Yes your draft should increase slightly the colder it gets outside (but only to a point) Also, it's impossible to set the draft without a draft gauge... without one it's only a guess. I was a oil heat service tech for 20 years. It's all physics. "Thermal" draft ... the difference between the hot flue gas temperature and the outside ambient temperature... hot air rises, draft is nothing more than a negative pressure or vacumm. You really can't set the barometric damper / draft stabilizer without a draft gauge. Also a barometric damper can't create more draft, it can only regulate what draft your chimney is already capable of pulling, and that will vary depending on the height of the chimney, lined or unlined, the condition of the chimney, outside air temperature, etc etc.
Use a barometric damper.... the good type with a well balanced counter weight. As the draft increases the the damper will open and when the draft is lower the damper will close, Keeping (as best possible) a steady and constant draft. Again, you really must use a draft gauge initially to set the barometric damper/draft regulator.
There's also currential draft...like when the wind is blowing hard outside. As wind blows over the top of the chimney it creates "spikes" of static pressure... ever see or hear your damper flapping? thats most likely caused by currential draft or "static" pressure. And of course there's "forced" draft But thats another story.
Just a side tidbit/fact : A chimney that goes up through the center of the house will pull more draft than a chimney located on the outside exterior wall of the house (all things being equal).