Zinc is toxic to begin with, the problem is if it doesn't stay on the pipe. It can come off in a fine powder though once overheated, it can be picked up on your hands if you touch the pipe or perhaps become airborne. Either way it can be ingested and heavy metals reek havoc on the brain and central nervous system. The only way to remove them from your body is chelation. As you say, with your stoker the pipe may never approach temps where it could become an issue. However, hand fired units are not that hard to goof and cook it up.
I'm not suggesting everyone run out a replace their existing galvanized, but because the stovepipe is removed for service fairly frequently, just use caution when handling used galvanized particularly when overheated and replace it with black when the time comes. That caution amounts to not eating, smoking or putting fingers in orifices prior to washing ones hands. It can't be absorbed through the skin to my knowledge, it must be ingested into the body.
I have no idea at what temperature it starts to give off the gas, I'll see what I can find.
Here is a link to the effects if it does gas off.http://en.wikipedia.org/wiki/Metal_fume_fever