Your old one could have rust or ash or the pivots rusted and may not be adjusted properly (thats what the draft gauge if for). That may be why it wasn't working correctly.
YES, all draft should be checked with a draft gauge (manometer). The weight setting is approx. You should be close however for starters. Also, with build of fly ash on the back side, it will change the draft settings eventually, so it's a good idea to check it at various times throughout the season and clean the back of it. It may not change much, but always a good idea to check it.
Yes, it should open and close depending on the draft, if windy, it will flap quite a bit.
NOTE: There is "Manometer Loaner" program if you don't want to buy one. Check the top of this thread and request to be on the list.