Thanks for all the ideas. I'm pretty sure ashes aren't the problem. When I shake it down I've been making sure I go until I can see an orange glow from underneath, so I think I'm good there. I'm pretty confident that my damper is operating correctly and the stove seems to be firing consistently. I'm more and more convinced its not the furnace but a loss of heat through the house. The house isn't freezing or anything, but burning a consistent 500 to 550 degrees its just holding 70, which is fine now but could put me in trouble on those colder nights that are coming.
The stove is located in the basement right next to the steps that lead to my first floor. My basement doesn't have a door going directly outside, you have to go down from the first floor. For an older house, I don't think it's that drafty down there. No coal chutes or anything like that, just three small windows that are in fairly good shape. I know the first floor of my house is well insulated, as I just remodeled it over the summer and was sure to insulate it well. However, I have not gotten to the upstairs yet so it is still outdated and more importantly, uninsulated. The stairs that lead from the first floor to the second floor are right above the stairs that go from the first floor to the basement. After reading what everyone has said, I'm wondering if the heat isn't getting sucked right up the stairs and out the uninsulated upstairs and out the ridge rent on the roof? As I said, it's a cape cod house so there is a small space between my upstairs ceilings and the peek of the roof and I know there is no insulation there. I may try insulating under the stairs that go to the second story to try and keep the heat on the first floor and allow it to radiate upstairs through the ceiling. Not sure if it will solve the proble but I think it's worth a shot until I get to remodeling the upstairs this summer and can get it properly insulated.