If your set up already had a marginal draft it may cause a problem.
If your set up could suck the cat across the room and up the chimney, then it would probably not be a problem!
Since you just installed the manometer, I would vacuum it out so you could know what your draft number is with a clean pipe. Then you will have a basis for comparison. In a few weeks if you see your draft struggling to get to the "clean" number it would be time to inspect for build up and if there is fly ash there again check to see the effect on draft by vacuuming it out. If the draft number still doesn't get back to the clean pipe number then it is time to look for fly ash in another part of the stove or chimney. I could tell when it was time to vacuum off the top of the baffle in my Harman in this way. It also clued me in to the fact that the cap was off the bottom of my SS chimney one time when I couldn't get a good draft number after cleaning out both the horizontal flue pipe and shutting the stove down and vacuuming it out. You can bet that cap was the first thing I checked every time after that though!