What would cause smoke to come out of the fireplace into the house?

I would think that as long as the smoke stops coming in when you open the door, that the house is so airtight that it wont let the smoke out. Try cracking a window open just enough to let the house breath and I think it will solve the problem.