I have seen pictures of Regency and/or Georgian door knockers where the ornate 'head part' had a bolt (or just a bar, perhaps - the pictures weren't that detailed) coming out of the back that went through the door, making it straightforward to take the thing down. This ornate piece was large enough to be seen from the street, which wasn't far away (up a flight of steps), so even if you weren't close enough to the family to get a note when they were 'in Town', if you lived or visited in the neighborhood you could probably see it.

I have no idea if removing and replacing the door knocker was common practice among Le Bot Ton; perhaps it was made up by an author to add texture to stories, and those of us who read such stories just started believing it (though my guess is it was a thing at least some people in that period did).