There are plenty of cases were the same Post Office services multiple zip codes for multiple cities. My local Post Offices serves at least two cities and each city has their own zip code. I'm fairly certain no state would be able to tax you simply based on your postal address. If you physically lived in another state year round it would be hard to claim you as a resident. I can't believe a court would side with the State of Washington if it went to court.
Your city may still be large enough to justify a full service Post Office. If not, perhaps Postal services could move into a local store and still offer most services. There is no reason a local store couldn't do 95% of what a full service Post Office offers at a lower cost. We have store locally that offer many Post Office functions at their customer service counter while using maybe 20 square feet of space. To offer PO Boxes and 95% of the services would certainly require more space.