Yep. I do as much joint-compound-sanding-dust cleaning as possible with a broom, dustpan, and mask. Unless you buy an auto-clean high end extractor, joint compound dust will clog up just about anything and cleaning a vac after sucking up this dust is a PITA. For small repairs I use a soft brush and dustpan, followed by a wet rag. For larger jobs its a broom and dust pan, using the vac and crevice tool to get into corners.
For me it takes longer to clean a vac after sucking up joint compound dust than to just us a broom or brush and dust pan.