I have a Lee Valley sanding drum for use in my drill press. I also purchased some extra sanding sleeves. After loosen the nut at the bottom of the drum it is still very difficult to pull off the sanding sleeve off the rubber drum and put another grit on. Anybody have a similar issue? Any solutions?