I recommend a fairly coarse sandpaper, as some here have recommended already -- though, it can be difficult to find the right type of sandpaper which doesn't load up or wear out too quickly.
Remember, the coarser the abrasive, the more time will be spent removing scratches on the next level of abrasive.
On my granite bench there is a steel screw for holding a magnet. The magnet is wrapped in paper. This is used regularly to remove iron particles from the abrasive. With something like a plane, it is used every three or four full length passes.
jtk
"A pessimist sees the difficulty in every opportunity; an optimist sees the opportunity in every difficulty."
- Sir Winston Churchill (1874-1965)