Agree, I see no appreciable wear to the pulley so I'd not open that can of worms by removing it unless you're working on a degree in industrial education.
Now that it's off you can determine if the shaft is tapered by 1) the difference in shaft diameters and also 2) if tapered I doubt the key will be parallel to the shaft.