I'm almost always looking at the furthest part of the fence (well behind the blade) when I'm ripping. obviously, you have to pay attention to the blade to keep your fingers, but the board should be pressed firmly against the fence along the entire length. Focusing on doing that AND avoiding the blade works best. Also, don't stop moving the board forward if possible.
Edit: I don't push anything thru under ~ 4" wide with my bare hand. I use a pusher.