In my experience, shooting goes rather quickly. How much material are you removing?
Technique may also play a role in your shooting work.
First is getting the blade set up for shaving end grain. It needs to be very sharp. In my experience setting the blade to take as thin of shaving as possible on edge grain will get you close. In my experience, the blade will then need to be set a hair deeper to make a shaving on end grain. It should make a shaving and not dust. The piece being worked is held with one hand and the plane with the other. Hold the work against the fence and move the plane so the toe is touching the work while the plane is against the side of the bed. (the bed is what the work is on) Then push the plane forward to take the first shaving. Withdraw the plane only to the toe then with the hand holding the work move the piece as needed to register against the plane's toe while holding the plane against the bed and make another shaving, repeat.
With a little practice this will go rather fast.
Here is something that made shooting a bit more comfortable with a bench plane:
https://sawmillcreek.org/showthread.php?130114
Derek Cohen has two much more elegant 'hot dogs' on his web site:
http://www.inthewoodshop.com/ShopMad...LV%20LAJ1.html
http://www.inthewoodshop.com/ShopMad...ck%20pics.html
It is only different ways different folks do similar tasks. In your example of a drawer, all my components usually have the ends smoothed and squared on a shooting board. This makes for easier, smoother marking with a wheel or pin gauge. Trimming for final fit of a drawer for me isn't done on a shooting board.
Different strokes for different folks.
My current shooting board can be used from either side:
https://sawmillcreek.org/showthread.php?244777
jtk