At least in the case of a Japanese saw, the kerf is quite narrow, thus the saw is very easy to pull (not much wood removal) and, started properly, it's hard to get the saw to veer appreciably off course (enough to require more than a few swipes with a block plane.)
But a Western saw is not much different (particularly with a back saw,) just has a slightly wider kerf. An eighth of an inch leaves a lot of wiggle room.