Make sure you use a good firm sanding block. You need to apply even pressure.
I'm not a fan of any of the fancy padded blocks. I just use a scrap of...
No - there's really no practical way to speed up any part of the process. This is one of those thing that only time can do.
Even kiln dried pressure...