Yes. The point is to fill the cracks, not coat the surface. You're sanding away the filler that is on the surface, leaving just what is in the cracks.

It's easier to sand thin single layers than thicker double layers.

It's also easier to sand thoroughly dry layers.

Which explains why I have models with a coat of filler that have been lying around unfinished for YEARS. Or maybe I'm just lazy and hate sanding.