direction of movement in space

Since there really is no "up" or "down" in space, what direction are astronauts being pulled toward when they are "unattached" in space?
(meaning if they were to not be holding on to anything or be in anything such as a space shuttle/station)