I recall that the movie attempts to fill this plot hole by saying that the aliens were leaving the planet but that the injured, sick, or otherwise weakened aliens were left behind. I think the one holding the boy in the final confrontation was perhaps the same one that lost a digit when the protagonist was using a knife to see it’s reflection through the door gap when it was locked in a pantry.

I also recall the man in the car who was leaving that home with the pantry saying “btw we locked one of those things in the pantry, I think something was wrong with it. k bye”

I don’t know how much of this I made up.