It gets harder to believe, but be honest

Yes, that's right. Drones also get replaced; have you had that happen to your household members before? One evening, you went to bed after saying goodnight to a drone sibling or parent, and the second morning, someone conscious and new completely took over their place. They jut disappeared, so their bodies don't always stay here either. That also tells us that the slots here are limited, which can be seen as another restriction proving that the town can't do anything. Some of these people who got droned before being replaced return to Mayfield too, but it can be both with or without previous memories of this place. Either way, it's like their being here never impacted their home world, wouldn't you say? So in that case, isn't it more likely that people simply go back, instead of picturing the worst case scenario like what everyone saw at the Dairy? The town likes to mess with our minds too, so naturally it'll draw the most horrifying possibilities to intimidate us. Does that make sense?