:shycouch:

Why would a single-cell organism evolve into a sentient multi-cell organism that tap dances?

Why wouldn't it?

Because it lacked free will.

Why would it?

All we really know is it might.

autonomy seems like an emergent property, rather than a specific function you design for. If you make something intelligent enough, then autonomy comes with the territory. And futurists who are talking about the singularity and AGI are specifically aspiring towards such a creation.

It's definitely common but I disagree with the notion that this is inherent. The biological separation makes emergent behaviours as you describe them to be less likely of an outcome imo. The sexbot doesn't actually have a sex drive (there's a good pun there that I can't quite stitch together), it's just mimicking human behaviour. The application of that mimicry using AGI is a far more realistic threat than AGI posing an existential threat to to the human race for its own self preservation.

Autonomy is something that emerges as a property long before the kind of Intelligence that humans have. Ants are basically autonomous. Current AI is basically autonomous. Its decision making skills will only increase the more advanced you make it. And AGI by definition would be every bit as capable and adaptive in its decision making skills as a human being - or else it wouldn't qualify as AGI.

Is a male love bot's sex drive its hard drive?

What about in terms of personal autonomy?