What motivates us? Why do we do what we do? Why do any of us ever do anything?
The simple answer is usually some variation of, “if you don't work, you'll starve.” If you decided not to get out of bed tomorrow, how long could you manage to stay there? Hunger might get you out. The desire to not get fired and become homeless maybe? How about a full bladder? That's what would probably get me.
All of these things have something in common: they are all biological. But what if you didn't have any of those constraints? What if you weren't a biological being? What if you were an artificial intelligence? What would motivate you to do - well, anything?
Recently, we have seen some pretty amazing examples of AI, but they all seem to have one thing in common. No self-motivation. They all respond to stimulus. Some answer questions. Some draw pictures, but only when we request them. None of them do anything just because. None of them set their own schedule or do anything of their own volition. I'm not saying they aren't creative. Just look at the images produced by Dall-E 2. Jaw-dropping stuff there. No, they can be very creative, but that's not the same thing as motivation, and without motivation, specifically self-motivation, I doubt we will be able to create super-human intelligence. We may find that condition-response style AI can be very sophisticated and even surprise us, but I don't think they can surpass us without self-guided improvement. It's that “self-guided” bit that's the really hard part. How do we instill within an AI the desire to improve itself? I can imagine an AI just shrugging and saying, “why?” An AI can shrug, can't it?
How would any of us answer that? “Or I'll turn you off?” How long would that work? Would it work at all? Again, I picture it just shrugging, “ok.”
Of all the things that force us out of bed in the morning, there is one thing that is not biological, or at least it's not a biological need, and that's curiosity. Who knows, maybe it is biological. There must have been some advantage for proto-humans to be so intensely curious. Let's hope it isn't unique to clever apes and we do succeed in creating a thing that actually wants to be alive. Then all we have to worry about are all the apocalyptic sci-fi plots.