I just saw something about how modern technology has oscillated between open and closed systems, and currently AI is stuck in a closed system. The point was to break out of that into an open system again, as that is when breakthroughs and development occur. However, given how badly we've done with the "fake" AI and its dangers, I think we need to be cautious on finding the next phase. AGI in fiction is not valid science itself, but how it affects things are lessons and warnings of what-ifs that we do not want to go down.

And sure, there's plenty that say it will never happen. Great. What if they're wrong, and we do what we always do and run into the fire because it's bright and shiny?