Technology

The United Nations is supporting a new international architecture to help countries make informed decisions about AI.

In 2025, the UN General Assembly established the Independent International Scientific Panel on Artificial Intelligence, made up of 40 experts from every region of the world serving in their personal capacity.

The panel's role is scientific rather than regulatory. It assesses, on a regular basis, the latest evidence on AI's opportunities, risks and impacts and produces independent reports that governments can use when developing policy.

This is approximately what I recommended some time back on Reddit. Can't have Sam Altman or advocacy groups driving it

need reasonably objective, unexcitable observers. Also, not just to governments but to the public.

There are a lot of unknowns, but at least we can start building consensus on some points and identifying the known unknowns.

It's also not just computer, technical unknowns. Like, one of the big questions is how much existing stuff done by humans can we realistically automate (Sam Altman has an incentive to say "everything, tomorrow"), and what. Economists can't predict the impact without technical data on capabilities, and right now, we just don't know a lot of that, and we don't know where we'll be in a decade. But they can firm up a set of questions to ask and to watch for. "If we can achieve capability X, then this is what the likely global impact is."

Those predictions will probably be bad at first. But...that's science. You start with a model and start refining and revising.

I'm also interested in coming to some kind of firmer consensus as to superintelligence risks. Right now, we have a lot of ideas flying around that run the gamut from If Anyone Builds It, Everyone Dies to "it's impractically hard to build and is thus a non-issue" and little consensus as to what lines we might cross that would expose us to superintelligence risk. Needless to say, this is is basically useless if you're trying to do anything from a policy standpoint.

I don't think that we should kick the can down the road on that. The problem is that our major advantage, as humanity, in dealing with superintelligence, is time. We get to make the first move, structure the environment. So we don't want to throw that away. If we need to figure out how to contain, how to control superintelligent systems, we likely need to do so before we have people actually building them. And there's a lot of potential economic and military and so forth benefit in building a superintelligent system, so people are probably going to try to build such systems. But there aren't similar incentives associated with safety surrounding superintelligent systems. People may not just go out and do that on their lonesome. So if we need to fund them or whatever, we should be doing so without waiting for those systems to exist.