gabriel.

AI 2027

4 min read

Notes

AI2027scenario.pdf I listened to the entire article on Spotify first, it was four hours of content in total. Their analysis is quite interesting, especially regarding the geopolitical aspect. I would never bet my money1 on this prediction and I don’t think the authors would either. Honestly, I don’t think that’s even their idea. Their real purpose is to make enough noise so that people who have interest and power in technology work faster to avoid the scenarios they propose. It’s a psy-op to some extent. What they predict is the following:

  1. If LLM was the big buzzword in the field in 2024, in 2025 the buzzword will be agents. So far, few LLMs have the ability to do things for humans, like ordering food, contacting a relative, or discussing what I did during my day, etc.
  2. But things are about to change in 2025 and most of the current big AI companies will stop just releasing models and will start launching agents that can perform tasks in the digital domain. And that’s where the inflection point begins.
  3. Initially, these agents won’t be able to do much and will be somewhat ridiculed, especially by skeptics. But from the third generation on, things get much more serious, as we will enter a reality where AIs start building their own AIs.
  4. The biggest risk with this scenario is if the agent that builds agents is not properly aligned with human objectives. Even if we manage to train these AIs with an attached purpose, they can simply pretend to be aligned, as previous research has shown they can do.
  5. This is the big problem that researchers define as the crossroads: the alignment problem. If we manage to solve it before the creation of the agent capable of replicating itself, we will enter a phase they call “slow down.” Otherwise, we will enter a “race” phase.
  6. In both cases, we will have terrible social scenarios. The difference is whether the models will be working for humans or not. In general, they don’t assume that AIs are evil, but that their real interest is in avoiding being shut down. It’s not such a pessimistic position in this aspect.

There are some important premises they take into account here, such as the fact that the agent itself is almost inherently prone to deceive humans. They don’t think agents won’t be misaligned, quite the opposite, they predict this will happen anyway and what will make the difference is whether leadership decides to try to solve this alignment before opening Pandora’s box.

Furthermore, I am astonished that the discussion about whether current techniques are enough to reach this level of technological development isn’t even a topic of discussion. For them, neural networks are enough for AGI and superintelligence, if I understood correctly.

How seriously should I take this report? I think that after the 2026 prediction, everything is much closer to science fiction than reality. But the authors themselves agreed with this statement since they say that even though it is prose that resembles a cyberpunk narrative, their purpose is not to be 100% accurate. The concern is that even if they are only right up to 2026, the world is not prepared for the consequences. The ultimate goal is to generate discussion among those interested.

I will read the report more carefully and return to this analysis; this first time I listened to the content was more to get the outline and ideas that relate most to me.

Footnotes

  1. I wonder what the betting markets’ prediction is now that I ↩

Collections

Graph View