A History of the Future, 2025-2027
A scenario, part 1.
Below is part 1 of an extended scenario describing how the future might go if current trends in AI continue. The scenario is deliberately extremely specific: it’s definite rather than indefinite, and makes concrete guesses instead of settling for banal generalities or abstract descriptions of trends.
The return of reinforcement learning
From 2019 to 2023, the main driver of AI was using more compute and data for pretraining. This was combined with some important "unhobblings":
Post-training (supervised fine-tuning and reinforcement learning for instruction-following) helped the LLMs be usable without difficult prompting.
Starting in 2024, Anthropic showed that judgement and taste in data curation—and the evaluation metrics that guide data curation—could give you a "magic sauce" effect in perceived LLM quality.
Most real-world LLM uses, of course, involved generating a sequence of tokens to try to achieve some task. So there were a lot of untapped gains from doing reinforcement learning (RL) for performance on concrete domains, rather than just RL for the models following instructions and being "safe"—i.e. a combination of avoiding PR hazards, and preparing for misuse mitigations on actually capable models down the line.
OpenAI fires the starting gun in 2024 with the release of o1, which was based on RL on chains-of-thought (COT), i.e. the model is trained to reason step-by-step towards correct answers, i.e. "test-time compute" in the horror-filled annals of machine learning jargon. In late 2025 they release “GPT o5” (“GPT” to normal people, and “o5” to those keeping track of the version number), a model which can take text, image, audio, video, computer screen state, real-life footage, whatever, process and understand it (choosing itself whether it should do chain-of-thought reasoning before answering or not), and output text, image, audio, video, computer actions.
Whereas the labs had spent almost four years racing down the scaling graph on pretraining compute, they had not yet done so for COT RL, and had not uncovered the subtler tricks to doing this well. This meant there was a lot of low-hanging fruit, so progress—and replication—was fast. In early 2025, DeepSeek spooks the entire American business scene with their release of R1. In spring 2025, Anthropic ships Claude 4, which also has inference-time compute abilities that trigger if the model is asked a question where that helps.
Anthropic keeps their largest Claude 4 model internal and secret from the very start. It gets used for (most importantly) producing training data for the smaller Claude 4s, and (experimentally) in doing internal evaluations of AI-driven AI R&D, starting with some adversarial robustness research on Claude 3.5. Inference costs on the biggest models are a big part of the rationale. Anthropic continues being focused on intelligence over product, and enterprise products over consumer products. They make only minor gains among consumers, but Claude is increasingly adopted among enterprises, programmers, knowledge-workers, and nerds. (Ironically, OpenAI has the consumer advantage despite focusing more on reasoning and less on the LLM being personable and writing well.)
In 2025, thanks to RL, “agentic” AI is here, but only kind of. Anthropic and OpenAI have computer-use features that work, except a bit spottily and are designed to never authorise a payment or send an email or do anything important without human confirmation. Google releases an AI agent for things like Google Cloud Platform configuration schlep, which the programmers love. A bunch of startups are competitive with the major lab products, in particular because no one has yet had time to pour ungodly amounts of compute into the COT RL. However, most "agentic" AI applications remain LLM scaffolds, i.e. a hard-coded flowchart of LLM prompts and other API calls.
Meta is trialling some unholy autonomous AI features across their apps (such as AI agents going around leaving comments on user’s posts to “maximise engagement”), but they still seem like gimmicks.
Code generation tools like Cursor and Lovable and Zed and Poolside and Magic.dev and ten million others are getting very good. For most apps, you can in fact just drop in a prompt and have the app running within a few minutes, though managing infrastructure is still a pain and technical debt tends to accumulate if the AI stacks many changes on top of each other. Some form of COT RL is used in the training stack for many but not all leading coding tools. LLM scaffolds still reign over unspecialised general agents.
Gemini-3 ships in 2025 after a vast pretraining run. It’s good but a disappointment; the final culmination of pretraining scaling laws in an era where products, inference-time compute, data curation (mostly synthetic now, but behind the scenes there’s some very important human judgement going on), and real-world interaction ability are key. Google DeepMind (GDM) is building powerful maths models, and making progress on reasoning architectures that don’t rely on external COT and are better-suited for maths.
After 2025, RL starts getting harder and the distance between the leading labs and the rest increases again. RL is simply less efficient than pretraining, partly because the necessity of letting models try long sequential chains of actions makes parallelism harder. The labs have now scaled up RL compute quite far, so the resource bar for being in the game rises. Also, RL is notoriously hard. First, subtle bugs are easy to make and hard to notice: is the RL agent not learning because you made a bug, or because it just won't learn? Second, there are more choices to make (e.g. you have to pick a reward function and scoring method, rather than the cross-entropy loss you default to with pretraining). OpenAI, Anthropic, and Google take some distnace to the rest in RL and overall general capabilities. However, the other labs don't necessarily see this as a loss—Meta deliberately focuses more on integrating AI into its products over 2025 and 2026, xAI focuses more on engineering use-cases, and both xAI and DeepSeek remain competitive. Also, the issues with RL mean that there are some more hairy technical problems that temporarily slow progress as labs one after another internally work through them, though this is not at all obvious from outside a lab.
In early 2026, xAI starts deploying an early version of an AI that can do engineering CAD work kind-of-well, as long as a human is looking over its shoulder and checking its work. This improves a lot after Tesla and SpaceX (are forced to) actually start using it, but it’s not yet groundbreaking; sheer data quantity remains an issue.
The next big advance is OpenAI's late-2026 release of o6. It has improved a lot in computer use, and generally in unifying its various input and output types (e.g. it can use text and images more effectively together in its output, process and output longer videos, etc.). Second, it has a more advanced memory architecture, including a built-in longer-term memory that allows instances to learn over time. Thirdly, it’s of course generically a bit smarter, a bit faster in token output, and so on. In particular, OpenAI has finally almost caught up to Claude’s personality level. It is also way more impressive to normal people because it can also—if prompted to do so—generate real-time video and audio of a talking face. OpenAI doesn’t explicitly encourage this, but winks at this, since it knows this will get some users addicted (especially as they now have a more nuanced policy for sexually explicit model outputs than the previous blanket ban).
Many people in Silicon Valley declare this AGI, and predict the immediate automation of all office jobs. In practice, it falls short in a hundred subtle ways that make it not a drop-in replacement, in particular with remaining unreliability in its ability to use computers and weaknesses at planning and completing long-horizon tasks. But the smart money is betting that these issues will be solved within a year.
Also in late 2026, Anthropic releases Claude 5 Haiku and Claude 5 Sonnet. Claude 5 Haiku is a cheap model roughly on par with Claude-3.5-Sonnet in smartness while having an output speed of hundreds of tokens per second. They come with an upgraded version of computer use that is far faster and more seamless. Again, the largest model is kept internal. Its training data curation and post-training finetuning was focused on programming, ML research, MLOps, and maths. Anthropic employees started adopting it internally in mid 2025, giving researchers and engineers what's essentially a team of AI interns to manage. They then spent 6 months giving the models tailored feedback, which they massively boosted with methods for dataset augmentation, and filtered for correctness with scalable oversight techniques like debate, before feeding it back into the model as finetuning data. In 2024, Anthropic internally estimated a +5-10% productivity boost from internal use of Claude-3.5-Sonnet and early training checkpoints of Claude-4; in 2025, this rose to +25%, and with Claude 5 Opus it started out at +35% but has gradually accelerated with more and more finetuning to +60% by mid 2026, and the number is still climbing. OpenAI does not have a comparable setup internally, partly because it’s politically less feasible due to the lower-trust environment, but also because it's a lower priority since they believe less in recursive self-improvement.
Codegen, Big Tech, and the internet
Coding is a purely digital job that is economically highly valuable, has a lot of training data and often provides a clean feedback signal of success, and that the AI-affiliated companies all already have expertise in. All this makes it ideal for AIs to be good at, quickly. In 2023-2026, the biggest economic impact of LLMs is their use in coding.
In 2023, models got good enough for programmers to prefer them to looking up human guidance on sites like StackOverflow. In 2024, coding copilots were a real productivity boost, perhaps +10% to +50%, for pure software engineering tasks (higher for things that are more boilerplate and when the coder has less background in what they're doing, lower for more research-y tasks or when working with familiar domains). In 2025, there are two big new advances. First, chain-of-thought RL meant that spending more LLM tokens converted more efficiently into better code. Second, a bunch of the obvious improvements to the workflow were made, such as the AI automatically running tests or checking that the UI looks right, and autonomously trying again if not, rather than maintaining the human as a tab-switching, prompt-writing monkey that does this for the AI. As a result, by 2026 codegen looks solved. There are some wrinkles left related to cloud infrastructure stuff, especially when there’s little training data on some aspect and/or a heavy and unavoidable button-clicking component, but these are quickly getting fixed, especially as computer use gets good and allows the models to better click buttons and surf the internet for documentation.
For a while, everyone’s paranoid about security in the fully AI-written codebases, and a bunch of security consulting and cybersec firms make a killing. However, it soon turns out codegen stuff is actually more secure than human code because the LLMs reliably do the standard correct thing over the weird bespoke thing whenever it comes to security, and this eliminates a lot of security vulnerabilities that humans would write in. The security consulting and cyber firms quickly become LLM wrapper companies with excellent marketing arms, and stop being used by most people apart from risk-averse large companies and governments. However, as is statistically obvious, there are a bunch of high-profile blowups, and it remains true that existing code can now be much more easily attacked since all you need is an o6 or Claude subscription.
By 2027, the price of creating a simple app is a few dollars in API credits or GPU hours. The price of a particularly complicated piece of software is on the order of $100 to $10k. The tech stack has shifted almost entirely to whatever there was the most data on; Python and Javascript/Typescript are in, almost everything else is out. Average code quality as judged by humans declines, but this is fine because humans don't read it and the LLMs can deal better with bloated code.
The coding advances trigger a massive flood of non-coders or amateurs flooding in and trying to make money off B2B SaaS or freelance programming. Agentic non-technical people are launching niche startups at massive rates, since you can ship a full-featured product in a few hours if you’re willing to burn money on API credits. Lots of these projects run into “tech debt hell” eventually. For a while programmers can earn heavy consulting fees (or cofounder roles) by coming in, chatting to the AI about the codebase, and telling it to make architectural changes that let the next features be added cheaper because it will take fewer lines of code on top of the better-structured codebase. However, just asking the AI “what’s wrong with this codebase” and then “how would you fix it” also works quite well if the prompting is good. The codegen scaffolds quickly evolve to be good at reflectively prompting AIs and managing the tech debt hell better, but it’s hard to notice this unless you’re working with them actively, leading to a lot of misinformed doubts about the capabilities based on early disappointments. The labs also start including more qualitative criteria in their codegen RL—not just "did the code run and pass the tests", but also asking another LLM to grade the style and extensibility of the code. In effect, there's a race over whether the AIs will learn good code practices from RL self-play, or from explicit human scaffold-crafting and prompting. Note that the latter is getting easier too, as the tooling improves, and AIs write the scaffold code and distill human programming knowledge into prompts. For example, in late 2025 Anthropic also ship an automated tool for building an LLM scaffold from observations of an arbitrary real-world digital work process.
Big Tech starts using the codegen tools heavily for new projects, but integration into older projects is slower because the codegen scaffolds are worse at interfacing with large existing codebases than writing small ones from scratch. This gets mostly solved over the course of mid-2025 to mid-2026, but gives the "Little Tech" startups a temporary tailwind. Big Tech headcounts grow, as they hire more people both to flatter the egos of managers—they are drowning in cash anyway—and in particular many product managers to oversee the AI codegen agents that are unleashing a massive series of new products now that they're mostly no longer constrained by development taking lots of time. Internal company office politics becomes even more of a rate-limiter: if teams are functional, the AI codegen boost means more products shipped, whereas if teams are not, the gains are eaten up by employees working less or by factional fights within companies. Microsoft launches “365 Days of Microsoft”, where every day of the year they release a new software product or a big update to a previous one; they move increasingly into more niche enterprise markets that they had previously ignored as part of a new paradigm shift. Google is more scattered and launches a thousand new features integrated into their product suites that—on paper–compete with existing startups, and—in practice—serve to expand the empires of enterprising Google middle-managers. Google gets a reputational hit as a shipper of sloppy products, but they have a few big hits and their customers are a captive market that will continue using Search and Drive, giving them room to flail around.
There are a few corporate scandals as AI codegen products fail, leading to a frenzy of effort at testing and fuzzing the AI outputs. But Big Tech is still all-in, at least until late 2026: they’re all feeling the AGI, and that if they miss it that’s an existential mistake, and if it’s all a bubble then at least they were only as bad as all the other Big Tech firms. The one slow actor is Apple, due to its cultural bias towards quality and assurance. Apple ships Apple Intelligence integrations but that’s about it.
Predictably, the super-abundance of software and the extreme competition in it drives down prices. SaaS companies aren’t yet experiencing an extinction wave because humans react to change slowly, but it doesn’t look good and investors start getting skittish. The big advantage that everyone points to is having locked-in customers or network effects; otherwise, the conventional wisdom goes, you're dead. But there are a bunch of companies and tools that let you circumvent attempts at customer lock-in. You can program yourself an X.com in an afternoon, have computer-using AI agents trawl X, Reddit, etc., pull in content to your site, and propagate your posts and replies automatically to all the other platforms. Some companies fight tooth and nail to try to make people stay on their platform (and thus see their ads); some just charge API prices and hope to at least get revenue. “Web4” comes to mean a programmable internet that is customised to everyone. A hundred startups jump on this bandwagon. Some established companies create carefully de-risked APIs and let users program customisations and integrations into their sites (i.e. let users ask codegen models to do such programming). The Web4 wave generally runs into the problem that most people don’t actually want to customise things; they want someone to have already thought through the interface and features on their behalf, are fine with existing setups, and not very eager to re-imagine the internet. But increasingly, if users dislike something about a site, they will build their own version, connect it to the original with AI schlep, and then lure over the few percent of users that are triggered by the same thing. Technical barriers like scraping limits are hard as AI agents can be made to browse in increasingly human-like ways (one successful startup explicitly engages in a scraping race against scraping-detection methods by fine-tuning a computer use agent on real human mouse moving patterns). An increasingly common barrier is asking humans for government ID or other real-world verification (with the privacy constraints mitigated with zero-knowledge proofs, if it's a fancy libertarian or crypto -affiliated thing). This too is spreading, also because some people want sites where they can talk to verified real humans.
By 2026, more code gets written in a week than the world wrote in 2020. Open source projects fork themselves into an endless orgy of abundance. Some high school students build functionally near-identical versions of Windows and Google Drive (and every video game in existence) from scratch in a month, because they can and they wanted one new feature on top of it. Everyone and their dog has a software product line. Big Tech unleashes a torrent of lawsuits against people cloning their products, echoing the Oracle v Google lawsuit about Java, but those lawsuits will take years to complete, and months feel like decades on the ground.
Silicon Valley is exuberant. The feeling at Bay Area house parties is (even more than before) one of the singularity being imminent. Some remain skeptical though, rightfully pointing out that the post-scarcity software isn’t the same as post-scarcity everything, and that genuine “agency” in the long-horizon real-world planning sense hasn’t really arrived, and under the hood everything is still rigid LLM scaffolds or unreliable AI computer use agents.
Business strategy in 2025 & 2026
Even though Meta, DeepSeek, and others are behind in raw intelligence and reasoning all throughout 2025 and 2026, they threaten the big labs because they are giving away (both to consumers, and freely to developers through open-weights releases) a level of performance across audio and video and image and text that is “good enough” for most use cases. SOTA performance is no longer needed for many use-cases, especially low-end consumer entertainment (e.g. image gen, chatbots, etc., which Meta is banking on), or most classification, processing, or business writing tasks.
OpenAI is especially vulnerable, since they rely heavily on consumers, and are also increasingly a product company that competes with products built on their API, driving many to switch. Their strategy (internally and to investors, though not publicly) is to be the first to achieve something like a drop-in agentic AI worker and use that to convert their tech lead over open source into >10% of world GDP in revenues. They’ve raised tens of billions and make billions in revenue from their products anyway, so they can bankroll these efforts just fine.
Anthropic remains a jewel of model quality and a Mecca of technical talent that gets surprisingly little attention from the rest of the industry. Analogies to Xerox PARC abound, but there are whispers of internal AGI being imminent, and no one else can claim the ideological mandate of heaven for safe AGI. The talent and money spigots stay on.
xAI and DeepSeek continue releasing open-source consumer models. Both also have a specialty in maths-y STEM and engineering stuff, aided by data collection efforts (with xAI being able to work closely with SpaceX and Tesla engineers) and inference-time compute methods. xAI also continues trying to leverage real-time access to X.com data to its benefit, but this isn't a major advantage or revenue source.
In 2024, thousands of startups were chasing after a lot of different use cases, and some started making serious money, but it was still very early days for actual products. The big winners were companies like Perplexity that use LLMs to trivially improve some LLM-compatible user case (like search), companies like Glean and Hebbia that are doing various enterprise LLM integration schlep, and legal LLM companies like Harvey (since law is intensely textual and high-revenue). However, the real money is still in infrastructure / “selling shovel”, in particular Nvidia.
By the end of 2025, there is no technical bottleneck to remote doctor appointments or most legal work being done entirely by AI. However, diffusion takes time. Also, in many countries lawyers barricade themselves behind a cluster of laws that forbid lawyer-automating AI. Getting hired as a new lawyer, or any kind of white-collar analyst, is getting harder though, as decision makers expect AI to reduce their need for entry-level white-collar workers of every kind, and firing people is much harder than not hiring them in the first place. Healthtech AIs are gradually working their way through regulatory hurdles over 2025-2026, and are clearly better than the average doctor at all the parts of the job that rely only on reasoning and knowledge. However, AI doctor appointments are only trialled at any significant scale in 2026, by Singapore and Estonia. Significant integration of AI in the non-patient-facing parts of the healthcare system is underway in the UK, many EU countries, South Korea, and China by 2026, but again diffusion is slowed by the speed of human bureaucracy.
There are lots of “AI agent” companies automating things like customer service, various types of search (e.g. for shopping / booking flights / etc.), and back-office computer processes. The big cloud hanging over them in 2025 is whether AI codegen scaffolds soon get good enough that they are trivial to replace, and whether generalist AI agents soon get good enough to kill both. In 2026 the first question starts being answered in the affirmative, as lowered barriers to coding create a flood of new entrants and a ruthless bloodbath of competition. However, even the release of o6 in 2026, despite some initial hype, does not yet cause much evidence of the generalist AI agents taking over both by the end of 2026.
There’s lots of LLM evals startups like Braintrust.dev and HumanLoop and Atla, that are mostly struggling to differentiate themselves against each other or to define a new testing/reliability/verification paradigm for the LLM scaffold era, but are growing fast. There’s a lot of LLM agent oversight solutions, but by the end of 2026 none manage to make a massive leap, and the unlocking of new AI uses remains bottlenecked on incumbents' risk tolerance and a slow buildup of knowledge about best practices and track records. A surprisingly retro success is call-centres of humans who are ready to jump in and put an AI agent back on task, or where AI agents can offload work chunks that are heavy on trust/authentication (like confirming a transaction) or on button-clicking UI complexity (like lots of poor legacy software), to human crowdworkers who click the buttons for them, while the AI does the knowledge/intelligence-intensive parts of the job on its own.
Many of the really successful startups are in the spaces that Big Tech won’t touch or has trouble touching: anything controversial (the sexual and the political), and anything too edgy or contrarian or niche/vertical-specific.
The fact that the explosion of codegen threatens Big Tech’s moat, plus some disappointment at the unreliability of o6 after so much hype, plus some general memetic force that means the “current thing” can be AI only for so long, combines to cause a market correction near the end of 2026. Software is starting to seem stale and boring. Investors want to see “real AGI”, not just post-scarcity in software. Google DeepMind’s maths stuff and xAI’s engineering stuff are cool; OpenAI and LLMs are not. Amazon’s AWS & physical store is cool, Google Search and Facebook are not.
Maths and the hard sciences
A compressed version of what happened to programming in 2023-26 happens in maths in 2025-2026. The biggest news story is that GDM solves a Millennium Prize problem in an almost-entirely-AI way, with a huge amount of compute for searching through proof trees, some clever uses of foundation models for heuristics, and a few very domain-specific tricks specific to that area of maths. However, this has little immediate impact beyond maths PhDs having even more existential crises than usual.
The more general thing happening is that COT RL and good scaffolding actually is a big maths breakthrough, especially as there is no data quality bottleneck here because there’s an easy ground truth to evaluate against—you can just check the proof. AIs trivially win gold in the International Mathematical Olympiad. More general AI systems (including increasingly just the basic versions of Claude 4 or o5) generally have a somewhat-spotty version of excellent-STEM-postgrad-level performance at grinding through self-contained maths, physics, or engineering problems. Some undergrad/postgrad students who pay for the expensive models from OpenAI report having had o3 or o5 entirely or almost entirely do sensible (but basic) “research” projects for them in 2025.
Mostly by 2026 and almost entirely by 2027, the mathematical or theoretical part of almost any science project is now something you hand over to the AI, even in specialised or niche fields.
In 2026, xAI also tries to boost science by launching an automated peer-reviewer / paper-feedback-giver specialised in STEM subjects, that can also run followup experiments automatically, and soon take a paragraph setting the direction and convert it to basically a full paper. Cue a thousand academics blasting it for mistakes in its outputs. The fair assessment is that it’s impressive but not perfect (somewhat like having a brilliantly fast but easily-distracted and non-agentic undergrad research assistant), but still better than all but the highest-effort human peer-reviewers. Elon Musk gets into feuds about its quality online, becomes radicalised about peer-review and academia, and starts the “Republic of Papers” as a side-feature on X to explicitly try to replace academia (it helps that, in 2026, the higher education bubble seems to be bursting in America, partly triggered by fears about AI job automation but also due to political headwinds). Everyone has Opinions.
In 2026, GDM releases work on new maths-oriented AI architectures that include an advanced more flexible derivative of MCTS that also searches for new "concepts" (i.e. new definitions that shrink the length of the most promising proof-tree branches) while doing the proof-tree search. Their maths models prove a long list of new theorems and results, including, in 2027, solving a few more long-standing prize problems, this time in a less ad-hoc and more credibly entirely-AI way. Demis Hassabis talks about "solving physics" within the next year, through a program that includes GDM collaborating with leading physicists.
In 2028, GDM’s collaboration with the theoretical physicists bears fruit: general relativity and quantum mechanics are unified with a new mathematical frame. There are a few candidate new theories with different values of some parameters that can only be verified by expensive experiments, but it seems clear that one of these candidate theories is correct. It's not "solving physics" or a final theory of everything, but it is clearly a major breakthrough in mathematical physics. The technical work owed a lot to a truly enormous compute-budget for RL self-play, the construction of a massive dataset of physics papers, critiques of them, and tokenised observational data by a physicist-and-AI-agent team, and close collaboration with a large number of leading physicists who gave feedback to the AI on the developing theories. Credit for the Nobel Prize is the subject of much discussion, but eventually (in 2030) ends up split between Demis Hassabis, one of the physicists who was most involved, and the most important AI system. Everyone has Opinions.
Corporate Google likes the PR win of achieving the century's greatest physics breakthrough so far, but the application of this mathematical firepower they are most hopeful about is formally verifying the correctness of software. This is especially pressing as there’s a lot of shifting tides in the cyber world. Codegen itself is on net a defense-dominant technology (as discussed earlier). Most of the hacks are either due to sloppy mistakes by early codegen products, or some adversary using AI tools to direct a disproportionate amount of effort on attacking some piece of legacy software that is still used, or on which a codegen-written program (indirectly) depends. There’s increasing demand for really air-tight software from a US defense establishment that is obsessed with cyber advantage over especially China, but also Russia, Iran, and North Korea. Also, easily proving the correctness of code will allow better feedback signals for codegen models, and help in the ambitious efforts underway to rewrite massive parts of the existing tech stack. So in addition to making leaps in the hard sciences, GDM’s other big applied goal is a world where the correctness of all essential code is proven. They have an early success in a late-2026 plugin for several popular languages that is essentially a type-checker on steroids (though of course, this is adopted less by the humans and more by the AIs that now write almost all of the code).
Initially, the US government tries to restrict the diffusion of code verification tools, since they don’t want China to get provably-correct coding capabilities. However, the open source community is only about 6 months behind in verification as it makes some leaps and bounds in 2027-2028, especially since there are thousands of former software engineers and mathematicians without much to do as they wait for the AIs to do their work for them.
As a result, by 2028 feats of intellect that would’ve taken Euler decades are done in a few minutes to mathematically prove that, conditional on the CPU's physical integrity working, some code is an utterly impregnable and flawless pizza delivery routing system. However, verification is not adopted nearly everywhere because there’s a cost multiplier over just getting an AI codegen tool to write unverified code (and AI codegen has continued plummeting in cost, not that anyone really notices anymore).
Societal response
On the soft skills side, by 2025 experiments show that models have reached human-level persuasion capabilities in controlled text-only chat settings. However, this doesn’t really matter, since it’s not how most human persuasion works; part of models being bad at long-horizon planning is weaknesses in strategic relationship-building with relevant actors over longer timescales. There also isn’t yet widespread use of models to manipulate politics. First, there just isn’t a particularly tech-savvy political campaign or movement to influence opinion, except for China gradually experimenting with increasingly more AI in their censorship bureaucracy. Second, models still seem worse than the best humans at that “spark” that lets some people create persuasive, viral ideas. Third, the memetic selection pressures acting on the collective output of humanity on the internet are already superhuman at discovering memetic viruses and persuasive ideas than any individual human, so passing any individual human capability threshold in this domain is not automatically a society-steering ability.
However, some 1-1 use-cases do work. AI scam calls with deepfaked audio and video start being a nuisance by mid 2025 but are mostly reined in by a series of security measures pushed by platforms (and by regulation in the EU), people creating new trust protocols with each other (“what’s our secret passphrase?”), increased ID verification features, and growing social distrust towards any evidence that's only digital.
Lots of people are talking to LLMs for advice. Some swear by Claude 4 in particular. Character.ai -like startups are having a boom. There is a lot of public discussion about people increasingly talking to AIs instead of having human friends and partners (which is boosted after multimedia Llama models are finetuned to be good at sexual image, audio, and—in 2026—video output). There's a hikikomori-like trend, strongest in California, South Korea, and China, where a minority of people forsake almost all human social contact and instead interact with AIs that are superhumanly risk-free and pliable, and come with superhumanly nice voices and avatars. In 2026, Australia and Canada ban talking to non-educational AIs with voice capabilities or human-like avatars for under-16s.
The written text quality of models remains surprisingly mediocre. Claude does best, and is great when prompted right, but “ChatGPTese” remains a thing that afflicts especially OpenAI and Google (though the former improves in 2026), and any human who writes mediocre prompts. There are loads of LLM slop content websites, but not a single blog written by an LLM becomes widely read among intellectual or elite circles.
As the codegen wave of 2026 hits, many consumers feel a few weeks of wonder and whiplash at the agentic AIs that can now do parts of their job, and at the massive orgy of abundance in software, and then this becomes the new normal. The world of atoms hasn’t changed much. Most people by late 2026 just assume that AIs can do basically everything digital or intellectual, and become surprised when they learn of things that the AIs can’t do.
Alignment research & AI-run orgs
In 2025, someone adds some scaffolding on top of an OpenAI Operator instance, making it in-theory capable of earning money through freelance work to pay for its own API costs, including automatically buying more credits for itself and find more freelance work. However, the economics doesn't work out, so it can't actually survive on its own without subsidies. In early 2026, a similar concept actually is economically-viable, and some are launched as an experiment by tech-savvy freelancers looking for some easy money, or by people who are just curious. A few blow up, mostly by doing various things related to memecoin manias and going viral as a result. In late 2026, one such autonomous AI scaffold with a memecoin windfall reasons about next steps, tries to incorporate a US business for itself, cold-emails a bunch of humans to ask for ID, and manages to get one of them to give an ID so it can incorporate the business. By 2027, there are a few experimental digital businesses run by AIs, but they're not very competitive, and often rely on what's effectively a subsidy in human interest due to them being novel.
Alignment research in 2025-2027 is driven by Anthropic (though of course most of their research is on GPU performance engineering, inference-time compute techniques, and other things focused on raw capabilities progress). SAEs peak in popularity in late 2024 before being mostly forgotten, but there’s a new interpretability paradigm that starts being put together in late 2025 based on identifying more general geometric structures in activation space. AI control setups are tested against misalignment “model organisms” that, by 2027, are trivially capable of hacking out of a normal environment. Model weight security at Anthropic is excellent for a private company, but this just means the attackers target OpenAI instead (and the gap between labs and open source is never more than a year in 2025-2027). And, of course, Anthropic internally writes endless safety cases. The general message in them is that a lot is resting on either an interpretability breakthrough or on AI control working on superhuman models. The low amount of evidence gained on “alignment” is frustrating to many; models have been caught scheming endlessly but always in fairly artificial setups, or in messy circumstances where it's not clear what the model should've done. The most important work seems to be work on properties upstream of scheming, such as a stream of work on corrigibility kickstarted by the 2024 Greenblatt et. al. paper "Alignment faking in large language models". The alarmingness of the early evidence against corrigibility was offset by promising empirical work on meta-learning techniques to encourage corrigibility in late 2025 and early 2026. By 2027 it's known how to train a model such that it either will or won't be amenable to being trained out of its current goal. Anthropic reveals this and some other safety-related insights to OpenAI and Google, and asks the State Department to reveal it to Chinese labs but is denied.
By 2027, the new interpretability paradigm is seeing progress, with AIs doing essentially all of the engineering and much of the detailed ideation. This reveals a taxonomy of patterns and feature representation types within neural networks. A few are neat and clean, but mostly models’ internals turn out to be messy and with massive redundancy between different parts. The notion of a model having a singular “goal component” looks less likely, at least if certain choices are made during training.
A test case of the new alignment techniques at Anthropic is the training in 2027 of a new model, Claude 5 Epic or just "Claude Epic", based on Claude 5 Opus -curated training data. Company leadership internally thinks it will be a full AGI. The interpretability team will be observing the model at checkpoints and watching it develop. Countless safety cases have been written; the hope is still to run evals, use AI control setups, and hope for some last-minute firmer guarantees from the interpretability work. Some at Anthropic are entirely convinced just by the scalable oversight work that’s already been done. Others expect the hard part of intent alignment to rear its head at any moment.
One of the avenues that seemed most promising in 2025 was interpreting AI chains-of-thought (COTs), something far easier to make meaningful progress on than interpretability. However, over 2026-2027, much more compute is poured into RL, and the COTs become less legible, as the models drift towards shorthand scripts that are more effective for them than writing out their thoughts in English. Work done by Anthropic and several academic labs leads to techniques for encouraging human interpretability of the COTs, by adding a COT interpretability term to the RL loss function and having some clever training details to avoid the model goodharting the interpretability term. However, this comes at a hit to performance. By 2027, another line of work is humans studying model COTs in detail and learning the ways it thinks; some mathematicians in particular pick up neat mental tricks from studying the COTs of models. However, overall COT interpretability declines, and it's generally accepted we won't know exactly what the models are thinking or why, even if COT analysis and the new interpretability techniques can give some general understanding in 2027.
By 2027, evaluations are showing that frontier models—including open-source models—could meaningfully help in engineering pandemics, if bad actors so chose. There's a messy but moderately effective effort by AI safety organisations and several agencies within governments to have some sort of misuse mitigation measures in place, in particular for API-accessible models. However, in the absence of a major incident, governments don't care enough, and open-source models seem hard to contain. Also, some bioterrorism continues being blocked by wet lab skills and just continuing good luck regarding the absence of a motivated bioterrorist. The other potentially catastrophic source of misuse is cyber, but it increasingly seems like this will be solved by default, in particular because AIs are good at writing secure code and formal verification is increasingly used for critical code.
The previous year of insane AI codegen stuff going on everywhere and the continued steady progress in AI has made it more intuitive to people that there won’t be a lot of “money on the table” for some nascent AGI to eat up, because it will enter a teeming ecosystem of AI systems and humans and their interactions. For example, though there are technically some self-sustaining AIs paying for their server costs, they struggle to compete with purposeful human+AI entities that deliberately try to steal the customers of the AI-only businesses if they ever get too many. The cyber competition is also increasingly tough, meaning that any single rogue AI would have a rough time defeating the rest of the world. However, no evidence by the end of 2027 has ruled out a sharper takeoff, and those who believe in it are increasingly either frantic and panicking, or then stoically equanimous and resigned, expecting the final long-term agentic planning piece to slot into place at any moment and doom the world. Also, the labs are openly talking about recursive self-improvement as their strategy.
Government wakeup
In 2025 in the Chinese government, thinking about AGI is stuck somewhere inside CCP internal machinations. Xi Jinping has heard of it and occasionally thinks about it, but doesn’t take it seriously as a near-term thing. However, some senior staff are properly "AGI-pilled" (and split between advocates of safety and racing, but without an overwhelming favourite yet, though also it’s clear that once the issue does get serious, national security will be by far the loudest voice in the room).
The EU is preparing to burden its (small) AI sector with bureaucracy (the EU AI Act). In 2025-2026, there are some high-profile cases of AI companies not offering services in Europe due to regulations. However, the EU & UK governments are best at tracking AI because of their efforts on safety research through the EU AI Office and the UK AISI. The UK government also announces planning reforms targeted at letting private companies build data centres more easily.
Going into 2026, India is trying to ride a new IT wave using open-sourced Llama, DeepSeek, and Mistral models to create products to try to replace its current IT services -heavy export industry. A lack of domestic data centres is making this harder, and political tangles make fast movement hard.
In 2025, the Trump administration tightens export controls on China (in particular preventing even older Nvidia chips like the H20 from being sold to China), tries to pressure other countries to not buy Chinese GPUs, and makes it easier to build power (especially nuclear & gas) and data centres within the US. Otherwise there is little US political action on AI. Behind the scenes, the defense establishment gets more involved in the AI scene. There are secret NSA and CIA projects researching AI for offensive & defensive cyber. More Chinese infiltration of American cyber systems is discovered. High-level government conversations behind closed doors are upping the apocalyptic rhetoric about how essential it is for the US to win in AI-powered cyber. All the major US AI labs have some government partnership related to this.
As internal CCP machinations grind along, and the evidence about big AI effects on programming rolls in through late 2025 and 2026, the CCP gets more serious about AI. Like in the US, once the strategic and national security implications rise in salience, other issues (including safety) fall. The CCP prepares their 15th Five Year Plan for 2026, which involves massive subsidies and investment for AI. DeepSeek leads the domestic AI industry, but the CCP has made it clear they will make the big calls. There is a conversation behind closed doors about whether to end the open-sourcing of DeepSeek models, but the CCP comes out in favour, in particular to try to get the rest of the world to build on top of Chinese AI models (and also helped by the press that the early 2025 DeepSeek R1 release caused). Huawei is shipping GPUs that are only about 12-16 months behind Nvidia. China’s worse startup ecosystem means that AI agent adoption is slower than in the US, though. However, China’s surveillance state has been on an AI adoption spree. In particular, censorship is instantaneous with LLMs. By 2026, there are widespread educational "Xi Jinping Thought AI Tutors" that most CCP members are mandated to have weekly sessions with. Retaining control of society now seems increasingly easy, allowing the CCP to focus more on geopolitics and military, and less on the consumer economy. At the same time, Xi Jinping has an overly-rosy view of Chinese military AI capabilities because people tell him what he wants to hear.
There's a shadow conflict playing out, almost entirely out of public attention, between US and Chinese cyber forces trying to get into each other's critical infrastructure while reducing the extent to which their own infrastructure is compromised. Contrary to publicly-available information, America probably has the upper hand, but it's also clear that both could inflict high damage on the other.
AI starts to figure in US domestic politics in 2026, but is not yet a top issue. The upcoming replacement of most human white-collar work looks more and more plausible, especially after OpenAI's release of o6. Job losses are not yet high, though, as human orgs take time to react to change. Even in software, where mass firings could perhaps most plausibly be done, many are afraid to try it first. Non-technical managers tend to treat the technical stuff as blackbox wizardry and are scared to break it, and technical managers don't want to reduce the size of their empires. The main effect is that hiring new software engineers basically stops, but the disaffected—a small group of nerdy, elite-coded, low-voting-rate youngsters—are not politically important. Other white-collar office jobs are also reducing entry-level hiring, as increased demand for productivity is instead met by existing employees just using AI more.
The US government, like China, decides against legally restricting the open-sourcing of AI models. This is influenced by pro-innovation arguments, China doing the same, and the defense-specific AI programs being done under classification with closed-source models anyway. The AI labs have also grown more reliant on government cooperation for things like power grid connection permits, data centre construction permits, and lobbying to avoid ruinous tariffs on GPUs. They also all want the money flow of Pentagon contracts and the prestige of working on US defense. This means that there's a tacit agreement that if the government hints they should or shouldn't do something, they are very likely to march to that beat.
Starting in late 2026, many of the governments worried about fertility decline get concerned about everyone talking to AIs instead of each other. South Korea bans “personalised AI companions” in 2027, and the EU requires people to register if they use them and imposes various annoying limits that drive down usage. However, the addicts can just use open-source models to circumvent regulations. Some countries spend lots of money on getting the "creator influencers"—influencers turbo-charged by generative AI—to extol the virtues of marriage and kids. By 2027, though, the more forward-looking politicians are—in private—starting to realise that once the economy transitions to being AI-powered, national interests are not harmed if the human population plummets. The “intelligence curse” is starting to set in.
Thanks to Luke Drago, Duncan McClements, Theo Horsley, and Bilal Chughtai for comments.