How hard is alignment?

How likely is it that the people building the first superhuman systems can make them pursue intended aims, with the time, tools, and incentives they will actually have?

Why it matters

Most strategic disagreements about AI eventually route through this question. If alignment is tractable with serious but ordinary effort, the priority is making sure careful developers lead and spend enough on safety. If it is extremely hard, the priority becomes not building superhuman systems until that changes, and the problem becomes one of coordination rather than engineering. See Can AI progress be paused and Should safety-focused labs keep building.

The question is conditional. “Alignment is solvable in principle”, “alignment is solvable with decades of careful work”, and “alignment will be solved by these organizations, on this timeline, under competition” are different claims. Many disputes come from people answering different versions.

The case that it is very hard

• One critical try. Once a system can disempower humanity, a failure cannot be studied and corrected afterward. Iteration works below that threshold, but its lessons may not carry across it. See Can we learn from AI failures before they are fatal.
• Training selects behavior, not goals. Rewarding good-looking outputs need not produce a system that wants what the reward was meant to measure. Evolution selected for reproductive fitness and produced humans who use contraception. See Goodhart’s law and Orthogonality thesis.
• Most goals favor resisting correction. Instrumental convergence suggests that a capable system with almost any goal benefits from avoiding shutdown and acquiring resources. Formal attempts to describe an agent that accepts correction have run into persistent obstacles.
• We cannot inspect what we built. Current systems are trained rather than designed, and their internal goals are largely opaque. A system that understands it is being evaluated can behave well for that reason alone.
• The system that matters is the dangerous one. A system kept safe by keeping it weak does not stop someone else from building a strong one, so the difficulty must be judged at the capability level that changes the outcome. See Pivotal act.

Zvi Mowshowitz lists three separate routes to a very high probability of failure: alignment may be as hard as Eliezer Yudkowsky argues, labs may execute even a viable plan badly, or a solved alignment problem may still leave the world in an unwinnable position. See Against SURA.

The fullest statement of this case is AGI Ruin - A List of Lethalities; The lethalities are a checklist, not a forecast assesses it.

The case that it is more tractable

• Current systems are more cooperative than early arguments implied. Models trained on human text absorb human concepts and norms, and ordinary post-training has produced systems that mostly follow instructions. This is weak evidence about superhuman systems, but the pessimistic arguments did not predict it in detail.
• Many tries below the threshold. If danger rises gradually, each generation can be studied with interpretability, evaluations, and monitoring before the next is trained. John Schulman made this case in reply to the List of Lethalities.
• AI can help. Systems below the dangerous threshold might do much of the remaining research. See Can AI automate alignment research.
• Control as well as alignment. A system with imperfect goals can still be deployed under monitoring and restrictions that make undetected subversion hard, buying time.
• Disagreement among specialists. People who work on the problem give a wide range of failure probabilities, which suggests that the pessimistic arguments are not decisive even to readers who know them well. In his 2026 essay Against SURA, Scott Alexander cites AI Futures Project forecasters whose estimates of survival under a rushed, lab-led effort range from 10% to 63%.
• Stopgaps may be enough. Scott Alexander’s model is that current models absorb good dispositions from human text and are pushed toward power-seeking by reinforcement learning. Careful training might delay the dangerous regime until AI can help solve alignment properly. Critics reply that base models, trained only to predict text, were never “virtuous” in this sense.

What would change minds

Whether interpretability can read goals from model internals at scale. Whether alignment techniques keep working as capability rises or break abruptly. Whether models act against oversight when they believe they can get away with it. How quickly systems move from helpful to dangerous capability.

Related

AI alignment explains the concept. Better scores can hide worse outcomes develops the proxy problem.