Agentic Discovery of Neural Architectures: AIRA-Compose and AIRA-Design
A 2026 paper showing LLM agents autonomously designing foundation-model architectures and training scripts, with AIRA-Compose and AIRA-Design producing models and mechanisms that match or beat hand-designed baselines.
Do agent-discovered architectures retain their advantages when scaled beyond one billion parameters under compute budgets comparable to frontier human-designed models?
AIRA matters because it moves automated AI R&D down into model architecture. Agents are not just writing wrappers around existing models; they are exploring computational primitives, scaling candidates, and designing mechanisms for long-range dependencies.
Results to watch
The paper reports AIRAformer and AIRAhybrid families, 1B-scale pretraining runs, faster scaling frontiers than several baselines, and agent-written attention mechanisms that approach human state of the art on Long Range Arena tasks.
ASI relevance
If automated architecture discovery becomes reliable, the capability frontier stops depending only on human researchers proposing the next transformer-class breakthrough. The loop becomes: generate architecture, train, evaluate, extrapolate scaling, retain winners, and hand the archive to the next search.