AlphaEvolve: A Gemini-Powered Coding Agent for Designing Advanced Algorithms¶

Designing a new algorithm requires:
  1. Deep mathematical intuition
  2. Patience to test thousands of variants
  3. Skill to verify correctness
  4. Creativity to escape local optima

Strassen (1969) found that 2x2 matrix mul needs 7 (not 8) multiplications.
For 56 years, nobody improved 4x4 complex-valued matrix multiplication.
  Best known: 49 scalar multiplications (Strassen recursive)
  Lower bound (theoretical): unknown
  Search space: astronomically large
  Verification: exact algebraic identity must hold

Ask Gemini: "Design a faster matrix multiplication algorithm."

Problems:
  - LLM hallucinates incorrect identities
  - Cannot verify its own arithmetic
  - One-shot generation gets stuck on textbook answers
  - No mechanism to refine through failure
  - No memory of which paths were dead ends

Even reasoning models (o1, Gemini Thinking) plateau quickly:
  Single-shot reasoning doesn't explore the search space
  Cannot accumulate insight across thousands of attempts

Classical evolutionary algorithms:
  - Random mutations rarely produce valid programs
  - Need handcrafted DSLs (Domain Specific Languages)
  - Cannot leverage prior knowledge
  - Scale poorly to complex algorithm spaces

AlphaTensor (2022): Used RL on a custom tensor game.
  Worked, but limited to one specific problem.
  Each new problem = new game = new RL training run.

Need: A general-purpose algorithm discovery engine that
  1. Writes real code (not toy DSLs)
  2. Evolves over thousands of iterations
  3. Verifies correctness automatically
  4. Generalizes across problem domains
  5. Improves on human-authored SOTA

                    +---------------------------+
                    |   Problem Specification   |
                    |  (code skeleton + eval)   |
                    +-------------+-------------+
                                  |
                                  v
        +---------------------------------------------------+
        |              Population Database                  |
        |  (programs, scores, lineage, prompt history)      |
        +---------------------------------------------------+
              |                                ^
              | sample parents                 | insert child
              v                                |
        +-----------+   prompt    +------------+----+
        |  Prompt   +-----------> |  Gemini Flash   |
        | Sampler   |             |  + Gemini Pro   |
        +-----------+             +----------+------+
                                             |
                                             | new code
                                             v
                                  +----------+--------+
                                  |  Evaluator Pool   |
                                  |  (run tests, get  |
                                  |   numerical score)|
                                  +----------+--------+
                                             |
                                             v
                                       insert into DB

Gemini 2.0 Flash:
  - Fast, cheap, high throughput
  - Generates the bulk of mutation proposals
  - Used for breadth: explore many directions

Gemini 2.0 Pro:
  - Slower, more capable, higher cost
  - Used sparingly for hard cases and refinement
  - Used for depth: produce careful, complex edits

The ensemble works because evolution rewards quantity:
  Most Flash mutations fail; cheap to discard.
  Pro mutations succeed more often when stuck.
  Combined cost-per-discovery is dominated by Flash.

AlphaEvolve does NOT ask the LLM to rewrite full programs.

Instead, it requests structured diffs:

  <<<<<<< SEARCH
  def matmul_block(A, B):
      C = np.zeros((4, 4))
      for i in range(4):
          for j in range(4):
              for k in range(4):
                  C[i,j] += A[i,k] * B[k,j]
      return C
  =======
  def matmul_block(A, B):
      # 48-multiplication scheme discovered by AlphaEvolve
      m1 = (A[0,0] + A[1,1]) * (B[0,0] + B[1,1])
      ...
      return assemble(m1, m2, ..., m48)
  >>>>>>> REPLACE

Benefits:
  - Localizes mutation to one region
  - Preserves correct surrounding code
  - LLM stays focused
  - Diffs apply atomically; failures are easy to revert

For each new generation, the sampler builds a prompt with:

  1. Problem description (static)
  2. Best programs found so far (top-k by score)
  3. Recent diverse programs (for exploration)
  4. Inspirations: human-authored references when available
  5. Critic feedback: what failed last time
  6. Few-shot examples of successful diffs

This is essentially RAG over the program archive.
The prompt itself evolves as the population evolves.

Every problem requires a programmable scorer:

  evaluate(program) -> float (higher = better)

Examples:
  - Matrix mul: count scalar multiplications, verify identity
  - Sorting: average comparisons over benchmark inputs
  - Scheduler: simulate cluster, measure cost-per-job
  - Kernel: run on TPU, measure wall-clock latency
  - Hardware: synthesize circuit, report area + power

The scorer must be:
  - Deterministic (or low-variance)
  - Fast (called millions of times)
  - Correctness-checking (reject invalid programs)
  - Numerical (so evolution can rank)

Standard 4x4 matrix multiply: 64 scalar multiplications
Strassen recursive (1969):    49 scalar multiplications

For complex-valued 4x4 matrices, 49 stood as the record
for 56 years. No human improved it.

AlphaEvolve discovered: 48 scalar multiplications.

AlphaTensor:
  - Specialized RL agent
  - Trained on a custom tensor decomposition game
  - Found 47 multiplications for 4x5 by 5x5 mod 2
  - Did NOT improve 4x4 complex case
  - Required separate training for each problem

AlphaEvolve:
  - General-purpose evolutionary code agent
  - Same system that does scheduling and kernels
  - Found 48 multiplications for 4x4 complex
  - Beats AlphaTensor on 14 of the matrix-mul targets it tested
  - No retraining per problem

Key advantages over AlphaTensor:
  1. Operates on real code, not abstract tensors
  2. Can express continuous-valued algorithms
       (AlphaTensor was restricted to specific finite fields)
  3. Leverages Gemini's prior knowledge of math
  4. Evolution explores larger neighborhoods than RL search
  5. Self-supervised by exact algebraic verification

Problem: Bin-pack jobs onto Google's machines.
  Google's existing scheduler is decades-mature.
  Any improvement compounds across millions of machines.

AlphaEvolve produced: a new heuristic function for Borg.
  Continuously running in production since rollout.
  Recovers ~0.7% of Google's worldwide compute capacity.
  Translates to thousands of machines worth of headroom.
  Code is human-readable and was reviewed before deployment.

Problem: Optimize Verilog circuits for TPU arithmetic units.
  Hand-tuned by hardware engineers for years.

AlphaEvolve found: a circuit-level rewrite of a key arithmetic block.
  Verified by hardware engineers as correct.
  Integrated into an upcoming TPU.
  First time an LLM-derived design has shipped in Google silicon.

Problem: Speed up the matrix-multiply kernel that
         dominates Gemini training cost.

AlphaEvolve generated tiling and pipelining variants:
  - 23% speedup on the target kernel
  - 1% reduction in total Gemini training time
  - Translates to days of accelerator time saved per training run

For FlashAttention specifically:
  - 32.5% speedup on the FlashAttention kernel
  - Discovered novel block-size and pipelining schedule

AlphaEvolve generated faster sorting for small arrays
(the hot path inside libstdc++-style sort routines).

Better than the human-tuned versions used in production code.
Now compiled into Google's internal libraries.

DeepMind tested AlphaEvolve on 50+ open problems in
mathematics and computer science.

Results:
  ~75% of problems: matched the best human-known solution
  ~20% of problems: BEAT the best known solution

Examples:
  - Kissing number bounds in high dimensions
  - Erdos minimum-overlap problem
  - Sums-of-squares decompositions
  - Ramsey-type constructions

FunSearch (2023, predecessor):
  - Operated on a single Python function (the "priority function")
  - Limited to problems expressible as scoring a function
  - Required heavy human framing of each problem

AlphaEvolve:
  - Evolves arbitrary blocks of arbitrary languages
       (Python, C++, JAX, Verilog, even pseudo-code)
  - Multi-file evolution: can mutate several locations at once
  - Scales from one-line tweaks to whole-module rewrites

Traditional EA:
  - Sequential generations
  - Wait for whole population to evaluate
  - Wastes time on slow evaluations

AlphaEvolve:
  - Continuous insertion: any worker can submit any time
  - Diverse sampling: avoid premature convergence
  - Map-elites style archive: keep best per niche
  - Scales to thousands of parallel evaluators

Naive mutation: edit a random token.
  -> Almost always produces broken code.

AlphaEvolve mutation: ask Gemini for a targeted diff.
  -> Almost always parses, often compiles, sometimes scores higher.
  -> The LLM imports decades of programmer intuition for free.

This is the core wager of the paper:
  "Evolutionary search becomes tractable when the mutation
   operator already knows how programs work."

Cheap pre-screen: compile + tiny test
  -> Discard 80% of mutations in milliseconds.

Medium screen: standard test suite
  -> Discard another 15%.

Full evaluation: real benchmark
  -> Run only 5% of candidates at full cost.

This 100x's the effective search budget.

The same Gemini model is used across:
  - Pure math (matrix multiplication)
  - Discrete optimization (scheduling)
  - Hardware (Verilog circuits)
  - Software (training kernels)
  - Combinatorics (sorting)

Specialization happens entirely in the prompt and evaluator.
No fine-tuning, no RLHF, no per-domain training run.

                  | FunSearch (2023) | AlphaTensor (2022) | AlphaEvolve (2025)
------------------+------------------+--------------------+--------------------
Substrate         | one Py function  | tensor decomposition| arbitrary code
LLM               | Codey (small)    | none (RL only)     | Gemini Flash + Pro
Search            | evolutionary     | RL with MCTS       | evolutionary
Domains           | math + combin.   | matrix mul only    | math, sched, hw, sw
Production use    | none             | none               | Yes (Google scale)
Beat human SOTA?  | sometimes        | yes (some cases)   | yes (many cases)
Per-problem train | no               | yes                | no
Code generated    | tiny snippets    | n/a                | full programs/diffs

+-------------------------------------------------------------+
|                    Problem Specification                    |
|  - Initial program (code skeleton)                          |
|  - Evaluator (programmable scorer)                          |
|  - Optional: human references, hints, constraints           |
+-------------------------------------------------------------+
                              |
                              v
+-------------------------------------------------------------+
|                  Distributed Controller                     |
|  - Maintains population database                            |
|  - Schedules workers across thousands of cores              |
|  - Handles checkpointing, retries, observability            |
+-------------------------------------------------------------+
                |                            |
                v                            v
+----------------------------+    +-----------------------------+
|       Mutation Workers     |    |     Evaluation Workers      |
|  - Sample parents          |    |  - Run cascade (cheap->full)|
|  - Build prompt            |    |  - Verify correctness       |
|  - Call Gemini Flash/Pro   |    |  - Compute fitness score    |
|  - Apply diffs             |    |  - Report back              |
+----------------------------+    +-----------------------------+
                |                            |
                +------------+---------------+
                             v
                  +---------------------+
                  | Population Archive  |
                  | (best-per-niche +   |
                  |  diverse sample)    |
                  +---------------------+

Even though AlphaEvolve is internal to DeepMind, the recipe
is reproducible. To build something similar:

1. A frontier code-capable LLM (Gemini, Claude, GPT-4-class)
2. A programmable, deterministic evaluator
3. An archive that samples by score AND by diversity
4. Diff-based mutation prompts
5. Significant compute (at least 10K-100K LLM calls per problem)

Cost rough order:
  Hard math problems: ~$10K-$100K of LLM compute per discovery
  Production deployment: amortized over years of savings
  Net: extremely positive ROI for Google-scale problems

+ Problem has a fast, deterministic evaluator
+ Solutions are short-to-medium length code
+ Existing human solutions exist as starting point
+ Improvements are continuous (not one-shot binary)
+ Domain has rich structure the LLM has seen before

Example: Matrix mul, kernels, schedulers, circuits.

- No clean evaluator (e.g. "is this proof elegant?")
- Solutions require novel mathematical concepts not in training data
- Search space is purely combinatorial with no smooth gradient
- Evaluator is expensive (>1 minute per run)
- Problem requires real-world experiments

For: It beat 50+ year old records held by top mathematicians.
Against: It still requires human-authored evaluators and skeletons.

Most accurate framing:
  AlphaEvolve is a force multiplier on human algorithmic intuition,
  not (yet) a replacement. The human picks the problem and writes
  the evaluator. The system explores the space at superhuman scale.

Current: AlphaEvolve refines and recombines existing ideas.
Open:    Can it invent genuinely new mathematical concepts?
         Strassen's 1969 paper introduced the idea of bilinear
         algorithms. Could AlphaEvolve introduce a new framework?

So far: All discoveries fit within existing frameworks.

AlphaEvolve only works where verification is easy.
Most of science is the opposite: hypotheses are easy, verification is hard.

For domains without cheap evaluators (biology, social science, physics
experiments), AlphaEvolve's recipe does not directly apply.

This is the fundamental limit on LLM-driven science:
  We can only optimize what we can grade.

AlphaEvolve is a proof of concept that AI can do real research:
  - Original contributions
  - Beat human SOTA on long-standing problems
  - Generate code that ships in production

Five-year question:
  How much of mathematics, computer science, and engineering
  research can be automated this way?

DeepMind's bet: a lot. The recipe is general.

Many important problems lack one.
"Make this UX better" or "prove this theorem elegantly"
have no automatic scoring function.

Tens of thousands of LLM calls per discovery.
Evaluation can dominate cost (e.g. running TPU benchmarks).
Only worth it for high-value problems.

Like all evolution, AlphaEvolve hill-climbs from current population.
Can miss radically different solutions that require
crossing fitness valleys.

DeepMind has not released AlphaEvolve as a tool.
Open-source efforts (OpenEvolve, others) are reproducing
the recipe in 2025-2026.

Every production deployment was reviewed by a human expert
before shipping. AlphaEvolve does not yet self-deploy.

Direct predecessor on matrix multiplication.
AlphaTensor: RL agent on a single problem.
AlphaEvolve: General agent that beats AlphaTensor on the same problems.

Direct predecessor on LLM + evolution.
FunSearch: small LLM, single function, narrow.
AlphaEvolve: frontier LLM, full programs, broad domains.

Sibling project for Olympiad-level geometry proofs.
Both use neural + symbolic loop. AlphaGeometry is theorem-focused;
AlphaEvolve is algorithm-focused.

LLM agent that learns Minecraft skills via code.
Same idea - LLM writes code, environment evaluates -
applied to game-playing instead of algorithm discovery.

Single-prompt search through reasoning steps.
AlphaEvolve generalizes this to multi-day population-level search.

Self-reflection feedback loop for LLM agents.
AlphaEvolve uses similar idea but at population scale instead of
single-trajectory.

Iterative critique-and-improve with LLMs.
AlphaEvolve replaces single-thread refine with parallel evolution.

Any field with code + automatic evaluation:

- Compiler optimization passes
- Database query planners
- Solver heuristics (SAT, MILP, constraint programming)
- Network routing
- Game-playing engines
- Cryptographic primitive design
- Numerical method optimization (ODE solvers, PDE schemes)
- Biology pipeline tuning (where simulators exist)
- Robotics controllers (in simulation)

Hyperscalers (Google, Meta, Microsoft, Amazon) have:
  - Frontier LLMs in-house
  - Massive compute
  - Proprietary problems with built-in evaluators
       (training kernels, ad ranking, scheduling, hardware)

Expect every hyperscaler to build an AlphaEvolve clone for
their own production stack within 12-18 months of this paper.

Open-source clones (OpenEvolve, evolve-py) lower the barrier.
Verticals with strong evaluators (compilers, solvers, EDA tools)
become candidates for "AI does the algorithm design" companies.

Defensibility comes from:
  - Owning the evaluator (proprietary benchmark + verifier)
  - Owning the problem distribution (real customer workloads)
  - Owning the deployment surface (where the algorithm runs)