Computational creativity and AlphaEvolve

In Big Data and Graph classes, we explore the power of evolutionary algorithms: systems inspired by natural evolution that can generate intelligent solutions on their own. What's fascinating is that these methods are not only working, they're solving problems that once seemed stagnant: from matrix multiplication to hardware design. One of the most surprising advances is AlphaEvolve, a Google DeepMind system that combines language models with computational evolution to discover new algorithms. From this experience, I wrote this article to understand how this system works.

The image at the top of this note shows an experiment of my own where I tried to reproduce the logic of AlphaEvolve by applying it to the classic problem The Kissing Number Problem. Although my version differs from the original DeepMind model, it retains its spirit: directed evolution, adaptive mutations, and rich telemetry. Instead of C programs, I worked with 3D spheres; instead of an LLM, I applied geometric operators. The result: a visual and executable demo in Colab to explore how efficient patterns emerge. This exercise sought to bring complex ideas to an interactive and understandable educational environment.

1. Introduction and Motivation

In the history of computing, the design of algorithms has been an eminently human task, guided by mathematical intuition, heuristic creativity and formal reasoning. From the classical algorithms of Euclid to modern techniques in deep learning, the search for efficient algorithmic solutions has been one of the most powerful engines of scientific and technological progress. However, this tradition of manual design presents fundamental limitations in the face of the increasing complexity of contemporary problems.

First, many domains of mathematics and computer science present immense, unstructured, and highly non-convex solution spaces, where intuitive human strategies are inefficient or outright inoperable. Matrix multiplication, for example, is a seemingly simple problem with internal structures still unexplored; although known since Strassen's work in 1969, new improvements for modest dimensions (such as 4×4 matrices) are hardly being discovered today thanks to non-traditional methods. The same is true in discrete geometry, combinatorial optimization, hardware design, and training large-scale AI models.

In this context, a new category of algorithmic discovery systems emerges that combine large language models (LLMs) with automatic evaluation techniques and programmatic evolution. These systems are no longer limited to completing code or generating syntactically valid solutions: they can propose, explore and refine complex solutions with semantic and adaptive structure. The most advanced proposal in this line to date is AlphaEvolve, an evolutionary coding agent developed by DeepMind that seeks to automate algorithm discovery through iterative cycles of mutation, evaluation, and selection, guided by the implicit reasoning of LLMs such as Gemini Pro.

Unlike previous approaches, AlphaEvolve is not solely looking to find a good solution within a given domain. Its objective is to build, in a partially autonomous way, new, interpretable and efficient algorithms, capable of addressing unsolved problems or improving existing algorithmic structures. The system represents a paradigm shift: from manual algorithm design to AI-assisted evolution, where the human role shifts from direct discovery to problem formulation, process monitoring, and interpretation of results.

2. What is AlphaEvolve?

AlphaEvolve is an algorithmic discovery evolutionary agent developed by DeepMind that combines the generative versatility of language models (LLMs) with principles of computational evolution and automatic verification. Its objective is not only to find valid solutions to complex computational problems, but also to systematically explore the space of possible programs, discovering new algorithmic structures through guided mutations and rigorous evaluation.

In functional terms, AlphaEvolve operates on an iterative cycle of program generation, evaluation, selection, and mutation, following an architecture inspired by classical evolutionary algorithms. However, it differs from these in three key ways:

1. LLM-assisted semantic mutation: Unlike traditional random mutations (such as bit reversal or node rearrangement), AlphaEvolve uses large language models, specifically Gemini Pro and Gemini Flash, to apply structural transformations to the source code. These mutations consist of operations such as SEARCH/REPLACE or functional modifications, but performed with semantic knowledge: the model understands the purpose of a piece of code and proposes plausible variants that can preserve, enhance, or modify its behavior. This approach allows for the generation of syntactically correct and conceptually different children, thus increasing the diversity and depth of the exploration.

2. Automated assessment and fitness function: Each program generated is executed and evaluated through an objective and quantifiable evaluation function. Depending on the domain, this function can measure the mathematical correctness of an output, the computational efficiency of a procedure, the quality of a geometric solution, or even the speed of convergence in an AI training problem. This metric fulfills the role of fitness in the evolutionary sense, and is used to select the most promising programs in each generation.

3. Structured population and directed evolution: AlphaEvolve maintains an active population of programs, which represents a subset of the total possible solution space. In each generational cycle, certain programs are selected as parents, variants (children) are generated through mutations, and the population is updated by retaining the best solutions according to their fitness score. Unlike classical biological evolution, there is no crossover between individuals, but a targeted and intelligent mutation, which allows convergence to be accelerated without sacrificing diversity.

In essence, AlphaEvolve can be understood as a hybrid system of heuristic search and algorithmic generation, in which the exploration of the solution space is carried out with the help of the semantic power of LLMs, while the exploitation of viable solutions is carried out by evolutionary selection mechanisms. The system behaves as an autonomous agent that not only tests alternatives, but also structurally learns to transform code into useful directions, making use of the statistical knowledge implicit in language models.

This architecture allows it to adapt to multiple application domains, from classic discrete math problems (such as the kissing number) to deep training system optimization, digital circuit design, and generation of high-performance kernels for GPUs. The overall nature of its design makes it a universal algorithmic search framework, the scope of which transcends any specific domain.

The image below represents how AlphaEvolve automates the algorithmic discovery cycle, with the human acting as the initial configurator and the LLMs as evolutionary agents within an organized and evaluated system with quantitative criteria.

3. Featured Apps

The impact of AlphaEvolve is not merely theoretical. Since its implementation, this evolutionary agent has produced tangible and measurable advances in key domains of computing, mathematics, and systems engineering. These applications stand out for sharing a common characteristic: they are problems with deep internal structures, where traditional methods had already reached apparent limits, and where automatic exploration guided by artificial intelligence has revealed new solutions.

3.1 Efficient matrix multiplication

Matrix multiplication is an elementary operation in linear algebra, but also one of the fundamental bottlenecks in scientific computing, graphics, neural network training, and signal processing. Since Strassen's algorithm (1969), which reduced algorithmic complexity by finding faster algorithms, it has been a high-level mathematical challenge to multiply 4X4 matrices with 49 operations.

AlphaEvolve improved on this mark by discovering an algorithm for multiplying 4×4 matrices using only 48 scalar multiplications, surpassing even the results obtained by its predecessor, AlphaTensor, which had focused on binary optimization for specific dimensions. This advance not only implies a theoretical improvement, but can also be implemented in hardware and kernels to obtain real gains in computational efficiency. The ability to rediscover and improve such a fundamental procedure suggests a new paradigm in symbolic algebraic optimization.

3.2 Optimizing AI Model Training

AlphaEvolve has been used to accelerate essential components in the Gemini architecture, including its core training based on large-scale operations. By finding more efficient ways to break down complex matrix operations into subproblems, the system was able to speed up one of the most critical routines in the training pipeline, resulting in a 1% reduction in total training time.

Beyond the one-off savings, this result shows how AlphaEvolve can automate a process that usually requires weeks of specialized engineering. With this system, improvements can be discovered in a matter of days, without compromising interpretability or functional validation.

3.3 Digital Hardware Design

One of the most innovative uses of AlphaEvolve was in the context of circuit design for Tensor Processing Units (TPUs). By intervening directly into Verilog code, the standard hardware design language, AlphaEvolve proposed rewrites that eliminated unnecessary bits in optimized arithmetic modules. These rewrites were automatically verified and proved to be correct, efficient, and, crucially, compatible with the human design flow.

This marks a radical change: it is no longer just a matter of generating software code, but of co-designing digital architectures, establishing a bridge between AI and microelectronics at a structural level.

3.4 Open-ended math problems: the kissing number

In pure mathematics, AlphaEvolve tackled the kissing number problem —an age-old problem of packing spheres into high-dimensional spaces—. In its 11-dimensional variant, the system discovered a configuration with 593 tangent spheres, improving the lower limit known until that time. He achieved this by designing a heuristic algorithm based on gradient optimization, which is automatically coded.

This type of contribution has two profound implications:

• Suggests that AlphaEvolve can do active mathematical research.
• Demonstrates their ability to operate on abstract geometric structures with physical and algebraic interpretation.

The image at the top of this article is a representation of The Kissing Number Problem.

3.5 GPU kernel optimization

AlphaEvolve also intervened in domains where human engineering was supposed to have already reached the limit: low-level CUDA kernel optimization. In particular, the system was able to accelerate the performance of the FlashAttention kernel, a critical component in Transformer models, by up to 32.5%. This is notable because these kernels are already extremely fine-tuned by specialized compilers such as Triton or TVM, and substantial improvements are rarely achieved without manual redesigns.

The fact that AlphaEvolve can mutate low-level instructions, without breaking compatibility or semantics, suggests a compiler-heuristic-type optimization capability that no other autonomous system had achieved.

4. Implications and Future

The emergence of systems like AlphaEvolve marks a turning point in the paradigm of algorithmic discovery: we went from an era where computational knowledge was designed manually, to a new stage where algorithms can emerge, evolve and adapt through semi-autonomous processes guided by language models. This transition not only redefines the role of the engineer or computational scientist, but also opens philosophical, technical, and epistemological questions about the place of artificial intelligence in the development of knowledge.

4.1 Epistemological and scientific implications

AlphaEvolve challenges the traditional way we think of mathematical and computational discovery. For the first time, it is possible for a machine to:

• Pose algorithmic hypotheses creatively.
• Code functional solutions without direct human intervention.
• Iteratively improve your proposals based on objective assessments.

This type of agency poses a new category of scientific production: research evolutionarily assisted by AI, where the human being does not act as a direct creator of solutions, but as an architect of the exploration space and supervisor of the evaluation process. In this sense, AlphaEvolve represents a kind of "algorithmic discovery assistant" that operates under selection criteria, but with semantic creative freedom.

4.3 Limitations and open challenges

As I mentioned in the class, despite its achievements, AlphaEvolve faces significant limitations, among which it is worth highlighting:

• Despite its remarkable advances, AlphaEvolve faces profound limitations that must be considered from both a technical and epistemological perspective.
• Technical limitations. The system does not guarantee global convergence, and it can stagnate at local optimums. Its architecture does not ensure an ergodic coverage of the search space, and it depends critically on external mechanisms to maintain evolutionary diversity. In addition, certain transformations of the code are difficult to interpret, especially when they emerge from successive chains of non-trivial mutations.
• Epistemological limitations. AlphaEvolve does not produce explanations, nor does it generate formal intuitions about the discovered algorithms. This raises the question of whether we are dealing with genuine discoveries or simply instrumental findings with no associated theory. Ultimately, the interpretive burden falls on the human, who must translate these discoveries into explicit mathematical knowledge.
• Practical limitations. Finally, not all problems can be formulated in such a way as to allow for automatic evaluation. This restricts AlphaEvolve's domain of applicability to highly formalizable and computationally assessable contexts.

These limitations imply that the human role remains essential, not only as a designer of the system, but also as an interpreter, evaluator, and regulator of the evolutionary process. The collaboration between natural and artificial intelligence is not a replacement, but a strategic symbiosis to expand the boundaries of knowledge.