Show original animation
The purpose of this article is to analyze how artificial intelligence is being applied in fields such as biology. A recent paper, "Simulating 500 million years of evolution with a language model", presents a significant advancement in the application of AI to molecular biology. It draws an interesting analogy between large-scale language models (LLMs), which generate text by analyzing linguistic patterns, and their ability to generate novel proteins by learning patterns in amino acid sequences.
1) The Importance of Proteins in Life and Science
Proteins are fundamental macromolecules essential for life, playing a central role in the biology of all organisms. These structures, composed of amino acid sequences, are involved in a wide variety of cellular processes and are crucial for maintaining and ensuring the functionality of biological systems. From catalyzing chemical reactions to regulating gene expression, proteins serve as the structural and functional building blocks of life.
2) Protein Space and Its Exploration
The protein space refers to the set of all possible amino acid sequences that can form a functional protein. Since proteins are composed of combinations of 20 essential amino acids and can vary in length from a few dozen to several thousand amino acids, the number of possible sequences is astronomically large.
However, within this vast space, only a tiny fraction corresponds to proteins that are both functional and stable in a biological environment. Evolution has explored only a portion of this space over billions of years, selecting proteins that provide an adaptive advantage to living organisms.
3) Unexplored Regions (Proteins That Could Exist)
Since evolution has only tested an infinitesimal fraction of the protein space, there are regions containing proteins that are still unknown but could have useful functions. These unexplored regions may contain proteins with properties that nature has never developed.
Artificial intelligence has revolutionized computational biology, enabling the design of new proteins with optimized and completely innovative functions. One of the most significant breakthroughs in this field is ESM3, a multimodal Transformer-based deep learning model that allows for the efficient prediction, analysis, and design of proteins.
The following table outlines its main characteristics:
| Aspect | Description |
|---|---|
| Language model applied to proteins | ESM3 employs an architecture similar to natural language models (such as GPT-4) but adapted to protein amino acid sequences. It works through token prediction, where each amino acid is treated as a "word" within a sentence. |
| Multimodal tokenization | Represents not only the amino acid sequence but also the three-dimensional structure and biological function of proteins. It uses advanced encoding techniques to map proteins into a deep learning space. |
| Bidirectional self-attention | Allows the model to capture long-range relationships within a protein, which is crucial for predicting its structure and function. It employs a masked token mechanism to train on predicting hidden protein fragments, similar to how a language model completes incomplete sentences. |
| Protein generation and optimization | Can generate new proteins following specific instructions, such as "design an enzyme that degrades plastics" or "create a fluorescent protein." It uses iterative optimization techniques to enhance the properties of generated proteins. |
Source: Own Elaboration
Using language models for protein design might seem counterintuitive at first, as these models were originally developed to process written texts. However, proteins can be considered biological languages, where amino acids function as "words" following structural and functional patterns.
The following image, taken from the paper, visually demonstrates how language models can be used to "write" new proteins, validating the idea that proteins can be understood as a "biological language" where amino acids function as words.
4) Results
The ESM3 model has been used to design esmGFP, a new fluorescent protein with no direct equivalent in nature. This protein is functionally similar to GFP (Green Fluorescent Protein), widely used in biotechnology and biomedicine to tag and visualize cellular processes.
However, esmGFP differs from natural fluorescent proteins because its amino acid sequence was entirely generated by artificial intelligence. Most notably, its sequence identity is 500 million years of evolution apart from any known fluorescent protein, meaning its design did not follow a natural evolutionary path but rather was a computational exploration of new amino acid combinations that enable fluorescence.
5) Reflection
A fascinating element is the idea that proteins can be considered "biological languages," which aligns perfectly with how ESM3 generated esmGFP. Just as natural language models can learn the grammatical structure of a language and generate coherent sentences, ESM3 has learned the patterns governing protein sequences and functions, producing a new fluorescent protein without following conventional biological evolution.
The following table reflects this parallel:
| Natural Language Models | Language Models for Proteins (ESM3) |
|---|---|
| Learn the grammar and syntax of a language | Learn the structural grammar of proteins |
| Predict the next word in a sentence | Predict amino acids in protein sequences |
| Generate coherent texts from prior data | Generate functional proteins from evolutionary patterns |
| Can write sentences never seen before | Can design proteins never before discovered |
Source: Own Elaboration
If we think about it, the protein was generated using a Transformer-based language model, demonstrating that the rules governing amino acids can be modeled similarly to the rules governing human language. The idea behind this is that if AI models can generate comprehensible texts by learning linguistic structures, they can also generate functional proteins by learning evolutionary patterns in amino acid sequences. In this case, the "language of life" can be deciphered and manipulated using AI, accelerating the exploration of the protein space far beyond what evolution has achieved in millions of years.
The following image is a representation of this protein:
References
Simulating 500 million years of evolution with a language model
Your reading notebook
The note is saved only in this browser.
This archive note retains its original publication context.
View original archive file ↗