Design Proteins Using Large Language Models: Enhancements and Comparative Analyses
Kamyar Zeinalipour, Neda Jamshidi, Monica Bianchini, Marco Maggini, Marco Gori
2024-08-14
Summary
This paper discusses how large language models (LLMs) can be used to design and generate high-quality protein sequences, which are essential for understanding biological processes.
What's the problem?
While LLMs have shown great success in handling text, they have not been widely used for generating protein sequences. Traditional methods often require large amounts of data, which can be hard to gather, especially for specific proteins.
What's the solution?
The authors introduce a new approach that uses several pre-trained LLMs to generate valid protein sequences. They trained these models on a smaller dataset of 42,000 human protein sequences, allowing them to create biologically realistic proteins even with limited data. The models were tested against established protein-focused models and showed comparable performance. They also evaluated the models using standard metrics to ensure their effectiveness.
Why it matters?
This research is significant because it demonstrates the potential of using AI and language models in the field of biology, particularly in protein design. By making their models publicly available, the authors encourage further research and collaboration, which could lead to advancements in medicine and biotechnology.
Abstract
Pre-trained LLMs have demonstrated substantial capabilities across a range of conventional natural language processing (NLP) tasks, such as summarization and entity recognition. In this paper, we explore the application of LLMs in the generation of high-quality protein sequences. Specifically, we adopt a suite of pre-trained LLMs, including Mistral-7B1, Llama-2-7B2, Llama-3-8B3, and gemma-7B4, to produce valid protein sequences. All of these models are publicly available.5 Unlike previous work in this field, our approach utilizes a relatively small dataset comprising 42,000 distinct human protein sequences. We retrain these models to process protein-related data, ensuring the generation of biologically feasible protein structures. Our findings demonstrate that even with limited data, the adapted models exhibit efficiency comparable to established protein-focused models such as ProGen varieties, ProtGPT2, and ProLLaMA, which were trained on millions of protein sequences. To validate and quantify the performance of our models, we conduct comparative analyses employing standard metrics such as pLDDT, RMSD, TM-score, and REU. Furthermore, we commit to making the trained versions of all four models publicly available, fostering greater transparency and collaboration in the field of computational biology.