Open-FinLLMs: Open Multimodal Large Language Models for Financial Applications
Qianqian Xie, Dong Li, Mengxi Xiao, Zihao Jiang, Ruoyu Xiang, Xiao Zhang, Zhengyu Chen, Yueru He, Weiguang Han, Yuzhe Yang, Shunian Chen, Yifei Zhang, Lihang Shen, Daniel Kim, Zhiwei Liu, Zheheng Luo, Yangyang Yu, Yupeng Cao, Zhiyang Deng, Zhiyuan Yao, Haohang Li, Duanyu Feng
2024-08-23
Summary
This paper introduces Open-FinLLMs, a series of large language models specifically designed for financial applications, which can handle complex data types like tables and time series.
What's the problem?
While large language models (LLMs) are powerful for generating text, they often lack the specific financial knowledge needed to perform well in finance-related tasks. Additionally, they struggle with multi-modal inputs, such as combining text with numerical data or charts.
What's the solution?
The authors developed Open-FinLLMs, starting with FinLLaMA, which was trained on a vast dataset of 52 billion tokens that includes text, tables, and time-series data. They then fine-tuned this model with 573,000 financial instructions to create FinLLaMA-instruct. Finally, they introduced FinLLaVA, which is capable of understanding and processing image-text data. Their evaluations showed that these models outperformed existing models in various financial tasks.
Why it matters?
This research is important because it enhances the ability of AI systems to work effectively in finance, a field where accurate data analysis is crucial. By improving how these models understand and generate financial information, it can lead to better decision-making tools for businesses and individuals.
Abstract
Large language models (LLMs) have advanced financial applications, yet they often lack sufficient financial knowledge and struggle with tasks involving multi-modal inputs like tables and time series data. To address these limitations, we introduce Open-FinLLMs, a series of Financial LLMs. We begin with FinLLaMA, pre-trained on a 52 billion token financial corpus, incorporating text, tables, and time-series data to embed comprehensive financial knowledge. FinLLaMA is then instruction fine-tuned with 573K financial instructions, resulting in FinLLaMA-instruct, which enhances task performance. Finally, we present FinLLaVA, a multimodal LLM trained with 1.43M image-text instructions to handle complex financial data types. Extensive evaluations demonstrate FinLLaMA's superior performance over LLaMA3-8B, LLaMA3.1-8B, and BloombergGPT in both zero-shot and few-shot settings across 19 and 4 datasets, respectively. FinLLaMA-instruct outperforms GPT-4 and other Financial LLMs on 15 datasets. FinLLaVA excels in understanding tables and charts across 4 multimodal tasks. Additionally, FinLLaMA achieves impressive Sharpe Ratios in trading simulations, highlighting its robust financial application capabilities. We will continually maintain and improve our models and benchmarks to support ongoing innovation in academia and industry.