Summary
AnyGPT is a unified multimodal large language model from Fudan University and Shanghai AI Laboratory that can understand and generate text, speech, images and music within a single framework.
AnyGPT is a unified multimodal large language model from Fudan University and Shanghai AI Laboratory that can understand and generate text, speech, images and music within a single framework.
It converts every modality into discrete tokens — text, images, audio and music — so one autoregressive transformer can process arbitrary combinations of inputs and outputs without changing the standard LLM architecture. It was trained on AnyInstruct-108k, a dataset of 108,000 multi-turn conversations interleaving modalities, and achieves zero-shot results comparable to specialised models on tasks such as image captioning and speech recognition.
AnyGPT is primarily a research model aimed at developers and researchers exploring multimodal AI, with a public demo and sample code rather than a consumer product. Its any-to-any design suits digital assistants, content creation and interactive media experiments.
Get more likes & reach the top of search results by adding this button on your site!


Get top updates in AI to your inbox every weekend. It's free!