Liquid AI, a startup co-founded by former researchers from MIT’s CSAIL, has unveiled its first multimodal AI models, marking a departure from the popular transformer architecture. The company aims to build foundation models beyond Generative Pre-trained Transformers (GPTs) and has already achieved superior performance compared to transformer-based models of similar size.
Known as Liquid Foundation Models (LFMs), these models come in three different sizes and variants, denoted by the number of parameters they possess. The LFMs have outperformed Meta’s Llama 3.1-8B and Microsoft’s Phi-3.5 3.8B on various benchmarks, including the Massive Multitask Language Understanding (MMLU) test, which covers STEM fields.
One of the key advantages of LFMs is their memory efficiency. Liquid AI’s LFM-3B model requires only 16 GB of memory, while Meta’s Llama-3.2-3B model needs over 48 GB. This makes LFMs suitable for deployment on edge devices and various use cases, including financial services, biotechnology, and consumer electronics.
Liquid AI’s approach to training post-transformer AI models involves computational units rooted in dynamical systems, signal processing, and numerical linear algebra. The resulting LFMs can model sequential data such as video, audio, text, time series, and signals. They offer real-time adjustments during inference without the computational overhead associated with traditional models.
The company has optimized the LFMs for deployment on hardware from NVIDIA, AMD, Apple, Qualcomm, and Cerebras. While still in the preview phase, Liquid AI invites early adopters and developers to test the models and provide feedback. The company plans to refine its offerings based on user feedback before a full launch event scheduled for October 23, 2024, at MIT’s Kresge Auditorium.
Liquid AI aims to position itself as a key player in the foundation model space by combining state-of-the-art performance with unprecedented memory efficiency. The company’s commitment to transparency and scientific progress is evident through its release of technical blog posts and engagement in red-teaming efforts.
