IBM Unveils Granite 4.0 Tiny Preview: Compact Open-Source AI Model for Efficient Deployment

Details

• IBM has released the preview version of Granite 4.0 Tiny, the latest model in its upcoming Granite AI family.
• The model is fully open-source and accessible on Hugging Face under the Apache 2.0 license.
• Granite 4.0 Tiny is designed to operate efficiently on consumer-grade GPUs costing under $350, significantly improving accessibility.
• Despite only being partially trained (2.5 trillion out of 15 trillion tokens planned), it achieves performance similar to the Granite 3.3 2B Instruct model while using 72% less memory.
• Once fully trained, IBM expects it to match the performance of its 8B Instruct variant.
• The model features a hybrid Mamba-2/Transformer Mixture-of-Experts (MoE) architecture, providing Mamba’s linear scaling advantage for longer contexts and Transformers’ precision.

Impact

The Granite 4.0 Tiny preview underscores IBM's push for efficient, accessible AI by providing high performance in a compact model. Its low hardware requirements could broaden advanced AI adoption across industries and developers. This move also positions IBM as a significant player in open-source, cost-effective AI, challenging competitors like Meta and Stability AI in the space.