Hugging Face launches Storage Buckets, S3-like mutable storage for AI workflows

March 10, 2026

Details

  • Hugging Face has shipped Storage Buckets, a new S3-compatible object storage service designed for high-throughput AI workloads that Git and Git LFS cannot efficiently handle.
  • The service supports fast writes, overwrites, and directory synchronization, addressing performance bottlenecks in checkpoint storage, processed data management, agent traces, and log handling.
  • Buckets are powered by Xet, the data versioning backend that Hugging Face acquired from XetHub in August 2024 to replace Git LFS as their primary data storage solution.
  • The offering emphasizes cost efficiency and speed compared to traditional Git-based approaches, which struggle with large file mutations and frequent updates typical in machine learning pipelines.
  • Users can manage buckets through the Hugging Face Hub CLI using S3-like commands, providing familiar interfaces for teams already working with object storage.

Impact

This release directly addresses a critical gap in the AI development stack. While Git excels at versioning code and small configuration files, it becomes prohibitively slow and expensive for the massive, mutable datasets and checkpoint files that dominate modern machine learning workflows. By offering native S3 compatibility through Buckets, Hugging Face positions itself to compete with cloud storage providers and standalone solutions like Wasabi, Backblaze B2, and MinIO—platforms that already serve AI/ML workloads but lack the integrated model and dataset ecosystem that Hugging Face provides. The move leverages Xet's architecture, which uses Merkle trees and content-defined chunking for efficient storage deduplication, potentially delivering cost advantages over raw S3 alternatives. This plays into Hugging Face's broader strategy to consolidate the ML development experience by offering models, datasets, compute, and now storage within a single platform. For teams currently juggling multiple services—Hugging Face Hub for models, AWS S3 or similar for checkpoints, and Git for code—Buckets simplifies infrastructure and reduces vendor lock-in risk. Over the next 12-24 months, this could shift where teams store training artifacts and accelerate adoption of Hugging Face's ecosystem as a comprehensive MLOps platform.

Read more