Anthropic Releases Open-Source Circuit-Tracing Tools to Advance AI Transparency

Details

• Anthropic has made available an open-source technique for building attribution graphs, which help map the decision pathways inside large language models.
• This initiative was developed in partnership with Anthropic Fellows and Decode Research, and features an interactive visualization interface called Neuronpedia.
• The tools are compatible with leading open-weight models, including Gemma-2-2b and Llama-3.2-1b.
• Researchers can now trace neural circuits, adjust model features, and run behavioral hypothesis tests more easily.
• A detailed walkthrough and hands-on demo notebook accompany the release, enabling in-depth analysis of multi-step model reasoning.

Impact

By opening up these interpretability resources, Anthropic bolsters AI safety research and invites greater collaboration throughout the AI research ecosystem. This move helps close the transparency gap in the industry and could inspire more open-source contributions from other model developers.