Independent AI Research,
Done in the Open

We build open-source AI infrastructure, pioneer interpretability and open-weight safety, and help researchers without frontier-lab resources do frontier work.

190+ Publications
62,000+ Citations 154M+ Model Downloads
Join the community Explore our research

Focus Areas

DataOpenly licensed The PileThe Common Pile
CodeOpen source GPT-NeoXlm-evaluation-harness
ModelsPublic, with checkpoints PythiaGPT-NeoX-20BComma

Open Models and Tools

Open research on large models requires open training data, open training and evaluation code, and models whose full development is public. We have built all three since 2020.

Our datasets include the Pile and the Common Pile. Our training framework is GPT-NeoX and our evaluation framework is the LM Evaluation Harness. Model suites such as Pythia are released with their training data, code, and intermediate checkpoints. All of it is openly licensed and in use across the research community.

Datasets Code Models
A training loss curve with saved checkpoints along it, one magnified to show internal structure forming

Interpretability Over Time

Most interpretability research studies a finished model. We study how model internals develop over the course of training, and we have argued that a science of AI must study training dynamics rather than only the final artifact.

The Pythia suite was built for this purpose. It provides checkpoints throughout training together with the exact data each checkpoint had seen, and it is now widely used in research on learning dynamics, memorization, and how capabilities form.

Pythia Don't Just Fix It in Post
Deep Ignorance Figure 1: filtered models match the baseline on general capability while biothreat proxy capability stays lower through 300M tokens of adversarial fine-tuning

Open-Weight Safety

Once model weights are public, safeguards cannot rely on API monitoring or on post-training methods that fine-tuning can remove. We work on safety methods that remain effective when anyone can inspect and modify the model, which in practice means intervening in the training data and the training process.

Deep Ignorance is one such method. It filters hazardous material out of pretraining data so that the model never learns it, and the resulting safeguard is substantially more resistant to adversarial fine-tuning than post-training alternatives. With FAR.AI we co-organize the Open-Weight Pretraining Safety Accelerator, which gives researchers access to the pretraining runs needed to test interventions of this kind at scale.

Deep Ignorance Development Cheatsheet
Schematic of benchmark saturation: older benchmarks flattening against a score ceiling while newer ones still separate models

Evaluation

We maintain the Language Model Evaluation Harness, which is the framework behind the Open LLM Leaderboard and the standard tool for evaluating open models. Maintaining it has shown us how much reported numbers depend on details of prompting, scoring, and implementation.

We also study evaluation itself. This includes when benchmarks saturate and stop distinguishing between models, how the age of a benchmark affects comparisons across models, and how results should be reported so that they can be compared across groups. Through the EvalEval Coalition, which we co-founded with Hugging Face and the University of Edinburgh, we work on shared evaluation infrastructure with other institutions.

LM Evaluation Harness Benchmark Saturation EvalEval Coalition

Recent Papers

Paper

Bergson: An Open Source Library for Data Attribution

EMNLP System Demonstrations · October 24, 2026

 Paper

Don't Just "Fix it in Post'': A Science of AI Must Study Learning Dynamics

ICML · July 10, 2026

 Paper

Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMs

ICLR · April 23, 2026

 Paper

The Common Pile v0.1: An 8TB Dataset of Public Domain and Openly Licensed Text

NeurIPS Datasets and Benchmarks · December 2, 2025