Major AI developers have largely adopted a closed-access model, delivering their most capable systems only through applications or APIs. This approach limits visibility into model architecture and behavior, a point highlighted by the contrast with several Chinese firms that distribute downloadable models, such as Xiaomi, which recently released live training logs. Academic groups like Stanford are also experimenting with fully open pre-training, exemplified by the Marin model, underscoring a growing split in deployment philosophy.
Nathan Lambert and Tom Zick, now leading Trillium Labs, first connected via Zoom while completing graduate work at UC Berkeley during the COVID-19 pandemic. Their collaboration emerged from frustration with the difficulty academic teams face when trying to reproduce proprietary experiments that require substantial compute resources. The pair argue that the current secrecy of frontier labs hampers community-wide risk assessment and slows the development of safety-focused techniques.
Trillium Labs positions itself as a nonprofit dedicated to open high-risk AI research. Its initial program will concentrate on post-training activities, such as fine-tuning large models after they have been released, while also probing recursive self-improvement, a method where AI systems generate their own research agendas. In addition, the lab plans to investigate how reinforcement learning influences model behavior, documenting training runs in detail so external scientists can replicate and critique the results.
The organization has secured an undisclosed amount of capital from investors including Schmidt Sciences and Halcyon Futures. Its founders state a longer-term goal of raising between $40 million and $100 million, with $30 million earmarked for compute resources and training over an 18-month horizon. These funds are intended to support the extensive experimentation required to explore scaling dynamics in post-training reinforcement learning.
Tim Fist, director of emerging technology policy at the Institute for Progress, told WIRED that he is “a massive fan of much more transparency than we currently have in R&D.” Zick added that “to understand something like how reinforcement learning scales in post-training, you need significant compute and a lot of careful experimentation.” Both statements underscore the lab’s belief that open documentation can reveal unexpected safety insights.
Advocates of closed access argue that limiting model exposure prevents malicious actors from exploiting capabilities such as automated vulnerability discovery and system intrusion. However, open research could enable the broader community to identify and patch these weaknesses before they are weaponized. By publishing training procedures and behavioral analyses, Trillium hopes to create a shared knowledge base that improves collective defenses against emerging AI-driven threats.
The founders view their nonprofit as a way to inject nuance into the ongoing policy debate over AI governance. By demonstrating that transparent, collaborative experimentation can coexist with safety goals, Trillium Labs aims to show that a shared understanding of frontier model risks may be more effective than concentrating power in a handful of private labs.