This is absurd.
The White House has decided not to publicly release its framework for evaluating advanced AI models — the very framework it just reviewed with OpenAI, Anthropic, Microsoft, and other major players. Three sources familiar with the discussions confirmed this to Axios, and the reaction from the AI safety community has been swift and sharp. “Baffling” is the word making the rounds, and honestly, it’s generous.
What We Know
Here’s the situation: the Trump administration has been pushing AI companies to submit their models to the US government for vetting. An executive order signed in June 2026 formalized this expectation, calling on companies to present their systems for evaluation before release. The goal, ostensibly, is to patch vulnerabilities and ensure national security interests are protected.
But now the framework that would guide those evaluations — the rulebook, the criteria, the standards by which these trillion-dollar systems get a thumbs up or thumbs down — is being kept internal. No public release. No open comment period. No external scrutiny.
This comes at a moment when Anthropic CEO Dario Amodei has been publicly warning that most people still don’t grasp how close we are to AI systems that outperform any human at any cognitive task. Whether you agree with his timeline or not, the point stands: the stakes of AI evaluation are enormous and growing fast.
Why Secrecy Defeats the Purpose
Let me be direct about why this matters for anyone building with or reviewing AI tools.
An evaluation framework only works if people trust it. And trust requires transparency. If the public — including independent researchers, civil society organizations, and competing nations — cannot see the criteria being used, there’s no mechanism for accountability. There’s no way to identify gaps, biases, or blind spots in the methodology. There’s no basis for global coordination on AI safety standards.
You’re essentially asking the world to trust that a closed-door meeting between the most powerful government on earth and the companies it’s supposed to regulate produced a fair and thorough set of benchmarks. That’s not how credible standards get built. That’s how regulatory capture happens.
And this isn’t happening in a vacuum. The Wall Street Journal reported earlier that Trump administration officials directed the Center for AI Standards and Innovation to pause public reports on its AI testing work. So the unit responsible for public-facing AI evaluation is being muzzled at the exact moment these systems are getting more capable and more widely deployed.
My Take as a Reviewer
I review AI tools for a living. I test agents, I break chatbots, I evaluate capabilities against claims. And I can tell you that evaluation methodology is everything. The questions you ask determine the answers you get. The benchmarks you choose shape what “safe” and “capable” even mean.
When those choices are made behind closed doors with only the companies being evaluated in the room, you don’t have safety testing. You have theater.
Think about what this means practically. If OpenAI or Anthropic releases a new model and says it passed the White House evaluation framework, what does that claim actually mean? Without public access to the criteria, independent researchers can’t verify anything. Journalists can’t fact-check anything. Competitors can’t identify whether the bar was set fairly or selectively.
It also creates a two-tier system of information. The companies in that room know exactly what they’re being measured against. Everyone else — smaller AI startups, open-source developers, international researchers — is operating blind. That’s not a safety regime. That’s an incumbent protection program.
What Should Happen Instead
Release the framework. It’s that simple.
If there are genuinely sensitive national security components — specific attack vectors that shouldn’t be public, for instance — redact those narrowly. But the overall evaluation methodology, the categories of risk being assessed, the performance thresholds, the testing protocols? Those should be open documents subject to public comment and academic review.
Every credible safety standard in every other industry works this way. Aviation safety standards are public. Food safety protocols are public. Nuclear safety criteria are public. The idea that AI evaluation standards need to be secret to be effective contradicts decades of evidence about how good regulation actually functions.
The White House had a chance to set a global benchmark for AI accountability. Instead, it chose opacity. And in the AI safety space right now, opacity isn’t caution — it’s negligence dressed up in a suit.
I’ll be watching to see if any of the companies involved push back publicly. But I’m not holding my breath.
🕒 Published: