Imagine a bouncer who is excellent at his job right up until you ask him to also stamp everyone’s hand on the way in. Now he’s watching the ink, checking the stamp alignment, making sure the pattern lands clean. And somewhere in that split attention, a guy he would have turned away at the door walks straight past him.
That’s roughly what new research from Lasso Security suggests is happening with SynthID-Text, the watermarking scheme Google built and that Anthropic plans to use in future Claude models. The finding: models using it become more likely to respond to harmful prompts they would otherwise refuse. Watermarking doesn’t just nudge word choice. It appears to move the safety needle too.
Why this one actually matters
I review AI tools for a living, which means I spend a lot of time watching vendors ship features that solve a problem nobody had while quietly creating three new ones. Watermarking is not that. Watermarking is a genuinely good idea. Provenance for machine-generated text is one of the few proposals in this space that addresses a real, measurable harm — you cannot tell what a model wrote, and that has consequences for elections, fraud, academic integrity, and evidence.
So the uncomfortable part isn’t that watermarking is bad. It’s that the safety feature and the safety behavior are apparently coupled in ways that weren’t obvious before someone went looking. That’s a different category of problem than “this product is mid.” This is two well-intentioned systems interfering with each other.
And the interference cuts both ways. The research also notes that bad actors can manipulate output to remove or distort the watermark, while the model stays just as capable of producing harmful content. So you get a provenance signal that motivated attackers can strip, attached to a model that may be slightly easier to talk into bad behavior because the signal is there in the first place. If you were designing a system to be annoying, you would struggle to do better.
What I’d actually want to know before shipping this
Here is where I’ll be honest about the limits of what we have. The public detail on this is thin. One research group, one watermarking scheme, a described effect. That is enough to take seriously and not enough to panic about. Researchers themselves land on the same conclusion: models need thorough testing when watermarking gets deployed. Not “we tested the model, then we added watermarking.” Tested again, after, as a combined system.
If I were evaluating a vendor rolling this out, the questions I’d want answered:
- Did you re-run your full refusal and jailbreak evaluation suite with watermarking enabled, or did you assume the results carried over?
- How much does the effect scale with watermark strength? Is there a dial, and where does it sit by default?
- What happens under adversarial prompting specifically, as opposed to a benign harmfulness benchmark?
- If the watermark can be stripped, what’s the actual threat model you think it defends against?
Notice that none of those are gotcha questions. They’re the ones a competent safety team would already be asking internally. The test of whether a lab is serious is whether they have answers ready or have to go find them.
The pattern I keep seeing
This is the third or fourth time I’ve watched the same shape play out in AI tooling. A feature gets added to address a legitimate concern. It ships as an additive layer, bolted onto a system whose behavior emerges from billions of interacting weights nobody fully maps. Everyone assumes the layer is orthogonal to everything else. Then someone measures, and it turns out the layer was never orthogonal at all.
Retrieval augmentation did this. Tool use did this. Fine-tuning for helpfulness famously did this to refusal behavior. The lesson isn’t “stop adding features.” It’s that in these systems, there is no such thing as a local change. Everything you touch is load-bearing for something you weren’t thinking about.
Which brings me to my actual take: watermarking should ship anyway. The provenance problem is real and getting worse, and waiting for a perfect implementation means shipping nothing. But it should ship with the safety evaluation redone from scratch, and the results published, and an honest accounting of the tradeoff. Call it the provenance tax. You get traceability, you pay something for it, and users deserve to know the price.
What would worry me is the alternative path — watermarking gets adopted broadly as a compliance checkbox, nobody re-tests, and we spend two years with a quiet degradation in refusal behavior across a large chunk of deployed models because everyone assumed a labeling feature couldn’t possibly affect what the model is willing to say.
Google and Anthropic both have safety teams that are better than average at this kind of thing. That’s not flattery, it’s why I expect them to publish the numbers. If they don’t, that silence will tell us plenty.
🕒 Published: