On October 9, Pawel Huryn posted a test of five AI “humanizers” on work writing. He scored how human each output read, and he also counted how many times each one changed what the text actually said.

That second number should decide whether one of these tools ever touches a customer email.

His examples are the kind that end up in a dispute file. A plain “rewrite this to sound human” prompt told a customer their £45 refund had already gone through. The original said “we’re refunding.” Emulate-1, which is sold as undetectable, dropped the deadline for canceling a subscription before it auto-renews. And blader/humanizer, the most popular skill in the category, apologized to a customer for a deposit deduction the company believed was fair.

Read that last one as a lawyer would. An apology in a dispute can look like an admission.

What he measured

He used twelve new AI-written work texts from sales, marketing, support, product and internal notes. Every tool ran on the same model. Four AI judges from four different labs saw what the writer meant and picked, blind, the version they would send. A fact checker listed every dropped, changed or added item. The judges were Grok, DeepSeek, Qwen and OpenAI Codex, and each saw the candidates in two random orders, which leaves less room for position bias to decide the pick.

Tool	Judges would send it	Meaning changes
work-humanizer	48%	18
blader/humanizer	20%	33
”Rewrite this to sound human, without changing the meaning”	4%	27
”Rewrite this to sound human”	1%	42
Emulate-1	0%	120

Huryn built work-humanizer and designed the test. He says so in the post, and the repository lists the caveats, starting with small samples, AI judges instead of people and no contracts or legal documents in the set. I’d treat the 48% as his claim. The meaning-change counts are checkable by hand, which is why I trust them more.

The detector rewards the worst offender

On Pangram, a commercial AI-text detector, work-humanizer, blader/humanizer and the one-line prompt all stayed at 2 to 4% human. Pangram caught every one.

Emulate-1 reached 87% human. It was also the tool that changed the meaning 120 times.

So if a team’s goal is “doesn’t read as AI”, the winning tool is the one rewriting the most commitments. I understand the pressure, especially since LinkedIn started letting people flag posts as AI slop. Still. A customer whose promised refund never shows up does not care how human the email sounded.

There’s a smaller result in the repo worth reading too. When work-humanizer was given the writer’s own notes as input, all four outputs scored 100% human on Pangram. Four texts. Not proof. It does match what Andrej Karpathy got at from another angle when he started asking his AI to write in the controlled English of aircraft maintenance manuals. The fix lives in what you hand the model before it writes.

The test I’d run before buying

If someone is about to roll out a rewrite tool to support, sales or HR, here’s an afternoon that’s worth more than any vendor demo.

Pull 15 real emails your team sent last month. In each one, highlight what commits the company, like amounts, dates, conditions, apologies, “we’ll follow up by Friday.” Run them through the tool. Then compare the highlighted list in the original against the rewrite, line by line, and count.

Use support and billing emails for this. Marketing copy can drift a little and nobody gets hurt. A support reply that drifts becomes a promise someone has to keep.

How many changes per email? Did any touch money? Did any add an apology? Did any drop a date?

A yes on either of the last three keeps the tool off customer email. Internal notes, fine.

Huryn’s repo had 104 stars the day I opened it. blader/humanizer has more than 56,000. Popularity tells you how many teams are already running one of these. It says nothing about what they’re promising customers by accident.

If you’d rather not run the count alone, send me a note and we’ll set it up on your team’s own emails.