What happened
Researchers from MIT and Cornell University have shown that letting an AI text detector keep learning after release can sharply improve how well it spots machine-written text, including text altered to pass as human, MIT News reported. The approach, called test-time adaptation, lets a deployed model adjust itself using new, unlabeled text it receives instead of being retrained from scratch.
In tests on disguised AI writing, a commercial detector caught about 27 percent of samples, while the adapted model identified 93 percent. In experiments simulating gradual changes in how people write, a detector using the method reached 85 percent accuracy, compared with 73 percent for traditional approaches. The project relied on API credits from Pangram to access its AI text detectors.
Kevin Ren, a Cornell Tech graduate student, is lead author, with Cornell Tech assistant professor Nikhil Garg as senior author and MIT Sloan's Manish Raghavan as co-author. The work will be presented at the Conference on Neural Information Processing Systems. The team cautioned the method may weaken as LLMs sound more natural and verifiable human text grows scarce, and Raghavan said it is unclear whether the problem is solvable.
Context
The spread of LLMs that produce human-like text has spawned many commercial AI-detection products. Developers usually train them on internet text known to be human or machine-written, but they can fail when incoming text differs from that training data, a problem known as distribution shift. Ren described a cycle in which humanizer software emerges to defeat each detector release, pushing companies to retrain on the altered output.
The briefing
The week in tech, in one email.
The stories worth your time, every week. Unsubscribe anytime.
