Many educators use AI detection software to judge whether student writing is AI-generated. While the desire to preserve academic integrity is commendable, detectors of AI-generated writing are not sufficiently accurate to justify their use in schools.

Students are being falsely accused. AI detection software commonly flags human-generated text as AI-written, so it will accuse many who have never used AI on their assignment.

AI writing detection software is easily fooled by students who have AI skill. The performance of the software goes down significantly if the student is skilled at AI prompting, or makes small changes to the AI output. 

AI writing detection software performs worse when the writer is not a native speaker. This means that specific student populations are more likely to be falsely accused.

AI writing detection software will not get markedly better. AI chatbots are designed to mimic human text, and they are increasingly good at doing so. Better AI means worse future AI writing detector performance. Newer AI detection tools also analyze the behavior of a student throughout an assignment, such as by analyzing typing patterns. However, that too is mimicable by AI, and many consider such systems overly intrusive.

Software to detect AI chatbot writing is commonly used at all education levels.

This is a travesty. Even students who do not consider misusing AI on their assignments are afraid. Trust between the student, teacher, and institution is a critical foundation for learning.

We call on school leaders to ban the use of AI detectors on student work.

Additional resources

“Can Using a Grammar Checker Set Off AI-Detection Software?: A college student says she was falsely accused of cheating, and her story has gone viral. Where is the line between acceptable help and cheating with AI?”, EdSurge, https://www.edsurge.com/news/2024-04-04-can-using-a-grammar-checker-set-off-ai-detection-software April 4, 2024.

Perkins, M., Roe, J., et al., “GenAI Detection Tools, Adversarial Techniques, and Implications for Inclusivity in Higher Education,” arXiv, arXiv:2403.19148, Mar. 28, 2024.

Pan, W. H., Chok, M., J., et al., “Assessing AI Detectors in Identifying AI-Generated Code: Implications for Education”, arXiv, https://arxiv.org/abs/2401.03676 Jan. 8, 2024.

Weber-Wulff, D., Anohina-Naumeca, A., Bjelobaba, S. et al., “Testing of detection tools for AI-generated text,” Int J Educ Integr 19(26), https://doi.org/10.1007/s40979-023-00146-z Dec. 25, 2023. - “our conclusion is that the systems we tested should not be used in academic settings.”

Carter, T., “Some universities are ditching AI detection software amid fears students could be falsely accused of cheating by using ChatGPT”, Business Insider, https://www.businessinsider.com/universities-ditch-ai-detectors-over-fears-students-falsely-accused-cheating-2023-9 Sep. 22. 2023.

Edwards, B., “OpenAI confirms that AI writing detectors don’t work: No detectors ‘reliably distinguish between AI-generated and human-generated content’,” ARS Technica, https://arstechnica.com/information-technology/2023/09/openai-admits-that-ai-writing-detectors-dont-work/ Sep. 8, 2023.

Liang, W., Yuksekgonul, M., et al., “GPT detectors are biased against non-native English writers”, arXiv, arXiv:2304.02819, Apr. 6, 2023.

Sadasivan, V. S., Kumar, A., et al., “Can AI-Generated Text be Reliably Detected?”, arXiv, https://arxiv.org/abs/2303.11156 Mar. 17, 2023.