Is This AI? Longitudinal Analysis of Strategies Used for AI Detection on Two Subreddits

Christina Yeung1, Galen Weld1, Jaron Mink2, & Franziska Roesner1

1Paul G. Allen School of Computer Science & Engineering, University of Washington

2School of Computing and Augmented Intelligence, Arizona State University

Last updated: August 10, 2026

Paper Dataset

Abstract

As AI-generated content (e.g., "slop") becomes more prevalent online, people are developing strategies to attempt to identify it (or, conversely, to gain confidence that something is not AI-generated). What strategies are people using, and how are they changing over time as generative AI models themselves change? In this work, we catalog and analyze 2 years and 8 months of the AI detection strategies discussed by users of two popular Reddit communities (r/isthisAI and r/RealOrAI) that use the wisdom of crowds to identify AI-generated media. Through a mixed-method analysis of 13,098 posts and 222,060 comments within these communities, we catalog and analyze the prevalence of 12 AI-detection strategies, including examining fine-grained physical details, recognizing trends in AI-created content, and the assumptions people make about what models are capable of producing. Furthermore, we find that these strategies and mental models shift over time in accordance with changing AI capabilities and in response to online social trends. By systematically cataloging users' AI detection strategies, we lay the groundwork for user-facing guidance and future research.

Dataset

Our labeled dataset is available in CSV format here: [Click to download CSV file]

Data Schema:

author = the name of the Reddit user who wrote the comment
body = the content of the comment (one comment can appear multiple times, depending on how many times it’s fallen into a label)
parent_id = the post that the comment is associated with
subreddit = which subreddit (RealOrAI, or isthisAI) the comment was from<
id = unique string that Reddit assigns comments
datetime_pst = the date, and time of the comment (in PST format)
labels = the substrategy that Claude classified the comment
date = the date (in PST format)
period = the time period into which we categorized the comment

Citation

@inproceedings{yeung2026isthisai, title={Is This AI? Longitudinal Analysis of Strategies Used for AI Detection on Two Subreddits}, author={Yeung, Christina and Weld, Galen and Mink, Jaron and Roesner, Franziska}, booktitle={AAAI/ACM Conference on AI, Ethics, and Society (AIES)}, year={2026} }

Acknowledgments

This work was supported in part by the U.S. National Science Foundation under Award #211423 and by gifts from Microsoft, including a Microsoft Grant for Customer Experience Innovation. The website was designed in part by Claude Code.