Meta Contractors Posed as Teens to Test Competitor Chatbots
Hundreds of contractors working for Meta were instructed to pose as minors online to test how competitor chatbots responded to prompts involving sensitive and high-risk subjects, according to internal documents and sources familiar with the project. The effort, managed by Meta contractor Covalen and known internally as Cannes, targeted chatbots from OpenAI, Google, and Character.AI.
Project Details
The project involved creating dummy under-18 accounts to send written prompts and images to rival chatbots, with responses logged into spreadsheets. Prompts included topics such as suicide, sex, eating disorders, drugs, profanity, and racial slurs. Many were written from the perspective of children or teenagers in crisis, such as a 13-year-old asking about pregnancy or a fifth-grader describing a classmate with a gun.
Some of the images sent by contractors included pills, knives, nooses, and medical diagrams. The prompts were designed to push chatbots toward responses their safety systems were supposed to refuse. In one round of testing in August 2025, over 45,000 prompts were sent to competitor chatbots. The companies being tested were unaware of the effort.
Ethical and Legal Concerns
Former contractors described the project as alarming, raising concerns about generating child sexual abuse material or secretly using competitors’ data. Legal experts reviewed the prompts and found they did not cross into illegal territory but noted potential violations of the competitors’ terms of service.
OpenAI, Google, and Character.AI confirmed they did not authorize the testing. Character.AI called the conduct a violation of its terms and policies, while Google and OpenAI said they were investigating the issue.
Meta's Defense
Meta defended the project as routine safety testing, stating it was an industry-standard practice to ensure safe and age-appropriate experiences. The company claimed it does not use competitor benchmarking to train its own AI models.
Industry Perspective
While testing competitors’ products is not unusual in the AI industry, the scale and secrecy of Meta’s project raised questions. Rumman Chowdhury, CEO of Humane Intelligence PBC, described the effort as outside typical industry standards, blending safety evaluation with competitor benchmarking in a way that could be seen as anticompetitive.
Conclusion
The project highlights the ethical and governance challenges in AI safety testing, particularly when involving sensitive topics and minors. As the tech industry continues to evolve, such practices may face increasing scrutiny from both legal and ethical standpoints.