Hundreds of people are reading ChatGPT conversations (and you probably agreed to it) After last week's math controversy, this is the second time in a week the question has come up: what actually happens to what you type into a chatbot? Do you paste your blood test results into ChatGPT to ask what the values mean, or rehearse the conversation with your GP in advance? Then a hired reviewer may read that conversation and give the answer a score. That's according to internal documents and real user conversations obtained by tech site 404 Media. The setting that makes this possible is on by default for every free, Plus and Pro account. What's going on? Internally, the program is called Project Lily. OpenAI, the company behind ChatGPT, hires hundreds of people through intermediaries who open a real conversation in a dashboard, summarize in one sentence what the user wanted, and rate four possible ChatGPT answers on a scale of 1 to 7. They watch for flattery, unnecessary emojis and what the instructions call AI talk. The goal is a chatbot that sounds less sycophantic, a problem that has dogged OpenAI for some time. The reviewers don't see a username. They do sometimes see a summary of the memory: what the user previously used ChatGPT for and, in some cases, roughly where that person lives. A filter is supposed to remove personal data before a conversation reaches a human. OpenAI's own description of that filter says it makes mistakes, especially with short texts. In some of the conversations 404 Media saw, the user explicitly asked ChatGPT to keep the contents to itself. When asked where OpenAI has told users that people can read their conversations, no answer came. After publication, the company pointed to a help page. The setting "Improve the model for everyone" was updated, with a better explanation of how to turn it off. That humans read along is still not mentioned.