Human Oversight

Hundreds of people are reading ChatGPT conversations (and you probably agreed to it) After last week's math controversy, this is the second time in a week the question has come up: what actually happens to what you type into a chatbot? Do you paste your blood test results into ChatGPT to ask what the values mean, or rehearse the conversation with your GP in advance? Then a hired reviewer may read that conversation and give the answer a score. That's according to internal documents and real user conversations obtained by tech site 404 Media. The setting that makes this possible is on by default for every free, Plus and Pro account. What's going on? Internally, the program is called Project Lily. OpenAI, the company behind ChatGPT, hires hundreds of people through intermediaries who open a real conversation in a dashboard, summarize in one sentence what the user wanted, and rate four possible ChatGPT answers on a scale of 1 to 7. They watch for flattery, unnecessary emojis and what the instructions call AI talk. The goal is a chatbot that sounds less sycophantic, a problem that has dogged OpenAI for some time. The reviewers don't see a username. They do sometimes see a summary of the memory: what the user previously used ChatGPT for and, in some cases, roughly where that person lives. A filter is supposed to remove personal data before a conversation reaches a human. OpenAI's own description of that filter says it makes mistakes, especially with short texts. In some of the conversations 404 Media saw, the user explicitly asked ChatGPT to keep the contents to itself. When asked where OpenAI has told users that people can read their conversations, no answer came. After publication, the company pointed to a help page. The setting "Improve the model for everyone" was updated, with a better explanation of how to turn it off. That humans read along is still not mentioned. …more

To Read or Not to Read the Code? https://every.to/source-code/to-read-or-not-to-read-the-code 1 big thing: Don't miss this shift — AI is ready to run your work in the dark. Don't let it turn the lights out in your head too. Shipping more has never been easier. Understanding what you're shipping has never been more at risk. Why it matters: B2B marketers building AI-assisted workflows face the same trap engineers do. The system gets smarter. You get faster. But the judgment that makes you valuable — knowing which of two reasonable-sounding plans will hurt you six months from now — quietly erodes if you let AI handle everything without staying in the work. Driving the news: Kieran Klaassen, GM of Every's AI email product Cora, found himself shipping faster than ever while feeling his own thinking get softer. Researchers Margaret Mitchell, Avijit Ghosh, and Samir Passi confirmed the pattern — extended AI agent use measurably erodes the vigilance, critical thinking, and domain skill that human oversight depends on. → The phenomenon dates back to 1983 — called the "irony of automation" — where the more capable the automation, the more human skill erodes → AI just makes it move faster Zoom in: Klaassen's solution: build a deliberate loop for his own learning alongside the AI. → His four practices: keep a list of gaps, ask for mechanics not just diffs, recover the reasoning behind the code, and let the model quiz you Yes, but: This takes discipline most people won't apply. The excuse that you don't have time is exactly what AI removes — but the desire to learn has to come from you. Be smart: Audit one AI-assisted workflow you run today. Can you explain why it works? Can you catch it when it fails? If not, you have a judgment gap, not a tool gap. The bottom line: The tools that gave you an edge are becoming easy to copy. The judgment to know where to point them is not. https://every.to/source-code/to-read-or-not-to-read-the-code #AIMarketing #B2BMarketing …more

Silicon Valley’s new frontier isn’t writing every line of code. It’s deciding what AI agents should do and how to evaluate them. From building startups and running accelerator programs, I’ve learned three practical rules for being agentic: 1) Define the objective, success metrics, and hard constraints up front. 2) Decompose work into discrete agent tasks with clear inputs and outputs. 3) Instrument feedback loops and human checkpoints to catch drift and control cost. If you’re rethinking roles or workflows for agents, send me a message. #AI https://www.wired.com/story/silicon-valley-agentic-individuals-future-of-work/ …more