Your Moat Just Became Promptable
Why employees are not inly leaking confidential data into AI. They are leaking the company’s learning curve, and by doing so turning private ideas, judgment, and intuition into generalized capability.
The new problem of corporations are not employees who are stealing, but employees who are improving. And here is why.
On X, a data protection employee shared how employees keep uploading confidential company information into ChatGPT.
As point of writing this, the post had >600k views, >11k likes – and I believe not because it is an IT security anomaly, but because it is already the new normal behavior of knowledge workers who are working under pressure — and who have, due to increased LLM use — already atrophied their ability to think independently.
Where it gets relevant is the fact that employees have access to Microsoft Copilot. Major enterprises sign dedicated AI tool contracts, like with Microsoft Copilot, that states in a contract that inputs/outputs are not used for training y default.
OpenAI, Meta, Anthropic all trained their AI models on petabytes of ILLEGALLY pirated content. So first, whether you should trust these contracts is one thing. The other thing is: Even if you do, employees will not only use these sanctioned tools, they will use whatever tool is the most capable for a specific task – without the organization knowing it – and they will put confidential information into these unvetted third-party AI systems which will collect prompts, uploaded files, and chat history to train the next generation models.
The REAL problem is something else: When it comes to confidentiality, companies still think in files (customer lists, legal documents, codebase, M&A decks, IP material).
However, in my opinion, the even riskier materials do not look ‘that’ confidential: the REAL confidential information you must worry about is what was previously trapped inside people’s brains, in meetings, on whiteboards. These idea information were never written down and thereby previously protected only by some kind of NDA.
However, now AI makes the unfinished thinking “promptable”. People start dictating their ideas into the LLM, and promptable becomes exportable.
And, the employee will use whatever model produces the best work. This might be ChatGPT on a private account, an unauthorized AI agent, PDF AI tools, meeting notetakers. Employees don’t do this maliciously, rather quite rationally, because their incentive is output.
And because the outputs look good, the efficiency is improving, employees are rewarded for doing so, without the organization being aware of it.
The problem will arrive with the next generation AI models. Suddenly, we have more capable AI systems that used confidential information as training data. – the ideas, creativity, intuition of your employees. Suddenly what defined your MOAT, becomes accessible to any competitor using the same model for a few hundred dollars.
Competitors will never need to see your exact confidential information. They will likely NEVER leak. The problem is on an entirely different scale: if enough private thinking that previously defined the MOATs of companies, is suddenly entering AI model training, the entire sector will get “averaged” upward, all while MOATs disappear.
The IT security question is what confidential are employees uploading.
The CEO question is: Which part of how we think must never become accessible as training material for the world?