AI Security Isn't a Feature Anymore — It's the Entire Product

Something unsettling happened this week in the world of artificial intelligence, and it wasn't just one incident — it was a pattern. Security researchers used Anthropic's Claude Mythos to breach macOS. ChatGPT's desktop app suffered a security incident. AI chatbots from Google, OpenAI, and Anthropic started spitting out people's real phone numbers. OpenAI had to rush out safety updates to help ChatGPT better recognize context in sensitive conversations.

Taken individually, each story is concerning. Taken together, they reveal a fundamental tension at the heart of AI development: we're building systems that are simultaneously our most sophisticated security tools and our most glaring security vulnerabilities.

The Anthropic macOS breach is particularly telling. Here we have an AI system so capable at understanding code and system architecture that it can identify previously unknown vulnerabilities and develop working exploits. This is genuinely useful for security research — finding bugs before bad actors do is the entire point of white-hat hacking. But the same capability that makes AI valuable for defense makes it terrifying for offense. We've handed attackers a force multiplier that never gets tired, never misses obvious patterns, and scales infinitely.

Meanwhile, the phone number leakage problem exposes a different kind of security failure — one rooted in training data and the fundamental architecture of large language models. These systems were trained on vast swaths of the internet, including directories, contact lists, and public records that real people never intended to become AI training fodder. Now that data is surfacing in unpredictable ways, and there's no easy fix. You can't just patch out a phone number when it's distributed across billions of model parameters.

Apple's reported consideration of allowing agentic AI on the App Store adds another layer to this security puzzle. The company has historically been the industry's privacy hawk, but it's now facing pressure to participate in the AI agent revolution while maintaining its security posture. How do you allow AI systems that can take actions on behalf of users while preventing abuse, data leakage, and malicious behavior? The UK tax authority's $175 million bet on AI fraud detection suggests one answer: fight fire with fire, using AI to police AI.

But this creates an arms race with no clear endgame. As AI systems become better at finding vulnerabilities, they also become better at exploiting them. As they become better at detecting fraud, they become better at committing it. We're building increasingly sophisticated defenses against threats that our own technology is making more potent.

The real issue isn't whether AI can be secured — it's that we're deploying these systems at scale before we understand their security implications. OpenAI's rushed safety updates for ChatGPT are a perfect example. These aren't features planned from the start; they're patches applied after problems emerge in production, with millions of users already exposed.

This week's news should be a wake-up call. AI security can't be an afterthought, a feature to be added later, or a problem to be solved with more AI. It needs to be fundamental to how these systems are designed, trained, and deployed. Because right now, we're in the absurd position of using AI to secure AI, while AI creates new security problems faster than we can address them.

The question isn't whether AI will have security issues — it's whether we'll take them seriously before the consequences become catastrophic.