Some insider knowledge: Lilli was, at least a year ago, internal only. VPN access, SSO, all the bells and whistles, required. Not sure when that changed.

McKinsey requires hiring an external pen-testing company to launch even to a small group of coworkers.

I can forgive this kind of mistake on the part of the Lilli devs. A lot of things have to fail for an "agentic" security company to even find a public endpoint, much less start exploiting it.

That being said, the mistakes in here are brutal. Seems like close to 0 authz. Based on very outdated knowledge, my guess is a Sr. Partner pulled some strings to get Lilli to be publicly available. By that time, much/most/all of the original Lilli team had "rolled off" (gone to client projects) as McKinsey HEAVILY punishes working on internal projects.

So Lilli likely was staffed by people who couldn't get staffed elsewhere, didn't know the code, and didn't care. Internal work, for better or worse, is basically a half day.

This is a failure of McKinsey's culture around technology.

> One of those unprotected endpoints wrote user search queries to the database. The values were safely parameterised, but the JSON keys — the field names — were concatenated directly into SQL.

I was expecting prompt injection, but in this case it was just good ol' fashioned SQL injection, possible only due to the naivety of the LLM which wrote McKinsey's AI platform.

I don’t love the title here. Maybe this is a “me” problem, but when I see “AI agent does X,” the idea that it might be one of those molt-y agents with obfuscated ownership pops into my head.

In this case, a group of pentesters used an AI agent to select McKinsey and then used the AI agent to do the pentesting.

While it is conventional to attribute actions to inanimate objects (car hits pedestrians), IMO we should be more explicit these days, now that unfortunately some folks attribute agency to these agentic systems.

> This was McKinsey & Company — a firm with world-class technology teams [...]

Not exactly the word on the street in my experience. Is McKinsey more respected for software than I thought? Otherwise I'm curious why TFA didn't just politely leave this bit out.

One interesting takeaway here is how quickly AI agents expose weaknesses in internal systems.

Many enterprise tools were designed assuming human interaction, where authentication flows, manual reviews, and internal processes add implicit safeguards.

But once you introduce autonomous agents that can systematically probe endpoints, missing authorization checks or misconfigured APIs become much easier to discover and exploit.

I suspect we’ll see a growing need for automated validation layers that continuously test internal AI tools for access control, data exposure, and unintended behaviors before they’re widely deployed.

I've got no idea who codewall is. Is there acknowledgment from McKinsey that they actually patched the issue referenced? I don't see any reference to "codewall ai" in any news article before yesterday and there's no names on the site.

https://www.google.com/search?q=codewall+ai

- "The agent mapped the attack surface and found the API documentation publicly exposed — over 200 endpoints, fully documented. Most required authentication. Twenty-two didn't."

Well, there you go.

What I don't see in this article that should be explicit:

If your data is in this database, it's gone. Other people have it. Your sensitive data that you handed over to their teams has vanished in a puff of smoke. You should probably ask if your data was part of the leak.

Fail to see how a state actor would not have come across this already.

Meanwhile, you're paying top dollars to a consulting firm that resolves back to an LLM to provide its services.

I can only remember a McKinsey team pushing Watson on us hard ages ago. Was a total train wreck.

They’ve long been all hype no substance on AI and looks like not much has changed.

They might be good at other things but would run for the hills if McKinsey folks want to talk AI.

> named after the first professional woman hired by the firm in 1945

Going out of their way to find a woman's name for an AI assistant and bragging about it is not as empowering as the creators probably thought in their heads.

Why was there a public endpoint?

Surely this should all have been behind the firewall and accessible only from a corporate device associated mac address?

Could the author please provide the prompt that was used to vibe write this blog post? The topic is interesting, but I would rather read the original prompt, as I am not sure which parts still match what the author wanted to say, vs flowerly formulations for captivating reading that the LLM produced.

Its so funny its a SQL injection because drum roll you can't santize llm inputs. Some problems are evergreen.

One interesting takeaway here is how quickly organizations are deploying AI tools internally without fully adapting their security models.

Traditional application security assumes fairly predictable inputs and workflows, but LLM-based systems introduce entirely new attack surfaces—prompt injection, data leakage, tool misuse, etc.

It feels like many enterprises are still treating these systems as just another SaaS product rather than something closer to an autonomous system that needs a different threat model...

I think the underlying point is valid. Agents are a potential tool to add to your arsenal in addition to "throw shit at the wall and see what sticks" tools like WebInspect, Appscan, Qualys, and Acunetix.

How long until a hallucinated data breach that spreads globally. There's a few inconsistencies and the typical low effort language AI has.

Cool but impossible to read with all the LLM-isms

That data is worth billions to frontier AI labs. I wonder if someone is already using it to train models

I wonder how these offensive AI agents are being built? I am guessing with off the shelf open LLMs, finetuned to remove safety training, with the agentic loop thrown in.

Does anyone know for sure?

At first glance, I thought this was about an AI agent named "Hacks McKinsey."

I’m waiting for the agentic models trained on virus and worm datasets to join the red team!

parameterized values but raw key concatenation is the kind of thing that looks safe in code review. easy to miss for humans, but an agent will just keep poking at every input until something breaks.

If the AI was poisoned to alter advice, then maybe McKinsey advice would actually be a net good.

And: AI agent writes blog post.

Music to my ears! Couldn't happen to a better company!

With all we've been learning from stuff like the Epstein emails, it would have been nice if someone had leaked this data:

> 46.5 million chat messages. From a workforce that uses this tool to discuss strategy, client engagements, financials, M&A activity, and internal research. Every conversation, stored in plaintext, accessible without authentication.

> 728,000 files. 192,000 PDFs. 93,000 Excel spreadsheets. 93,000 PowerPoint decks. 58,000 Word documents. The filenames alone were sensitive and a direct download URL for anyone who knew where to look.

I'm sure lots of very informative journalism could have been done about how corporate power actually works behind the scenes.

... in two hours:

> No credentials. No insider knowledge. And no human-in-the-loop. Just a domain name and a dream. ... Within 2 hours, the agent had full read and write access to the entire production database.

Having seen firsthand how insecure some enterprise systems are, I'm not exactly surprised. Decision makers at the top are focused first and foremost on corporate and personal exposure to liability, also known as CYA in corporate-speak. The nitty-gritty details of security are always left to people far down the corporate chain who are supposed to know what they're doing.

I wonder what is their security level and Observability method to oversee the effort.

this reads like it was written by an LLM

Not exactly clear from the link: were they doing red team work for McKinsey or is this just "we found a company we thought wouldn't get us arrested and ran an AI vuln detector over their stuff"?

You'd think that the world's "most prestigious consulting firm" would have already had someone doing this sort of work for them.

Are accounting and management consulting companies competent in cutting edge tech?

I have grown to despise this AI-generated writing style.

> Lilli's system prompts — the instructions that control how the AI behaves — were stored in the same database the agent had access to.

Being able to rewrite your own source. What's the worst that could happen?

McKinsey can eat shit