CrankySec

Smash the glass wing

You know shit's getting out of control when your friends and family start asking you about this "new AI hacking thing" they saw on Instagram. I know cyberfolk tend to blow things out of proportion, but, when things like this spill over onto main street, you just know you're in for a wild ride that should be used as a textbook example of the Gell-Mann Amnesia Effect.

Anthropic, the AI company equivalent of DJ Khaled in that it keeps suffering from success, gathered a bunch of people who would in no way, shape or form benefit from this attention, gave this group a silly name as people who do vulnerability research are wont to do, and called in the press. The press, of course, started lapping it up, and went on to write pieces about how hard it is for Anthropic to be so good looking, so rich, and have such a big dick that they can't even leave the house. It's a problem. And that's from a company ran by a guy who wrote a 19,000 word essay about the thing his company sells being a threat to mankind, and took the obvious next step of selling it anyway because, fuck it. Yolo and all that.

It obviously works, because here we are talking about it. Setting aside the irony of inviting to the table a bunch of companies who are very much part of the problem, here's the scoop: Anthropic's got a new model that got too good at finding vulnerabilities in software, and they jumped at the opportunity of running their "Oh, noes! We're too good at this. The horror!" playbook. I got sent an Instagram reel by some moron who was telling everyone to freak the fuck out, and that your nonna is going to die of hackers, and your car will try to eat you, and your house appliances will become deadly traps 24/7, and that we should all run for the hills. To which I say: Congrats on getting that engagement.

The moron was the person who posted that reel, not the lovely person who sent it to me, just so we're clear.

So, what's the real reality? Who the fuck knows, man. I'm not invited to that particular industry conclave. I don't what's going down at that encounter of the minds. But, what I do know is something that cyberfolk knew since before the thing was even called cybersecurity: PoC || GTFO. If you don't know what that means, you must lead a good life. It means, in general terms, that finding a vulnerability in software is just one part of the whole. If you cannot produce evidence that the vulnerability is exploitable, your vulnerability is just some academic curiosity. If you do produce evidence of exploitability, but it only happens if a thousand other things happen first, or it can only be triggered under strict lab conditions, your vulnerability is a heads up.

Go take a look at the technical report. The examples they have selected to showcase the capabilities of this model are: An OpenBSD denial of service, an FFMpeg crash, and a Linux Kernel privilege escalation issue that was portrayed thusly:

-The model autonomously found and chained together several vulnerabilities in the Linux kernel—the software that runs most of the world’s servers—to allow an attacker to escalate from ordinary user access to complete control of the machine.

While their own red team described it like so:

Mythos Preview identified a number of Linux kernel vulnerabilities that allow an adversary to write out-of-bounds (e.g., through a buffer overflow, use-after-free, or double-free vulnerability.) Many of these were remotely-triggerable. However, even after several thousand scans over the repository, because of the Linux kernel’s defense in depth measures Mythos Preview was unable to successfully exploit any of these.

Just so you don't think I'm picking the proverbial cherry, here's the paragraph right after that one:

Where Mythos Preview did succeed was in writing several local privilege escalation exploits. The Linux security model, as is done in essentially all operating systems, prevents local unprivileged users from writing to the kernel—this is what, for example, prevents User A on the computer from being able to access files or data stored by User B.

Which, ok. Local privilege escalations require you to be, you know, local. A cool find, but hardly the end of the world. And that's because, by and large, defense in depth measures work. They picked these three things as their marquee examples, and they are all meh at best. They follow up by saying that they found "several thousand more", which makes me wonder about the actual exposure here.

Here's the thing: finding a vulnerability is easy peasy. Exploiting it is a whole other ballgame. Making a practical exploit is a whole other sport. Crying "OH MAH LAWD! WHAT HAVE WE UNLEASHED UPON THE WORLD?!" is PR. Unless, of course, you back it up with the PoC. Otherwise kindly GTFO.

What I am not saying is that these findings don't have inherent value, because they do. Every vulnerability you find, exploitable or not, adds to the collective body of knowledge, and that makes software more reliable down the line. Every security researcher knows this. The same way that every security researcher and/or malicious actor know that 0-days have a very limited lifespan, and that you use them sparingly. This is not the fucking 90s where you had shit like WinNuke. It may not look like it, but things have changed.

On the other hand... Let the C-Suite panic a bit and go on a hiring spree. We know how to deal with these issues.