CrankySec

You're smarter than this

The whole debacle around Anthropic going all "Oh my god, I cannot believe we are this good!" with their Mythos model has made me realize that a lot of cyberfolk lack a very fundamental skill: skepticism. Everywhere you go, someone is yapping about 271 vulnerabilities in Firefox, and how this is a portent of terrible things to come. These people should shut the fuck up.

I will not try to out-beaut this beaut here because I cannot, but let's take a look at this specific 271 vulnerabilities number. On April 21, 2026, Mozilla published a blog by Firefox CTO Bobby Holley, in which he states the following with regards to Mythos:

This week’s release of Firefox 150 includes fixes for 271 vulnerabilities identified during this initial evaluation.

It sounds really impressive, since he prefaces this with this:

We wrote previously about our collaboration with Anthropic to scan Firefox with Opus 4.6, which led to fixes for 22 security-sensitive bugs in Firefox 148.

So, Opus 4.6 discovered 22 "security-sensitive bugs in Firefox 148", and Mythos discovered 271 vulnerabilities that were fixed in Firefox 150. That's a very impressive jump. A jump so impressive that anyone who doesn't have a vested interest in the hype or doesn't take the PR at face value would raise an eyebrow. It's not that everything that comes out of the PR machine is bullshit, but, as a cybersecurity professional, and as a citizen, you need to be able to parse these things and figure out what's real and what's bullshit. Here's a completely unrelated book recommendation.

And, before we proceed, let's just state for the record that words matter. "Security-sensitive bugs" and "vulnerabilities" are very different things. For that matter, everyone involved in this thing is using the word "vulnerability" very loosely, too. And that's fine, as long as you define it, and stick to your definition. You can't lump those "271 vulnerabilities" together as if they are the same, because they are very much not. That's why we have a whole taxonomy of vulnerabilities, and why folks try to give them criticality values.

Let's take a look at the actual release notes for Firefox 150:

I am no mathematician, but I am counting 41. You're 230 vulnerabilities fixed short, Bobby. Unless, that is, you're counting things differently without telling us. Maybe you're counting the same vulnerability multiple times? Maybe one fix solved 200 of those? Who knows? It is kinda weird for you to say that Firefox 150 "includes fixes for 271 vulnerabilities", and a section of your own release notes for Firefox 150 literally titled "Security Vulnerabilities fixed in Firefox 150" lists 41 fixes, with three being credited to Mythos, and one of those being High impact. So, which one is it, Bobby? Is it 271 or 3? That's why words matter. And the folks who make money from the hype know that words matter, and they know that confusion works in their favor.

A quick sidebar here: Look at the release notes for Firefox 148, and the release notes for Firefox 150. If the metric is "vulnerabilities fixed", Opus 4.6 did a better job than Mythos.

On top of that, it is very good to remind yourself that Anthropic is not a cybersecurity company. That's not what they do. That's not their business. It is tangentially related at best. Mythos is not a fuzzer. It's not a vulnerability scanner. It's not a SAST/DAST tool. It's a fucking LLM. Cybersecurity practitioners should know that just because you're a big company, it doesn't automatically mean you know everything about everything. We see this all the time: people who are good at one thing thinking they are automatically good at other things. And that's probably truer in Silicon Valley than anywhere else in the world. That's how you get Stanford dropouts reinventing bodegas or charging hundreds of dollars for a juice pouch squeezer.

Don't listen to software developers/AI maximalists quoting some fella who identifies as someone who helps "technology teams tell their story, efficiently and effectively" when they write about application security and vulnerability management. Or do. I'm not your boss.

Cybersecurity practitioners must be able to sniff appeals to authority from a mile away. Just because the CTO of a browser (for real, though: how does that work? Being the CTO of a product?) said something, it doesn't mean it's true, or accurate, or that it's been said in good faith with no ulterior motives. It also doesn't mean it's not. That's why you don't dismiss it as bullshit right off the bat, either.

And cyberfolk are not immune to that either: remember a few years ago when a dude went on stage at DEF CON and said that broadcasting fake ADS-B data would cause airplanes to crash into each other, the media lapped it up, and a lot of people freaked out just for someone from the actual industry to come out and say "this is not how any of this works"? Pepperidge Farm remembers.

Being skeptical is very important for people in cybersecurity. The "big claims require big evidence" mantra needs to be internalized. You all talk a big game about social engineering, and fail to recognize it when it's aimed at you. Maybe this Mythos thing is orders of magnitude better at finding bugs in code. Maybe it isn't. The reality is that those who are making a big deal about it are being very coy about it, and unwilling to show the receipts. Will it be able to find bugs it introduced itself? How will you know? You see LLMs doing stupid shit all the time, so are you confident that any LLM will produce bug-free code? Catch its own bugs? If it can catch its own bugs, why didn't it write the code correctly in the first place? Bobby wrote that "The zero-days are numbered", but I think this statement will have to be revised when you let the same system with its own biases review and audit its own work.

P.S.: Did you know that we're on Mastodon and Bsky? Give us a follow because I love arguing with people online. We also have a store. And a Discord. But if you've soured on Discord, I understand.