CrankySec

Let's Delve In

A lot of peeps out there shocked, and I mean SHOCKED!, about this whole Delve thing. If you don't know what I'm talking about, go have a read. If you're lazy, here's a summary: looks like YC graduate compliance company Delve has allegedly defrauded clients. Allegedly. No one's been charged with anything, and no one's been found guilty of anything. So... Allegedly. Their whole business model revolves (revolved? Who knows!) around a platform that helps companies achieve compliance with SOC 2, ISO/IEC 27001, HIPAA, and GDPR through some "AI-powered" platform. Someone from this company mistakenly posted a link to a bunch of reports, and the folks behind the LONG article linked above did some digging. It doesn't look great. It's not THAT surprising either.

You see, I was a PCI-DSS QSA for a long time. And I've been certified as a Lead Auditor for ISO/IEC 27001 for the last two editions of the standard. And I've worked with third-party risk management for years. This is not me tooting my own horn (plug to our business later), just a way to show you that a) I've written a LOT of PCI-DSS reports on compliance, b) I've seen first-hand how ridiculous the ISO/IEC 27001 certification is, and c) I've read a LOT of SOC 2 reports.

Let's start with the PCI-DSS years. First of all, let me tell you that, from all those other compliance programs, this one is the one that takes things more seriously than the others. PCI-DSS Reports on Compliance have to go through a quality-assurance cycle before they are released to the client for their signature. That means that little old me wouldn't be able to just write whatever and send it to the client. At the very least, I'd have to collude with a QA person who was sitting thousands of miles away from me. On top of that, the organization that runs that thing, the PCI SSC, does audit the QSA firms, and does put them on a remediation status when they fuck up. In terms of marketing, having your company's name in red letters on the PCI SSC website is not a good look at all. Fuck up consistently, or fuck up too hard, and they will simply yank your ability to conduct PCI DSS assessments.

Things are not that simple, though. As we've mentioned before, the whole thing is weird because you can just go to a more, let's say, friendly QSA company if the current one is poking around too much, or asking too many questions. Every participant in this ecosystem is incentivized to give you a report on compliance that is cheap, fast, and somewhat defensible. And that's extremely hard to do. For one, the report itself is insanely long. The report template alone is 354 pages long. And they expect you to write a detailed account of how you tested every single requirement, what you saw, and why you think what you saw during the testing is kosher. It's a mega pain in the ass.

It's not unusual for the reporting to take way longer than the assessment itself. I fucking hated writing that thing. And you still had to write the Attestation of Compliance, which is yet another document. You simply cannot do cheap, fast, and somewhat defensible while playing by the rules and being a profitable company. My then-employer tried to automate the writing of the report at one point. It was great: it cut the time of writing that thing by some 80%. That was just automating the writing of the report: everything else still needed to be done: onsite visits, interviews, evidence collection, the works. Then, the PCI SSC took a look at a sample of our reports and realized, correctly, that they all read the same. Because they pretty much did. What did they do? They told my then-employer to knock it off. Can't do that. That's how you get your name in red letters. The reality remains, though: to be a QSA the way they want QSAs to be, you need to know your shit, be thorough, and write fast. Given how much I was paid, that's asking too much. People will cut corners. It's simply inevitable. But that's a risky proposition because the SSC is not the AICPA or whoever the fuck governs ISO/IEC 27001 audits.

When I took my second ISO/IEC (we use full names here) 27001 lead auditor class, I was told that the test would be open book, and, if you failed, you just took it again up to three times. Like, right then and there. Obviously no one went home without the title of ISO/IEC 27001:2022 Lead Auditor. Literally the following Monday, I got a call from someone at BSI asking me if I would like to teach the class. It is a known fact that they will just take anyone with a pulse and a piece of paper that proves at best that you are not functionally illiterate. Don't take this as indisputable truth, but I have never heard of anyone or any company that got punished for being shit at ISO/IEC 27001. And, trust me: I've been audited against ISO/IEC 27001. They were ALL shit. They still had to go onsite, do interviews, collect evidence, take notes, and write a report. It wasn't like what's being alleged in that article.

The cherry on top of this trash cake is SOC 2. That one has got to be a joke. WHY is the AICPA, ostensively named the Association of International Certified Professional ACCOUNTANTS, in charge of this shit at all? You're accountants, people. What are you doing writing requirements for information security?? "Oh, but the CPA is assisted by a subject matter expert!" The fuck they are. Thanks for the tips and tricks about conducting audits, but we'll take it from here. When I did third-party risk assessments, I came across a LOT of SOC 2 reports. And they were ALL terrible. You don't learn anything about the security posture of your vendor by reading that garbage. And let's not even mention the many times a vendor just sends you a SOC 2 report. Like, any report. The first one they see. That one that covers something that has got absolutely nothing to do with the service being provided. It's like asking for the Hyundai car company's SOC 2, and getting the one for the Hyundai Heavy Industries instead. It's ridiculous. And I've never heard of anyone getting even a slap to the wrist.

The whole ecosystem is rotten to the core. You cannot expect someone to perform these audits to the standards they should with the constraints they have. They should be well compensated, but they aren't. They should undergo good training, but they don't. They should have time to do a good job, but they don't. What Delve has done (allegedly) was just taking this to the logical conclusion: If no one really cares about this whole thing, let's cut the extra steps and go straight to providing you with the document you paid for. If someone doesn't paste the wrong link at the wrong time at the wrong place, this would just be one of those open secrets everyone knows. Because, trust me: we all know those things are worthless.

I really do hope that things like OSCAL can make this process less pathetic, because the current situation is just silly.

I'll now plug my services: If you care about those things and need help, hit me up at sc at crankysec dot com, and we can talk about it.