The most expensive security failures were boring

Ask people to picture a major breach and they imagine a genius in a dark room, writing an exploit nobody has ever seen.

Read the actual reports and you find something less cinematic: an expired certificate, a VPN account nobody remembered, a person tapping “approve” on their phone. The damage was enormous. The causes would barely fill a ticket.

That should worry you and encourage you at the same time. Here are six cases, and what each one has to say about your own systems.

Equifax: the patch, and the certificate nobody renewed

In March 2017 a serious flaw in Apache Struts was made public. According to the US Government Accountability Office, unidentified individuals were scanning Equifax’s systems for it just two days later, and they found a vulnerable server running the company’s online dispute portal.

Equifax had scanned its own network a week after the announcement. The scan missed that server.

The data theft began in May. The attackers found a database holding unencrypted usernames and passwords, and used them to reach 48 more databases beyond the three behind the portal. They ran about 9,000 queries, and the attack lasted about 76 days before anyone noticed.

Why so long? A device meant to inspect encrypted traffic had been failing silently, because its digital certificate had expired about 10 months before the breach. When an administrator replaced the certificate, inspection resumed and the signs of intrusion appeared.

At least 145.5 million people in the US were affected. The settlement with US regulators was at least $575 million, and potentially up to $700 million.

The question for you: what would tell you if one of your security tools stopped working today?

Colonial Pipeline: one account nobody was using

In 2021 a ransomware attack led to the shutdown of one of the largest fuel pipelines in the United States. Mandiant, which investigated, said the way in was a VPN account that was no longer in use. Its password had turned up in a batch of leaked passwords.

Colonial’s CEO told the US Senate that this legacy VPN “did only have a single factor authentication”. He also said the password was not a weak one. Mandiant’s testimony described it as relatively complex, and said an employee had likely reused it somewhere else that was breached.

So a strong password did not help. The company paid a ransom of 75 bitcoin, worth about $4.4 million at the time.

The question for you: who still has access to your systems and no longer needs it? Can you produce that list this afternoon?

Capital One: a firewall with too much power

In 2019 an intruder obtained credit card application data covering about 100 million people in the US and 6 million in Canada. The route had two ordinary mistakes in it.

First, an open-source web application firewall was misconfigured, so it could be tricked into sending requests on the attacker’s behalf. That is server-side request forgery, or SSRF. The intruder aimed those requests at the cloud provider’s metadata service, which hands temporary credentials to servers.

Second, those credentials were far too powerful. As Brian Krebs reported, the firewall was allowed to list and read the files in any of the company’s storage buckets. A firewall has no reason to read credit card applications.

Fix either mistake and this attack looks very different. That is the argument for least privilege: you are planning for the day something else goes wrong.

The question for you: if your most exposed server were taken over, what could its credentials reach?

Uber: one tap on a phone

A second login factor would likely have stopped the Colonial intruder. It did not stop the one at Uber in 2022.

Uber’s own account says the attacker likely bought a contractor’s corporate password on the dark web, after malware infected the contractor’s personal device. The attacker then tried to log in again and again. Each attempt sent an approval request to the contractor’s phone, and each one blocked the attacker, until the contractor accepted one.

From there the attacker reached other employee accounts and gained elevated access to tools including G-Suite and Slack.

Push approval depends on a tired person making the right decision every time. Phishing-resistant methods such as hardware security keys and passkeys remove that decision, because there is nothing to approve by accident.

The question for you: which of your important logins can still be approved with a single tap?

Heartbleed: one missing check

OpenSSL’s advisory for this 2014 bug opens with the words “A missing bounds check”. A client could tell a server how much data to echo back, and the server believed it without comparing that number to what it had actually received.

Each request could reveal up to 64 KB of the server’s memory. Attackers could simply ask again. That memory could hold passwords and private keys.

The bug was introduced in December 2011 and made public in April 2014, sitting in widely used, openly readable code for more than two years.

The question for you: which code do you trust only because everybody else uses it?

xz: the attack that half a second stopped

Here is one that went the other way.

In March 2024 a Microsoft engineer named Andres Freund was doing unrelated work on Debian’s development version when he noticed that SSH logins were using an odd amount of CPU. His measurements were modest: a failed login that took 0.299 seconds before an update took 0.807 seconds after it.

He followed the half second down to liblzma and found a backdoor in the release packages of xz Utils, a compression library. It was given the highest possible severity score. A contributor known as “Jia Tan” had spent roughly two years earning the project’s trust and gaining the right to publish releases.

It reached testing distributions and was caught before the stable releases most servers run.

The most patient attack on this list was stopped by someone who found half a second strange and refused to let it go.

The question for you: when something on your systems behaves slightly oddly, does anyone have the time to find out why?

What the six have in common

Very little of this called for new technology. Most of the failures sat in ordinary gaps, so these are worth checking first:

Know what you run. A scan only protects the systems it can actually see.
Monitor your monitoring. A silent security tool looks exactly like a quiet day.
Remove old access. Unused accounts have no owner to notice when they are misused.
Limit what each credential can reach. Assume one will be stolen, and decide now how far it gets.
Treat small oddities as leads. Attackers try to look like normal traffic, so slightly abnormal is often all you get.

None of that needs a genius. It needs someone to go and look, which is why it so often doesn’t get done.