No doubt you’ve heard of the Cloudflare security leak that’s becoming known as “Cloudbleed.”

There’s been a lot of outrage over it.

But many commentators are missing its most important lesson.

First, a summary of what has happened.

Few weeks ago, a Google researcher noticed that under certain circumstances, Cloudflare was serving corrupted content. Further investigation revealed that Cloudflare had a bug in an HTML parser. The bug would be triggered by certain HTML errors within a requested page.

The bug caused Cloudflare’s edge servers to inject additional content into the requested pages as they served them.

As explained in a blog post from Cloudflare CTO John Graham-Cumming, “our edge servers were running past the end of a buffer and returning memory that contained private information such as HTTP cookies, authentication tokens, HTTP POST bodies, and other sensitive data.”

Here’s a vital point.

The private information included data from requests to other Cloudflare users on that server.

As noted by Google’s Tavis Ormandy, “we observed encryption keys, cookies, passwords, chunks of POST data and even HTTPS requests for other major cloudflare-hosted sites from other users… I’m finding private messages from major dating sites, full messages from a well-known chat service, online password manager data, frames from adult video sites, hotel bookings. We’re talking full https requests, client IP addresses, full responses, cookies, passwords, keys, data, everything.”

Compromised data have also included API endpoints, OAuth tokens (not just OAuth1, but also OAuth2), PII (personally identifiable information), geolocation info, and other secrets.

The important point here is that the Cloudflare users who (unknowingly) were being compromised, weren’t even necessarily the ones triggering the bug. Their sites, and the private data of their customers, were compromised merely because they shared Cloudflare resources with the triggering users.

How long has this been happening? Cloudflare says this bug has existed since September 22 of last year. (For various reasons, it got much worse in mid-February this year.)

So far, over 3,400 domains have been identified as being potential triggers.

The potential victims are: every domain that has ever shared Cloudflare resources, at any time since last September 22nd, with any one of these 3,400 domains.

Cloudflare has now identified and fixed the bug responsible for all this. And by listening to company executives, it might seem that this incident wasn’t a big deal.

As Mr. Graham-Cumming told the BBC, “I am not changing any of my passwords. I think the probability that somebody saw something is so low it’s not something I am concerned about.”

But others strongly disagree with this assessment.

To them, this leak is disastrous.

Cloudflare has said that the bug would be triggered in one of every 3.3 million HTTP requests. At first, that doesn’t sound like much. But consider that already in 2015, Cloudflare was handling four million requests per second. (Presumably, the number is even higher now.)

And this bug has existed since last September.

Furthermore, it might be even worse than these numbers would indicate. It doesn’t necessarily require 3.3 million requests to trigger each leak.

After all, if a bad actor had noticed that a particular page also included some private data from other websites, he could just request that same page over and over, and trigger the bug each time.

Or, it would have been trivial for a hacker to sign up for his own Cloudflare account, and set up an error-ridden page. Then he could just request his own page over and over again, to harvest private data from other Cloudflare users.

But let’s say that no malicious actors were able to take advantage of this security hole. (Although we don’t know this to be true.)

That still wouldn’t mean this incident is over. Far from it.

Private customer data has been injected into a countless number of public web pages, for quite some time.

And while this was occurring, many of these pages were being cached across the Web.

Efforts are now underway at some of the search engines to purge their caches of all pages with compromised data. But it seems safe to assume that not all caches will be purged. Indeed, some will probably be mined instead. (Would we expect a Chinese or Russian web company to not try to get compromising data on Western firms?)

Nor are search engines the only organizations today with large caches. The Internet Archive obviously has one. Many ISPs maintain large caches too. Even many private companies cache large portions of the web, if they use appliances like Blue Coat.

For that matter, how many hackers rushed to scrape the major search engine caches, before they were fully cleared? (Even Google’s cache still had easy-to-find compromised data, widely available for many hours after the incident announcement.)

A full clean-up of this mess is impossible.

Lastly, there are other ramifications of Cloudflare’s leaks.

Perhaps Mr. Graham-Cumming thinks that compromises of his own personal data are “not something to be concerned about”, but many organizations are in regulated industries. They have strict legal obligations, both for maintaining security of their customer’s data, and for notifying customers of any breaches. The penalties for noncompliance can be severe.

No doubt, executives at these organizations strongly disagree that this months-long security breach is “not something to be concerned about.” Many are in crisis mode as a result of it.

So what can be learned from this whole fiasco? Many commentators are saying that this incident can be blamed on an over-centralization of the Web, or an over-reliance of cloud services, or something like that.

But these conclusions aren’t correct.

Here’s the real lesson from this.

From deep in the middle of Cloudflare’s blog post:

“Because Cloudflare operates a large, shared infrastructure, an HTTP request to a Cloudflare web site that was vulnerable to this problem could reveal information about an unrelated other Cloudflare site.”

Here’s a key point to this entire incident. Its victims were not at fault. Their websites didn’t trigger the parser bug.

They became victims merely by sharing infrastructure with other users.

In fact, although this is the worst example (so far) of shared-resource vulnerabilities, it’s not the first time that organizations have been victimized by these sorts of problems.

At Reblaze, our customers do not share resources.

Every single customer receives a unique private cloud: an entire dedicated stack (including DNS servers, load balancers, logs, database, etc.) for that customer’s use alone.

This eliminates all possible shared-infrastructure vulnerabilities, such as this latest Cloudflare incident. We’re the only cloud web security company that does this.

But it is the only correct approach—as this incident clearly shows.

Reblaze has many other advantages over competing web security solutions. To get a free demo, contact us.