“I worry this is the big botnet flexing,” CEO said. But outage was self-inflicted.
So they have no idea what caused this file to double in size and propagate itself across the network. And their fix was just to replace it with an older version.
There doesn’t appear to be anything preventing this problem from happening again.
Prince explained that the problem “was triggered by a change to one of our database systems’ permissions which caused the database to output multiple entries into a ‘feature file’ used by our Bot Management system.
They know what caused the file size increase, exactly what system it broke and how.
Cloudflare will work on “hardening ingestion of Cloudflare-generated configuration files in the same way we would for user-generated input; enabling more global kill switches for features; eliminating the ability for core dumps or other error reports to overwhelm system resources; [and] reviewing failure modes for error conditions across all core proxy modules,” according to Prince.
And they’re planning to prevent future occurrences.
Seriously did you not read the article?
Yeah, what or who made the change to the database? Why did they do it? What are they “hardening ingestion of Cloudflare-generated configuration files” against? Do they know? Are they trying not to be specific on purpose or are they being incompetent?
Seriously just read the article, Cloudflare have made a surprising amount of detail public.
If you don’t understand what’s in the article that’s fine… just ask for the bits you don’t understand to be explained.