Cloudflare Attributes Major Outage to Internal Error, CEO Apologizes

Cloudflare published a post incident explanation on November 20 saying the widespread outage that disrupted services for clients including social platforms and major internet services was caused by an internal error rather than by hostile actors. The company traced the failure to an unintended change in database permissions that led to multiple entries being written to a feature file used by its Bot Management system. Those duplicate entries caused the file to balloon, hit a hard coded file size limit, and trigger cascading failures across the network.

According to Cloudflare, the chain of events began with a permissions change that allowed excessive entries to be created in a component intended to identify automated traffic. The feature file grew rapidly until it exceeded an internal safety threshold that had been embedded in code as a fixed limit. When the threshold was breached, processes relying on that file failed, causing traffic handling across Cloudflare nodes to falter and many customer sites and services to become unreachable.

Engineers worked through the outage and core traffic was reported restored within hours. Full recovery was announced later in the day after teams implemented fixes and conducted verification across the global network. Cloudflare characterized the incident as the companys most severe outage since 2019 and said it would undertake immediate mitigations to reduce the chance of recurrence.

Chief Executive Matthew Prince issued an apology and outlined several short term measures the company instituted during and after the outage. Those steps included deploying global kill switches to isolate the problematic feature, limiting the size and frequency of error dumps that contributed to the overload, and changing operational procedures around database permission changes. Cloudflare also said it would revise internal processes and tooling to prevent similar permission changes from producing runaway data growth in critical files.

The outage revived debate among customers and industry observers about dependence on a small number of infrastructure providers that sit between end users and the broader internet. For many websites and apps, companies such as Cloudflare perform essential roles including traffic routing, security filtering, and content delivery. When those services falter, outages can propagate widely and quickly, prompting clients to reassess redundancy strategies and single point dependencies.

Cloudflare said it will publish a more detailed post incident report with timelines and technical analysis in the coming days and will work with affected customers to evaluate impacts. The episode underscores the operational fragility that can arise from complex automated systems, and it is likely to accelerate conversations about transparency, testing of fail safe mechanisms, and the design of guardrails in software that governs critical internet infrastructure. For end users the practical lesson is familiar and immediate, outages at pivotal infrastructure providers can ripple across services people rely on every day.