About

ProPublica is a nonprofit newsroom that investigates abuses of power.

To publish their news, ProPublica wanted a dynamic site with the performance and stability offered by a static site generator, and the simplicity offered by their content management system (CMS). With their CMS, they had begun to see tradeoffs between improved user experience internally, and increased complexity of the webpages. This was causing latency and taxing the site to the point of making it crash.

How they use Workers

• Reverse proxy: ProPublica worked with the digital agency, Happy Cog, to improve stability and latency by separating their origin and web servers. They use Workers as a reverse proxy so their origin servers would now only create and update content, while the reverse proxy would act as the web server focused just on responding to requests.

• Part of a caching layer in between Internet and CMS: Their solution included various parts: a cache data store, the origin server housing the CMS, the reverse proxy, a queue, and a queue Worker. As part of the solution, Workers receives the requests and then checks the cache data store. If the cache has expired, Workers then adds a request to the queue for that resource’s cache to be updated. As requests come into the queue, it generates the webpage from the origin and updates the corresponding cache in the data store.

Why Workers?

• Performance: Workers is a distributed serverless technology, running on the edge as close as possible to end users
• Redundancy: With the reverse proxy able to serve content in the cache, their origin servers could be offline, but the site could remain available and stable running on Cloudflare's network
• Stability: Since the web server just retrieves and serves resources, and does not generate them, the site can now handle more traffic without risk of crashing