Shopify modular monolith scaled to 30 TB per minute during Cyber Monday

Shopify uses a modular monolith architecture built with Ruby on Rails and MySQL.

2021 was their biggest Black Friday Cyber Monday Together with the Google Cloud infrastructure team, they achieved near-perfect uptime while averaging ~30TB/min of egress traffic across our infrastructure. That’s a massive ~43PB/day!

Here are some of the most interesting stats

https://twitter.com/ShopifyEng/status/1465806691543531525

59 points | by vladmihalcea 867 days ago

8 comments

  • codegeek 867 days ago
    Show this to everyone who shits on monolith and only wants to build a scalable serverless microservices whatever.
    [-]
    • cardosof 867 days ago
      Some people just want to build CVs, not products.
      [-]
      • dmlittle 867 days ago
        A service oriented architecture is not a bad thing. It has its benefits as well as it’s downsides. The key is to know when to use the pattern and when not to.

        I’m not sure what exactly “modular monolith” means but I’m guessing it’s some kind of mono repo with different services that share a large core but can scale independently. At Shopify’s scale I doubt you can just use a single service and database and not run into problems (both technical and development productivity wise).

    • speedgoose 867 days ago
      The secret is to build scalable monoliths. You probably need a few microservices to achieve that at this scale.
      [-]
      • 10000truths 867 days ago
        The secret is keeping performance in mind throughout your whole tech stack and in your application code. Do that, and you might not even need to scale beyond a couple machines for redundancy, depending on your SLOs.

        I’ve said this a couple times before, but it bears repeating - nginx can easily serve a million 1 kB static files per second over TLS from just one machine with a modern Xeon/EPYC CPU. Serving dynamic content doesn’t have to perform any worse than one or two orders of magnitude below that.

  • jlgaddis 867 days ago
    > ... averaging ~30TB/min of egress traffic ...

    That's 4 Tbps after converting to units that everyone else in the world uses for network traffic (assuming the calculator installed atop my neck is operating bug-free this soon after booting up today).

    [-]
  • ksec 867 days ago
    Not trying to downplay those results and achievements.

    But do each store front get their own instance of both Database and the App? If that is the case isn't that equivalent to running millions of Wordpress instances?

    I think I asked this before but I forgot what the answer was.

  • kello 865 days ago
    Does github still use Rails in a sort of monolith? I like examples of "monoliths" in the wild that gasp scale as well / better than dozens of microservices painfully working together just to show me a metrics dashboard. Dont get me wrong, not against microservices, it's just that more often than not such architectures have brought more pain than good in my career. I will most of the time gladly trade the overhead of maintaining/managing microservice-glue-infrastructure for a large modular monolith.
  • dyeje 865 days ago
    RaIlS doEsn'T ScAlE.
  • andrewdubinsky 867 days ago
    That's impressive.
  • yuppie_scum 867 days ago
    Near perfect
  • BilalBudhani 867 days ago
    BuT rAiLs DoEsNt ScAlE