Why Discord, Pinterest, Moz and Bleacher Report use Elixir. Case studies

• Elixir is a relatively new language, created in 2011 with origins in Erlang. Read more about the characteristics of Elixir in this article.
• Despite young age, it has many adopters including tech giants such as Discord, Bleacher Report, Pinterest and Moz, whose case studies we are about to examine in this article.
• There is a rising demand for web developers with Elixir knowledge and skills. Want to stand out from the crowd? Start our Free Elixir Course today!

Discord: Handling growth and concurrency issues

Discord is a voice and chat service for gamers that has experienced continuous growth in the last years. Initially serving groups of less than 25 users, it arrived at the situation where there are up to 30,000 concurrent users at high-trafficked Discord servers like /r/Overwatch. As an early adopter of Elixir, Discord benefited a lot from its use. Let’s take a look at some particular issues the app was able to overcome thanks to it.

Genstage

In summer 2016, the mobile push notification system in Discord has encountered issues with burst notifications during traffic spikes. Shortly after /r/Overwatch’s Discord had passed 25,000 concurrent users, the system was significantly slowing down or even coming to a halt. Push notifications would either come late or would not appear at all.

Sending push notifications to Google’s Firebase Cloud Storage messaging service turned out to be the bottleneck. A decision was made to overcome its limitations by sending push requests to Firebase not through HTTP, but via XMPP.

To avoid further system overloading, GenStage was employed.

According to Elixir creators, GenStage is a behavior for exchanging events with back-pressure between Elixir processes.

Thanks to its key feature called back-pressure, GenStage knows exactly how many push requests Firebase’s XMPP connection can handle. In extreme situations, push requests are buffered. Whereas normally the buffer is empty, it may happen it fills up. That’s when GenStage’s Push Collector dumps the incoming push requests.

This is how no part of Discord’s notification system gets overloaded nowadays. The use of GenStage makes it possible for Discord to handle over a million push requests per minute. Learn more about GenStage on GitHub

Voice services which scale

The backend infrastructure of Discord’s voice services such as Discord Gateway, Discord Guilds and Discord Voice is written in Elixir. According to the Discord development team, it allows for lots of code reuse. This sole fact makes the code much more transparent and allows the voice services run on multiple platforms, such as web (Chrome, Firefox, Edge etc.), standalone apps (Windows, Linux, macOS) and phone (iOS, Android).

Thanks to the Elixir-driven efforts on both server architecture and client end, Discord was able to handle 2.6 million concurrent voice users with egress traffic of more than 220 Gbps (bits-per-second) and 120 Mpps (packets-per-second) in September. You can read about the details here.

Bleacher Report: From 150 servers to just 5

Bleacher Report is a leading real-time, mobile-first sports website which has faced concurrency issues during the major traffic spikes. Billions of page views and sent push notifications per month and over 250,000 monthly users - this is how the reality looked for the website’s engineers in early 2017.

Originally written in Ruby on Rails, Bleacher Report could no longer scale. After examining the alternatives, the engineering team of Bleacher Report decided to use Elixir for its backend. Reasons? The Ruby-inspired Elixir syntax makes it easy for Ruby developers to make the transition, which - combined with the vast performance advantage - made for an easy choice.

After switching to Elixir, Bleacher Report’s infrastructure was reduced from 150 servers to merely five, according to TechWorld.

Needless to say, this reduction saved BR huge amounts of money.

Pinterest

What have you pinned to your board lately? We know at least some of you use Pinterest for visual inspiration, as in the third quarter of 2018, the service crossed 250 million active users, who have pinned 175 billion items on 3 billion virtual pinboards. The social media giant has benefited from a similar backend and cost reduction as Bleacher Report.

It seems that, at least in some part, Pinterest owes theses benefits to Elixir. After converting to this language in 2014, the system delivered over 14,000 notifications per second, working on merely 15 servers - half less in comparison to the time it was written in Java.

The rate-limiting system for Pinterest’s API and Ads API is written in Elixir and responds faster than its predecessor.

Another reason for the use of Elixir by the 12.3 billion $ worth company is clarity of code. The new system reduced the amount of code to around 1000 lines, which is a tenfold less in comparison with the old Java version.

Read more about Pinterest’s conversion to Elixir here.

Moz

Moz is a SEO software development company. In 2016, the backend of Moz Pro, its digital marketing and SEO suite underwent a general overhaul. The goals included improving speed and performance of the application, and unlocking new features.

Moz Pro serves keyword ranking reports to its users in a cyclical manner. As more data were added to its toolset, the app’s architecture scaled poorly and did not perform well with large sets of data. In result, Moz Pro was unable to serve features like complete campaign history to its users.

The MySQL database storage turned out to be the main bottleneck of Moz Pro.

The Seattle-based company tried several solutions to improve the performance of the service. After having tested its own internal prototype using CSV files and an SQLite solution, Moz developers chose a database-free architecture and a data indexing model which runs on Elixir.

Elixir enables processing rankings from cold storage in Amazon S3 and loading it into memory. What’s more, data can be indexed up front, resulting in a ready-to-query structure. The data are smaller: they occupy 63 times less disk space in comparison with their corresponding MySQL tables.

The Elixir-driven API serves the data 20 times faster. Moz Pro can move the data from the cold storage stage to a finished “shard” in under 25 seconds in an average campaign. More complex campaigns take about two minutes to be presented. Which, in contrast to 13 (in case of the former) and 45 minutes (for the latter) is a huge progress, isn’t it?

The list goes on

The above case studies are only a foretaste of Elixir’s global adoption. Other companies who have been using this language (or at least are said to) include Netflix, Yala, Wistia, Slack and Adobe are said to use Elixir in production. This list is not complete though, as new adopters join this community on regular basis.

UPDATE

• PepsiCo eCommerce revealed that they are utilizing web development with Elixir and Phoenix for their marketing automation application. Learn about the details behind the use of Elixir programming in te CPG sector's leading company.

• If you want to know who else uses both the language and the framework, just take a look at the list of famous websites and apps built with Elixir and Phoenix.

• You can find out about more companies which use Elixir in production on this website

Wrap up

Concurrency and scalability issues seem to have been the main reasons behind the decisions of Discord, Bleacher Report, Pinterest and Moz to switch to Elixir. All of them have improved the performance of their services thanks to it. The change also resulted in code amount cuts, and significantly reduced the server infrastructure, saving those companies lots of money.

If you are still curious if Elixir is a right choice for your company, don’t hesitate to take part in our Free Elixir Course.

Wondering if Elixir is a good fit for your business? Get in touch with us and let's talk about it!

A big 'thank you' goes out to Michał Buszkiewicz for quality checking and valuable contribution to this article. And to Kuba Bakalarczyk for an awesome graphic.