HomeCase studiesBroadside: mass email transmission

Broadside: mass email transmission

In a nutshell

It all started with “Can you build a system which send a million emails a day, reliably?” Large companies with millions of customers or partners often need to send out millions of emails a day, with order confirmations, e-tickets, and what not.

Challenges, not problems

All companies send emails from their corporate email system. Very few of these email systems can handle a surge of 200,000 emails generated from their business servers in an hour. Mass email transmission is a different animal altogether. And you can’t have your customer walking away because he is not receiving his airplane e-ticket on time.

The second challenge is that any system sending out millions of emails every day begins to look like …. yes, you guessed it: a spammer! We cannot have all the spam blockers in the world ganging up and pointing fingers at us, can we? Honest, tax-paying email transmission systems would go belly-up in no time.

The third issue here is simply one of size. How do you process, filter, log, and send out emails in such quantity that one or two servers fall woefully short? How do you orchestrate a dozen or fifty servers to work together to spread the good word without a hiccup? Excellent scalability and cluster management tools and processes are required.

Solutions, not ideas

What we built here looks nothing like all the corporate email systems our lead engineers have been designing and implementing since the mid-nineties.

The Broadside system uses huge amounts of parallel processing to handle huge data surges of email, and multiple distributed databases to keep track of everything. One executive can control the entire flow of one campaign or one batch from one console, irrespective of how many servers are at play.

The system runs on a cluster of Linux servers, and can scale up to 50+ servers without any problems. Each server is so finely optimised that our largest corporate customers have not been able to load more than two servers even during their peak hours.

Broadside is the only system we have ever designed where, given the right load, a server is stressed simultaneously on all four of its resources: CPU speed, RAM capacity, disk I/O speed, and network bandwidth. This ain’t no financial accounting package.

Broadside: a term from all-out gun battles during the golden era of sailing ships, when a ship would fire all its cannons from one side of its hull in a terrifying barrage of mayhem. We like the name!

Now that’s value

Our customers are some of the largest retail stockbroking firms, retail banks, and the largest securities depository of India.

For them, sending out a small batch of 200,000 or 1,500,000 emails and knowing that delivery is tracked and reported for each message, is a big load off their chests. Our systems run 24×7 so that they can reach their customers on time, maintain their customers’ trust. Brokers in India need to deliver trade confirmation notices to investors within a few hours of closing bell at the stock exchange, or else they incur the wrath of the regulatory powers. These brokers trust our systems to deliver on time. Broadside has risen to every reliability and scalability challenge thrown at it.