Skip to main content

Can Twitter's backbone handle sudden jump in popularity?

For the past week or so, I’ve been noticing - and hearing some rumblings about - a greater-than-normal number of overload errors spewing from Twitter. Even though the overloads likely are stemming from the recent mainstream Twitter craze - which is a good thing - it’s clear that Twitter is not equipped to be handling this level of traffic. From an article published by ZDNET

That’s not a good thing.

In the month of February, U.S. traffic to Twitter was up more than 1,000 percent from a year earlier, according to comScore. Worldwide, the number of visitors in February was up more than 700 percent from the previous year. It’s probably safe to say that the growth curve will stay on the upswing for March and April, too. That means more traffic and more overload messages.

Over at the Tech Broiler blog, Jason Perlow has put up a post that explains why he thinks the backbone technology that’s processing these millions and millions of tweets is insufficient - especially given the growth numbers. Perlow refers to the processes as transactions - yeah, just like a major financial institution might manage. He explains that at any given time, millions of rows of Twitter data are being both written and read, which is different from, for example, Google’s search engine, which tends to be more “read-intensive.” Perlow writes:

Google is able to distribute their database and “spiders” the web as it needs to, inserting data into it using a sophisticated algorithm and fully in control of when it happens. While many millions of users query Google’s database on demand, data isn’t inserted into it so quickly that its requirements are I/O bound at the database level — much of what is being queried is being pulled out of cache, which is residing in memory. Google is able to minimize the number of disk writes and stripe reads over a large amount of systems using fairly off-the-shelf components. So can Twitter do the same thing as Google and build a large distributed cluster of systems, such as with Aster Data’s “Beehive” or a few hundred racks of Kickfires to solve the problem? In a word, no.

Perlow - who fully discloses his IBM connection - says Twitter needs a mainframe. Maybe two. I can’t speak for the needs of Twitter’s backbone with any real degree of expertise. But I do now that whatever is back there powering Twitter today obviously is not beefy enough to handle the growth in traffic. Sure, we’re in tough economic times and an investment in a mainframe might be tough to swallow - especially since Twitter still hasn’t announced a solid plan for how it plans to make money.

Also see: Twitter: A fine ‘pre-business’ but un-monetizable and a deadly acquisition target

But Twitter can’t just keep throwing up error messages and Fail Whale screens. How will the company grow membership - or even hang on to the members it has now - if the experience starts becoming more of a hassle than it’s worth? It has to make an investment of some sort - and Perlow goes on to explain why different approaches won’t work:

Twitter’s performance problems cannot be solved in a massive scale out distributed systems manner.. Twitter is doing all those simultaneous row inserts while it is doing reads, which is murder on CPU and on IOPS and you would eventually run into scalability problems, and the build out costs using [mid-range] systems would be cost prohibitive. ZDNET

Comments

Popular posts from this blog

A Golden Age for Cheapskates

In a Lousy Economy, People Dig a Bit Deeper to Turn Up Deals
By Nancy Trejos
Washington Post Staff Writer
Sunday, April 19, 2009

Last fall, the Woodbine, Md., mother of three figured out a great way to get some: online giveaways. She has entered about 40 so far. She has won T-shirts, cleaning products, a small portable vacuum, olive oil, beef jerky and -- best of all -- a Nintendo DS on Web sites such as http://MeTime.com, http://TheMotherhood.com and http://5minutesformom.com.

"The lure of free stuff is quite appealing," she said. "I never considered myself a winner. I don't think I ever won bingo. My name was never drawn from a hat. However, I've been extraordinarily successful at the giveaways."

The recession has emboldened a certain kind of consumer: The mooch. With dwindling retirement savings, a higher cost of living and wobbly job market, they don't just want discounts on items they used to pay full price for without a second thought. They want freebi…

THE ANCIENT SHIP APPEARS AGAIN AT SHORT SANDS YORK BEACH MAINE

A RECENT NOR EASTER HIT THE NEW ENGLAND COAST AND AS THOUGHT THIS WRECK SURFACED AGAIN FROM ITS BURIED DEMISE--The skeleton only appears periodically - the last time was in 2013 - always after a significant coastal storm, and always attracting attention. Archaeological work conducted in 1980 indicated the wreck is a sloop of about Revolutionary War age. It is likely a “pinky,” a type of vessel with a high, narrow stern and square rigging easily maneuverable along the coast of Maine. Pinkies were popular as fishing and cargo vessels. The first sighting of the skeleton was in 1958, and then it has surfaced periodically right up to the present day - and usually after a good spring nor’easter. Word spread quickly about this sighting via social media, and people came over the weekend and on Monday as well. First, they had to navigate the seaweed- and rock-strewn streets of York Beach. They were also precluded from parking cars in most of the Ellis Park lot, which was buried under a layer …

CREATE YOUR OWN MOSAIC

I've always felt a special fascination for mosaics. From Roman tile compositions to current digital collages, I think they're an awesome artistic expression.
That's why I was so happy when I found Andreamosaic.
This tool enables you to create amazing mosaics from your digital pictures in a very easy way.
It requires having a large collection of photos to use as tiles, that's true. Fortunately the installation file already includes a pack of 500 sample photos that produce excellent results.
The program's interface is quite dull; in fact it's simply a gray window with too much text on it. But the mechanics are easy to understand so you'll be creating your own mosaics in no time!
Plus, the program includes a 20-page manual that explains everything thoroughly. Just remember that the more tiles you use, the longer it will take to generate the mosaic and the larger the final file will be.

BY ELENA SANTOS CREATE YOUR OWN MOSAIC

How To Create A Photo Mosaic In Photos…