TorrentFreak

The place where breaking news, BitTorrent and copyright collide

Twitter Uses BitTorrent For Server Deployment

Twitter is calling in the help of BitTorrent to deploy files across its many servers in a more efficient way. The project dubbed ‘Murder’ is based on the Open Source BitTornado BitTorrent client. Aside from assisting Twitter it is available to other developers at no cost.

twitter murderLarge scale web-services such as Twitter need thousands of servers to manage the flow of updates that are sent out by their millions of users. As a result, keeping all these servers updated with the latest data can be time and resource intensive.

To improve the deployment of files across their servers, Twitter is calling in the help of BitTorrent. With BitTorrent, Twitter is planning to distribute files faster and more efficiently, saving time and precious resources and improving the scalability of Twitter’s operation.

Twitter’s new project, codenamed ‘Murder’, will not use the bandwidth of Twitter users. Instead, it will transform the site’s servers into a large BitTorrent swarm that will distribute file updates using BitTorrent technology.

The setup is pretty straightforward. Murder uses a ‘seeder’ server where the new files will be distributed to thousands of ‘peer’ servers.

Because all servers assist in the deployment of the files, it will only take a fraction of the time it would otherwise take when files are distributed from a central server. This server-to-server BitTorrent technology also explains the name ‘Murder’ which is used to describe a flock of crows.

The Murder project is developed by Twitter’s Larry Gadea and built based on the BitTornado BitTorrent client, which was the first client to implement web-seeding. The code is open to the public and licensed under the free software Apache License.

Eric Klinker, CEO of BitTorrent Inc. is excited about Twitter’s adoption of BitTorrent. “We are thrilled to collaborate with them on this and hope that more Web Monsters out there look to this and other applications of BitTorrent in solving some of the hard problems of the Internet,” he said.

“The distributed nature of BitTorrent means an operation that once took many dozens of minutes, now happens in less than a dozen seconds. These efficiencies will reduce maintenance windows, site downtime and exposure to security vulnerabilities,” Klinker added. Although Klinker speaks about “collaboration” with Twitter it is not clear what role BitTorrent Inc. will play in the Murder project.

At this point it is still unclear when Murder will be implemented or released to the public but Twitter is expected to release more details on the upcoming project in the weeks to come.

Related Posts

Previous Post | Next Post

  • Mike

    Sounds sweet

  • Stoned

    I hope this opens the mind of some, for they to see that BitTorrent protocol only use is not piracy.

    And it is a good solution to speed up some things.

    Kudos Twitter!

    P.S.- I would also love to see Twitter recieving letters from their ISP saying they have been downloading copyrighted material…=P

  • anoit7

    Yeah! Bit Twitter!

  • No


    At this point it is still unclear when Murder will be implemented or released to the public but Twitter is expected to release more details on the upcoming project in the weeks to come.

    It is released and available through the code repository.

  • Joe

    this is kind of a dumb idea, it’s much more effective to write a simple app which pushes software updates etc out to all the servers, this way file versioning etc can all be monitored and logged much more efficiently than bittorrent.

    They have some of the brightest engineers in the world, and this is the best solution they can come up with? Pretty pathetic!

    BTW – RIAA/MPAA will be monitoring Twitter torrent traffic for infringements now :P

  • http://www.eZee.se www.eZee.se

    Another big boy using BT, not real news in my view – just a natural evolution. I’m surprised TF would take the time to write about it..oh well, I guess BT news is BT news.

  • Pingback: Twitter Uses BitTorrent For Server Deployment – TorrentFreak | Twitter Traffic Reports

  • Anonymous

    oh noes quick mpaa/riaa/ifpi/bpi sue twitter there using bit torrent and we all know (if your lead to believe) is illegal.

    STUPID BUGGERS.

    hopefully this will put the end to myths about bit torrent about time the parasites were put right.

  • Pingback: Kampf gegen den Fail-Whale: Twitter setzt auf BitTorrent | Basic Thinking Blog

  • Pingback: Twitter Uses BitTorrent For Server Deployment – TorrentFreak | Twitter Traffic Experts

  • Pingback: SolidNode » Blog Archive » Twitter Using BitTorrent to Speed Servers

  • Anonymous

    @8 thanks for the mods, it was getting crazy

    Good for Twitter and the BT protocol most of all, a positive step forward!

  • Walrus

    @5

    File versioning? You do know that bittorrent has extensive error correction, right? The pieces downloaded must match the hashes provided by the tracker exactly. Just layer something over the bittorrent protocol to track what murder clients have and what they need and leave the actual distribution up to bittorrent.

  • Anonymous

    @Joe: Then you end up with a bottleneck at the server distributing the update, which in their case was their git repository. Murder solves this problem.

  • Walrus

    @9

    Er. I didn’t mean error correction. Error CHECKING. The only “correction” would be redownloading that piece.

  • Annie Moose

    Glad to see someone finally recognizes the broad capabilities and flexibility of Bittorrent! Pre-Bittorrent, of course, P2P sharing was unwieldy for large networks. But with Bittorrent, it’s a much more efficient way to spread traffic out over a network–and because each of the servers can download from each other, rather than just one central server, it seems like this would definitely reduce “bottlenecks” and increase update speed.

    Kudos to Twitter for adding some respectability to a perfectly legal network type and protocol!

  • AMQP

    If Twitter needs a good middleware for messaging, there is AMQP.

    I have used OpenAMQ in the past, it works like a charm.

  • bigbadgirl

    extreme usage of torrenting

  • Ninja

    Wow, pretty nice. In their case they can’t really use the help of their users to distribute the updates as it might open security breaches – correct me if I’m wrong please. But it sure is an idea for reducing the load on, lets say, an online media store. The users could be offered a discount on their subscription if they helped with the distribution using the mp3/flac/whatever they downloaded from the network.

    Bittorrent is sweet. The future has been consolidated and has become present. We’ll see more companies following for sure. ISPs might not be that thrilled if the companies start using their clients bw to help though lmao…

  • Jim

    Id like to think that the internet will BE bittorrent, bittorrent not just being a protocol, but a way of communication.

    There is no reason for there not to be constant communication, its wasted resource. Every machine can pass traffic and share the burden, in the end the world becomes one giant circulation of data and everybody gets whatever they want!

    :D

  • CapnS

    Jim: I don’t think you understand how the internet (or BitTorrent’s protocol for that matter) actually works do you?

  • Reasoned Brain

    Jim doesn’t, that’s true. What he suggested is similar to having a multi-core processor (core=server in this analogy) but only ONE core doing 90% of all the work (the ‘central’ server uploading the new version to ALL other cores).

    Bittorrent allows the “other” cores to upload, once received a different part of the files/data.

    Example with only 3 servers: Server 1 (central) uploads part A to Server 2, part B to Server 3. Server 2 uploads part A (it has grabbed it from Server 1) to Server 3 and Server 3 uploads part B to Server 2… while this happens Server 1 also uploads part C. etc

  • neostyles

    See, bittorent does have some great legal uses. :)

  • Pingback: Twitter utiliza BitTorrent para la distribución en sus servidores | Bitelia

  • United Hackers Association

    HOW is the company BIT TORRENT involved with this open source this article needs make a distinction cause there is a big one if twitter makes a mistake and goes to bram the word will go out and vry few will use it

    keep that in mind twitter on your p2p travels the community remembers those like BRAM that screwed people around selling out to warner brothers.

    this also might help that open source project update and get modernized and maybe twitter will ask for better encryption like a ssl /ssh tunnel that cant be throttled UNLESS htey wish to thorttle there businesses

  • Wait a sec…

    Didn’t Twitter just denounce bit torrent a few days ago?

  • Terminator

    To avoid any confusion, the summary is that Twitter will use bittorrent to transfer data “server to server” only. End users like me and you are not part of the swarm. Any way it is glad to see bittorrent being used for more “legal purposes” :)

  • SomKen

    I can has .torrent file?

  • Pingback: Twitter implementará tecnología de Bittorrent para la distribución de datos en sus servidores | TekCrispy

  • Pingback: Twitter chiede aiuto a BitTorrent

  • Pingback: Twitter userà BitTorrent per distribuire i file tra gli utenti. | TuttoVolume

  • Pingback: links for 2010-02-10 | Stéphane Thibault

  • Pingback: Twitter Uses BitTorrent For Server Deployment « blog-me-not

  • Pingback: Mörder-Projekt – Datenabgleich via Bittorrent auf Produktivsystemen at webCONSUL

  • Pingback: Twitter utiliza BitTorrent para distribuir en sus servidores | MuyInternet

  • Pingback: Pirate Home Page » Twitter Uses BitTorrent For Server Deployment

  • Pingback: Twitter utiliza BitTorrent para la distribución en sus servidores @ Blog DDS.media

  • Nigel Stolting

    Nice to see a company not re-inventing the wheel and re-using a technology that is already available.

  • phineas

    so im supposed to offer bandwidth for text that is 99.9% nonsense. no thanks.

  • supaspoida

    Although someone else mentioned that the source to this is already available, they neglected to provide a link. Here it is: http://github.com/lg/murder

  • Pingback: Joegle » Hand Banana

  • Pingback: Twitter chiede aiuto a BitTorrent « YBlog

  • Pingback: Twitter stringe accordo con BitTorrent per la distribuzione dei file tra gli utenti | Internet

  • Pingback: Twitter utiliza BitTorrent para la distribución en sus servidores | Adobe User Group Granada

  • american

    There’s no need to call for a psychic on this one.

    Within months, it will get lawsuits from RIAA and MPAA and other failing craps.

    I can see Twitter being the next big Torrent wholr.

    Twit a file to users that follows you.

  • Pingback: Twitter deve usar tecnologia de Bittorrent em seus servidores - Twitter Brasil

  • BTGuard - BitTorrent Anonymously

NewsBits

Even more news...

  • Blu-ray Anti-Piracy Tech Stops Discs and Promotes Purchases

    An anti-piracy system present in all official Blu-ray players since 2012 has received a fresh update...

  • Foxtel Breeds Pirates by Locking Up Game of Thrones

    One of the main reasons why people turn to piracy is the lack of legal alternatives....

  • UK Student Admits Breaching Sony Copyrights With Leak of PS3 SDK

    Last year an Internet user known as El Nomeo leaked version 3.70 of Sony’s Playstation3 SDK...

  • Pirates Can Be Identified Despite Sharing IP Addresses, ISP Claims

    Carrier-Grade Network Address Translation is a network mechanism through which many Internet subscribers can share the...

  • Feds Seize Cash from Major Bitcoin Exchange’s Dwolla Account

    The U.S. Government has taken a significant action against the web’s top Bitcoin exchange by seizing...

MostDiscussed

Below are TorrentFreak's most discussed articles of the past month. Join the discussion if you like.

CopyQuote

Left Quote

“The Pirate Bay has been one of the most important movements in Sweden for freedom of speech, working against corruption and censorship.

Peter Sunde Left Quote

PopularArticles

A selection of some TorrentFreak's classics dug up from our archives.