The Pirate Bay argued that BitTorrent trackers have been made redundant by technologies such as DHT and PEX. In addition, The Pirate Bay team said that they might move away from torrents entirely and switch to offering magnet links instead.

“We’re talking to the other torrent admins on doing magnet links and DHT and PEX for all sites. Moving away from torrents and trackers totally – like pick a date and all agree ‘from this date, we’ll not support torrents anymore’,” a Pirate Bay insider told TorrentFreak at the time.

Now, two years later, that date is coming soon.

Today, The Pirate Bay made the first step towards this new future by making magnets the default download links instead of torrents. TorrentFreak was further informed that in “a month or so” the largest torrent site on the Internet will stop serving torrent files indefinitely.

The announcement is bound to lead to confusion and uncertainty among many torrent users, but in reality very little will change for the average Pirate Bay visitor. Users will still be able to download files, but these will now be started through a magnet link instead of a .torrent file.

The Pirate Bay team told TorrentFreak that one of the advantages of the transition to a “magnet site” is that it requires relatively little bandwidth to host a proxy. This is topical, since this week courts in both Finland and the Netherlands ordered local Internet providers to block the torrent site.

Perhaps even better, without the torrent files everyone can soon host a full copy of The Pirate Bay on a USB thumb drive, which may come in handy in the future.

Unlike the site’s users, existing torrent sites that scrape .torrent files from The Pirate Bay will have to make some drastic changes. If they want to continue serving .torrent files they will have to fetch them from DHT. Also, hotlinks to .torrent files will stop working and will soon redirect to The Pirate Bay’s detail page for the files in question.

One of the potential downsides of using magnets is that it could take a bit longer for downloads to start, especially if there are relatively few people sharing a file. This is because the .torrent file has to be fetched from other users instead of being downloaded directly from the site. More background on these and other technicalities can be found here.

The good news is that all mainstream BitTorrent clients support magnet links. This wasn’t the case back in 2009, but when The Pirate Bay hinted that in the future they could become a magnet-only site, all developers quickly made their clients fully compatible.

There’s no doubt that a torrent-less Pirate Bay will certainly mark the end of an era. At the moment it’s hard to predict what the impact of The Pirate Bay’s decision will be on the BitTorrent community. But torrents, however, will never disappear completely.

Source: TorrentFreak, for the latest info on copyright, file-sharing and anonymous VPN services.

However, the torrent eco-system didn’t collapse because in part TPB would rely on other BitTorrent features to facilitate transfers between peers.

Although they had been introduced years before, following this announcement the relatively obscure technologies of Magnet links, DHT and Peer Exchange would be thrust into the spotlight. From their quiet beginnings people suddenly wanted to know all about these trio of terms, which prompted us to create a user guide. The mainstream acknowledgement of the trackerless torrent had truly arrived.

While .torrent files continue to be the weapon of choice for most BitTorrent users, during the last year more and more sites have brought Magnet links on board, a development encouraged by the founders of The Pirate Bay. These days most of the major torrent clients support them.

As will become clear by the conclusion of this article, getting to grips with Magnets sooner rather than later may prove prudent. To this end, today we take a look at some addons, extensions, sites and services which make the use of Magnet links just a little bit easier.

Mainline DHT Plugin for Vuze

When DHT was first introduced back in 2005, the first client to implement it was Vuze (then Azureus). Unfortunately it uses a system that’s incompatible with those used in almost every other client. That means that Vuze users are cut off from everyone else. Not to worry though as with this Mainline DHT plugin, BitTorrent users are one happy swarm again.

The plugin can be downloaded here or through the client’s plugin installer.

Magnet Tracker

Magnet Tracker is a handy script that scans a web page looking for torrent hashes. The script then displays a window in the bottom right of a compatible browser window which allows the user to download a Magnet link. Magnet Tracker supports many of the main torrent sites and even offers functionality on Wikileaks in response to the site’s adoption of the technology last year.

As can be seen from the configuration window below, the script also allows for the customization of the Magnet link, such as with the addition of any number of additional trackers to add that extra downloading power should it be needed.

Magnet Tracker requires Firefox and GreaseMonkey to run and can be downloaded here.

Magnet Catcher

Continuing on the theme, Magnet Catcher strips the concept of adding Magnet links to a page right back. As can be seen from the before and after screenshots below, this script adds Magnet links straight to the main searchpage of a site containing torrent hashes, with no need to click-through to the torrent details page.

Magnet Catcher requires Firefox and GreaseMonkey to run and can be downloaded here.

Magnet Link Generator

If you already know the hash value of the material you want to download from BitTorrent, this basic webpage will convert it into a Magnet link.

Don’t let the foreign text put you off – simply paste the hash into the box and click the button and a Magnet link will appear. Click this and your client will pull its peers from DHT and PEX, no torrent site needed.

Go to Link Generator

Mgnet.me – The Magnet URI shortening service

As can be seen from the example below, one of the downsides to Magnet links is that they can be very long and therefore difficult to share.

magnet:?xt=urn:btih:d39ebe536d55ae4e422767f21eb997da11fe471c&dn=TorrentFreak.TV
.S02E12.HDTV.x264-TFTV.mov

Crucially they often have too many characters to be shared via Twitter, they are not clickable in IM apps like GTalk and MSN, and can be unfriendly on the eye. The Mgnet.me service changes all that.

Introduced earlier this year, Mgnet.me is a shortening service, much like Bit.ly or Tinyurl, designed specifically to convert Magnet URIs into shorter and more manageable links. As can be seen from the screenshot below, it also provides HTML code and a feature to post a newly shortened Magnet link directly to Twitter.

Go to Mgnet.me

Magnet Search Engines and Indexes

While many torrent sites are now offering Magnet links alongside the more traditional .torrent file, there are other sites which specialize in Magnet links only. The beauty for these sites is that they don’t have to carry .torrent files since they can generate Magnet links on the fly – as long as they know the hash of content in question that is.

Sailr

The first search engine in our list is the ultra-minimal Sailr. As shown in the screenshot below, this Magnet-only search engine has Mgnet.me implementation as standard.

Go to Sailr

BTDigg

BTDigg, is the first ever search engine for trackerless torrents. Rather than searching for .torrent files around the web like most engines, BTDigg scans BitTorrent’s DHT (Distributed Hash Table) to discover new files instead.

In theory, BTDigg should be able to discover any torrent sitting in a torrent client anywhere in the world, providing the torrent has DHT enabled in its settings. This means that torrents don’t even have to be uploaded to a torrent site, further decentralizing the BitTorrent ecosystem.

Go to BTDigg

Conclusion

Even though Magnet links (and their friends DHT and PEX) have been around for some time, the BitTorrent community has been relatively slow to adopt them as their primary route to content. The reason for that is a simple issue of supply and demand. While torrent sites, .torrent files and associated BitTorrent trackers remain in plentiful supply, most users will feel little need to make Magnet links their primary metadata choice.

However, that will all change when their favorite torrent site becomes censored by PROTECT IP or other blocking measures planned around the world.

In future – and armed with just a hash value – users may suddenly find themselves mysteriously attracted to Magnets.

Source: TorrentFreak, for the latest info on copyright, file-sharing and anonymous VPN services.

The Pirate Bay boasted the title of “the world’s largest BitTorrent tracker” for half a decade, but a year ago this title no longer applied. On November 17, 2009 The Pirate Bay decided to shut down its tracker for good. According to the Pirate Bay team central trackers had become obsolete.

“Now that the decentralized system for finding peers is so well developed, The Pirate Bay has decided that there is no need to run a tracker anymore, so it will remain down! It’s the end of an era, but the era is no longer up-to-date,” the team announced.

The Pirate Bay argued that BitTorrent trackers have been made redundant by technologies such as DHT and PEX. In addition, The Pirate Bay team said that they might move away from torrents entirely and switch to offering Magnet links instead.

“We’re talking to the other torrent admins on doing magnet links and DHT and PEX for all sites. Moving away from torrents and trackers totally – like pick a date and all agree ‘from this date, we’ll not support torrents anymore’,” a Pirate Bay insider told TorrentFreak at the time.

The announcements led to confusion and uncertainty among many torrent users, but in reality very little changed for the average torrent user. The Pirate Bay’s dominant position as a tracker has been taken over by two new ones, and even after a year .torrents are still available on The Pirate Bay.

What was interesting to see, however, is the response that came from the development community and torrent site owners. Before last year most torrent clients didn’t have support for Magnet links, and those that did spent little time on making them compatible and easy to use. However, after The Pirate Bay’s call for Magnet support this quickly changed.

BitTorrent clients such as Transmission, BitComet and Ktorrent all implemented support for Magnets this year. The clients that already offered Magnet support, such as uTorrent and Vuze, didn’t sit still either and spent time optimizing their implementation.

Similarly, the operators of other torrents sites were also listening in and nearly all of the larger torrent sites that didn’t already offer Magnet links soon added them. In January, this was followed by the launch of the first ‘Magnet-only’ torrent index named TorrIndex. Clearly, the words uttered by The Pirate Bay operators had not been in vain.

As for the tracker that was shut down, aside from the sentimental value it hasn’t really been missed. OpenBitTorrent and PublicBitTorrent quickly took over and have been going strong ever since, and not without a reason.

Larger torrents with thousands of peers will work just fine without a central tracker thanks to technologies such as DHT and PEX. But the majority of torrents out there only have a handful of peers and for these files a central tracker is still an essential part of the downloading process.

The shut down of The Pirate Bay tracker last year marked the end of an era, but as it stands now BitTorrent trackers are not defunct yet.

Source: TorrentFreak, for the latest info on copyright, file-sharing and anonymous VPN services.

The key thing to understand is that nobody is being forced to use Magnet links or trackerless torrents. While these long-standing technologies may prove to be the future, they will co-exist with tracker-enabled torrenting for quite some time. For now, nobody will be forced to immediately change their existing downloading habits, although it may be wise to switch to a BitTorrent client that is compatible with these technologies.

In an attempt to clear some of the mystique surrounding DHT, PEX and Magnet links we will walk through all three briefly, hoping to assure those who’ve become confused earlier this week.

DHT

Using DHT instead of trackers is one of the things The Pirate Bay is now trying to encourage, and torrent downloads that rely solely on this technology are often referred to as “trackerless torrents.” DHT is used to find the IP addresses of peers, mostly in addition to a tracker. It is enabled by default in clients such as uTorrent and Vuze and millions of people are already using it without knowing.

DHT’s function is to find peers who are downloading the same files, but without communicating with a central BitTorrent tracker such as that previously operated by The Pirate Bay.

DHT is by no means a new technology. A version debuted in the BitTorrent client Azureus in May 2005 and an alternative but incompatible version was added to Mainline BitTorrent a month later. There is, however, a plugin available for Azureus Vuze which allows it access to the Mainline DHT network used by uTorrent and other clients.

Peer Exchange (“PEX”)

Peer Exchange is yet another means of finding IP addresses. Rather than acting like a tracker, it leverages the knowledge of peers you are connected to, by asking them in turn for the addresses of peers they are connected to. Although it requires a “kick start”, PEX will often uncover more genuine peers than DHT or a tracker.

Magnet links

Traditionally, .torrent files are downloaded from torrent sites. A torrent client then calculates a torrent hash (a kind of fingerprint) based on the files it relates to, and seeks the addresses of peers from a tracker (or the DHT network) before connecting to those peers and downloading the desired content.

Sites can save on bandwidth by calculating torrent hashes themselves and allowing them to be downloaded instead of .torrent files. Given the torrent hash – passed as a parameter within a Magnet link – clients immediately seek the addresses of peers and connect to them to download first the torrent file, and then the desired content.

It is worth noting that BitTorrent can not ditch the .torrent format entirely and rely solely on Magnet links. The .torrent files hold crucial information that is needed to start the downloading process, and this information has to be available in the swarm.

Pirate Bay links cf. Mininova links: When the Magnet link specification first came out, in January last year it called for a particular format (“base32 encoded”). The links that EZTV, Mininova and ShareReactor have displayed for some time all conform to that original specification. In May of last year the specification was changed, in favor of “hex encoding”, and that is the format of the links being displayed by The Pirate Bay. Torrent clients should accept either format.

Compatible Clients

All the main torrent clients: uTorrent 1.8.5, Vuze 4.3.0.2, BitTorrent 6.3, BitComet 1.16, and Transmission 1.76 (and others) support Peer Exchange and DHT (via a plugin in the case of Vuze). Neither BitComet nor Transmission yet support Magnet links but Transmission is planning to include Magnet link support in the upcoming 1.8 release. Bearing in mind that no site, including The Pirate Bay, has yet abandoned support for traditional torrent files, there is plenty of time for support to be added.

We hope that this article has cleared some of the smoke that was generated by The Pirate Bay’s announcements earlier this week. There is no need to panic, cry or be angry, and it’s not a problem if you’re still confused after reading this article. Torrents will still be available and aside from some extra downloading options thanks to sites that add Magnet links, nothing drastic will change in the near future.

Props to ‘Adapa’ for contributing to this article.

Source: TorrentFreak, for the latest info on copyright, file-sharing and anonymous VPN services.