TorrentFreak

The place where breaking news, BitTorrent and copyright collide

BitTorrent’s Future? DHT, PEX and Magnet Links Explained

This week The Pirate Bay confirmed it would shut down its tracker for good, instead encouraging the use of DHT, PEX and magnet links. This move confounded many BitTorrent enthusiasts, who although wishing to adapt, were confronted with hard to grasp terminology and technology. Time for some explaining.

The Pirate Bay’s recent confirmation that they had closed down their tracker since DHT and Peer Exchange have matured enough to take over, was coupled with the news that they had added Magnet links to the site. This news has achieved its aim of stimulating discussion, but has also revealed that there is much confusion over how these technologies work.

The key thing to understand is that nobody is being forced to use Magnet links or trackerless torrents. While these long-standing technologies may prove to be the future, they will co-exist with tracker-enabled torrenting for quite some time. For now, nobody will be forced to immediately change their existing downloading habits, although it may be wise to switch to a BitTorrent client that is compatible with these technologies.

In an attempt to clear some of the mystique surrounding DHT, PEX and Magnet links we will walk through all three briefly, hoping to assure those who’ve become confused earlier this week.

DHT and PEX in action

dht pex

DHT

Using DHT instead of trackers is one of the things The Pirate Bay is now trying to encourage, and torrent downloads that rely solely on this technology are often referred to as “trackerless torrents.” DHT is used to find the IP addresses of peers, mostly in addition to a tracker. It is enabled by default in clients such as uTorrent and Vuze and millions of people are already using it without knowing.

DHT’s function is to find peers who are downloading the same files, but without communicating with a central BitTorrent tracker such as that previously operated by The Pirate Bay.

DHT is by no means a new technology. A version debuted in the BitTorrent client Azureus in May 2005 and an alternative but incompatible version was added to Mainline BitTorrent a month later. There is, however, a plugin available for Azureus Vuze which allows it access to the Mainline DHT network used by uTorrent and other clients.

Peer Exchange (“PEX”)

Peer Exchange is yet another means of finding IP addresses. Rather than acting like a tracker, it leverages the knowledge of peers you are connected to, by asking them in turn for the addresses of peers they are connected to. Although it requires a “kick start”, PEX will often uncover more genuine peers than DHT or a tracker.

Magnet links

Traditionally, .torrent files are downloaded from torrent sites. A torrent client then calculates a torrent hash (a kind of fingerprint) based on the files it relates to, and seeks the addresses of peers from a tracker (or the DHT network) before connecting to those peers and downloading the desired content.

Sites can save on bandwidth by calculating torrent hashes themselves and allowing them to be downloaded instead of .torrent files. Given the torrent hash – passed as a parameter within a Magnet link – clients immediately seek the addresses of peers and connect to them to download first the torrent file, and then the desired content.

It is worth noting that BitTorrent can not ditch the .torrent format entirely and rely solely on Magnet links. The .torrent files hold crucial information that is needed to start the downloading process, and this information has to be available in the swarm.

Pirate Bay links cf. Mininova links: When the Magnet link specification first came out, in January last year it called for a particular format (“base32 encoded”). The links that EZTV, Mininova and ShareReactor have displayed for some time all conform to that original specification. In May of last year the specification was changed, in favor of “hex encoding”, and that is the format of the links being displayed by The Pirate Bay. Torrent clients should accept either format.

Compatible Clients

All the main torrent clients: uTorrent 1.8.5, Vuze 4.3.0.2, BitTorrent 6.3, BitComet 1.16, and Transmission 1.76 (and others) support Peer Exchange and DHT (via a plugin in the case of Vuze). Neither BitComet nor Transmission yet support Magnet links but Transmission is planning to include Magnet link support in the upcoming 1.8 release. Bearing in mind that no site, including The Pirate Bay, has yet abandoned support for traditional torrent files, there is plenty of time for support to be added.

We hope that this article has cleared some of the smoke that was generated by The Pirate Bay’s announcements earlier this week. There is no need to panic, cry or be angry, and it’s not a problem if you’re still confused after reading this article. Torrents will still be available and aside from some extra downloading options thanks to sites that add Magnet links, nothing drastic will change in the near future.

Props to ‘Adapa’ for contributing to this article.

Related Posts

Previous Post | Next Post

  • vyvyan

    I always wonder why is TF so partial to deluge. Deluge supports PEX/DHT as well as magnet links, now for a long time

  • audiovissual

    Still confused

  • tutame ex inferis

    But it’ll be all irrelevant soon anyway, ’cause you sorry ass will be hauled infront o0f court and banged up

  • Mr.Afghanistan

    Magnet Links sucks big time.
    P2P Tracker always been faster.

    P2P Trackers not going anywhere. so don’t talk bullsh*t.

    Magnet Links been always around and very unsuccessful, Magnet Links can’t replace current P2P technology.

    TPB changed their Tracker system coz they were sick of Anti piracy companies.

  • Huh?

    Seriously, you thought this would clear things up for the casual downloader who doesn’t upload in code and such? I don’t get it at all.

  • http://www.eZee.se www.eZee.se

    Ah, I was wondering which site was going to be first in doing this, TF, Ars, Wired, TC etc

    Good job TF, appreciate it, and will link to this article.

  • freeinternet

    ok no tracker is needed, but you still need a database and a frond end for the torrent+magnet links , so people can search and download them, correct?

  • Pingback: TorrentFetcher Searches for Torrents from Your Desktop | Plates55 Blog

  • ah, makes perfect sense..

    uh what?

  • Soundwave (Have A Cigar)

    I don’t understand how anyone can have been using BitTorrent all this time and not know what these things are.

    Don’t you look things up when you don’t understand? If I call you an imbecile, and you don’t know what it means, it is in your best interest to look up the meaning of the word.

    And this article couldn’t have made it any simpler. Do you even know what a peer is? God damn.

  • Soundwave (Have A Cigar)

    Ask specific questions. What part don’t you understand?

  • Nick

    Ok, so you explained it technically, but I don’t think you’ve explained WHY this is better (for the user) than trackers.

  • Still don’t get it

    So how does DTH find peers when there’s no tracker telling them?

  • dextryn

    I personally think that this is a good way to go for the public tracker system.
    In my opinion private trackers won’t go this route for a while because of the ratio system. Without any way to determine ratios, private trackers would have a hard time staying alive.

    That being said, anyone have an available TL invite? I can show some ratio screenshots other trackers as well as a speedtest if you’d like.

    email me at inthistwilight65 (at) gmail (dot) com

    If you’re looking for some kind of trade I can get you an IPT invite and a Demonoid one (when it comes back up).

  • Anonymous

    Question: how do you find the FIRST peer to connect to?

  • neb

    It’s still file sharing………

    POS

  • OpenTPB.com

    There is severe concern that the non-centralized distributed peer groups could play into an entrapment situation where the voluntary peer groups, especially without an independent external infrastructure node (such as TPB or another public tracker), as the operational structure of the new PEX and peer groups is identical to the policy definition of international organized crime; As if the protocols for isolated peer groups were designed and implemented based on UN’s definition of voluntary support of and participation in organized crime structures.

    http://OpenTPB.com has additional information and will continue to expand this list of definitions and concerns.

    Although using a centralized tracker provides a plausible security risk, it also gives you pseudo-random peers from which to acquire data. Using an isolated peer group you control without an infrastructure style origin, your Socially devised network assumes you have personally approved and associated with all the others in the group; The peer group may allow rogue or undesired data to enter your community’s cluster, rendering your entire peer group as sponsors of criminal concerns, such as injected pictures of mickey mouse.

    Please take high concern of the security and ethical constraints when using new tools. Without a centralized public server such as TPB or the rest which are considered by international law as an infrastructure provider of services, not responsible for the content of the forums or postings or torrents, you take severe liability for the content introduced onto your personally sponsored private network of peers regardless if you want compromising content or not.

    It takes one rogue seed or entry point to inject deadly content. When using a torrent from a tracker, you know the source data’s characteristics and hope the resulting data is clean. Avoid isolated peer groups, they are easily targeted for entrapment and criminal exploit.

  • blah

    oh noes, the big o’mighty pirate bay has started to load its ships with magnet links now that it has played out its role of being any useful. let’s all follow suit!

    the truth is, however, that magnet links will always be a slower alternative than downloading the torrent directly and DHT and PEX will never alone find more peers than if they were coupled with trackers (they would, in fact, most certainly find less peers).

    why hasn’t TF thrown its piratebay prayer rug in the closet yet? the future of bittorrent is not determined by some pharisaic dealers at the bay.

  • Damien

    A very insightful article, thank you. But I agree with #1: why don’t you mention Deluge ?

  • Borderliner

    No central point (that can fail). That’s the whole point. Magnetlinks are pretty much like a normal WWW URL, hence while they do need to be displayed they’re even harder to classify “illegal” than .torrents (especially as they don’t generally point to *where* the data can be obtained). PEX works between cliets and don’t need any third party at all. DHT does the same.

    All in all: DHT/PEX/magnetlinks mean a slower start when downloading, but ensure a longer life for torrents as they remove the need for a tracker.

  • Paolo

    @freeinternet
    Right, but since a Magnet link is only 50 characters long, it may be easily spread (and made searchable) in a variety of different ways unsuitable for .torrents (which, typically, have sizes of 20-100 kB).

    The sky’s the limit: Tweeter, identi.ca, serverless forums, simple steganography, KAD archives (made easily searchable through eDonkey or Emule for example), archives of magnet links hosted in hosting services and so on.

    And I am confident there are other ways I can’t even imagine right now.

    @Mr.Afghanistan
    Check your facts and get informed. Do proper tests as we did, and you’ll see that you lose just a few seconds, and only in the “bootstrap” phase, using Magnet+DHT+PEX instead of .torrent+tracker.

    @Still don’t get it
    This article may be useful:
    http://en.wikipedia.org/wiki/Distributed_hash_table

    Particularly relevant is the Kademlia algorithm.

    @Nick
    Main advantage for the user is the capability to continue file sharing without trackers and without torrents. Copyright enforcement legislation is moving (we are opposing to this but it’s unclear if we can win) toward trackers shutdown if the tracker is suspected to track copyrighted contents.

    Another very important advantage is that Magnet links dissemination is much much easier (see above) than .torrent dissemination.

  • troy69

    I’m still confused about this whole thing?!

    Plz tell us in simple words what we have to do yo still download/upload files? I just don’t get it!

    Now I know “how to” but I don’t understand the yechnology behind it all, is that something I got to learn to keep doing what I’m doing?

  • what if…?

    this is a bit complicated for me to understand.the hole thing.
    I have tried that magnet link to compare with normal .torrents,but it wasn’t downloading at all ( about 0.5 to 5kb/sec) even after being connected to a lot of peers and downloading for about 20mns.
    what was wrong?why did it not work at all?
    did i do anything wrong,or did i forget to something?
    could someone enlight me,please?

  • Soundwave (Have A Cigar)

    #16 OpenTPB.com,

    Who cares? The content is still hash checked, and peers are still banned for failed hash checks.

    I keep hearing the same question: why do we need this change?

    DHT + PEX eliminates the need for a tracker. If you know anything about BitTorrent, you know that trackers go offline, then you can’t connect to anyone without DHT.

  • Still don’t get it

    @Paolo: Of course I read the Wikipedia Article before qsking the question. But that isn’t very plastic. So my nick remains.

  • Paolo

    @OpenTPB.com

    Your analysis seems cute, but lacks references. Your concerns about “intent and direct willingness” are not new: KAD “network” accessed with eDonkey or eMule has been working for several years in even more sophisticated way than DHT (KAD allows search functions too).

    I hope you will put more references to legislation of various countries, since it is my impression that your analysis is flawed with regards to “private systems”. Why would you define accessing p2p through DHT a “private system”?

  • what if…?

    by the way: using utorrent 1.8.5

  • xxploit

    @Soundwave
    I guess it’s safe to say that stupid people are ignorant, and you can probably argue that ignorance is bliss. So why would a stupid person want to ruin their blisful life by learning something new? They just get on with the oblivious day to day repetetive shit.

    (Funny thins is that they probably won’t even get what i’m saying)

  • pavel_at_bitsnoop_com

    @19

    Magnet links indeed are very similar to just an URL, but copyright-enforcers sometimes prefer to play dumb. “Linking to copyrighted content!!11oneone”

    We run DMCA policy like this – if content is DMCA-crapped, we disable .torrent download link and display links to Google and a Magnet URL. Torrent is still searchable, can have comments, scrapes updated, etc.

    Surprisingly it seems that pirate-fighters that send DMCA requests to us can live with that – as linking to Google is not a crime. As well as diplsaying torrent/file names.

    So far we’ve had just a single case of a Goth musicians (if you can call them that) backed by RIAA-wielding lawyer demanding to take down even links to Google/Magnet. Well, they also demanded to block any files with names containing a list of generic Goth-song/Lovecraft keywords – but were asked to fuck off. :-)

    I can see a number such cases increasing greatly in near future as pirate-fighters will start to realise that they are losing battle against torrents. Time will tell.

    What is scary is that even Google themselves block search results on DMCA notices, internet censorship is tightening.

  • Paolo

    @by what if…?

    Do you use Azureus? If so:

    it is important to keep in mind that Azureus DHT is an implementation of Kademlia algorithm incompatible with uTorrent and all the other clients, which implement so called “mainline DHT”. So you might have a “split” between peers who use Azureus and all the others. This issue is easily resolved by installing the plugin mainlineDHT in Azureus (to do the opposite, e.g. to make uTorrent compatible with Azureus, is quite complicated).

    I think you can install it directly using a menu function inside Azureus.

    http://azureus.sourceforge.net/plugin_details.php?plugin=mlDHT

    If you don’t use Azureus, then you might have a different problem, but it’s impossible to say anything without further details.

  • Unions and Intersections

    The unicode characters, already established, for Union and Intersection, would be appropriate methods of representing public (union) magnet availability and private (intersection) availability, there are also various other established symbols of horizontal form, however the group based availability would most effectively make use of these known concepts and related characters.

    HTML Entity (decimal) ∩
    HTML Entity (hex) ∩
    HTML Entity (named) ∩

    HTML Entity (decimal) ∪
    HTML Entity (hex) ∪
    HTML Entity (named) ∪

  • yosh

    If DHT would mean ditching .torrent this would certainly be a good change, but unfortunately they still don’t.

  • Paolo

    @troy69

    Just click on “Magnet link” instead of “.torrent” file.

    Make sure your client has DHT and Peer Exchange (PEX) enabled and that magnet links are associated to your favorite client (usually everything is already configured, by default).

    That’s all. :)

  • Anonymous

    There will be lots of fake data for your client to reject and sloooow down your downloads.

  • Paolo

    @by yosh

    That function is very hard to implement in nowadays DHT. However, you can have a look to KAD network, if you wish that (nothing forbids to put magnet links archives or even .torrent in KAD network). Easiest way to perform searches inside KAD is, in my opinion, using clients like eMule.

    Nonetheless, I assure you that leaving torrents,trackers and huge centralized indexers is a nightmare for antipiracy federations and for their lawyers. This is the real breakthrough of this key-step to decentralization.

  • Soundwave (Have A Cigar)

    #32 Anonymous Coward

    Prove it, or pose it as a question.

  • Paolo

    @Anonymous

    You say: “There will be lots of fake data for your client to reject and sloooow down your downloads.”

    Apparently there should be no difference at all in comparison with a tracker. Actually, after hundreds of tests, we did not detect any difference in speed. Only the “bootstrap” is longer, but just for a few seconds.

    Am I missing something here?

  • Charlie

    Totally digging this article! It’s a great idea to help out your audience, although I’m still a bit confuzed…

  • Anonymous

    One of the jobs of a tracker is to ban clients sending fake or corrupt data. The anti-file sharers will take advatage of this.

  • townie2

    many private torrent sites make you disable PEX and DHT.

  • Paolo

    @townie2

    Private torrent sites must continue using trackers. It is their problem if they are willing to limit dissemination of culture, art and science and enforce ratios. Under my point of view, private trackers and indexers are enemies of the concept “sharing is caring” and free access to knowledge, so they have to solve the problems they create to themselves on their own.

  • Anonymous

    Well.. as a matter of fact, I’ll speak, eh, rather write.

    You’ve 3 ways, besides trackers, of getting peers: Local Peer Discovery (LPD), DHT, and Peer Exchange(PEX).

    DHT goes look on the “network” for people that has what you’re looking for. And connect to those you are interested. Then, our little friend PEX will exchange the peers, so you’ll end up with more peers to download. In the mean time, Local Peer Discovery will probably try some ip addresses trying to find peers.

    What does the tracker? Speeds up the connection to the peers, because is like having an address list. Is easier than just popping up and asking who the hell has this or that.

    Alright, now let’s go with magnetic links: They are just the hash of the file you want to download. You’ll look for people that has the .torrent (which contains vital information of the torrent, blocks, parts, size, etc..) and get it, then “run it” and you’ll be then probably start downloading. TORRENTS can’t dissapear, at least from now, because you need that vital information to certify and know what to download.

    What does at the end this do to the common user? A slower start, maybe. Doesn’t have to download the .torrent him/herself since it is done by the client. The torrent will never die because of lack of tracker. It’ll only die because there are no seeders and no enough parts to make one.

    What are the adventages for the sites running up? They won’t store .torrents, probably. They will save bandwitdh. And no tracker will be needed, so less legal troubles for them.

  • Paolo

    @Anonymous

    Ok, now I get your point, but it’s wrong. The function to ban nodes which send corrupt data has always been a task of the clients. Trackers have never been able to do that efficiently (it would be an overwhelming task on large swarms to check EVERY single piece of data sent). Also, I think that most torrent clients let you configure the number of corrupted segments received before banning a peer.

    So don’t be worried, nothing changes.

  • Paolo

    @Anonymous

    Ok, now I get your point, but it’s wrong. The function to ban nodes which send corrupt data has always been a task of the clients. Trackers have never been able to do that efficiently (it would be an overwhelming task on large swarms to intercept and check EVERY single piece of data sent, and it would not be p2p anymore!). Also, I think that most torrent clients let you configure the number of corrupted segments received before banning a peer.

    So don’t be worried, nothing changes.

  • hemimaniac

    First off towards the whole Deluge thing, one can’t really blame them for not mentioning, tho it is a great open source client, its home website lately has been down more then it has been up, not exactly a “bell ringer” for continued upgrades and support ( And i love Deluge as a gui client so this pains me to add)

    Next, as far as i can see, two different trends are going to emerge from this kind of move, those who are the casual downloader ( plaguing bigger private sites for years – The HnR’r yes well all see them, show up, grab a file and burn, then is the first one to flame in other comments because there only one seed and he/she is only downloading @1.5 | kbs, IMO those peeps are just ripe to load up their favorite “limewire” like software and have-at-it! On the other hand, IMO there is the way of the .torrent, a true bonified way of getting ACCURATE and COMPLETE info, any doubt to this? take a client that has been allowing dht, magnet links for as long as i can remember, limewire,bearshare or what not and load them up, 4-5 pages of grief for 1 file, not to mention dupe listings and rubbish everywhere!

    Either way, IMO, there will always be 2 sides to this argument, and NOTHING that one side says will influence the other, and furthur more, depending on wich side you see from, there is now way to embrace the other without feeling different in some way

  • hemimaniac

    My deepest apologies, that should read

    plaguing bigger private sites for years not what i missed,

  • anon

    DHT, PEX and Local peer discovery are horrible for limted bandwidth users.

    If you want the fastest speed and be able to download/upload while browsing, disable all this crap.

    1 reliable tracker is all you need.

  • hmm

    thanks TF for yet another informative article to the masses. god i hope you don’t pay much attention to the trolls!

  • RoestVrijStaal

    Bittorrent goes back to oldschool-P2P:
    The Gnutella1 and Gnutella2 networks are using DHT and Magnet all the time ( because the Bittorrent-protocol is based on them ). And they are still up and running.
    Conclusion: Bittorrent = forever. What will be next? Adding the DHT-implementation KAD-protocol of ED2K or an other DHT-implementation? :3

  • German Navy

    Is it a legal work around to avoid hostin .torrent files? I still dont get it.

    http://www.plentyoftorrents.com

  • Anonymous

    heh. This is like that New Math us old timers had to learn years ago. ;)

    Still confused to be honest but thanks for the article. Maybe after a few uses, we’ll all get the hang of it. I for one understand DHT and PEX. It’s just when magnet gets thrown in is where I get confused. Just as long as I can keep downloading *.torrent files that allow me to do DHT and PEX, I should be fine.

    Gotta admit though it’s moot as I believe I’ve only used TPB once.

  • Anon

    To those offering up advice on how DHT/PEX works, while I am just as friendly as the next file sharer, I think, and I could be wrong, that we should be a little more tight lipped about how all this works. Its a known fact that the corporate trolls frequent these forums and any advice you give also goes directly to them.

    I know they can look this stuff up just like those who really don’t understand how all of this works. My point is we should be a little bit careful of what info we make available. The battle lines have been drawn, spies are amongst us, don’t give them any weapons to use against us. Loose lips sink ships!

    Just my humble opinion. Flaming commences in 3…2…

  • bittorrent forever

    You missed too far this time.

    The key problem of bit torrent is not tracker or trackerless. The problem is these site (TPB) is that TPB is setup to be a failure.

    TPB inflate the number of users anbd downloaders. It upset the MPAA. And MININOVA too. MININOVA claims that it reached 10B torrent downloads, but how many of these downloads come from the real end ueers? How many of them come from the search engine.

    If pirates shouting all the time to the world how much money he made from rubbery, he is doomed.

    It is going to be the same. Even TBP goes to megnet, if they do not change the current business model, they can still be hunt down by MPAA.

    The most important thing in this p2pdownloading thing is, keep low profile, do not upset the mainstream.

    A real pirate neverl claim he is a pirate. A real pirate is the one who take, and hide.

  • who cares

    no clearing up should have been needed. if you can figure out how to install a torrent client, then find sites that give you torrent files, you should be smart enough to understand that technically nothing changed. you are still downloading from a swarm of peers just like you did with a .torrent file. the only thing missing is the .torrent file. the rest of the process is essentially the same and the differences are not enough to warrant a technical spec explaination. just know that you are still getting what you want, the way you always have gotten it, just minus a generic file that has not been necessary for a long time. private sites will still have to offer .torrent files so they can track ratio’s. if you are a member of such a private site, again, you are smart enough to understand that nothing serious has changed. to those who think dht is evil (and you h33t people know who you are) wake up and smell roses, it is no more or no less safe than not using it.

  • bittorrent forever

    There are 1,300,000 torrents totally on mininova.org, if there are 100 search engines, it will be 130,000,000 downloads. If you build your search engine smart enough, it would not repeatedly download the same torrents again. But if it is not, the crawler could download the same torrents many times. Then, you will see your site traffic is very high.

    How do I know this? Because, I have several crawlers walking on lot of torrent sites everyday.

    (I do not have torrent site, but I do have crawlers running at this moment. Not just one crawler, I have 10 crawlers running on two computers, searching lot of torrentsites all the time).

    I am telling you, lot of site claimed they have some 2~3M torrents, it is fake. They never had.

    So, I really do not understand why these sites claimed they are biggest, largest, most advanced, most downloaded …. why bluffing?

    I mean downloading copyright movie is stealing. Admit it. We know. But why I agree some people go download? Because you can not buy all the movies or music. And you really do nto have time to do that.

    As long as it is not hurting the mainstream industry, who cares?

    The people like downloading, they will rarely have chance to go cenima, ro drive too far to DVD shop. The people who enjoy high quality, will always be high quality staff.

    So, why the hell MPAA is hunting us? Because some pirates are shouting on the street, I am stealing your goods, and I am stealling a lot. Catch me if you can.

    Then, that is why we are here.

  • *

    @German Navy

    Magnet links substitute hosting .torrent files.

    DHT substitute running peer trackers.

    Ask for more detail.

    …

    @Anon (50) – I wouldn’t worry about it at all, as you say, it’s all commonly accessible knowledge anyone can read about online in precise detail if they want. Any media-friendly companies interested in exploiting the technology are already well and truly capable of understanding the protocols involved.

    I would suggest that security through obscurity isn’t really security at all. The strongest systems are those that are wide open to scrutiny from the start. BitTorrent is kinda like that in many ways already, which is why it’s proved so resiliant to attack… it’s good protocol, doesn’t need to hide.

  • Anonymous

    Afaik DHT has a central point too. Your client should connect to a database to reach others downloading the same hashed torrent. 2 questions depending on the answer:
    1. If there is a central point in DHT then why do we call it decentralized?
    2. If there isn’t any central point, then how can it find the peers????

  • Anonymous

    @55

    seconded, I had the same question

  • rawr

    People should remember that one of the sites sued back in the days of emule was ed2k-it.com which was taken over by isohunt (as they were being sued at the same time anyway). Ed2k-it did not host .torrents, only ed2k links, which are the equivilent of magnet links. Removal of .torrent in favour of magnet links doesn’t solve any problem what so ever.

  • *

    @Anonymous (55/56)

    1. There are fallback DHT ‘bootstrapping’ servers as I understand it that kick in if the client doesn’t already have it’s own valid local node list to get started with, so yeah, that much is a centralized step (and a weakness), but the generic operation of the DHT network after this process has enacted once should allow for clients to lose any dependence on a centralized entity as they build and store their own local list of nodes for future use to enable them to contact other DHT nodes directly instead.

    I *think* that’s the gist how it goes anyway, though don’t quote me on it… =)

    2. As above, if the client has previously built a locally stored list of valid nodes (file:”dht.dat” in uTorrent) it shouldn’t need to fallback on a central bootstrapping server to get primed again. I imagine you could prime it manually if you ever really needed too also, so don’t imagine it an especially vulnerable attack vector for anyone wishing to halt people’s use of DHT.

  • General Snus

    I understand PEX & Magnet links no problem. PEX is simple, it asks your existing peers who their peers are. Magnet links have the hash so that when a peer is found, it gets the rest of the info in the torrent (like names of files and size) from there. Very simple. (Although I think uTorrent’s implementation could be improved – with 1.8.4 it automatically starts any magnet link, without the dialog that allows me to tell it to start or not, where to save it, what files to get, etc… I do realize it would have to find a peer with all the .torrent info before it could ask me half those questions, but anyways thats not the reason I’m writing).

    The only thing I don’t understand is how DHT finds peers, or wtf DHT really is. I’ve read the wiki, but it isnt very clear either.

  • FU7777

    To xxploit:

    Were you abused as child? Please get some help for your insane rage and ranting, and your inability to answer quite valid questions, that you may happen to have a good deal of knowledge of. However, not all of us were fortunate enough to be blessed with omniscience. (Yes, you can look that word up as I would be willing to bet you don’t have a clue what it means!)

    Also, learn how to spell, otherwise you’re no better than the so-called “stupid” people you are calling ignorant. Is your own ignorance bliss for you? Because you have certainly cornered the market on it.

    So FU very much!

  • Traum

    I don´t understand DHT PEX what_ever, but I still use my BT client and it works. Maybe it takes some time and many more articles like this and understand teh difference… ;-/

  • Soundwave (Have A Cigar)

    #46, Anon,

    I have limited bandwidth, and haven’t used a tracker in about 6 months.

    As already stated, relying exclusively on DHT over a tracker, the torrent takes longer to get up to speed. Beyond that however, DHT has no problem saturating your bandwidth.

    In addition, you should have no problem browsing the internet while using DHT and/or PEX. At least I don’t, but it’s possible that could be because I use uTP in addition to TCP.

    (uTP is a Mainline/uTorrent exclusive protocol at the moment)

    So, as far as I’m concerned, you’re wrong.

  • dunno

    what i want to know is when one of these damn bittorrent clients is going to put kad support in their client kad+dht+pex+magnet= unstoppable awesomeness
    but we all know thats not going to happen because people are ignorant

  • Soundwave (Have A Cigar)

    The other thing is, with DHT and PEX enabled (should be on by default), you may find considerably more peers than with just one tracker. It depends on the tracker.

    I’m sure you’ve noticed if you’ve downloaded a torrent with more than one tracker, some trackers show 10 peers, and another tracker might show 350.

    So, if the torrent you download has one tracker, and the tracker is only showing 10 peers, DHT will show the full 350, for example.

  • BlueDragon

    As often the technical TF articles are raising quite more questions than giving answers…
    Excellent topic but should be treated more in depth IMO.

    @ General Snus
    This might give you an answer to your question about DHT: http://en.wikipedia.org/wiki/Distributed_hash_table
    Each file name is associated with a key (hash) and the keys are distributed amongst the nodes (clients) in a defined way. When one needs the data (file name) the key is searched amongst the nodes. Each nodes are linked to some others in a particular way allowing to minimize the necessary changes in the distributed hash table when nodes are coming or leaving.
    As with PEX, DHT needs a bootstrap to connect to the first other peer (node) able to share relevant information so in any case a central tracker is needed to start with but it can then continue without the need of a centralized tracker…

    PEX: Connected / dropped peer lists (IP:Port) are exchanged every 30 seconds with other connected peers.
    http://www.azureuswiki.com/index.php/Peer_Exchange

  • bernard

    I don’t care what anybody says, running DHT, PEX, and LPD enabled slows the small guy down.

    I normally run fine at 85KB down with nothing enabled, tracker only.

    For the entire week I ran with everything enabled, and for the entire week, I averaged 25KB to 45KB. SAD SAD SAD.

    So I tried lower my outgoing for a day from 12KB to 6KB and then 1KB. The best download I got was still a choppy 45KB.

    Within two minutes of turning DHT, PEX, and LPD off, I am running maxed out at 85KB again.

    You guys who don’t see any degradation probably have much better upload pipe than I.

  • Xcel

    LMAO!…. *Just shakes head*

  • bernard

    Something else I noticed, once I turned off DHT, etc., the peers quantity didn’t change, so theoretically, as long as everyone continues to actively stay connected, the pool to connect from is still available.

    IOW, people like I can turn on DHT to gather a pool of peers and then shut it off to regain speed.

  • spidermike

    bernard, maybe your router’s crap and struggles with udp/translations/whatever. some do… but dht/pex doesn’t consume naff all bandwidth wise in my experience, and i have it running 24/7 and keep a close eye on things. an average 1 kilobyte/sec would probably be a high estimate…

    but yeah, once you have a list of peers you certainly don’t need to keep dht/pex/lpd/trackers active, the job’s done unless you want more peers.

  • bernard

    @ spidermike

    I’ll take the router point into consideration. I know it fails when I hit too many connections or half-opens. But that’s just it, it fails = reboot, it doesn’t ‘half-work’.

    Having used several different cable and DSL providers, I’ve found the low end DSL (which I’m currently using) tends to be very strict pipe-wise. For instance, my actual down/up capabilities are 90KB/17KB, but using one affects the other. The best compromise I have for steady p2p is 85/12. Like if I made the connection 87/12, I would suddenly have lost many packets and my actual throughput would now be closer to 75/7.

    I don’t believe the router is at fault here, I’ve used it on the same connection with a 180/35 account and it ran fine, other than similar tuning for those speeds.

  • Bobe-On (Distributed Search Engines, and Tor, etc.)

    Thanks to TF and thead for the extra info…

    On 28 Nov 21, 2009 at 01:42, pavel_at_bitsnoop_com wrote:

    What is scary is that even Google themselves block search results on DMCA notices, internet censorship is tightening.

    “…Big companies control the entry point to the data you seek, and neither you nor web masters who run the sites have a say in the matter… Majestic-12 is developing a search engine scalable to billions of web pages that is based on support by the community.”

    – www . majestic12 . co . uk

    YaCy P2P Search Engine:

    “Advantages

    * As there is no central server, the results cannot be censored, and the reliability is (at least theoretically) higher.
    * Because the engine is not owned by a company, there is no centralised advertising.
    * Because of the design of YaCy, it can be used to index the ‘hidden web’, like Tor, I2P or Freenet.”
    – Wikipedia

    See also FAROO (Wikipedia)

    “Information [still] wants to be free.”
    – Stewart Brand
    (at the first Hackers’ Conference)

  • sp

    i don’t see how this makes you safer or better off from getting sue by the media companies.
    dht or no dht you ip adress is still displayed.

  • SMH

    As the copyright police go from friendly neighbor light footwalk patrol to putting on their SWAT gear, torrenting will probably go underground.

    This means that trusted circles authenticated with something like a PGP-key or biometric verification identifying the peer (circle member) will have to go into effect—if they don’t exist already. I am not talking just about private tracking groups with their high level invites, although they might certainly claim to have a higher level security that excludes anti-piracy org intruders and their weasley torrent poisoners.

    Of course, I don’t know how much a peer can feel secure if he can get a “circle” of 1 million members worldwide (!) or whatever the minimum number is to constitute an effective swarm. Such a group membership count should really be–what?—three to four times whatever a decent swarm is (?).

    Ultimately we will all need to become members of the trusted circle with absolute high level authentication, since the corporate interests can outbid anyone with respect to the best justice system that money can buy.

  • Hom3r

    I HATE how people will use a technology or software and have NO IDEA how it works or what it actually does.

  • JImmy Somo

    No freakin way dude thats crazy.

    RT
    ultimate-privacy.br.tc

  • hmmm

    When bittorrent becomes edonkey/kad.

    Actually this is very bad, because people who use private trackers will be prevented from downloading stuff from those dht/pex sites.

  • @75

    “I HATE how people will use a technology or software and have NO IDEA how it works or what it actually does”

    What, like your brain.

  • Anonymous

    That’s their problem. Out of all the mainstream networks, only BT has the private site concept. Maybe it’s time for them to go into the history books like trackers.

  • Smoothy

    >It is worth noting that BitTorrent can not ditch the .torrent format entirely and rely solely on Magnet links. The .torrent files hold crucial information that is needed to start the downloading process, and this information has to be available in the swarm.

    Actually you can ditch torrent files entirely with magnet links and DHT enabled.

  • Cujo

    interesting about all the comments here ,, we go from “i’m an idiot” to “i’m a master” hehehe
    very interesting :D

  • Shin no Noir

    @BlueDragon
    “PEX: Connected / dropped peer lists (IP:Port) are exchanged every 30 seconds with other connected peers”

    It actually depends on the client how often it sends a PEX message. There is no official standard for PEX. I’ve seen clients that never send an ut_pex message (Azureus) and I’ve seen clients that send enormous ut_pex messages containing a few thousands peers (Opera).

  • Anonymous

    Quote :”DHT, PEX and Local peer discovery are horrible for limted bandwidth users.”
    Actually,limited bandwidth user are horrible for bit-torrent.Lets face it, you are not the ones pulling the large loads .

  • peter

    Quote :”DHT, PEX and Local peer discovery are horrible for limted bandwidth users.”
    Actually,limited bandwidth user are horrible for bit-torrent.Lets face it, you are not the ones pulling the large loads .

  • Bad MoJo

    The big advantage of Magnets is that since they are small, they can be included in the HTML source. Check the source of a page you are hitting and the hash for the torrent will be in the page. That means that since you can magnetise that with firefox you never download a torrent. The asshats cant indict every web hit, (although they would probably like to.) and have no idea whether you magnetised it or not.

  • Darth_yoda

    Hey Torrent freak Guys thanks for clarifying what DHT and PEX are but on Utorrent there is something else called LOCAL PEER DISCOVERY. Can you clarify what this is and how it works aswell? Thx

  • ]v[EGADET]-[

    “There is no need to panic, cry or be angry” hahaha, silly

  • fewfefewefeq47676

    Hmmm, they could better implement a system like Kademlia (used in Emule) which removes all need for sites and is searchable.

    OK Emule is pretty old and slow, if you can search BT via Kademlia and have fast downloads that would be the best of both worlds.

    Or start from scratch with a new protocol as BT is getting old as well.

  • Anonymous

    7 Nov 21, 2009 at 00:52 by freeinternet

    You need databases for a lot of things if you are talking about searchs yes we do need a database of a trusted source to find those hashs that are good or there could be a “reputation” scheme so you only track the reputation of people anonimously through encrypted keys.

    In short the hashes themselves don’t need to be stored they can be searched on the cloud but the reputation of the source needs to be tracked.

    12 Nov 21, 2009 at 01:03 by Still don’t get it

    The simplest way to explain that is maybe saying that DHT send a smoke signal and all other peers respond to it.
    DHT send a resquest anouncing it is in some address and everybody with a client listening respond to it and start negotiations(handshakes). The old way was that clients contacted a tracker server and send their data to it and it maintained a list of peers that contacted it.

    14 Nov 21, 2009 at 01:08 by Anonymous

    There are 2 ways that I know of the “shoot and pray” and the “initial tracker”.
    DHT can broadcast its intentions to a range of IPs and wait for a response and keep going until it finds someone. This could take a long time that is why most clients today have a specific server that distribut lists of peers that contacted them in the last hour. I think they call the thing a “seed list” it greatly improves the times and your own client in some cases will save a list of peers that he last contacted and see if they are online so the work to find fast peers is not all lost but this approach is time sensitive if you have been a month offline you will probably not find the same peers as before, and maybe a solution to this is identifying clients not by IP but by encrypted keys that way no matter what IP the client is on it will always be indentifyed but of course this raises privacy concerns and has to be done in a way that anonymous communications can be stablished and that probably is not the bittorrent protocol.

    16 Nov 21, 2009 at 01:10 by OpenTPB.com
    A valid concern that is why a reputation system is requeried for DHT to work properly or it will be like emule that you don’t know what is being transferred until you get the entire file.
    About liability that can be aliviated by using anonymity measures that are not provided by the bittorrent protocol at the moment and with so much people being throtled anonymous clients are looking better and better everyday as the speeds are mostly the same.

  • Xcel

    Why ask why, just drink bud dry, LoL…

    It would havve been much easier to just let Ppl figure it out for themselves…
    I mean really , the new versions of allot of BT clients simply default to DHT/PEX when trackers are down, Ppl continue to dl their torrents in blissful ignorance, but NOOOOOoooooooooo, somebody actually had to try to explain it all to them and now Ppl think they have issues when they have absolutely none!

    First of all, Trackers are needed to initially up a torrent, it just does not appear magically with swarm attached, LoL

    Second, Magnet links are only “slower” to *START* after your client finds the metadata necessary, your downloads can (and in most cases *are) actually be faster…

    Third, DHT/PEX and “local Peer Discovery” all working together is a beautiful thing..

    LPD finds peers that are closest to you, this is good because it can/will actually improve transfer speeds…

    DHT/PEX finds peers/seeds all over the internet,that have the same file, it is not restricted to a(or a few)trackers..this is *GOOD* the more peers the better…

    ………

    Now with UTorrents new Beta – 2.0, there are even more technologies being tested, eg; UDP tracker support for dling torrents using the UTP protocol with UTs own version of “STUN”…

    Ok,ok, ok Now theres probabbly waaaay to much information for allot of you, LMAO…

    All of this can be really allot easier, “Dont worry-Be happy, and DL and seeed till ya bleed…
    Let the app writers and beta testers work the bugs out and worry about the “How & Whys & Whats”

  • Anonymous

    90 Nov 21, 2009 at 15:42 by Xcel

    Agreed is difficult to try and explain things but the process is a very needed one.

    It scales to other things too, like politics.

    I find myself frustrated by the ignorance of many but then I realize that I too am ignorant if I can’t find a simple way to showcase my line of reasoning.

    So although the process is painful we should strive to continue and do our best to inform others how things work for the sake of transparency, fairness and a balanced society.

  • ws

    @by*

    You state:

    Magnet links substitute hosting .torrent files.

    DHT substitute running peer trackers.

    Ask for more detail.

    I am asking for more detail.

    How do I create a magnet link so I don’t have to host my torrents on Mininova, TPB or another site?

    Also two new trackers state:

    OpenBitTorrent is a bittorrent tracker free for anyone to use. You don’t need to register, upload or index a torrent anywhere, all you have to do is to include the OpenBitTorrent tracker URL in your torrent.

    PublicBitTorrent is a bittorrent tracker free for anyone to use. You don’t need to register, upload or index a torrent anywhere, all you have to do is to include the PublicBitTorrent tracker URL in your torrent.

    Again how would you know my torrent exists for you to download if I don’t upload or index it somewhere?

    It has been 25 minutes since I created a torrent with no trackers and no one has connected to it so far.

    I believe there is a fallacy in the statement that a torrent doesn’t have to be uploaded to a hosting site.

  • l33t

    wow all you people need to getz a life!

  • Smoothy

    To correct my comment in 80. Yes, you do need the torrent file (it contains all the information for the file being seeded) but you don’t need a tracker.

  • ws

    Update

    It’s been an hour since I created my unhosted trackerless torrent and so far no connections.

    How does this work again?

  • Smoothy

    @ Ws

    >How do I create a magnet link so I don’t have to host my torrents on Mininova, TPB or another site?

    That’s a good question. I assume for other to download your “torrent” you somehow have to manage to create a magnet link and then publish it somewhere for others to see it.

  • Smoothy

    @ ws

    Here’s how you create a magnet link in utorrent – http://answers.yahoo.com/question/index?qid=20091117120318AAKusEz

  • Sam

    OK people, stop being so critical of TF. If you are tired of hearing about TPB on TorrentFreak, I am tired of hearing about this endless and pointless healthcare debate on CNN. And you can see in the anchors’ faces that they’re sick of it too, but it’s the news, so they have to report it. They also have to factcheck claims about reform to avoid being called biased.

    Lay off TorrentFreak. If you don’t want to hear another word about pirate bay, start your own blog. Call it “TorrentButNotTPBFreak” or “FOX News” or something like that.

  • HarryJoMinor

    Wow, I never would have figured that dude!

    Jess
    ultimate-privacy.br.tc

  • Anonymous

    Which to me, Smoothy, sounds a lot like what we already have. Some site will host magnet links which in MAFFIA eyes is assisting copywrong. Good question “ws”. Any answers out there from the “experts”?

  • ws

    @ Smoothy

    Thanks!

    Paste it somewhere else?

    I right clicked on a torrent as instructed and nothing happened. Sorry. (I must state I am using uTorrent in Wine on openSUSE 11.2 Linux)

    Back to my original question. How do I get it in my download list if it doesn’t have to be uploaded or indexed anywhere? How does anyone know I have a new torrent ready for upload?

  • ws

    Update2

    I added the OpenBitTorrent tracker to my unhosted torrent after an hour of seeding without a tracker, and still no peers have connected, so after 2 hours of seeding I’ve added the PublicBitTorrent tracker.

    Since both claim I don’t need to upload or index a torrent I should get some peers connecting soon! LOL!

  • Anon

    ws, what is the hash for your torrent?

  • Velocy

    Thanks all for explanations but I’m still a bit confused of how DHT technically works.

    One of you said it connects to the “network” and starts looking for peers which are hosting the same file you’d like to download… Okay… but what’s this “network” you are talking about ?
    You can’t just browse the entire Internet to find someone out there who has a file to share, it’s just technically impossible.

    I understand how PEX works once you found at least one peer. But how the hell can you find this first peer ?
    You still need to connect to a server and request for at least one peer, right ? For me this sounds like a little tracker anyway.

    So what’s the point ? And how the first peer is chosen amongst the whole list ? And anyway, is there a list of peers somewhere ?

    Still confused… thanks for next replies

  • ws

    @anon

    Sorry this is supposed to work without anyone knowing the torrent exists according to my understanding of what I have read so far.

    Still no peers btw.

  • Anon

    If no-one knows it exists, no-one can even look for your torrent. This is why you have no connects.

    This thread is explaining that a tracker is unnecessary (because the DHT finds peers) and the .torrent file can be downloaded directly from your BT client using the Magnet Link mechanism.

    But until you publish the hash, no-one know how to locate your torrent

  • filosofixit

    torrentfreak should add this to their spamfilter :

    ”
    (rt/jess/whatever)
    ultimate-privacy.br.tc
    “

  • leakz.net

    @ws

    no one will be able to know about ur torent if u dont publish it. U need to publish it on some bittorrent indexer. What DHT, PEX does is it compensates for the need of trackers. So anyone who only has ur torrent file(or gets it from a magnet link) will be able to download it even if there are no trackers that track that torrent.

    BTW, I dont think trackers shud be nuked. Afterall, that is how the indexing sites(and their users) know whether a torrent is active at all. Without trackers what are we(indexing site admins) to scrape from ??

    http://leakz.net

  • John

    @16
    omg, thats ALOT of bullshit in one page.

    Just read the section about https and you realize this guy has NO CLUE.

  • Ninja

    Trackers are needed at least in the beginning of the torrent life to ensure the health of the other methods.

    Maybe in the future the trackers won`t be needed but for now they can`t be disregarded.

    This magnet thing along with open trackers such as openbt are awesome and reduce the risks of losing any lawsuits dramatically. I mean, how can one be accused of infringing copyright if they only make information about files available? And how can you accuse a tracker of facilitating illegal transfers if they have no clue of what is being transferred?

    That will leave a single choice to the labels: go after the file sharers themselves. Which will be amusing since we were over 20M using TPB only. If you broaden the range of services and sites that number is probably over 100M. If you add e-mails and msn transfers then that number ges over the roof.

    It is getting late for the labels and other media companies to catch up with the future…

  • dude

    yeah… the tracker has been an unnecessary crutch for a while now. One that is just causing problems for those that operate them.

    Magnet Links are just for those who don’t want to have to fiddle around with all those torrent files. Bittorrent needs them. If everything was magnet links, bittorrent would collapse. Download the actual torrent is always best, but if some sites want to steer away from legal trouble, they may upload the torrents on more popular sites & just use magent links on their site..

  • ws

    @leakz.net and everyone else interested.

    Exactly my point. The torrent needs to be uploaded and indexed at a torrent hosting site.

    However, if you read my original post, both OpenBitTorrent and PublicBitTorrent claim, “You don’t need to register, upload or index a torrent anywhere, all you have to do is to include the tracker URL in your torrent.”, and the claim is the same for magnet links in my understanding and the site not the client you are using creates the magnet link.

  • Anon

    ws, you’re wrong. Post the hash for your torrent here, and people will start connecting to it to prove the point.

    At the moment, there’s no way to know WHICH torrent on OBT is yours. OBT is making your torrent available to anyone who can identify it, but ATM no-one can because you haven’t posted any way to identify it.

    (BTW, the hash is displayed on the ‘General’ tab in uT and you can right-click on it to copy it to the paste buffer).

  • Anon

    To clarify where you’re mistaken, what OBT & PBT are saying is that you don’t have to “register, upload or index [...]” in order for their trackers to do their function of tracking the torrent. You haven’t done any of those things and the trackers are working.

    However, you seem to be inferring that they’re saying that you’re done once you’ve got OBT to track your torrent.

    What I’m saying is there’s an additional step you have to take, which is to publicise your torrent. Until you do that, although your torrent is available, no-one can find it.

    OBT & PBT are not claiming that their service does the publicity bit (cos it doesn’t).

  • &peer_id=-AN0N1840

    @ws

    Again how would you know my torrent exists for you to download if I don’t upload or index it somewhere?

    You do indeed need to publish it in some way for others to know (a) that it exists, and (b) what it represents. All a tracker is ever aware of is info pertaining to a “info_hash” value that represents a calculated sum of the files contained with a torrent (example: C76631D43B3923D10620C088EEEF643154692728 is a calculated info_hash that represents a particular bundle of information) and some other rudimentary info like your IP:port, what client you’re running, the size of the data in the torrent, and so on. The tracker knows nothing descriptive about the content, not even file names, and does not publish information of its availiability anywhere… this much is up to you.

    You can make a tracker aware of any info_hash that you generate simply by contacting it via an /announce url, and all it will do (basically) is sit there keeping a note that you, at ip:port, are announcing C76631D43B3923D10620C088EEEF643154692728.

    Pass the .torrent to a friend and when they run it they also contact the tracker, then the tracker knows about 2 people announcing the same C76631D43B3923D10620C088EEEF643154692728, and passes your ip:ports to each other so you can contact each other directly to start negotiating file transfer.

    DHT does as much, but you don’t contact a central tracker as such, you join into a swarm of nodes that as a whole perform the same basic task: find other nodes announcing the same info_hash (with obvious benefits to resiliency if you have 100,000 highly distributed DHT nodes, versus a single point source tracker in some datacentre somewhere).

    …

    ***note: i’m a noob – if you know better, correct anything i’ve said here***

  • Anon

    And if you magnetise the hash like this

    magnet:?xt=urn:btih:C76631D43B3923D10620C088EEEF643154692728

    anyone can download your torrent of The Yes Men just by clicking it or pasting it into the Open URL box

  • consuminghatred

    I just have one question, and I hope someone will give me an answer.

    Lets, hypothetically assume, that all we ever use is DHT/PEX/LPD and that there existed some kind of indexer for those magnet links.

    How could I, as a downloader, know of the quality of the torrent (seed/peers) if there is no tracker to actually get that information?

    I would have to wait and see for myself when the torrent begins its download in my client?

    I would have to rely exclusively on comments from other peers/seeders that may or may not post something on the indexer site?

  • leakz.net

    ^^117 yes and thats exactly what my point is. If only magnet links are used than there is no way to know seeds/peers before u start the download.

    “ONLY magnet links” can NEVER be the ideal future for bittorrent.

    http://leakz.net

  • Anon

    Couldn’t you add a function to your indexer to enable it to scrape the DHT for Seed/Peer numbers?

  • Anonymous

    LPD(local peer discovery) is for LAN. You can turn it off. Correct me if LPD is good for anything.

  • The Bats

    Seems to me it would be simpler to go out and actually buy the shit.

  • Anonymous

    @117:

    That is called “reputation system” and yes you would have to trust other users comments.

    About the disponibility, DHT allow for it to be know over the bittorrent protocol if I’m not mistaken but is a bit different, you will get your info directly from others clients and will have to wait and see what they respond.

    Yah I know it adds a uncertainty to it but actually it works I tried and all magnet links to this date download without a problem. The annoying part of it is that if the magnet URI don’t have the name of the file it will show the hash as a name.

    magnet:?xt=urn:btih:7ZRGOWS3TU6NUFHVWWYMZ4N2ESJYNDQH&dn=LoL.some.video.name.here.LoL

    The Hash is bogus and the name too but is just to explain the name thing, if you were to download that it would show in your client as “LoL.some.video.name.here.LoL”

    If you want to know what “dn” means read here.
    http://en.wikipedia.org/wiki/Magnet_URI_scheme

    Besides magnet links are plain text you can see what you are getting and can even add trackers if you feel the need to, just copy&paste.

    It could be even more safer then a torrent file that you can’t see inside.

  • Phenom

    @92

    I’ve tried to keep it simple, and to take a step back from the technical details:

    The web page you search for torrents on is not a tracker. Mininova/TPB are not trackers, they are torrent search engines. Some of these torrent search engines (like TPB used to) ALSO run a tracker, but these are different things. You don’t download .torrent files from the Tracker.

    A Tracker keeps a list of peers that are looking for a specific hash (aka the swarm). The hash is what you talk to the tracker/other clients with and is a unique value that is calculated from the .torrent using a Hashing Algorithm.

    OpenBitTorrent is a tracker only, it just keeps track of peers looking for specific hashes.

    When your client talks to the tracker, it sends the hash its looking for and in return receives a list of addresses it can try to download from. When a tracker receives a request for a hash it doesn’t know about, it starts keeping track of it as well, and that’s how the swarm is built.

    DHT does the exact same thing as a tracker, only in a de-centralized way if you are trying to download.

    A Magnet link is just the hash of the torrent in a URL. To get this hash, the Torrent search engine still needs the .torrent file to calculate the hash from initially.

    If you download the .torrent from a site like Mininova, your client calculates the hash and then starts finding peers using trackers, DHT etc.

    If you use a magnet link, your client takes the hash from the magnet URL and tries to download the .torrent file from peers it finds using a tracker or DHT etc. Magnet links can also include the address of a tracker. You still need the .torrent, because that contains information about the actual files you are attempting to download. Once your client has the .torrent file, it continues just as it would if you had downloaded the .torrent from a site directly.

    This is an example of a magnet link I just made up:

    magnet:?xt=urn:btih:a123fcfad15e3f478&dn=MONKEYSAREAWESOME&tr=http%3A%2F%2Fdenis.stalker.h3q.com%3A6969%2Fannounce

    This one has three parts:

    xt – The torrent hash
    dn – Name of the torrent
    tr – Tracker URL

    For the casual downloader, Magnet links replace .torrent files.

  • cosy

    ppl who dont get this are retarded or dont understand english

  • ws

    I think my point has been made.

    You have to post something, somewhere for a torrent to be visible to others.

    Anon wants me to post the hash here, or I could just upload it the torrent file to TPB or another hosting site. Then you could click on the magnet link and copy and paste it into your client or download the torrent file and add it.

    Thanks for all the feedback!

  • ws

    @The Bats

    All of the stuff I download are current TV shows.

    Since I am too poor to afford cable, and wouldn’t want to pay for 500 channels I don’t want anyway, it is the only way I get to watch them.

    I do subscribe to Netflix and get older shows and movies through them.

    Thanks for your suggestion!

  • Anon

    ws, what you said was “I believe there is a fallacy in the statement that a torrent doesn’t have to be uploaded to a hosting site.”

    Which seems to have now changed to “if I create a torrent and nobody ever finds out about it, then nobody will ever download it”, which I think we could’ve guessed.

    If you had’ve posted the hash, then perhaps the difference would’ve been more obvious as people would now be downloading your torrent without you ever uploading your .torrent file to a hosting site.

    Perhaps the difference seems too trivial for you to notice, but it is, infact, the point of this thread: if hosting .torrent files on a website ever becomes legally untenable, then listing torrent titles along with their associated magnet links (or even just hashes) can achieve the same effect will less legal exposure for the indexer.

  • LtJackboot

    I personally am not interested in why it works or how it works just as long as there is something coming down the pipe to deal with the the changes in file sharing. I was there when Napster was young and every time the NetNazis have tried to stop us from getting content someone has come up with a new way. This latest thing is the same as Napster disappearing in favor of Gnutella and then Limewire etc. keep up the good fight fellow geeks.

  • baka pinkuu

    Several people have asked basically the same question and not gotten answers, but Velocy is the most recent one:

    One of you said it connects to the “network” and starts looking for peers which are hosting the same file you’d like to download… Okay… but what’s this “network” you are talking about ?
    You can’t just browse the entire Internet to find someone out there who has a file to share, it’s just technically impossible.

    I understand how PEX works once you found at least one peer. But how the hell can you find this first peer ?

    If you don’t want to say because giving out that info would help the MAFIAA jack@5535 to find people to pick on, please just say so and people will know to stop asking. But please don’t answer with another more-technical iteration of “You find peers because your computer looks for them,” because that’s a frustrating, circular answer. The question is “Where and how does it look for them?”

  • ws

    @anon

    Just where the hell am I supposed to list these torrent titles along with their associated magnet links (which nobody has adequately explained how to create) or even just hashes?

    Sounds to me it’s all going to have to be private.

    When I click on create new TORRENT in uTorrent I am creating a new torrent not a magnet link, the same in Vuze, Transmission or any other client. Then I upload it to a hosting or indexing site, which seems to be adding the magnet link.

    Also, when I use a magnet link the .torrent file gets saved in the Store .torrents in directory I have selected for that purpose.

    And the answer to my question isn’t you can post it here. LOL!

  • Anon

    OK, when you make the torrent in uT, look at the General tab, and you’ll see the Hash listed. You can right-click copy it to the clipboard to make it easy to paste into wherever you want it to be.

    To make a magnet link, just prepend

    magnet:?xt=urn:btih:

    to the hash like I did in #116. I foolishly assumed TF would recognise it as a URL and clickify it automatically, so let’s do that manually:

    <a href=”magnet:?xt=urn:btih:C76631D43B3923D10620C088EEEF643154692728″>Click here!</a>

    gets you

    Click here!

    And now anyone who clicks that link will start downloading the YesMen torrent.

    I seem to recall that Vuze has a ‘copy Magnet URL’ function, to automatically make the link, but doesn’t that take all the fun out of it?

    Paolo posted lots of ideas for what you could do with a magnet link that you couldn’t do with a .torrent file in post #20

    The only real problem you seem to have at the moment is that sites like TPB (the ones that people go to to search for torrents ATM) are set up to accept .torrent files, which makes magnets a bit redundant. However, as Pavel pointed out in #28, this might not always be the case, and TF is explaining one of the possible fall-backs if that actually happens.

    Of course you don’t have to post a hash or magnet link here, but I was rather hoping that with a bit of prompting, you might be able to help TF’s explanation of how magnets work by actually working through the process in this thread.

  • Anon

    Hmm, the link hasn’t worked. It seems TF’s engine eats the magnet prefix.

    The code I posted would work on a site that doesn’t do that (I think).

  • Soundwave (Have A Cigar)

    #120

    “LPD(local peer discovery) is for LAN. You can turn it off. Correct me if LPD is good for anything.”

    Exactly. For sharing with people on your network. This is useless for most people.

  • Soundwave (Have A Cigar)

    ^^Not to be confused with DHT or PEX.

  • consuminghatred

    I thankful for the reply, but I think one of the biggest problems of this system is the lack of availability of information about the the health of the torrent.

    The seed/peer ratio, to me, at least. Is a DEFINITIVE reason why I choose one torrent from another.

    Comments, of course, also indicate the risk (if they found a virus/trojan) or the quality (usually in video downloads) but the seed/peer ratio is still the most important and having to download the torrent via magnet links, waiting for the client to load and then looking seems to be a long and unnecessary process making it much more difficult to weed out “unhealthy” torrents (regarding download speed and availability)

  • ws

    @anon

    “OK, when you make the torrent in uT, look at the General tab, and you’ll see the Hash listed. You can right-click copy it to the clipboard to make it easy to paste into wherever you want it to be.”

    Well according to Paolo in his #20 post wherever I want it to be could be a lot of places, I currently don’t want to be.

    However the steganography idea kicks ass! Still I’d have to have a site where everyone interested would want to go to see if I had a new torrent or not, and then decode the steganography.

    Thanks for explaining how to create a magnet link though!

  • FU, TF

    @baka pinkuu

    In µTorrent, if it’s a new install, or doesn’t yet have a ‘dht.dat’ file generated where it stores a cache of previously known DHT contacts, it contacts a server at ‘router·utorrent·com’ which in turn responds with addresses of active DHT nodes for the client to connect to.

    µT stores these node addresses locally in a ‘dht.dat’ file it generates for later use, so on next startup an attempt to establish new links to the DHT network shouldn’t require any further contact with a central entity, like that at ‘router·utorrent·com’.

  • Mr Scared

    OK this stuff just makes me confused.
    I don’t see how you can find other users without being connected to a centralized server?

    Or does it make the peers into dht servers?

    Browse safe. https://p-r-o-x-y.in

  • ws

    OK!

    I just created a magnet link in uTorrent, but first I had to create the torrent, then I right-clicked the torrent file (not the hash) and selected Copy Magnet URI.

    What is confusing there, is that it doesn’t indicate that anything happened, until I opened a text editor and pasted the link into the editor.

    So, I see how this could be used if there was somewhere to paste the link.

  • Anon

    Mr Scared, there aren’t DHT servers and clients, it’s a peer-to-peer system just like bittorrent. The server FU was talking about just gets you into the DHT network if you don’t know of ANY other peers.

    ws, sorry I missed that option hiding in plain sight. It’s a bit easier than magnetising the hash.

  • Pharaoh

    So why bother with public BT at all if it’s basically just ED2k now?

  • Bucketmonkey

    There’s a dead rodent stapled to my testicles unfortunately. Send help please, my nuts are composing.

  • Anonymous

    You don’t have to use TPB. There are other trackers and sites. TPB can stick it up their butt.

  • baka pinkuu

    @FU, TF
    Thank you for the answer, it was perfect. (^_^)

    @Bucketmonkey Send help please, my nuts are composing.
    (‘o’)
    Over the course of my life, I’ve heard hundreds of jokes about men thinking with their genitals.
    But I have to hand it to you… that’s the first time I’ve ever heard of a man writing music with them. I look forward to hearing your the result.

  • Just as confused

    I don’t think anyone on here has gotten the gist of what ws is asking. I believe what he is asking is this, in a normal BT upload you create a .torrent file and then upload that file to TPB, Mininova, IsoHunt, etc. Now, if you create a magnet link, on what site do you post that link so others can use it to access your torrent?

    In other words, is there a MagnetBay, or MiniMag, MagHunt, etc., web site to search for magnet links? Correct me if I’m wrong ws.

  • Just as confused

    @Mr Honeypot/Scared

    From the home page of p-r-o-x-y.in

    Please do not use this site for illegal activities. All connections are logged.

    Stop posting your obvious attempts to lure unsuspecting people into a setup.

    We look after our own, we will always be smarter than you, and we will prevail! Damn trolls!

  • Physuss

    @145 Just as confused. That is the beautiful thing about magnet links. (see #20 Paolo)
    You could write on a wall/t-shirt the Hash(Hex) and a short description of what the files are, and we could get them.

    Eg. I just ripped the FLAC/MP3 of a new album that everybody wants. I create a torrent, and it gets a Hash. I go on ANY website. and post a comment along the lines of “New Album by Artist in FLAC. Hash= xxxxxxxxxxxxxxxxxxxxxxxxxxx”. All YOU would have to do is G009le “new album by Artist” and my comment on the random site would pop up. You now have the uniqe name of the files. With DHT and PEX that is all you need to find me seeding the torrent. (Corrections ?)

  • Smoothy

    @ Pharaoh

    >So why bother with public BT at all if it’s basically just ED2k now?

    The difference is that each individual is sharing a fewer select number of files (rather than their entire library) so downloads can be a lot faster.

  • Smoothy

    @ Pharaoh

    Also I’d like to add that Ed2k still has its place as a long term repository for files. Its excellent for rare, old material that can’t be downloaded through torrents. Bittorrent is more built for distributing new content. Saying that though I’ve seen old (public) torrents that are a good couple of years old that are still healthy and well seeded.

  • Anonymous

    Operation: Internet Crusade starts today, November 22, 2009 at 9:00:00 PM (UTC time)

    We are Anonymous.
    We are Legion.
    We do not Forgive.
    We do not Forget.
    Expect us.

  • Anonymous

    to get the magnet link of a torrent just simply right click on it in the torrent list(utorrent) and select copy magnet uri. Now you can right click and paste it anywhere!

  • …

    Ok i get the whole idea with DHT, PEX and magnet links. But when i look at the magnet-URI format:

    magnet:?xt=urn:btih:&dn=&tr=

    i see that it still needs and tracker-url, can someone explain this to me, as it was stated above that the big pro about this technology is that its nog using a tracker anymore?

  • Anon

    @…

    Try to think of this as three separate problems:

    Identifying the torrent you want

    Acquiring the .torrent file

    Locating other peers in the swarm for the torrent

    Magnet links only address the second problem; they allow you to acquire the .torrent file from the swarm rather than a central host.

    How you get into the swarm once you’ve got the .torrent file is a separate issue.

    At the moment, trackers are a very good way to join a swarm, but if running BT trackers becomes problematic (eg for legal reasons, not that we’re close to that ATM), then the Distributed Hash Table provides the same functions as a tracker, but, again, without using a central host.

    The issues of how you advertise your torrent so people can find it are not addressed by the DHT, Peer Exchange or Magnet Links.

  • Brian

    I couldn’t give a monkey’s if the tracker’s down just as long as I can still be guaranteed a good idea of the seed/peer ratio.

  • Bad Mojo

    @152
    … you are absolutely right.

    Check out Firefox add-on magnetize.
    It does just this and adds the trackers you choose. It comes canned with all the open trackers but allows you to add any others. Currently it works with Torrentz and Isohunt. It extracts the hash from the http source. Then utorrent does the rest.

    I am unsure how any of this would help an uploader. But it makes downloading pretty transparent.

  • Anonymous

    @152:

    The tracker field on a magnet link is optional you can have it or not if you use DHT.

    Besides I see a lot of people saying that a magnet link is for the torrent file. I have my doubts about that as I understood the thing it doesn’t depend on a torrent file for downloading anything it just needs a hash of the file to be shared to ask other clients. But I could be wrong if bittorrent does it differently from others.

    The only filed really needed on a magnet link is the “xt” field that have the file hash.

  • Anonymous

    A quote from wikipedia.

    “The most common use of magnet links is to link to a particular file based on a hash of its contents, producing a unique identifier for the file, similar to an ISBN or catalog number. Unlike traditional identifiers, however, content-based signatures can be generated by anyone who already has the file, and so do not need a central authority to issue them.”

  • Anon

    @Anonymous #156

    As the original post states “BitTorrent can not ditch the .torrent format entirely and rely solely on Magnet links”. The .torrent file contains the hashes for the all pieces in the torrent, and you need that information to check that the pieces have been received without corruption.

    Why don’t you try it? :)

    You’ll see your BT client use the magnet link to download the .torrent file (which will appear in \Documents and Settings\Username\Application Data\uTorrent – Application Data is a hidden folder).

    It will then use the .torrent file to download the data exactly the samy way as if you’d downloaded the .torrent file with a web browser.

  • Smoothy

    @156

    Yes, you need a torrent file as it contains the information necessary to correctly download the file (bittorrent splits a file(s) into numerous small pieces and the .torrent file contains the hashes of these pieces) . With the DHT/magnet-link system you don’t need to host the torrent file (publishing the magnet link will do) and you don’t need the torrent file to specify a tracker (as DHT takes over the job of the tracker). That’s how I understand it anyway. I’m quite confident that the DHT/magnet-link system can replace trackers and the need to host torrent files.

  • Anonymous

    @158 Nov 22, 2009 at 15:55 by Anon

    Hmmm…I don’t use windows so your path is probably wrong still, what you are saying is probably true to some magnet links already on the net.

    Which defeats the purpose of having magnet links in the first place.

    But I will play with you because I will assume I’m wrong and need some more information to have a clear picture.

    I just used EZTV to test what you said and configured Deluge to copy any torrent file to the desktop(of Fedora) and added the magnet link for merlin.

    magnet:?xt=urn:btih:XA6TEKQOYEF2DD6MNR7JF5R75RP3W2TA&dn=Merlin.2008.S02E09.WS.PDTV.XviD-RiVER&tr=http://tracker.openbittorrent.com/announce

    It didn’t show any torrent file on the desktop yet still the download already started.

    And to make sure that any torrents added would have been put on the desktop I downloaded another torrent and it did show up.

    I’m still looking for the place deluge store the torrents but I didn’t find it yet.

    So for now I have to say that what you are saying is not proved.

    But I don’t know if Deluge store torrents from magnet links in different places or if just don’t download anything.

  • ws

    Ok, It looks like it is agreed that you still need to publish (host) the magnet link somewhere so those interested in the file can find it.

    So when all the torrent hosting and indexing sites are closed because everyone is using magnet links and publishing them on their t-shirts it is a win for the industries.

  • Anonymous

    Well you were right it does have a torrent there with the hash.

    b83d322a0ec10ba18fcc6c7e92f63fec5fbb6a60

    Deluge stores in linux their torrents in /home/[user name]/.config/deluge/state

  • no help at all yet

    Is there a site I can search for magnet links?

  • Anonymous

    @163 Nov 22, 2009 at 16:48 by no help at all yet:

    The sites you already use may have them. For other protocols there is like gnuntella.

    Here what I found on google(never tested):

    http://magnet-uri.sourceforge.net/

    search in the page for “Some websites that publish MAGNET URIs:”

  • Phenom

    @160

    Or you can just not be an idiot and read what has been repeated multiple times.

    The first thing your client does is download the .torrent file after it starts downloading with the magnet link. Where your client stores this .torrent file depends entirely on the client you are using.

    This removes the need for .torrent files to be hosted, since you can get it from seeds or other peers (which all have the .torrent)

    @163

    eztv.it, thepiratebay… pretty much every major torrent site has magnet links now as an alternative to downloading the .torrent file.

  • Anon

    The Pirate Bay have recently added Magnet links to their listings.

    Other sites, eg ISOHunt, don’t post magnet links, but they do display the hash. You can turn the hash into a magnet link by putting “magnet:?xt=urn:btih:” infront of the hash when you paste it into your BT client’s Open URL box.

    There’s even a Firefox plugin that will convert the hash to a magnet link for you.

  • Anonymous

    @165 Nov 22, 2009 at 17:00 by Phenom:

    I like to be an idiot :)
    I don’t trust people, they are not reliable, people make mistakes, people lie, people get emotional about things and try to protect their ideas like it was their first born so no I can’t just accept what people say.

    I however can test and ask questions and if that bother you I’m sorry.

  • Dustin Wirght

    Wow, sounds pretty mixed up to me dude, I mean really.

    Jess
    ultimate-privacy.br.tc

  • ws

    @anon

    That Firefox plugin is experimental according to the add-on site and they state:

    Experimental add-ons are newer add-ons which have not yet undergone our public review process. Many of these add-ons may be in prototype form. Since they are untested by our editorial team, they may be alpha, beta or pre-production in quality, performance and features.

    Caution should be used when installing experimental add-ons, as they have not been tested by an editor and may harm your computer configuration.

    It claims: Generates magnet links for torrent meta-indexers, letting you pull the torrent out of the swarm without visiting the website that hosts it.

    I still want to know how I can do that if I don’t know the torrent exists? There are a lot of swarms going on at this moment and I want to pull a torrent out of one of them. How?

  • Anonymous

    @169 Nov 22, 2009 at 17:19 by ws:

    I think I can answer that base on my limited knowledge of the bittorrent protocol.

    To make a magnet link apparently you have to:

    - Create a torrent file.
    - Hash the torrent file.
    - Publish the magnet link somewhere.

    The magnet link contains a hash that uniquely identifies a torrent file so your client will ask other clients if they have them and will download that file.

    An analogy is:

    A library, you get an ISBN number(hash) go to a library and ask for the librarian(your peer) if he has that book(file).

    That helps you?
    Hope so because I don’t know how to dumb it down more.

  • no help at all yet

    @Phenom and #166 Anon

    Thank you, finally, that is what I’ve been trying to get out of all these techno-babblers on here. A straight answer.

    Ok, so next idiot question is this, how is hosting a magnet link different from a torrent file on a central site like TPB, EZTV… in the eyes of the MAFFIA? To them, they are still providing an easy way to access copywronged files.

  • no help at all yet

    @Anonymous Insulter #170

    And you were born a genius or were you just dropped on your head? Repeatedly!

  • Anonymous

    @169 Nov 22, 2009 at 17:19 by ws:

    Let me correct myself here.

    In the part “hash the torrent file.” it may be “Extract the hash from the torrent file”, because there is a hash inside the torrent file the is used as an identifier for it.

  • no help at all yet

    @Anonymous Insulter #170
    So dipshit, where does ‘ws’ need to publish his magnet link so a loser/leech like you can download and not seed it back?

  • Anon

    ws, if your complaint is that this isn’t a completely mature technology yet, I infer that Ernesto agrees with you as he titled this thread “BitTorrents Future” with a question mark :)

    I certainly agree that the way BT clients handle magnets needs a bit of work (uTorrent doesn’t even let you paste a raw hash into a box, you have to add the magnet:?… stuff) and the indexers are still oriented towards .torrent files rather than magnets.

    I think the intent of this thread is more of an FYI explaining how up-and-coming technologies solve problems that don’t yet exist. People who are currently happy uploading/downloading .torrents on indexers should keep doing that, but if indexers start to move to posting magnet links instead of hosting .torrents, hopefully people now know what that means (basically an extra minute waiting for the torrent to start up).

    @no help
    Pavel at #28 explains his experience that there really is a difference, legally, between posting magnets and hosting .torrents, even if it’s not really obvious why.

    And for example AnimeSuki posts magnet links, but does NOT host any .torrent files at all (they link to .torrents hosted by the fansubbing groups). I don’t know if they do that because they don’t want the technical burden of hosting the files, or for legal reasons, but there does seem to be real-world evidence that magnet links are not the same as offering .torrent files for download, even if they achieve the same effect.

  • Anonymouse

    rtorrent also can DHT.

  • ws

    @170 Nov 22, 2009 at 17:31 by Anonymous

    I know all of that thank you.

    Read my #169 post again. I still want to know how I can pull a torrent out of a swarm, as the add-on is claiming I can do, without visiting a site, and knowing the torrent exists?

    For example you create a torrent and publish the magnet link somewhere. I don’t know it exists but want to pull it out of the swarm, since I don’t have to go to a website and know the thing exists.

    I have to go somewhere and get the magnet link so I can add it to my client, and then it will ask other clients if they have the file so I can download it.

    Whether a .torrent file or a magnet link it has to be published (hosted). The nice feature as pointed out is that magnet links can be hosted anywhere. I nominate sites of the MAFFIA.

  • no help at all yet

    @ws
    I think the best answer so far came from Phenom and #166 Anon. You still have to go to TPB, Mininova, EZTV… to post and search for magnet links. I see no real improvement in plausible deniability against the MAFFIA.

  • Anon

    @173 “it may be Extract the hash from the torrent file”, because there is a hash inside the torrent file the is used as an identifier for it.”

    No, the hash is created by running the hashing algorithm against the .torrent file. If you were to then put that hash into the .torrent file, the new file would have a different hash.

  • Anon

    @ #4 You are correct TPB shut down tracker b/c of the court cases

  • ws

    I was thinking about that library analogy where,

    “An analogy is:

    A library, you get an ISBN number(hash) go to a library and ask for the librarian(your peer) if he has that book(file).”

    You know the library provides copy-written material for free. Sure they paid for the items they provide, but how many millions have the publishers lost to the users of the library. My library provides not only books, but movies and music.

    So I need to know Sarah Palin’s book EXISTS, so I can go to the library, LOOK UP the ISBN(hash) and ask the librarian(my peer) if they have the book(file).

    Torrent sites are just libraries!

  • ws

    Um sorry!

    I meant torrent sites are the card catalogue.

  • #170

    @176 Nov 22, 2009 at 18:05 by ws

    That add on if I’m not mistaken search for hashs in websites and transform them into magnet links.

    It does not make them out of thin air.

    Here:
    http://www.rohitab.com/discuss/index.php?showtopic=35291

    I think is the developer explaining what it does.

    Your question is:
    How does a magnet link finds a torrent file in a swarm?

    It get those from its peers and not from a website.

    Still the magnet link have to be somewhere to be found.

    @178 Nov 22, 2009 at 18:27 by Anon
    Please do the following test and see for yourself why I changed my statement.

    - Get a torrent that you download and find out the info hash stored in it.

    - Make a magnet link with it and wait until it downloads.

    It works here.

  • Anon

    “- Get a torrent that you download and find out the info hash stored in it.”

    How do you do that?

    If your answer is “read it off the bittorrent client’s info screen”, consider the possibility that the client is calculating the hash by digesting the .torrent file, not reading the hash value from inside the file

  • Smoothy

    @177

    You make a good point. But it adds another layer to the mix (magnet-link -> torrent) which the legal departments of the ‘MAFFIA’ have to traverse. How can posting a simple, short, incomprehensible line of text somewhere on the net really be equivalent to aiding copyright infringement?

  • Anon

    And hashes have a ‘legitimate’ function; integrity checking (You’ll see all the Linux distro sites post SHA1 or MD5 hashes for all their releases).

    I think it’s difficult to argue those hashes are legal, but the BT hashes are always illegal.

  • Anonymous

    @182 Nov 22, 2009 at 19:26 by Anon

    Yep! that is a possibility and until I can get the “torrent meta info viewer” functioning I have no way of knowing that for sure except for the wikipedia article that states.

    BTIH (BitTorrent Info Hash)

    Hash sum used on BitTorrent.

    xt=urn:btih:[ BitTorrent Info Hash (Hex) ]

    And I also tried hashing the torrent files and comparing them with SHA1 and MD5 and did not find a match with what I collected so if you be so kind to check it out and tell us how it did go it would be great to dispel any doubts.

    Programs you can use to get the hash info from a torrent file:

    - GTorrentViewer(linux)
    - Transmissioncli
    transmissioncli -i .torrent
    - Deluge also shows the metadata info.
    - MakeTorrent 2.1(win) Don’t know if it works.

    :)

    Cheers

  • Anonymous

    Hmmm…here the MAN(manual) pages for the transmissioncli explaining the “-i” switch in detail and what it does.

    http://linux.die.net/man/1/transmissioncli

  • ws

    @181 Nov 22, 2009 at 19:19 by #170

    What part of this statement don’t I (or you) understand?

    Generates magnet links for torrent meta-indexers, “letting you pull the torrent out of the swarm without visiting the website that hosts it.”

    I read it as I can pull a hat out of my derrière without going to the store and buying that hat in the first place.

    I have to visit some website(store aka “meta-indexer”) to get the magnet link, therefore I have to know someone has posted a link to a file I want.

  • Anon

    Hmm, I don’t think I can run any of those programs just at the moment.

    I’m not really arguing with you, I just think that the way that BT calculates the hash from the torrent and then uses it as a unique key for identifying the torrent to trackers etc is really much cleverer than you’e giving it credit for.

    The main point is that to put the hash into the .torrent, you’d have to change the file, which would change the hash.

    There are hashes in a .torrent file, but they’re for the pieces that will be downloaded. I would think that’s what GTorrentViewer is showing you (and if it displays the .torrent hash too, it calculates it by digesting the .torrent)

  • Anon

    “-i, –info
    Shows information from the specified torrent file, such as the cryptographic hash, the tracker, announcement, file size and file name.”

    I suspect that’s poorly phrased. The tracker list is not in the .torrent either.

    So although what that actually says does support your assertion, I think that should read ‘information about’ rather than ‘information from’, which is really a rather subtle difference.

  • #170

    @182 Nov 22, 2009 at 19:26 by Anon

    Ok! now I double checked and if you get the info-hash from inside of the torrent it will start a download.

    I used GTorrentViewer to extract the info-hash and build a magnet link.

    It is running here right now.

    here a source on how the info-hash is made:
    http://lists.ibiblio.org/pipermail/bittorrent/2005-October/001676.html

  • Anon

    ws, you always seem within millimeters of grasping this and then it slips through your fingers.

    No-one’s ever argued you don’t need some way of finding out that the torrent exists and what its hash is. As we said, Paolo covered this nicely at #20. There is a slight problem ATM in that the main indexers are set up for .torrents, not magnet links, but that’s a separate issue.

    Consider a site like AnimeSuki, which publishes (HTTP) links, but not .torrent files. In future, should the links to the .torrent files become a problem, they could switch to listing magnet links which, according to Pavel #28 would be OK.

  • Anon

    #170, right so we’re both half-right.

    When the .torrent is made, some kind of key is made, presumably by digesting the pieces’ hash list and stored in the header of the .torrent file.

    To make the BT hash, you then digest that key with SHA1, but that hash is NOT stored in the .torrent file.

    Seems an odd way to do it, but it probably reduces the load on indexers etc as digesting the key would be much faster than digesting the whole file.

  • A non mouse

    @Smoothy
    Would there not be a need of at the very least a description of what the mag-link is for? Your not going to just randomly click on a mag-link and hope you get something good are you? I’m not trying to be an ass here but all the technical discussion has gone way beyond what is needed here. All most BT users want is an efficient way to search for and down/up load the files they want. They don’t care about all the techno-babble being spewed out here.

    @ws
    Just give up. They will never listen. They are too busy flagellating each other with their superior knowledge. I believe the answer is you still have to go to sites like TPB, Isohunt… to up/download the mag-link for a torrent.

  • Physuss

    @184 Anon Quite right! The integrity checking function of the Hash (Magnetlink) also provides a robust way of insuring you are getting the actual file you wanted. malware is always a concern.

    To the analogy:think of it like this, A swarm is many people (seeders) who might have the whole book you want or just a few pages. You need the table of contents (.torrent file) of that book to know if you have copied all the pages, or are still missing some. But because of fear of getting sued for hosting ONLY the table of contents and a list of people who have the book, some library(tracker sites) are now trying to only have an encoded version of the NAME of the table of contents (Magnetlink in Hash form). Magnetlink equals us announcing “Hey, anybody have the Table of contents (.torrent) to this book I want?” DHT lets you copy the table of contents from others. You then photocopy the pages they have. PEX now lets you run around and photocopy the pages that their friends have.

    its P2P in a very true sense.

  • #170

    @187 Nov 22, 2009 at 19:57 by ws:

    Well I can see the confusing part now(I think).

    Torrent meta-indexers = search sites that gather information from other sites. One of those was “torrentspy” I think. It got their information from other sites but did not had a server or anything hosting the torrents files.

    So what the guy said in his website is true you don’t need to go to the website hosting the torrent file because the search engine has the hash and will create a link for ya to get the torrent from your peers.

    The “fail” part maybe is the “meta-indexer” part he could have used “torrent search engines” or something else more understandable.

    Or have phrased in another form. It is descriptive and technically true but only to people who understand how servers and websites work and that is a fail LoL

    Hope this help you understand.

  • #170

    @191 Nov 22, 2009 at 20:20 by Anon:

    We both learned something new today :)

    yay!

  • ws

    @190 Nov 22, 2009 at 20:12 by Anon

    I grasp it completely, and I will reiterate both OpenBitTorrent and PublicBitTorrent claim:

    They are a bittorrent tracker free for anyone to use. You DON’T NEED to register, upload or index a torrent anywhere, all you have to do is to include the tracker URL in your torrent, and I’ve seen several places that state a magnet link lets you pull the torrent out of the swarm without visiting the website that hosts it.

    You have to register at a hosting site, upload and index the torrent. The hosting or indexing site appears to be adding the magnet link.

    So if I want XYZ movie, I have to go to a hosting site(or use my rss feed), either go to my favorite user I know is going to provide the torrent or do a search, then download the torrent or magnet link, load it into my client and wait for seeds and peers to connect until I have XYZ movie.

    @192 Nov 22, 2009 at 20:23 by A non mouse

    That’s the point I’ve been making all along. That it has to be hosted somewhere. You can’t just pull it out of the swarm without knowing it exists as has been claimed.

  • Anon

    ws: “You can’t just pull it out of the swarm without knowing it exists as has been claimed.”

    Where has that been claimed?

  • ws

    So anyone want to download my experiment? Here is the magnet link to a small text file called experiment.

    magnet:?xt=urn:btih:HFHPPEPUOYCFWJSJWFJEVVZS4VZLPIMK

  • Anon

    “experiment” DL’d successfully :)

  • ws

    @197 Nov 22, 2009 at 20:59 by Anon

    It is claimed here at the Magnetiser add-on site.

    https://addons.mozilla.org/en-US/firefox/addon/47391

    or can’t I read?

  • Anon

    It reads:

    “This is a test of the magnet link system. It is only a test.
    DO NOT PANIC!”

  • Anon

    ws: Eww, their phrasing is horrible. “Without visiting the website that hosts it” is very ambiguous. For a start, as we just demonstrated, no website has to host anything. For your experiment torrent, there isn’t a website to not visit.

    “Host” means something very specific: to make a data set available for download. So although TF now ‘hosts’ the magnet link for ‘experiment’, it doesn’t ‘host’ the torrent file, which you’ve never uploaded to any website.

  • ws

    @199 Nov 22, 2009 at 21:04 by Anon

    Cool, now all I need is a place to HOST my magnet links and I’ll never upload a torrent file again!

    Did anyone pull my experiment out of the swarm while Anon was downloading it? Curious minds want to know. :>)

  • #170

    @196 Nov 22, 2009 at 20:49 by ws:

    They are a bittorrent tracker free for anyone to use. You DON’T NEED to register, upload or index a torrent anywhere, all you have to do is to include the tracker URL in your torrent, and I’ve seen several places that state a magnet link lets you pull the torrent out of the swarm without visiting the website that hosts it.

    And that statement is completely true you don’t need to UPLOAD, REGISTER, INDEX or anything because the HOST for the TORRENT FILE is your peers(swarm). Your bittorrent client is downloading the TORRENT FILE from your peers not a tracker or hosting service or website or whatever the TORRENT FILE is coming from your PEERS not the original HOSTING SITE or a WEBSITE.

    They are not saying you will not have to search for the magnet link, you will still have to go somewhere to find those links and probably those places will be sites like mininova, the pirate bay and others.

    Do the test:

    - Download Big Buck Bunny
    It’s a CC 3.0 license so as long as you give credits you can distributed and use it.

    - Make a torrent out of it.
    - Get the info-hash that your bittorrent client shows you.
    - Make a magnet link.
    - Send the link to someone via email and ask them to download the film.

    If you do that you don’t need to register anywhere, you don’t upload the torrent file to anywhere(except other peers), you don’t index the torrent because there are no trackers. All they said is true isn’t?

    I don’t see where you saw written or implied that it is going to pull something that nobody knows exists.

    ps: All caps was used to emphases only not shouting.

  • A non mouse

    @ws
    I think the answer is “42″!;-}

  • Smoothy

    @Ws

    Just downloaded your ‘experiment’ and am seeding it now :P

  • ws

    @204 Nov 22, 2009 at 21:18 by A non mouse

    “42″ will host magnet links, or is that the number from which all meaning can be derived? LOL!

    Let’s not deteriorate the discussion anymore than necessary. I think it’s pretty played out anyway.

    It was very interesting. Thanks to all that contributed!

  • Smoothy

    @ A non mouse

    >Would there not be a need of at the very least a description of what the mag-link is for? Your not going to just randomly click on a mag-link and hope you get something good are you?

    Yes, you do. Mininova does this already.

  • ws

    @205 Nov 22, 2009 at 21:27 by Smoothy

    Ok, one last post. LOL!

    Interesting, in that my client is only showing 1 seed, and 4 peers. DHT shows 1 seed and 1 peer, PEX shows no seeds and 2 peers.

    BTW the magnet is not the torrents hash.

  • Anon

    “BTW the magnet is not the torrents hash.”

    magnet:?xt=urn:btih:394EF791F476045B2649B1524AD732E572B7A18A

    There’s more than one way to construct a magnet link from the hash.

  • Smoothy

    @ Ws

    At my end DHT is showing 2 seeds. And PEX is showing 2 peers as well.

  • ws

    @209 Nov 22, 2009 at 21:36 by Anon

    So true, as you have stated before.

    However I did that right click on the torrent in utorrent and selected Copy Magnet URI, which leads me conclude that utorrent already constructed the magnet link. You can’t copy something that isn’t created, can you?

    @smoothy

    I guess my client was a little slow catching up as it’s showing that now.

  • Anon

    Oh, neither is ‘right’ or ‘wrong’, it’s just a different way of encoding the same number. ‘Your’ way is slightly shorter, ‘my’ way is what the Magnetiser plugin does – just prepend a header to the hexadecimal hash value.

    I was responding to Smooth’s observation that the Magnet is not the hash, which sort of contradicts all the posts I’ve made saying it is the hash.

    It is the hash, but it might not look like the hash if you use the newer encoding.

    The magnet link can’t encode the link as a raw binary number as the web servers are expecting ASCII coded text – some of the 256 possible values of the raw bytes are reserved for print control purposes and can’t be printed as characters in a consistent way. A hexadecimal number will always print correctly ‘cos it’s just 0-9 A-F with no special characters.

  • ws

    @207 Nov 22, 2009 at 21:17 by #170

    I missed this post somewhere, in all the reading.

    I want to provide the file, not download one, thank you. I’m still reading that as: if I use their trackers in my torrent, that I want others to find, I don’t need to register, upload, host or index the damn thing anywhere.

    I’ve already created torrents with those trackers and didn’t register, upload or index them anywhere as an experiment, as it states, and no one connected to them.

    Magnet links are the way to go! Just need some central place to publish them. Why don’t you start a web site for us? Thanks! I’m ready to start!

  • A non mouse

    @ws
    You didn’t miss the post, TF just takes too long to “moderate” long comments on here sometimes as I never saw the post either. It then pops up in the middle of a comment stream and if you don’t go back and look you might miss it. Very annoying at times.

    To the TorrentFreak moderators, if you would at least leave a placeholder of commenter and post number there would be a LOT less confusion on some of these postings. Just a suggestion.

  • Anonymous

    @ws:

    Didn’t people not download your TXT or they are imagining something?

    Are going to keep playing with people?

  • Anon

    ws’s torrent worked just fine.

    I gave up too quickly on the ‘the hash isn’t in the .torrent’ thing, though.

    Comparing the description in #170′s link: ” d8:announce…info[d6:length...e]e – Compute the SHA1 hash of the bytes between the brackets.” with some actual .torrent files reveals that the bytes between the brackets is, infact, pretty much the whole file. There’s just a little bit in the header outside the the part that’s digested for the hash so you can add trackers to the .torrent without changing the hash.

    Therefore the hash isn’t inside the .torrent at all. The hash is calculated by digesting almost the whole .torrent file.

  • ws

    @219 Nov 22, 2009 at 23:46 by Anonymous

    They did! Here is the link if anyone else wants to try it.

    magnet:?xt=urn:btih:HFHPPEPUOYCFWJSJWFJEVVZS4VZLPIMK

    Let me know if it works in Vuze. I can’t get that client to work with any magnet link. uTorrent works fine.

    Didn’t realize I was playing with people. Just thought I was pointing out false statements. Last I’ll post on the subject.

    I know what I’m doing and why I’m asking the questions. I’ve been uploading torents to TPB for a couple years now, stopped using TPB’s tracker long before eztv did.

    Why you keep playing with me?

  • Anon

    Weren’t earlier versions of Vuze only capable of reading the hexadecimal Magnet links (the format I posted)?

    And also, of course, Vuze has its own implementation of the DHT, so it’s not obvious to me how it would find my uTorrent host, or whether it can ask for the .torrent file to be transferred.

    I’m guessing it would work if you have the current version of Vuze with the Mainline DHT plug-in.

    I’ll leave my client seeding your experiment torrent for the next few days so you (or anyone else) can experiement on it.

  • Anon

    “Weren’t earlier versions of Vuze only capable of reading the hexadecimal Magnet links (the format I posted)?”

    Nope, according to this very article (hmm, that seems such a long time ago), I’ve got it backwards, hexadecimal is the new format. That seems odd because base32 coding is shorter and just sticking a header on the hex hash seems an obvious thing to do, whereas base32 coding is a bit obscure.

  • Pingback: Novell Strips BitTorrent DHT Technology from openSUSE | We R Pirates

  • Pingback: Novell Strips BitTorrent DHT Technology from openSUSE – FUCK THE RIAA

  • dralezero

    Basically with DHT your computer becomes a tracker, and everyone else using DHT is a tracker. Your own computers are communicating the lists of Peers instead of communicating with a central tracker that has the list.

  • Jeff

    Earlier versions of Azureus/Vuze were only capable of handling the base32 encoded magnet links. The ability to parse the hex ones was added with version 4.3.0.2.

    Incidentally, TPB torrents will initially appear to go to the now closed TPB tracker, but will then redirect to Openbittorent.

  • Pingback: Novell Strips BitTorrent DHT Technology from openSUSE | InstantIdiocy

  • Pingback: ????????? ? ??????? (????: Google Chrome OS) (21 ??? 2009) | ????????? ? ??????? | ????? ?????

  • redbaron

    Great article. Thanks for the explanations. Short but enough to understand what’s what. :)

  • billy bob

    52 Nov 21, 2009 at 04:06 by bittorrent forever
    You missed too far this time.

    The key problem of bit torrent is not tracker or trackerless. The problem is these site (TPB) is that TPB is setup to be a failure.

    TPB inflate the number of users anbd downloaders. It upset the MPAA. And MININOVA too. MININOVA claims that it reached 10B torrent downloads, but how many of these downloads come from the real end ueers? How many of them come from the search engine.

    If pirates shouting all the time to the world how much money he made from rubbery, he is doomed.

    It is going to be the same. Even TBP goes to megnet, if they do not change the current business model, they can still be hunt down by MPAA.

    The most important thing in this p2pdownloading thing is, keep low profile, do not upset the mainstream.

    A real pirate neverl claim he is a pirate. A real pirate is the one who take, and hide.

    Well – this is the problem with Torrents in general. The technology took somehting that has been traditionally an underground movement (and rightfully should have stayed that way) and dumped – full force into the mainstream so that everyone from “soccer-moms” to “little billy” can click a link from a webpage – copy and paste into a Client and click start.

    I think the technology itself is a great leap forward and has many uses and great potential – but at the cost of putting a spotlight on something that used to be behind the scenes has simply gotten out of control and both the general public and the media outlets simply don;t know what to make of it.

  • Bobe-On

    @ Nov 23, 2009 at 17:57 by billy bob:

    Whose side are you on?

    I disagree, and perhaps quite vehemently:
    The more people who use something, the better and more robust that thing can become, due to its use, such as with involvement, feedback, support, etc., and the more who can benefit from it. Like FLOSS.
    To suggest something like that remain “elite” or “underground” seems to suggest that that’s its best use; or to lend support/capitulate to those who would want to remove or control it.
    Fuck that.
    Power to the people.
    ‘Sharing is caring.’

  • Pingback: How can etailers handle the Christmas rush? – Econsultancy | OnlyDedicatedServers.com

  • Kickass_Sid

    Really useful article!! Thanks!

  • consuminghatred

    I don’t think I quite understand how this new shift towards magnet links will help TPB from avoiding being a MAFIAA target.

    Even before, while offering links to the torrents (which are not illegal and are not, in themselves copyrighted content) they were chased and demonized by the big corporations.

    Based on technicalities the p2p community had the upper hand, we weren’t, technically, downloading through the TPB any kind of copyrighted work. So how is this supposed to help?

    The MPAA and its analogs will still say they are helping commit copyright infringement by facilitating it.

    I don’t see the point, really, or maybe I have misunderstood some part of this or their real intent.

  • ws

    @230 Nov 24, 2009 at 04:14 by consuminghatred

    From the PB blog post:

    “Now that the decentralized system for finding peers is so well developed, TPB has decided that there is no need to run a tracker anymore, so it will remain down! It’s the end of an era, but the era is no longer up2date. We have put a server in a museum already, and now the tracking can be put there as well.

    By moving to a more decentralized system of handling tracking (DHT+PEX) and distributions of torrent files (Magnet Links), BitTorrent will become less vulnerable to downtime and outages:

    * With decentralized peer acquisition, there is no central tracker that can be down.
    * With decentralized fetching of metadata (torrents) we don’t need to rely on a single server that stores and distributes torrent files.”

    The key word in all of that is DECENTRALIZED.

    That is the intent and the future, since a site still could be accused of helping commit copyright infringement, by publishing the magnet links to your torrents.

  • Pingback: BitTorrent’s Future: DHT, PEX, and Magnet Links Explained [BitTorrent] · TechBlogger

  • Pingback: Open Systems Journal » Blog Archive » BitTorrent’s Future: DHT, PEX, and Magnet Links Explained [BitTorrent]

  • Pingback: BitTorrent’s Future: DHT, PEX And Magnet Links Explained | Lifehacker Australia

  • Pingback: Future of Bittorent, DHT, PEX and Magnet Links Explained | Bay Area Tech Pros

  • Pingback: Novell Strips BitTorrent DHT Technology from openSUSE @ blog.idtorrent.org

  • Pingback: BitTorrent’s Future: DHT, PEX, and Magnet Links Explained [BitTorrent] | CHARGED's Digital Lifestyle at Work or Play

  • Pingback: BitTorrent’s Future: DHT, PEX, and Magnet Links Explained [BitTorrent] | Interesting news

  • Your Mom

    Good Read thanks.

  • Pingback: BitTorrent’s Future: DHT, PEX, and Magnet Links Explained « Hubcom.org – Tech News 24-7

  • What BS

    I knew something was coming when Warner Bros. bought Pirate Bay.

    If a candle maker invents a light bulb and patents it and files the patent so nobody can use light bulbs and have to buy candles, is that fair? No way! Sharing files are sharing purchased property and these swindlers can’t keep up with technology. Patent laws are the problem. All stories in movies today are just rehashed Biblical, Shakespearian or folklore plagiarism.

  • Pingback: MININOVA EST MORT

  • Pingback: Mininova is dead... Is time to think about alternative to torrents ?

  • jpbit

    Billy Bob, actually Bram Cohen developed BitTorrent as a content distribution system for general use that would help limit the bandwidth requirements of the original content’s source by turning everyone in the swarm into a source for the content. In fact, quite the opposite of designing it for the use of some self styled elites who think too highly of themselves.

  • Pingback: Fault tolerance in virtualised environments – The Register | OnlyDedicatedServers.com

  • Pingback: BitTorrent’s Future: DHT, PEX, and Magnet Links Explained [BitTorrent] | Computer Tips and Tricks

  • ravin

    does anyone wanna swap an iptorent invite for anothr good private torrent site invite? email me preston_raver@hotmail.com

  • www.desiimg.com

    this technology is still growing..
    but I wonder why I get low speeds in magnet links ?

  • Myp2p

    Can someone help please, I just want to know if Magnet links are safer? Thanks

  • BTGuard - BitTorrent Anonymously

NewsBits

Even more news...

  • 404 Fail: Six Strikes’ Piracy Alternatives Go Missing

    After years of negotiating and planning the “six strikes” copyright alert system finally saw the light...

  • Google Doesn’t Believe Kim Dotcom is Real

    Those who enter Kim Dotcom into Google’s search box will get millions of results. Still, Google...

  • Nintendo’s Miyamoto: Piracy More Concerning Than Used Games Market

    In an interview with Computer and Video Games, Nintendo designer Shigeru Miyamoto says that his company...

  • Monitoring BitTorrent Activity on a Network Using WireShark

    As all readers should know by now, BitTorrent activity is public and can be easily logged...

  • The Pirate Bay Suffers Downtime

    The Pirate Bay is not responding at the moment and most proxies appear to be down...

MostDiscussed

Below are TorrentFreak's most discussed articles of the past month. Join the discussion if you like.

CopyQuote

Left Quote

“The Pirate Bay has been one of the most important movements in Sweden for freedom of speech, working against corruption and censorship.

Peter Sunde Left Quote

PopularArticles

A selection of some TorrentFreak's classics dug up from our archives.