Unpack RAR Archives before you release a Torrent
Written by Ernesto on November 23, 2006Here’s a great essay explaining why people shouldn’t put RAR archives in torrents. They are useless and they make “selective downloading” impossible.
Found at The Piratebay
Why RAR archives do not belong in torrents
It has become a common practise to create torrents which contain umpteen RAR files, often in the form of a Release from The Scene, instead of simply directly adding the actual files to be distributed.
Though some might think this practise lends a bit of underground mystique to the common BitTorrent world, that is the only thing that might possibly be said in its favour. Others think the use of RAR solves some real problems. The truth is it adds nothing but nuisance
and even renders a useful feature of BitTorrent clients useless.
This essay aims to explain why it is a good idea to extract your
RAR archives before creating a torrent.
Why are there umpteen RAR files to begin with?
This essay is not trying to argue that sceners should change the way they package their releases. As a common BitTorrent user, the author couldn’t care less. Since sceners transfer their files by other means than BitTorrent, there really are some problems that are
solved by the elaborate packaging.
The first problem is that of files being corrupted in transfer. One of the features of the RAR format is the use of checksums to ensure file integrity. If a RAR file is damaged in transit, the idea is that thanks to the checksum you will notice this when the archive is extracted, so that you will not unknowingly end up with a corrupted file.
The second problem is that of re-downloading when something goes wrong. With the use of multi-volume archives (that’s the umpteen .r01 and so on files) you don’t have to download the whole thing again. However, RAR checksums are not very practical for multi-volume
archives. Which is why releases also come with .sfv files containing checksums for each individual RAR file.
The third problem is that many file transfer protocols do not support the parallel downloading of one file from multiple sources. Multi-volume archives let you work around this manually.
BitTorrent solves these problems for you
Most people know that solving the third problem, parallel downloading from multiple sources, is the very essence of BitTorrent. That multi-volume RAR archives are still common in torrents suggests that some have missed the fact that BitTorrent also takes care of the
first and second problem for you. And it does this automatically, without requiring any other tools. What the sceners do manually, the BitTorrent users need not bother with.
Each .torrent file includes information on how to split the files being distributed into many smaller pieces that can be transferred individually, even out of order. This is much like the multi-volume RAR archives, although the pieces are smaller and you never have to deal with them yourself. So when a scener might have to manually re-download a file of several megabytes, the BitTorrent user doesn’t even notice that their client automatically re-downloaded a few kilobytes when needed. Your BitTorrent client can do this, because
the .torrent file also comes with checksums for each little piece. (Checksums which, by the way, are created using a much better algorithm than that used in .sfv files). That is how BitTorrent also solves the first and second problem for you.
What RAR archives in torrents really achieve
So if BitTorrent already takes care of the problems the RAR archives was supposed to solve, what do they really achieve? To begin with the obvious, before you can use the files you just downloaded, you must extract them. Since practically all of the material
distributed with BitTorrent comes in compressed formats to begin with, like music and movies, the little compression RAR adds is useless. Instead you need twice as much free disk space. First for the RAR archives and then again for the extracted files. And if you want to
play nice and seed for a while like you should, you must keep the RAR files around.
But it gets worse. Sometimes you don’t want all of the files in a torrent. A good BitTorrent client lets you download just the files you want. But if the torrent contains a RAR archive, you must download it all anyway to eventually get the files you really want.
So to summarise, RAR archives in torrents waste disk space and hinder useful BitTorrent client features, without adding anything of value. In other words, they are a useless pain in the ass.
But The Scene has rules, and RAR is cool!
Face it, if you get your warez with BitTorrent, you’re not part of The Scene. And that goes for the people that will download your torrents as well. They will not care about rules or archive formats. What they will care about is if you waste their time and disk space.
What about NFO files and credit?
That’s simple. If you care about giving credit where credit is due, don’t rename the files you extracted from the RAR archives. And let any .nfo files be part of the torrents you create as well. But keep the useless stuff like .sfv files and samples out. Who needs samples when you can just open the file directly?
By Anonymous 2005. Verbatim copying and redistribution of this essay
are permitted provided this notice is preserved.
Previously: Brein Sues Provider for Hosting BitTorrent Site
Next: TorrentPod Episode 14
59 Responses
hmm,,, intresting,, dont know why but intresting nevertheless ;p
I agree with this essay! It’s so annoying when I find a certain J-pop album or single I’ve been wanting to listen to on JPOPSUKI’s tracker, and it’s in a rar or zip! Everyone should unpack or unzip their rar and zip archives before sharing their contents on BitTorrent.
I’m glad I’m not the only one that thinks like this. It’s a royal pain in the ass having to unrar a heap of files that I downloaded from Bt. Only to find out that it’s a fake file, like the most recent episode of SVU was.
And another thing, if they’re unrar’d people are more likely to seed the file longer. To be quite honest, when I have a bunch of rar files on my hard drive, I just delete them as fast as I can.
They’re nothing but a waste of space, and only useful if you’re uploading to Usenet, which nobody really uses anymore since most ISP’s have crippled them to the point of uselessness, or have removed them all together.
Hmm it has a interesting point.
i can see how non-compressing stuff like music and Videos, can work great.
but i will have to massively dis-agree with this concept is applied to Video Games and Applications, imagine trying to download a operative System or application that isnt saved as a ISO, (you get that allot with older games) and if its compressed it can shave around 10% off the file size. i remember downloading a old Game it was saved as a rar (around 70mb) and upon un-compressing it, the total file size around 400-600mb…..
….now i need to find a program that can do compression like cutting the size down around 5 times small than orginal size :-0
Totally agree with you here, archive files in torrents are useless.
Who needs samples when you can just open the file directly? >
[quote comment="24556"]Who needs samples when you can just open the file directly? >[/quote]
Samples are useful if you want to check out the quality of the video, without downloading the whole file.
the scene is stupid anyways, all their stuff is for “private” usage only. no xdcc/www/bt is their “rules”.
I don’t understand it at all.
The splitting by torrents is well known to all users, since you cant notice it in your torrent client.
The checksum part for individual parts can you see within the client as well, since you will get an error of an exact piece number in the log, so you can download it again if needed.
About the multiple files, thats the only one that is worth mentioning, most private trackers have rules for splitting the main content in rars and all other files around it, then the problem is solved too!
About the double size, its not an issue with movies at all, since you can play them in players that play without unpacking. With warez the story is different; when I assume you will burn the program away or delete the installer after the usage of it, then the problem still doesn’t apply.
Beside the above, rar’s are a great way to distribute because its a standard for everyone, else you would create a mess in the torrent world with all kind of archive specialists.
Sorry to disagree but it’s nice being able to confirm that the video or game you are getting is unaltered and the actual content that a group released.
A group takes pride (how ironic) in releasing content and they would quickly get ignored by the scene if their warez contained viruses or stolen or low quality content.
The benefit of unaltered warez is then less risk of virus and re-encoded videos by some basement wanna-be.
The way to confirm the warez is to go to a scene tracking site, check a few comments on the release and possibly get the official check-sum file (sfv).
I do agree about the fact that rars get deleted more quickly bringing down potential seeders. However this can and will be solved with bigger drives however!
[quote comment="24564"][quote comment="24556"]Who needs samples when you can just open the file directly? >[/quote]
Samples are useful if you want to check out the quality of the video, without downloading the whole file.[/quote]
Argh, my comment was borked. I actually quoted the sentence “Who needs samples when you can just open the file directly?” from the text. Seems like the > killed the rest.
I find samples damn useful (nfos too) and thus question the text…
I’ve been producing torrents for 6 months now, I’m glad somebody actually bothered to spell it out for everybody to see…
These should be rules that are dictated by common sense, anybody who downloaded a handful of torrents should have figured these out.
I’m downloading ~100 Gb of data through BT each month and if I hadn’t plenty of space , RARs would have been a real pain. Both for seeding and unpacking.
Really dont know id .rar or .zip is really worth the time reqd for the uploader to create the archive.The worst part is when there is a password and no one knows what it might be. About 3 weeks back I downloaded “Snow Falling On Cedars” which was in .rar archive format consisting of 29 parts of 50 MB each. The password was mentioned as “Cedars0011″ but when I tried to unpack there was a CRC error saying that the password was wrong. Now I searched again and found that a large number of people had the file with no password and consequently were against seeding a useless file to other peers.
By the time a working password was found most people had decided to get rid of the file.As a result a rather desirable torrent was reduced to a half dead torrent.( Might even be totally dead by now!).
Well as mentioned Samples definitely are useful, SFVs obviously would be useless without the rars but NFOs definitely do have to stay. Rar’ed archives in torrents ONLY make sense when the content comes from the scene, if it’s non-scene then raring is quite useless but for scene content it is worth keeping in original format. The are a few reasons for this, firstly is speed, uploaders will be getting the content in Rar’ed format, they’d need to unpack it first then hash it and upload. Secondly is errors, yes bittorrent will fix any errors in errornous transfers but what happens if the uploader unpacked the torrent and it became corrupt at that point, well they would have to rehash the whole torrent and reupload, if the same happened with a rar which didn’t transfer proper then only that bit needs uploading as either a “fix” torrent or the whole thing can be rehashed but the rest of the content will stay exactly the same. Also leaving the torrent in rars allows people to jump on the torrent easily if they downloaded it from somewhere else.
There are two other flaws mentioned either in your article or comments though, firstly Rars are NEVER compressed, they are used purely for file splitting. Compressing would take a long time and add very little if any additional compression, uncompressing would also take much longer. Secondly rars from the scene will NEVER be passworded, if they were they would be nuked, passwording rars is lame.
I agree 100% with this essay, it’s totally useless.
RAR packages are often compressed depending on the type of content.Different file types offer a highly varied potential for archiving compression. Multimedia files do not compress well.Game packages au contraire compress quite well.A 1 GB game might compress to less than 200 MB when compressed using a proper algorithm. However even for games Uharc archiver ( If you want more info http://en.wikipedia.org/wiki/UHarc) gives better performance than Winrar or Winzip.
Splitting is useful for the archaic users who just cant get enough of the floppy or when uploading to file servers like rapidshare.de where size limits are applicable to files( not all cases!).
So the verdict on rar archives is…….For torrents it is just a way of making money ( checkout http://www.ppnow.com or the links to various shareware rar password crackers in the archive) or for harassing peers!
So we are supposed to think the uploader of a RAR file is a cool scenester, and there was I thinking they were just a clueless numpty.
Thanks for spelling out clearly, insightfully and (I would have thought) incontravertibly what anyone who’s actually ever downloaded a RAR-ed video already knew - they’re an unneccessary pain in arse.
As for ISOs - fair enough, as it does make the file size smaller, and what good is half an ISO anyway?
FULLY AGREED!
Rar files suck my rim and are so annoying
HOW CAN I OPEN THIS TANGENT FILE
I dont agree. At all. But I doesnt really care either since I havent been in contact with TPB for a long time anyway.
http://thepiratebay.org/tor/3563005/Why_.rar_files_are_OK_to_use
tux u ar right about jumping into the download after downloading from somewhere else
jose u ar very stupid not sceners
sceners are d ppl who make d blody stuf u download
you guys are idiots… I won’t even bother to say why…
I agree, and only have one correction to make. The original reason for multi-rar sets is this: At one time, when all of this scene stuff started, people used floppies to transfer the stuff around offline and to back up, so, filesplitting software (and, later, multi-archive software) was used to make nice, easy to use chunks of 1.44MB or, in some cases, 2.88MB that could be easily stored on floppies, later, zip drives became handy and some people started making them 100MB instead and some people still on dialup complained, so they had a big meeting about it (yes, the sceners) and came away with two groups, each of which adopted a different standard, one was 50MB (or 2 archives per zip disk) and the other was closer to 25MB (or 4 archives per zip disk) (I also noticed 12.5 MB files for obvious reasons) of course, now that CDs and DVDs are around, the multi-archive thing is less about backup and offline transfer, and more about tradition and some of the reasons you mentioned. Hope this helps :-)
the reason they split it into rar files is pretty simple. All the stuff goes on Usenet first, where the maximum file size have to be pretty low before the Usenet servers add the files to their archive. It’s not because “it’s how we used to do it, why change it?”.
And I’m not speaking for or against rar archives. I don’t really care that much, I rarely get anything bad.
all the rar files ive downloaded have been corrupt, rar is crap!!!
@David: outdated version of WinRar will incorrectly report some archives as corrupted or password protected. Upgrade to latest RAR!
Can choose individual files because it’s RAR’d? I don’t feel that it’s in good practice to archive multiple files together in the first place.
There is the odd APP/WAREZ torrent with the installer and a few patches archived, sure that’s fine, you may often want both so it’s okay. If you want one thing or another, is it so hard to hunt down another torrent without the RAR archives? I’ll admit that some torrents are ZIPs inside a RAR inside a RAR with an NFO file or something - that’s just stupid.
Really, if you’re so against RAR archives, hunt for a torrent with the extracted files. You’ll more often than not [especially if it's in demand] find at least one with some decent seeders.
In my recent experience, I had some trouble with my hard drive, merging partitions, and lost several files, mostly torrents, above a certain file size. The damage was less than it would have been thanks to those torrents in RAR archives. So they are useful, dude, so lighten up.
The author is an idiot and a newb of the worst kind. Sceners DO NOT rar up millions of clips and files or songs that a downloader might want. They rar up SINGLE releases (ie: a whole movie, a whole iso, etc.). Even a 0day release contains only ONE single release. The crap he might download might have millions of different things rar’d up together but that is not an official scene release. One of the main reasons that releases are rar’d up with an sfv is to check integrity as well as make it convenient to get a replacement file if one is bad file from one site (or tracker for that matter).
And a couple of important points you should remember.. First, your a leech. Which means your entire article is crying and whining about how YOU find it difficult to leech what you want YOUR way. Second, P2P is an offshoot from the scene. Without the scene and its DECADES of existence you wouldn’t be here with your web blog trying to influence people who do not know any better that YOUR way is right or better than years of experience and technical evolution.
And lastly.. YOU GET 90% OF WHAT YOU DOWNLOAD FROM THE SCENE IN THE FIRST PLACE AND THEY DO NOT CARE WHAT YOU THINK.
Ryche
I was going to post, but Ryche said everything I wanted to say. I concur with Ryche, so should you. Good work buddy.
jonny166
Can anyone help me. Call me blonde but when I click on a .001 file of a game I have downloaded in winrar, it asks me for a password. Can anyone tell me how or where I am to find what the password is ???
OMG…
The autor (and a few) is an idiot…
OMG…
The autor (and a few) is an moron…
Jasper van Weerd and Ryche are right..
im sorry to say..i find its hard to agree with ur pointless argument..
i used to hate RAR files..but as i come to learn bout it..used it..and distributed it..i say u need to practically learn more before u say anything..thank u..
can anyone crack a RAR password for me??
can anyone crack a Multi RAR password for me ??????????Crankifornia_6@yahoo.com
I hate RAR files. Because they are so damn hard to get to run right. As a matter of fact still haven’t been able to successfully get a RAR file to run right. Stick with ZIP. It is so much easier to get to run.
Since you are not supposed to download the files i dont think the scene will care what you think..
Oh my, what an incredible bunch of retards there are standing up for The Scene. Did any one of you read this line: “This essay is not trying to argue that sceners should change the way they package their releases.” It is merely pointing out that it is pointless to upload the rar’d files to a torrent tracker. If your only worry is that it will not be an original Scene release, then only download from private trackers.
Also, if The Scene and its wannabee groupies don’t care, then why have you all replied?
One more thing that has not been mentioned in these comments with regard to the RAR format: there is no effective Free Software solution to unpacking such archives.
Certainly a binary “unrar” program is available. Under a silly up-yer-bum license, and no source is made available. (How many security booboos lurk in there, that we haven’t yet heard of? Will the author update the binary-only program for “minor” platforms?)
This problem is felt more seriously by PowerPC users (i.e. GNU/Linux or FreeBSD on a Mac or some such), who cannot even resort to proprietary software as an interim solution. To a free software purist the situation is simply unacceptable.
For those who want to compress their ISO images and the like, I would suggest 7zip (compresses really, really well) or just plain old zip (compresses a bit less, but still takes the “air” out of your ISOs plenty fine).
are there step-by-step instructions to using WinRAR to unpack these type of RAR files?
[quote comment="143582"]are there step-by-step instructions to using WinRAR to unpack these type of RAR files?[/quote]
What a n00b!!!
Search the fucking web!
I agree, RAR sucks!!! It’s just for n00bs like the lamer who asks about a “step-by-step” guide. Poor boy.
torrents that aren’t rar’ed live longer, FACT
1. “Sceners DO NOT rar up millions of clips and files or songs that a downloader might want. They rar up SINGLE releases (ie: a whole movie, a whole iso, etc.).”
Or a whole season of a show.
Want a particular episode first (or only)? With regular video files, I can tell my client to make certain files a high priority, others a low priority, and not download some files at all. I can’t do that with RAR.
My torrent client gives me percentage completion for each file I’m downloading. If a file reaches 100%, I can start watching it right away. I can’t do that with RAR.
My video player is quite good about dealing with “corrupted” files, including partially-downloaded torrents. Even if a file hasn’t reached 100%, I can usually start watching if it’s close, and it’ll skip over the incomplete parts. (So much for needing samples.)
And all this still ignores one of the big annoyances about RAR — having to uncompress them at the end. There’s many factors involved there: Extra time to decompress them. Double the space to hold both the RARs and the decompressed results. Risk that the file might be password protected (or fake altogether) and you’ve just downloaded gigs for nothing.
2. “One of the main reasons that releases are rar’d up with an sfv is to check integrity as well as make it convenient to get a replacement file if one is bad file from one site (or tracker for that matter).”
Read the article. “BitTorrent solves these problems for you”. And it does it better than RAR, too.
With BitTorrent, you check file chunks *as* you download, rather than just at the end. Plus, if a chunk is incorrect, it’s much smaller and easier to replace than an entire RAR part, and your client does so automatically.
3. “And a couple of important points you should remember.. First, your a leech.”
Even BitTorrent “leechers” are still helping everyone else download, by re-uploading file data to everyone else. They may not provide content themselves, but they do still help spread the existing content to those who want it. Releases wouldn’t be worth very much if nobody actually downloaded and used them.
4. “Without the scene and its DECADES of existence you wouldn’t be here with your web blog trying to influence people who do not know any better that YOUR way is right or better than years of experience and technical evolution.”
BitTorrent is the latest result of those “years of experience and technical evolution”. It solves almost all the problems RAR files once solved, but better.
The one unsolved problem is compression, but as stated, most large files are already compressed and don’t benefit from RAR.
Multi-volume RAR files were just a cheap, early imitator of what BitTorrent now provides, and are only useful when you *aren’t* using BitTorrent.
Just because we’ve done something one way for decades doesn’t mean we shouldn’t use a better way when it comes along.
5. “And lastly.. YOU GET 90% OF WHAT YOU DOWNLOAD FROM THE SCENE IN THE FIRST PLACE AND THEY DO NOT CARE WHAT YOU THINK.”
Actually, I get what I download from people who take Scene stuff and then release it as a torrent. *These* are the people who should be unRARing the data before creating their torrent.
If the Scene is going to continue doing things on a less-reliable medium like Usenet, then there’s no reason they shouldn’t use RAR or whatever they feel is best.
All we ask is that the data be unRARed *before* it hits the torrent trackers. That’s the way BitTorrent is meant to be used, and that’s the way it works best.
I have to totally agree with the essay. RAR files in BitTorrent is so annoying. The one thing you fail to mention is that it is one more application loaded on a PC that I, for one, do not need.
I tend to unpackage my downloads, then re-package them without RAR, as you suggest, leaving in the credits.
Any new torrents I upload, I DO NOT RAR ‘cos it’s crap, outdated and serves no useable purpose. It is simply self-gratification for numpties who like to make life difficult for themselves.
Now go and write “RAR is crap” 100 times then split it into multiple 50MB files if it’ll make you feel any better.
Z
PLZZ!
if u upload a torrent DONT rar it.
use common sence ffs! who cares even if the file gets cut in half, 3gb vs 1.5gb, with a 56k modem i would care. its not 1990 anymore. and its not cool. many times files dont get that much smaller anyway.
RAR sucks
if its rar’ed i look for another torrent thats not rar’ed. rar is crap. finally some smart people that understand this, and dont just go with the flow!
thx
dont listen to fanboy-scene-wanna-be-people just unrar and up the torrent.
This is so wrong!
I can agree that for example music albums sholdn’t be rar’d, but
* apps
* movies
* games
etc etc still should, since you can’t know how by what means it was transferred before/will be transferred after and how many times that’ll happen - not keeping the scene releases intact only makes for worse and worse quality (video artefacts and the like).
Oh, you freakin rar idiots! Like what game saved you 10%? I’m gonna call B.S. on ya Bud… jpg’s png’s game exe’s mp3’s mpg’s vob’s all have rar-like compression built-in and every game I know uses it to the max to keep games from running onto too many disks, so rar just winds up as a container and - - - a PAIN IN THE ASS! So stop it already! If the files are not in an archive now - do us both a HUGE favor and don’t waste yours and my time and drive space packing them up AND if they are packed now, when you unpack them to use them - re-seed a new torrent with the files you unpacked (I’ll give you a kiss —- XXX— @!%$! HEY NO TOUNGE!! - dammit I hate that!)
SCRU - The Scene Release Unpacker
http://www.tmoogle.com/scru
Belligerent Engine, that’s not correct. The sources of unrar are available and it works on even more obscure platforms than yours.
Regarding RAR itself. It has no purpose on any modern P2P network because all of them support resuming, swarming, checksuming and more. If the self-announced elite insists on using outdated technology like FTP, so be it but everyone with a brain should know better.
There are of course exceptions where archives like ZIP/RAR/TAR are suitable like software source files where you have large number of files which are rather useless on their own.
I think RAR etc. are also used because some early bittorrent implementations had issues with torrents that contained many files because they’d try to open each of these files even though there’s usually a limit of a few hundred (or even less) open files. Nowadays these just open the files they really need to access at any time.
I read about half way through the comments…
I think that rar’ing the release at this time serves a purpose. You can’t download parts of the release and skip things you don’t want.
becuase you have to download the entire release everyone seeds everything… downloading is faster for people who want the whole thing and not just that one song (or whatever)…
And in the year 2008 with a common 8mps download speed downloaders still bitch about wasting hours as opposed to the days we waited for a file 10 years ago.. RAR’ing may cut off a couple mb’s but in the time it takes for you to take a piss, the difference of an unpacked file could be downloaded already. It’s not as if every torrent you download must be ready within the hour. There is no point to RAR files that are generally already under a good compression. And if not, the time it takes to download the extra bit for the unpacked torrent file is minuscule. I’d much rather spend 10 more minutes waiting for a file to finish, then 10 unpacking the file with the grim hope it’s even the file you wanted in the first place.
I’m sure it’s also less work/time to upload a file or two, than RARing and uploading them.
Do the simple, tried and true. Make a nonRAR torrent. It’s simpler for us, and simpler for you.
I agree with the article/essay.
The Scene can release how they wish, but they don’t release .torrent files. Who uploads .torrent files in RAR format? Second-Hand people get the RAR files from The Scene, and make it a .torrent.
THOSE people should unpack first and then make a .torrent file. If it’s crap, torrent sites have comments too.
Sometimes a torrent claims to be “an image” but doesn’t say if it’s .ISO, .UIF, .NRG, .IMG…etc. If the unpacked image file were the source of the torrent, then we’d all know.
Agreed, get rid of rar’s and zip’s in bit torrent.
I totally agree, those “scene” kiddies are a bunch of tards for use Rar.
Real men use tarballs like Gzip. Damn win-lammers, they must die!
I think that rar format is used to reduce size in sending any information across a large expanse of people. What annoys me is that these torrent programs are designed to share files over a global area, So why bother putting a password on a file that is supposed to be shared and give people reason to not trust these files as they are designed to be shared….
Thank you, Ernesto, for this desperately-needed article! I'm glad to see from these comments that there are others out there, too, who see the light. I'm with David–fed up with corrupt archives! Everything compressed ends up corrupt at some point; I'd rather wait a little longer downloading the original file uncompressed than wait TWICE AS LONG having to re-download a "compressed" version three times before I can get it without corruption! And I feel REALLY sorry for those of you who have to go through this re-downloading with miserable 56k dial-up connections!! Already been there, done that, I sympathize with you!!
Responses are closed
All remaining responses will continue to be archived. Use the TorrentFreak forums if you want to discuss something.