I recently lost all of my music collection due to a harddisk crash on my fileserver, which means that I have to start collecting all over again, and believe me, some rare songs are pretty hard to find.
On the other hand I often find a nice song on Youtube and decide that I have to keep it. And after a bit of looking around I found the handy youtube-dl script which automatically parses the pages content and downloads the video file, much like keepvid.com does, but all from the shell.
After we have the video file all we have to do is dump the audio and convert it to a format we like (most likely MP3 or OGG). Until today I used mplayer and mencoder to dump and encode the audio, but I stumbled across this article on howtoforge.org that uses ffmpeg, and is more flexible and avoids me one intermediate step
So now I have a really simple script that fetches the video and converts it, which looks like this:
#!/bin/bash
# Just additional setting to use high quality video
YOUTUBE_OPTS="-b"
# Please make sure youtube-dl is in your path
youtube-dl $YOUTUBE_OPTS -o temp.flv "$1"
ffmpeg -i temp.flv -ab 128000 -ar 44100 "$2"
Please make sure, as stated above that youtube-dl is in your path. Usage is pretty darn simple:
ripyoutube <youtube-url> <mp3-name>
Don’t forget to enclode the URL and filename into doublequotes
I love my music, I really do. Problem is however that I have far too much of it (2980 Songs, 16 GB currently, not counting audiobooks etc…) to carry it around with me all the time. It just doesn’t fit on my iPod Mini, nor does it fit on my notebook (I know shame on me, but I need the diskspace for work too). So I always have to carry around a selection of the songs I want to listen too, and far too often just the song I want to listen too is on my fileserver, and the trouble to transfer it to my notebook takes too long for me to be still in the mood to listen to it… Pitchfork is the solution to all of my problems, well almost all… Pitchfork allows me to stream my music over my Local Network, and over the internet, with a nice and responsive interface that facilitates access to my songs.
Installing MPD
Since Pitchfork is based upon MPD, which will take care of streaming the data to the various clients and maintain the song database, we will first have to set it up. Sadly my distro (OpenSuSe 10.2) doesn’t have a version of MPD in its package management, so I had to download the sources and compile it myself:
wget http://www.musicpd.org/uploads/files/mpd-0.12.2.tar.bz2
tar -xvjf mpd-0.12.2.tar.bz2
cd mpd-0.12.2/
Now we have the sources, and we have to check that we have all the dependencies. Depending on how you are going to use MPD you have to have the following packages (along with their devel packages):
libshout: because we will be streaming the songs to an Icecast server
oss: as far as I know this is the most widely used and best supported. If it doesnt work take a look here
libmad: for mp3 support
libogg: for OGG Vorbis support
libvorbis: for Metadata support for Vorbis formats
Quite a long list, but this is about the most essential stuff, if you have some special needs take a look at the official dependency listing.
Now it’s time to get to the actual compilation:
./configure
make
sudo make install
After the configure step, you’ll see a list of options that are turned on or off, if something is missing it probably means that you are missing some development packages.
So now we have mpd installed, the next step is to configure it. First you have to move the example configuration to the correct location:
sudo cp doc/mpdconf.example /etc/mpd.conf
Then we edit it according to our needs, especially the location of the Songs, Logfiles and Database should be changed:
music_directory "/storage/Music/"
playlist_directory "/storage/Music/playlists"
db_file "/var/lib/mpd.db"
log_file "/var/log/mpd.log"
error_file "/var/log/mpd.error"
pid_file "/var/run/mpd.pid"
The pid_file has to be uncommented since we will run MPD in daemon mode, and we want to shut it down later. Now let’s create these files and change the owners:
“mpd” being the user that will later run the daemon (if you don’t have an mpd user refer to your distro documentation on how to create one. It is really important that the mpd user is in the sour). Next we have to set some more settings:
user "mpd"
audio_output {type"oss"
name "Direct OSS output"}
audio_output {type"shout"
name "Icecast Stream"
host "localhost"
port "8000"
mount "/mpd.ogg"
password "youricecastpassword"
bitrate "96"
format "44100:16:1"}
mixer_type "oss"
mixer_device "/dev/mixer"
mixer_control "PCM"
mixer_type "software"
Initializing the database:
mpd --create-db
This will insert all your songs into the database for faster access. Now MPD is set up and should work. Try with
sudo mpd
MPD will complain that it can’t connect to the Icecast server (which we’ll setup a few lines below this, but everything else should work fine.
Installing Icecast
As I said, my Music resides on a fileserver, and I want the songs to be streamed directly to my other machines, so what’s better than Icecast, an opensource streaming solution, used by many internet radios out in the wild. Luckily most distros have it in their packagemanagement software (I know openSuSe has ) so it shouldn’t be too hard to get it up and running, just make sure that the source password in /etc/icecast.xml matches the password you specified above, in the /etc/mpd.conf file.
Installing Pitchfork
Installing pitchfork is pretty straight forward. All you need is this:
Please refer to your distro documentation to see how to setup these. My setup uses an apache 2 server with mod_php, and Firefox 2 on my notebook. Now to the installation itself, depending on your distribution the document root of your webserver is /var/www/htdocs or /srv/www/htdocs, change to that directory and do the following:
wget http://pitchfork.remiss.org/files/pitchfork-0.5.2.tar.bz2
tar -xvjf pitchfork-0.5.2.tar.bz2
cd pitchfork-0.5.2/
chmod a+rwx config/
The rest of the configuration is done through the browser, just point your browser to http://localhost/pitchfork-0.5.2/ if you installed pitchfork on the computer you’re currently on, or replace localhot with the IP of your server. Change whatever you’d like to change (but it should not be necessary as the important configuration is done in MPD itself), and then press Save and you’ll be taken to the Pitchfork interface. Voila, it’s all done
Installing mpdscribble (optional)
I’m a huge fan of Last.fm, but sadly I can’t update my profile using the Player, because the songs are streamed to it. So why not use the MPD to update them for me? For this purpose I will be using mpdscribble:
wget http://www.frob.nl/projects/scribble/mpdscribble-0.2.12.tar.gz
tar -xvzf mpdscribble-0.2.12.tar.gz
cd mpdscribble-0.2.12
./configure
make
sudo make install
Now create the basic configuration
mkdir ~/.mpdscribble
It is suggested that mpdscribble is run as your user since it has read access to your username and password-hash. Edit the configuration file ~/.mpdscribble/mpdscribble.conf as follows:
username = "lastfm_username"
password = "md5sum of lastfm password"
Where the md5sum can be found using either this tool or the command line md5sum tool:
echo -n "your_lastfm_password" | md5sum
And that’s it, mpdscribble is set up and ready to be run:
Before we start discussing what are the good and the bad habits when you use BitTorrent we have to define what the goal of a seeder is. First of all a Seeder is intended as the first seeder in this article, the one who creates the Torrent-File and therefore the first who uploads any data to the other users. He wants his file to be spread to the other users and possibly detach it from himself, by detach I mean that the torrent continues to be spread without the need that the first seeder has to be still seeding. However as I explain beyond
Don’t Zip!
Often people package their files together in one compressed archize (zip-files, or one of the other compression mechanism like, rar or ace) and then generate a Torrent-File out of them. There are several reasons why this not a good idea.
A waste of disk space: zipping your files is a real waste of disk space for both the seeder and the leecher, because to be seeding the file you have to have the compressed file at hand, while to use its content you need to uncompress it. Let’s see the usual scenario: A user downloads a Film with BitTorrent. The file that is being downloaded is a zip archive containing both the Film and some junk files that nobody whants to see and will be the first to be deleted. The user has to extract the movie from the archive to watch it. Therefore he needs twice the space to uncompress it, and maybe, as it so often happens, first needs to delete something else to free the needed space first. Once uncompressed he has the movie twice on the disk, so he can throw away one of the two. Which one will he choose? Right: the compressed one, thus another possible seed has vanished…
It’s completely useless because nearly all the data on the net is already compressed with media-specific codecs (see DivX for movies, OGG or MP3 for music, pdf for Documents) the general purpose compressions like zip, rar or alike have very small or no effect whatsoever.
Seeds disappear: basically this is already explained above, but I have to press on this fact that disappearing seeds are the main reason for torrents to disappear too. Users often choose the Torrents with the most Seeds, as they are most likely to last until they finished downloading, thus the number of seeders not only makes downloads faster but also makes sure that new users will come.
Don’t steal Torrents!
What is stealing anyway? We don’t speak of physically stealing a property from each other, since often the contents aren’t owned by anyone, but instead we speak of stealing when one downloads a torrent and then creates a new torrent, only to get the credits for it. Again this is one of the worst inventions in the history of BitTorrent, or in fact P2P in general, stealing torrents has a really bad impact on the performance of a torrent. The base principle of BitTorrent is that for a single file we have a large user base that share the file amongst each other. By stealing a torrent you split the userbase in many different smaller userbases that do not share amongst each other, thus making a death of the torrents more likely. Re-Seeding is no good either. Re-Seeding means that a torrent that is almost dead is recreated with a new torrent file and registering it in the tracker once more. It tends to split the userbase just like stealing the files, thus speeding up the death of the torrent. As long as there is at least one active seeder it is not a good idea to artificially enforce a republishing of the file, making it temporarily available in tracker news, as this is the only effect it has. Also reseeding tends to confuse the users as they don’t know anymore which file to download and what exactly the difference is.
Don’t stop seeding!
This is easy to understand, how do you expect others to keep on seeding when even you don’t seed anymore? Also you ensure that there is at least one full active copy around, therefore encouraging new users to download your file (files that have no seeds active are less popular and most likely to die out soon). Keeping the Torrents active is hard work but it’s worth it. We are gearing towards quality torrents and not sheer mass. By creating this small effort on the side of the seeder we ensure that the things that he publishes are really worth it and not just the usual junk for the masses. Off course it’s not needed that all the torrents a seeder has ever published are in his sharing queue, it’s enough that the seeder checks periodically which torrents need a seeder (I strongly recommend that at least a 5% of seeders should be the absolute minimum) and if needed he just puts the torrents into action. I’m currently working on a tool that allows Seeders and others to create RSS-Feeds with theis torrents to watch , and therefore making it easier to coope with the duties of a seeder.
Don’t use Multitracker
As said before splitting the userbase into smaller one is a really bad thing to do, and the most common error while creating a Torrent is that of specifying multiple trackers. This may sound as a good idea, because if the main tracker is down there are still the fallback trackers, but, if you chose a good tracker, it is really unlikely that the tracker becomes unresponsive or is taken down completely. If a BitTorrent client can’t contact the first tracker he will attempt the others, thus distributing the users over more trackers and slowing down everything. Many believe that if a client is given a multitracker torrent it will check all torrents specified and connect to them all, thus creating a sort of meta-tracker that is made up of the others, but for most clients this is not true, they will just select a single tracker to which they connect. With more and more clients supporting Azureus’ DHT Tracker the need for multitracker torrents is non-existant, because if a tracker is down, one just uses the DHT tracker which is never down.
Sadly most BitTorrent Services, are shortlived, Torrents are kept on the main page only for a few days or even less, and therefore they are published only for that time. I’d like to thank mininova and ThePirateBay for taking another aproach, away from the shortlived "publish-on-mainpage" and more an archive of good and working torrents. If we want to use the BitTorrent infrastructure for serious distribution of information, this is the way to go, information should be easily accessible , not only for a short period of time!
For some time now my SuSe Linux 9.2 Professional installation has become slower and slower, partly because I filled it up with programs and partly because I messed around with the X-Server and KDE. I was pretty new to Linux and so I had some problems learning how things are done the Linux Way
Now I think I could manage a Linux System and keep it clean from birth to death (of the Notebook, not the OS )
I got a bit pissed off with SuSe because of the problems I had, and was therefore looking for another distro, but then I found openSuSe which looks nice and I can keep some features I really like in SuSe:
SCPM: Profile Switching has never been as easy as with this tool.
Yast: Although I started to hate its slowness, it’s a really good tool, aggregating all the functionality in a single place.
Backing up my stuff
Before starting the installation I had to back my stuff up, especially the crucial data such as Thunderbird mail folder and Firefox settings. I just copied my entire homefolder to another machine, I’ll only have to remember not to overwrite some hidden folders (like the .kde folder…) when copying back.
Software list
Now it’s time to decide what to install. There are a few applications I absolutely need:
) so it shouldn’t be too hard to get it up and running, just make sure that the source password in /etc/icecast.xml matches the password you specified above, in the /etc/mpd.conf file.
