Search This Blog

Tuesday, November 30, 2010

Media Server - The Media

Now that the RAID is setup, ready to receive all sorts of media, now's the time to start populating it.

I will be using Transmission BitTorrent client to download torrents, TED to monitor for new TV shows, and MediaTomb to share this media to things on my network (computers, PS3, etc.).

Transmission comes with Ubuntu Desktop, so that's easy.  Just open it (Applications >  Internet > Transmission), go to Edit > Preferences.  In here, it's pretty self-explanatory.  I created a Torrents directory under /mnt/Media (My RAID partition), which has Downloading and Complete folders in it.  I've set Transmission to monitor the Torrents folder for new .torrent files, the Save To location is Complete, and Keep Incomplete in Downloading.  From here, TED will be configured to download .torrent files to /mnt/Media/Torrents.

Ensure that the Java Runtime Environment (jre) is installed using Synaptic Package Manager, then download TED.  Once it has downloaded, go to the folder and unzip it.  Delete the MacOS, Windows, and Sun folders.  Right click on ted.jar, and click Open With OpenJDK Java 6 Runtime.  Now, configure TED: Extra > Preferences.  "When ted finds a torrent..." browse to your .torrent location (ie /mnt/Media/Torrents), and select that folder.  Now TED will download the .torrent file, and Transmission will automatically download it.

For sharing, I am using MediaTomb.  You will also need ffmpeg and VLC to enable media conversion on the fly.  Synaptic Package Manager is the best choice for this.  Once installed, download MediaTomb (also via SPM), and then follow these instructions.

References:
http://www.geek.com/articles/chips/feature-linux-media-server-using-ubuntu-810-2009065/
http://robertpyke.blogspot.com/2008/07/ubuntu-mediatomb-vlc-and-ps3-match-made.html

Thursday, November 25, 2010

Media Server - Setting up the RAID 5

Now the fun part...

Moving right in to creating the RAID 5 for all the media:
sudo fdisk /dev/sdb
n   (to create a new partition)
p   (for primary partition)
1   (to create partition 1)

....If for some reason at this point it tells you there is already a partition there, that's fine.  Just press d, and it will delete it, then go back to "n".

<enter>   (to begin on cylinder 1 of the hard drive)
<enter>   (to end on the last cylinder of the hard drive)
t   (for type of partition this will be)
fd    (to make it a RAID autodetect)
w    (to write changes)

Repeat these steps for each hard drive you want in the array... (ex: /dev/sdd, /dev/sde, /dev/sdf)

Now you can either use apt-get install, or use the Synaptic package manager (whichever you prefer), but we need mdadm to proceed to creating the RAID.  If you are using this as a tutorial, mdadm was used to create the RAID 1 for the OS drives.

sudo mdadm --create --verbose /dev/md0 --level=5 --raid-devices=4 /dev/sdb1 /dev/sdd1 /dev/sde1 /dev/sdf1

This will start creating the RAID 5 (--level=5) using 4 hard drives (--raid-devices=4), on their first (and only) partitions (/dev/sdb1 /dev/sdd1 /dev/sde1 /dev/sdf1)

If you want to see the status of the RAID build, use:
cat /proc/mdstat
It should say something like:
md0 : active raid5 sdf1[4] sde1[2] sdd[1] sdb[0]
          5860535808 blocks level 5, 64k chunk, algorithm 2 [4/3] [UUU_]
          [>............................]  recovery =  0.5% (11036672/1953511936) finish=545.0min speed=59392K/sec
unused devcies: <none>

This is saying that its creating the RAID.  This will take some time, depending on the size of your hard drives, how many you are using, etc.  Make sure that your power settings will not sleep your computer after a period of time, and don't do things like system updates that will cause a computer restart. It's best just to leave it alone until the process is complete.

If you want to continuously watch the status (ie: you don't have anything to do for several hours):
watch cat /proc/mdstat
Once you are bored with that <Ctrl+c> to get out of watching it

Sweet... you have a RAID!  However, nothing will be managing it.  Tell mdadm to scan the raid config file or /proc/mdstat for missing information:
sudo mdadm --detail --scan >> /etc/mdadm/mdadm.conf

Go into this file and comment out (or delete) any other entries with /dev/md0 in them:
sudo gedit /etc/mdadm/mdadm.conf
You should see the "ARRAY /dev/md0..." line at the very bottom.  Any other lines in this file will need to have a "#" in front of them so that md0 will get mounted properly.  Also, I took out "metadata=00.90" because it gave me errors, and everything still seems to work fine.

Now we need to format the RAID:
sudo mke2fs -t ext4 -j /dev/md0
-or-
mke2fs.ext4 /dev/md0  (this is the one that worked for me)
This will make an ext4 filesystem with journaling on the RAID (/dev/md0)
If you are using ext3 filesystem, leave off -t ext4

Create a mount point anywhere on the system for your md0:
sudo mkdir /mnt/Media
Putting it in /var means that everyone can read it.  You may also want to put it in /mnt and then change permissions later.

Edit the fstab so that the RAID will get mounted each time the system boots:
sudo vi /etc/fstab
Ađd the following line to the bottom of the file:
/dev/md0 /var/media ext4 defaults 1 2
From here you can reboot to see if the RAID gets mounted at /var/media, or you can mount it now without a reboot:
mount -a
This will read from the fstab and mount everything from it.

Hopefully this was all very helpful.

End notes:
I went back to not using a RAID1 for the OS. After giving it a day of thought, there really is no reason for me to waste a 250GB hard drive on something that can easily be reinstalled and configured from a CD.  Plus, I've been running Ubuntu on my MacBook for over a year now with no issues, so there should be none on the server.  If the time comes down the road where the hard drive where Ubuntu is installed fails, I would just replace it with a new one and reconfigure everything from a backup that will reside on the RAID.
That all being said, if you used a RAID for the OS, then the RAID for the media would be md1 for the above steps that reference md0.

Reference:
sonniesedge

Wednesday, November 24, 2010

Media Server Update - Setting up RAID 1

To set up the OS on a RAID 1 (the 2 250GB HDDs), I followed this wiki pretty much verbatim:
https://wiki.archlinux.org/index.php/Convert_a_single_drive_system_to_RAID

To get to this point, however, I simply installed Ubuntu using all the normal setup options onto a single 250GB hard drive.  I have a 100GB partition for the ext4 filesystem, a 10GB partition for swap, and whatever was left over for future use.  This was on /dev/sda (sda1 being the 100GB, sda2 being the 10GB).  So when I went to go set mine up, I started with a fresh wipe of /dev/sdc - nevermind the incorrect cable configuration - and followed the wiki.  The only difference I noted was that he created the swaps physically before the OS on the hard drive (starting with cylinder 1), so when the wiki talked about created a swap at /dev/sdb1, I simply did mine at /dev/sdc2.

I'm using a later version of Grub, so I did not configure the menu.lst (Grub 2 will do this for me automagically).

Quick tips in the alter Fstab part:
blkid will get you the UUIDs of the hard drives... but why would you want to break out the pen and paper, then risk the chance of getting a character wrong? Try this:
mv /mnt
cp /mnt/new-raid/etc/fstab fstab.txt
blkid >> fstab.txt
This copies the current fstab and appends the output of blkid to the end of the fstab file
Now, pear it down to the info you need:
vi fstab.txt
Comment out the old stuff, and make the new stuff match the old stuff (ie: take out the "...", labels, etc.)  New to VI? Take a look at these basic vi commands.

After running mdadm --detail --scan >> /etc/mdadm/mdadm.conf
you may like to vi that file as well to remove the metadata=00.90 (at the very end) to avoid getting an error.  If not, update-initramfs will just throw a flag and then say it ignored it... which shouldn't be an issue

After running the perl script, it should come back with nothing.  To exit the perl script command line, Ctrl+C

Tuesday, November 23, 2010

Media Server Update - (cont from Part 1)

To get the most up to date video output, I attempted enabling propriety drivers under System > Administration > Additional Drivers.  Once I rebooted, it took me straight to a command line vice the nice Ubuntu Desktop GUI.  My advice: do NOT use the drivers Ubuntu recommends (atleast while running 10.10 and with a GeForce GTS450 video card).

My workaround:
Download updated drivers from here to your home directory (~/).
sudo stop gdm
sudo apt-get remove nvidia-current
cd ~/
sudo ./NVIDI<tab> -a   (something like NVIDIA-Linux-x86_64-260.19.12.run
startx  (or sudo shutdown -r now)

The ./ before NVI... is important, don't forget that or it won't run.  Also, if it won't run, try ls -a.  If it doesn't atleast say -rwx, then you can't execute it. So:
chmod 755 NV<tab>
Now try to run it again.

I got an error saying the pre-install script failed, would I like to continue?  I chose yes, clicked yes on all the other prompts, then "sudo reboot".
Once rebooted, the graphical desktop was back.  Not to say I don't like command line stuff.... I just would rather only use it when needed, and not for everything.

More trials and tribulations to follow.

Ubuntu Media Server - Part 1

My Goal:
1) Create a home media server that can stream video across the LAN to an iPhone, PS3, TiVO, and several computers
2) Use the same computer to display high definition video and output full surround sound
3) Possibly use for a gaming platform (not a must - but could be nice)

My Setup:
Motherboard - Intel DP55KG
    -Carries up to 16 GB of RAM
    -Supports the Intel i7 processors
    -Has optical audio (TOSLink) in/out
    -Has 8 SATA ports for many HDDs (plus 2 eSATAs on the back)
    -Built-in bluetooth module (convenient; one less thing to purchase to make my keyboard / mouse work)
Processor - Intel i7-870 (1156 chipset)
Memory - OCZ 3P1600LV4GK
    -4 GB total
    -I originally went with the Corsair Dominator 8GB via 2 DDR3 sticks, but thought to be having parity issues and no boot.  After sending those back and ordering the OCZ, I had the same E7, E9, and 5A postcode errors.  After further research, turns out my video card was not properly powered.  In the future, I may go back to the 8GB sticks.
Video - Nvidia GeForce GTS450
    -This gets me 2 DVI outs, and 1 mini HDMI out
    -Mini HDMI to HDMI cables are easy to find; ~$20 on Amazon
Hard drives -
    - 2 SATA Seagate Barracuda 7200 250GB for the OSes (100 GB for ubuntu, 10 GB for swap, and ~140GB for possible future Windows install)
    - These are setup in a RAID 1 format
    -4 SATA Western Digital WD20EARS Caviar Green 2 TB
    - These are setup in a RAID 5 format, with 1 spare for all media storage

Misc:
Case - NZXT Phantom Full Tower (has remove enough for the 6 hard drives and plenty of external connections.  I love how many fans this thing has and is capable of.  Currently, out of the box, I have the Processor heat sink fan, an 80mm in front, 2 80mm on the side, and 1 200mm on the top (not including the PSU and Graphics card fans)... and this running with all HDDs is still very quiet with very little power draw.  My APC unit is showing a range of 110-140 Watts power draw.
Processor heat sink - Corsair H50 high performance heat sink
 - self contained water cooling... the fan / radiator on the back of the case has 2 hoses that go down to the top of the processor.  Internally there is a pump.  Hookup is quite simple and all necessary adapters are included for common processors
PSU - Ultra X4 750W ATX Power Supply

More to follow on the Ubuntu configuration.  I will be using 10.10 (Maverick Meerkat) with a variety of software to automate my streaming and organizational processes as much as possible.