Search This Blog

Wednesday, June 11, 2014

Replace a RAID 5 disk that has failed (Linux / Ubuntu)

If you take a look at my last couple of blog entries, you'd know that I had a hard drive that was approaching imminent failure:
I got the new drive in the mail from Amazon which was a different model, but the same size (2.0 TB) WesternDigital Green (WD20EZRX).  Once I decided that using a SATA 3 drive on a SATA 2 bus was going to work, I went for the purchase.

On to the replacement:
Using mdadm, tell the Linux RAID to not recognize the disk as usable:
mdadm --fail /dev/md0 /dev/sda1

If you are like me, and you don't know which one is which, use the Disk Manager tool and write down the serial number of the drive.  This will correlate to the number on the printed label of the physical drive.  Note: it is handy to tape a piece of paper to the inside of your computer listing all of your drive serial numbers and the associated partition for future reference.  I actually had forgotten that I did this the last time a drive failed, wrote down the serial number of my drive, and then realized the paper was in the computer.

Power down the machine, remove the faulty drive, and replace with the new one.

Once the drive is replaced, power on your computer.  You should see a /dev/md0 fail event upon startup.  Mine said something to the effect of 3 out of 4 devices available, 1 removed.. etc.

Next, format the new drive with fdisk:
sudo fdisk /dev/sda

This will bring you into the fdisk program.  Type m for the help menu and available input options.  Perform these in order:
p - print the current configuration and verify there is no partition already.  This is a quick idiot check to make sure you are configuring the correct drive.
n - new partition
p - make it a primary partition
<enter> - accept the default start sector (should be 2048)
<enter> - accept the default end sector (should be the end of the hard drive)
t - change the type of the partition
fd - make it a Linux RAID autodetect
p - verify all of your settings are correct.  It should look something like this:
w - write your changes to the file

This will write the new partition table, exit fdisk and return you to the command line.  Execute partprobe to ensure your system will recognize the new partition.

Tell mdadm that the drive is now available:
sudo mdadm --add /dev/md0 /dev/sda1

Your data from the other 3 drives will now be rewritten across the new sda1 drive.  This will take some time, but can be monitored:
watch cat /proc/mdstat

It is important to leave your machine on and uninterrupted until the rebuilding process is complete.

Aren't RAID 5's a beauty?  I love having automatic hardware failure protection... assuming not more than 1 drive fails at a time.  I hope you found this useful.  If you have any questions or comments feel free to post in the comments below.

Next up will be to create a RAID 1 using my existing system drive and a spare unused drive I've had sitting around.... without losing any data.  Should be fun!

Tuesday, June 3, 2014

Checking and repairing a RAID in Linux

Recently I've been having a weird issue where I will sit down at my computer after it has not been used in a while and it shows a black screen with a blinking "_" in the top left corner.  The only thing I can do to recover from this is to issue the Alt+Prt Sc+REISUB to force an emergency file system sync and reboot (click the link for details on all the inputs).  Once the machine was back up, I quickly started researching what caused the issue by checking out dmesg and kern.log. I also ran some smartctl tests and noticed there were some bad blocks on my RAID 5 (4x2TB).  I started down the rabbit hole of repairing bad blocks, only to find out I could be causing more harm than good.  I vaguely remember attempting this before on a non-RAID, and ending up with more unusable blocks than when I started.  Before doing too much damage to my RAID, I decided to do some more research.  Turns out, with a Linux software RAID (mdadm), I can easily find and repair my issues using one simple command:

sudo echo 'check' > /sys/block/md0/md/sync_action

Of course, my RAID is on md0, so change this to wherever your mount your disks if different.  It is wise to do this while the volume is not mounted (sudo umount /dev/md0), otherwise you risk damage.  This command will start the filesystem check but will not keep you up to date on its progress.
To check up on the progress, issue:

watch cat /proc/mdstat

This will take a long time, depending on the size of your drives; mine started out with ~290 minutes to finish.  To quit watching, Ctrl+C.

To pause the check:

sudo /usr/share/mdadm/checkarray -x /dev/md0

Resume:

sudo /usr/share/mdadm/checkarray -a /dev/md0

Once it has completed, check the mismatch count:

cat /sys/block/md0/md/mismatch_cnt

If output returns 0, then you're all set and your RAID array should be as repaired as it can be.  If it returns something other than 0, you can synchronize the blocks by issuing:

sudo echo 'repair' > /sys/block/md0/md/sync_action
watch cat /proc/mdstat
And, once the repair is complete, check it again:
sudo echo 'check' > /sys/block/md0/md/sync_action
watch cat /proc/mdstat

For more info, check out the Thomas Krenn Wiki.

Thursday, April 24, 2014

Make magnet links work in Xubuntu

When trying to open magnet links in Xubuntu, sometimes you will get an error.  For example, when searching in Catfish and a folder is clicked, I got this:
"Unable to detect the URI-scheme of /home/user/folder/folder".

You might also get this in Chrome when trying to open a magnet to a torrent file.  For some reason, Firefox works fine with magnet links (probably uses gnome-open instead of the system's opener by default).

To fix the problem, edit /usr/bin/xdg-open: sudo gedit /usr/bin/xdg-open

In there, find the lines that look like this:

if [ x"$DE" = x"" ]; then
    DE=generic
fi
And change it to this:
if [ x"$DE" = x"" ]; then
    DE=generic
fi
#xdg-open workaround for bug #1173727:
DE=gnome
This will force Xubuntu to think you are using the Gnome Desktop Environment, and will in turn use gnome-open instead of xdg-open.  When Xubuntu detects the XFCE display manager, it calls exo-open "$1" which is not capable of handling magnets.  This workaround will get you going until the bug has been fixed.

Monday, April 21, 2014

Replace Ubuntu One with Insync

Ubuntu announced the pending shutdown of their Ubuntu One services recently.  I depend on Ubuntu One to sync my ~/Documents folder between my desktop and laptop, and have even bragged to Windows users how easy it is to get files from one computer to the next using Ubuntu One.  It seems that Canonical does not make enough money through their services, would rather focus on other projects, and so are shutting it down completely on June 1st.  This led me to finding an alternative solution.

Google Drive allows one to upload up to 15 GB of data to the cloud for free (100 GB is only $2/month).  There are many services out there that do the same sort of thing, such as Dropbox or Amazon Cloud Drive, and a quick search will reveal other cloud storage solutions.  The major downfall to using something other than Google Drive is that the common amount of free space given for storage is somewhere between 2 to 5 GB, with steep prices if you wish to expand.  I am an avid Android user, so I already make use of my pictures getting automatically pushed to Drive, as well as awesome Gmail integration.  So for me, the decision was simple: find a way to sync my two computers using Drive.

Introducing InSync.  At the most basic level, it allows a user to automatically sync data between a specified folder on their computer (/home/user/Insync, or whatever you desire to name it) and Drive.  This is useful, but other services offer the same thing (Grive, SyncDrive, etc).  But where InSync prevails is all the other stuff it is capable of:

  • Automatic conversion of Google Docs to Office (LibreOffice/OpenOffice compatible)
  • Built-in sharing without a browser
  • Recent changes feed
  • Window manager integration (ie Nautilus) - right-click a file to sync
  • Symlink, junction and alias support (key feature; more on this below)
  • Multiple Google account support
  • Watch any folder for changes
  • Support for almost every platform
  • And many others
As you can see by their features, InSync has done a great job at integrating Google Drive into the desktop environment.  Back to my original issue, however, is that I needed to sync two computers to be mirror images of each other.  This process is not glaringly simple in InSync, so this is how you do it:

  1. Ensure all your files are synced between the two machines using Ubuntu One or other methods
  2. Download InSync on one of the computers (we'll say desktop, for ease of explanation).
  3. Complete the installation, choosing the "Advanced Setup" when prompted
  4. Authenticate InSync with your Google account and choose where to store your files (/home/user/Insync).
  5. Once InSync has finished installing, Nautilus users (Ubuntu) may want to install package insync-nautilus in order to have right-click menu integration.
    1. sudo apt-get install insync-nautilus
    2. Restart Nautilus by clicking the prompt once that completes, or by logging out and back in.
    3. Another method is by opening terminal (Alt F2) and typing nautilus -q. This will kill the window manager.  You start it back by opening a folder either from Ubuntu's side bar, or pressing the Super key and typing home, and opening the folder.
  6. Now the fun part: create a symlink inside your InSync folder to your Documents folder:
    1. Right-click Documents in your home folder > Add to Insync > Your Google Account
    2. OR: ln -s ~/Documents ~/Insync/Documents
  7. This will sync all of your Documents to Google Drive.  Give it some time to finish.  Once the InSync icon in the toolbar looks like: 
    or 
    you are ready to proceed.
  8. On your other computer (laptop), prepare it for syncing by installing InSync, and choosing Advanced Setup again.  Do not put anything in the InSync folder, as your desktop and laptop are currently mirror images of each other via Ubuntu One.
    1. Don't forget to install the Nautilus integration and restart Nautilus (step 5)
    2. Go to InSync > Your Google Account Name > Settings > "Selectively sync your files & folders"
    3. Choose the Documents folder from your Google Drive, and Apply Changes
    4. Allow all of your Drive's documents to sync to your laptop
  9. On the laptop, rename ~/Documents to ~/Docs
    1. mv ~/Documents ~/Docs
  10. In the terminal on your laptop:
    1. ln -s ~/Insync/Documents ~/Documents
    2. This creates a symlink from Insync's Documents directory to a non-existing ~/Documents on your Laptop. (note** non-existing because you have renamed the ~/Documents folder on step 9)
    3. Note that this symlink is opposite of step 6.2, because we want to create the illusion of a ~/Documents folder that is actually under ~/Insync/Documents, allowing all previous shortcuts to continue working.
That's it! Your files on your laptop and desktop will stay in sync with each other.  

Now go disconnect Ubuntu One! 

Monday, January 6, 2014

Google Chrome profile error on Ubuntu

Lately Google Chrome has been acting strange for me.  Every few times I would open Chrome, I would get an error that "Your profile could not be opened correctly".  However, it was very inconsistent, so the cause was hard to nail down.  When this would happen, I would get about 10 error windows stacked on top of one another, I would press OK several times, and then have to sign in to my profile again using the settings menu.  After a while this started to annoy me, so I started to hunt down a solution.

I am using Chrome 31.0.1650.63 and Ubuntu 13.04 (Xubuntu variant), although the below fixes should apply to most versions until Google fixes the issue.  To check your versions in

Chrome: Go to Settings ()  > About Chrome

Ubuntu: In terminal, lsb_release -a


The fix for me was to kill all the zombie processes that Chrome left behind from the last time(s) it ran.  The easiest way to do this is:
pgrep -l chrome

Make sure that all the processes shown are only chrome.  If they are, then:
pkill chrome


You may also try:
killall -9 chrome
Although this may not work if the pkill chrome command did not work.

You should now check that all chrome processes were killed by issuing the same pgrep -l chrome command.  If so, then restart Chrome and see if you get the same error.  If there are still processes, you may have to go in more depth to make it go away:
ps -ef | grep chrome
Many results may show up, but look for the PID, which is the second column of each line:
benmctee 11841     1  6 07:30 ?        00:02:23 /opt/google/chrome/chrome

11841 is the process ID (PID).  Issue the kill command as follows:
kill -TERM 11841
kill -SIGTERM 11841
kill -15 11841

Repeat the above kill process for each PID that appeared in the ps -ef | grep chrome command.

An alternative way to go about this is to use the top command:
top (opens the process manager)
s (goes to the sort screen)
The display will show all fields like the following window:
Notice how the top line shows %CPU on the far right.  This is what top is currently sorting by.  I don't really care at this point which process is causing the most CPU usage, since I know that we want to kill the Chrome process.  Scroll down to COMMAND, and press s.  Your window should now show COMMAND in the upper right:
Press q to get back to the main top screen.  Everything will be sorted now, but in reverse order:
 Type uppercase R to sort everything alphabetically, and then scroll down to chrome.

Notice that there are a multitude of instances that get started when Chrome runs.  To kill the processes, type k, and then enter the PID of the process you desire to kill, and then press <enter>.  To quit top, type q.

Killing each process can take some time, which is why I prefer to use the pgrep and pkill commands from earlier.

If you still have troubles with Chrome, you can follow other steps as outlined in Saravanan Thirumuruganathan's article over on Wordpress.

Thursday, January 2, 2014

Garmin GPSMap 62S under Ubuntu

I recently picked up Geocaching, thanks to my wonderful parents. It is a highly addictive sport in which caches, mostly small containers with (at a minimum) a paper log sheet inside, are hidden all around the globe. Once a cache is hidden, the cache owner will post its coordinates to geocaching.com, at which point other cachers can use a GPS receiver (GPSr) or a GPS enabled smartphone with the geocaching app installed to search for it. Once found, the cacher will log it as found on the app or website, sign the log, and move on to the next cache. The smartphone way is great for beginners, but oftentimes one must log a DNF (did not find) because phone accuracies are usually no better than 16 ft. This is where the use of a handheld GPSr comes in handy. Models like the Garmin GPSMAP 62s have a geocaching feature, and users can log a find with it and then upload that data once back home and connected to a computer.

This tutorial is meant for users of a Garmin GPSr who also use Ubuntu.  It will install the Garmin plugin, as well as QLandkarte GT, a very useful GPS program written for the Linux OS.

Install:

sudo add-apt-repository ppa:andreas-diesner/garminplugin
sudo add-apt-repository ppa:mms-prodeia/qlandkarte
sudo apt-get update
sudo apt-get install garminplugin qlandkartegt qlandkartegt-garmin

For the most up-to-date location of install files, head on over to:
https://launchpad.net/~mms-prodeia/+archive/qlandkarte
You will still need to add the repository, like in the above steps.

A quick run down of someone using QLandkarte GT for geocaching:
http://sourceforge.net/apps/mediawiki/qlandkartegt/index.php?title=A_Grand_Day_Out

Open QLandkarte by searching for it within the Ubuntu window, or execute qlandkartegt from terminal.  We must first get a map to use within the program.  There are many ways to go about this, and are all based on your preference.  My first stop would be GPS File Depot.  They have many custom maps for Garmin that are ready for download.  Another way to obtain maps if they don't have what you are looking for is to export one directly from OpenStreetMaps (OSM).  The problem with doing it directly from their website is that you are limited to a small number of tiles (I was unable to download the entire island of Oahu).

In order to download larger regions than what OSM allows, I recommend using this link to select the region you would like to download.  From there, you select the region / state / etc, and then go to the download page.  Since we will be using QLandkarte GT, download the "osm_generic_tiles.zip" file.

  1. How to load a map and caches into QLandkarte GT:
    1. Copy this URL: http://mt.google.com/vt/x=%2&y=%3&z=%1
      1. I originally starting writing this article based around Google Maps, but after reading this, I decided that the best support is given (and free) by OpenStreetMaps.
    2. Download your desired region from the link above or here.
    3. Once your zip file has downloaded, unzip the contents into your desired directory (/home/user/geocaching/maps).
    4. In QLandkarte: File > Load Map
    5. Select the file with .tdr as the extension.  Another file dialog will open, and it will ask for an img file, where you will choose the filename_mdr.img (ie 63240000_mdr.img).  Your map will now load into the software.
    6. Generate a Pocket Query on Geocaching.com for the caches you wish to load to your GPSr
    7. Download the pocket query, unzip the GPX file(s), and load those into qlandkarte
      1. File > Load Geo Data > select the GPX file.
      2. To add a second GPX file: File > Add Geo Data.  You will have to choose 'Add Geo Data' vice 'Load Geo Data' when loading waypoints as the GPX data will replace the caches you loaded in the previous step.
    8. You should now have all of the caches that were created in the pocket query layered on top of the Map in QLandkarte GT
  2. Exporting Caches:
    1. File > Export Geo Data > name the file something.gpx (I used 20130311.gpx to indicate the date on which it was created).
    2. Ensure all waypoints are selected, and click OK.
  3. Exporting the Map:
    1. Ensure you are still on the map tab
    2. Choose Map (menu) > Select Sub map
    3. Select an area to export.
    4. Click the name of the selection in the Maps tab, and then click Export map
    5. Select the folder you wish to export to, create a name, select Garmin Custom Map, and click Export
    6. Connect your Garmin to your computer and copy the newly created img file to the Garmin folder on your GPSr's SD card.  Please note that you must have the same directory structure on the SD card as in the default Garmin internal storage.
      1. As you can see, I have two volumes mounted, GARMIN and 7.9 GB Volume.  The 7.9 is the SD card and the GARMIN is the GPSr internal storage.  I drop the .img files in the 7.9\Garmin root directory, and the cache file into the GPX directory.  The CustomMaps directory is empty, and is probably left over from previous experiments.  I had no success in being able to open the custom map on the GPSr when I copied the img file to that directory.
    7. While you are here, also upload the GPX file to the GPX folder.  Both the GPX folder and the Custom Maps folder are under the Garmin directory.

That's it!  Unmount the GPSr and try it out.

If you run into problems with QLandkarte not doing something because of permissions, try running it as sudo from the terminal: sudo qlandkartegt

As always, please leave a comment below if you find any of these instructions incomplete or wrong.  I would greatly appreciate a more efficient way of going about updating the Garmin within Ubuntu, so if you have any tips, please let me know.  I keep looking at Open Cache Manager as a way to manage things the way the Windows folks do with GSAK, but it has no map export feature as of yet, so that's off the table.  However, it's a great cache manager tool, and you should check it out at their website.