Thursday, October 24, 2013

How to backup/wipe your hard disks using dd

I recently sold my old Dell Inspiron 1420 and bought a shiny new Lenovo Thinkpad T430. Now I intend to write a separate post about my experience with this new machine, but in this post I would like to cover some basics of the command line tool dd. This tool was extremely handy and helped me to securely wipe all data on my old hard drive before I parted with it. You can never rely on plain formatting to ensure that all sensitive information (personal files, saved passwords, etc.) from your old hard disk has been erased. However, there are a lot of tools out there that are an overkill for this task. This is where dd steps in - it is simple and present in most Linux distributions including the Ubuntu Live USB. However, it is also a very dangerous tool so make sure that you understand what you are doing before you blindly copy paste the commands into your terminal. You may end up with your data being permanently lost. So let's begin.

First of all the, basic structure of dd. A typical dd command will look like this:
user@pc: ~$ sudo dd if= of= bs= conv=
Here, if stands for the input file (could also be a partition or an entire drive), of for the output file (could also be a disk partition or an entire drive), bs stands for byte size (default value is 512) and the conv stands for conversion options. Specifying the byte size and conversion options are purely optional but highly recommended because of the following reasons:

a. The default value of byte size will be extremely slow. However, a little tweaking of this default value can increase the speed of the backup/wipe operation by up to 25 times! For example,
sets the block size to 4k, an optimal size for hard disk read/write efficiency and therefore, cloning speed.

b. Because dd is not very user friendly, it will not print any progress report while it running. But that is not the worst part, it will simply stop if it encounters a bad sector in your hard drive. The conversion options allow us to modify this behavior. For example,
The noerror option all dd to just write zeros when it comes across bad sectors and then continue with the rest of the data. The sync option ensures that the sectors on the target and source device are aligned. The notrunc option or 'do not truncate' maintains data integrity by instructing dd not to truncate any data.

If you wish to know more, I highly recommend reading the wiki here (German only) or here.

Important: Always find out the correct hard drive name/partition using sudo fdisk -l or sudo blkid. Incorrect use of the dd command can result in permanent loss of data!

a) Backup
If I wish to backup my linux partition which is /dev/sdb1 to a file on my external hard drive, the dd command will look like this:
user@pc: ~$ sudo dd if=/dev/sdb1 of=/media/EXTERNAL/hdd_backup.img bs=1M conv=noerror,notrunc,sync
b) Erase
If I intend to erase all data on my hard drive so that no one can recover it, I simply have to write the entire hard disk with zeros. Or better, I can write random values all over. Even better, I can run multiple passes of the same command and I will know that the data is beyond recovery even by . However, the process to generate and write random values is time consuming and unless you are a spy who is trying to destroy vital information, writing zeros (with 2 passes instead of 1) is good enough for the rest of us. Also, destroying the hard disk physically would be a better options in the former case. Let us say, I have to wipe the data on my /dev/sdb. I have two options, namely
Writing zeros:
user@pc: ~$ sudo dd if=/dev/zero of=/dev/sdb bs=1M conv=noerror,notrunc,sync
Writing random values:
user@pc: ~$ sudo dd if=/dev/zero of=/dev/sdb bs=1M conv=noerror,notrunc,sync
It goes without saying that you will need to boot from a Live USB in order to be able to wipe your primary hard drive. At this point, I should also mention that the byte size value is (obviously) mentioned in bytes but the following suffixes can be used for convenience.

KB1000 (ie 1KB equals 1000 bytes)
K1024 (ie, 1K equals 1024 bytes)
MB1000000 (= 1000 * 1000, that corresponds to 1000000 bytes 1MB)
M1048576 (= 1024 * 1024, 1M equals 1048576 bytes)
GB1000000000 (= 1000 * 1000 * 1000, ie 1GB equals 1 billion bytes)
G1073741824 (= 1024 * 1024 * 1024, d, h, 1G byte corresponds 1073741824)

Now, a little searching online led me to this website which tries to identify an ideal byte size (bs=131072) in order to optimize the speed of operation. But because I had not though this through I also have some of my own statistics to share. Because I had two disks, I could also compare the speeds using SATA and IDE interfaces. Here are the results:

SATA 5400 RPM Data written (GB) Time taken (mins) Average Speed (MB/s)
sudo dd if=/dev/urandom of=/dev/sdb 83 399.5 3.5
sudo dd if=/dev/zero of=/dev/sdb 159 115.0 23.0
sudo dd if=/dev/zero of=/dev/sdb bs=131072 750 165.5 75.5
IDE (PATA) 5400 RPM Data written (GB) Time taken (mins) Average Speed (MB/s)
sudo dd if=/dev/urandom of=/dev/sda 56 333.9 2.8
sudo dd if=/dev/zero of=/dev/sda 71 115.2 10.3
sudo dd if=/dev/zero of=/dev/sda bs=131072 160 72.5 36.8

As is pretty evident, the writing of randomly generated data is extremely slow and it will be faster to perform two passes of zero writes instead of one random write. Not to mention, it is next to impossible to recover data from a drive written with zeros, so using random values is pretty much overkill for the average Joe. Further, a change of byte size from the default values can speed up the entire process by up to 4 times.

Monday, October 21, 2013

How to fix missing sound indicator in Xubuntu 13.10

After my recent upgrade to Xubuntu 13.10 Saucy Salamander, I noticed that the sound indicator menu was missing. The volume indicator seemed to think that the volume was muted, and presented no drop-down menu when clicked. The sound was working fine and so were the multimedia keys on my keyboard. After some searching on the internet, I found that this was a more common problem. This bug has been reported on launchpad as well.

The workaround mentioned is to edit the service file in the following fashion. First bring up the terminal window and type the following command (replace mousepad with the editor of your choice)
user@pc: ~$ gksudo mousepad /usr/share/dbus-1/services/indicator-sound.service
The service file will now open and its contents will look like this:
[D-BUS Service] Name=com.canonical.indicator.sound
Now, all you need to do is to comment out the last line and enter a new one instead
[D-BUS Service] Name=com.canonical.indicator.sound
Save and close this file and reboot your computer.

Please note that use this method only if you are using XFCE as your sole desktop environment. If you also use other DEs, for e.g. Unity, along with XFCE, then this method will break the sound indicator in your other DEs. In this case, you should follow the steps outlined in this article.

While we are at it, I might as well list some of the known issues in Xubuntu 13.10
  • indicator-sound no longer functions with xfce4-indicator-plugin (1208204)
  • Gmusicbrowser's albuminfo-plugin is deactivated by default and causes the app to hang if enabled (1223808)
  • Restart button fails to work in Update Manager (1232363)
  • User Administration - a new User is added correctly, but Administration app crashes on close (1185396)
  • Lock screen slow to appear on resume from suspend (1229486)

Wednesday, October 16, 2013

XFCE Panel: Left Alignment Workaround

After my recent upgrade to Xubuntu 13.04, I noticed that my lower panel was left aligned. Although it is completely functional, I want my panel to extend to cover the entire width of the screen. It just makes it easier for me to locate the indicators for volume, notes and gmusicbrowser. Also, the panel keeps resizing as I open more applications which I find a little annoying. Here is what I did.

1. First of all, we need to fix the length of the bottom panel. To achieve this, right click on the bottom panel and navigate to Panel > Panel preferences. In this window make sure that the Length (%) is all the way up to 100% and the "Automatically increase the length" option is unchecked. Make the required changes and click on close - the changes are applied immediately without any confirmation.
 2. What we have now is a panel of fixed length. However, the indicators still keep moving dynamically. To make this nuisance disappear, we need to add a separator of expanding length to the right of the indicators. To achieve this, right click on the bottom panel once again and navigate to Panels > Add New Items. Scroll and select "Separator" and click on add.
3. Now that you have added the Separator, move it so that it now comes between the indicators/notification area and the window buttons. Also, click on the properties box on the right and choose the style as "Transparent" and check the option for "Expand". Now click close.
That should do it. If you followed the above steps, you should finally get your bottom panel to look like this
If you have messed up the configuration in the above steps or if you just wish to restore the default panels, you can do so easily by renaming (or deleting) the ~/.config/xfce4/panel/ directory and then restarting xfce4-panel.
user@pc: ~$ mv  ~/.config/xfce4/panel ~/.config/xfce4/panel-backup
user@pc: ~$ killall xfce4-panel
If the panel doesn't restart automatically, then press Alt+F2 and run:

Installing a new hard disk without reinstalling Ubuntu/Windows

I own a 7-year old Dell Inspiron 1420 laptop, which comes with a 160 GB hard drive. My optical drive stopped working a couple of years ago and I had been contemplating getting a secondary hard drive. The only thing that kept me from it was the whole process of reinstalling my dual boot configuration on it. While Ubuntu installation is not too difficult and is done pretty quickly, I really hate having to reinstall Windows. The OS installation alone takes up a lot of time, not to forget the time required for installing additional programs like a good web browser, PDF reader, a compression program, antivirus, an office suite and vlc/mplayer - these are perhaps the most basic necessities that are needed to make your Windows functional.

A couple of months back, I decided to finally get an additional drive 750 GB drive and a caddy to place it securely in place of my optical drive. Choosing a caddy is easy - you just need to take care of two things : the size of the optical drive (12.5 or 9 mm) and the interface (IDE/SATA) that it uses to connect the motherboard. This is the same interface which is used to connect your optical drive and can be checked when you take out your optical drive.

Cloning the hard disk

Important Note:  /dev/sda refers to the complete hard drive and includes all partitions. /dev/sda1 refers to a specific partition of the same hard drive.

My home folder is on the same partition as the Ubuntu install. So I just cloned the two partitions using the terminal. My Ubuntu and Windows partitions were on /dev/sda1 and /dev/sda2 respectively. A good way to make sure you have the correct partition is to use the GParted tool (included in the Live CD) or the fdisk -l command at the terminal.
user@pc: ~$ sudo fdisk -l
Then I connected my external drive and used the following commands:
user@pc: ~$ dd if=/dev/sda1 of=/media/EXTERNAL/Ubuntu.img bs=4096
user@pc: ~$ dd if=/dev/sda2 of=/media/EXTERNAL/Win7.img bs=4096
I used this method because it does not require installing additional programs. For more ways of cloning, this article on lifehacker is a good read.

In the meantime, I used unetbootin to create a Ubuntu bootable USB. When the backup was done, I simply shut down my computer and replaced my primary hard drive with the new spacious one I had bought. The next step was to boot into Ubuntu using the live USB I had just made and then partition the new drive. I made a partition system similar to my previous one -
1. ext4 (Ubuntu)
2. NTFS (Windows 7)
3. NTFS (storage of common data files like Dropbox/Firefox profile/Music/etc.)

Note that I do not use any separate swap partition on my computer.

Of course, I made sure that the new partitions were at least the same size or bigger than the previous ones to ensure that the image file can be written back on them. When the partitions were made and properly formatted, I restored the OS installations on my new drive using the following commands:
user@pc: ~$ dd if=/media/EXTERNAL/Ubuntu.img of=/dev/sdb1 bs=4096
user@pc: ~$ dd if=/media/EXTERNAL/Win7.img of=/dev/sdb2 bs=4096
Yes, the new hard drive was mounted as /dev/sdb in my case. But you should check using the fdisk -l command to make sure you are restoring the images on the correct drives/partitions. Now, we are almost done.

Reinstalling the bootloader

One thing to notice here is that I cloned individual partitions instead of the whole drive. While this would have been also possible, I did it this way because I did not have enough free space on my external disk. Which is why the next step becomes important. I have two OS installations on the new hard drive now but no bootloader. To make this computer recognize the OS installs, I installed grub2 on the drive in the following way (Source: HowToUbuntu):

1. While still in the live CD/USB environment, open a terminal and mount the partition your Ubuntu Installation is on. Again, it can be found out by using GParted or the fdisk command.
user@pc: ~$ sudo mount /dev/sdb1 /mnt  
2. Now bind the directories that grub needs access to to detect other operating systems, in the following fashion
user@pc: ~$ mount --bind /dev /mnt/dev && sudo mount --bind /dev/pts /mnt/dev/pts && sudo mount --bind /proc /mnt/proc && sudo mount --bind /sys /mnt/sys
3. Now we need to change the apparent root directory of the live environment to the one on the Ubuntu partition we just mounted
user@pc: ~$ sudo chroot /mnt
4. Now install, check, and update grub. Grub is installed on the whole drive and not a single partition, so use /dev/sdb instead of /dev/sdb1 for the following set of commands
user@pc: ~$ grub-install /dev/sdb
user@pc: ~$ grub-install --recheck /dev/sdb
user@pc: ~$ update-grub
5. Now the grub installation is complete. All that is left is to exit the chrooted system and unmount everything.
user@pc: ~$ exit && sudo umount /mnt/dev && sudo umount /mnt/dev/pts && sudo umount /mnt/proc && sudo umount /mnt/sys && sudo umount /mnt
6. Shut down and turn your computer back on, and you will be met with the default grub2 screen.

That's it! Now you should have a completely running system with the same configuration as you had before. I have not tried doing this with a significant change in the system configuration. Even with those changes, Ubuntu should port to the new system configuration just fine. The only problem that you could face is with Windows because sometimes its WGA kicks in if it detects a change in your hardware.