DD Rescue

Old News ;-)

[Jul 08, 2010] Save Your Data from the Click of Death - by Kyle Rankin

July 14, 2004 | O'Reilly Sysadmin

Related link: http://www.garloff.de/kurt/linux/ddrescue/

One thing I like about Linux is that there are always tools out there to discover that save you a lot of time or in this case data. I always love to walk a new user through transferring files from one drive to another and tell them “now run watch df  to check the output.”

“Watch? Oh WOW why didn’t I know about this before?”

Well, I had another one of those moments the other day when I ran across dd_rescue. This utility addresses the issue of using dd to image a drive that has bad blocks or otherwise has errors. When dd hits an error, it fails, so you are left with using skip and seek arguments to try to route around the bad blocks. Dd_rescue addresses this by simply skipping bad blocks it finds (or trying X number of times before skipping if you wish), so you can image even failing drives full of bad blocks.

Of course, dd_rescue has its problems as well. While it does skip bad blocks, it takes it some time to do so, and since bad blocks are often grouped together, it can take a very long time to dig through a group of bad blocks. If this group is at the front of your data, you are wasting the valuable last breaths of your hard drive looking at bad blocks, instead of recovering good ones.

The dd_rhelp script addresses this by acting as a front-end for dd_rescue. When dd_rescue finds a bad block, dd_rhelp has it jump ahead a bit, and then read blocks in reverse until it finds the tail end of the bad block group. It then continues forward and grabs all of the good blocks first. Then, after it finishes grabbing your good data, it goes back to the bad data and attempts to recover what it can from there.

dd_rhelp has an added advantage in that it can also resume at any point in the recovery by keeping track of where it is in dd_rescue’s logs. So if you have to Ctrl-C from the process, you can just re-run it and dd_rhelp will pick up where it left off. Also, it has a nice little ASCII progress meter, which is useful for keeping track of where you are.

Of course, it makes sense to not run this rescue while you are booted into the bad drive, so either move the drive to another system, or use some sort of rescue disk like Knoppix.

So, to use this, download dd_rescue version 1.03 or later (1.03 is also packaged in Debian unstable at the moment, but testing seems to have 1.02) from its site, and then get the latest version of dd_rhelp. While older versions of dd_rescue will work by themselves, dd_rhelp requires 1.03. Building dd_rhelp is a simple matter of untarring and running ./configure && make, and it even compiled just fine under Knoppix.

Once the tools are ready, you can recover a drive with

# dd_rhelp /dev/hda1 /mnt/hdb1/hda1_rescue.img

Then run fsck on the image you have created, mount it loopback (mount -o loop /mnt/hdb1/hda1_rescue.img /mnt/hda1 in my case) and then browse through it and recover the files you need.

Now you can see if you still have a warranty on the drive and return it, demolish it with a sledgehammer, or take it apart and use it as a desk accessory.

What are some of your “click of death” horror stories? IBM Deathstar or otherwise!

Comments

lbruno:

An alternative to dd_rescue
dd conv=noerror

Sums:

An alternative to dd_rescue
Sorry, not really. dd with conv=noerror writes nothing in the image in case of a bad block. Your image gets smaller and may be not mountable. dd_rescue writes Zeros in the image. The size of your image becomes the size of the partition.

HAND

Oliver

Antonio_Diaz:

A real alternative to dd_rescue + dd_rhelp
Is ddrescue http://freshmeat.net/projects/dd_rescue/ (don't confuse it with Garloff's dd_rescue).

ddrescue will:
1) Read the non-damaged part of the disk, skipping the damaged areas.
or
1) Read a list of bad blocks (damaged areas) from a file.

2) Try to read the damaged areas, splitting them into smaller pieces and reading the non-damaged pieces, until the hardware block size is reached.

3) Try to read the damaged hardware blocks until the specified number of retries is reached, or until interrupted by the user.

4) Optionally write a list of bad blocks to a file.

Ddrescue is now part of the GNU project. Its new homepage is http://www.gnu.org/software/ddrescue/ddrescue.html.

[Mar 11, 2010] g4u.

this is bit-by-bit copying so NTFS is supported. It works best with identical size and geometry. People complain that the image resulting from g4u is very big. This is normal as g4u clones the whole disks with all blocks, not attributing if they contain any valid data or if they are empty/unused. To find empty/unused blocks (and not clone them), g4u would need intimate understanding of the contained filesystem, which is different again for each filesystem - Windows FAT, Linux Ext2/3/ReiserFS/..., BSD FFS, Solaris UFS, etc.

Creating a dd-dcfldd Image Using Automated Image & Restore (AIR) HowtoForge - Linux Howtos and Tutorials

Automated Image & Restore (AIR) is an open source application that provides a GUI front end to the dd/dcfldd (Dataset Definition (dd)) command. AIR is designed to easily create forensic disk/partition images. It supports MD5/SHAx hashes, SCSI tape drives, imaging over a TCP/IP network, splitting images, and detailed session logging. To date, the AIR utility has only been developed for use on Linux distributions. In its simplest form, AIR provides a convenient interface to execute the dd set of commands. It eliminates the risk of "fat fingering" an error in the shell terminal and ultimately makes using the dd command more user-friendly for those who are not as experienced. Please note that using the AIR front end still requires some basic knowledge of how the dd (or dcfldd) commands work.

The dd command has been around for quite a while. It is well known throughout the Unix/Linux community, well documented, and as I can only imagine extensively used. A dd image is a bit by bit image of a source device or file. The uses for dd range from creating and maintaining system backups and restore images to the forensic application of imaging evidence that will be returned to the lab and examined.

This tutorial is not designed to teach the use of the dd command; this is well documented and a simple internet search will yield a plethora of results. Instead, the intent of this mini "how-to" is to introduce users to the AIR front end application, increase overall awareness of the utility, and provide a brief example of creating a dd image using this tool.

DISCLAIMER: I do not claim to be expert at using dd or Automated Image & Restore.

dd_rescue Advantages

Imagine, one of your partitions is crashed, and as there are some hard errors, you don’t want to write to this hard disk any more. Just getting all the data off it and retiring it seems to be suitable. However, you can’t access the files, as the file system is damaged.

Now, you want to copy the whole partition into a file. You burn it on CD-Rom, just to never lose it again. You can setup a loop device, and repair (fsck) it and hopefully are able to mount it.

Copying this partition with normal Un*x tools like cat or dd will fail, as those tools abort on error. dd_rescue instead will try to read and if it fails, it will go on with the next sectors. The output file naturally will have holes in it, of course. You can write a log file, to see, where all these errors are located.

The data rate drops very low, when errors are encountered. If you interrupt the process of copying, you don’t lose anything. You can just continue at any position later. The output file will just be filled in further and not truncated as with other Un*x tools.

If you have one spot of bad sectors within the partition, it might be a good idea, to approach this spot from both sides. Reverse direction copy is your friend.

The two block sizes are a performance optimization. Large block sizes result in superior performance, but in case of errors, you want to try to salvage every single sector. So hardbs is best be set to the hardware sector size (most often 512 bytes) and softbs to a large value, such as the default 16k.

Install dd_rescue in Debian

Install ddrescue using the following command

#apt-get install ddrescue

Install ddrescue in Ubuntu

sudo apt-get install ddrescue

This will complete the installation

ddrescue Syntax

dd_rescue [options] infile outfile

Now we will see how to use ddrescue under damaged disk

If you have a damaged hard disk /dev/sda1 and you have an empty space hard disk /dev/sda2 Now if you want to copy data from /dev/sda1 to /dev/sda2 use the following commnd

# dd_rescue /dev/sda1 /dev/sda2/backup.img

If you are using ubuntu linux use the following command

sudo dd_rescue /dev/sda1 /dev/sda2/backup.img

This copies an image of /dev/sda1 to sda2

Now you need to check the backup image consistency this will check for is there any problems with this image.

#fsck -y /dev/sda2/backup.img

If you are using ubuntu linux use the following command

sudo fsck -y /dev/sda2/backup.img

After finishing this checking you need to mount your disk image in to your other hard disk

#mount /dev/sda2/backup.img /mnt/recoverydata

If you are using ubuntu linux use the followingprebackup.img /mnt/recoverydata

This will mount all the data from the backup.img under /mnt/recoverydata now you can try to access the data it should work without any problem.

Restore image

If you want to restore this image use the following command

#dd_rescue /dev/sda2/backup.img /dev/sda1

If you are using ubuntu linux use the following command

sudo dd_rescue /dev/sda2/backup.img /dev/sda1

Copy Disk Image to remote machine using SSH

If you want to copy your disk image to remote machine over ssh you need to use the following command

#dd_rescue /dev/sda1 - | ssh username@machineip ‘cat /datarecovery/backup.img’

If you are using ubuntu linux use the following command

sudo dd_rescue /dev/sda1 - | ssh username@machineip ‘cat /datarecovery/backup.img’

This will be prompetd for password of the username you have menctioned in the above command after entering the password dd_rescue strats copying obviously it will take some time to copy over the network.

Possible Error

If you see the following error at the time of copying you can ignore this error

dd_rescue: (warning): output file is not seekable!
dd_rescue: (warning): Illegal seek

If you want to take this image in compressed format you can use the following command format

#tar zcvf - /dev/sda1 | ssh username@machineip ‘cat@@>/tmp /datarecovery/backup.tar.gz’

If you are using ubuntu linux use the following command

sudo tar zcvf - /dev/sda1 | ssh username@machineip ‘cat@@>/tmp /datarecovery/backup.tar.gz’

If you want to know more available options check dd_rescue man page

recover-data-from-a-dead-hard-drive-using-ddrescue

Like dd, dd_rescue does copy data from one file or block device to another.dd_rescue is a tool to help you to save data from crashed partition. It tries to read and if it fails, it will go on with the next sectors where tools like dd will fail. If the copying process is interrupted by the user it is possible to continue at any position later. It can copy backwards.

[Feb 15, 2007] Recover Data from a Damaged hard disk using dd_rhelp -- Debian Admin

December 19, 2006 (debianadmin.com) by Admin @ 4:59 pm. Filed under Backup

dd_rhelp is a bash script that handles a very usefull program written in C which is called dd_rescue, it roughly act as the dd linux command with the characteristic to NOT stop when it falls on read/write errors.

This makes dd_rescue the best tool for recovering hard drive having bad sectors. In short, it’ll use dd_rescue on your entire disc, but will try to gather the maximum valid data before trying for ages on badsectors. So if you leave dd_rhelp work for infinite time, it’ll have the same effect as a simple dd_rescue. But because you might not have this infinite time , dd_rhelp will jump over bad sectors and rescue valid data. In the long run, it’ll parse all your device with dd_rescue.

You can Ctrl-C it whenever you want, and rerun-it at will, it’ll resume it’s job as it depends on the log files dd_rescue creates.In addition, progress will be shown in a ASCII picture of your device beeing rescued.

Why use dd_rhelp and not dd_rescue ?

dd_rhelp uses dd_rescue to compute a recovery path through the device that will focus on valid data recovering. This recovery path will go through all the device, exactly as dd_rescue could do it on its own without any path. This means that dd_rhelp will save you time ONLY IF YOU INTEND TO CANCEL ITS JOB BEFORE THE END of a full recovery.

Because a considerable amount of time is taken to try to rescue badsectors. This amount of time can be mesured in days,
month, years, depending on your device capacity and its defectiveness. You might not want to spend this time knowing that 99 percent of this time will be taken to look at badsector and won’t lead to any more data recovering.

dd_rhelp shifts this useless waiting time to the end. Using dd_rescue strait throughout your device make your waiting time dependent on the badsector distribution.

Install dd_rhelp

First you need to download latest version from here

Preparing your system

#apt-get install make gcc g++ ddrescue

If you are using ubuntu linux use the following command

sudo apt-get install make gcc g++ ddrescue

Download dd_rhelp using the follwoing command

wget http://www.kalysto.org/pkg/dd_rhelp-0.0.6.tar.gz

Now you have dd_rhelp-0.0.6.tar.gz file

Extract this file using the following command

tar xzvf dd_rhelp-0.0.6.tar.gz

cd dd_rhelp-0.0.6

Run the following commands

# ./configure

# make

# make install

If you are using ubuntu linux use the following command

sudo ./configure

sudo make

sudo make install

This will complete the installation and this will install program under /usr/local/bin/dd_rhelp

dd_rhelp Syntax

dd_rhelp {filename|device} {output-file} [{info}]
or dd_rhelp --help
or dd_rhelp --version

Now we will see how to use ddrescue under damaged disk

If you have a damaged hard disk /dev/sda1 and you have an empty space hard disk /dev/sda2 Now if you want to copy data from /dev/sda1 to /dev/sda2 use the following command

#dd_rhelp /dev/sda1 /dev/sda2/backup.img

If you are using ubuntu linux use the following command

sudo dd_rhelp /dev/sda1 /dev/sda2/backup.img

Output looks like below

/usr/bin/tail: Warning: “+number” syntax is deprecated, please use “-n +number”
=== launched via ‘dd_rhelp’ at 0k, 0 >>> ===
dd_rescue: (info): ipos: 104384.0k, opos: 104384.0k, xferd: 104384.0k
errs: 0, errxfer: 0.0k, succxfer: 104384.0k
+curr.rate: 10178kB/s, avg.rate: 8532kB/s, avg.load: 31.3%
dd_rescue: (info): /dev/sda2 (104391.0k): EOF
Summary for /dev/sda2 -> backup.img:
dd_rescue: (info): ipos: 104391.0k, opos: 104391.0k, xferd: 104391.0k
errs: 0, errxfer: 0.0k, succxfer: 104391.0k
+curr.rate: 355kB/s, avg.rate: 8519kB/s, avg.load: 31.3%
/usr/bin/tail: Warning: “+number” syntax is deprecated, please use “-n +number”

This copies an image of /dev/sda1 to sda2

Now you need to check the backup image consistency this will check for is there any problems with this image.

#fsck -y /dev/sda2/backup.img

If you are using ubuntu linux use the following command

sudo fsck -y /dev/sda2/backup.img

After finishing this checking you need to mount your disk image in to your other hard disk

#mount /dev/sda2/backup.img /mnt/recoverydata

If you are using ubuntu linux use the following command

sudo mount /dev/sda2/backup.img /mnt/recoverydata

This will mount all the data from the backup.img under /mnt/recoverydata now you can try to access the data it should
work without any problem.

Restore image

If you want to restore this image use the following command

#dd_rhelp /dev/sda2/backup.img /dev/sda1

If you are using ubuntu linux use the following command

sudo dd_rhelp /dev/sda2/backup.img /dev/sda1

Copy Disk Image to remote machine using SSH

If you want to copy your disk image to remote machine over ssh you need to use the following command

#dd_rhelp /dev/sda1 - | ssh username@machineip ‘cat /datarecovery/backup.img’

If you are using ubuntu linux use the following command

sudo dd_rhelp /dev/sda1 - | ssh username@machineip ‘cat /datarecovery/backup.img’

This will be prompetd for password of the username you have menctioned in the above command after entering the password

dd_rhelp strats copying obviously it will take some time to copy over the network.