DD Reference

[Mar 10, 2010] mount_dd

freshmeat.net

mount_dd is a small tool for mounting a dd image with a GUI. You can mount it in read-write or read-only mode.

dd(1) convert-copy file - Linux man page

Copy a file, converting and formatting according to the operands.

bs=BYTES

force ibs=BYTES and obs=BYTES

cbs=BYTES

convert BYTES bytes at a time

conv=CONVS

convert the file as per the comma separated symbol list

count=BLOCKS

copy only BLOCKS input blocks

ibs=BYTES

read BYTES bytes at a time

if=FILE

read from FILE instead of stdin

iflag=FLAGS

read as per the comma separated symbol list

obs=BYTES

write BYTES bytes at a time

of=FILE

write to FILE instead of stdout

oflag=FLAGS

write as per the comma separated symbol list

seek=BLOCKS

skip BLOCKS obs-sized blocks at start of output

skip=BLOCKS

skip BLOCKS ibs-sized blocks at start of input

status=noxfer

suppress transfer statistics

BLOCKS and BYTES may be followed by the following multiplicative suffixes: xM M, c 1, w 2, b 512, kB 1000, K 1024, MB 1000*1000, M 1024*1024, GB 1000*1000*1000, G 1024*1024*1024, and so on for T, P, E, Z, Y.

Each CONV symbol may be:

ascii

from EBCDIC to ASCII

ebcdic

from ASCII to EBCDIC

ibm

from ASCII to alternate EBCDIC

block

pad newline-terminated records with spaces to cbs-size

unblock

replace trailing spaces in cbs-size records with newline

lcase

change upper case to lower case

nocreat

do not create the output file

excl

fail if the output file already exists

notrunc

do not truncate the output file

ucase

change lower case to upper case

swab

swap every pair of input bytes

noerror

continue after read errors

sync

pad every input block with NULs to ibs-size; when used

with block or unblock, pad with spaces rather than NULs

fdatasync physically write output file data before finishing

fsync likewise, but also write metadata

Each FLAG symbol may be:

append

append mode (makes sense only for output)

direct

use direct I/O for data

dsync

use synchronized I/O for data

sync

likewise, but also for metadata

nonblock

use non-blocking I/O

nofollow

do not follow symlinks

noctty

do not assign controlling terminal from file

Sending a USR1 signal to a running 'dd' process makes it print I/O statistics to standard error and then resume copying.

CW$ dd if=/dev/zero of=/dev/null& pid=$!

CW$ kill -USR1 $pid; sleep 1; kill $pid

18335302+0 records in

18335302+0 records out 9387674624 bytes (9.4 GB) copied, 34.6279 seconds, 271 MB/s

Options are:

--help

display this help and exit

--version

output version information and exit

References

buffer(1), cdrecord(1), cstream(1), cycbuff.conf(5), gpart(8), myrescue(1), sg_dd(8), sg_read(8), sgm_dd(8), sgp_dd(8), spax(1), star(1), varnishd(1), wodim(1), xfs(5), xfs_copy(8), xfs_repair(8)

docs.sun.com man pages section 1M System Administration Commands - dd

The Source of All Tape Knowledge

dd The name dd stands for "copy and convert". Don't see it? Well, "cc" was already taken for the C compiler, so the author chose the next letter in the alphabet. The syntax has sort of an evil, JCL-like quality to it. According to The Jargon File, the interface was a prank.

Using dd

Most people use dd incorrectly. This is because dd is a piece of junk that should be replaced.

For example, one common misusage of dd is to try and get 64k blocks written to the tape with this command:

tar -cf - args... | dd of=/dev/rmt8 bs=64k

This won't work because (as you will see below), the bs argument gives you only one buffer. The dd process will attempt to read 64k chunks from the pipe into this buffer, but will only receive a maximum of PIPE_BUF bytes (usually 4 or 8k). It will then write this buffer out to the tape as a single record (it will not pad this block to 64k, fortunately).

GNU dd

This data taken from GNU fileutils 3.12

When dd starts up, it parses all the arguments on the command line in order. Note that the bs= argument will override any previous ibs= or obs= arguments. If neither the obs nor the ibs argument is presented, and bs is given, and no character-translation conversions are performed, then only one buffer will be used (more on this later). In all other cases, two buffers (input and output) are used. If you don't specify any *bs args, ibs and obs default to 512.

Next, based on the translations that you have specified, dd builds a translation table. This table is a 256 entry array, specifying a character-by-character mapping that is the composite of all specified translations. The actual order of application of translations is not the same as what is on the command line. It is:

1. ebcidic_to_ascii
2. lower_to_upper
3. upper_to_lower
4. ascii_to_ebcidic
5. ascii_to_ibm

Note that not all conversions can be specified at once. You have your choice of only one conv in {ascii,ebcdic,ibm}, {lcase,ucase}, {block,unblock}, {unblock,sync}.

Finally, dd enters the copy stage. It allocates enough room for the input buffer, and if using a two-buffer scenario, allocates an output buffer as well. It performs any skips on the input, then performs any seeks on the output.

The main loop of the copy stage occurs now. It attempts to read input_blocksize characters into the input buffer. Errors here may be trapped, depending on command line options. If a full input block is not read (for example, when reading from a communication line, the end of a file, a pipe or special file, especially tapes), the partial block count is incremented. If the sync option is in effect, partial input blocks are NUL padded and treated as full input blocks.

At this point, if we are single-buffering, we write the block out. TODO: finish up here (I got bored)

AIX dd

AIX dd is broken. From their manpage:

3. Use the backup, tar, or cpio command instead of the dd com- mand whenever possible to copy files to tape. These commands are designed for use with tape devices. For more information on us- ing tape devices see the rmt special file.

6. To ensure that only whole blocks are written to the output device (such as an 8mm tape in fixed-block mode), specify the ibs flag, the obs flag, and the conv=sync flag. The ibs flag must be a multiple of the obs flag.

This was hard-won knowledge for me. When dding a tar file directly out to tape, I ended up using:

dd if=foo.tar of=/dev/rmt1.5 ibs=1 obs=10240 conv=sync

Alternatively, you can use catblock, which is more efficient. Note that their comment about ibs being a multiple of obs is simply wrong, as my example demonstrates.

Solaris dd

Solaris dd is also not necessarily broken, but somewhat unintuitive. From the manpage:

When dd reads from a pipe, using the ibs=X and obs=Y operands, the output will always be blocked in chunks of size Y. When bs=Z is used, the output blocks will be whatever was available to be read from the pipe at the time.

In other words, don't expect "bs=Z" to be the same as "obs=Z ibs=Z". That's because, like GNU tar, it probably uses one buffer if you put "bs=Z", whereas "obs=Z ibs=Z" forces it to use two buffers. GNU dd may be subject to the same deficiency here. (TODO: check) I suppose it depends on how it treats a short read.

Mounting disk images on Linux loopback device

Obtaining a Disk Image

To obtain my disk image, I simply issued:

 
 rachael:#  dd if=/dev/hda of=/mnt/nebula/hda_dd.image
 4757130+0 records in
 4757130+0 records out
 

You can't simply mount a disk with the loopback device, however. You need some additional information. You will want to fetch a copy of the partition table, including the all important cylinder number we will use later. Invoke the magic of fdisk:

 
 rachael:/home/jasonb#  fdisk -l
 Disk /dev/hda: 4871 MB, 4871301120 bytes
 255 heads, 63 sectors/track, 592 cylinders
 Units = cylinders of 16065 * 512 = 8225280 bytes
    Device Boot    Start       End    Blocks   Id  System
 /dev/hda1   * 1       463   3719016    7  HPFS/NTFS
 /dev/hda2   464       592   1036192+   5  Extended
 /dev/hda5   464       479    128488+  82  Linux swap
 /dev/hda6   480       592    907641   83  Linux
 

Later, you can use this information to verifiy your image is sane.

Verifying the Sanity of Your Image

 
 faith:/home/jasonb#  fdisk -C 592 /nebula/hda_dd.image
 Command (m for help): p
 Disk /nebula/hda_dd.image: 0 MB, 0 bytes
 255 heads, 63 sectors/track, 592 cylinders
 Units = cylinders of 16065 * 512 = 8225280 bytes
        Device Boot    Start       End    Blocks   Id  System
 /nebula/hda_dd.image1   * 1       463   3719016    7  HPFS/NTFS
 /nebula/hda_dd.image2   464       592   1036192+   5  Extended
 /nebula/hda_dd.image5   464       479    128488+  82  Linux swap
 /nebula/hda_dd.image6   480       592    907641   83  Linux
 

Looks familiar, no? If all went well, it should be identical to the image yanked from the original disk.

Accessing Specific Partitions in the Image

Now, the fun begins. There are three ways to mount partitions from the image. You can simply use the stock kernel's loopback device, an enhanced loopback device offered by NASA, or extract the partition from the image and mount that directly with the loopback device. In all instances, the loopback device is the final destination. The journey varies with each, however. Let's look at the former most approach first.

Mounting with a Specified offset

 
 faith:/home/jasonb#  fdisk -l -u -C 592 /nebula/hda_dd.image
 Disk /nebula/hda_dd.image: 0 MB, 0 bytes
 255 heads, 63 sectors/track, 592 cylinders, total 0 sectors
 Units = sectors of 1 * 512 = 512 bytes
        Device Boot    Start       End    Blocks   Id  System
 /nebula/hda_dd.image1   *63   7438094   3719016    7  HPFS/NTFS
 /nebula/hda_dd.image2       7438095   9510479   1036192+   5  Extended
 /nebula/hda_dd.image5       7438158   7695134    128488+  82  Linux swap
 /nebula/hda_dd.image6       7695198   9510479    907641   83  Linux
 

The offset must be specified in bytes, so now you must take the starting offset, in this instance 63, and multiply it by 512 bytes. From this we obtain 32256. (This assumes 63 sectors per track and 512 bytes per sector.) The file system type in this case is NTFS, so let us mount this partition from within the image using the usual loopback method.

 
 faith:/usr/src#  mount -o loop,offset=32256 \
   -t ntfs /nebula/hda_dd.image /mnt
 faith:/usr/src#  ls /mnt
 AUTOEXEC.BAT
 boot.ini
 CONFIG.SYS
 Corel
 Documents and Settings
 IO.SYS
 MSDOS.SYS
 NTDETECT.COM
 ntldr
 PUTTY.RND
 Program Files
 pagefile.sys
 RECYCLER
 System Volume Information
 WINNT
 

If you are using util-linux prior to version 2.12b, specifying an offset that required more than 32-bits was not possible. If you have util-linux 2.12b or newer, you can safely skip the next few sections. (You may still wish to extract individual partitions from your disk image using dd discussed at the end of this guide.)

Attempting to mount my ext3 partition near the end of the disk with a 2.11 version of util-linux yields (7695198 * 512 = 3939941376):

 
 faith:/usr/src#  mount -o loop,offset=3939941376 \
   -t ext3 /nebula/hda_dd.image /mnt
 mount: wrong fs type, bad option, bad superblock on /dev/loop0,
        or too many mounted file systems
 

Fortunately, we aren't done yet. The second method utilizes a loopback device designed to mount partitions within the image without an offset limitation. In fact, no offset need be specified at all.

Mounting with a Special Patch

You will need to patch your kernel to use the enhanced loopback device. This patch alters the way the loopback device works. You will no longer be able to mount partitions via the loopback device beyond /dev/loop0. If you use /dev/loop[1-7] this could be a show stopper for you; Check out the last method.

The patch is currently available against 2.4.20 and 2.4.21 prepatch 4. You will need to fetch the patch from NASA HQ's public FTP server. It's the enhanced_loop-x.x-linux-2.4.x-xfs.patch file located there. You can also fetch the XFS patch for 2.4.21-pre4 and the 2.4.21-pre4 patch itself as of this writing. I used 2.4.21-pre4 with Alan Cox's -ac7. For convenience, a patched kernel ready for compiling is also available.

 
 faith:/usr/src/linux-2.4.20#  patch \
   -p1 < ../enhanced_loop-0.2-linux-2.4.21-pre4-xfs.patch
 patching file drivers/block/loop.c
 patching file Makefile
 Hunk # 1 FAILED at 1.
 1 out of 1 hunk FAILED -- saving rejects to file Makefile.rej
 

Don't worry about the Makefile reject; It's just the EXTRA_VERSION variable. (That's because I used -ac7.)

Now, recompile your kernel in the usual way (I use Debian GNU/Linux's make-kpkg command) and make sure you enable the loopback device if it isn't already. When that task is complete, reboot with your shiny new kernel.

To accomodate the enhanced loopback device, some new entries need to be created in /dev. A script named createdev is available to handle that task for you, and it can be run at start up if you're running devfs to recreate the entries for you at boot. You can fetch the script from NASA HQ. You may need to comment out the sourcing of the RedHat functions within the script if you aren't on a RedHat based distribution, like Debian. By default the script will create enough entries in /dev for a fifteen disks with up to fifteen partitions. You can adjust that to your requirements within the script. It will blow away any existing /dev entries it has added if you change configurations, so you need not tend to them yourself.

 
 faith:/nebula#  vi createdev
 faith:/nebula#  bash createdev start
 faith:/nebula# 
 

Once you've run the script, you should find a entries like the following in your /dev directory:

 
 faith:/#  ls /dev/loop[a-zA-Z]*
 /dev/loopa    /dev/loopd12  /dev/loopg2
 /dev/loopj6   /dev/loopn   /dev/loopa1
 /dev/loopd13  /dev/loopg3   /dev/loopj7
 /dev/loopn1   /dev/loopa10  /dev/loopd14
 /dev/loopg4   /dev/loopj8   /dev/loopn10
 /dev/loopa11  /dev/loopd15  /dev/loopg5
 /dev/loopj9   /dev/loopn11
 

With the kernel up and running, you also need to acquire a modified copy of losetup, the loopback setup program. If you're running an RPM based distribution, you're in luck. You can fetch the modified losetup by making another journey to NASA HQ's FTP server. Rebuild it with rpmbuild -bb and install. If you're running Debian GNU/Linux, as I am, you can install the rpm package with the usual apt-get command. Then, you could either build the RPM package and use alien to convert it to a Debian package or use rpm2cpio to create a cpio archive of the RPM. For the latter, you can extract the source from the resultant cpio archive and compile:

 
 faith:/usr/src#  rpm2cpio loop-utils-0.0.1-1.src.rpm > loop-utils.cpio
 faith:/usr/src#  cpio -i < loop-utils.cpio
 39 blocks
 faith:/usr/src#  tar -zxvf loop-utils-0.0.1.tar.gz
 loop-utils-0.0.1/
 loop-utils-0.0.1/COPYING
 loop-utils-0.0.1/Makefile
 loop-utils-0.0.1/loimginfo.c
 loop-utils-0.0.1/lomount.c
 loop-utils-0.0.1/lomount.h
 loop-utils-0.0.1/loop.h
 loop-utils-0.0.1/loop.sgml
 loop-utils-0.0.1/losetgeo.c
 loop-utils-0.0.1/lotest.c
 loop-utils-0.0.1/nls.h
 loop-utils-0.0.1/partinfo.c
 faith:/usr/src#  cd loop-utils-0.0.1
 

You may wish to edit the Makefile, which sticks things in /usr by default. I changed it to /usr/local and added ${prefix} as the path for the sbin_prefix variable. It originally had no value at all, but is later used when installing the losetup binary, which would instead end up in your /sbin directory. Oops.

 
 faith:/usr/src/loop-utils-0.0.1#  make
 gcc -Wall -Wstrict-prototypes -O6 -DVERSION='"0.3.9"' \
   -DLOG2_NR_PARTITION='4'   -c -o losetgeo.o losetgeo.c
 gcc   losetgeo.o   -o losetgeo
 gcc -Wall -Wstrict-prototypes -O6 -DVERSION='"0.3.9"' \
   -DLOG2_NR_PARTITION='4'   -c -o loimginfo.o loimginfo.c
 gcc   loimginfo.o   -o loimginfo
 gcc -Wall -Wstrict-prototypes -O6 -DVERSION='"0.3.9"' \
   -DLOG2_NR_PARTITION='4'   -c -o partinfo.o partinfo.c
 gcc   partinfo.o   -o partinfo
 gcc -DMAIN -D_FILE_OFFSET_BITS=64 lomount.c -o losetup.o
 <warnings...>
 ld losetup.o -o losetup
 gcc -Wall -Wstrict-prototypes -O6 -DVERSION='"0.3.9"' \
   -DLOG2_NR_PARTITION='4'   -c -o lotest.o lotest.c
 gcc   lotest.o   -o lotest
 sgml2latex loop.sgml
 Processing file loop.sgml
 sgml2html -s 0 loop.sgml
 Processing file loop.sgml
 sgml2info loop.sgml
 Processing file loop.sgml
 echo "START-INFO-DIR-ENTRY" > loop.info.2
 echo "* Loop: (loop). Block device loopback package." \
     >> loop.info.2
 echo "END-INFO-DIR-ENTRY" >> loop.info.2
 cat loop.info.2 loop.info > loop.info.3
 rm loop.info.2
 mv loop.info.3 loop.info
 

Now, let's test drive our new loopback device.

 
 faith:/nebula#  /usr/local/sbin/losetup -d /dev/loopa
 faith:/nebula#  /usr/local/sbin/losetup /dev/loopa hda_dd.image
 faith:/nebula#  mount -t ntfs /dev/loopa1 /mnt
 faith:/nebula#  ls /mnt
 AUTOEXEC.BAT
 boot.ini
 CONFIG.SYS
 Corel
 Documents and Settings
 IO.SYS
 MSDOS.SYS
 NTDETECT.COM
 ntldr
 PUTTY.RND
 Program Files
 pagefile.sys
 RECYCLER
 System Volume Information
 WINNT
 faith:/nebula#  umount /mnt
 faith:/nebula#  /usr/local/sbin/losetup -d /dev/loopa
 faith:/nebula# 
 

Nifty, eh?

Mounting by First Extracting the Partition

We pass dd bytes at a time size (bs option) of 512. Next, we pass it the starting sector of my ext3 partition from the fdisk output above, 7695198, as the number of blocks to skip ahead in the image. Last, we calculate the size as explained in the Sleuth Kit Informer above by taking the starting and ending sectors of the partition, subtracting them, then adding one (9510479 - 7695198 + 1 = 1815282).

Ronald Woelfel raised an interesting question about a missing sector on partitions with an odd number of sectors, which was explained thusly by Brian Carrier of Sleuth Kit fame: ”The reason that you noticing the difference is likely because your linux system has the 2.4 kernel, which has a bug when accessing disk or partition devices. If a partition or disk has an odd number of sectors, the last sector is not read.”

 
 faith:/home/jasonb#  dd if=/nebula/hda_dd.image of=/nebula/test.image \
   bs=512 skip=7695198 count=1815282
 1815282+0 records in
 1815282+0 records out
 

Once dd completes, you can mount the image as you normally would:

 
 faith:/home/jasonb#  mount -o loop -t ext3 /nebula/test.image /mnt
 faith:/home/jasonb#  ls /mnt
 bin    dev     home    lib opt   sbin  var
 boot   etc     import  lost+found  proc  tmp   vmlinuz
 cdrom  floppy  initrd  mnt root  usr   vmlinuz.old
 faith:/home/jasonb#  umount /mnt
 

Enjoy!

Links and Useful Resources

How to use dd to extract individual partitions from a disk image

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotes :  Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce :  Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS :  Programming Languages History : PL/1 : Simula 67 : C : History of GCC development :  Scripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month :  How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: March 12, 2019