Ghosting as a new backup method

A Slightly Skeptical View on Ghost

Dr. Nikolai Bezroukov

Version 0.80

Copyright 1998-2012, Dr. Nikolai Bezroukov. This is a copyrighted unpublished work. All rights reserved.

Note: Earlier version of this paper was published in Softpanorama biulletin, Vol 21, No. 1, 2009

Contents

• Introduction
• Origins
• History of Development
• Ghost 2003

• Corporate edition of Norton Ghost

• Norton Ghost 9-11
• Norton Ghost 12-13
• Norton Ghost 14-15
• Speed
• Corporate edition of Norton Ghost
• Webliography

Introduction

The history of Ghost is the history of a useful innovation if not destroyed, was severely damaged by Symantec. Currently Acronis True Image replaced Ghost on the marketplace.

Ghost is an innovative attempt to make unix dd style backup process more flexible from a little know New Zealand firm. Which was bought by Symantec. The latter in its classic style destroyed it with inept marketing and unability to move from DOS as the base of the program. So it acquired Power Quest which has a similar program called Drive Image, the main  competitor of the original Ghost with some innovative features. And destroyed it too.  Later in 2005 Symantec decided to use Ghost brand for the backup program from Veritas which was acquired (and destroyed) at this time.

After realizing that this was huge blunder Symantec revived the original style partition backup program in Ghost 14 and later.  See  Softpanorama Norton Ghost Page for details. Corporate edition has somewhat different path and has some additional features.  Symantec has a long & sordid history of acquiring products or complete companies and then destroying their products.

Origin and key ideas

If we talk about the basic idea behind Ghost, then we should consider it to be a derivative of Unix dd. In principle Ghost is just a hybrid of sector-by-sector disk copying utility and regular "per file" backup based on understanding of the filesystem of the partition which it working with.  Like in dd the disk/partition contents are copied either directly to another disk or partition, or ( more typically ) they are saved as a file that can later be restored to a disk or partition. But here similarities end. Ghost is hybrid idea: it has two layers -- partition layer on which it deals with partition resizing, etc and filesystem layer on which it deals with file present on the filesystem.

As it needs to understand the filesystem structure Ghost perform much more complex operations the DD (for example the restore on a partition of smaller size). But there is no free lunch and versatility and simplicity of dd are gone. Each new version of filesystem requires new version of ghost. Also ghost can't backup deleted files like DD can: it backs up only disk blocks that contain "valid" files as reflected in filesystem directories. That means that generally it is less tolerant for the errors.  And its ability to backup damaged filesystems is really inferior in comparison with DD. Also typically it does not have good recovery methods for damaged archive. Acronis true image is very bad in the respect and you really find yourself in a very difficult position due to a minor error in archive. 

At the same time there are some improvement over DD capabilities. Among them:

• It typically produces compressed images.  See also Working with disk images
   
• It can restore disk/partition backup to a disk/partition of different size expanding or compacting the filesystem on the fly.
• It also has the ability to restore individual files and folders (this capability was first unleashed in Ghost 2001). After you created your image, you can then restore not only the whole image, but each individual file or folder on it. Recently Linux loopback filesystem provided similar capabilities to DD images.  Ghost can even update images with new files although I do not  think that this is such a great idea. 

Norton Ghost is not a single product developed from one version to another. After initial acquisition Symantec played its typical game with it -- the original product was discarded, competitor was bought but also destroyed, and so on and so forth. In its current incarnation this is a different product which is not a derivative of the original Ghost. It have little in common other then name and broad function. 

• Old versions of Symantec Ghost (up to 2003) -- classic Ghost -- is a DOS-based utility. They all based on original design by Binary Research that Symantec acquired. From the very beginning Ghost has pretty amazing capabilities of working with disk images, essentially creating a class of its own. Binary Research created significant momentum and has good penetration in the Windows marketplace when Symantec acquired the product.  From the very beginning Ghost was able to work with NTFS images. Later it acquired the possibility to extract individual files via special utility, Ghost Explorer. It has interface similar to Windows Explorer and supports drag-n-drop of individual files. That make Ghost is one of the cheapest backup utilities available on the market, especially suitable for personal and small organizations use. 
   
• Later versions of Ghost starting from 9 (Ghost 2005) and including Ghost 11 (Ghost 2006) are based on a completely different product  -- Power Quest Drive Image. Ghost 9.0 (Ghost 2005) actually ships Ghost 2003 as its part. The main difference is that Power Quest Drive Image has the ability to create images without rebooting in DOS.  this is an important improvement although there are cases where using DOS is safer (for example creating an image of the windows partition. This capability also makes writing image to DVD simpler and faster. But please be aware that it might create some problems in case your computer is infected with adware/spyware, or worse with a rootkit.
• Ghost 12-13 was just an attempt to use Ghost brand for the backup program from Veritas which was acquired (and later destroyed) in 2005.
• Ghost 14 and 15 were partially successful attempt to return to the original idea.  The it was too little too late. At this time Acronis True Image became primary ghost-like program on the marketplace.

I think that the simplest and the most flexible way to use Ghost for a regular user is to create a FAT32  partition to store images (with current large drives it is possible to do on the same drive) or to use a special USB drive devoted to images.  On desktop it is very convenient to install the second harddrive and use it for images. You need approximately 30G for the partition that can store the image of the C partition in case you have Microsoft Office and a dozen of other applications installed (a must for any home user).  With current 320GB or larger desktop hardrives that's peanuts.  On laptop with its smaller (usually 240 GB harddrive or 120  GB SSD) usage of a separate USB drive is a better deal. 

Ghost and similar programs (ghosters) typically have three option for backup:

• backup of the disk
• backup of the partition
• Copy of the partition to another drive.

Please note that Ghost and its derivatives are sensitive to the version of filesystem used. Different versions of Window use different versions of NTFS. Generally the year of release of Ghost should be at least a year higher then the year of the release of your version of Windows for it to work reliably, so for Windows XP Ghost 2001-2003 are OK. I would say that for Windows 2003 (NTFS 3.1), only Ghost 2003 and up are safe. 

History of Development

Ghost was not developed by Symantec; like most Symantec products it was bought when an original developer already achieved significant market success.  Here is the summary of Ghost development adapted from Wikipedia,

Ghost was originally produced by Binary Research ( Auckland, New Zealand ). It was this codebase that was purchased by Symantec in 1998.

Murray Haszard wrote Ghost in 1996, based on his experience with file-copying program that worked via parallel and serial interface previously produced by Binary Research.The name Ghost originated as an acronym for "General Hardware-Oriented Software Transfer".

In this age of huge organizations producing monster software it's pretty refreshing to see that individual talent still matter. Initial version of Ghost was just an implementation of a single brilliant idea -- create a hybrid program that behaved like DD but at the same time have flexibility closer to file based backup. It also should be able to clone Windows system disk and system partitions. Software should be able to preserve Windows system disk in workable condition despite minor changes in the size of the target partitions. As we can see that while initial design was Windows oriented, Ghost can actually be written for Linux without any problems.  

The original Binary Research version of Ghost was an instant success  and created a market for Windows system disk/partition cloning software.

Murray Haszard wrote Ghost in 1996, building on experience with a parallel and serial file-copying program previously produced by Binary Research. Initially, Ghost supported only FAT filesystems directly, but it could also copy (although not resize) other filesystems by performing a sector copy. Ghost added support for the NTFS filesystem later in 1996, and also provided a program, Ghostwalker (DOS name: ghstwalk.exe), to change the Security ID (SID) that made Windows NT systems distinguishable from each other. Ghostwalker is also capable of modifying the name of the Windows NT-based computer from its own interface. Ghost added support for the ext2 filesystem in 1999 and for ext3 subsequently.

The last version of Ghost released before Symantec acquisition was version 4.0. It added support of multicast technology for cloning imaging over the network  (following the lead of a competitor, ImageCast). Multicasting allows sending a single image simultaneously to many machines without putting greater stress on the network than by sending an image to a single machine. This version also introduced Ghost Explorer, a Windows program which allowed a user to browse the contents of an image file and extract individual files from it. Explorer as subsequently enhanced allowed users to add and delete files on FAT (and later on ext2 and on ext3) filesystems in an image. Ghost Explorer could extract files from NTFS images but not edit NTFS images. Ghost Explorer could work with images from older versions but only slowly; version 4 images contained indexes to find files rapidly. Version 4.0 also moved from real-mode DOS 286 to protected-mode. The additional memory available allowed Ghost to provide several levels of compression for images, and to provide the file browser.  Gdisk, a scriptable partition manager was also added. Gdisk serves a role similar to Fdisk, but has greater capabilities.

Ghost 4.1 released in 1998 allowed for password-protected images. Version 5.0 moved to 386 protected mode. Unlike the character-based user interface of earlier versions, 5.0 used a GUI. The Binary Research logo (two stars revolving around each other) played on the main screen while the program idled. That was the last version before acquisition.

Ghost 6.0 included a console application in 2000 to simplify the management of large numbers of machines. The Console communicates with client software on managed computers to allow a system administrator to refresh the disk of a machine remotely.

As a DOS-based program, Ghost required machines running Windows to reboot to a DOS environment to run it. Ghost 6.0 required a separate DOS partition when used with the Console. Ghost 7.5 in 2002 created a 'Virtual Partition' instead - a DOS partition which actually exists as a file within a normal Windows filesystem. This significantly eased systems management. Ghost 7.5 also allows writing to as well as reading from an image of an NTFS partition. Ghost 7.5 could also write images to CD-R drives, and later versions can also write DVDs. Ghost 2003, a consumer version of Ghost, does not include the Console but has a Windows front-end to script Ghost operations. The machine still needs to reboot to the Virtual Partition, but the user doesn't need to interact with DOS.

The success of Ghost created a lot of imitation like Drive Image. The latter became the main competitor and has some important features that Ghost did not have, including the ability to create images without rebooting to DOS

At the end of 2003, Symantec acquired its competitor, Power Quest, discarded the original product and rebranded Drive Image as Norton Ghost 9.0. It was released on August 2, 2004 as a new consumer version of Ghost and was based on version 7 of Drive Image product.

After the acquisition the original Ghost codebase was abandoned. This is not the first time Semantic "killed" the product in such a way. Like Computer Associates Symantec is a company that is living by acquisitions and like CA (and before CA) it developed a bad habit of purchasing other utility companies to cut down on the competition and either rename comparing product to sell it as the next version of the Symantec line (that happened with several great programs that were acquired by Symantec and later killed).

Remember Central Point Utilities and Xtree? They used to compete with the Norton utilities and Norton Commander that Symantec bought before and they vanished after Symantec bought the company.  And Norton Utilities which were a very good product in DOS days, went downhill in Symantec hands and gradually become a less useful (and more dangerous) bloatware.  Symantec folks seems fail to understand the fact that the value of Norton Utilities was in their ability to serve as a troubleshooting/productivity tools directly from floppy/CD without installation on the computer.  So they convert it into installation-based set of Utilities and a lot of good qualities of the original Norton Utilities disappeared without any trace. But let's return to Ghost.

There are several ways to run Ghost, but the classic way is to using it after booting from the DOS boot disk (I recommend using Windows 98 boot disk).  You need to launch ghost.exe (ghospe.exe in older versions) to create a backup or restore the partition or drive.

To summarize the historical development of Norton Ghost we can say that there are two major version of Ghost: Ghost 2003 (the last version that used rebooting to DOS for operation) and Ghost 2004-2006 (based of PowerQuest Drive Image product).

Ghost 2003

Ghost 2003 was the last version of the original product line. It is a DOS executable and all cloning operation are performed after reboot to DOS. Earlier versions of Ghost (until version 8, or Ghost 2003, see below) required direct manual rebooting of the computer using DOS bootable disk or CD. Ghost 2003 can boot from virtual partition or boot remotely using a network boot (using 3Com Boot Services).  This is a more convenient operation but rebooting might create problem in docked laptops as DOS might not see the drives present in dock after the reboot.

Ghost 2003 provides the Ghost Boot Wizard to help automatically create a bootable floppy disk. It make sense to create several of them just in case, write the code that is used on the floppy label and attach one to the computer. 

Norton Ghost 2003 supports Linux EXT3, EXT2, Microsoft FAT and NTFS. It consist of the following main programs (those should be updated via live update to version of May 28, 2003 or later): 

• Ghost.exe -- the main program that can create and restore images
• GhostWalker.exe -- images browser program
• GDisk.exe -- formatting utility,  which is a better tool then Microsoft fdisk
• Ghwrap.exe
• Ghreboot.exe

After you created the DOS boot disk using the Ghost wizard you need to modify it. The disk contains 251K of free space, enough for Volkov commander with several useful plug-ins. 

Norton Ghost 2003 can write disk images to several popular DVD/RW drives (very slow, not recommended), as well disks connected via USB 1.1, 2.0, or FireWire (pretty fast, recommended). Drive should be plugged directly into the main USB ports on the computer to allow Ghost to access it.

As Ghost is a DOS program and in order to restore the image, the image file should be visible from DOS,  but Symantec went to substantial length to provide drivers to many "post-DOS" devices including Firewall and USB drives. Still the most reliable way is to use a regular harddrive with FAT32 partitions. Network drive is also a possibility but here you depends on reliability of the second computer.

Ghost 2003 hides classic, simple way of using the program by booting into DOS by creating a virtual DOS partition (which is very convenient for laptops that does not have floppy drive), but you should not forget about the basic fact: ghost still is a DOS-based command line utility.  Ghost 2003 obnoxiously installs itself into the windows tray but you can remove it from the tray by using configuration option (disable  tray option).

All-in-all not all new features introduced by Symantec in later versions of Ghost are completely useless and Symantec still managed not to destroy the product five years after the acquisition ;-). So far Ghost did not share the fate of Norton Utilities.

Actually the Ghost 2003 Virtual Partition and is a rather slick idea: when you start the cloning operation from within Windows, Ghost automatically restarts your computer into a DOS environment, performs the cloning operation, and then boots the computer back into Windows. To create DOS environment Ghost uses a Virtual Partition with IBM PC DOS. The Virtual Partition is a file stored on the hard disk. When Ghost restarts the computer, the computer uses the information from that file to load DOS and other required files, and to run Ghost.exe. As I mentioned before this way in a laptop environment you do not need a floppy drive to create an image, which is important for many laptop users.

Note: The DOS executable file for Norton Ghost 2003 is ghost.exe. In earlier ghost versions the DOS executable file is Ghostpe.exe.

Interestingly, suppose your image file is stored on a network drive or a friend’s machine and your hard disk fails. You can connect to the network or any other standalone machine over a peer-to-peer link, either through your parallel port, USB port, or even your network card. The Ghost boot disk contains some generic drivers for these hardware connections. Network administrators can also use Norton Ghost to deploy multiple machines on a network but "Professional version" is better equipped for that. If you get machines of the same configuration, then you can configure one, create a ghost image, and copy it across the remaining machines.

Ghost also has many command-line parameters, such as those for checking image file integrity, etc.

This version of Ghost also supports Linux EXT2 and EXT 3 file systems apart from FAT16, FAT32, and NTFS

Norton Ghost 2005-2006

Norton Ghost 9  (which is actually  a rebranded Drive Image, a different product from the original Norton Ghost) have so called Symantec Recovery Disk. If your operating system is not responding, this tool lets you boot your PC, restore a backup image and perform system fixes and virus scans. In Norton Ghost 2003 you can just boot from a DOS floppy (or CD in case of laptop) and restore the image without much fuss.

Norton Ghost 9.0 also offers increased driver and interface support for USB, USB2 and FireWire removable storage devices in comparison with Ghost 2003.  That provides the possibility of direct burning of image on DVD (operating that was possible in Ghost 2003 but was too slow to be useful). If a disk or partition is no longer accessible due to corruption of the disk's system area or file system, or more often due to some spyware/adware infection, disk image can be a life saver. If you previously created a disk image or partition image file, you can use that file to restore the disk or partition. If the harddrive is physically damaged, you can restore the disk or partition from the image to a different hard disk.

What is really impressive is that Ghost image is not a monolithic archive. Norton Ghost 9 can mount images as a drive so that the backup copy looks just like the original drive. This even more slicker solution that  contained a special utility that is capable of to extract files from the image  (Ghost explorer) in Ghost 2003. Both solutions are pretty amazing and provide flexibility not available in Unix world.

The speed of Norton Ghost 2003 in disk to disk operation is really amazing: approx 10 min to backup a partition with ~4G of data (many small html files) with the image size slightly over 2G and just 3-4 min to restore this partition. For typical windows XP C drive (say 10G of data) Ghost 2003 takes approximately 25 min and produces image approximately 7G in size. For comparison to create a RAR archive for the 4G partition takes over an hour on the same computer. 

Corporate edition of Norton Ghost

Ghost 8.0 Corporate Edition evolves from a backup and recovery product into a configuration and patch management platform.

Ghost 8.0 has some remote management features that enables administrators to clone Microsoft Corp., Windows NT, 2000 or XP machines and send out software hot fixes or operating system configuration changes to multiple workstations at once. New features also make it easier to migrate user settings such as Windows "My Documents" folders, desktop shortcuts and configurations between computers, Symantec said.

To make multiple target cloning more efficient, multicasting feature from original Ghost was reintroduced for this version. Bandwidth throttling features enable companies to use the multicast feature without taking up too much of the network's available resources, the company said.

Ghost customers can still "unicast," copying a desktop image to a single machine, or deploy a single image to all machines on a network subnet, Symantec said.

Symantec also included features to limit the impact of system cloning and restoration on corporate networks. Unlike earlier versions of Ghost, Version 8.0 Corporate Edition has the ability to create staging areas on local machines. Those enable administrators to store local copies of user profiles, software fixes or even entire "images" of the computer operating environment, Symantec said.

Unlike other disk contents, the staging area is not overwritten when Ghost restores files to the machine and the files in it can be used to quickly install key software components. Earlier versions of Ghost required all the information captured during the cloning process or used in a restoration to be sent back and forth over the corporate network, Symantec said.

Symantec also introduced improved asset tracking features. IT administrators can now track what hardware and software is running on remote machines from the Ghost 8.0 management console and use filters to see which machines qualify for software patches or other updates, the company said.

A "ghost boot" feature simplifies deployment of the Ghost product by allowing administrators to browse for network machines and remotely install the Ghost client.

Webliography

• Softpanorama Norton Ghost Page
• Ghost (software) - Wikipedia, the free encyclopedia
• Symantec Support - Norton Ghost 9.0
• Symantec Ghost Tutorials

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotes :  Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce :  Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS :  Programming Languages History : PL/1 : Simula 67 : C : History of GCC development :  Scripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month :  How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Created May 16, 1996; Last modified: March 12, 2019