Backing Up And Restoring Linux With SystemImager
(Open source abandonware)

NOTE: This is an interesting project which now is abandonware (last changes are from 2015). Like many such projects it died of overcomplexity.  It can be simplified and revived without bell and wiles as key ideas are sound and there are commercial systems like Bright cluster manager which use the same approach (rsync based restoration os nodes)

SystemImager was a popular tool created by Brian Elliott Finley,  while he was working in Argonne National Laboratory. The name "System Imager"  is trademarked.  Systemimager is licensed under GPL v 2.  Now this is an open source abandonware.

Source code is now available from GitHub - finley-SystemImager.  Key utilities are

• si_getimage   (648 lines)
• si_pushupdate
• si_updateclient  (817 lines) -- updates client from the image on the master
• si_pushoverrides  -- distribute configuration files from  the image server to the clients or group of them, using the SystemImager overrides.

As it is based on rsync SystemImager is distribution-agnostic and that means that it can support heterogeneous set of nodes (aka grid). In other words it allows the deployment of any Linux distribution (standard or even customized) to any kind of target machine as long as you can providea bootable image for this machine. Typical environments include computer labs and render farms, but it has proven particularly popular in clustered computing environments, such as grid and high performance computing.

The main goal of the project was to make deployment of large numbers of computers easy, but trying to achive that they went of the couse and make things way too complex. 

Rsync works with files based (rather than disk blocks) so SystemImager images are stored as a directory hierarchy of files representing a comprehensive snapshot of a machine. This snapshot is containing all the files and directories from the root of the server file system.

Images can be acquired in multiple ways. The standard method of image creation involves cloning of a pre-installed machine, so called the golden-client. In this way, the user can customize and tweak the golden-client's configuration according to his needs, verify it's proper operation, and be assured that the image, once deployed, will behave in the same way as the golden-client. Incremental updates are possible by syncing an updated golden-client to the image, then syncing that image to deployed machines using the si_updateclient command.

Images are hosted in a central repository on a server, called the image-server, and they can be distributed among the clients using several different transport mechanism: rsync
(the default), multicast (via Flamethrower1), SSL encrypted rsync (using a SSH tunnel), and (for a very large set of nodes) via BitTorrent.

Commercial implementation of the same ideas as SystemImager can be found in Bright Cluster Manager.  Both rely on PXE boot, which is not available in all environments.

To implement PXE boot you need to provide two critical peaces of infrastructure (see  Chapter 30. Setting Up an Installation Server )

1. TFTP server
2. DHPC server

If you do not control DHCP server you only option is to use private network, which is a common solution for HPC clusters, but has little value outside this niche.

Some ideas about how to resurrect SystemImager and make it simpler

This is too valuable abandonware to make it die.

A simpler implementation is to create initial install using the kickstart, then duplicate it into RAM drive, chroot to the duplicate and overwrites  the installed root filesystem with the image. that exclude the necessity of DHCP server, PXE, etc. Or more correctly to cut the complexity it make sense to make them independent.

In other words systemimager can be re-implemented as the kickstart post install script.  That limits the number of distribution it supports.

Idea of overrides

System imager supports 3 levels of hierarchy for the overrides over the image:

• Node override: to be distributed in a single node. This is probably the most useful type.
• Group override: to be distributed only in a group of nodes. This is a useful is you have different group of servers under maintenance.
• Global override: to be distributed to all the nodes. This is rarely useful as such changes can be incorporated into the image itself.  But there can be some experimental changes although it is unclear why to deploy them globally  using overwrite and do not create a separate image.

Implementation

SystemImager is mostly written in Perl. the problem is that in the current form it makes use of 18 packages which makes implementation rather complex. The current implementaion of SI uses the folowing packages:  

1. busybox

2. bc

3. devfsd

4. ISC dhcp

5. discover

6. dosfstools

7. e2fsprogs

8. jfsutils

9. xfsprogs

10. Linux kernel

11. parted

12. pxelinux

13. rsync

14. syslinux

15. raidtools

16. reiserfsprogs

17. systemconfigurator

18. uClibc

Implementation via kickstart will remove almost all of them.

Later it became a part of System Installation Suite (SIS), which includes SystemInstaller, SystemImager, and System Configurator.  Of them only System Configurator is still maintained.

SystemInstaller is a tool that allows you to install images directly to a SystemImager image server. System Configurator, which is also used by the standard SystemImager release, performs configuration of target machine uniquenesses such as IP addresses, network cards, and initial RAM disks needed to boot clients after installation.

NEWS CONTENTS

• 171226 : Package Repository · finley-SystemImager Wiki · GitHub ( Package Repository · finley-SystemImager Wiki · GitHub, )
• 171226 : ( , )
• 171226 : ( , )

Old News ;-)

Package Repository · finley-SystemImager Wiki · GitHub

Brian Elliott Finley edited this page on Oct 2 2016 · 4 revisions

• Home
• Development
• Documentation
• Package Repository

Simple style -- just copy and paste this one-liner to install the repo config file:

curl -s http://systemimager.org/install-repo.sh | sudo sh

Alternate Method

Choose the appropriate chunk below, then scrape and paste the entire chunk at once. No need to do one line at a time.

Ubuntu and Debian

# choose either the stable or testing repo
REPO=stable
REPOFILE=systemimager.org-$REPO.list
URL=http://download.systemimager.org/sources.list.d/$REPOFILE
sudo wget $URL -O /etc/apt/sources.list.d/$REPOFILE
wget http://systemimager.org/pub/[email protected] -O - | sudo apt-key add -

Red Hat, CentOS, and friends

# choose either the stable or testing repo
REPO=stable
REPOFILE=systemimager.org-${REPO}.repo
RELEASE=$(uname -r | sed -r -e 's/.*((el|fc)[0-9]+).*/\1/')
URL=http://download.systemimager.org/pub/yum.repos.d/$REPOFILE.$RELEASE
cd /etc/yum.repos.d/
curl --url $URL --output $REPOFILE
rpm --import http://systemimager.org/pub/[email protected] 

Browse the Repo

http://systemimager.org/pub/

Recommended Links

Softpanorama Recommended

Top articles

Sites

Top articles

Sites

Index of -pub-systemimager

	systemimager-datasheet-1.5.0.pdf	07-Aug-2015 17:45	113K
	systemimager-manual-v4.1.6.pdf	07-Aug-2015 17:10	472K

Package Repository · finley-SystemImager Wiki · GitHub

SystemImager - Wikipedia

SystemImager CF TWiki

SystemImager and BitTorrent: a peer-to-peer approach for Large-Scale OS Deployment

• SystemImager: http://www.systemimager.org/
• Creating Images Of Your Linux System With SystemImager: http://www.falkotimme.com/howtos/systemimager/index.php and http://www.howtoforge.com/howto_linux_systemimager
• My Restore Script: http://www.falkotimme.com/howtos/dedicated_server_backup_restore_systemimager/restore
• BusyBox: http://www.busybox.net/
• Strato: http://www.strato.de/