|Contents||Bulletin||Scripting in shell and Perl||Network troubleshooting||History||Humor|
|News||Unix Configuration Management Tools||Recommended Links||git log|
|Parallel command execution||Config files distribution: copying a file to multiple hosts||Slurping: copying a file from multiple hosts||Configuration Files Generation||SSH for System Administrators||rdist|
|Provisioning||Webmin||rsync||IBM Remote System Management Tool||LCFG A large scale UNIX configuration system||Baseliners|
|Software Distribution||Simple Unix Backup Tools||Enterprise Job schedulers||Sysadmin Horror Stories||Humor||Etc|
etckeeper is a collection of tools to let
/etc be stored in a git, mercurial, bazaar
or darcs repository. This lets you use git to review or revert changes that were made to
Or even push the repository elsewhere for backups or cherry-picking configuration changes. It was created
by Joey Hess
It hooks into package managers like yum and apt to automatically commit changes made to /etc during package
upgrades. It tracks file metadata that git does not normally support, but that is important for /etc,
such as the permissions of
It's quite modular and configurable, while also being simple to use. Still you need to understand the basics of working with version control to use it effectivly.
First, a big warning: By checking /etc into version control, you are creating a copy of files like /etc/shadow that must remain secret. Anytime you have a copy of a secret file, it becomes more likely that the file contents won't remain secret. etckeeper is careful about file permissions, and will make sure that repositories it sets up don't allow anyone but root to read their contents. However, you also must take care when cloning or copying these repositories, not to allow anyone else to see the data.
Since git mushes all the files into packs under the .git directory, the whole .git directory content needs to be kept secret. (Ditto for mercurial and .hg as well as bazaar and .bzr)
Also, since version control systems don't keep track of the mode of files like the shadow file, it will check out world readable, before etckeeper fixes the permissions. The tutorial has some examples of safe ways to avoid these problems when cloning an /etc repository.
Also note that
etckeeper init runs code stored in the repository. So don't use it on
repositories from untrusted sources.
etckeeper will notice if it's being run by way of sudo, and makes a commit with the author set to
the user who sudoed to root. This is useful when a system has multiple admins; as long as they use sudo
while doing their administration, and run
etckeeper commit to commit their changes,
git blame can show who was responsible for each change.
etckeeper has special support to handle changes to /etc caused by installing and upgrading
packages. Before apt installs packages,
etckeeper pre-install will check that /etc contains
no uncommitted changes. After apt installs packages,
etckeeper post-install will add any
new interesting files to the repository, and commit the changes.
You can also run
etckeeper commit by hand to commit changes.
There is also a cron job, that will use etckeeper to automatically commit any changes to /etc each day.
Version Control Systems are designed as a way to manage source code, not as a way to manage arbitrary directories like /etc. This means there are a few limitations that etckeeper has to work around. These include file metadata storage, empty directories, and special files.
Most VCS, including git, mercurial and bazaar have only limited tracking of file metadata, being
able to track the executable bit, but not other permissions or owner info. (darcs doesn't even track
executable bits.) So file metadata is stored separately. Among other chores,
sets up a
pre-commit hook that stores metadata about file owners and permissions into a
/etc/.etckeeper file. This metadata is stored in version control along with everything
else, and can be applied if the repo should need to be checked back out.
git and mercurial cannot track empty directories, but they can be significant sometimes in /etc.
pre-commit hook also stores information that can be used to recreate the empty directories
Most VCS don't support several special files that you probably won't have in /etc, such
as unix sockets, named pipes, hardlinked files (but symlinks are fine), and device files. The
pre-commit hook will warn if your /etc contains such special files.
Darcs doesn't support symlinks, so they are also stored in
A quick walkthrough of using etckeeper.
Note that the default VCS is git, and this tutorial assumes you're using it. Using other VCSes should be broadly similar.
etckeeper init command initialises an /etc/.git/ repository. If you installed etckeeper
from a package, this was probably automatically performed during the package installation. If not, your
first step is to run it by hand:
etckeeper init command is careful to never overwrite existing files or directories
in /etc. It will create a
.gitignore if one doesn't already exist (or update content inside
a "managed by etckeeper" comment block), sets up pre-commit hooks if they don't already exist, and so
on. It does not commit any files, but does
git add all interesting files for an
initial commit later.
Now you might want to run
git status to check that it includes all the right files,
and none of the wrong files. And you can edit the
.gitignore and so forth. Once you're
ready, it's time to commit:
cd /etc git status git commit -m "Baseline of etc" git gc # pack git repo to save a lot of space
After this first commit, you can use regular git commands to handle further changes:
passwd joeuser git status git commit -a -m "changed a password for joeuser"
Rinse, lather, repeat. You might find that some files are changed by daemons and shouldn't be tracked by git. These can be removed from git:
git rm --cached printcap # modified by CUPS echo printcap >> .gitignore git commit -a -m "don't track printcap"
etckeeper hooks into yum (and similar systems) so changes to interesting files in /etc caused by
installing or upgrading packages will automatically be committed. Here "interesting" means files that
are not ignored by
You can use any git commands you like, but do keep in mind that, if you check out a different branch or an old version, git is operating directly on your system's /etc. If you do decide to check out a branch or tag, make sure you run "etckeeper init" again, to get any metadata changes:
git checkout april_first_joke_etc etckeeper init
Often it's better to clone /etc to elsewhere and do potentially dangerous stuff in a staging
directory. You can clone the repository using git clone, but be careful that the directory it's
cloned into starts out mode 700, to prevent anyone else from seeing files like
etckeeper init fixes their permissions:
mkdir /my/workdir cd /my/workdir chmod 700 . git clone /etc cd etc etckeeper init -d . chmod 755 ..
Another common reason to clone the repository is to make a backup to a server. When using
push to create a new remote clone, make sure the new remote clone is mode 700! (And, obviously,
only push over a secure transport like ssh, and only to a server you trust.)
ssh server 'mkdir /etc-clone; cd /etc-clone; chmod 700 .; git init --bare' git remote add backup ssh://server/etc-clone git push backup --all
If you have several machine's using etckeeper, you can start with a etckeeper repository on one machine, then add another machine's etckeeper repository as a git remote. Then you can diff against it, examine its history, merge with it, and so on. It would probably not, however, be wise to "git checkout" the other machine's branch! (And if you do, make sure to run "etckeeper init" to update file permissions.)
root@darkstar:/etc>git remote add dodo ssh://dodo/etc root@darkstar:/etc>git fetch dodo root@darkstar:/etc>git diff dodo/master group |head diff --git a/group b/group index 0242b84..b5e4384 100644 --- a/group +++ b/group @@ -5,21 +5,21 @@ sys:x:3: adm:x:4:joey tty:x:5: disk:x:6: -lp:x:7:cupsys +lp:x:7:
Incidentally, this also means I have a backup of dodo's /etc on darkstar. So if darkstar is compromised, that data could be used to attack dodo too. On the other hand, if dodo's disk dies, I can restore it from this handy backup.
Of course, it's also possible to pull changes from a server onto client machines, to deploy changes to /etc. Once /etc is under version control, the sky's the limit..
The main configuration file is
etckeeper runs the executable files in
/etc/etckeeper/$command.d/. (It ignores the same
ones that run-parts(1) would ignore.) You can modify these files, or add your own custom files. Each
individual file is short, simple, and does only one action.
For example, here's how to configure it to run
git gc after each apt run, which will
save a lot of disk space:
cd /etc/etckeeper/post-install.d (echo '#!/bin/sh' ; echo 'exec git gc') > 99git-gc chmod +x 99git-gc git add . git commit -m "run git gc after each apt run"
Here's how to disable the automatic commits after each apt run, while still letting it git add new files:
By default, etckeeper uses git. This choice has been carefully made; git is the VCS best supported by etckeeper and the VCS users are most likely to know.
[ It's possible that your distribution has chosen to modify etckeeper so its default VCS is not git -- if they have please complain to them, as they're making things unnecessarily difficult for you, and causing unnecessary divergence of etckeeper installations. You should only be using etckeeper with a VCS other than git if you're in love with the other VCS. ]
If you would like to use some other VCS, and
etckeeper init has already been run to
set up a git repository, you have a decision to make: Is the history recorded in that repository something
you need to preserve, or can you afford to just blow it away and check the current /etc into the new
In the latter case, you just need to follow three steps:
etckeeper uninit # deletes /etc/.git! vim /etc/etckeeper/etckeeper.conf etckeeper init
In the former case, you will need to convert the git repository to the other VCS using whatever tools
are available to do that. Then you can run
etckeeper uninit, move files your new VCS will
use into place, edit
etckeeper.conf to change the VCS setting, and finally
init. This procedure is clearly only for the brave.
Two blog posts provided inspiration for techniques used by etckeeper:
isisetup had some of the same aims as etckeeper, however, unlike it, etckeeper does not aim to be a git porcelain with its own set of commands for manipulating the /etc repository. Instead, etckeeper provides a simple setup procedure and hooks for setting up an /etc repository, and then gets out of your way; you manage the repository using regular VCS commands.
A valuable tool I have been using for many years is etckeeper, it works by essentially turning your /etc directory into a git repository. This is a fantastically useful set of tools as any configuration changes can be logged and also reverted quite easily. Install and setup is exeptionally easy too! Packages are available for most distributions, but my scenario (Fedora,CentOS,RHEL) was:
yum install etckeeper
Once the package was installed an initialisation must be performed:
This essentially runs a “git init” in the /etc directory setting up the directory ready.
That’s all there is to the installation.
Using it is a matter of committing changes when they are made, my workflow generally consists of running a check to see if all previous changes were committed, make the change, commit the change.
Will check the /etc directory for uncommitted changes, if they exist they can be committed in the same way as any new changes:
Running this command will present the familiar commit log screen in your favourite editor as it is essentially running a git commit from within the etc directory. Once the commit log is saved any changes are then stored within the version control system. A cron job is also in place to ensure a daily commit takes place, incase commits have been missed.
Now this is cool and extremely useful, but extending the git elements to push to a remote repository gives your etc that extra bit of resilience. Hook scripts are already present within /etc/etckeeper/commit.d/99push to recognise if a remote repository is configured and push to it on commit. Adding a remote repository is fairly simple, in my case I push to a gitlab (think self hosted github) server which I run.
First up a repository needs to be created in which to push to, I won’t go into detail here as there are hundreds if not thousands of Git tutorials out there. Gitlab has a repository created for each server and the ssh public key of each server stored to enable access.
git remote add origin git@gitlab01:etckeeper/server01.git
git push -u origin master
Will set the remote repository and populate it.
The last element to configure is the etckeeper config file, changing
(or whatever branch you choose to use)
And thats it! You an amazingly simple piece of software which could potentially save your Apache server, your Dovecot server or maybe even your job!
Softpanorama hot topic of the month
Very helpful Git command cheat sheet: http://cheat.errtheblog.com/s/git
Git on Wikipedia: http://en.wikipedia.org/wiki/Git_(software)
Git for the lazy: http://www.spheredev.org/wiki/Git_for_the_lazy
Starting git using just 10 commands: http://blog.xkoder.com/2008/08/13/git-tutorial-starting-with-git-using-just-10-commands/
How To Manage -etc with Version Control Using Etckeeper on CentOS 7 DigitalOcean
Etckeeper - ArchWiki
Seeing the fnords
Keep Configs Under Control With Etckeeper -- ServerWatch
FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of environmental, political, human rights, economic, democracy, scientific, and social justice issues, etc. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit exclusivly for research and educational purposes. If you wish to use copyrighted material from this site for purposes of your own that go beyond 'fair use', you must obtain permission from the copyright owner.
ABUSE: IPs or network segments from which we detect a stream of probes might be blocked for no less then 90 days. Multiple types of probes increase this period.
Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers : Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism : The Iron Law of Oligarchy : Libertarian Philosophy
War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda : SE quotes : Language Design and Programming Quotes : Random IT-related quotes : Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce : Bernard Shaw : Mark Twain Quotes
Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 : Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law
Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds : Larry Wall : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS : Programming Languages History : PL/1 : Simula 67 : C : History of GCC development : Scripting Languages : Perl history : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history
The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month : How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite
Most popular humor pages:
Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor
The Last but not Least
Copyright © 1996-2016 by Dr. Nikolai Bezroukov. www.softpanorama.org was created as a service to the UN Sustainable Development Networking Programme (SDNP) in the author free time. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License.
Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.
FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.
This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...
|You can use PayPal to make a contribution, supporting development of this site and speed up access. In case softpanorama.org is down you can use the at softpanorama.info|
The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the author present and former employers, SDNP or any other organization the author may be associated with. We do not warrant the correctness of the information provided or its fitness for any purpose.
Last modified: November, 04, 2016