Systemd tutorial

19 December 2011

You've read all about systemd, the new Linux init daemon. You know what it does, and why. Now it's time to dig in and learn how to make it sit up and beg — or at least start, stop, and get information on services.

Starting and Stopping Services

My earlier piece, "Here We Go Again, Another Linux Init: Intro to systemd" discusses the concepts behind systemd and what it is supposed to do. Now it's time to learn how to use it to control services on our systems. systemd is backwards-compatible with sysvinit and Upstart, so you can try it out by installing it on any Linux that uses sysvinit or Upstart without a lot of extra work. Arch Linux, Debian, and OpenSUSE all include systemd in their software repositories.

A conspicuous omission from distros that support systemd is Ubuntu. There are various reasons given for this, and I don't care because I'm tired of geekfights and just want to get on with things. Another way is to fetch yourself a copy of Fedora 15 or 16, which runs systemd by default.

systemadm is a nice graphical systemd manager (figure 1). It's still a baby so it might crash or something, but you can try it by installing the systemd-gtk package on Fedora, the systemd package on Arch, or the systemd-gui for Debian. No, of course distros cannot use consistent package naming, because that is against the rules.

As pretty as systemadm is, let us adjourn to the command line for the rest of this article. Watch the prompts in the examples to see which ones require root privileges. You can see the status of everything systemd controls by running systemctl with no options:

$ systemctl
UNIT                                              LOAD   ACTIVE SUB       JOB DESCRIPTION
proc-sys-fs-binfmt_misc.automount                 loaded active waiting       Arbitrary Executable File Formats File System Aut
sys-devices-pci0...000:00:1b.0-sound-card0.device loaded active plugged       82801I (ICH9 Family) HD Audio Controller
sys-devices-pci0...-0000:05:00.0-net-wlan0.device loaded active plugged       Centrino Wireless-N 1000

How do you see only available services, running or not?

$ systemctl list-units -t service --all
UNIT              LOAD   ACTIVE   SUB   JOB   DESCRIPTION
ceph.service      loaded inactive dead        LSB: Start Ceph distributed filesystem daemons at boot time
chronyd.service   loaded active   running     NTP client/server
cman.service      error  inactive dead        cman.service

These will spit out a whole lot of output, probably a hundred lines or more. Want to see only active services?

$ systemctl list-units -t service

You can check the status of any individual service, such as the cman.service, which has an error flag in the previous example:

 $ systemctl status cman.service
cman.service
	  Loaded: error (Reason: No such file or directory)
	  Active: inactive (dead)

How nice, it tells us what the problem is. Here is what a normal running service looks like, with complete information on the service:

$ systemctl status sshd.service
sshd.service - OpenSSH server daemon
	  Loaded: loaded (/lib/systemd/system/sshd.service; enabled)
	  Active: active (running) since Thu, 15 Dec 2011 12:11:05 -0800; 2h 26min ago
	Main PID: 2091 (sshd)
	  CGroup: name=systemd:/system/sshd.service
		    \   2091 /usr/sbin/sshd -D

On Fedora you can still use the old service and chkconfig commands. But why not start learning the new systemd commands? This is how to start a service:

# systemctl start sshd.service

Or use restart to restart a running service. This stops a service:

# systemctl stop sshd.service

Those are in effect only for the current session, so if you want a service to start at boot do this:

# systemctl enable sshd.service

And that's all. No hassling with startup scripts. This disables it from starting at boot:

# systemctl disable sshd.service

You can check to see if a service is already running; 0 means it is currently running and 1 means it is not:

$ systemctl is-enabled sshd.service; echo $? 
enabled
0

You can use systemctl halt, poweroff, or reboot to shutdown or reboot the system. All three send a wall message to users warning that the system is going down.

Processes, cgroups, and Killing

systemd organizes processes with cgroups, and you can see this with the ps command, which has been updated to show cgroups. Run this command to see which service owns which processes:

$ ps xawf -eo pid,user,cgroup,args 
  PID USER     CGROUP                      COMMAND
 1338 root     name=systemd:/user/carla/2       \_ gdm-session-worker [pam/gdm-password]
 1358 carla    name=systemd:/user/carla/2           \_ /bin/sh /etc/xdg/xfce4/xinitrc
 1487 carla    name=systemd:/user/carla/2               \_ /usr/bin/ssh-agent /bin/sh -c exec -l /bin/bash -c "startxfce4"
 1515 carla    name=systemd:/user/carla/2               \_ xscreensaver -no-splash
 1517 carla    name=systemd:/user/carla/2               \_ xfce4-session

cgroups were introduced into the Linux kernel a few years ago, and they are an interesting mechanism for allocating and limiting kernel resources. In systemd, cgroups are used to corral and manage processes. When new processes are spawned they become members of the parent's cgroup. The cgroup is named for the service it belongs to, and services cannot escape from their cgroups so you always know what service they belong to.

When you need to kill a service you can kill the cgroup, and snag all of its processes in one swoop instead of playing find-the-fork-or-renamed-process. Another way to view the process hierarchy is with the system-cgls command, as shown in Figure 2.

Figure 2: Partial output from system-cgls showing process cgroups.

There is my old friend Avahi daemon. So instead of hunting down and killing the two Avahi processes the old-fashioned way, systemd lets me do it in one command:

# systemctl kill avahi-daemon.service

Lennart Poettering, the main author of systemd, has written a series of articles for sysadmins. They're not indexed, so here are links for your convenience. These cover customizing startup services, runlevels, gettys, and everything you need to know to control systemd

NEWS CONTENTS

• 200102 : Understanding and Using Systemd ( Understanding and Using Systemd , )

Old News ;-)

Understanding and Using Systemd

Thursday, 18 September 2014 15:36 Carla Schroder

Image courtesy Wikimedia Commons, CC BY-SA 3.0

Like it or not, systemd is here to stay, so we might as well know what to do with it.

systemd is controversial for several reasons: It's a replacement for something that a lot of Linux users don't think needs to be replaced, and the antics of the systemd developers have not won hearts and minds. But rather the opposite, as evidenced in this famous LKML thread where Linus Torvalds banned systemd dev Kay Sievers from the Linux kernel.

It's tempting to let personalities get in the way. As fun as it is to rant and rail and emit colorful epithets, it's beside the point. For lo so many years Linux was content with SysVInit and BSD init. Then came add-on service managers like the service and chkconfig commands. Which were supposed to make service management easier, but for me were just more things to learn that didn't make the tasks any easier, but rather more cluttery.

Then came Upstart and systemd, with all kinds of convoluted addons to maintain SysVInit compatibility. Which is a nice thing to do, but good luck understanding it. Now Upstart is being retired in favor of systemd, probably in Ubuntu 14.10, and you'll find a ton of systemd libs and tools in 14.04. Just for giggles, look at the list of files in the systemd-services package in Ubuntu 14.04:

$ dpkg -L systemd-services

Check out the man pages to see what all of this stuff does.

It's always scary when developers start monkeying around with key Linux subsystems, because we're pretty much stuck with whatever they foist on us. If we don't like a particular software application, or desktop environment, or command there are multiple alternatives and it is easy to use something else. But essential subsystems have deep hooks in the kernel, all manner of management scripts, and software package dependencies, so replacing one is not a trivial task.

So the moral is things change, computers are inevitably getting more complex, and it all works out in the end. Or not, but absent the ability to shape events to our own liking we have to deal with it.

First systemd Steps

Red Hat is the inventor and primary booster of systemd, so the best distros for playing with it are Red Hat Enterprise Linux, RHEL clones like CentOS and Scientific Linux, and of course good ole Fedora Linux, which always ships with the latest, greatest, and bleeding-edgiest. My examples are from CentOS 7.

Experienced RH users can still use service and chkconfig in RH 7, but it's long past time to dump them in favor of native systemd utilities. systemd has outpaced them, and service and chkconfig do not support native systemd services.

Our beloved /etc/inittab is no more. Instead, we have a /etc/systemd/system/ directory chock-full of symlinks to files in /usr/lib/systemd/system/. /usr/lib/systemd/system/ contains init scripts; to start a service at boot it must be linked to /etc/systemd/system/. The systemctl command does this for you when you enable a new service, like this example for ClamAV:

# systemctl enable [email protected]
ln -s '/usr/lib/systemd/system/[email protected]' '/etc/systemd/system/multi-user.target.wants/[email protected]'

How do you know the name of the init script, and where does it come from? On Centos7 they're broken out into separate packages. Many servers (for example Apache) have not caught up to systemd and do not have systemd init scripts. ClamAV offers both systemd and SysVInit init scripts, so you can install the one you prefer:

$ yum search clamav
clamav-server-sysvinit.noarch
clamav-server-systemd.noarch

So what's inside these init scripts? We can see for ourselves:

$ less /usr/lib/systemd/system/[email protected]
.include /lib/systemd/system/[email protected]
[Unit]
Description = Generic clamav scanner daemon
[Install]
WantedBy = multi-user.target

Now you can see how systemctl knows where to install the symlink, and this init script also includes a dependency on another service, [email protected].

systemctl displays the status of all installed services that have init scripts:

$ systemctl list-unit-files --type=service
UNIT FILE              STATE
[...]
chronyd.service        enabled
[email protected]         static
[email protected]     disabled

There are three possible states for a service: enabled or disabled, and static. Enabled means it has a symlink in a .wants directory. Disabled means it does not. Static means the service is missing the [Install] section in its init script, so you cannot enable or disable it. Static services are usually dependencies of other services, and are controlled automatically. You can see this in the ClamAV example, as [email protected] is a dependency of [email protected], and it runs only when [email protected] runs.

None of these states tell you if a service is running. The ps command will tell you, or use systemctl to get more detailed information:

$ systemctl status bluetooth.service
bluetooth.service - Bluetooth service
   Loaded: loaded (/usr/lib.systemd/system/bluetooth.service; enabled)
   Active: active (running) since Thu 2014-09-14 6:40:11 PDT
  Main PID: 4964 (bluetoothd)
   CGroup: /system.slice/bluetooth.service
           |_4964 /usr/bin/bluetoothd -n 

systemctl tells you everything you want to know, if you know how to ask.

Cheatsheet

These are the commands you're probably going to use the most:

# systemctl start [name.service]
# systemctl stop [name.service]
# systemctl restart [name.service]
# systemctl reload [name.service]
$ systemctl status [name.service]
# systemctl is-active [name.service]
$ systemctl list-units --type service --all

systemd has 12 unit types. .service is system services, and when you're running any of the above commands you can leave off the .service extension, because systemd assumes a service unit if you don't specify something else. The other unit types are:

• Target: group of units
• Automount: filesystem auto-mountpoint
• Device: kernel device names, which you can see in sysfs and udev
• Mount: filesystem mountpoint
• Path: file or directory
• Scope: external processes not started by systemd
• Slice: a management unit of processes
• Snapshot: systemd saved state
• Socket: IPC (inter-process communication) socket
• Swap: swap file
• Timer: systemd timer.

It is unlikely that you'll ever need to do anything to these other units, but it's good to know they exist and what they're for. You can look at them:

$ systemctl list-units --type [unit name]

Blame Game

For whatever reason, it seems that the proponents of SysVInit replacements are obsessed with boot times. My systemd systems, like CentOS 7, don't boot up all that much faster than the others. It's not something I particularly care about in any case, since most boot speed measurements only measure reaching the login prompt, and not how long it takes for the system to completely start and be usable. Microsoft Windows has long been the champion offender in this regards, reaching a login prompt fairly quickly, and then taking several more minutes to load and run nagware, commercialware, spyware, and pretty much everything except what you want. (I swear if I see one more stupid Oracle Java updater nag screen I am going to turn violent.)

Even so, for anyone who does care about boot times you can run a command to see how long every program and service takes to start up:

$ systemd-analyze blame
  5.728s firewalld.service
  5.111s plymouth-quit-wait.service
  4.046s tuned.service
  3.550s accounts.daemon.service
  [...]

And several dozens more. Well that's all for today, folks. systemd is already a hugely complex beast; consult the References section to learn more.

Recommended Links

Softpanorama Recommended

Top articles

Sites

Top articles

Sites

...

• Rethinking PID 1
• systemd for Administrators, Part 1
• systemd for Administrators, Part II
• systemd for Administrators, Part III
• systemd for Administrators, Part IV
• systemd for Administrators, Part V
• systemd for Administrators, Part VI
• systemd for Administrators, Part VII
• systemd for Administrators, Part VIII
• systemd for Administrators, Part IX
• systemd for Administrators, Part X
• systemd for Administrators, Part XI

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotes :  Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce :  Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS :  Programming Languages History : PL/1 : Simula 67 : C : History of GCC development :  Scripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month :  How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: March, 12, 2019