Chapitre 6. The systemd daemon

Table des matières

6.1. Basic Usage
6.2. System Start and Target Management
6.3. Advanced Usage
6.4. More information

The program systemd is the process with process ID 1. It is responsible for initializing the system in the required way. systemd is started directly by the Kernel and resists signal 9, which normally kills processes. All other programs are either started directly by systemd or by one of its child processes.

[Note]Using System V init rather than systemd

Being fully compatible, both systemd and System V init (SysV init) can act as a drop-in replacement for each other. If you prefer to use the SysV init rather than systemd, press F5 on the boot screen and choose System V. To make this a permanent choice, install the package sysvinit-init which will replace the systemd-init package.

Starting with openSUSE 12 systemd is a replacement for the popular SysV init daemon. systemd is fully compatible with SysV init (by supporting init scripts). One of the main advantages of systemd is the fact that it considerably speeds up boot time by aggressively paralleling service starts. Furthermore, systemd only starts a service when it is really needed. For example, the printing daemon cupsd is not started during boot time, but rather when a user prints a document for the first time after having booted the system. systemd also supports Kernel Control Groups (cgroups), snapshotting and restoring the system state and more. See http://www.freedesktop.org/wiki/Software/systemd/ for details.

6.1. Basic Usage

The SysV init system utilized several different commands to handle services— the init scripts, insserv, telinit and others. systemd makes it easier to manage services, since there is only one command to memorize for the majority of service handling tasks: systemctl.It uses the command plus subcommand notation like git or zypper:

systemctl [general OPTIONS] subcommand [subcommand OPTIONS]

See man 1 systemctl for a complete manual.

[Tip]Terminal Output and Bash Completion

If the output goes to a terminal (and not to a pipe or a file, for example) systemd commands send long output to a pager by default. Use the --no-pager option to turn off paging mode.

systemd also supports bash-completion, allowing you to enter the first letters of a subcommand and then hit →| to automatically complete it. This feature is only available in the bash Shell and requires the installation of the package bash-completion.

6.1.1. Managing Services in a Running System

Subcommands for managing services are the same as for managing a service with SysV init (start, stop, ...). The general syntax for service management commands is as follows:

systemd
systemctl reload|restart|start|status|stop|... <my_service(s)>.service
SysV init
rc<my_service(s)> reload|restart|start|status|stop|...

systemd allows to manage several services in one go. Instead of executing init scripts one after the other as with SysV init, execute a command like the following:

systemctl start <my_1st_service>.service <my_2nd_service>.service

The following table shows a list of the most important service management commands for systemd and SysV init:

Tableau 6.1. Service Management Commands

Task

systemd Subcommand

SysV init Subcommand

Starting

start
start

Stopping

stop
stop

Restarting

Shuts down services and starts them afterwards. If a service is not yet running it will be started.

restart
restart

Restarting conditionally

Restarts services if they are currently running. Does nothing for services that are not running.

try-restart
try-restart

Reloading

Tells services to reload their configuration files without interrupting operation. Use case: Tell Apache to reload a modified httpd.conf configuration file. Note that not all services support reloading.

reload
reload

Reloading or restarting

Reloads services if reloading is supported, otherwise restarts them. Services is not yet running it will be started.

reload-or-restart
n/a

Reloading or restarting conditionally

Reloads services if reloading is supported, otherwise restarts them if currently running. Does nothing for services that are not running.

reload-or-try-restart
n/a

Getting detailed status information

Lists information about the status of services. The systemd command shows details such as description, executable, status, cgroup, and messages last issued by a service (see Section 6.1.2, « Debugging Services »). The level of details displayed with the SysV init differs from service to service.

status
status

Getting short status information

Shows whether services are active or not.

is-active
status

6.1.2. Debugging Services

By default, systemd is not overly verbose. If a service was started successfully, no output will be produced. In case of a failure, a short error message will be displayed. However, systemctl status provides means to debug start-up as well as operation of a service.

systemd comes with its own logging mechanism (« The Journal ») that logs system messages. This allows to display the service messages together with status messages. The status command works similar to tail and can also display the log messages in different formats, making it a powerful debugging tool.

Show Service Start-Up Failure

Whenever a service fails to start, use systemctl status <my_service>.service to get a detailed error message:

www.example.com: ~ # systemctl start apache2.service
Job failed. See system journal and 'systemctl status' for details.
www.example.com: ~ # systemctl status apache2.service
   Loaded: loaded (/lib/systemd/system/apache2.service; disabled)
   Active: failed (Result: exit-code) since Mon, 04 Jun 2012 16:52:26 +0200; 29s ago
   Process: 3088 ExecStart=/usr/sbin/start_apache2 -D SYSTEMD -k start (code=exited, status=1/FAILURE)
   CGroup: name=systemd:/system/apache2.service

Jun 04 16:52:26 g144 start_apache2[3088]: httpd2-prefork: Syntax error on line
205 of /etc/apache2/httpd.conf: Syntax error on li...alHost>
Show Last n Service Messages

The default behavior of the status subcommand is displaying the last ten messages a service issued. To change the number of messages to show, use the --lines=n parameter,

systemctl status ntp.service
systemctl --lines=20 status ntp.service
Show Service Messages in Append Mode

To display a « live stream » of service messages, use the --follow option, which works just like tail -f:

systemctl --follow status ntp.service
Messages Output Format

The --output=mode parameter allows to change the output format of service messages. The most important modes available are:

short

The default format. Shows the log messages with a human readable timestamp.

verbose

Full output with all fields.

cat

Terse output without time stamps.

6.1.3. Permanently Enabling/Disabling Services

The service management commands mentioned in the previous section let you manipulate services for the current session. systemd also lets you permanently enable or disable services, so they are automatically started when requested or are always unavailable. You can either do this by using YaST, or on the command line.

6.1.3.1. Enabling/Disabling Services on the Command Line

The following table shows a list of enabling and disabling commands for systemd and SysV init:

[Important]Service Start

When enabling a service on the command line, it is not started automatically. It is scheduled to be started with the next system start-up or runlevel/target change. In order to immediately start a service after having enabled it, explicitly run systemctl start <my_service>.service or rc<my_service> start.

Tableau 6.2. Commands for Enabling and Disabling Services

Task

systemd subcommand

SysV init subcommand

Enabling

systemctl enable <my_service(s)>.service

insserv <my_service(s)>

Disabling

systemctl disable <my_service(s)>.service

insserv -r <my_service(s)>

Checking

Shows whether a service is enabled or not.

systemctl is-enabled <my_service>.service

n/a

Re-Enabling

Similar to restarting a service, this commands first disables and then enables a service. Useful to re-enable a service with its defaults.

systemctl reenable <my_service>.service

n/a

Masking

When « disabling » a service, it can still be started manually. To completely disable a service, you need to mask it. Use with care.

systemctl mask <my_service>.service

n/a

Unmasking

A service that has been masked can only be used again after it has been unmasked.

systemctl unmask <my_service>.service

n/a


6.1.3.2. Enabling/Disabling Services with YaST

Start the YaST module with YaST+System+System Services (Runlevel). In the default view, the Simple Mode, it displays an overview listing all the available services and the current status of each service (see Figure 6.1, « System Services (Runlevel) »). The left column shows the name of the service, the center column indicates its current status and the right column gives a short description. For the selected service, a more detailed description is provided in the lower part of the window. To enable a service, select it in the table then select Enable. The same steps apply to disable a service.

The Expert Mode provides detailed control over the runlevels (refer to Section 6.2.1, « Targets vs. Runlevels » for more information on runlevels). The runlevel into which the system boots by default is displayed at the top. Normally it defaults to runlevel 5 (full multiuser mode with network and X). A suitable alternative might be runlevel 3 (full multiuser mode with network). Runlevel 4 is undefined to allow creation of a custom runlevel.

To enable or disable individual services use the table in this window. It lists the services available, shows whether they are currently enabled on your system and, if so, for which runlevels. To define the runlevels in which a service should be started, select the respective row with the mouse and activate the checkboxes below that represent the runlevels. A brief description of the currently selected service or daemon is provided below the table overview.

[Warning]Faulty Runlevel Settings May Damage Your System

Faulty runlevel settings may make your system unusable. Before applying your changes, make absolutely sure that you know their consequences.

Figure 6.1. System Services (Runlevel)

System Services (Runlevel)

With Start/Stop/Refresh, decide whether a service should be activated. Refresh status checks the current status. Set/Reset lets you enable or disable a service in the same manner as with the Simple Mode interface. Selecting OK saves the changed settings to disk.

6.2. System Start and Target Management

The entire process of starting the system and shutting it down is maintained by systemd. From this point of view, the Kernel can be considered a background process to maintain all other processes and adjust CPU time and hardware access according to requests from other programs.

6.2.1. Targets vs. Runlevels

With SysV init the system was booted into a so called « Runlevel ». A runlevel defines how the system is started and what services are available in the running system. Runlevels are numbered, the most commonly known ones are 0 (shutting down the system), 3 (multiuser with network) and 5 (multiuser with network and displaymanager).

systemd introduces a new concept by using so-called « target units ». However, it remains fully compatible to the runlevel concept. Target units are named rather than numbered and serve specific purposes. For example, the targets local-fs.target and swap.target mount local filesystems and swap spaces.

The target graphical.target provides a multiuser system with network and displaymanager capabilities and is equivalent to runlevel 5. Complex targets, such as graphical.target act as « meta » targets by combining a subset of other targets. Since systemd makes it easy to create custom targets by combining existing targets, it offers great flexibility.

The following list shows a list of the most important systemd target units. For a full list refer to man 7 systemd.special.

Selected systemd Target Units

default.target

The target that is booted by default. Not a « real » target, but rather a symlink to another target like graphic.target. Can be permanently changed via YaST (see Section 6.1.3.2, « Enabling/Disabling Services with YaST »). To change it for a session, use the Kernel command line option systemd.unit=<my_target>.target at the boot prompt.

emergency.target

Starts an emergency shell on the console. Only use it at the boot prompt as systemd.unit=emergency.target.

graphical.target

Starts a system with network, multi-user support and a displaymanager.

halt.target

Shuts down the system.

mail-transfer-agent.target

Starts all services necessary for sending and receiving mails.

multi-user.target

Starts a multi-user system with network.

reboot.target

Reboots the system.

rescue.target

Starts a single user system without network.

In order to remain compatible to the SysV init runlevel system, systemd provides special targets named runlevelX.target mapping the corresponding runlevels numbered X.

Tableau 6.3. SysV Runlevels and systemd Target Units

SysV runlevel

systemd target

Purpose

0

runlevel0.target, halt.target, poweroff.target

System shutdown

1, S

runlevel1.target, rescue.target,

Single user mode

2

runlevel2.target, multi-user.target,

Local multiuser without remote network

3

runlevel3.target, multi-user.target,

Full multiuser with network

4

runlevel4.target

Unused/User Defined

5

runlevel5.target, graphical.target,

Full multiuser with network and displaymanager

6

runlevel6.target, reboot.target,

System reboot


[Important]systemd Ignores /etc/inittab

The runlevels in a SysV init system are configured in /etc/inittab. systemd does not use this configuration. Please refer to Section 6.2.2, « Custom Targets » for instructions on how to create your own bootable target.

6.2.1.1. Commands to Change Targets

Use the following commands to operate with target units:

Task

systemd command

SysV init command

Change the current target/runlevel

systemctl isolate <my_target>.target

telinit X

Change to the default target/runlevel

systemctl default

n/a

Get the current target/runlevel

systemctl list-units --type=target

With systemd there is usually more than one active target. The command lists all targets currently active.

who -r

or

runlevel

Change the default runlevel persistently

Use the YaST runlevel editor or run the following command:

ln -sf /lib/systemd/system/<my_target>.target /etc/systemd/system/default.target

Use the YaST runlevel editor or change the line

id:X:initdefault:

in /etc/inittab

Change the default runlevel for the current boot process

Enter the following option at the boot prompt

systemd.unit=<my_target.target>

Enter the desired runlevel number at the boot prompt.

Show a target's/runlevel's dependencies

systemctl show -p "Requires" <my_target.target>

systemctl show -p "Wants" <my_target.target>

« Requires » lists the hard dependencies (the ones that must be resolved), whereas « Wants » lists the soft dependencies (the ones that get resolved if possible).

n/a

6.2.2. Custom Targets

On SysV init SUSE systems, runlevel 4 is unused to allow administrators to create their own runlevel configuration. systemd allows ro create any number of custom targets. It's suggested to start by adapting an existing target such as graphical.target.

[Warning]Avoiding Overwritten Customizations

Always do systemd customization in /etc/systemd, never in /lib/systemd. Otherwise your changes will be overwritten by the next update of systemd.

Procédure 6.1. Create a Custom Target

  1. Copy the configuration file /lib/systemd/system/graphical.target to /etc/systemd/system/<my_target>.target and adjust it according to your needs.

  2. The configuration file copied in the previous step already covers the required (« hard ») dependencies for the target. To also cover the wanted (« soft ») dependencies, create a directory /etc/systemd/system/<my_target>.target.wants.

  3. For each wanted service create a symbolic link from /lib/systemd/system into /etc/systemd/system/<my_target>.target.wants.

  4. Once you have finished setting up the target, reload the systemd configuration to make the new target available:

    systemctl daemon-reload

6.2.3. Debugging System Start-Up

systemd offers means to analyze the system start-up process. You can conveniently review the list of all services and their status (rather than having to parse /varlog/). systemd also allows to scan the start-up procedure to find out how much time each service start-up consumes.

6.2.3.1. Review Start-Up of Services

To review the complete list of services that have been started since booting the system, just enter the command systemctl. It lists all active services like shown below (shortened). To get more information on a specific service, use systemctl status <my_service>.service.

Exemple 6.1. List Active Services

jupiter.example.com: ~ # systemctl
UNIT                                LOAD   ACTIVE SUB       JOB DESCRIPTION
[...]
systemd-random-seed-load.path       loaded active waiting       Random Seed
acpid.service                       loaded active running       ACPI Event Daemon
apache2.service                     loaded failed failed        apache
avahi-daemon.service                loaded active running       Avahi mDNS/DNS-SD Stack
bluez-coldplug.service              loaded active exited        LSB: handles udev coldplug of bluetooth dongles
console-kit...-system-start.service loaded active exited        Console System Startup Logging
cron.service                        loaded active running       Command Scheduler
cups.service                        loaded active running       CUPS Printing Service
[...]
LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.
JOB    = Pending job for the unit.

107 units listed. Pass --all to see inactive units, too.

To restrict the output to services that failed to start, use the --failed option:

Exemple 6.2. List Failed Services

jupiter.example.com: ~ # systemctl --failed
UNIT                   LOAD   ACTIVE SUB    JOB DESCRIPTION
apache2.service        loaded failed failed     apache
NetworkManager.service loaded failed failed     Network Manager
plymouth-start.service loaded failed failed     Show Plymouth Boot Screen

[...]

6.2.3.2. Debug Start-Up Time

To debug system start-up time, systemd offers the systemd-analyze command. It shows the total start-up time, a list of services ordered by start-up time and can also generate an SVG graphic showing the time services took to start in relation to the other services.

Listing the System's Start-Up Time
jupiter.example.com: ~ # systemd-analyze
Startup finished in 2666ms (kernel) + 21961ms (userspace) = 24628ms
Listing the Services Start-Up Time
jupiter.example.com: ~ # systemd-analyze blame
  6472ms systemd-modules-load.service
  5833ms remount-rootfs.service
  4597ms network.service
  4254ms systemd-vconsole-setup.service
  4096ms postfix.service
  2998ms xdm.service
  2483ms localnet.service
  2470ms SuSEfirewall2_init.service
  2189ms avahi-daemon.service
  2120ms systemd-logind.service
  1210ms xinetd.service
  1080ms ntp.service
[...]
    75ms fbset.service
    72ms purge-kernels.service
    47ms dev-vda1.swap
    38ms bluez-coldplug.service
    35ms splash_early.service
Services Start-Up Time Graphics
jupiter.example.com: ~ # systemd-analyze plot > jupiter.example.com-startup.svg

6.2.3.3. Review the Complete Start-Up Process

Above mentioned commands let you review the services that started and the time it took to start them. If you need to know more details, you can tell systemd to verbosely log the complete start-up procedure by entering the following parameters at the boot prompt:

systemd.log_level=debug systemd.log_target=kmsg

Now systemd writes its log messages into the kernel ring buffer. View that buffer with dmesg:

dmesg | less

6.3. Advanced Usage

The following sections cover advanced topics for system administrators. For even more advanced systemd documentation, please refer Lennart Pöttering's series about systemd for administrators at http://0pointer.de/blog/projects.

6.3.1. System Log

Section 6.1.2, « Debugging Services » explains how to view log messages for given service. However, displaying log messages is not restricted to service logs. You can also access and query the complete log written by systemd—the so-called « Journal ». Use the command systemd-journalctl to display the complete log starting with the oldest entries. Refer to man 1 systemd-journalctl for options such as applying filters or changing the output format.

6.3.2. Snapshots

You can save the current state of systemd to a named snapshot and later revert to it with the isolate subcommand. This is useful when testing services or custom targets, because it allows you to return to a defined state at any time. A snapshot is only available in the current session and will automatically be deleted on reboot. A snapshot name must end in .snapshot.

Create a Snapshot
systemctl snapshot <my_snapshot>.snapshot
Delete a Snapshot
systemctl delete <my_snapshot>.snapshot
View a Snapshot
systemctl show <my_snapshot>.snapshot
Activate a Snapshot
systemctl isolate <my_snapshot>.snapshot

6.3.3. Kernel Control Groups (cgroups)

On a traditional SysV init system it is not always possible to clearly assign a process to the service that spawned it. Some services such as Apache, spawn a lot of 3rd party processes (e.g. CGI or Java processes), which themselves spawn more processes. This makes a clear assignment difficult or even impossible. Additionally, a service may not terminate correctly, leaving some of its children alive.

systemd solves this problem by placing each service into its own cgroup. cgroups are a Kernel feature that allows aggregating processes and all their children into hierarchical organized groups. systemd names each cgroup after its service. Since a non-privileged process it not allowed to « leave » its cgroup, this provides an effective way to label all processes spawned by a service with the name of the service.

To list all processes belonging to a service, use the command systemd-cgls. The result will look like the following (shortened) example:

Exemple 6.3. List all Processes Belonging to a Service

~ # systemd-cgls --no-pager
├ user
│ └ root
│   └ 1
│     ├ 2279 sshd: root@pts/0
│     ├ 2282 -bash
│     └ 2541 systemd-cgls --no-pager
└ system
  ├ 1 /sbin/init splash showopts
  ├ apache2.service
  │ ├ 2535 /usr/sbin/httpd2-prefork -f /etc/apache2/httpd.conf -D SYSTEMD -k start
  │ ├ 2536 /usr/sbin/httpd2-prefork -f /etc/apache2/httpd.conf -D SYSTEMD -k start
  │ ├ 2537 /usr/sbin/httpd2-prefork -f /etc/apache2/httpd.conf -D SYSTEMD -k start
  │ ├ 2538 /usr/sbin/httpd2-prefork -f /etc/apache2/httpd.conf -D SYSTEMD -k start
  │ ├ 2539 /usr/sbin/httpd2-prefork -f /etc/apache2/httpd.conf -D SYSTEMD -k start
  │ └ 2540 /usr/sbin/httpd2-prefork -f /etc/apache2/httpd.conf -D SYSTEMD -k start
  ├ xdm.service
  │ ├ 2250 /usr/bin/xdm
  │ ├ 2253 /usr/bin/X -nolisten tcp -br vt7 -auth /var/lib/xdm/authdir/authfiles/A:0-ii8Goo
  │ ├ 2263 -:0         
  │ └ 2276 /usr/bin/xconsole -notify -nostdin -verbose -exitOnFail
  ├ ntp.service
  │ └ 2202 /usr/sbin/ntpd -p /var/run/ntp/ntpd.pid -g -u ntp:ntp -c /etc/ntp.conf
  ├ sshd.service
  │ └ 1743 /usr/sbin/sshd -D
    

See Chapitre 10, Kernel Control Groups (↑System Analysis and Tuning Guide) for more information about cgroups.

6.3.4. Killing Services (Sending Signals)

As explained in Section 6.3.3, « Kernel Control Groups (cgroups) », it is not always possible to assign a process to its parent service process in a SysV init system. This makes it difficult to terminate a service and all of its children. Child processes that have not been killed will remain as zombie processes.

systemd's concept of to confine each service into a cgroup makes it possible to clearly identify all child processes of a service and therefore allows to send a signal to each of these processes. Use systemctl kill to send signals to services. For a list of available signals refer to man 7 signals.

Sending SIGTERM to a Service

SIGTERM is the default signal that is sent.

systemctl kill <my_service>.service
Sending SIGNAL to a Service

Use the -s option to specify the signal that should be sent.

systemctl kill -s SIGNAL <my_service>.service
Selecting Processes

By default the kill command send the signal to all processes of the specified cgroup. You can restrict it to the control or the main process. The latter is for example useful to force a service to reload its configuration by sending SIGHUP:

systemctl kill -s SIGHUP --kill-who=main <my_service>.service

6.4. More information

For more information on systemd refer to the following online resources:

Homepage

http://www.freedesktop.org/wiki/Software/systemd

systemd for Administrators

Lennart Pöttering, one of the systemd authors, has written a series of blog entries (13 at the time of writing this chapter). Find them at http://0pointer.de/blog/projects.

Control Centre: The systemd Linux init system

http://www.h-online.com/open/features/Control-Centre-The-systemd-Linux-init-system-1565543.html

Booting up: Tools and tips for systemd, a Linux init tool

http://www.h-online.com/open/features/Booting-up-Tools-and-tips-for-systemd-1570630.html