The author selected the Free and Open Source Fund to receive a donation as part of the Write for DOnations program. Show
IntroductionIn this two-part tutorial, you will learn how to configure a Linux service to restart automatically after a reboot or crash using Part One covers general Linux service
management concepts like the Part Two offers a step-by-step tutorial for completing a practical and common PrerequisitesTo complete this tutorial, you will need:
Introducing the Service Management DaemonLinux services can be made self-healing largely by changing the way they are handled by the service management daemon, also known as the
As Linux has evolved, so has the behavior of the
Most modern Linux distributions have gradually migrated away from System V and currently using systemd. Older style We will cover systemd in this article as this is the most recent and common service manager used in Linux distributions today. However, we will also talk about System V and Upstart when necessary and see how systemd evolved from there. To give you an idea:
To understand the RunlevelsA runlevel represents the current state of a Linux system. For example, a runlevel can be the shutdown state of a Linux server, a single-user mode, the restart mode, etc. Each mode will dictate what services can be running in that state. Some services can run in one or more runlevel but not in others. Runlevels are denoted by a value between 0 and 6. The following list shows what each of these levels means:
Runlevels 2, 3, and 4 vary by distribution. For example, some Linux distributions don’t implement runlevel 4, while others do. Some distributions have a clear distinction between these three levels. In general, runlevel 2, 3, or 4 means a state where Linux has booted in multi-user, network-enabled, text mode. When you enable a service to auto-start, Linux is actually adding it to a runlevel. In System V, for example, the OS will start with a particular runlevel; and, when it starts, it will try to start all the services that are associated with that runlevel. In systemd, runlevel has become target, and when a service is made to auto-start, it’s added to a target. We’ll discuss targets later in this article. Introducing the System V init DaemonSystem V uses an
So, when the An When a process or service such as MySQL Server starts under System V, its binary program file has to load into memory. Depending on how the service is configured, this program may continuously
execute the background (and accept client connections). The job of starting, stopping, or reloading this binary application is handled by the service’s In System V, an To make a service restart after a crash or reboot, you can usually add a line like this to the
To enable a System V service to start at system boot time, run this command:
To disable it, run this command:
To check the status (running or stopped), run this command
Introducing the Upstart DaemonAs the serialized way of loading jobs and services became more time consuming and complex with System V Upstart
To keep things simple, Upstart
is backward-compatible with System V. The
In between these events, a service can be in a number of states, like waiting, pre-start, starting, running, pre-stop, stopping, etc. Upstart could take action for each of these states as well, creating a very flexible architecture. At startup, Upstart will run any System V The files have a naming style of
/etc/init/cron.conf
Introducing the systemd DaemonThe latest in Linux One of its functions is to work as a system and service manager for Linux. In this capacity, systemd controls how a service should behave if it crashes or the machine reboots. You can read about systemd’s systemctl here. systemd is backward-compatible with System V commands and initialization scripts. That means any System V service will also run under systemd. This is possible because most Upstart and System V administrative commands have been modified to work under systemd. systemd Configuration Files: Unit FilesAt the heart of systemd are unit files. Each unit file represents a specific system resource. Information about the resource is kept track of in the unit file. Service unit files are simple text files (like Upstart .conf files) with a declarative syntax. This makes the files easy to understand and modify. The main difference between systemd and the other two Directory StructureIn Red Hat-based systems like CentOS, unit files are located
in two places. The main location is If a unit file with the same name exists in both locations, systemd will use the one under systemd init Sequence: Target UnitsA special type of unit file is a target unit. A target unit filename is suffixed by .target. Target units are different from other unit files because they don’t represent one particular resource. Rather, they represent the state of the system at any one time. Target units do this by grouping and launching multiple unit files that should be part of that state. systemd targets can therefore be loosely compared to System V runlevels, although they are not the same. Each target has a name instead of a number. For example, we have How it brings the server up to that stage is where the difference lies. Unlike System V, systemd does not bring up services sequentially. Along the way, it can check for the existence of other services or resources and decide the order of their loading. This makes it possible for services to load in parallel. Another difference between target units and runlevels is that in System V, a Linux system could exist in only one runlevel. You could change the runlevel, but the system would exist in that new runlevel only. With systemd, target units can be inclusive, which means when a target unit activates, it can ensure other target units are loaded as part of it. For example, a Linux system that boots with a graphical user interface will have the graphical.target activated, which in turn will automatically ensure multi-user.target is loaded and activated as well. In System V terms, that would be like having runlevels 3 and 5 activated simultaneously. The table below compares runlevels and targets:
systemd default.targetsystemd System V had the default runlevel defined in a file called inittab. In systemd, that file is replaced by default.target. The default target unit file lives under /etc/systemd/system directory. It’s a symbolic link to one of the target unit files under /lib/systemd/system. When you change the default target, you are essentially recreating that symbolic link and changing the system’s runlevel. The inittab file in System V also specified which directory Linux will execute its As units are activated, they are activated all in parallel or all in sequence. How a resource unit loads may depend on other resource units it wants or requires. systemd Dependencies: Wants and Requiressystemd wants and requires control how systemd addresses dependency among service daemons. As mentioned before, Upstart ensures parallel loading of services using configuration files. In System V, a service could start in a particular runlevel, but it also could be made to wait until another service or resource became available. In a similar fashion, systemd services can be made to load in one or more targets, or wait until another service or resource became active. In systemd, a unit that requires another unit will not start until the required unit is loaded and activated. If the required unit fails for some reason while the first unit is active, the first unit will also stop. This ensures system stability. A service that requires a particular directory to be present can thus be made to wait until the mount point to that directory is active. On the other hand, a unit that wants another unit will not impose such restrictions. It won’t stop if the wanted unit stops when the caller is active. An example of this would be the non-essential services that come up in graphical-target mode. Practical Example: Understanding the systemd Startup SequenceTo understand service startup behavior under systemd, we are using a CentOS 8.3 Droplet. We will follow the .target rabbit-trail as far as we can. systemd’s startup sequence follows a long chain of dependencies. First, let’s run this command to list the default target unit file:
This shows output like the following:
As you can see, the default target is actually a symbolic link to the multi-user target file under /lib/systemd/system/. So, the system is supposed to boot under multi-user.target, which is similar to runlevel 3 in System V init. multi-user.target.wantsNext, let’s run the following command to check all the services the multi-user.target file wants:
This should show a list of symbolic link files, pointing back to actual unit files under /usr/lib/systemd/system/:
Other than
The output shows:
basic.targetYou can run the following command to see if there are any required units for basic.target:
As it turns out, the basic.target requires sysinit.target:
And it wants a few targets as well:
The command will return the following:
Going recursively, you can see if the sysinit.target will require any other targets to be running as well:
There will be none. However, there will be other targets wanted by sysinit.target:
An output like this will appear:
Examining a systemd Unit FileGoing a step further now, let’s look inside a service unit file, the one for sshd:
It looks like this: /etc/systemd/system/multi-user.target.wants/sshd.service
You can see the service unit file is clean and easy to understand. The first important part is the After clause in the The Since ConclusionIn this article you learned about System V, Upstart, and systemd service management daemons. You explored startup scripts and configuration files, important parameters, startup sequence, and the commands that control service startup behavior. In part two of this article, we will apply these skills to a real-life example and use systemd to configure MySQL. Once completed, your MySQL instance will automatically restart after a reboot or crash. And while you will use MySQL as your example application, you can substitute any number of services, such as the Nginx or Apache web servers. |