systemd and Services

What systemd is, and why it replaced init scripts#

systemd is the init system on nearly every modern Linux distribution (Ubuntu, Debian, RHEL/Fedora, Arch) — the very first process the kernel starts (PID 1), responsible for bringing up every other service, managing dependencies between them, and supervising them for the rest of the machine's uptime. It replaced older init systems (SysV init, Upstart) that managed services with shell scripts in /etc/init.d/ — systemd's equivalent is a declarative unit file, and it natively understands dependencies, parallel startup, automatic restarts, and structured logging, none of which a plain shell script gets for free.

Writing a unit file#

A unit file describing a service is a plain text file, conventionally placed in /etc/systemd/system/ for anything you manage yourself (as opposed to /usr/lib/systemd/system/, reserved for units installed by packages).

# /etc/systemd/system/myapp.service
[Unit]
Description=My App API server
After=network.target postgresql.service
Requires=postgresql.service

[Service]
Type=simple
User=deploy
Group=deploy
WorkingDirectory=/var/www/myapp
ExecStart=/usr/bin/node /var/www/myapp/server.js
Restart=on-failure
RestartSec=5
Environment=NODE_ENV=production
EnvironmentFile=/var/www/myapp/.env

[Install]
WantedBy=multi-user.target

Breaking down the three sections every unit file has:

• [Unit] — metadata and ordering. Description is just a human-readable label shown in status output. After=network.target postgresql.service says "don't even attempt to start until networking and Postgres have started" — this only controls order, not a hard dependency. Requires=postgresql.service is the actual dependency: if Postgres fails to start, systemd won't start this service either (and will stop this one if Postgres later stops).
• [Service] — how to actually run and supervise the process. Type=simple (the default) tells systemd the process in ExecStart is the main process and stays running in the foreground — other Type values exist for daemons that fork and background themselves. User/Group run the process as an unprivileged account rather than root, the same least-privilege principle covered for containers in the Docker security page. Restart=on-failure with RestartSec=5 means systemd automatically restarts the process 5 seconds after a crash — without this, a crashed process just stays dead until someone notices.
• [Install] — what happens on systemctl enable. WantedBy=multi-user.target means "start this automatically as part of normal multi-user boot," which is what makes the service survive a reboot without manual intervention.

Controlling the service with systemctl#

After adding or editing a unit file, systemd needs to be told to re-read it:

$ sudo systemctl daemon-reload

Skipping this step is the single most common reason "I edited the unit file but nothing changed" — systemd caches parsed unit files and won't notice an edit until told to reload.

# Start it right now (does not affect whether it starts on boot)
$ sudo systemctl start myapp

# Enable it to start automatically on every future boot
# (does not start it right now, unless combined with --now)
$ sudo systemctl enable myapp

# Do both in one step — the usual command for a new service
$ sudo systemctl enable --now myapp

# Check its current status
$ sudo systemctl status myapp
● myapp.service - My App API server
     Loaded: loaded (/etc/systemd/system/myapp.service; enabled)
     Active: active (running) since Wed 2026-08-26 09:14:03 UTC; 2min ago
   Main PID: 18420 (node)
      Tasks: 11
     Memory: 84.2M
        CPU: 1.203s
     CGroup: /system.slice/myapp.service
             └─18420 /usr/bin/node /var/www/myapp/server.js

# Stop it, and disable it from starting on future boots
$ sudo systemctl stop myapp
$ sudo systemctl disable myapp

# Restart after a config or code change
$ sudo systemctl restart myapp

# Reload config without a full restart (only for services whose
# ExecReload is defined — not every unit supports this)
$ sudo systemctl reload myapp

start/stop and enable/disable are genuinely independent axes, and mixing them up is a classic mistake: start affects only the current boot session; enable affects only future boots. A service that's enabled but never started won't be running right now; a service that's started but not enabled will vanish on the next reboot.

Viewing logs with journalctl#

systemd captures every managed service's stdout/stderr automatically into the journal, a structured, indexed log store — no separate logging setup required for a basic service, unlike the older convention of every daemon managing its own log file under /var/log/.

# All logs for one specific service
$ journalctl -u myapp

# Follow logs live, exactly like tail -f
$ journalctl -u myapp -f

# Only logs since the last boot
$ journalctl -u myapp -b

# Only the last 50 lines
$ journalctl -u myapp -n 50

# Only errors and worse (see the priority table below)
$ journalctl -u myapp -p err

# Logs in a specific time range
$ journalctl -u myapp --since "2026-08-26 09:00" --until "2026-08-26 10:00"

Priority levels, from most to least severe — useful for filtering with -p:

journalctl -u myapp -p err shows entries at err and every level more severe than it (crit, alert, emerg) — exactly the first place to look when systemctl status shows a service as failed and you need to know why.

Common mistakes#

• Editing a unit file and skipping systemctl daemon-reload — systemd keeps using its cached copy of the old file until explicitly told to re-read it.
• Confusing enable with start (or disable with stop) — they control independent things: whether it's running right now versus whether it starts automatically on the next boot.
• Omitting Restart=on-failure (or an equivalent restart policy) on a long-running service — without it, a crash leaves the service down indefinitely until someone notices and restarts it manually.
• Running a service's ExecStart as root when a dedicated User=/Group= would do — the same least-privilege reasoning that applies to containers applies here too.
• Forgetting After=/Requires= for a real dependency (like a database) — without it, the service can start before its dependency is ready and fail intermittently, especially right after a full system reboot when everything starts at once.