Simple systemd Tricks

· 752 words · 4 minute read
Technology
systemd · linux · sysadmin

Per my earlier post I’ve been learning and using systemd more and more last year. I presented a couple of examples in my talk, but I thought I’d throw up some more; one rich source of writing-my-own-service-files has been Debian 7 boxes I’ve moved over to systemd. Since there are very few service files out of the box, there’s plenty of scope to roll my own to solve simple problems.

One example is the tt-rss update service:

::::IniLexer
[Unit]
Description=Tiny Tiny RSS update daemon
After=network.target postgresql.service
Requires=postgresql.service

[Service]
User=www-data
Group=www-data
WorkingDirectory=/xxx/xxx/tt-rss
Type=simple
ExecStart=/usr/bin/php ./update_daemon2.php
Restart=always
PrivateTmp=true

[Install]
WantedBy=multi-user.target

A few notes:

  • I grabbed this from a Fedora package. One nice thing about systemd units is that they’re pretty portable; changing the user and DB dependency (postgresql instead of mysql) was all that was required.
  • The daemon doesn’t double-fork, which is a pain for sysvinit, which then requires a wrapper to run it properly. A second nice thing about systemd is that I don’t need to mess about with runit, daemontools, the Debian’s start-stop-daemon, supervisord, or one of the numerous other options. And since systemd is widely adopted, I don’t have to guess which solution will be in vogue with the platform I’m working on.
  • It doesn’t bother starting the daemon until the database server is running. No point firing up services you know are going to fail. No debugging service races at startup time!
  • If it dies, Restart=always will start it up again. The tt-rss process is pretty reliable, but this directive is pretty useful for less stable stacks like some Rails containers or node.js.
  • If I was on a newer version of systemd I’d use the InaccessibleDirectories directive to reduce the attack surface presented by a daemon that goes out and pulls content from untrusted sources. Alas, Wheezy is stuck on systemd 44 so that’s not an option.

Another is my postfix unit:

::::IniLexer
[Unit]
Description=Postfix Mail Daemon
After=network.target
Requires=var-spool-mail.mount

[Service]
Type=forking
ExecStart=/usr/sbin/postfix start
ExecStop=/usr/sbin/postfix stop
Restart=always

[Install]
WantedBy=multi-user.target

Some notes here:

  • By way of comparison the postfix sysvinit script is 266 lines long.
  • This will only spin up postfix if my mail spool is actually there. No delivering mail to a n empty directory and then having a subsequent mount over the top confuse matters.

Just to round things out, let me add the configs from my presentation; first up, let’s look at a PostgreSQL dev instance on a RHEL 7 beta box:

::::IniLexer
.include /usr/lib/systemd/system/postgresql.service

[Unit]
Description=PostgreSQL dev database server
After=network.target
Requires=var-lib-pgsql.mount

[Service]
# Port number for server to listen on
Environment=PGPORT=5434

# Location of database directory
Environment=PGDATA=/var/lib/pgsql/data-dev

BlockIOWeight=500
OOMScoreAdjust=-500

This supports a dev instance of PostgreSQL that’s co-resident with a bunch of other instances on the same server.

  • The include at the top line is a nice clean way of pulling in the system-wide file. When the system upgrades I don’t have to diff a full service file, just check my overrides are applicable.
  • This file overrides/appends only the lines I care about varying.
  • I’ve added a stanza to check the PG filesystem is mounted, because, well, duh.
  • A seperate port is a standard way to break out instances.
  • The data directory needs to hived off.
  • There’s a couple of instances on the same system, and this isn’t the most important; I bounce the BlockIOWeight down a bunch from the 1000 default so this instance won’t hog the environment.
  • PostgreSQL is like most RDBMSes in that having the OOM killer visit can lead to bad things, so I score it appropriately.

And the last one is the node.js example; much like the tt-rss example above, node.js leaves the details of how to daemonise up to you. For a simple life, I can write a few lines for PID 1 to take care of it:

::::IniLexer
[Service]
ExecStart=/bin/npm start --production /var/www/ghost-tony/
Restart=always
StandardOutput=syslog
SyslogIdentifier=ghost
User=ghost
Group=nobody

[Install]
WantedBy=multi-user.target
  • Web-2.0 MVP ninjas give no fucks about system integration, so making it easy to take their deliverables and deploy them easily is a blessing. Not doing a tedious cut-and-paste of the same boilerplate shell is a win.
  • As previously, Restart makes an appearance, because when our 0.10-release framework/app server falls over, we don’t want to have to restart it.
  • A fully productionised version of this will have PrivateTmp and InaccessibleDirectories for a cheap and easy security buff.
  • Syslogging is a boon, no matter what Lennart says.

So there we are: a few simple systemd examples that show how much cleaner and more functional things can be with a saner PID 1.

Share