Author Archives: Andrew Wells

About Andrew Wells

I have been developing on the LAMP stack since about 2006. I run Ubuntu XFCE on my desktop and have a history of managing Ubuntu and CentOS servers. I code web applications mostly in PHP but have experience with other languages as well. When I'm not working, I can be found working in my home lab or out snowboarding, hiking, camping, or biking depending on the season.

Configure Minecraft Server with Systemd and Backups on Ubuntu 22.04

Base OS

Install Ubuntu 22.04 Server. At setup, create your preferred user for managing the server. We will create a user specific to the Minecraft service later.

Update and install some software.

[root]$ export NEEDRESTART_MODE=a
[root]$ apt update && apt -y dist-upgrade
[root]$ apt -y install git htop openjdk-17-jre-headless screen rdiff-backup

Minecraft Server Setup

Add a user.

[root]$ useradd -m -s /bin/bash bukkit

Create some directories.

[bukkit]$ mkdir -p /home/bukkit/{minecraft,mcbackup,scripts}

Download the server jar.

[bukkit]$ cd /home/bukkit/minecraft
[bukkit]$ wget https://download.getbukkit.org/craftbukkit/craftbukkit-1.20.4.jar
[bukkit]$ ln -s craftbukkit-1.20.4.jar minecraft_server.jar

Create the eula file. This is necessary to run for the first time.

[bukkit]$ echo "eula=true" > /home/bukkit/minecraft/eula.txt

Test run? You can do that by running this:

[bukkit]$ java -Xmx1024M -Xms1024M -jar /home/bukkit/minecraft/minecraft_server.jar nogui

Once the world generates, feel free to join from the client. This is a good checkpoint before moving into the service/backup configurations.

Once done, type “stop” into the running server to stop it.

Systemd Service

The service is pretty easy, just basically creating a few files. You don’t have to use vim; use your editor of choice and paste in the contents for each file.

This file allows you to start/stop/monitor the service and logs.

[root]$ vim /etc/systemd/system/minecraft.service

File contents:

[Unit]
Description=Minecraft Server

[Service]
Type=simple
WorkingDirectory=/home/bukkit/minecraft
ExecStart=java -Xmx1024M -Xms1024M -jar /home/bukkit/minecraft/minecraft_server.jar nogui
ExecStop=/home/bukkit/scripts/stop.sh
KillSignal=SIGINT
SendSIGKILL=no
TimeoutStopSec=300
User=bukkit
Restart=on-failure
Sockets=minecraft.socket
StandardInput=socket
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target

The stop command needs to be in its own script for some reason.

[bukkit]$ vim /home/bukkit/scripts/stop.sh

#!/usr/bin/bash

echo "stop" > /run/minecraft.stdin
sleep 45

[bukkit]$ chmod +x /home/bukkit/scripts/stop.sh

Adding a socket that will allow sending commands to the running service. Finding how to do this is what finally got me away from using “screen” to hold the running process.

[root]$ vim /etc/systemd/system/minecraft.socket

File contents:

[Unit]
PartOf=minecraft.service

[Socket]
ListenFIFO=%t/minecraft.stdin

Allow the bukkit user to manage the service

The following will walk you through creating a sudoers file that will allow the bukkit user to run service commands, but only for the minecraft service.

[root]$ vim /etc/sudoers.d/bukkit

Add this:

bukkit ALL=(ALL) NOPASSWD:/usr/bin/systemctl start minecraft
bukkit ALL=(ALL) NOPASSWD:/usr/bin/systemctl stop minecraft
bukkit ALL=(ALL) NOPASSWD:/usr/bin/systemctl restart minecraft
bukkit ALL=(ALL) NOPASSWD:/usr/bin/systemctl status minecraft
bukkit ALL=(ALL) NOPASSWD:/usr/bin/journalctl -u minecraft -f

Fix the permissions.

[root]$ chmod 0440 /etc/sudoers.d/bukkit

Command Reference

Starts the service if it wasn’t running.

[bukkit]$ sudo systemctl start minecraft

Stops the service.

[bukkit]$ sudo systemctl stop minecraft

Restarts the service.

[bukkit]$ sudo systemctl restart minecraft

Get the status of the service.

[bukkit]$ sudo systemctl status minecraft

Watch the log output.

[bukkit]$ sudo journalctl -u minecraft -f

Send a command to the service.

[bukkit]$ echo "say hellooooooo" > /run/minecraft.stdin

Most of the Systemd configuration was found in this StackExchange Answer.

Local Backup

I backup the VM routinely, however, I backup the world files hourly. This can be done locally.

Create this file.

[bukkit]$ vim /home/bukkit/scripts/mcbackup.sh

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

#!/bin/bash
 
# remove old backups #
rdiff-backup --remove-older-than 7D --force /home/bukkit/mcbackup/
 
# save any chunks in memory #
echo "save-all" > /run/minecraft.stdin
sleep 10
 
# disable level saving while the backup runs #
echo "say ** Backup Yo Stuffz! May lag a sec during backup. **" > /run/minecraft.stdin
echo "save-off" > /run/minecraft.stdin
 
# run the local backup #
rdiff-backup --exclude /home/bukkit/minecraft/plugins/dynmap/web /home/bukkit/minecraft /home/bukkit/mcbackup
 
# enable disk writes again #
echo "save-on" > /run/minecraft.stdin
echo "say ** Done **" > /run/minecraft.stdin
[bukkit]$ chmod +x /home/bukkit/scripts/mcbackup.sh

Consider adding the script to bukkit’s crontab.

Test a Restore

Run the script manually a few times or let the cron go a few times. Then run the below rdiff-backup command (just the first line) and see the output.

[bukkit]$ rdiff-backup --list-increments ~/mcbackup
Found 3 increments:
    increments.2024-01-13T03:29:55Z.dir   Sat Jan 13 03:29:55 2024
    increments.2024-01-13T03:30:46Z.dir   Sat Jan 13 03:30:46 2024
    increments.2024-01-13T03:32:21Z.dir   Sat Jan 13 03:32:21 2024
Current mirror: Sat Jan 13 03:32:33 2024

The below command will restore the most recent backup available (Sat Jan 13 03:32:33 2024) based on the above output. Useful if you just died or something 🙂

[bukkit]$ sudo systemctl stop minecraft
[bukkit]$ rdiff-backup --force -r now /home/bukkit/mcbackup /home/bukkit/minecraft
[bukkit]$ sudo systemctl start minecraft

Refer to rdiff-backup documentation for more restore options.

Nagios Plugins 2.2.1 with SSH Remote Exec for Clients

Configure the Server

I assume you already configured the server using this guide: Nagios 4.4.1 Install from Source on Ubuntu 18.04

Log in to your Nagios server and su into the nagios user. You might have to su from root since that account usually doesn’t have a password.

[root@server]$ su - nagios

Out of curiosity, see if you have a .ssh folder.

[nagios@server]$ ls -alh ~/.ssh
ls: cannot access '/home/nagios/.ssh': No such file or directory

More than likely you don’t. So that also means that there is no SSH key pair. Let’s create one.

[nagios@server]$ ssh-keygen

Basically mash the enter key until you see the fingerprint art. We don’t want a passphrase on this key because it will be used by automated scripts.

Now output the public key and save locally for later.

[nagios@server]$ cat .ssh/id_rsa.pub

Configure the Client(s)

Install Option #1 – Install From Source

[root@client]$ useradd -m -s /bin/bash nagios

[root@client]$ cd ~
[root@client]$ wget http://www.nagios-plugins.org/download/nagios-plugins-2.2.1.tar.gz
[root@client]$ tar -xzvf nagios-plugins-2.2.1.tar.gz

By the way, this tar command was written without referencing the docs.

Install some required packages.

[root@client]$ apt-get install libgd-dev php build-essential unzip libnet-snmp-perl libperl5.26 libpq5 libsensors4 libsnmp-base libsnmp30 libtalloc2 libtdb1 libwbclient0 samba-common samba-common-bin smbclient snmp whois libmysqlclient-dev libssl-dev

Now install them

[root@client]$ cd ~/nagios-plugins-2.2.1
[root@client]$ ./configure --with-nagios-user=nagios --with-nagios-group=nagios
[root@client]$ make
[root@client]$ make install

Option #2 – Apt Install

[root@client]$ useradd -m -s /bin/bash nagios

[root@client]$ apt-get install nagios-plugins

I reference all plugin paths from the source install, so create a link if you do the deb package.

[root@client]$ ln -s /usr/lib/nagios/plugins /usr/local/nagios/libexec

Additional Plugin Install

[root@client]$ cd /usr/local/nagios/libexec
[root@client]$ wget https://raw.githubusercontent.com/justintime/nagios-plugins/master/check_mem/check_mem.pl
[root@client]$ chmod +x check_mem.pl

Authorize Connections

[nagios@client]$ mkdir ~/.ssh
[nagios@client]$ vim ~/.ssh/authorized_keys

Paste in the SSH public key created earlier on the server.

Fix some file permissions.

[nagios@client]$ chmod 0600 ~/.ssh/authorized_keys

Test Commands

Run this lengthy command on the server:

[nagios@server]$ /usr/local/nagios/libexec/check_by_ssh -H 10.0.36.7 -C "/usr/local/nagios/libexec/check_load -w 2,1,1 -c 8,4,4"
OK - load average: 0.14, 0.10, 0.09|load1=0.140;2.000;8.000;0; load5=0.100;1.000;4.000;0; load15=0.090;1.000;4.000;0;

Note that the load average returned is from the client, not the server.

Example Configuration

Below is an example config to start with. You can have a generic command that will call most any remote command. In the service definition, pass the remote command in the first parameter and the warning/critical thresholds in the following parameters. The fourth parameter, $ARG4$ can be used to pass any additional parameters if needed.

define command {
        command_name    check_by_ssh
        command_line    /usr/local/nagios/libexec/check_by_ssh -t 30 -H '$HOSTADDRESS$' -C "/usr/local/nagios/libexec/$ARG1$ -w '$ARG2$' -c '$ARG3$' $ARG4$"
}

define hostgroup {
    hostgroup_name     hg-linux
    alias              Linux Servers
}

define service {
    use                     generic-service
    hostgroup_name      hg-linux
    service_description CPU Load
    check_command       check_by_ssh!check_load!4,2,2!8,4,4
    check_interval      1
    retry_interval      1
}

define host {
    use                     host-generic
    host_name               server-01
    alias                   server-01
    address                 192.168.0.10
    hostgroups              hg-linux
}

Sanitize Docker Environment

I have a few docker hosts used for development, and sometimes I need to clean out all images/containers/network groups and start from scratch. These commands do just that. Obviously don’t run these on any machines w/ persistent data. Use with caution!

Linux Hosts (no Docker GUI)

Relatively safe cleanup – these commands remove dangling images/volumes that are likely no longer used and just taking up space.

#[root]$ docker system prune -f
#[root]$ docker rmi $(docker images -f "dangling=true" -q)
#[root]$ docker volume rm $(docker volume ls -q -f dangling=true)

More destructive, 

#[root]$ docker kill $(docker ps -a -q)
#[root]$ docker rm $(docker ps -a -q)
#[root]$ docker rmi $(docker images -q) -f

MacOS Hosts (Docker GUI)

I’ve had times where the Docker image completely fills up, and cleaning up images to not free up space. So it’s just easier to delete the Virtualbox image and restart Docker.

Delete the Docker VM disk file:

[user]$ rm -f ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2

Restart Docker using the Docker GUI.

Gitlab EE/CE with Runners and Docker Registry on Ubuntu 16.04

Where do you store your code projects – Github? What about private projects…do you pay for private repos? You can have more control over your projects and even configure automated tests and builds – all using open source software! Fire up your spare hypervisor and read on to learn how to build your own dev project host!

Continue reading →

Nagios 4.x Install from Source on Ubuntu 14.04

I am just starting to upgrade / install Ubuntu 14.04 on my servers, so it’s time to create a guide to install Nagios 4 on Ubuntu 14.04. Apparently Nagios Core install scripts wern’t well tested for Debian systems. But in any case, it can be installed with a couple modifications.

This guide is going to be very similar to my other Nagios install guides.

Continue reading →

Offline Migration of KVM Guests

It’s time to enter into summer mode with my lab. By that I mean that it’s time to move important KVM guests off of the HP G5 server and shut that space heater down. Here are my notes with moving guests. If you are looking to move guests from a CentOS to Ubuntu hypervisor, I encourage you to read over my findings. You could also reverse-engineer the article if you are going from Ubuntu to CentOS as well.

Continue reading →

iftop

Format Correctly in Windows Putty Session

[root]$ NCURSES_NO_UTF8_ACS=1 iftop -i eth0

Config File Examples

Edit the config file on Ubuntu (and probably Debian) boxes:

[user]$ vim ~/.config/htop/htoprc

Edit the file on CentOS (and probably RHEL) boxes:

[user]$ vim ~/.htoprc

Config for 8-Core Boxes

# Beware! This file is rewritten by htop when settings are changed in the interface.
# The parser is also very primitive, and not human-friendly.
fields=0 48 17 18 38 39 40 2 46 47 49 1
sort_key=46
sort_direction=1
hide_threads=0
hide_kernel_threads=1
hide_userland_threads=0
shadow_other_users=0
show_thread_names=0
highlight_base_name=0
highlight_megabytes=1
highlight_threads=0
tree_view=0
header_margin=1
detailed_cpu_time=0
cpu_count_from_zero=0
update_process_names=0
color_scheme=0
delay=15
left_meters=CPU AllCPUs2 Memory Swap
left_meter_modes=1 1 1 1
right_meters=Hostname Tasks LoadAverage Uptime Clock Battery
right_meter_modes=2 2 2 2 2 2