Debian BOINC Client

Contents

Server Installation

Physical Device

Install Debian 9 (Stretch) 64bit or newer onto the device using the Minimal setup option plus the SSH server option during the final steps on the installation. BOINC does not require a lot of disk space requiring custom partitioning, the default installation options should work in most cases.

Cloud Server

Spin up a basic Debian 9 (Stretch) 64bit or newer cloud instance at the provider of your choice; note that a BOINC client is CPU and RAM intensive, so be careful which provider you choose. This guide is written for a 1 CPU, 512M to 1G RAM, 20G disk cloud server with 1TB transfer/month which is billed at a flat monthly rate by the provider; tuning is provided to avoid noisy-neighbor behavior and keep resource usage within reasonable limits.

Be Careful of Costs - Cloud providers typically charge for time spent running (uptime) plus bandwidth charges. Research costs carefully and ensure the client is configured to meet your budget. The below tuning is only a guide and may need tweaked to specific circumstances.

The below instructions have been tested on various standard Debian instances supplied by cloud providers. The actual CPU resource limits vary depending on type of provider/instance, however - be sure to pay attention to the CPU usage tuning details.

User Setup

Use a very secure password - at a minimum use pwgen -sB 15, strong password security encouraged!

Set up a non-root user and add to the sudo group, then add a password. Use this user for SSH access and become root once logged in with sudo; if you have used SSH keys to log in as root, copy to this new user's setup as well if needed:

apt-get update
apt-get install sudo

export MYUSER="frankthetank"
useradd -m -d /home/${MYUSER} -s /bin/bash -g users -G sudo ${MYUSER}
passwd ${MYUSER}

If you are unable to use ssh-copy-id from your workstation to add a new SSH key, perform the work manually:

mkdir /home/${MYUSER}/.ssh
cp /root/.ssh/authorized_keys /home/${MYUSER}/.ssh/
chmod 0700 /home/${MYUSER}/.ssh
chmod 0600 /home/${MYUSER}/.ssh/authorized_keys
chown -R ${MYUSER}:users /home/${MYUSER}/.ssh

SSH in as this user and test sudo several times - log out completely between tests

Disable root Login

If the above is successful and you are capable of gaining full root privileges via the non-root SSH session using sudo, now disable root logins in SSH from the outside world for an additional security layer. The root account still remains usable, just not via direct SSH access.

The task is to set PermitRootLogin no - the setting varies from one provider to another, sometimes it's already set (either yes or no), sometimes it's commented out. This small scriptlet should handle these 2 most common cases, be careful and investigate for yourself:

_SCFG="/etc/ssh/sshd_config"
if $(grep -iEq '^PermitRootLogin[[:space:]]+yes' "${_SCFG}"); then
  sed -i.bak -e 's/^PermitRootLogin.*/PermitRootLogin no/gi' "${_SCFG}"
else
  sed -i.bak -e 's/^#PermitRootLogin.*/PermitRootLogin no/gi' "${_SCFG}"
fi

After confirming the change is correct, restart the SSH core daemon (it will not log you out):

systemctl restart sshd

Test logging in again to ensure the changes are as expected. Do not log out of the active, working SSH session as root until you've confirmed in another session you can log in as your non-root user and still gain sudo to root.

Server Hardening

1. The Debian default vimrc (set mouse=a, /usr/share/vim/vim80/defaults.vim) messes up middle-mouse click paste when remote via SSH, override the setting to just disable the mouse:

echo 'set mouse=' >> /root/.vimrc

2. Install a few basic packages to make life a little nicer; typically the minimal install / cloud instances are stripped down and need a few things added, both for security and ease of use. Adjust as needed, at a minimum ensure the below are in place:

apt-get update
echo "iptables-persistent iptables-persistent/autosave_v4 boolean true" | debconf-set-selections
echo "iptables-persistent iptables-persistent/autosave_v6 boolean true" | debconf-set-selections
echo "unattended-upgrades unattended-upgrades/enable_auto_updates boolean true" | debconf-set-selections
apt-get install sysstat unattended-upgrades iptables-persistent man less vim

The smem package will pull in a lot of X dependencies due to an embedded recommendation, install it while disabling that feature. This utility can be used to quickly query memory usage (including swap) on the memory constrained cloud server:

apt-get install smem --no-install-recommends

3. Enable journald to store logs on disk instead of just RAM. By default, the journald system is in automatic mode - on boot it will create the ephemeral tmpfs /run out of RAM, but will only transition to storing the journal on disk (out of RAM) if this directory exists.

mkdir /var/log/journal

4. Enable sysstat for ongoing statistics capture of your instance (use sar to view):

sed -i.bak -e 's|^ENABLED=".*"|ENABLED="true"|g' /etc/default/sysstat

5. Enable unattended-upgrades to ensure that all Security updates are applied:

cat << 'EOF' > /etc/apt/apt.conf.d/02periodic
APT::Periodic::Enable "1";
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Download-Upgradeable-Packages "1";
APT::Periodic::AutocleanInterval "5";
APT::Periodic::Unattended-Upgrade "1";
EOF

6. Enable the basic iptables rules to allow only port 22:

cat << 'EOF' > /etc/iptables/rules.v4
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
COMMIT
EOF

cat << 'EOF' > /etc/iptables/rules.v6
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p ipv6-icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A INPUT -j REJECT --reject-with icmp6-adm-prohibited
-A FORWARD -j REJECT --reject-with icmp6-adm-prohibited
COMMIT
EOF

7. If using a cloud server, add a bit of swap if needed - using swap is not bad in and of itself, the Linux kernel will attempt to cache it's small bits of data if available. Using swap in place of real RAM is bad, however - the tuning below will avoid actual application swapping.

# 512M file, probably overkill
dd if=/dev/zero of=/swap.file bs=4096 count=128000
chmod 0600 /swap.file
mkswap /swap.file
echo '/swap.file none swap defaults 0 0' >> /etc/fstab
swapon /swap.file

8. Finally, ensure all the services are enabled and apply all outstanding updates; reboot as needed for a new kernel. If you don't reboot here, you'll need to service foo restart each one individually (just reboot, it's easier):

systemctl disable remote-fs.target
systemctl enable sysstat unattended-upgrades netfilter-persistent

apt-get dist-upgrade -y

reboot

fail2ban Setup

Optional: configure fail2ban to keep an eye on the SSH port for brute force attacks.

Note: fail2ban tends to consume a fair amount of memory the longer it runs; if the cloud server is memory constrained, you may wish to skip this step or disable the service later. Use smem to monitor it periodically.

apt-get install fail2ban sqlite3

cat << 'EOF' > /etc/fail2ban/jail.local
[DEFAULT]
ignoreip  = 127.0.0.1/8
bantime   = 600
maxretry  = 3
backend   = auto
destemail = root@localhost

[ssh]
enabled  = true
port     = ssh
filter   = sshd
logpath  = /var/log/auth.log
maxretry = 6
EOF

systemctl enable --now fail2ban

Additionally, add a weekly cron task to purge the database of old IPs (bug in 0.9.x series) and to restart the daemon to free up it's RAM usage:

cat << 'EOF' > /etc/fail2ban/dbpurge.sql
delete from bans where timeofban <= strftime('%s', date('now', '-7 days'));
vacuum;
.quit
EOF

cat << 'EOF' > /etc/cron.weekly/f2b-cleanup
#!/bin/sh
if [ -x /usr/bin/sqlite3 ]; then
  sqlite3 /var/lib/fail2ban/fail2ban.sqlite3 < /etc/fail2ban/dbpurge.sql
fi
systemctl restart fail2ban.service
EOF

chown root:root /etc/cron.weekly/f2b-cleanup
chmod 0755 /etc/cron.weekly/f2b-cleanup

BOINC Client Setup

Note: Debian 9 uses /var/lib/boinc-client and Debian 10 uses /var/lib/boinc as the default data directory. If you installed using 9 and dist-upgrade to 10, the upgrade process will create a symlink connecting the old name to the new name.

The BOINC client is basically a wrapper application; it will "phone home" periodically and download new work units from the upstream project(s) and manage running those work units, then submit the results of the work to the upstream projects.

1. Install the boinc-client and boinctui software; the client will automatically start and enable as part of the package installation process. As of this writing errors appeared in the logfile about missing directories which are being added:

apt-get install boinc-client boinctui

# Debian 9
mkdir /var/lib/boinc-client/{slots,locale}
chown boinc:boinc /var/lib/boinc-client/{slots,locale}

# Debian 10
mkdir /var/lib/boinc-client/{slots,locale}
chown boinc:boinc /var/lib/boinc-client/{slots,locale}

systemctl restart boinc-client

3. Enable the client to always run, as this is a dedicated researching instance:

boinccmd --set_run_mode always
boinccmd --set_network_mode always

4. Cloud server: tune the client settings to fit within the resources, and keep it from being a noisy-neighbor; as this is a dedicated instance, most RAM and disk will be allocated, but the CPU thresholds are reduced. As an example of two very different platforms, if you're using a Google Cloud f1-micro VM which has only 0.2 (20%) of a vCPU, a good throttle is 19%. But if you're using a full vCPU instance, 49% is an acceptable limit for most providers to avoid being a noisy neighbor. You will need to research the exact right throttle for your specific circumstances.

1 TB/month is roughly 400 KB/s sustained bandwidth

4.1 First, use systemd to control the CPU throttle, not the settings in the BOINC client. The BOINC client uses CPU idle detection and frequently pauses the process as it detects other things happening, which is undesirable in this specific setup. Instead, we will use systemd to create a cgroup (resources control group) to confine the BOINC process - and it's children, the work units - to a smaller percentage of the real CPU.

The CPUQuota option was added in systemd version 213 and requires CONFIG_CFS_BANDWIDTH=y to be configured in the active kernel. Debian 9 meets both of the requirements.

Use the built-in edit capability of systemctl to create the unit override setting(s), it will create /etc/systemd/system/boinc-client.service.d/override.conf and place you into edit mode:

EDITOR=vi systemctl edit boinc-client

# Add the below to the new file being edited and save
[Service]
CPUQuota=49%

Inform systemd of the new content just created, then restart the BOINC client to activate:

systemctl daemon-reload
systemctl restart boinc-client

4.2 Next, set up the BOINC client to use 100% of what it thinks is the whole CPU and to only pause when it thinks other processes are using 100%, such that it basically never stops running at all while confined to it's custom cgroup. Remember that on the outside, it's only actually using 49% of the CPU in this throttled configuration.

Notice that we still set the real disk space and RAM thresholds, we are only telling it to use 100% of CPU as that's the only item we throttled from the outside. If you cannot use the above systemd throttle, you must set the CPU settings below to their actual lower values desired!

# Debian 9
vi /var/lib/boinc-client/global_prefs_override.xml

# Debian 10
vi /var/lib/boinc/global_prefs_override.xml

<global_preferences>
  <suspend_cpu_usage>100.000000</suspend_cpu_usage>
  <cpu_usage_limit>100.000000</cpu_usage_limit>
  <disk_max_used_pct>75.000000</disk_max_used_pct>
  <ram_max_used_busy_pct>70.000000</ram_max_used_busy_pct>
  <ram_max_used_idle_pct>90.000000</ram_max_used_idle_pct>
  <vm_max_used_pct>50.000000</vm_max_used_pct>
  <leave_apps_in_memory>1</leave_apps_in_memory>
  <run_if_user_active>1</run_if_user_active>
  <run_on_batteries>1</run_on_batteries>
  <run_gpu_if_user_active>1</run_gpu_if_user_active>
  <daily_xfer_limit_mb>8000.000000</daily_xfer_limit_mb>
  <daily_xfer_period_days>30</daily_xfer_period_days>
</global_preferences>

boinccmd --read_global_prefs_override

Note: the above step 4 instructions can also be followed on a Physical device to limit the CPU use, this is not unique to a cloud server. Throttling a physical device to 95% of the CPU power leaves a little room for other things (nightly updates, fail2ban, ssh sessions, etc.) to breathe a little easier.

BOINC Project Setup

The easiest way to manage BOINC projects is to use an online Account Manager; much like the BOINC client manages downloading and running work units, the Account Manager handles which projects you wish to join, as well as helping connect all those projects together easily in one view. One of the most popular Account Managers is BOINC Account Manager (BAM!); it's already integrated to the boinctui and boinc-client software and has a handy setup guide:

Follow the above set of steps, then when reaching step 8 "Attach the BOINC client to BAM!" return here and follow these steps. There are multiple ways to do this, using boinctui simply makes everything a lot easier.

Tip: When making your accounts, use the same username / password combination for the various projects that you use with BAM! setup. They all interconnect with each other, using the same credentials leads to a better experience and is easier to manage. However, be sure to use a unique password for this setup, do not use an already known password!

1. As your non-privileged user (not root!) start the application: boinctui. The very first time it starts a TUI dialog will appear asking where to connect for the BOINC client. Accept the defaults, 127.0.0.1 and no password.

Note: a password can be added by root to /etc/boinc-client/gui_rpc_auth.cfg, then it's used for the above first-time connection dialog. This will add one more layer of security and is recommended; do not use the BAM! password, just generate something random. I recommend pwgen -sB 15 at a minimum.

2. Within the boinctui TUI now on screen, press F9 to bring up the menu along the top. Arrow over to the Projects menu item, then choose the option to start using an account manager and press Enter.

3. From the list of account managersi now shown, select the one near the top named BAM! and a new dialog will appear asking for your login and password. This is the same login created on https://boincstats.com/en/bam/ above.

At this point a few actions will be happening upstream, be patient - in general, the following is happening when you first join a new server to your BAM! account:

After just a bit of time (it varies per project; as a rule of thumb, wait at least 5 minutes) perform two more first-time only actions to get everything aligned:

The above two steps can be skipped and they will eventually happen on their own, they are normal regular tasks performed by the BOINC client wrapper software. Performing them manually the first time just helps make sure everything is working as expected. After a day or so (some projects only update statistics once a day) you will now be able to track the performance of your server in both BAM! and the specific projects themselves.

Final Checks

Note that it may take a bit of time (not long) for tasks to be assigned to your new cloud client for processing; you should be able to get an immediate confirmation the client is connected and working with a general overview:

boinccmd --get_state

Once the Projects attach and start sending tasks, use top and you should see the active binaries crunching data using the CPU and RAM, and can be observed (see the man page for all options):

boinccmd --get_simple_gui_info
boinccmd --get_project_status
boinccmd --get_tasks

The boinctui application is a very nice alternative, it presents a nice windowed interface which can fully control the client options with menus, and presents all the information cleanly. Pretty much anything boinccmd can do, boinctui can do as well.

boinctui

See the above section on configuring boinctui it's first time run.

References