11 min to read

How to harden Linux with Lynis and Puppet

Securitea? I prefer chamomile.

Featured image

Lynis is a super handy open-source security auditing tool that provides excellent advice for tightening up the screws from the defaults that come with your favorite Linux distribution.

While Linux distros tend to fare fairly well off the bat for default security controls, there’s still plenty to be desired. This makes automated security auditing tools like Lynis invaluable for improving your security baseline.

Lynis comes in both free and commercial flavours, but for the purposes of this article, we’re going to focus purely on what you can get from the free version, and how you can start applying the tool’s recommendations with Puppet code.

Table of Contents

Our System

First up, this guide assumes you’re running Puppet 7. Both Community and Enterprise edition should be fine, but we’re only going to be testing with the Community edition which is free and open source. Puppet Enterprise has its own way of assigning classes to nodes through the web console, so if you’re on the enterprise version, YMMV.

Everything we’re doing here is running on Rocky Linux 8. This should run fine on your favorite flavor of Linuxy goodness though so long as it supports Puppet Server and Puppet Agent.

This article assumes you’re using the official Puppet Control Repository.

For simplicity’s sake, we’re also going to pick on just the example node and role which comes out of the box with the control repo. It’s laid out roughly like this:

.
├── CODEOWNERS
├── LICENSE
├── Puppetfile
├── README.md
├── data
│   ├── common.yaml
│   └── nodes
│       └── example-node.yaml
├── environment.conf
├── hiera.yaml
├── manifests
│   └── site.pp
├── scripts
│   ├── config_version-r10k.rb
│   ├── config_version-rugged.rb
│   └── config_version.sh
└── site-modules
    ├── profile
    │   └── manifests
    │       └── example.pp
    └── role
        └── manifests
            └── example.pp

We’re going to be using the latest version of Lynis from their GitHub repository. Since Lynis is providing advice based on current industry advice and known threats, it’s always a good idea to run the latest version. If your copy of Lynis is behind, you’re going to be exposing yourself to unnecessary risk.

Lastly, we’re going to be using BASH. There are plenty of other shells available with their own pros and cons, but I guarantee on almost any major distro BASH is either going to be the default shell, or at least installed by default. If you’re not using BASH, that’s fine too, you may need to adjust the scripts a little to work around any BASHisms that are in there.

The Process

1. Clone the Lynis from GitHub

First, open a terminal and ssh to your test system. Once you’re in, run the following to create yourself a checkout directory and pull down Lynis from GitHub. If you don’t have git installed on the server already, you can either install it with yum install git, or use scp to copy it over from another system.

mkdir Checkout
cd Checkout
git clone https://github.com/CISOfy/lynis
cd lynis

inside the Lynis directory you’ll find the application - a shell script called Lynis - along with its supporting database, libraries, etc. If the Lynis shell script isn’t executable by default, make sure you can run by adding execute permissions using chmod.

2. Run your first scan with Lynis

Since we’re baselining the example system, we’re going to run Lynis as root. This means that Lynis will have a complete view of the system and will give you very detailed output in its findings about what to look at tuning. Lynis has an unprivileged mode as well (penetration testing mode), however that’s beyond the scope of what we’re diving into in this article.

To run our scan, we’re first going to use sudo to escalate our permissions. Once we’ve assumed root’s identity, we run Lynis in auditing mode with lynis audit system.

sudo -s
./lynis audit system

We’re using sudo -s instead of sudo -i in this case as we don’t want to spawn a login shell as the target user (in this case root). Instead, all we want to do is elevate our permissions to root while retaining our current working directory and everything in our existing environment.

Lynis will initially complain that we’re running it as root. This is a security feature to reduce the risk that you’re going to run a tainted version of Lynis that someone has fiddled with. After all, a security auditing tool is also an attack tool, just one you’re using to identify flaws to fix rather than to exploit.

image

Since we just cloned Lynis from the official GitHub repo and we’re 100% intending to run it as root, you can safely press enter and ignore this error for now. You could also change ownership of Lynis to root like it suggests, but this isn’t necessary for our purposes.

Once you press enter, Lynis will run for a bit and will produce a lot of output. A lot of output.

The first block gives you a very brief overview of your system, some handy details of things like where Lynis is outputting its logs to (/var/log/lynis.log by default), and some very brief details about the audit itself.

image

After this, there’s a short couple of sections about what system tools and plugins it’s picked up on, followed by brief output from each of the tests as Lynis completes them. The test results are presented as succintly as possible (mostly in pass/fail format). We can safely skip over these as we’re looking for the recommendations in the results section.

The results section contins the meat of the report - what Lynis recommends you do to improve your system’s security. Each recommendation comes with a link to the control in Cisofy’s online documentation, and while not every control is fully doccumented, the recommendation should at least give you enough to work with to figure out what it’s suggesting you do.

image

Finally, at the bottom of the report, Lynis will provide a security hardening index. This score is a representation of how well your system is hardened according to Lynis. While it should not be your only metric for how secure your system is, it can be used to get an idea of how well secured Lynis considers your system to be, with a score of 1 being incredibad, and 100 being almost unusably hardened.

image

Depending on who I’m building a hardened system for I usually aim for no less than 85%, with higher being better. Remember that security is always a game of managed risk, and for every control you introduce there will be a trade-off somewhere in usability.

Now that we’ve got our report and Lynis’ recommendations, we can move on to the fun part with Puppet.

3. Create a hardening profile in Puppet

Now that we’ve got some recommendations from Lynis, we’re going to create our hardening profile. For this article, we’re just going to focus on hardening kernel attributes using sysctl. Tuning kernel attributes is common practice as most distributions begin with fairly permissive defaults.

Navigate to where you’ve cloned your control repository and run this script to create your hardening profile:

cat <<'EOF' >site-modules/profile/manifests/linux_hardening.pp
class profile::linux_hardening (
  Hash $sysctl_keypairs = undef,
) {
  $sysctl_keypairs.each |$key, $value| {
    sysctl { $key:
      ensure => present,
      value  => $value,
    }
  }
}
EOF

This profile expects a flat hash to be passed in where the hash’s keys are the names of the kernel attributes to be modified, and the values are the values to assign to configure the attribute. The loop statement will run through each key/value pair in the hash and create a new sysctl resource for each of them which Puppet will then enforce, creating the resulting entries in /etc/sysctl.conf. This could be made more robust by also taking into account the ensure statement, but for the purpose of this example, we can assume that the attributes that will be configured with sysctl will not be values we want to reconfigure later.

Remember to also add your harding profile to the role assigned to your test server. Your role will end up looking something like this:

class role::example {
  profile::base
  profile::soe
  profile::linux_hardening
  profile::some_service
}

4. Create a hash of attributes to harden in Hiera

With the hardening profile in place, we now need to feed it configuration data. Create the following block in the node data for example-node under data/nodes/example-node.pp.

profile::linux_hardening::sysctl_keypairs:
  fs.protected_fifos: 2
  fs.protected_regular: 2
  fs.suid_dumpable: 0
  kernel.dmesg_restrict: 1
  kernel.kptr_restrict: 2
#  kernel.modules_disabled: 1
#  kernel.perf_event_paranoid: 2
  kernel.sysrq: 0
  kernel.yama.ptrace_scope: 1
  net.core.bpf_jit_harden: 2
  net.ipv4.conf.all.accept_redirects: 0
  net.ipv4.conf.all.log_martians: 1
  net.ipv4.conf.all.send_redirects: 0
  net.ipv4.conf.default.accept_redirects: 0
  net.ipv4.conf.default.accept_source_route: 0
  net.ipv4.conf.default.log_martians: 1
  net.ipv6.conf.all.accept_redirects: 0
  net.ipv6.conf.default.accept_redirects: 0

Two attributes have intentionally been commented out. The first - kernel.modules_disabled - is disabled as setting it to a value of 1 will prevent any further kernel modules being loaded dynamically. This is a setting you want to enable when your system is ready for production, but while working on hardening and configuration it’s better to leave it off until you know you no longer need to play with modules on the fly.

The second attribute - kernel.perf_event_paranoid has been disabled as configuring it can brick your system if not done carefully. Higher levels of paranoia further and further restrict who can profile the system. In some cases, this can lead to blocking legitimate applications from performing their required functions during startup if the system is misconfigured.

The documentation for all of the flags we’re playing with can be found at kernel.org’s page on sysctl.

5. What’s next?

Once you’ve applied your hardening, you can jump back on your test system and run puppet agent with puppet agent -t to apply the new hardening. Once that’s done, rerun lynis and you should see your hardening score go up. After tinkering for a little while, you should be able to hit a score of 80 or higher without too much trouble.

Often times you can find modules on Puppet Forge that will give you control over services Lynis recommends you harden. If not, you can also control a lot of standard configurations in a repeatable manner with augeas, a tree-based configuration editing tool designed to make modifying configuration files more reliable and consistent. It also has its own Puppet module that nicely wraps its functionality so you can use it from your own classes. Finally, if augeas doesn’t work for you, you can configure most services directly using [file][puppet-file] resources.

TLDR

Deploy and run Lynis:

mkdir Checkout
cd Checkout
git clone https://github.com/CISOfy/lynis
cd lynis
sudo -s
./lynis audit system

Assess the recommendations for tuning, pick a recommendation to work on, and develop Puppet code to address it. e.g. here’s just the Puppety goodness for the controls we’re applying in the article above:

# Create a hardening profile which applies a hash of sysctl keypairs
cat <<'EOF' >site-modules/profile/manifests/linux_hardening.pp
class profile::linux_hardening (
  Hash $sysctl_keypairs = undef,
) {
  $sysctl_keypairs.each |$key, $value| {
    sysctl { $key:
      ensure => present,
      value  => $value,
    }
  }
}
EOF

# Create hiera data to populate the keypairs hash
cat <<'EOF' >>data/nodes/example-node.pp
profile::linux_hardening::sysctl_keypairs:
  fs.protected_fifos: 2
  fs.protected_regular: 2
  fs.suid_dumpable: 0
  kernel.dmesg_restrict: 1
  kernel.kptr_restrict: 2
#  kernel.modules_disabled: 1
#  kernel.perf_event_paranoid: 2
  kernel.sysrq: 0
  kernel.yama.ptrace_scope: 1
  net.core.bpf_jit_harden: 2
  net.ipv4.conf.all.accept_redirects: 0
  net.ipv4.conf.all.log_martians: 1
  net.ipv4.conf.all.send_redirects: 0
  net.ipv4.conf.default.accept_redirects: 0
  net.ipv4.conf.default.accept_source_route: 0
  net.ipv4.conf.default.log_martians: 1
  net.ipv6.conf.all.accept_redirects: 0
  net.ipv6.conf.default.accept_redirects: 0
EOF

Don’t forget to apply the profile to a role so it gets applied during a puppet run.

Rinse-repeat until your hardening index is at least above 80, and you’ve satisfied all relevant security controls for your environment.