Installing RHEL 8 on Hetzner root servers

Hetzner is a very popular provider for so called root servers and VPS (Virtual private Servers) located in Germany with datacenters in Germany and Finnland. They are quite affordable and have good services as well.

The default installation image, sorry Hetzner, is crap (i.e. no logical volumes). The rescue system is not only a nice tool to recover from botched system configurations, it also comes with an image installer which allows users to install a custom system. The default selection is Arch, CentOS 7.6, some Debian and Ubuntu versions but no CentOS8 or even RHEL of any version.

Game over? No, you can create an own image, just put it on a web server and configure the installation accordingly.

Create your image

I just installed a VM from the ISO image that can be downloaded here.

Select the “Minimal” installation, register the machine at Red Hat with subscription-manager register, attach the RHEL OS with subscription-manager attach –auto and install tar and mdadm.

Caution! You need to put the image on a public available web space! Unregister your system and delete the SSH host keys!

subscription-manager unregister
subscription-manager clean
rm -rf /etc/ssh/ssh_host_*

An other important fact is that the Hetzner installer only knows about CentOS and only about version 7. On CentOS8 and RHEL8, the dracut binary is located elsewhere than on RHEL7, you need to create a Symlink.

[root@localhost /]# ln -s /usr/bin/dracut /sbin/dracut

For the same reason you need to name the tarball CentOS-<something>

The next step is to create the tarball. Important to know is that the tarball must not contain /dev, /proc and /sys folders.

[root@localhost /]# tar cJvf CentOS-80-el-x86_64-minimal.tar.xz --exclude=/dev/* --exclude=/proc/* --exclude=/sys/* --exclude=/CentOS-80-el-x86_64-minimal.tar.xz /

Copy the resulting tarball to a public web space:

[root@localhost /]# scp CentOS-80-el-x86_64-minimal.tar.xz blog.delouw.ch:/var/www/html/blog.delouw.ch/pub

Yes, you can use my image if you like: http://blog.delouw.ch/pub/CentOS-80-el-x86_64-minimal.tar.xz

Reboot your server into the rescue mode

Point your browser to https://robot.your-server.de/server, select your server, click on “Rescue” on the server menue, select “64 bit” and click on “Activate Rescue System”. The root password will be shown which will be needed to log in to the system.

Configure the installer

Configuring the installer is straight forward. Create a file called config.txt with the following content:

DRIVE1 /dev/sda
DRIVE2 /dev/sdb
SWRAID 1
SWRAIDLEVEL 1
BOOTLOADER grub
HOSTNAME localhost.localdomain
PART /boot ext2     1024M
PART lvm   rhel       all

LV rhel   root   /       xfs     20G
LV rhel   swap   swap    swap     16G
LV rhel   tmp    /tmp    xfs      1G
LV rhel   home   /home   xfs      4G
LV rhel   var   /var   xfs      4G
LV rhel   var_log   /var/log   xfs      4G
LV rhel   var_log_audit   /var/log/audit   xfs      4G

IMAGE http://blog.delouw.ch/pub/CentOS-80-el-x86_64-minimal.tar.xz

The partitioning is according to PCI-DSS. Note that the installer can not set the mount options needed by PCI-DSS (and a lot of other regularities).

Side note: The Hetzner installer refuses to install the image if /boot is of type xfs. You need to choose ext2,3 or ext4. I’m using ext2 as this is the most efficient file system, fsck on unclean shutdown for 1GByte takes just a few seconds, it does not matter.

Run the installer

Just run it and expect an error….

installimage -a -c config.txt

Hetzner Installer

The installer tries to install some packages which will, of course, fail as the system is not registered to RHSM yet.

[14:26:11] # Running some centos specific functions
[14:26:11] # chroot: chkconfig iptables off
[14:26:11] :   error reading information on service iptables: No such file or directory
[14:26:11] # chroot: chkconfig ip6tables off
[14:26:11] :   error reading information on service ip6tables: No such file or directory
[14:26:11] # chroot: chkconfig postfix off
[14:26:11] :   error reading information on service postfix: No such file or directory
[14:26:11] # Testing and setup of cpanel image
[14:26:11] # chroot: yum check-update
[14:26:12] :   Updating Subscription Management repositories.
[14:26:12] :   Unable to read consumer identity
[14:26:12] :   This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
[14:26:12] :   Error: There are no enabled repos.
[14:26:12] # chroot: yum -y install polkit
[14:26:12] :   Updating Subscription Management repositories.
[14:26:12] :   Unable to read consumer identity
[14:26:12] :   This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
[14:26:13] :   Error: There are no enabled repos.
[14:26:13] => FAILED
[14:26:13] :   report install.conf to rz-admin: 1281870
[14:26:13] :   report debug.txt to rz-admin: ok
[14:26:13] cleaning up

You can just ignore that error messages, it does not matter.

Finalizing

Reboot the system and log in with the root password provided for the rescue system when activating the rescue system at https://robot.your-server.de/server.

Change the password (needless to say?)

Continue to configure the system according your needs.

Have fun 🙂

Migrating from CentOS8 to RHEL8

There are various reasons why to migrate from CentOS to RHEL. Quicker access to bugfixes and new minor releases as well as having a fully commercially supported system. Unfortunately most providers do not have an option to install RHEL but CentOS instead.

There are different tutorial on the net how to migrate from RHEL to CentOS but almost no information about the other way round. It is quite simple and at the end of the day you have only Red Hat Packages installed.

In 2012 I wrote an article about Migrating from CentOS6 to RHEL6. Subsequently I’ve posted a similar article for RHEL 7: Migrating from CentOS7 to RHEL7. Now its time for an update.

Disclaimer

Some of the procedures can be destructive for your system and/or your data. I’m not taking any responsibility for any damage casue. Take a full backup of your system before even thinking about trying this procedure!

Also an important to note is that such a procedure is not supported by Redhat.

Requirements

There are only two things you need

  • A valid RHEL subscription obtained from Redhats online store
  • A RHEL8 ISO-Image which corresponds with your current CentOS minor release (or newer) which can be downloaded at Redhat downloads

Preparations

Be sure you activated your subscription.

Mount the ISO image on your CentOS8 machine:

[root@centos8 ~]# mount /dev/cdrom /mnt -o loop

Go to /mnt/BaseOS/Packages and install the packages we need:

[root@centos8 Packages]# rpm -e subscription-manager-rhsm-certificates --nodeps
[root@centos8 Packages]# yum localinstall subscription-manager-1*
[root@centos8 Packages]# yum localinstall subscription-manager-rhsm-certificates-1*

(Re)Move your CentOS repos
To avoid conflicts between CentOS and Redhat Repositories you need to get rid of them. Remove them or just keep a copy.

[root@centos8 Packages]# mkdir /etc/yum.repos.d.centos
[root@centos8 Packages]# mv /etc/yum.repos.d/CentOS-* /etc/yum.repos.d.centos

Force-remove the centos-release and yum RPMs

[root@centos8 Packages]# rpm -e centos-release --nodeps
[root@centos8 Packages]# yum localinstall redhat-release-8*

Unmount the ISO
This is important, otherwise yum reinstall will fail!

[root@centos8 ~]# umount /mnt

Register your system

To get access to RHEL repositories, you need to register your system. The username “user@example.com” must be replaced with your username. The ID is a randomly generated UUID.

[root@centos8 ~]# subscription-manager register
Registering to: subscription.rhsm.redhat.com:443/subscription
Username: user@example.com
Password: 
The system has been registered with ID: 8aa7ef30-e834-4889-9e07-c5e8ce31e681 
The registered system name is: centos8.example.com
[root@centos8 ~]# 

Attach the system to a subscription

Usually it is just good enough to auto-attach the subscription needed.

[root@centos8 ~]# subscription-manager attach --auto
Installed Product Current Status:
Product Name: Red Hat Enterprise Linux for x86_64
Status:       Subscribed

[root@centos8 ~]#

Review enabled repositories

Sometimes you dont want to use all the repos provided. The simplest way is just to disable all and re-enable those you need.

[root@centos8 ~]# yum repolist
Updating Subscription Management repositories.
Red Hat Enterprise Linux 8 for x86_64 - AppStream (RPMs)                                                 3.0 MB/s | 9.3 MB     00:03    
Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs)                                                    2.9 MB/s | 8.1 MB     00:02    
repo id                                              repo name                                                                     status
rhel-8-for-x86_64-appstream-rpms                     Red Hat Enterprise Linux 8 for x86_64 - AppStream (RPMs)                      6,031
rhel-8-for-x86_64-baseos-rpms                        Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs)                         2,193
[root@centos8 ~]# 

Adjust with subscription manager repos –disable|enable as needed.

Changing the Distribution

Now we have all requirements met, lets reinstall the packages.

[root@centos8 ~]# yum -y reinstall "*" 

Cleanup

Now we need to manually clean up the CentOS specific packages that have not been reinstalled.

[root@centos8 ~]# rpm -qa --queryformat "%{NAME} %{VENDOR}\n" | grep -i centos | cut -d' ' -f1
containers-common
centos-logos
glibc
libfprint
containernetworking-plugins
python3-dnf-plugin-spacewalk
platform-python
glibc-common
python3-rhnlib
slirp4netns
gpgme
rhn-client-tools
dnf-plugin-spacewalk
podman
python3-libs
qemu-guest-agent
glibc-langpack-en
container-selinux
python3-gpg
runc
buildah
fuse-overlayfs
python3-rhn-client-tools
oci-systemd-hook
[root@centos8 ~]# 

Most of this packages are not problematic to remove and reinstall. Unfortunately this is not the case with the glibc packages. Do not try to remove them with –nodeps, you system will stop working. The problem is that the CentOS delivered packages seems to be newer than that provided by RHEL. This can be solved by downgrading the packages.

That may be the same with other packages as well.

[root@centos8 ~]# yum downgrade glibc

At the end, there will be one package left: python3-gpg. Due to some dependencies it can only be reinstalled after the rest of the packages are reinstalled/downgraded.

[root@centos8 ~]# yum reinstall python3-gpg

Check if there are still RPMs of vendor “Centos” installed:

[root@centos7 ~]# rpm -qa --queryformat "%{NAME} %{VENDOR}\n" | grep -i centos | cut -d' ' -f1

This should return nothing, almost all is now RHEL8. The only traces left are the previously installed Kernels. They will get deleted over time when installing (updating) new Kernels.

In my case I just used CentOS8 “Server” installation. If you plan to go down this road, please clone the system first for testing before migrating the actual system.

Support by Redhat

Will the converted machine be supported after this procedure? Well, officially it is not supported, but if there are no traces of CentOS left on the machine…

As mentioned before, better install RHEL in the first place 🙂

Using Data Deduplication and Compression with VDO on RHEL 7 and 8

Storage deduplication technology has been on the market for quite some time now. Unfortunately all of the implementations have been vendor specific proprietary software. With VDO, there is now an open source Linux native solution available.

Red hat has introduced VDO (Virtual Data Optimizer) in RHEL 7.5, a storage deduplication technology bough with Permabit in 2017. Of course it has been open sourced since then.

In contrast to ZFS which provides the same functionality on the file system level, VDO is an inline data reduction which works on block device level, it is file system agnostic.

Use cases

There are basically two major use cases: VM Storage and Object Storage Backends.

VM Storage

The main use case is storage for virtual machines where a lot of data is redundant, i.e. the base operating system of the VMs. This allows to deduplicate the data on disk on a large scale, think about 100 VMs where the operating system takes about 5Gbyte each will be reduced to approx. 5 Gbyte instead of 500 Gbyte.

Typically VM storage can be over committed by factor 10.

Object- and Block storage backends

As a backend for CEPH and Glusterfs, it is recommended to not over commit more than factor 3. The reason for the lower over commitment is that the storage administrator usually does not know what kind of data will be stored on it.

Availability

VDO is available since RHEL 7.5, it is included in the base subscription. At the moment it is not available for Fedora (yet).

The source code is available on github:

At the moment the Kernel code is not yet in the upstream Mainline Kernel, it is ongoing work to get it into the Mainstream Kernel.

Typical setup

Physical disk -> VDO -> Volumegroup -> Logical volume -> file system.

Block device can be a physical disk (or a partition on it), multi path device, LUKS disk, or a software RAID device (md or LVM RAID).

Restrictions

You can not use LVM cache, LVM snapshots and thin provisioned logical volumes on top of VDO. Theoretically you can use LUKS on top of VDO, but it makes no sense because there is nothing to deduplicate. Needless to say that VDO on top of a VDO device does not make any sense as well. Be aware that you can not make use of partitioning or (LVM) Raid on top of VDO devices, all that things should be done in the underlying layer of VDO.

When using SAN, check if your storage box already does deduplication. In this case VDO is useless for you.

Installation

Its straight forward:

[root@vdotest ~]# yum -y install vdo kmod-kvdo

Create the VDO volume

In this test case, I attached a 110Gbyte disk, created a 100 GByte partition and will over commit it by factor 10.

Warning! As of writing this article, never use a whole physical disk, use a partition instead and leave some spare space in the disk to avoid data loss! (see further below)

[root@vdotest ~]# vdo create --name=vdo1 --device=/dev/vdb1 --vdoLogicalSize=1T

Creating volume group, logical volume and file system on top of the VDO volume

[root@vdotest ~]# pvcreate /dev/mapper/vdo1
[root@vdotest ~]# vgcreate vg_vdo /dev/mapper/vdo1
[root@vdotest ~]# lvcreate -n lv_vdo vg_vdo -L 900G
[root@vdotest ~]# mkfs.xfs -K /dev/vg_vdo/lv_vdo
[root@vdotest ~]# echo "/dev/mapper/vg_vdo-lv_vdo       /mnt    xfs     defaults,x-systemd.requires=vdo.service 0 0" >> /etc/fstab

Display the whole stack

[root@vdotest ~]# lsblk /dev/vdb
NAME                MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
vdb                 252:16   0  110G  0 disk 
└─vdb1              252:17   0  100G  0 part 
  └─vdo1            253:7    0    1T  0 vdo  
    └─vg_vdo-lv_vdo 253:8    0  900G  0 lvm  /mnt
[root@vdotest ~]#

Populate the disk with data

The ideal test for VDO is to put some real-life VM-Images to the file system on top of it. In this case I scp’ed three IPA server and some instances to that file system. This kind of systems are all quite similar, the disk space saved is tremendous. The total size of the vm images is 105G

Lets have a look:

[root@vdotest ~]# df -h /mnt
Filesystem                 Size  Used Avail Use% Mounted on
/dev/mapper/vg_vdo-lv_vdo  900G  105G  800G  12% /mnt
[root@vdotest ~]# 
[root@vdotest ~]# ll -h /mnt
total 101G
-rw-------. 1 root root 21G Dec 17 09:41 ipa1.lab.delouw.ch.qcow2
-rw-------. 1 root root 21G Dec 17 09:47 ipa1.ldelouw.ch
-rw-------. 1 root root 21G Dec 17 09:54 ipa2.ldelouw.ch
-rw-r--r--. 1 root root 21G Dec 17 09:58 ipaclient-rhel6.home.delouw.ch
-rw-------. 1 root root 21G Dec 17 10:03 ipatest.delouw.ch.qcow2
[root@vdotest ~]#

Lets use the vdostats utility to display the actually used storage on disk:

[root@vdotest ~]# vdostats --si
Device                    Size      Used Available Use% Space saving%
/dev/mapper/vdo1        107.4G     15.2G     92.2G  14%           89%
[root@vdotest ~]#

Performance Tuning

There are a lot of parameters that can be changed. Unfortunately the documentation available at the moment is rudimentary, thus its more a guesswork than facts.

  • Number of worker threads of different kind
  • Enable or Disable compression

On machines with a lot of CPUs using more threads than the defaults can dramatically boost performance. man 8 vdo gives a glimpse of the different parameters related to threads.

Compression is a quite expensive operation. On top of that, depending on the kind of data you are storing, it does not make much sense to use compression (Well, deduplication is kind of compression as well).

Pitfalls

Be aware! With every storage deduplication solution there comes a big pitfall: The logical volume on top of VDO shows free disk space while the actual disk space on the physical disk can be (almost) exhausted. You need to carefully monitor the actual disk usage.

The fill grade can rapidly change if the data to be stored contains a lot of non-deduplicatable and/or compressible data. A good example is virtual machine images containing a LUKS encrypted disk, In such a case, use LUKS on the storage, not on the VM level.

Even if you update one virtual machine, the delta to other machine images will grow and less physical space is available.

VDO comes with a few Nagios plugins which are very useful for alerting administrators in the cause the available physical disk is filling up. They are located in /usr/share/doc/vdo/examples/nagios

According to df -h, on my test system there is still 800 Gbyte available. What happens if I store my 700 Gbyte Satellite 6 image? The data is mostly RPMs which are already compressed quite well. Lets see….

After a transfer of approx 155 Gbyte, the physical disk got full and the file system is inaccessible. I was hitting the worst case that can happen: Complete and unrecoverable data loss.

The df command shows some 241 Gbyte free.

[root@vdotest ~]# df -h |grep mnt
/dev/mapper/vg_vdo-lv_vdo                900G  241G  660G  27% /mnt
[root@vdotest ~]#

The vdostat command tells a different story, like expected.

[root@vdotest ~]# vdostats --si
Device                    Size      Used Available Use% Space saving%
/dev/mapper/vdo1        107.4G    107.4G      0.0B 100%           59%
[root@vdotest ~]# 

When attempting to access the data, there will be an I/O error.

[root@vdotest ~]# ll -h /mnt
ls: cannot access /mnt: Input/output error
[root@vdotest ~]# 

Thats bad. I mean really bad. The device is not accessible anymore.

xfs_repair does not work. Do not attempt to make use of the -L option! Your file system will be gone.

Recovering from a full physical disk

Lets resize the partition instead. First unmount the file system

[root@vdotest ~]# umount /mnt

Delete and recreate the partition using fdisk

[root@vdotest ~]# fdisk /dev/vdb
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): d
Selected partition 1
Partition 1 is deleted

Command (m for help): n
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p): 
Using default response p
Partition number (1-4, default 1): 
First sector (2048-41943039, default 2048): 
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-41943039, default 41943039): 
Using default value 41943039
Partition 1 of type Linux and of size 20 GiB is set

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.
[root@vdotest ~]# partprobe
[root@vdotest ~]#
[root@vdotest ~]# vdo growPhysical -n vdo1

Run a file system check.

Now you are able to mount the file system again and your data is available again.

Documentation

Red Hat maintains a nice documentation about storage administration, VDO is covered by an own chapter. https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html-single/storage_administration_guide/#vdo

Conclusion

The technology is very interesting and will kick some ass. Storage deduplication will be more and more important, with VDO there is now a Linux native solution for that.

At the moment it is quite dangerous to use VDO in production. Filling up a physical disk without spare space is an unrecoverable error, a complete data loss. That means: Always create the VDO device on top of a partition that is not using the whole disk or another device that can grow in size to prevent data loss.

If you plan to use VDO in production make sure you have a proper monitoring in place that alerts quite ahead of time to be able to take corrective action.

Nevertheless: Its cool stuff and I’m sure the current situation will be fixed soon.

Audit your systems for security compliance with OpenSCAP

OpenSCAP logoIntroduction to (Open)SCAP

SCAP stands for Security Content Automation Protocol. It is an open standard which defines methods for security policy compliance, vulnerability management and measurement etc. This article focuses on the operating system compliance part of SCAP.

It comes originally from the US National Institute of Standards and Technology (NIST) to provide a way for US government agencies to audit its systems for regulatory compliance.

OpenSCAP is a NIST validated open source implementation of SCAP.

Why should I make use of OpenSCAP anyway?

Lot of people will ask this question to them self, in particular System Administrators and Engineers since they are not IT Security Officers.

The simple answer is that you just sit down with the IT Security Officer once and define which systems need to be compliant to what regulatory, With OpenSCAP you can always ensure the systems are configured according the the policy (or policies).

Organizations that need to be compliant according to a official policy will sooner or later facing an external security audit. I experienced that several times, its a nightmare. If you can proof that your systems are scanned regularly with the SCAP standard, you will be very well prepared, an external auditor will not bug you for a long time.

Abbreviations, abbreviations, abbreviations

Its obvious, government agencies love abbreviations 😉 Lets explain the two most important ones.

XCCDF

Extensible Configuration Checklist Description Format. This files, i.e. /usr/share/xml/scap/ssg/content/ssg-rhel7-xccdf.xml contain descriptions used for auditing a system against compliance to a policy.

This files are usually included in your distribution and are updated if needed.

OVAL

Open Vulnerability and Assessment Language. Its used to detect vulnerabilities and patches.

Since vulnerabilities and patches are popping up very quickly they need to be downloaded and distributed to all systems to be audited on a regular base (i.e. daily).

OVAL files can be downloaded as listed below:

Organizations using System Management Tools such as Red Hat Satellite or SUSE Magager will not profit from OVAL patch scans as those products will report which patches have been applied or not by themself. Nevertheless, additional OVAL scans add the benefit of vulnerability scanning regardless of installed patches.

More Abbreviations

More abbreviations and a short description of them can be found here: https://www.open-scap.org/resources/acronyms/

OpenSCAP Scap Security Guide (SSG)

There are a lot of regulations out there. Government of some countries releases policies and sometimes SCAP content for some Operating Systems, mostly RHEL and Windows. The SSG Project works on collecting and implementing content for this policies for the operating systems as well as for some other software such as JBoss. Included in the scap-security-guide are the most important US Government and PCI-DSS for RHEL. Only available for Debian at the moment is the content for the French ANSSI DAT-NT28.

The only Linux distributions I’m aware of that provides packages for scap-security-guide are RHEL and Fedora. However, upstream there is some content for more distributions available. I really hope that all important and fine distributions such as SLES, Debian and Ubuntu will jump on the bandwagon.

Regulations covered by OpenSCAP SSG

Here a list of what is available for the most important Linux distributions.

Red Hat Enterprise Linux 7

  • PCI-DSS (Payment Card Industry – Data Security Standard), Commercial – USA
  • C2S (Commercial Cloud Services), Government – USA
  • USGCB/STIG (United States Government Configuration Baseline/Security Technical Implementation Guide), Government – USA
  • CNSSI 1253 (Committee on National Security Systems), Government – USA
  • CJIS (Criminal Justice Information Services), Government – USA

Debian and Ubuntu

Officially there is nothing available. Its is currently under development, see https://github.com/OpenSCAP/scap-security-guide/tree/master/Ubuntu/16.04 and https://github.com/OpenSCAP/scap-security-guide/tree/master/Debian/8.

As of 2017-03-04 compiling fails.

  • ANSSI DAT-NT28 (Agence nationale de la sécurité des systèmes d’information), Government – France

Suse Linux Entrprise Server

Suse does not provide the scap-security-guide package and there is no XCCDF content for regulatory compliance checks delivered by Suse. However, some basic tests are available. It is not clear if Suse has some plans to join the scap-security-guide community, would be nice to see that. SLES customers can open a support case at https://scc.suse.com/login and ask for enhancement.

Using SCAP content without scap-security-guide

You can make use of SCAP content without the OpenSCAP security guide. Its rather complex and not covered in this article.

Installing the required packages

RHEL 7

[root@server ~]# yum -y install scap-security-guide

All required dependencies will be installed as well

Debian and Ubuntu

root@ubuntu:~# aptitude install python-openscap

All required dependencies will be installed as well

SLES12sp2

sles12sp2:~ # zypper install openscap openscap-content openscap-extra-probes openscap-utils

All required dependencies will be installed as well

Tailoring profiles

For most users it is probably too much to secure its systems according to military standards which includes turning off USB support and the like.

The most important civil regulatory by far is PCI-DSS. Each company handling kind of Credit- or Debitcard data must obey the current standard. As of writing this article this is version 3.2.

PCI-DSS is a de-facto standard in Enterprise Linux environments.

Of course it makes sense for all kind of companies to secure its systems. On systems which are not exposed, security policies can be more relaxed.

Also good to know is that some tests simply do not apply to your system. I.e. if you are using a centralized identity management software such as Redhat IdM with IPA or Microsoft Active Directory then the central instance will take care about the password policies, not the particular system to be audited.

Installation of the SCAP Workbench

The Scap Workbench is available in RHEL to be installed by yum, a binary for Windows and Mac OS is available as well. Needless to say that the source code is available.

Downloads: https://github.com/OpenSCAP/scap-workbench/releases

Usage

In the following examples, we disable the check for AIDE.

SCAP-Workbench Screencast

SCAP-Workbench Screencast

You can save the tailoring file as a single XML file or even better safe it as an RPM for easy distribution to all your systems.

Scanning

The usage is the same on all tested Linux distributions. Be aware, XCCDF scanning makes no sense w/o any SCAP content. If your distribution does not provide you the necessary data, 3rd party providers may.

RHEL 7 comes with the scap-workbench which is GUI that allows you to scan the local or remote systems via SSH. The scap-workbench is a nice tool to scan a handful of servers manually but not to scan a whole zoo of servers.

You also can scan your systems with the CLI on the host itself. Kind of automation can be done with i.e with Ansible.

Manual Scan

The oscap info command gives you an overview which profiles are available.

[root@server ~]# oscap info /usr/share/xml/scap/ssg/content/ssg-rhel7-xccdf.xml
Document type: XCCDF Checklist
Checklist version: 1.1
Imported: 2017-02-14T13:33:08
Status: draft
Generated: 2017-02-14
Resolved: true
Profiles:
        standard
        pci-dss
        C2S
        rht-ccp
        common
        stig-rhel7-workstation-upstream
        stig-rhel7-server-gui-upstream
        stig-rhel7-server-upstream
        ospp-rhel7-server
        nist-cl-il-al
        cjis-rhel7-server
Referenced check files:
        ssg-rhel7-oval.xml
                system: http://oval.mitre.org/XMLSchema/oval-definitions-5
        ssg-rhel7-ocil.xml
                system: http://scap.nist.gov/schema/ocil/2
        http://www.redhat.com/security/data/oval/Red_Hat_Enterprise_Linux_7.xml
                system: http://oval.mitre.org/XMLSchema/oval-definitions-5
[root@server ~]# 

Lets choose pci-dss and start a scan:

[root@server ~]# oscap xccdf eval --profile pci-dss --results scan.xml --report scan.html /usr/share/xml/scap/ssg/content/ssg-rhel7-xccdf.xml
Title   Ensure Red Hat GPG Key Installed
Rule    ensure_redhat_gpgkey_installed
Ident   CCE-26957-1
Result  pass

Title   Ensure gpgcheck Enabled In Main Yum Configuration
Rule    ensure_gpgcheck_globally_activated
Ident   CCE-26989-4
Result  pass
[Lot of Output immited]

The parameter –results saves the result in a HTML file.

Automated scanning with Redhat Satellite 6

Users of Redhat Satellite 6 can schedule scans of large server farms. The screenshots shows you how compliance tests can be presented to a IT Security Officer.

Compliance Report

Compliance Overview

The Compliance report shows a overview of hosts and a brief look at how many test have been failed.

Compliance Report Detail view

Compliance Report Detail view

The Compliance report detail shows which test have been failed. It also provides a description of each topic.

Host details

Host details

The detail view of a host shows that this host is not compliant. In this case, security errata must be applied and the host must be reconfigured to get compliant to the security policy.

Alternatives to OpenSCAP

There are a few alternatives to OpenSCAP as listed by the NIST’s Security Content Automation Protocol Validated Products.

Further reading