Upgrading RHN Satellite 5.6 to 5.7

This post guides you trough the upgrade procedure for a Satellite 5.6 using the embedded database on RHEL6-x86_64. Further it guides you to setup of Kerberos authentication of Satellite users with IPA.

Recently Redhat released Satellite Server 5.7. Despite Satellite 5.x will be outphased in the next few years, there are plenty of new features. The most significant new features are:

  • Upgraded PostgreSQL to 9.2
  • Authentication via IPA/SSSD/Kerberos
  • IPMI support
  • Renewed WebUI
  • Readonly API users

And finally… drum roll…. formal support for spacecmd 🙂

As always when you plan to upgrade your Satellite server to the latest version, you need to do some preparations first.

Download the ISO
As usual, visit the Download site and make sure you select 5.7 and the architecture fitting you system (x86_64 or S390)

Get a new Satellite Certificate
Satellite 5.7 needs a new certificate to get it activated. You can create it by your own at the Subscription Management Application site, ensure you attach enough subscriptions to your Satellite server(s). Alternatively open a support case.

Backup
Usually an upgrade runs smooth, but just in case… it is recommended practice to have a recent backup ready. If your Satellite is running on a virtual machine, power off, snapshot and power on to have a consistent backup ready. For physical systems, db-control and the choice of your backup software need to be visited.

Backup the rest of your Satellite:

Create a copy of your rhn configuration directory as we need some information from the old files after the upgrade.

[root@rhnsat ~]# cp -rp /etc/rhn/ /etc/rhn-$(date +"%F")

Update your OS and Satellite
First step is to update the operating system and the Satellite 5.6 and apply the latest database schema updates as well.

yum -y update && reboot

To update the database schema, run the following command. Ideally it looks as follows:

root@rhnsat ~]# spacewalk-schema-upgrade 

You are about to perform upgrade of your satellite-schema.

For general instructions on Red Hat Satellite schema upgrade, please consult
the following article:

    https://access.redhat.com/knowledge/articles/273633

Hit Enter to continue or Ctrl+C to interrupt: 
Schema upgrade: [satellite-schema-5.6.0.21-1.el6sat] -> [satellite-schema-5.6.0.21-1.el6sat]
Your database schema already matches the schema package version [satellite-schema-5.6.0.21-1.el6sat].
[root@rhnsat ~]# 

Functionality Check
It is recommended to restart and check a softwares functionality before upgrading to be able to pinpoint problems if there are some.

[root@rhnsat ~]# rhn-satellite restart

Cleanup
Its a good idea to review the software channels in use and delete unused channels as this can free up quite some diskspace and reduces the size of the database significantly.

[root@rhnsat ~]# spacewalk-remove-channel -c rhel-i386-rhev-agent-6-server
Deleting package metadata (20):
                  ________________________________________
Removing:         ######################################## - complete
[root@rhnsat ~]#

Delete old system snapshots not used anymore. The following example deletes all snapshots which are older than one month:

[root@rhnsat ~]# sw-system-snapshot --delete --all --start-date 200001010000 --end-date $(date -d "-1 months" "+%Y%m%d0000" 

Remove spacecmd from EPEL
Most Satellite users have spacecmd installed from EPEL. Its a good idea to remove it to avoid conflicts. It is also important to disable the EPEL repositories on Satellite servers as a simple yum update can bring your Satellite server into trouble.

RTFM
If not done yet, install the rhn-upgrade package which contains the instructions how to proceed.

yum -y install rhn-upgrade

The package contains not only SQL- and other useful scripts needed for the upgrade but also important documents to read. The are located in /etc/sysconfig/rhn/satellite-upgrade/doc.

For most users, the document satellite-upgrade-postgresql.txt applies.

Do not forget to read the updated product documentation as well:

Changing your file system layout
As there will be an updated PostgreSQL version needed which is part of the Software Collection and not installable from the base channel, you need to add a new file system in /opt/rh.
The new database is about the same size as before. Check your used disk space at /var/lib/pgsql

[root@rhnsat ~]# lvcreate /dev/vg_data -n lv_opt_rh -L 17G 
[root@rhnsat ~]# mkfs.ext4 /dev/vg_data/lv_opt_rh
[root@rhnsat ~]# tune2fs -c0 -i0  /dev/vg_data/lv_opt_rh

Exit your /etc/fstab accordingly and mount the file system with mount -a to check if it working as expected.

Lets do it
Mount the ISO image and run the installer.

[root@rhnsat ~]# mount satellite-5.7.0-20150108-rhel-6-x86_64.iso /mnt -o loop
[root@rhnsat ~]# cd /mnt
[root@rhnsat mnt]# 

If you are using a proxy to sync your satellite, provide the --diconnected flag.

[root@rhnsat mnt]# ./install.pl --upgrade --disconnected
* Starting Red Hat Satellite installer.
* Performing pre-install checks.
* Pre-install checks complete.  Beginning installation.
* RHN Registration.
** Registration: Disconnected mode.  Not registering with RHN.
* Upgrade flag passed.  Stopping necessary services.
* Purging conflicting packages.
* Checking for uninstalled prerequisites.
** Checking if yum is available ...
There are some packages from Red Hat Enterprise Linux that are not part
of the @base group that Satellite will require to be installed on this
system. The installer will try resolve the dependencies automatically.
However, you may want to install these prerequisites manually.
Do you want the installer to resolve dependencies [y/N]? y
* Installing RHN packages.
* Now running spacewalk-setup.
* Setting up SELinux..
** Database: Setting up database connection for PostgreSQL backend.
*** Upgrading embedded database.
** Database: Populating database.
** Database: Skipping database population.
* Setting up users and groups.
** GPG: Initializing GPG and importing key.
* Performing initial configuration.
* Activating Red Hat Satellite.
** Certificate not activated.
** Upgrade process requires the certificate to be activated after the schema is upgraded.
* Enabling Monitoring.
* Configuring apache SSL virtual host.
Should setup configure apache's default ssl server for you (saves original ssl.conf) [Y]? y
* Configuring tomcat.
* Configuring jabberd.
* Creating SSL certificates.
** Skipping SSL certificate generation.
* Deploying configuration files.
* Update configuration in database.
* Setting up Cobbler..
task started: 2015-02-08_154708_sync
task started (id=Sync, time=Sun Feb  8 15:47:08 2015)
running pre-sync triggers
cleaning trees
removing: /var/www/cobbler/images/ks-rhel-x86_64-es-4-u6
[..]
running shell triggers from /var/lib/cobbler/triggers/change/*
*** TASK COMPLETE ***
Cobbler requires tftp and xinetd services be turned on for PXE provisioning functionality. Enable these services [Y]? y
This portion of the Red Hat Satellite upgrade process has successfully completed.
Please refer to appropriate upgrade document in /etc/sysconfig/rhn/satellite-upgrade
for any remaining steps in the process.
[root@rhnsat mnt]#

The next step is having a look at diff /etc/rhn/rhn.conf /etc/rhn-$(date +”%F”)/rhn.conf
and edit /etc/rhn/rhn.conf accordingly. You will probably see missing things such as proxy, server.satellite.rhn_parent etc. Also change the setting disconnected to 0.

Upgrade database scheme
Before doing anything, first update the database schema to 5.7.

[root@rhnsat ~]# spacewalk-schema-upgrade

Activate the updated Satellite server
To subscribe the Satellite server to the appropriate software channels, it must be activated. Since it was activated before, the --ignore-version-mismatch parameter must be provided.

[root@rhnsat ~]# rhn-satellite-activate --rhn-cert=rhn-satellite57-2015-02-08.xml --ignore-version-mismatch

Initial Update of Software and database schema
There is a good chance that there are updates available for the Satellite Server as the ISO image will not be updated that often.

[root@rhnsat ~]# yum -y update

Even if no update was installed, there is a schema update available:

[root@rhnsat ~]# spacewalk-schema-upgrade 
Schema upgrade: [satellite-schema-5.6.0.21-1.el6sat] -> [satellite-schema-5.7.0.11-1.el6sat]
Searching for upgrade path: [satellite-schema-5.6.0.21-1] -> [satellite-schema-5.7.0.11-1]
Searching for upgrade path: [satellite-schema-5.6.0.21] -> [satellite-schema-5.7.0.11]
Searching for upgrade path: [satellite-schema-5.6.0] -> [satellite-schema-5.7.0]
Searching for upgrade path: [satellite-schema-5.6] -> [satellite-schema-5.7]
The path: [satellite-schema-5.6] -> [satellite-schema-5.7]
Planning to run spacewalk-sql with [/var/log/spacewalk/schema-upgrade/20150208-155657-script.sql]

Plase make sure you have a valid backup of your database before continuing.

Hit Enter to continue or Ctrl+C to interrupt: 
Executing spacewalk-sql, the log is in [/var/log/spacewalk/schema-upgrade/20150208-155657-to-satellite-schema-5.7.log].
The database schema was upgraded to version [satellite-schema-5.7.0.11-1.el6sat].
[root@rhnsat ~]# 

Cleanup
After startarting the Satellite Server, the package meta data should be automatically recreated. If not, run
/etc/sysconfig/rhn/satellite-upgrade/scriptsregenerate-repodata manually.

Rebuild the search index:

[root@rhnsat ~]# service rhn-search cleanindex

You don’t need to remove the old PostgreSQL version, this is done automatically.

Using IPA and Kerberos for authentication
Before configure the Satellite Server to use IPA, make sure it is enrolled and the HTTP service principal exists. If not, add it with the following command:

[root@ipa1 ~]# ipa-addservice HTTP/rhnsat.example.com@EXAMPLE.COM

Next will be getting a Kerbros Ticket of a user allowed to create Keytabs. In this example it is the user admin.

[root@rhnsat ~]# kinit admin
Password for admin@EXAMPLE.COM: 
[root@rhnsat ~]# 

Afterwards, run the setup script:

[root@rhnsat ~]# spacewalk-setup-ipa-authentication
Enabling authentication against [ipa2.example.com].
Retrieving HTTP/ service keytab into [/etc/httpd/conf/http.keytab] ...
Keytab successfully retrieved and stored in: /etc/httpd/conf/http.keytab
changed ownership of `/etc/httpd/conf/http.keytab' to apache
Configuring PAM service [spacewalk].
Will install additional packages ...
Loaded plugins: product-id, rhnplugin, security, subscription-manager
This system is receiving updates from RHN Classic or RHN Satellite.
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package mod_auth_kerb.x86_64 0:5.4-13.el6 will be installed
---> Package mod_authnz_pam.x86_64 0:0.9.2-1.el6 will be installed
---> Package mod_intercept_form_submit.x86_64 0:0.9.7-1.el6 will be installed
---> Package mod_lookup_identity.x86_64 0:0.9.2-1.el6 will be installed
---> Package sssd-dbus.x86_64 0:1.11.6-30.el6_6.3 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

============================================================================================================================================================
 Package                                      Arch                      Version                               Repository                               Size
============================================================================================================================================================
Installing:
 mod_auth_kerb                                x86_64                    5.4-13.el6                            rhel-x86_64-server-6                     30 k
 mod_authnz_pam                               x86_64                    0.9.2-1.el6                           rhel-x86_64-server-6                     13 k
 mod_intercept_form_submit                    x86_64                    0.9.7-1.el6                           rhel-x86_64-server-6                     17 k
 mod_lookup_identity                          x86_64                    0.9.2-1.el6                           rhel-x86_64-server-6                     19 k
 sssd-dbus                                    x86_64                    1.11.6-30.el6_6.3                     rhel-x86_64-server-6                    122 k

Transaction Summary
============================================================================================================================================================
Install       5 Package(s)

Total download size: 201 k
Installed size: 0  
Downloading Packages:
(1/5): mod_auth_kerb-5.4-13.el6.x86_64.rpm                                                                                           |  30 kB     00:00     
(2/5): mod_authnz_pam-0.9.2-1.el6.x86_64.rpm                                                                                         |  13 kB     00:00     
(3/5): mod_intercept_form_submit-0.9.7-1.el6.x86_64.rpm                                                                              |  17 kB     00:00     
(4/5): mod_lookup_identity-0.9.2-1.el6.x86_64.rpm                                                                                    |  19 kB     00:00     
(5/5): sssd-dbus-1.11.6-30.el6_6.3.x86_64.rpm                                                                                        | 122 kB     00:00     
------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                        41 kB/s | 201 kB     00:04     
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : mod_authnz_pam-0.9.2-1.el6.x86_64                                                                                                        1/5 
  Installing : mod_intercept_form_submit-0.9.7-1.el6.x86_64                                                                                             2/5 
  Installing : mod_auth_kerb-5.4-13.el6.x86_64                                                                                                          3/5 
  Installing : mod_lookup_identity-0.9.2-1.el6.x86_64                                                                                                   4/5 
  Installing : sssd-dbus-1.11.6-30.el6_6.3.x86_64                                                                                                       5/5 
  Verifying  : mod_intercept_form_submit-0.9.7-1.el6.x86_64                                                                                             1/5 
  Verifying  : sssd-dbus-1.11.6-30.el6_6.3.x86_64                                                                                                       2/5 
  Verifying  : mod_lookup_identity-0.9.2-1.el6.x86_64                                                                                                   3/5 
  Verifying  : mod_authnz_pam-0.9.2-1.el6.x86_64                                                                                                        4/5 
  Verifying  : mod_auth_kerb-5.4-13.el6.x86_64                                                                                                          5/5 

Installed:
  mod_auth_kerb.x86_64 0:5.4-13.el6                  mod_authnz_pam.x86_64 0:0.9.2-1.el6            mod_intercept_form_submit.x86_64 0:0.9.7-1.el6          
  mod_lookup_identity.x86_64 0:0.9.2-1.el6           sssd-dbus.x86_64 0:1.11.6-30.el6_6.3          

Complete!
** /etc/sssd/sssd.conf has been backed up to sssd.conf-swsave
Updated sssd configuration.
Turning SELinux boolean [httpd_dbus_sssd] on ...
        ... done.
Turning SELinux boolean [allow_httpd_mod_auth_pam] on ...

        ... done.
Configuring Apache modules.
** /etc/tomcat6/server.xml has been backed up to server.xml-swsave.ipa
Stopping sssd:                                             [  OK  ]
Starting sssd:                                             [  OK  ]
Stopping tomcat6:                                          [  OK  ]
Starting tomcat6:                                          [  OK  ]
Stopping httpd:                                            [  OK  ]
Starting httpd:                                            [  OK  ]
Waiting for tomcat to be ready ...
Authentication against [ipa2.example.com] sucessfully enabled.
As admin, at Admin > Users > External Authentication, select
          Default organization to autopopulate new users into.
[root@rhnsat ~]# 

Next, point your browser to https://rhnsat.example.com/rhn/admin/multiorg/ExternalAuthentication.do to finalize the setup.

Configure your browser for Kerberos
If you did not yet configured your browser to use Kerberos authentication, do so. Assuming you are using an IPA invironment, follow the instructions provided on the IPA servers. http://ipa2.example.com/ipa/config/browserconfig.html

Disclaimer
I take no responsibility about damaged Satellites, lost of data etc. in doubt, stick on the official product documentation at http://access.redhat.com

One thought on “Upgrading RHN Satellite 5.6 to 5.7

  1. mark says:

    Hi,
    I am failing upgrade from Satellite 5.6 to 5.7 with the internal PostgreSQL database…..
    # /usr/bin/spacewalk-schema-upgrade
    psql: FATAL: password authentication failed for user “rhnuser”
    Connect to database was not successful.

    Have you seen this before?

    thanks,
    Mark

    PS
    I used you blog to migrate from 5.5 to 5.6 and it worked like a charm!

Leave a Reply

Your email address will not be published. Required fields are marked *