Upgrading Postgresql from 9.5 to 10.5

We want to go to the most recent Postgresql for use by dCache, at least on head01.aglt2.org. Currently Postgresql is version 9.5 and has a slave server t-head01.aglt2.org.

The plan is as follows:

• Install the Postgresql 10 yum repo on head01 and t-head01 (Done August 20, 2018)
• Create scripts to
  • Initialize the Postgresql 10 DB (Done August 20, 2018)
  • Check the pg_upgrade (check_pg_upgrade.sh)
  • Run the pg_upgrade process using the --link option (Done August 20, 2018)
• Re-organize the ZFS space holding the database on head01 so everthing in /var/lib/pgsql and below is in one ZFS filesystem (Done August 20, 2018)
• Tune up the ZFS storage to better support Postgresql. See https://www.aglt2.org/wiki/bin/view/AGLT2/PostgresqlonZFS (Done August 21, 2018)
• Install Postgresql 10 on both head01 and t-head01 (Done August 21, 2018)
• Create the target (empty) Postgresql 10 DB using the above script on head01 (Done August 21, 2018)
  • As user 'postgres' run the initdb_10.sh script in /var/lib/pgsql on both head01 and t-head01 created the data directory /var/lib/pgsql/10/data
  • Update the following configuration files to match the settings from the 9.5 DB: postgresql.conf, pg_hba.conf and recovery.conf (t-head01 only)
• As user 'postgres' run the check_pg_upgrade.sh command to verify things are OK on head01 (Done August 21, 2018)
• Run the actual pg_upgrade
  • First shutdown dCache on head01
  • Next shutdown postgresql-9.5 on head01, wait 30 seconds, then on t-head01
  • Wait at least 15 seconds to settle things out...
  • Take a ZFS snapshot on head01 so we can roll-back in case of problems: zfs snap pgsql/dcache@pre-psql10-upgrade
  • Make sure that postgresql is off on both head01 and t-head01
  • As user 'postgres' run the /var/lib/pgsql/run_pg_upgrade.sh script
• Verify postgresql10 will start and that the DB looks OK on head01
• Start dcache on head01
• Now run the reseed-link-upgrade.sh on head01 (as 'root') to get t-head01 "updated" (see upgrade guide at https://www.postgresql.org/docs/10/static/pgupgrade.html)
• Verify postgresql10 will start and that the DB looks OK on t-head01

-- ShawnMcKee - 21 Aug 2018