Informix Storage Migration via Mirroring

Abstract

IBM Informix Dynamic Server (IDS) databases reside in "dbspaces", each composed of one or more "chunks" (files, logical volumes or whole disk devices). If the storage infrastructure is being upgraded, you might find that moving chunks via backup & restore or external copy would take longer than the outage window allows, particularly if this is between different sites.

This article describes a method of achieving this with only a very short downtime using Informix chunk mirroring.

Content

Best practice is that storage devices are never directly presented to IDS as chunks, but only symbolic links to them. This makes a variety of device change scenarios easier, and facilitates standardisation across replicas. It is a prerequisite for the technique described here. Also, you must restart IDS with "MIRROR 1" set in the $INFORMIXDIR/etc/$ONCONFIG configuration file beforehand for the database engine feature to be available.

The example described here is based on an actual case where iSCSI LUNs were replaced with logical volumes in a local SSD RAID 10 array on Linux. This was performed on both the primary instance and a High-availability Data Replication (HDR) secondary server.

The following files were created on both servers (generated in Excel where easier):

99-informix.rules

ENV{DM_VG_NAME}=="dbsvg", ENV{DM_LV_NAME}=="*", OWNER:="informix", GROUP:="informix", MODE:="660"

lvcreate.sh

vgcreate dbsvg /dev/sdb

lvcreate -L 74752M -n appdbs_1  dbsvg

lvcreate -L 74752M -n appdbs_2  dbsvg

lvcreate -L 74752M -n appdbs_3  dbsvg

lvcreate -L 74752M -n bigdbs_1  dbsvg

lvcreate -L 74752M -n bigdbs_2  dbsvg

lvcreate -L 74752M -n bigdbs_3  dbsvg

lvcreate -L 10240M -n llogdbs_1 dbsvg

lvcreate -L 10240M -n llogdbs_2 dbsvg

lvcreate -L 1000M  -n physdbs_1 dbsvg

lvcreate -L 100M   -n rootdbs_1 dbsvg

link.sh

DIR=/opt/informix/dbmirror

[ -d $DIR ] || mkdir $DIR ; cd $DIR

for i in /dev/mapper/dbsvg-*

do echo ln -sf $i $(echo $i | cut -d- -f2)

done

mirror.appdbs.map

/opt/informix/dbspaces/appdbs_1  0  /opt/informix/dbmirror/appdbs_1  0

/opt/informix/dbspaces/appdbs_2  0  /opt/informix/dbmirror/appdbs_2  0

/opt/informix/dbspaces/appdbs_3  0  /opt/informix/dbmirror/appdbs_3  0

mirror.bigdbs.map

/opt/informix/dbspaces/bigdbs_1  0  /opt/informix/dbmirror/bigdbs_1  0

/opt/informix/dbspaces/bigdbs_2  0  /opt/informix/dbmirror/bigdbs_2  0

/opt/informix/dbspaces/bigdbs_3  0  /opt/informix/dbmirror/bigdbs_3  0

mirror.llogdbs.map

/opt/informix/dbspaces/llogdbs_1 0  /opt/informix/dbmirror/llogdbs_1 0

/opt/informix/dbspaces/llogdbs_2 0  /opt/informix/dbmirror/llogdbs_2 0

mirror.physdbs.map

/opt/informix/dbspaces/physdbs_1 0  /opt/informix/dbmirror/physdbs_1 0

mirror.rootdbs.map

/opt/informix/dbspaces/rootdbs_1 0  /opt/informix/dbmirror/rootdbs_1 0

mirror.sh

for dbspace in $(ls mirror.*.map | cut -d. -f2)

do onspaces -m $dbspace -f mirror.$dbspace.map ; sleep 10

done

recover.sh

onspaces -s rootdbs -p /opt/informix/dbmirror/rootdbs_1 -o 0 -O -y ; sleep 10

onspaces -s physdbs -p /opt/informix/dbmirror/physdbs_1 -o 0 -O -y ; sleep 10

onspaces -s llogdbs -p /opt/informix/dbmirror/llogdbs_1 -o 0 -O -y ; sleep 10

onspaces -s llogdbs -p /opt/informix/dbmirror/llogdbs_1 -o 0 -O -y ; sleep 10

onspaces -s llogdbs -p /opt/informix/dbmirror/llogdbs_2 -o 0 -O -y ; sleep 10

onspaces -s appdbs  -p /opt/informix/dbmirror/appdbs_1  -o 0 -O -y ; sleep 10

onspaces -s appdbs  -p /opt/informix/dbmirror/appdbs_2  -o 0 -O -y ; sleep 10

onspaces -s appdbs  -p /opt/informix/dbmirror/appdbs_3  -o 0 -O -y ; sleep 10

onspaces -s bigdbs  -p /opt/informix/dbmirror/bigdbs_1  -o 0 -O -y ; sleep 10

onspaces -s bigdbs  -p /opt/informix/dbmirror/bigdbs_2  -o 0 -O -y ; sleep 10

onspaces -s bigdbs  -p /opt/informix/dbmirror/bigdbs_3  -o 0 -O -y

switch.sh

cd /opt/informix

mv dbspaces dbspaces.old

mv dbmirror dbspaces

mv dbspaces.old dbmirror

mirror_remove.sh

for dbspace in $(ls mirror.*.map | cut -d. -f2)

do onspaces -r $dbspace -y ; sleep 10

done

Note that temp dbspaces were not included above as:

  • they cannot be mirrored;
  • contents are not persisted and would never need migrating;
  • this system holds temp spaces in RAM disk (see article) so did not need relocating.

The new logical volumes were created – and correct chunk ownership enforced as defined in the provided udev rules file (the contents should all be on one line) – on both primary and secondary systems:

sh lvcreate.sh

cp 99-informix.rules /etc/udev/rules.d

udevadm control --reload-rules

The above is specific to Linux and run as "root", whereas all other following steps should work on any Unix variant and are run as "informix".

Symbolic links to the logical volume devices are created next on both systems; Informix mirror chunks are then defined on the primary with "onspaces -m" which automatically replicates the definition on the secondary but only synchronises contents on the primary; contents are then synchronised manually on the secondary with "onspaces -s":

sh link.sh    # both systems

sh mirror.sh  # primary only

sh recover.sh # secondary only

Once synchronisation had completed several hours later (checked with "onstat -d"), the switch of old/new chunks could be performed on the secondary (it's better to verify the process there first rather than on the primary):

onmode -ky    # stop IDS

sh switch.sh  # swap symlinks

oninit -v     # restart IDS

Depending on the IDS version, it may fail to bring online the root dbspace mirror on the secondary due to a defect, generating an assert. If so, recover the down chunk with:

onspaces -s rootdbs -p /opt/informix/dbmirror/rootdbs_1 -o 0 -O -y

During a very short planned maintenance outage on the primary some days later, the switch-over was repeated on the primary with:

onmode -l     # advance the logical log to create a recovery point

onmode -yuck  # stop IDS gracefully after a checkpoint

sh switch.sh  # swap symlinks

oninit -v     # restart IDS

After running for a suitable time to prove the new storage was working correctly, the mirrors were dropped from both servers by running this final command on the primary:

sh mirror_remove.sh

Conclusion

Migrating Informix to new storage can be conveniently and quickly achieved by defining mirror chunks on the new array with "onspaces", waiting for them to be synchronised, and then switching primary and mirror chunks, before eventually dropping the originals.

Downtime while copying the contents is eliminated, and only a very short outage is required to make them the new primary set. Moreover, in a future release, it will be possible to perform that step without no downtime using extended "onspaces" syntax. See this Request for Enhancement we have had accepted:

www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=104825

Disclaimer

Suggestions above are provided "as is" without warranty of any kind, either express or implied, including without limitation any implied warranties of condition, uninterrupted use, merchantability, fitness for a particular purpose, or non-infringement.