Just recently (31 December 2012) the first Patch Bundle (MOS patch name 13539664 – LNX64-112-CMT: PLACE HOLDER BUG FOR OAK PATCH 2.1.0.1.0) for the ODA was released. This Patch Bundle only contains firmware updates for the shared disks (HDD) and a new version of OAK (Oracle Appliance Kit).
Because the only information about patching the ODA came from the documenation library (where it is stated that rolling-updates are not supported) I was curious what this rather small patch (no Oracle software patches, only firmware updates and an new release of OAK) would impact the availability of the complete ODA system. In this post I will share my experience with this first ODA patch bundle.
Starting the patching process on the first node answered the question about the system availability within a couple of seconds: it started stopping the complete Grid cluster on the ODA! (So availability during patch installation – at least during patching of node 1 – was 0%).
Procedure for installing ODA PatchBundle (2.1.0.1.0)
We used the following procedure to install the ODA PatchBundle 2.1.0.1.0.
- Copy the PatchBundle file from MOS (p13539664_21000_Linux-x86-64.zip) to the /tmp of both ODA nodes (SC’s)
- (As root!) Unpack this PatchBundle on both ODA nodes (SC’s) using:
/opt/oracle/oak/bin/oakcli unpack -package /tmp/p13539664_21000_Linux-x86-64.zip - (As root!) Install the patch on ODA node 1 (SC0):
/opt/oracle/oak/bin/oakcli update -patch 2.1.0.1.0 - (As root – after reboot of ODA node 1) Install the patch on ODA node 2 (SC1):
/opt/oracle/oak/bin/oakcli update -patch 2.1.0.1.0 - (As root) Reboot ODA node 2 (SC1):
reboot
Duration
Because all Oracle Grid resources on the ODA are stopped, the time the complete the complete patching procedure is important. Of course the installation time described below is bases on a installation with just 1 Oracle RAC database running on with and no session logged in on the database. For a situation with multiple databases running and users/applications connected to these databases, stopping all Grid resources could take a lot longer (and of course also starting up all resources again).
On Node 1
The patch installation brings down the Oracle Grid cluster (all resources including a small sample RAC database) on both nodes and update the firmware of the shared HDD disks and the new version of OAK. After that the server automtically started rebooting the server.
The total time it took for patching ODA node 1 (SC0) until the RAC database instance on node 1 was opened (so including rebooting of the server and starting the GI again) took around 21 minutes and 30 seconds (21:30).
On Node 2
After all resources on node 1 where started (so the database was available again) we started the patching procedure on node 2. The patching process discovered that the firmware for the shared HDD disks was already updated, so it only needed to install the patch for OAK.
It was nice to see that it didn’t bring down the Oracle Grid resources (running on node 1) again because it determined that the firmware for the disks was already up to date. It only updated its local inventory with this updated information.
One strange thing was that after the patch procedure was completed, it didn’t mention to reboot the server. Although it wasn’t explicitly mentioned I decided to reboot it anyway because it also did so on the first node.
The total time it took installing the patch on ODA node 2 (SC1) until the RAC database instance on node 2 was opened, was 12 minutes and 50 seconds (12:50).
In total (so patching the complete ODA, including reboots and having an opened Oracle RAC database) took around 35 minutes.
The patch installation creates a logfile (on each ODA node): /opt/oracle/oak/pkgrepos/System/2.1.0.1.0/bin/tmp/oak-oakpatch-<hostname>–<date/time>.out
Conclusion
It’s nice to finally have a grasp of how the “one-command” patching of an ODA actually works. Lucky for us we don’t actually run production database on the ODA’s yet so the unavailability of the database wasn’t a problem.
What worries me with this PatchBundle kind of “one-command” kind of patching is that you cannot tell how long the databases running on an ODA are unavailable because it totally depends on what compontents get patches (this patch only contained HDD firmware updates and the small OAK package, but what if there are also Oracle software patches included – and the ODA documentation tells us that these won’t be rolling!). I’m afraid we have to wait until the next PatchBundle and see what’s in there……