Troubleshooting Oracle VCDB snapshots taking over fifteen minutes and reporting missing archivelogs
Delphix currently (as of 126.96.36.199) ensures that a Virtual Container Database (VCDB) and Virtual Pluggable Database (VPDB) are provisioned with log_archive_dest_X database parameters (typically log_archive_dest_1) being set to a Delphix supplied NFS mounted file system. During the provision process Delphix allows the setting of log_archive_dest_X to be another non-Delphix file system, however Delphix will establish a second destination and link this to the Delphix file system. This results in the VCDB creating archivelogs in both locations.
The Database Administrator of the VCDB can disable archiving to the Delphix mount point by altering the relevant log_archive_dest_X parameter. This results in archiving occurring into the non-Delphix file system only. This is currently not a supported configuration.
Having the VCDB archivelog creation configured in this way results in snapshots being created for the VPDB that cannot be provisioned from as archivelogs are reported as missing for the snapshot.
This knowledge base article examines what is put in place to produce this behaviour and how you overcome the missing logs reported for the snapshot and more importantly how you avoid the missing logs and 15 minute snapshot times altogether.
Applicable Delphix Versions
- Click here to view the versions of the Delphix engine to which this article applies
Major Release All Sub Releases 6.0 188.8.131.52, 184.108.40.206, 220.127.116.11, 18.104.22.168, 22.214.171.124, 126.96.36.199, 188.8.131.52, 184.108.40.206, 220.127.116.11, 18.104.22.168, 22.214.171.124, 126.96.36.199, 188.8.131.52, 184.108.40.206, 220.127.116.11, 18.104.22.168, 22.214.171.124
126.96.36.199, 188.8.131.52, 184.108.40.206, 220.127.116.11, 18.104.22.168, 22.214.171.124, 126.96.36.199, 188.8.131.52, 184.108.40.206, 220.127.116.11, 18.104.22.168, 22.214.171.124, 126.96.36.199, 188.8.131.52, 184.108.40.206, 220.127.116.11, 18.104.22.168, 22.214.171.124
126.96.36.199, 188.8.131.52, 184.108.40.206, 220.127.116.11, 18.104.22.168, 22.214.171.124, 126.96.36.199, 188.8.131.52
184.108.40.206, 220.127.116.11, 18.104.22.168, 22.214.171.124, 126.96.36.199, 188.8.131.52, 184.108.40.206, 220.127.116.11, 18.104.22.168, 22.214.171.124, 126.96.36.199, 188.8.131.52, 184.108.40.206
220.127.116.11, 18.104.22.168, 22.214.171.124, 126.96.36.199, 188.8.131.52, 184.108.40.206, 220.127.116.11, 18.104.22.168, 22.214.171.124, 126.96.36.199, 188.8.131.52, 184.108.40.206, 220.127.116.11, 18.104.22.168, 22.214.171.124
The Oracle Database Administrator (DBA) has made changes to the Oracle VCDB parameters influencing the creation of archivelogs and stopped archive logs from being created in the Delphix NFS mounted file system typically used to hold these logs. The DBA has set the creation of archivelogs to be a file system location outside of Delphix supplied mount points.
The following changes in a VCDB/VPDB pair will lead to the behavior appearing.
- Enable creation of archive logs to a non-Delphix file system location (if it wasn't set already during the provision process).
[oracle@oel7sitde2 ~]$ mkdir -p /u01/app/oracle/archive [oracle@oel7sitde2 ~]$ sqlplus / as sysdba SQL> alter system set log_archive_dest_2='location=/u01/app/oracle/archive/ MANDATORY' scope=spfile sid='*'; System altered.
- Disable archiving in the VCDB to the NFS mount point and enabled archiving in a non-Delphix based file system. The reset command is one way to disable this archiving, there are others not documented here, like editing parameter files and removing the values. In this case the Delphix location was /mnt/provision/vcdbarc1/archive. This location is removed as an archive location for the VDB.
SQL> show parameter log_archive_dest_1 NAME TYPE VALUE ------------------------------------ ----------- ------------------------------ log_archive_dest_1 string location=/mnt/provision/vcdbarc1/archive/ MANDATORY SQL> alter system reset log_archive_dest_1; System altered.
- Once this configuration is in place a subsequent snaspsync will look to have hung and no progress will be seen after initiating log sync for the VPDB. It will wait until the snapshot timeout is reached at around 15 minutes.
- The Job will eventually complete, however it will take a little over fifteen minutes to do so.
- Looking at the snapshot just captured for the VPDB, it will report the snapshot is not provisionable.
- A warning is also produced regarding logsync being unable to fetch the archivelogs from the VCDB.
There are two parts to resolving this behaviour permanently in VCDB's.
TO RESOLVE THE MISSING ARCHIVELOGS reported for the snapshots reporting archivelogs missing the following process is to be performed for each snapshot.
- The snapshot(s) captured where the warning around missing archivelogs is reported cannot be provisioned from in this state and the timeflow repair tool (Repair Missing Archivelogs) must be used to retrieve the missing logs from the target hosts file system.
- This will pull the missing logs from the target host into the engine and clear the missing logs error.
TO RESOLVE THE ONGOING SNAPSHOT PROBLEMS that result from configuring a VCDB to archive logs only to a file system location that is not a Delphix mount point establishing an archivelog destination that resides in Delphix mount points must be performed.
- Log into the VCDB invoking the SYSDBA privilege
sqlplus / as sysdba
- Execute the parameter change to set an archivelog destination to be the original archive location supplied for the VCDB when it was created.
SQL> alter system set log_archive_dest_1='location=/mnt/provision/VCDBARC1/archive/ MANDATORY';