In some circumstances, the Delphix Engine may raise the following issue as a Fault:
Title: Cannot provision a database from a portion of TimeFlow
Details (May vary): Database [Database Name] TimeFlow [TimeFlow Name] has missing logs because SnapSync forced archive logs [sequence range] to be skipped to make progress on a snapshot. Portions of the TimeFlow are unprovisionable.
Action: Use the CLI TimeFlow Log Fetch command to retrieve the missing logs. The logs can only be retrieved from a non-ASM location.
This warning indicates that Delphix has not been able to download all of the Archive Logs required to perform point-in-time (SCN-based) provisioning from an Oracle dSource.
Please note that this error will affect point-in-time provisioning but does not affect provisioning from a Snapshot. Issues which prevent provisioning from a Snapshot have a different, more severe error message, and such issues can be resolved from the Delphix web interface. More information on this error is available in the Knowledge Base article KB091 How To Resolve "Error: Archive Logs Cannot Be Retrieved Due To Deletion".
The "Details" section of the Fault provides more information on the specific cause:
- SnapSync forced archive logs [sequence range] to be skipped to make progress on a snapshot: This fault may occur following a SnapSync / Snapshot operation. During generation of a Snapshot, Delphix will prioritise the collection of the archive logs needed to allow the new Snapshot to be provisioned. When it "goes back" to collect the historic archive logs, they may no longer be present. This is most likely if the dSource has a short retention duration for Archive Logs.
Missing part of log sequence [sequence] because it has been deleted at the source: This occurs when an archive log has been removed from the dSource before it can be collected by Delphix. This is most likely to occur if the dSource is configured very short Archive Log retention or experiences an unusually large burst of activity.
Missing log sequence [sequence] because LogSync failed to fetch it more than [count] times: This fault suggests that the Delphix Connector was unable to send the logs to the Delphix Engine. This may indicate a temporary problem with the Source system or network.
The ability to confirm which archive log sequences are missing is through the Delphix Command Line Interface (CLI), and then using the TimeFlow Name from the Fault details, as follows:
myengine> cd timeflow myengine timeflow> cd oracle myengine timeflow oracle> cd log myengine timeflow oracle log> list timeflow='TimeFlow Name' missing=true CONTAINER TIMEFLOW INSTANCENUM SEQUENCE STARTSCN ENDSCN Database Name 'TimeFlow Name' 1 12345 123456789000 123456789050 Database Name 'TimeFlow Name' 1 12346 123456789050 123456789080 Database Name 'TimeFlow Name' 1 12348 123456789080 123456789154
This information can be used in combination with RMAN or with the assistance of the Database Administrators to identify and locate the missing log files.
There are two ways to resolve this issue - marking the issue as resolved or repairing the TimeFlow. There are also further steps that can be taken to minimise the risk of this issue occurring again.
Option 1: Marking the issue as resolved
Only do this if there is not a need to perform point-in-time recovery of this dSource for the missing timeframe, and then mark this Fault as 'Resolved' using the "Mark Resolved" button. Once the fault is marked as 'Resolved' there will still be no way to provision from the range in question.
Option 2: Repairing the TimeFlow
To perform this operation, first locate the "missing" Archive Logs and make them available on another server where the Delphix Engine can retrieve them via SSH/SCP.
Once these Archive Logs are available, follow the steps in the document CLI Cookbook: Repairing a TimeFlow.
Preventing future issues
The following changes may improve LogSync performance and reduce the risk of this issue reoccurring:
- Enabling the use of Block Change Tracking (as discussed in our Requirements document), on the dSource if it is not already enabled
- Increasing the retention of Archive Logs on the source system
- Increasing the available storage and network bandwidth for the dSource and Delphix Engine (in some cases, the Source database may not have enough network or storage bandwidth to supply log files to the Delphix Engine in a timely manner)
- Modifying the Sync Options to ensure that they are optimal for the dSource
For more information on steps described above, please refer to the following documents: