Skip to main content
Delphix

Provisioning to Oracle Source Environments May Cause Data Loss (TB032)

Alert Type

Data Availability / Data Loss

Impact

Under specific conditions, provisioning an Oracle virtual database (VDB) back to the same environment as its source database will cause the source database's datafiles and flashback logs to be deleted. Data loss can cause the source database to crash.

Contributing Factors

The issue may occur in the following Delphix Engine Releases:

  • Delphix Engine 3.2.0.0

  • Delphix Engine 3.2.1.0

  • Delphix Engine 3.2.2.0 and 3.2.2.1

  • Delphix Engine 3.2.3.0

  • Delphix Engine 3.2.4.0, 3.2.4.1 and 3.2.4.2

  • Delphix Engine 3.2.5.0 and 3.2.5.1

  • Delphix Engine 3.2.6.0

  • Delphix Engine 3.2.7.0 and 3.2.7.1

  • Delphix Engine 4.0.0.0 and 4.0.0.1

  • Delphix Engine 4.0.1.0

  • Delphix Engine 4.0.2.0

  • Delphix Engine 4.0.3.0

  • Delphix Engine 4.0.4.0

  • Delphix Engine 4.0.5.0

  • Delphix Engine 4.0.6.0 and 4.0.6.1

  • Delphix Engine 4.1.0.0

  • Delphix Engine 4.1.2.0

  • Delphix Engine 4.1.3.0, 4.1.3.1 and 4.1.3.2

  • Delphix Engine 4.1.4.0

  • Delphix Engine 4.1.5.0

  • Delphix Engine 4.1.6.0

  • Delphix Engine 4.2.0.0 and 4.2.0.3

  • Delphix Engine 4.2.1.0 and 4.2.1.1

  • Delphix Engine 4.2.2.0 and 4.2.2.1

  • Delphix Engine 4.2.3.0

  • Delphix Engine 4.2.4.0

  • Delphix Engine 4.2.5.0 and 4.2.5.1

  • Delphix Engine 4.3.1.0

  • Delphix Engine 4.3.2.0 and 4.3.2.1

  • Delphix Engine 4.3.3.0

  • Delphix Engine 4.3.4.0 and 4.3.4.1

  • Delphix Engine 4.3.5.0

  • Delphix Engine 5.0.1.0 and 5.0.1.1

  • Delphix Engine 5.0.2.0, 5.0.2.1, 5.0.2.2, and 5.0.2.3

  • Delphix Engine 5.0.3.0 and 5.0.3.1

Deletion of source database datafiles can only occur when the following conditions are all met:

  • A VDB is provisioned to its source database environment. This form of provisioning can occur:
    • As part of a normal VDB provision or refresh
    • When a dSource's Validated Sync uses the source database environment as a staging environment
    • When converting a dSource to a Live Source and the Live Source is running on the source database environment
  • The snapshot from which the VDB is being provisioned was taken at the same time as one or more new ASM or OMF datafiles were added to the source database

Deletion of source database flashback logs can only occur when the following conditions are met:

  • A VDB is provisioned to its source database environment. This form of provisioning can occur:
    • As part of a normal VDB provision or refresh
    • When a dSource's Validated Sync uses the source database environment as a staging environment
    • When converting a dSource to a Live Source and the Live Source is running on the source database environment
  • The source database is using ASM to store flashback logs

Symptoms

If the source database is actively using the datafiles, the Delphix Engine's attempt to delete the datafiles will fail – there will be no data loss. The Oracle alert log of the provisioned VDB will contain the following:

alter database create datafile 1050 as '/mnt/provision/Valo_362/datafile/1050_27.dbf'
WARNING: Cannot delete Oracle managed file +BIGDATA/alone/datafile/test2.18069.907675005
Completed: alter database create datafile 1050 as '/mnt/provision/Valo_362/datafile/1050_27.dbf'
alter database create datafile 1051 as '/mnt/provision/Valo_362/datafile/1051_28.dbf'
WARNING: Cannot delete Oracle managed file +BIGDATA/alone/datafile/test2.18068.907675009
Completed: alter database create datafile 1051 as '/mnt/provision/Valo_362/datafile/1051_28.dbf'
alter database create datafile 1052 as '/mnt/provision/Valo_362/datafile/1052_29.dbf'
Mon Mar 28 12:29:20 2016
WARNING: Cannot delete Oracle managed file +BIGDATA/alone/datafile/test2.18067.907675123
Completed: alter database create datafile 1052 as '/mnt/provision/Valo_362/datafile/1052_29.dbf'

If the source database is not actively using the datafiles, the Delphix Engine's attempt to delete the datafiles will succeed – there will be data loss. The Oracle alert log of the provisioned VDB will contain the following:

alter database create datafile 1050 as '/mnt/provision/Valo_9C1/datafile/1050_27.dbf'
Deleted Oracle managed file +BIGDATA/alone/datafile/test2.18069.907675005
Completed: alter database create datafile 1050 as '/mnt/provision/Valo_9C1/datafile/1050_27.dbf'
alter database create datafile 1051 as '/mnt/provision/Valo_9C1/datafile/1051_28.dbf'
Deleted Oracle managed file +BIGDATA/alone/datafile/test2.18068.907675009
Completed: alter database create datafile 1051 as '/mnt/provision/Valo_9C1/datafile/1051_28.dbf'
alter database create datafile 1052 as '/mnt/provision/Valo_9C1/datafile/1052_29.dbf'

If the source database's datafiles are deleted, it will fail to start due to missing datafile errors like the following:

ORA-01157: cannot identify/lock data file 1050 - see DBWR trace file
ORA-01110: data file 1050: '+BIGDATA/alone/datafile/test2.18069.907675005'

If the source database's flashback logs are deleted, the Oracle alert log of the provisioned VDB will contain the following:

alter database flashback off
Deleted Oracle managed file +BIGDATA/alone/flashback/log_1.30227.888670171
Deleted Oracle managed file +BIGDATA/alone/flashback/log_3.29398.888674221
Flashback Database Disabled 
Completed: alter database flashback off

Relief/Workaround

  • Do not provision Oracle VDBs to the same environment as their source database.
    If you are currently running a VDB on the same environment as the VDB source database and you plan to refresh your VDB in the future, migrate the VDB to another target environment.
  • Ensure environments containing source databases are not being used as staging environments for Validated Sync.
  • Ensure environments containing source databases are not being used as staging environments for Live Sources.
    If you currently have a Live Source on an environment with its source database, migrate the Live Source to another environment.

Resolution

This issue is fully resolved in Delphix Engine 5.0.4.0, and later releases.