FAQ: Broken Virtual Pluggable Database (KBA10373)

What is a broken vPDB?

A virtual pluggable database is broken if it cannot be operated upon from the Delphix Management interface such that operations like Disable, Enable, Refresh or Rewind all fail.

Any operations involving the opening or unplugging of the virtual pluggable database from its container database fail.

Why a vPDB gets into a broken state?

There could be many reasons for a vPDB to get into a broken state such as (not an exhaustive list): 

• Datafile is deleted or corrupted

• Datafile is renamed or moved

• Oracle DB instance crashes and archive logs needed to perform recovery are lost/deleted

These issues can be broadly categorized into two:

1. vPDB datafiles become unavailable on the Delphix mount location

ERROR at line 1:
ORA-65368: unable to open the pluggable database due to errors during recovery
ORA-01110: data file 37:
'/mnt/provision/VCDO_UR1-CDOMLOTG4A38/datafile/u01/app/oracle/oradata/CDOMLOSR38
1E/B79FACE892B3121AE0533DC36E0A86D7/datafile/o1_mf_system_hypxt9c7_.dbf'
ORA-01157: cannot identify/lock data file 37 - see DBWR trace file
ORA-01110: data file 37:
'/mnt/provision/VCDO_UR1-CDOMLOTG4A38/datafile/u01/app/oracle/oradata/CDOMLOSR38
1E/B79FACE892B3121AE0533DC36E0A86D7/datafile/o1_mf_system_hypxt9c7_.dbf'
ORA-01113: file 40 needs media recovery
ORA-01110: data file 40:
'/mnt/provision/VCDO_UR1-CDOMLOTG4A38/datafile/u01/app/oracle/oradata/CDOMLOSR38
1E/B79FACE892B3121AE0533DC36E0A86D7/datafile/o1_mf_users_hypxtoqg_.dbf'
UNPLUG_PDB_ERROR

2. vPDB datafiles need recovery but the necessary logs are not available

ERROR at line 1:
ORA-65368: unable to open the pluggable database due to errors during recovery
ORA-00279: change 4637955 generated at 04/17/2023 08:50:30 needed for thread 1
ORA-00289: suggestion :
/u01/app/oracle/fast_recovery_area/CDOMLOTG4A38/archivelog/2023_04_17/o1_mf_1_15
5_%u_.arc
ORA-00280: change 4637955 for thread 1 is in sequence #155
ORA-00278: log file
'/u01/app/oracle/fast_recovery_area/CDOMLOTG4A38/archivelog/2023_04_17/o1_mf_1_1
55_%u_.arc' no longer needed for this recovery
ORA-00308: cannot open archived log
'/u01/app/oracle/fast_recovery_area/CDOMLOTG4A38/archivelog/2023_04_17/o1_mf_1_1
55_%u_.arc'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 7
ORA-01113: file 48 needs media recovery
ORA-01110: data file 48:
'/mnt/provision/VCDO_PWE-CDOMLOTG4A38/datafile/u01/app/oracle/oradata/CDOMLOSR38
1E/B79FACE892B3121AE0533DC36E0A86D7/datafile/o1_mf_users_hypxtoqg_.dbf'
UNPLUG_PDB_ERROR

How to fix a broken vPDB?

A broken vPDB can be force refreshed or rewound. A force option has been introduced in the OracleRollbackParameters and OracleRefreshParameters in Delphix version 13, for the Rollback and Refresh operations respectively. This option is available in the CLI only.

ip-10-110-255-103 database 'VCDO_A4Z' rollback *> ls
Properties
    type: OracleRollbackParameters
    credential: (unset)
    force: false
    instanceNumber: (unset)
    syncAfterReprovision: true
    timeflowPointParameters:
        type: TimeflowPointSemantic
        container: (unset)
        location: LATEST_POINT
    username: (unset)

How does force parameter work for Refresh/Rewind?

The Refresh/Rewind with force option is similar to a regular Refresh/Rollback with one important distinction – if the force option is set, the virtual pluggable database is dropped including datafiles in its current timeflow if it cannot be unplugged from its container database.

Are there any implications of using the force option?

Since the Refresh/Rollback with force option drops the virtual pluggable database including datafiles in its existing timeflow (before creating the new timeflow for the vPDB), there will be the below implications:

• Undo Refresh or Rewind functionality will not be available 

• Timeflow before refresh/rewind will remain in the broken state, however, any Snapshots in that timeflow will continue to remain provisionable.

• In the case of Rewind, Delphix will not be able to identify the 'last good point in timeflow' for the vPDB.

Related Articles

The following articles may provide more information or related information to this article:

• CLI cookbook: Force refresh/rewind a virtual PDB