DB_SYNC (SnapSync) During VDB Provision With Custom db_domain Parameter Fails With Internal Error (KBA6402)

Issue

A VDB Provision automatically initiates a DB_SYNC (SnapSync) when the provision is complete and the dataset is online.  In some instances, an incorrect db_domain entry will cause the SnapSync to fail with an "internal error" and directs the user to contact Delphix Support.

In this instance, the dataset will  be accessible on the target environment. However, if the VDB is Disabled, a subsequent Enable will also fail with an internal error.

Applicable Delphix Versions

Click here to view the versions of the Delphix engine to which this article applies

Major Release	All Sub Releases
6.0	6.0.0.0, 6.0.1.0, 6.0.1.1, 6.0.2.0, 6.0.3.0, 6.0.3.1
5.3	5.3.0.0, 5.3.0.1, 5.3.0.2, 5.3.0.3, 5.3.1.0, 5.3.1.1, 5.3.1.2, 5.3.2.0, 5.3.3.0, 5.3.3.1, 5.3.4.0, 5.3.5.0 5.3.6.0, 5.3.7.0, 5.3.7.1, 5.3.8.0, 5.3.8.1, 5.3.9.0
5.2	5.2.2.0, 5.2.2.1, 5.2.3.0, 5.2.4.0, 5.2.5.0, 5.2.5.1, 5.2.6.0, 5.2.6.1
5.1	5.1.0.0, 5.1.1.0, 5.1.2.0, 5.1.3.0, 5.1.4.0, 5.1.5.0, 5.1.5.1, 5.1.6.0, 5.1.7.0, 5.1.8.0, 5.1.8.1, 5.1.9.0, 5.1.10.0
5.0	5.0.1.0, 5.0.1.1, 5.0.2.0, 5.0.2.1, 5.0.2.2, 5.0.2.3, 5.0.3.0, 5.0.3.1, 5.0.4.0, 5.0.4.1 ,5.0.5.0, 5.0.5.1, 5.0.5.2, 5.0.5.3, 5.0.5.4
4.3	4.3.1.0, 4.3.2.0, 4.3.2.1, 4.3.3.0, 4.3.4.0, 4.3.4.1, 4.3.5.0
4.2	4.2.0.0, 4.2.0.3, 4.2.1.0, 4.2.1.1, 4.2.2.0, 4.2.2.1, 4.2.3.0, 4.2.4.0 , 4.2.5.0, 4.2.5.1
4.1	4.1.0.0, 4.1.2.0, 4.1.3.0, 4.1.3.1, 4.1.3.2, 4.1.4.0, 4.1.5.0, 4.1.6.0

Resolution

This issue is ultimately caused by an invalid extra character in the db_domain parameter, causing an invalid service_name to be built in the JDBC connection string for the VDB.  In this example, the VDB named "failtest" was configured with the db_domain parameter incorrectly entered with a comma appended.

To work around this issue, the config parameter must be edited via CLI, and then the target environment must be refreshed. This will force the JDBC connection string to be updated with the corrected parameter, at which point VDB operations should be successful.

In the following example, we see the incorrect db_domain value and correct it.

DelphixEngine> /source; select failtest
DelphixEngine source 'failtest'> ls
Properties
    type: OracleVirtualSource
    name: failtest
    allowAutoVDBRestartOnHostReboot: false
    archivelogMode: true
    config: failtest
    configParams:
        compatible: '19.0.0'
        db_domain: 'delphix.com',
<edit>
DelphixEngine source 'failtest'> update
DelphixEngine source 'failtest' update *> set configParams.db_domain='delphix.com'
DelphixEngine source 'failtest' update *> commit
DelphixEngine source 'failtest'> ls
Properties
    type: OracleVirtualSource
    name: failtest
    allowAutoVDBRestartOnHostReboot: false
    archivelogMode: true
    config: failtest
    configParams:
        compatible: '19.0.0'
        db_domain: 'delphix.com' <<<<<<<<<<<<<<<<<<<<

Troubleshooting

Review the VDB configuration parameters by either reviewing the applicable VDB configuration template in the GUI or viewing the VDB configuration parameters via CLI. Examples of both are provided below.

GUI:

CLI:

DelphixEngine> source
DelphixEngine source> select failtest
DelphixEngine source 'failtest'> ls
Properties
    type: OracleVirtualSource
    name: failtest
    allowAutoVDBRestartOnHostReboot: false
    archivelogMode: true
    config: failtest
    configParams:
        compatible: '19.0.0'
        db_domain: 'delphix.com',    <<<<<<<<<<<<<<<<<<<<
        log_archive_format: '%t_%s_%r.dbf'
        memory_max_target: 1073741824
        memory_target: 1073741824
        nls_language: 'AMERICAN'
        nls_territory: 'AMERICA'
        open_cursors: 300
        processes: 300
        remote_login_passwordfile: 'EXCLUSIVE'
    configTemplate: (unset)
    container: seantest
    customEnvVars: (empty)
    linked: false
    logCollectionEnabled: false
    mountBase: /mnt/provision
    nodeListeners: (empty)
    operations:
        type: VirtualSourceOperations
        configureClone: (empty)
        postRefresh: (empty)
        postRollback: (empty)
        postSnapshot: (empty)
        postStart: (empty)
        postStop: (empty)
        preRefresh: (empty)
        preRollback: (empty)
        preSnapshot: (empty)
        preStart: (empty)
        preStop: (empty)
    redoLogGroups: 3
    redoLogSizeInMB: 1024
    reference: ORACLE_VIRTUAL_SOURCE-2
    runtime:
        type: OracleSourceRuntime
        accessible: false
        accessibleTimestamp: 2020-09-05T14:14:00.716Z
        activeInstances: (empty)
        bctEnabled: false
        databaseSize: 0B
        databaseStats: (empty)
        dnfsEnabled: false
        enabled: ENABLED
        notAccessibleReason: Environment monitor has not yet retrieved attributes for object "seantest".
        status: CHECKING
    runtimeMountInformation:
        type: UnixRuntimeMountInformation
        name: (unset)
        nfsVersion: 3
        nfsVersionReason: TUNABLE_OVERRIDE
    staging: false
    status: DEFAULT
    virtual: true