Skip to main content
Delphix

Troubleshooting "Bequeath connection to database 'XYZ' failed. TCP fallback connection is active." (KBA8428)

 

KBA

KBA# 8428

Applicable Delphix Versions

Click here to view the versions of the Delphix engine to which this article applies
Major Release All Sub Releases
6.0 6.0.7.0, 6.0.8.0, 6.0.8.1, 6.0.9.0, 6.0.10.0, 6.0.10.1, 6.0.11.0

Troubleshooting the "Bequeath connection to database 'XYZ' failed. TCP fallback connection is active." fault

Delphix Engine version 6.0.7.0 introduced a new featured Simplified Connection Management ( SCM ). This feature changes how all Delphix processes connect to an Oracle dSource or VDB to query the database.

Prior this release, Delphix would connect to the target database using a TCP connection, through the Oracle Listener, using a supplied username and password. The connection would either be initiated on the local host or from the Delphix engine, depending on the specific Delphix version and connection purpose.

This method of communication has several disadvantages. Consider the following issues: network connectivity needs to be opened between the Delphix and the Oracle hosts on any listener ports; limited support for advanced connection methods ( such as TCPS ); username and passwords must be added to Delphix and kept in sync.

From 6.0.7.0, all connections to the target database will be attempted using SCM first. With an SCM connection, the Delphix client process running on the Oracle host will initiate a connection directly to the database using the Oracle Bequeath method and OS authentication. This greatly simplifies deploying Delphix into environments: listener ports no longer need to be opened; Delphix only requires an OS login; Delphix no longer also requires a specific DB user.

When making an SCM connection, Delphix must use various libraries in the Oracle Home to which it is being connected. This increases the dependency Delphix has on the home having the correct library versions installed and permissions. In most cases these will be accurate, however in cases where the home has the incorrect permissions or has missing/invalid libraries this will result in SCM failing to connect to the database. 

In cases where an SCM connection fails, Delphix attempts to fallback to using a listener based TCP connection, initiated from the Oracle host, if a username/password has been previously configured. When a fallback occurs, the following fault will be raised

Oracle TCP fallback has been engaged. Nov 2, 2021 7:38 PM
Bequeath connection to database 'XYZ' failed. TCP fallback connection is active.
User Action
TCP connections are deprecated and will be removed in a future release.
Investigate and resolve the cause of the bequeath connection failure.

Prerequisites 

Delphix Version 6.0.7.0 or above.

In the examples shown in this article, the Oracle binaries are owned by the user "oracle", and the environment has been added to Delphix using the OS user "dlpxqa".

 

Resolution

Delphix maintains a pool of connections to the target databases in the client process running on the Oracle host. When attempting to resolve issues with SCM, it may be necessary to force the connection pool to be refreshed. This can be done by killing the Delphix client process. This should only be performed when there are no currently running actions (such as SnapSync, Provision, et cetera) on the host. If this is done while actions are running, the actions will fail.

This can be performed using the following command: 

ps -ef | grep 'Delphix_COMMON' | grep 'java' | grep 'client.jar' | awk '{print $2}' | xargs kill

When attempting to make an SCM connection, the Delphix client process loads various libraries from the Oracle Home. Most failures result from either the OS user not having access to these libraries or the incorrect versions of these libraries existing in the home. The following files are required.

$ORACLE_HOME/lib/libclntsh*
$ORACLE_HOME/lib/libocijdbc*.so
$ORACLE_HOME/jdbc/lib/*

The Delphix OS user must be able to cd to each of these directories, and read the files listed above. The default permissions when Oracle is installed allow this, however incorrect Oracle Home cloning or incomplete installations may result in incorrect permissions.

The following provide some examples of how these files may have the incorrect permissions set, and their correct settings.

[dlpxqa@mw-scm-tgt ~]$ ls -lad $ORACLE_HOME/lib
drwxr-xr-x. 4 oracle oinstall 12288 May 10  2021 /u01/app/oracle/product/19.11.0.0/dbhome_1/lib

$ORACLE_HOME/lib must be executable and readable by the Delphix OS user, The default permission is 0755.

[dlpxqa@mw-scm-tgt ~]$ ls -la $ORACLE_HOME/lib/libclntsh*
lrwxrwxrwx. 1 oracle oinstall       21 May 10  2021 /u01/app/oracle/product/19.11.0.0/dbhome_1/lib/libclntshcore.so -> libclntshcore.so.19.1
-rwxr-xr-x. 1 oracle oinstall  8052984 May 10  2021 /u01/app/oracle/product/19.11.0.0/dbhome_1/lib/libclntshcore.so.19.1
lrwxrwxrwx. 1 oracle oinstall       17 May 10  2021 /u01/app/oracle/product/19.11.0.0/dbhome_1/lib/libclntsh.so -> libclntsh.so.19.1
lrwxrwxrwx. 1 oracle oinstall       12 May 10  2021 /u01/app/oracle/product/19.11.0.0/dbhome_1/lib/libclntsh.so.10.1 -> libclntsh.so
lrwxrwxrwx. 1 oracle oinstall       12 May 10  2021 /u01/app/oracle/product/19.11.0.0/dbhome_1/lib/libclntsh.so.11.1 -> libclntsh.so
lrwxrwxrwx. 1 oracle oinstall       12 May 10  2021 /u01/app/oracle/product/19.11.0.0/dbhome_1/lib/libclntsh.so.12.1 -> libclntsh.so
lrwxrwxrwx. 1 oracle oinstall       12 May 10  2021 /u01/app/oracle/product/19.11.0.0/dbhome_1/lib/libclntsh.so.18.1 -> libclntsh.so
-rwxr-xr-x. 1 oracle oinstall 81393744 May 10  2021 /u01/app/oracle/product/19.11.0.0/dbhome_1/lib/libclntsh.so.19.1

$ORACLE_HOME/lib/libclntsh.so and its associated files must be executable and readable by the Delphix OS user, The default permission is 0755 for library files and 0777 for symlinks.

[dlpxqa@mw-scm-tgt ~]$ ls -la $ORACLE_HOME/lib/libocijdbc*.so | egrep "libocijdbc[0-9]+.so"
-rw-r--r--. 1 oracle oinstall  153448 Apr 20  2021 /u01/app/oracle/product/19.11.0.0/dbhome_1/lib/libocijdbc19.so

$ORACLE_HOME/lib/libocijdbcXX.so must be readable by the Delphix OS user. The default permission is 0644.

[dlpxqa@mw-scm-tgt ~]$ ls -lad $ORACLE_HOME/jdbc/lib
drwxr-xr-x. 2 oracle oinstall 195 Apr 26  2021 /u01/app/oracle/product/19.11.0.0/dbhome_1/jdbc/lib
[dlpxqa@mw-scm-tgt ~]$ ls -la $ORACLE_HOME/jdbc/lib/*
-rw-r--r--. 1 oracle oinstall 7493113 Apr 20  2021 /u01/app/oracle/product/19.11.0.0/dbhome_1/jdbc/lib/ojdbc10dms_g.jar
-rw-r--r--. 1 oracle oinstall 6195946 Apr 20  2021 /u01/app/oracle/product/19.11.0.0/dbhome_1/jdbc/lib/ojdbc10dms.jar
-rw-r--r--. 1 oracle oinstall 7463495 Apr 20  2021 /u01/app/oracle/product/19.11.0.0/dbhome_1/jdbc/lib/ojdbc10_g.jar
-rw-r--r--. 1 oracle oinstall 4441302 Apr 20  2021 /u01/app/oracle/product/19.11.0.0/dbhome_1/jdbc/lib/ojdbc10.jar
-rw-r--r--. 1 oracle oinstall 7449288 Apr 20  2021 /u01/app/oracle/product/19.11.0.0/dbhome_1/jdbc/lib/ojdbc8dms_g.jar
-rw-r--r--. 1 oracle oinstall 6160100 Apr 20  2021 /u01/app/oracle/product/19.11.0.0/dbhome_1/jdbc/lib/ojdbc8dms.jar
-rw-r--r--. 1 oracle oinstall 7419739 Apr 20  2021 /u01/app/oracle/product/19.11.0.0/dbhome_1/jdbc/lib/ojdbc8_g.jar
-rw-r--r--. 1 oracle oinstall 4408366 Apr 20  2021 /u01/app/oracle/product/19.11.0.0/dbhome_1/jdbc/lib/ojdbc8.jar
-rw-r--r--. 1 oracle oinstall   29207 Apr 20  2021 /u01/app/oracle/product/19.11.0.0/dbhome_1/jdbc/lib/simplefan.jar

Delphix uses an OJDBC library from $ORACLE_HOME/jdbc/lib/ , this directory must be executable and readable by the Delphix OS user, The default permission is 0755. Delphix will choose the best library based on the database version and available ojdbc libraries. All libraries in this directory should be readable by the Delphix OS user, the default permission is 0644.

Further assistance


If the issue fault persists after verifying and correcting any permissions issues, then a support case should be raised, and include