Skip to main content
Delphix

Masking Error "Connection Timed Out" (KBA1855)

  1. Last updated
  2. Save as PDF
 

 

KBA

KBA#1855

 

At a Glance   

Description This article provides a troubleshooting guide for the JDBC error Connection timeout and Connection Timed Out.
Root cause The error is thrown when the existing connection to the database has not received a response message within the time specified (this is hardcoded in JDBC).  Consequently, this will cause the job to fail and any further attempts will result in an error.
 

"Connection timeout" errors are often symptoms of underlying network issues or firewall rules blocking packets. If you encounter frequent timeouts, it's essential to collaborate with your network administrators to identify and resolve the underlying problem.
Where This error can appear anytime a connection is used - such as Test Connection, creating Rule Set, or during job execution. This can also happen in the browser session or when connecting to a service (such as LDAP). 

This KBA is specifically looking at errors during masking job executions.
Troubleshooting
Steps		 The main goal here is to find what caused the connection to time out and why? This will then define what actions are needed (resource change, network change, or other) to resolve the issue.

The steps below are only suggestions on where to start investigating:
1. Jobs The main step on the Masking Engine is to find the time of the event. It might also be that other jobs or connections have Connection errors.
  • Check timestamps - when the connection was up and when there were connection errors. 
  • Are there any other jobs that have Connection errors?
2. Scripts Verify scripts that start and check status are not timing out (and as a result causing the database or listener to drop). Scripts should be written to keep the (i.e., API) connection alive.
3. Network, Proxy, DNS This issue can be caused by the Network, including Proxy and DNS, where packet loss, routing issue or latencies can cause the JDBC connections to Time out. 
  • Check the DNS or try using the IP address directly.
  • Talk to your network administrators and check relevant logs. 
4. Firewall or Intrusion Detection The Firewall, Intrusion Detection (IDS) or Anti-Virus can have rules defined that will drop (even silently) packets and cause the JDBC connection to Time out.
  • Talk to your network administrators and check the relevant logs.
5. Host

It is possible that this error is caused by changes on the hosting platform (such as ESX) or another external host or service (also known as Noisy Neighbor).

  • Check resource saturation on hosts - such as memory, CPU, storage, ...
More Info More info on how to access logs: 

Other Connection errors:

Applicable Delphix Versions

Click here to view the versions of the Delphix engine to which this article applies
Date Release
Jan 25, 2024 19.0.0.0
Dec 20, 2023 | Jan 10, 2024 18.0.0.0 | 18.0.0.1
Nov 21, 2023 17.0.0.0
Oct 18, 2023 16.0.0.0
Sep 21, 2023 15.0.0.0
Aug 24, 2023 14.0.0.0
Jul 24, 2023 13.0.0.0
Jun 21, 2023 12.0.0.0
May 25, 2023 11.0.0.0
Apr 13, 2023 10.0.0.0 | 10.0.0.1
Mar 13, 2023 | Mar 20, 2023 9.0.0.0 | 9.0.0.1
Feb 13, 2023 8.0.0.0
Jan 12, 2023 7.0.0.0
Releases Prior to 2023
Major Release All Sub Releases
6.0

6.0.0.0, 6.0.1.0, 6.0.1.1, 6.0.2.0, 6.0.2.1, 6.0.3.0, 6.0.3.1, 6.0.4.0, 6.0.4.1, 6.0.4.2, 6.0.5.0, 6.0.6.0, 6.0.6.1, 6.0.7.0, 6.0.8.0, 6.0.8.1, 6.0.9.0, 6.0.10.0, 6.0.10.1, 6.0.11.0, 6.0.12.0, 6.0.12.1, 6.0.13.0, 6.0.13.1, 6.0.14.0, 6.0.15.0, 6.0.16.0, 6.0.17.0, 6.0.17.1, 6.0.17.2

5.3

 5.3.0.0, 5.3.0.1, 5.3.0.2, 5.3.0.3, 5.3.1.0, 5.3.1.1, 5.3.1.2, 5.3.2.0, 5.3.3.0, 5.3.3.1, 5.3.4.0, 5.3.5.0, 5.3.6.0, 5.3.7.0, 5.3.7.1, 5.3.8.0, 5.3.8.1, 5.3.9.0

5.2

5.2.2.0, 5.2.2.1, 5.2.3.0, 5.2.4.0, 5.2.5.0, 5.2.5.1, 5.2.6.0, 5.2.6.1

5.1

5.1.0.0, 5.1.1.0, 5.1.2.0, 5.1.3.0, 5.1.4.0, 5.1.5.0, 5.1.5.1, 5.1.6.0, 5.1.7.0, 5.1.8.0, 5.1.8.1, 5.1.9.0, 5.1.10.0

5.0

5.0.1.0, 5.0.1.1, 5.0.2.0, 5.0.2.1, 5.0.2.2, 5.0.2.3, 5.0.3.0, 5.0.3.1, 5.0.4.0, 5.0.4.1, 5.0.5.0, 5.0.5.1, 5.0.5.2, 5.0.5.3, 5.0.5.4

Troubleshooting Connection timed out

Connection timeout and Connection timed out errors are issues caused on the other side of the connection. The investigation should therefore start at the systems connected to the Masking Engine that can be the point of failure - these systems are outside the scope of this article. 

The details here will focus on getting the relevant details from the Masking Engine.

Analyzing Logs

This section shows how to use grep (Linux) (or using Select-String for Windows) to get more details from the Masking Engine logs.

The logs can be downloaded from the Job Monitor, from Admin > Logs, or using the API. For more details on how to access these, see KBA7988 listed above.

Search terms

In order to find the details in the logs, we need to look at all errors in the job. Search for (some modifications might be required based on the connector type and version):

  • E=1
  • IO Error
  • Connection timed out
Linux example

Below is an example using grep:

grep -E "E=1|IO Error|Connection timed out|Next Exception" logfile.log
Window example 

Below is an example using Select-String:

Select-String "E=1|IO Error|Connection timed out|Next Exception" logfile.log

Example Output

The commands above should return an output similar to (this example is masking an Oracle database):

[JOB_ID_n_m] 2023/07/15 14:14:33 - DelphixTableUpdate.0 - IO Error: Connection timed out
[JOB_ID_n_m] 2023/07/15 14:14:33 - DelphixTableUpdate.0 - Next Exception: SQLState( 08006) ErrorCode(17002)
[JOB_ID_n_m] 2023/07/15 14:14:33 - DelphixTableUpdate.0 - java.sql.BatchUpdateException: IO Error: Connection timed out
[JOB_ID_n_m] 2023/07/15 14:14:33 - DelphixTableUpdate.0 - Finished processing (I=18370000, O=0, R=18270000, W=18269999, U=18269999, E=1)

 

More about the logs

We will look at the logs in more detail in this section. The log output might differ and there might be other job steps that have 'Connection timed out' (such as a Custom Algorithm which has a connection to a database).

Salient entries in the log:

  • The key entry is Connection timed out
    • In this example, the step DelphixTableUpdate has failed.  
       
  • Notes around step: DelphixTableUpdate
    • The log entry Error updating batch indicates that the batch couldn't update (due to the Connection error).
    • In this case there is an additional error code Next Exception: SQLState( 08006) ErrorCode(17002) 
      • Note: there might not be a reason code in the logs (it depends on the JDBC).
      • In this case, it indicates what we already know database is unreachable. 
    • Due to the error, the job is trying to rollback to the latest Batch - this fails because The connection is closed. 
       
  • The JAVA stacks are from the JDBC client and differ between JDBC connectors (details irrelevant). 
2023/07/15 14:14:33 - DelphixTableUpdate.0 - ERROR (version 7.1.0.0-12, build 1 from 2017-05-16 17.18.02 by buildguy) : Error in step, asking everyone to stop because of:
2023/07/15 14:14:33 - DelphixTableUpdate.0 - Error updating batch
2023/07/15 14:14:33 - DelphixTableUpdate.0 - IO Error: Connection timed out
...
2023/07/15 14:14:33 - DelphixTableUpdate.0 - Next Exception: SQLState( 08006) ErrorCode(17002)
...
2023/07/15 14:14:33 - DelphixTableUpdate.0 - Caused by: java.sql.BatchUpdateException: IO Error: Connection timed out
...
2023/07/15 14:14:33 - DelphixTableUpdate.0 - Error performing rollback on connection
2023/07/15 14:14:33 - DelphixTableUpdate.0 - The connection is closed.
2023/07/15 14:14:33 - DelphixTableUpdate.0 - Finished processing (I=18270000, O=0, R=18270000, W=18269999, U=18269999, E=1)

 

 

 

 

 

Privacy Policy Terms of Use Legal About
© 2024 Delphix Corp