The internal application on the Delphix Engine that supports the browser interface (GUI), command line interface (CLI) and Web APIs may restart unexpectedly or become unresponsive. Virtual Databases (VDBs) will continue to run, but it may become impossible to administer the Delphix Engine. Scheduled jobs, e.g. including SnapSync jobs that generate new snapshots from dSources may be delayed, may not run, or may not complete. Other administration functions of the Delphix Engine may be similarly affected.
The issue can only occur when running the following version of Delphix software:
All Sub Releases
The issue is only known to affect Delphix Virtualization Engines. Masking Engines are not susceptible to the issue.
The issue is more likely to occur under heavier loads, including:
- Engines sustaining high Virtual Database (VDB) I/O activity.
- Engines where there are a larger number of VDBs or Sources, e.g. more than dozens of objects.
- Engines running many simultaneous jobs, e.g. SnapSync, Provision, or refresh.
- Existing sessions using the Delphix Admin or System Setup applications may become unresponsive.
- Attempts to login to an affected Delphix Engine may cause the browser to continually load or become unresponsive, and new sessions may fail.
- Commands issued in the CLI may not complete.
- Delphix jobs, e.g. SnapSync, running at the time an issue occurs may fail and may not automatically restart. Jobs may be failed with an error similar to:
<job_type> for "<object>" failed due to server restart during execution
If a Delphix Engine becomes non-responsive due to this problem, it should still be possible to manually restart the management service to restore access to the system:
- ssh into the delphix Engine using a user credential with the system administrator role. (Windows users may use an ssh utility like putty)
- Run the following CLI command
system restart ; commitCommand output similar to the following should be seen
delphix> system restart ; commit Restarting the management service. The current session will be re-established once the "service is available.
The issue is fully resolved in Delphix 220.127.116.11 and later releases.