The Delphix Engine includes the Delphix Storage Migration feature that allows for the removal of storage devices if they are no longer needed. Once a device is removed using this feature, the storage configuration of the Delphix Engine may be left in an inconsistent state. This may result in a failure of an affected Delphix Engine to start properly.
The failure to start will extend any Virtual Database (VDB) service outage associated with a restart or crash and will render the Delphix Engine administrative interfaces (Command Line Interface (CLI), Web API, and Graphical User Interface (GUI)) inoperative.
The problem can only occur in the following Delphix Engine Releases:
Delphix Engine 126.96.36.199 and 188.8.131.52
Delphix Engine 184.108.40.206 and 220.127.116.11
Delphix Engine 18.104.22.168 and 22.214.171.124
Delphix Engine 126.96.36.199
Delphix Engine 188.8.131.52
Delphix Engine 184.108.40.206 and 220.127.116.11
Delphix Engine 18.104.22.168
Delphix Engine 22.214.171.124 and 126.96.36.199
Delphix Engine 188.8.131.52
Delphix Engine 184.108.40.206 and 220.127.116.11
Delphix Engine 18.104.22.168
Delphix Engine 22.214.171.124 and 126.96.36.199
Delphix Engine 188.8.131.52 and 184.108.40.206 and 220.127.116.11
The problem will only occur following a restart or crash of the Delphix Engine appliance from which a storage device has been successfully removed using the Storage Migration Feature.
If the conditions specified in this section are satisfied, there is a high probability of experiencing the problem.
- The Delphix Engine will begin the normal boot / restart process but will not fully progress to a normal state. When viewing the Delphix Engine guest console, e.g. using ESXi, the Delphix Management Service and Delphix Boot Service services will persistently show as offline, as below:
- The system may appear reachable on the network. For example, a ping command might get a successful response. Otherwise the system will be unresponsive.
- If VDBs were not shutdown prior to the incident, they may hang or crash. In the system log of affected Oracle target hosts, messages like:
NFS server <ip address> not responding
may be seen on the console or in the system log.
- Attempts to initiate SSH connections to an affected Delphix Engine will fail with "connection refused."
- Attempts to initiate new browser connections to an affected Delphix Engine will fail with "This site cannot be reached", "refused to connect", or "connection refused" errors.
- Web API calls will timeout or fail
Avoid using the Delphix Storage Migration feature.
If device removal has been completed, but the problem has not occurred because the Delphix Engine has not been restarted, defer any planned restarts/reboots of the affected Delphix Engine and contact Delphix Support.
If this problem has already occurred, contact Delphix Support for remediation. Further reboots or restarts of the Delphix Engine will not be effective.
The issue is fully resolved in Delphix Engine OS Version 5.0.2016.04.29, and later OS releases.
Fresh Installations and Full Upgrades to Delphix Engine 18.104.22.168 will run OS Version 5.0.2016.04.29.
Deferred OS upgrades to Delphix 22.214.171.124 will run a prior version of the Delphix OS and are still susceptible to the problem described in this bulletin.
See the "Deferred OS Upgrade" section of "Upgrading to a New Version of the Delphix Engine" for information about how to determine the current OS version of a Delphix Engine.
Delphix Engine Release
Included OS Version
Minimum OS Version