The Delphix Engine includes the Delphix Storage Migration feature that allows for the removal of storage devices if they are no longer needed. During or following a device removal using this feature, the Delphix Engine may crash and restart unexpectedly. The system will normally return to service without manual operation; however, Delphix jobs running at the time of the failure, e.g. SnapSync or Provision jobs, may be interrupted and may need to be manually restarted.
Delphix Engine crashes can cause temporary loss of access to Virtual Databases (VDBs), or could cause VDBs to crash.
Recurring Delphix Engine crashes are possible.
The problem can only occur in the Delphix Engine Releases in which the Delphix Storage Migration Feature is available, including all of the following:
Delphix Engine 22.214.171.124 and 126.96.36.199
Delphix Engine 188.8.131.52 and 184.108.40.206
Delphix Engine 220.127.116.11 and 18.104.22.168
Delphix Engine 22.214.171.124
Delphix Engine 126.96.36.199
Delphix Engine 188.8.131.52 and 184.108.40.206
Delphix Engine 220.127.116.11
Delphix Engine 18.104.22.168 and 22.214.171.124
Delphix Engine 126.96.36.199
Delphix Engine 188.8.131.52 and 184.108.40.206
Delphix Engine 220.127.116.11
Delphix Engine 18.104.22.168 and 22.214.171.124
Delphix Engine 126.96.36.199 and 188.8.131.52 and 184.108.40.206
- Delphix Engine 220.127.116.11 and 18.104.22.168
- Delphix Engine 22.214.171.124 and 126.96.36.199
The problem can only occur during removal or following the successful removal of a storage device using the Delphix Storage Migration feature.
The problem is typically associated with the use of one or more VDBs subsequent to completion of the device removal feature. If all VDBs have been in use following a device removal, the probability of the issue occurring rapidly decreases over time. Likewise, the problem is more likely to occur as VDBs are first used following a device removal.
If a Delphix Engine experiences this problem, multiple recurrences are likely.
Storage devices added by the hypervisor to the Delphix guest machine but are not configured in the Delphix storage pool as data devices cannot contribute to this problem. See Adding and Expanding Storage Devices for more information on the procedure for configuring devices into a Delphix storage pool.
- Following an unexpected reboot, an alert may be be created with the following descriptive text:
Unexpected server restart The server is starting up following an unexpected shutdown around <date>. Contact Delphix Support
- Delphix jobs running at the time of the failure may fail with the alert:
<job_type> for "<object>" failed due to server restart during execution
- When a reboot occurs VDBs may experience temporary suspension of service. SQL Server VDBs may be inaccessible until they are restarted. In the log of affected Oracle target hosts, messages like:
NFS server <ip address> not responding
may be seen on the console or in the system log, where <ip address> is the network address of the affected Delphix Engine.
Avoid using the device removal feature on affected Delphix Engine releases.
This issue is partially resolved in Delphix OS Version 5.0.2016.04.29, the operating system delivered with Delphix Engine 188.8.131.52. In particular, this Delphix OS Version solved the case where system crashes could occur following the successful removal of a storage device using the Delphix Storage Migration feature.
The issue is fully resolved in Delphix OS Versions, 5.0.2016.08.23, 5.1.2016.09.01, and later releases. These OS versions were delivered in Delphix Engine releases 184.108.40.206 and 220.127.116.11, respectively.
Fresh installations and full upgrades will be be running the Delphix OS version specified as included OS Version in the table below. For Delphix Engine releases where the Included OS Version and Minimum OS Version are different, it's possible that a Deferred Upgrade was used and that the affected Delphix Engine could be running an older operating system version. See the "Deferred OS Upgrade" section of "Upgrading to a New Version of the Delphix Engine" for information about how to determine the current OS version of a Delphix Engine.
|Delphix Engine Release||Included OS Version||Minimum OS Version|