Applicable Delphix Versions
- Click here to view the versions of the Delphix engine to which this article applies
Major Release All Sub Releases 6.0 18.104.22.168, 22.214.171.124, 126.96.36.199, 188.8.131.52, 184.108.40.206, 220.127.116.11
The symptoms of a server hang are that the system is not reachable via NFS, the GUI, SSH, console logins, et cetera, and that the hypervisor still indicates the virtual machine (VM) is running. The server may respond to ping depending on the nature of the hang. Should such a condition arise where the system is otherwise unreachable, a non-maskable interrupt (NMI) may be sent from the hypervisor to cause the Delphix operating system (DxOS) to kernel panic and generate a crash dump. The resulting crash dump can be collected by Delphix Support for further analysis.
If the system does not respond, retry the procedure. The final recourse is to reset or power on/off the system which will not generate a core and reduce potential for root cause analysis.
It is important to note that this procedure will not be successful in all cases. VM hangs may occur for a variety of reasons related to the guest operating system, or other hypervisor issues. The following procedure is a best-effort to collect system state information at the time of a VM hang.
An administrative user with PowerShell access is required to complete this process.
An NMI to a VM in Hyper-V can be issued by an Administrator using the Debug-VM PowerShell cmdlet. During this process it is recommended to also monitor the VM console in another window.
In the example below, the Delphix VM named "SeanN HyperV Test" is currently running on server DEVSUPPORT-HV01.
The cmdlet syntax to issue an NMI with Debug-VM is:
PS C:> debug-vm "SeanN HyperV Test" -InjectNonMaskableInterrupt -Force
If executed successfully, the command prompt will return immediately.
Once the command is issued successfully, there will be no console updates while the memory dump is generated. It is critical that the VM not be restarted during this period; as it may seem the VM is unresponsive, restarting during this phase will cause the memory dump to fail and diagnostic details will be unavailable for any potential root cause analysis.
The Windows Event Log can also be reviewed to confirm the NMI was issued by navigating to Application and Services Logs - Microsoft - Windows - Hyper-V-VMMS - Admin. The event ID is 33500.
Once the crash dump is generated the VM will automatically restart from the original boot device, and normal Delphix VM boot activity should be observed.
Delphix Support engagement will be required to collect the diagnostic data from the Engine for further analysis.
The following articles may provide more information or related information to this article: