Skip to main content
Delphix

Large Storage Performance Test Causes the Management Service to Restart (KBA8655)

 

 

KBA

KBA# 8655

Applicable Delphix Versions

Click here to view the versions of the Delphix engine to which this article applies
Major Release All Sub Releases
6.0 6.0.0.0, 6.0.1.0, 6.0.1.1, 6.0.2.0, 6.0.2.1, 6.0.3.0, 6.0.3.1, 6.0.4.0, 6.0.4.1, 6.0.4.2, 6.0.5.0, 6.0.6.0, 6.0.6.1, 6.0.7.0, 6.0.8.0, 6.0.8.1, 6.0.9.0, 6.0.10.0, 6.0.10.1, 6.0.11.0

Troubleshooting

Running a Storage Performance Test with a very large testRegion value can lead to an unexpected restart of the Delphix engine management process.

In the following example, the Delphix engine has 4TB of storage assigned and the testRegion parameter for the storage performance test had been set to 3TB.  The initialization phase of the job completed successfully but the test failed shortly after the test workloads started, reporting that the management service was unavailable.

delphix-engine storage test create *> ls
Properties
    type: StorageTestParameters
    devices: xvdb,xvdc,xvdd,xvde (*)
    duration: 120
    initializeDevices: true
    initializeEntireDevice: false
    testRegion: 3TB (*)
    tests: ALL
delphix-engine storage test create *> commit
    `STORAGE_TEST-1
    Dispatched job JOB-1
    STORAGE_TEST_EXECUTE job started for "system".
    Initializing storage test.
    Starting storage device initialization.
    ETA: 1:50:57.

Storage device initialization complete.
Starting storage benchmarking.
Starting random read workload with 4 KB block size and 8 jobs.
Starting random read workload with 4 KB block size and 16 jobs.
-
Starting random read workload with 4 KB block size and 32 jobs.
Starting random read workload with 4 KB block size and 64 jobs.

The operation could not be completed because the management service is currently unavailable. If the system is starting up, please log out and try again at a later point. Otherwise, run "system restart" as a system user and try to reconnect at a later point. If the problem persists, please contact Delphix support.

Prerequisites 

The Delphix engine has been assigned multiple terabytes of storage space and a multi-terabyte testRegion value has been set for the storage test.

Resolution

This issue is under investigation within Delphix Engineering.

To avoid this issue, use the default testRegion value for the storage performance test.

If the issue persists when using the default testRegion value, please contact Delphix Support for further assistance.