SQL Server VDB Provisioning Reports Recovery Failed Error After an Instance Crash or Restart (KBA6225)

Issue

When Provisioning or Refreshing a VDB, the Delphix Engine may report failure during an ALTER DATABASE command, with the following error message:

Title: Provision virtual database "MyVDB".
Error: Recovery failed for virtual database "MyVDB"
Error Code: exception.db.mssqlvdb.vdb.restore.failed
Suggested Action: Make sure that the SQL Server instance "INSTANCENAME" on the target host "HOSTNAME" is up and that the user "DOMAIN\USERNAME" has privileges to restore a database on the host.

In cases relevant to this article, the command output section will include a failure of the ALTER DATABASE ... SET ONLINE command:

ALTER DATABASE [MyVDB] SET OFFLINE WITH ROLLBACK IMMEDIATE
Failed to restart the current database. The current database is switched to master.
ALTER DATABASE [MyVDB] SET ONLINE
Msg 5011, Level 14, State 7, Server HOSTNAME\INSTANCENAME, Line 1
User does not have permission to alter database 'MyVDB', the database does not exist, or the database is not in a state that allows access checks.
Msg 5069, Level 16, State 1, Server HOSTNAME\INSTANCENAME, Line 1
ALTER DATABASE statement failed.

(34 rows affected)
SPID;Status;Login;HostName;BlkBy;DBName;Command;CPUTime;DiskIO;LastBatch;ProgramName;SPID2;REQUESTID
----;------;-----;--------;-----;------;-------;-------;------;---------;-----------;-----;---------
Msg 50000, Level 11, State 127, Server HOSTNAME\INSTANCENAME, Line 4
error_online_database

(0 rows affected)
die : Error: recover_vdb failed
At C:\Program Files\Delphix\DelphixConnector\564d0d4f-15e9-a73e-dee6-250ae46ae3fe-vdb-5\SCRIPT\Provision.ps1:26 char:12
+         die <<<<  "$($args[0])"
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,die

Applicable Delphix Versions

Click here to view the versions of the Delphix engine to which this article applies

Major Release	All Sub Releases
6.0	6.0.0.0, 6.0.1.0, 6.0.1.1, 6.0.2.0
5.3	5.3.0.0, 5.3.0.1, 5.3.0.2, 5.3.0.3, 5.3.1.0, 5.3.1.1, 5.3.1.2, 5.3.2.0, 5.3.3.0, 5.3.3.1, 5.3.4.0, 5.3.5.0 5.3.6.0, 5.3.7.0, 5.3.7.1, 5.3.8.0, 5.3.8.1, 5.3.9.0
5.2	5.2.2.0, 5.2.2.1, 5.2.3.0, 5.2.4.0, 5.2.5.0, 5.2.5.1, 5.2.6.0, 5.2.6.1
5.1	5.1.0.0, 5.1.1.0, 5.1.2.0, 5.1.3.0, 5.1.4.0, 5.1.5.0, 5.1.5.1, 5.1.6.0, 5.1.7.0, 5.1.8.0, 5.1.8.1, 5.1.9.0, 5.1.10.0
5.0	5.0.1.0, 5.0.1.1, 5.0.2.0, 5.0.2.1, 5.0.2.2, 5.0.2.3, 5.0.3.0, 5.0.3.1, 5.0.4.0, 5.0.4.1 ,5.0.5.0, 5.0.5.1, 5.0.5.2, 5.0.5.3, 5.0.5.4
4.3	4.3.1.0, 4.3.2.0, 4.3.2.1, 4.3.3.0, 4.3.4.0, 4.3.4.1, 4.3.5.0
4.2	4.2.0.0, 4.2.0.3, 4.2.1.0, 4.2.1.1, 4.2.2.0, 4.2.2.1, 4.2.3.0, 4.2.4.0 , 4.2.5.0, 4.2.5.1
4.1	4.1.0.0, 4.1.2.0, 4.1.3.0, 4.1.3.1, 4.1.3.2, 4.1.4.0, 4.1.5.0, 4.1.6.0

Troubleshooting

This may occur if the SQL Server instance or host has recently been restarted, and SQL Server is still performing crash recovery of other databases.

If crash recovery is still occurring, messages similar to the following will continue to appear in the SQL Server ERRORLOG file:

Recovery of database 'another_database' (6) is 81% complete (approximately 64 seconds remain). Phase 3 of 3. This is an informational message only. No user action is required.

Resolution

To successfully Provision or Refresh a VDB after a SQL Server instance or server restart, it may be necessary to wait until SQL Server has finished performing crash recovery.

Crash recovery of an instance is not complete until after the following message appears in the SQL Server ERRORLOG file.

Recovery is complete. This is an informational message only. No user action is required.

Once the above message has appeared in the ERRORLOG, please retry the affected Provision or Refresh operation.

If VDB Provisioning continues to fail with this error message after recovery finishes, please contact Delphix Support for further troubleshooting.