ORA-27037: unable to obtain file status during runtime (KBA9107)

Issue

During the use of a Oracle VDB certain SQL statements fail with the following error:

ORA-01565: error in identifying file '/cs/delphix/UNYSVU70/datafile/+SNY94900_DATA1/pnyb3r10/datafile/undotbs1_7.dbf' 
ORA-27037: unable to obtain file status 
Linux-x86_64 Error: 13: Permission denied

The same error occured against multiple datafiles depending on the SQL query.

This error was due to the permissions on the Delphix mount point used by the VDB being changed after provision time.

Example:

oracle@target$ ls -la /mnt/provision/VDB1 
total 10 
drwxr-x--x 7 oracle asmadmin 7 Mar 14 12:17 . 
drwxr-x--- 9 delphix delphix 255 Mar 14 13:16 .. 
drwxr-x--- 2 oracle asmadmin 87 Mar 25 06:37 archive 
drwxr-x--- 4 oracle asmadmin 12 Mar 14 12:01 datafile 
drwxr-x--- 2 oracle asmadmin 2 Sep 13 2021 external 
drwxr-x--- 3 oracle asmadmin 3 Mar 14 11:44 script 
drwxr-x--- 2 oracle asmadmin 64 Mar 14 12:12 temp

You can see that the parent mount point /mnt/provision has a timestamp of 13:16 which is after the creation time of the VDB mount at 12:17.

This causes various issues when the database attempts to access certain files and will report errors like the following:

ORA-27037: unable to obtain file status 
Linux-x86_64 Error: 13: Permission denied

Applicable Delphix Versions

Click here to view the versions of the Delphix engine to which this article applies

Major Release	All Sub Releases
6.0	6.0.0.0, 6.0.1.0, 6.0.1.1, 6.0.2.0, 6.0.2.1, 6.0.3.0, 6.0.3.1, 6.0.4.0, 6.0.4.1, 6.0.4.2, 6.0.5.0, 6.0.6.0, 6.0.6.1, 6.0.7.0, 6.0.8.0, 6.0.8.1, 6.0.9.0, 6.0.10.0, 6.0.10.1, 6.0.11.0, 6.0.12.0, 6.0.12.1, 6.0.13.0
5.3	5.3.0.0, 5.3.0.1, 5.3.0.2, 5.3.0.3, 5.3.1.0, 5.3.1.1, 5.3.1.2, 5.3.2.0, 5.3.3.0, 5.3.3.1, 5.3.4.0, 5.3.5.0, 5.3.6.0, 5.3.7.0, 5.3.7.1, 5.3.8.0, 5.3.8.1, 5.3.9.0
5.2	5.2.2.0, 5.2.2.1, 5.2.3.0, 5.2.4.0, 5.2.5.0, 5.2.5.1, 5.2.6.0, 5.2.6.1
5.1	5.1.0.0, 5.1.1.0, 5.1.2.0, 5.1.3.0, 5.1.4.0, 5.1.5.0, 5.1.5.1, 5.1.6.0, 5.1.7.0, 5.1.8.0, 5.1.8.1, 5.1.9.0, 5.1.10.0
5.0	5.0.1.0, 5.0.1.1, 5.0.2.0, 5.0.2.1, 5.0.2.2, 5.0.2.3, 5.0.3.0, 5.0.3.1, 5.0.4.0, 5.0.4.1, 5.0.5.0, 5.0.5.1, 5.0.5.2, 5.0.5.3, 5.0.5.4

Resolution

The permissions on the Delphix mount point used by the VDB had been altered after provisioning which prevented the Oracle process from accessing all the required files.

The mount point should meet the requirements detailed in the documentation:

Troubleshooting