Resolving rpc.statd Error (KBA1202)
Applicable Delphix Versions
-
Major Release All Sub Releases All All
Issue
Enabling a virtual database (VDB) on an affected target will fail with an error suggestive of incorrect sudo privileges:
Error Failed to mount "Vdaisy_56B" because the mount command executed as user "sybase" on host "nstackrh69.delphix.com" failed for mount point "/home/sybase/toolkit/564d21b7-5756-10a6-694b-5bdb5f806c53-vdb-28/datafile". Error Code exception.db.generictarget.mount.failed Suggested Action Make sure the user has permission to mount file systems on the host and that NFS traffic from the host can reach the Delphix Engine.
The detailed portion of the error message showing the "Command Output" shows:
mount.nfs: rpc.statd is not running but is required for remote locking. mount.nfs: Either use '-o nolock' to keep locks local, or start statd. mount.nfs: an incorrect mount option was specified mount: only root can do that stty: standard input: Inappropriate ioctl for device stty: standard input: Inappropriate ioctl for device mount.sh: MOUNT_FAILED
Similarly, the debug log file from the Delphix Engine shows:
[2017-10-13 13:14:12,963][INFO][jcm.impl.JobInstance#execute:352][Worker-4161|JOB-2468|DB_PROVISION(ASE_DB_CONTAINER-153)][] job failed with user error
com.delphix.appliance.server.dco.events.enums.DbGenerictargetExceptions$MountFailed: exception.db.generictarget.mount.failed {Vdaisy_56B|sybase|nstackrh69.dcenter.delphix.com|/home/sybase/toolkit/564d21b7-5756-10a6-694b-5bdb5f806c53-vdb-28/datafile|mount.nfs: rpc.statd is not running but is required for remote locking.
mount.nfs: Either use '-o nolock' to keep locks local, or start statd.
mount.nfs: an incorrect mount option was specified
mount: only root can do that
stty: standard input: Inappropriate ioctl for device
stty: standard input: Inappropriate ioctl for device
mount.sh: MOUNT_FAILED
}
Troubleshooting VDB Provision Failures on Linux due to rpc.statd Error
Four disparate scenarios have been identified as causing this error.
Scenario 1
Usage of the hosts access control files "/etc/hosts.allow" and "/etc/hosts.deny" can prevent the mount command from succeeding because rpcbind is disallowed from making connections to the "localhost" address. Errors similar to the following can be seen in "/var/log/messages":
10:34:04 nstackrh69 kernel: RPC: server rpc.statd requires stronger authentication. 10:34:04 nstackrh69 rpcbind[9783]: connect from 127.0.0.1 to getport/addr(status): request from unauthorized host
Checking the two files, strict access is configured:
$ cat hosts.deny # # hosts.deny This file contains access rules which are used to # deny connections to network services that either use # the tcp_wrappers library or that have been # started through a tcp_wrappers-enabled xinetd. # ALL:ALL $ cat hosts.allow # # hosts.allow This file contains access rules which are used to # allow or deny connections to network services that # either use the tcp_wrappers library or that have been # started through a tcp_wrappers-enabled xinetd. # # ALL: 10.0.0.0/255.0.0.0
Scenario 2
This is caused by a bug in the statd startup script included within a range of Oracle Enterprise Linux software releases (Oracle Enterprise Linux 6.0 - 6.8 inclusive). The PATHenvironment variable in /usr/sbin/start-statd does not include the path for command cat. The detailed error message/logging from Delphix should include an additional error about being unable to find the "cat" command:
/usr/sbin/start-statd: line 10: cat: command not found;mount.nfs: rpc.statd is not running but is required for remote locking.
Review the "/usr/sbin/start-statd" script to confirm "/bin" is included in the PATH.
Scenario 3
Incorrect permissions on /etc/hosts.allow and /etc/hosts.deny can cause this error.
Run "ls -la /etc/hosts.*" to make sure that the user performing the mount operation has read access to these files.
Scenario 4
The rpcbind service is not running. Run the Linux "service" command as follows to see if the rpcbind service is running:
# service rpcbind status rpcbind is stopped
Resolution
Scenario 1
Grant rpcbind permission to the localhost address by adding the following line to "/etc/hosts":
# cat hosts.allow # # hosts.allow This file contains access rules which are used to # allow or deny connections to network services that # either use the tcp_wrappers library or that have been # started through a tcp_wrappers-enabled xinetd. # ALL: 10.0.0.0/255.0.0.0 rpcbind: 127.0.0.1
Scenario 2
Add /bin as a path to script /usr/sbin/start-statd
e.g.
PATH=/bin:/sbin:/usr/sbin
# cp /usr/sbin/start-statd /usr/sbin/start-statd.orig # diff /usr/sbin/start-statd /usr/sbin/start-statd.orig 7c7 < PATH=/bin:/sbin:/usr/sbin --- > PATH=/sbin:/usr/sbin
Scenario 3
Grant read permission to the appropriate user and groups. Normally these files have permission 644:
# chmod 644 /etc/hosts.allow # chmod 644 /etc/hosts.deny
Scenario 4
Start the rpcbind service from an account with root permission:
# service rpcbind status rpcbind is stopped # service rpcbind start Starting rpcbind:
Related Articles
External Links: