Troubleshooting "Exhausted available authentication methods" (KBA6009)
KBA
KBA# 6009
Issue
There are many different scenarios in which Delphix must establish an ssh connection to a host (adding a new environment, refreshing an existing an environment, provisioning a VDB, taking a SnapSync for a dSource, et cetera). Sometimes these ssh connections will fail because they are rejected by the host Delphix is trying to connect to.
The error shown by Delphix in response to the connection rejection is very generic. It simply states that it "Exhausted available authentication methods" and does not provide any additional information:
Error: Could not log into "snoopy.acme.com" with the provided username and password. Error Code: exception.ccc.authenticate.failed Suggested Action: Make sure that the host user and password are correct, and that the Delphix Connector is properly registered with the Delphix Engine. Command Output: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
From the Delphix side, we can only see that the host ("snoopy.acme.com" for example) rejected the login attempt because it has "Exhausted available authentication methods". The host will not give more information about why it rejected the login attempt because it does not want to give any helpful hints on what to try next in the event that someone is trying to hack into the host.
The "Exhausted available authentication methods" message can trigger for any number of reasons, such as:
- User account is locked
- Password is expired
- Incorrect permissions on the user's home directory or public keys
- User's home directory has been moved
Applicable Delphix Versions
- Click here to view the versions of the Delphix engine to which this article applies
-
Major Release All Sub Releases 6.0 6.0.0.0, 6.0.1.0, 6.0.1.1, 6.0.2.0, 6.0.2.1, 6.0.3.0, 6.0.3.1, 6.0.4.0, 6.0.4.1, 6.0.4.2, 6.0.5.0, 6.0.6.0, 6.0.6.1, 6.0.7.0, 6.0.8.0, 6.0.8.1, 6.0.9.0, 6.0.10.0, 6.0.10.1, 6.0.11.0 5.3
5.3.0.0, 5.3.0.1, 5.3.0.2, 5.3.0.3, 5.3.1.0, 5.3.1.1, 5.3.1.2, 5.3.2.0, 5.3.3.0, 5.3.3.1, 5.3.4.0, 5.3.5.0 5.3.6.0, 5.3.7.0, 5.3.7.1, 5.3.8.0, 5.3.8.1, 5.3.9.0 5.2
5.2.2.0, 5.2.2.1, 5.2.3.0, 5.2.4.0, 5.2.5.0, 5.2.5.1, 5.2.6.0, 5.2.6.1
5.1
5.1.0.0, 5.1.1.0, 5.1.2.0, 5.1.3.0, 5.1.4.0, 5.1.5.0, 5.1.5.1, 5.1.6.0, 5.1.7.0, 5.1.8.0, 5.1.8.1, 5.1.9.0, 5.1.10.0
Resolution
Most often this is due to the proper permissions not being applied to the public key directory.
1. If you are using public-key encryption for logging into your Unix-based environment:
- Select the Delphix operating system environment user and choose Public Key for the Login Type.
- Click View Public Key.
- Copy the public key that is displayed, and append it to the end of your
~/.ssh/authorized_keys
file. If this file does not exist, you will need to create it.
Note: If ~/.ssh/authorized_keys, ~/.ssh or ~ (the user's home directory) are writable by other users other than the Delphix Enviroment User, sshd will not allow public key authentication. The following are examples of permissions which conform to sshd's allowable permission convention:- Run
chmod 600 ~/.ssh/authorized_keys
- assign read/write privileges for your user. - Run chmod 700 ~/.ssh - assign read/write/execute privileges for your user.
- Run
chmod 755 ~
- assign home directory writable only by your user.
- Run
The public key needs to be added only once per user and per environment.
2. If you are not using public-key encryption (or the above is correct), it will be necessary to review the target host's security files to determine the reason the host rejected the login attempt. Please review the troubleshooting section for further information.
Troubleshooting
Check the security logs of the host that is rejecting the login attempt:
- Linux logs the errors in "/var/log/secure"
- for the following Authentication errors:
Oct 14 12:06:53 rh610-base sshd[15773]: Authentication refused: bad ownership or modes for directory /home/oracle/.ssh
Oct 14 12:07:40 rh610-base sshd[15801]: Authentication refused: bad ownership or modes for file /home/oracle/.ssh/authorized_keys
Oct 14 12:05:24 rh610-base sshd[5063]: Authentication refused: bad ownership or modes for directory /home/oracle
- On HP-UX 11.23 or higher with SSH 5.0, SSHD daemon logging of failed logins logs successfully to two files:
/var/adm/syslog/syslog.log
/var/adm/btmps - AIX - Please refer to IBM's documentation
- Solaris - Please refer to Oracle's documentation
For all platforms, you may need to login as "root" and run the "sshd" with debug flags enabled and choose an unused port to connect to.
Related Articles
The following articles may provide more information or related information to this article:
- How to Enable ssh/sshd Debugging for Solaris[TM] Secure Shell [ssh(1)] (Doc ID 1022291.1)
- How to Enable Debugging in PAM (Pluggable Authentication Module) (Doc ID 1009025.1)
- OpenSSH/Logging and Troubleshooting