How to Unlock ESXi hosts – you can not login at all

The root account of the ESXi hosts can be locked due to many failed login attempts. In this instance, you will not be able to connect to the host via web GUI or SSH. This can be due to many reasons, such as expired credentials / Brute force attacks and can cause Monitoring systems. Most of the time, I ended up with monitoring system related issues.

So, today we will look at the resolution steps.

There are requirements to be checked in advance following the below steps

Need physical console access or DCUI access (either using the iLO/iDRAC console)

The steps are as below (commands are in Italic and Bold)

1. Login to the DCUI console with the root credentials (do not worry you can access)

2. Enable SSH and shell access under the “Troubleshoot options”

3. Go to view logs and select syslogs and find the causing IP address

4. Then come back to DCUI main menu and press Alt + F1 to get the console

5. Execute the command pam_tally2 –user root to check how many failures and to identify the causing the IP address (in my case, the IP address of the monitoring system)

6. If you are confident on the source IP, you may unlock the root account by executing the pam_tally2 –user root –reset command

7. Just monitor for 15 minutes, if you notice re occurring failed attempts, you will have to change the IP or Power off the source (which we identified in step 3 & 5)

8. If you don’t notice any issues, you are good to go.

Permanent Device Loss (PDL) and All-Paths-Down (APD)

There is a new storage-related feature, VM Component Protection (VMCP), that protects virtual machines from possible storage issues.

There are two different types of methods that can be managed by VMCP:

PDL: It occurs when the storage array issues a SCSI sense code indicating that the device is unavailable (for example, a failed LUN).

APD: Usually, related to an underlying storage/networking issue, different from a PDL because the host doesn’t have enough information to determine if the device loss is temporary or permanent.

The below URL will provide all the necessary information to drill down the issue and the resolution steps;

https://kb.vmware.com/s/article/2004684

[source: VMware]