Troubleshooting issues with connecting to Hyper-V and Azure Stack HCI

Edit online

Troubleshoot, and understand the connection issues between Microsoft Hyper-V or Azure Stack HCI and License Metric Tool. Analyze the listed solutions to resolve connection problems.

Glossary

The connection between License Metric Tool and Hyper-V or Azure Stack HCI is established through the Windows Management Instrumentation (WMI) queries. These queries use either PowerShell over the DCE-RPC mechanism, or WinRM that uses SOAP, or XML over HTTP protocol. The License Metric Tool client communicates with the Hyper-V or Azure Stack HCI host through the VM Manager interface, VM Manager Tool. The following list describes the basic components of the virtual environment:

Server: The Hyper-V or Azure Stack HCI host where the WinRM service is located.
Client: License Metric Tool host that can be either a Windows or UNIX computer.

The client authentication on both Windows and UNIX systems uses the same authentication sequence: NTLM, NTLMV2, or HTTP Basic. The Windows domain members undergo Kerberos network authentication when you use PowerShell as the communication interface.

Troubleshooting multiple VM manager connections

When there are many Hyper-V or Azure Stack HCI VM manager connections that are configured with the WinRM interface, VM Manager Tool might not be able to correctly establish the connections simultaneously in multiple threads. In this case, you must set the vmm_thread_pool_size parameter to 1 to lower the number of connection threads. Additionally, consider adding more VM Manager Tools to distribute the load between them.

Manual troubleshooting

To manually troubleshoot Hyper-V or Azure Tack HCI connections, perform the following steps.

To verify whether the WinRm service is running, run the following command in the command prompt.
```
sc query WinRM
```
The query results should include the following information.
```
SERVICE_NAME: WinRM
STATE       : 4  RUNNING
```
Note: The Windows User Account Control (UAC) can affect user access to the WinRM service. If you use the Negotiate authentication scheme, only the Administrator account can access the service. To allow all accounts to access the service, set the following registry value.
```
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System\
LocalAccountTokenFilterPolicy to '1'
```
Verify the WinRM configuration properties by using command prompt.
1. Run the following commands on the Hyper-V or Azure Stack HCI server:
  - winrm get winrm/config/service
  - winrm enumerate winrm/config/listener
2. Run the following commands on the client:
  - winrm get winrm/config/client
  Note: If Hyper-V or Azure Stack HCI and License Metric Tool are installed on the same host, you can use a single command to retrieve the required information: winrm get winrm/config.
Set up a simple test case by changing some of the WinRM settings.
1. On the License Metric Tool server, add an asterisk to the Trustedhosts list.
```
winrm set winrm/config/client @{TrustedHosts="*"}
```
  TrustedHosts="*" forces the client to abandon authentication of the remote end. However, the remote end still requires the client authentication. Typically, TrustedHost is set on the client to the Hyper-V or Azure Stack HCI server name.
2. Allow Basic authentication and unencrypted traffic on a Hyper-V or Azure Stack HCI server.
```
winrm set winrm/config/service/auth @{Basic="true"}
winrm set winrm/config/service @{AllowUnencrypted="true"}
```
  Basic="true" enables HTTP Basic authentication through user and password in clear text. AllowUnencrypted= true allows the transfer of the unencrypted data through HTTP between the server and the client.
3. Check the WinRM configuration setup.
```
'winrm get winrm/config/client':
Client:
        NetworkDelayms = 5000
        URLPrefix = wsman
        AllowUnencrypted = true
        Auth
            Basic = true
            Digest = true
            Kerberos = true
            Negotiate = true
            Certificate = true
            CredSSP = false
        DefaultPorts
            HTTP = 5985
            HTTPS = 5986
        TrustedHosts = *
 
'winrm get winrm/config/service' on the Hyper-V server:
Service:
    RootSDDL = O:NSG:BAD:P(A;;GA;;;BA)(A;;GA;;;S-1-5-21-3273298017-2363932476
    -3643925056-1633)S:P(AU;FA;GA;;;WD)(AU;SA;GWGX;;;WD)
    MaxConcurrentOperations = 4294967295
    MaxConcurrentOperationsPerUser = 15
    EnumerationTimeoutms = 600000
    MaxConnections = 15
    MaxPacketRetrievalTimeSeconds = 120
    AllowUnencrypted = true
    Auth
        Basic = true
        Kerberos = true
        Negotiate = true
        Certificate = true
        CredSSP = false
        CbtHardeningLevel = Relaxed
    DefaultPorts
        HTTP = 5985
        HTTPS = 5986
    IPv4Filter = *
    IPv6Filter = *
    EnableCompatibilityHttpListener = true
    EnableCompatibilityHttpsListener = true
    CertificateThumbprint
```
  If the previous change to Basic = true resolved the authentication issue, apparently the client and the server are not able to negotiate the authentication protocol. The Basic authentication scheme is not recommended, unless WinRM is set up with HTTPS. It might cause security exposure by sending a user name, a password and the message body in clear text. Microsoft uses three protocols during the Negotiate scheme: Kerberos, NTLMV2, and NTLM. Usually, the client and the server choose the best authentication mechanisms they agree upon. License Metric Tool does not allow the Kerberos protocol. To check whether the client and the server allow the NTLMV2 or NTLM protocol, use the following registry query.
```
reg query HKEY_LOCAL_MACHINE\System\CurrentControlSet\control\LSA\MSV1_0
```
  The following parameters under this registry key control the authentication schema behavior:
  - NtlmMinClientSec
  - NtlmMinServerSec
  The parameters can be modified through the appropriate Group Policy: Computer Configuration > Windows Settings > Local Policies > Security Options:LAN Manager authentication level.
To manually identify the appropriate authentication mechanism for WinRM service on the remote host, use the following commands:
- Run the command from Windows PowerShell.
```
Test-WSMan -ComputerName http://<server_name>:<port> 
-Auth <authentification_scheme> -Cred <user_id>
```
  Where:
  
  <server_name>
  
  Is the host name of the Hyper-V or Azure Stack HCI server. If you are using HTTPS, the host name must match the CN in the certificate.
  
  <port>
  
  Is the port number on which the Windows Remote Management client for the HTTP or HTTPS transport listens.
  
  <authentification_scheme>
  
  Is the authentication scheme: Basic or Negotiate
  
  <user_id>
  
  Is the user ID to Windows PowerShell.
- Run the command from the command prompt.
```
winrm identify -r:http://<server_name>:<port> 
-auth <authentification_scheme> -u:<user_id> -p:<password>
```
  Where:
- <server_name>
  
  Is the host name of the Hyper-V or Azure Stack HCI server. If you are using HTTPS, the host name must match the CN in the certificate.
  
  <port>
  
  Is the port number on which the Windows Remote Management client for the HTTP or HTTPS transport listens.
  
  <authentification_scheme>
  
  Is the authentication scheme: Basic or Negotiate.
  
  <user_id>
  
  Is the user ID to WinRM.
  
  <password>
  
  Is the password to WinRM.

Event tracing

If License Metric Tool cannot connect to the Hyper-V or Azure Stack HCI host, use the custom Windows Event Tracing. When the settings of the Client and the server are conflicting, the Security event log generates the following error message: Unknown user name or bad password. You cannot determine that the actual issue is the unsupported authentication protocols. The Windows event tracing obtains the diagnostic data about the WMI and WinRM queries.

To start Event Tracing on the Hyper-V or Azure Stack HCI server, use the following commands:
- logman.exe start winrmtrace -p Microsoft-Windows-Winrm -o winrmtrace.etl -ets
- logman.exe start wmitrace -p Microsoft-Windows-WMI-Activity -o wmitrace.etl -ets
Run the VM Manager connection test. If no VM manager connection is defined, the command output is empty.
- vmman.bat -testconnection
- vmman.sh -testconnection
When the connection test finishes, stop collecting data traces:
- logman.exe stop winrmtrace -ets
- logman.exe stop wmitrace -ets
To change the formatting of the collected Event Trace Log files, use the following commands:
- Tracerpt winrmtrace.etl
- Tracerpt wmitrace.etl

The formatted WinRM Event Trace Log file, winrmtrace.etl, contains the information about the user authentication issue.

<Task>User authentication </Task>
<Message>Sending HTTP 401 response to the client and disconnect the connection after
 sending the response</Message>

Additionally, the file consists of the SOAP, or XML queries that are sent from the Client and provide essential data that is useful during troubleshooting the Hyper-V problems.

Troubleshooting connection status with Wireshark

Run the VM Manager connection test. If no VM manager connection is defined, the command output is empty.

vmman.bat -testconnection
vmman.sh -testconnection

Note:

To analyze the network traffic stream on UNIX with no GUI environment, use tcpdump. To dump all the traffic to the external file use

tcpdump -vvv -XX
-w tcpdump.out

. The tcpdump.out file can be viewed with Wireshark.

Successful connection test between License Metric Tool and Hyper-V or Azure Stack HCI with PowerShell interface.

Parameters:

vmm_communication_interface=POWERSHELL

PowerShell uses the NTLM authentication protocol.

The screen shows the Wireshark console with the Hyper-V connection details.

It does not connect to the WinRM ports: 5985 and 80, but it uses a DCE-RPC mechanism to connect to the Endpoint Mapper service via port 135.

Select the RemoteCreateInstance response entry.
To check whether the authentication finished successfully, expand the parameters, and check the value of HResult. If authentication was success, HResult is marked as S_OK.

PowerShell v2 by default uses NTLMv1 for negotiation. To update NTLM, use the following registry setting:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\LSA\LMCompatibilityLevel

For more information, see: How to enable NTLM 2 authentication.

Unsuccessful connection test between License Metric Tool and Hyper-V or Azure Stack HCI with NTLM interface.

Parameters:

```
vmm_communication_interface=NTLM
```
```
vmm_url=http://.../wsman
```
The default port: 80

The connection test failed, and the trace.log_vmmconf_<VM_manager_ID>.log contains the following error messages.

(...)com.ibm.license.mgmt.vmmanager.hyperv.net.HttpConnector::send::
An error occured while trying to send request to http://.../wsman

(...)com.ibm.license.mgmt.vmmanager.hyperv.net.HttpConnector::send:java.net.
ConnectException: Connection refused: connect

While the errors from the trace.log_vmmconf_<VM_manager_ID>.log file do not provide enough details, the Wireshark capture proves that the issue is related to TCP. Use the following command to check for listener on port 80.

winrm enumerate winrm/config/listener

If no listeners are detected, run the following command to set up listeners.

winrm set winrm/config/service @{EnableCompatibilityHttpListener="true"}

Note: The setup of listeners on port 80 can collide with the other HHTP services on this computer. To avoid the issues, specify a dedicated WinRM port, such as 5985 or 5986, on the vmm_url connection string.

Unsuccessful connection test between License Metric Tool and Hyper-V or Azure Stack HCI with NTLM interface. HTTP Error 401.

The connection test failed, and the trace.log_vmmconf_<VM_manager_ID>.log contains the following error messages.
```
(...)com.ibm.license.mgmt.vmmanager.hyperv.net.HttpConnector::retrieve::
Response Code is: 401
```
```
(...)com.ibm.license.mgmt.vmmanager.hyperv.net.HttpConnector::retrieve::
The following response code was returned while connecting to VM Manager 
http://.../wsman: responseCode = 401
```
The following Wireshark capture lists the NTLM negotiation.
The first four rows show the successful NTLM negotiation. The fourth row ends the so-called four-way NTLM handshake. The last line detects HTTP Error 401: Unauthorized, and expands into the following HTTP response.
The WWW-Authenticate: Negotiate response indicates that the server is ready to use the NTLM, NTLMV2, or HTTP Basic authentication scheme. However, the server already successfully responded to the negotiation sequence. Ensure that the WinRM AllowUnencrypted = true.
Note: VM Managers do not provide the HTTP body encryption. WinRM accepts only the HTTP body that is encrypted with Negotiate, or Kerberos protocol.
The Wireshark capture shows the unsuccessful NTLM negotiation handshake.
The four-way NTLM handshake ends with a 401 error. The most common reason behind this error is the lack of sufficient access rights. To check whether the accounts assigned to the Administrators group are allowed to access WinRM, configure the following Windows registry entry.
```
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\
System\LocalAccountTokenFilterPolicy
```

Unsuccessful connection test between License Metric Tool and Hyper-V or Azure Stack HCI caused by the bug related to vmm_login domain prefix interpretation.

When you add a VM Manager to License Metric Tool, make sure that you provide the administrator user name in one of the following formats:

user_name@domain, for example: test@cluster.com
user_name\domain, for example: test\cluster.com

Starting from application update 9.2.16, you can also provide the administrator user name in one of the following formats:

domain@user_name, for example: cluster.com@test
domain\user_name, for example: cluster.com\test

Note: Make sure that you upgrade the VM Manager Tool to version 9.2.16, not only the License Metric Tool server.

If you do not follow this pattern, and provide the user name in the incorrect format, an error is generated during the VM Manager authentication.

The screen shows the Wireshark capture that contains the domain authentication error.

When the error occurs, the user name is assigned to the incorrect field.