IBM Tivoli Storage Manager for Virtual Environments, Version 7.1

Troubleshooting

Solutions to Data Protection for VMware vSphere GUI and Data Protection for VMware command-line interface issues are provided.

Data Protection for VMware vSphere GUI or Data Protection for VMware vCloud GUI backup or restore operation fails

Complete these tasks to resolve a backup or restore failure:

Log on to the system where the Tivoli® Storage Manager backup-archive client is installed.
Start a command-line session:
- Open the Windows Start menu and select Programs > Tivoli Storage Manager > Backup Client Command Line.
- Open a terminal window.
If not already there, go to the installation directory:
```
cd C:\Program Files\Tivoli\TSM\baclient
```
```
cd /opt/tivoli/tsm/client/ba/bin
```
By default, error log files are in the installation directory.
View these Tivoli Storage Manager backup-archive client log files to see if an error was generated:
- dsmerror.log: All client messages.
- dsmwebcl.log: All web client messages.
- dsmj.log: All client Java™ GUI messages.
These log files are located in the directory you specify with the DSM_LOG environment variable or in the current working directory.
If neither of these files contain an error, run a backup-archive client backup and restore operation to see if it fails.
If the backup-archive client operations complete successfully, run a Data Protection for VMware command-line interface Backup and Restore operation. Set the appropriate trace parameters (as described in Profile parameters) so you can view any errors that might be generated.

Data Protection for VMware command-line interface backup fails with `scSignOnAsAdmin: Error 53`

In this situation, a Data Protection for VMware command-line interface backup operation failed and this error was generated to the backup-archive client dsmerror.log:

scSignOnAsAdmin: Error 53 receiving SignOnAsAdminResp verb from server

Typically, this error results when the VMCLI node name is different from its administrator name. Tivoli Storage Manager requires these two names to be the same.

Tivoli Storage Manager data mover nodes are not visible during a backup operation

Verify that the correct proxy node authority was granted on the Tivoli Storage Manager server. If the correct authority exists, then the data center mapping specified by the VE_DATACENTER_NAME profile parameter is incorrect. See Profile parameters for a complete description and correct syntax of the VE_DATACENTER_NAME parameter.

The `inquire_detail` command failed with `Return Code 53`

In this situation, the vmcli -f inquire_detail command failed and this error was generated to your log file:

ANS1033E (RC-53)  An invalid TCP/IP address was specified.

This error occurs when a node name does not match its administrator name. This issue can happen when you rename a node but do not rename its administrator. The solution is to either rename the administrator to match the new node name or register a new administrator for the new node.

The commands in these examples are issued from the Tivoli Storage Manager Administrative Command Line:

Rename the administrator at the same time you rename the node:

rename node <current_node_name> <new_node_name>
rename admin <current_admin_name> <new_node_name>

For example:

rename node DC_VC5 DC_WIN2K8_X64
rename admin DC_VC5 DC_WIN2K8_X64

As a result, the new administrator name matches the new node name.

rename node <current_node_name> <new_node_name>
register admin <new_admin_name> <password>

For example:

rename node DC_VC5 DC_WIN2K8_X64
register admin DC_WIN2K8_X64 DC_WIN2K8_X64PWD

As a result, the new administrator name matches the new node name.

Invalid sign on and invalid password errors received

In this situation, Data Protection for VMware command-line interface, Data Protection for VMware vSphere GUI, or Data Protection for VMware vCloud GUI operations failed and your log file contained one (or more) of these errors:

GVM1170E: A VMCLI command failed.

ANR2177I FRSV123015.TSMCLI has 1 invalid sign-on attempts. The limit is 5.

ANR0424W Session 125713 for node FRSV128215.TSMCLI (TSM4VE) 
refused - invalid password submitted.

This issue can occur when a VMCLI node set password attempt was made by using a password that was previously set.

When the VMCLI node password is set, the password is registered and then automatically changed. As a result, you do not know this changed password. If you must change the VMCLI node password after it is already set, the Tivoli Storage Manager Administrator must use the update node command to reset this node on the Tivoli Storage Manager server. When the node has been reset, set the VMCLI node password again and specify the new password.

The VMCLI node password is set by either of these methods:

Data Protection for VMware command-line interface: vmcli -f set_password command
Data Protection for VMware vSphere GUI: Configuration > Tasks > Edit Configuration > VMCLI Node
Data Protection for VMware vCloud GUI: Configuration > Tasks > Edit Configuration > VMCLI Node

Session timeout

The Tivoli Storage Manager server COMMTIMEOUT option affects the duration of the Data Protection for VMware session. If the processing time of the Data Protection for VMware operation exceeds this value, the server ends the session with Data Protection for VMware. Therefore, if you are sure that no error has occurred during a Data Protection for VMware operation and the COMMTIMEOUT value has been reached, increase the value. Likewise, if an error occurred but Data Protection for VMware did not report the error in a timely manner, then decrease the value for better real-time reporting.

vSphere Client displays a Data Protection for VMware vSphere GUI connection error

In this situation, the Data Protection for VMware vSphere GUI was installed successfully on the vCenter node. However, when viewing the Data Protection for VMware vSphere GUI in the vSphere Client Plug-in Manager (Menu > Plug-ins > Manage Plug-ins), the following message is shown:

Tivoli Data Protection for VMware
IBM Tivoli Storage Manager 2.1
Disabled
Tivoli Data Protection for VMware Plug-in
The following error occured while downloading the script plugin from
http://finapps.mycompany.com:9080/TsmVMwareUI/plugin/config.xml:
The request failed because of a connection failure. (Unable to connect 
to the remote server)

This issue can occur when the web server cannot access ports specified in the bootstrap.properties file. During installation, the web server accesses port 9080. However, when these additional ports cannot be accessed, the vSphere Client cannot connect to the Data Protection for VMware vSphere GUI.

Complete these tasks to resolve this issue:

On the system where the Data Protection for VMware vSphere GUI is installed, go to the following directory for your operating system:
```
/opt/tivoli/tsm/tdpvmware/common/webserver/usr/servers/veProfile
```
```
C:\IBM\tivoli\tsm\tdpvmware\webserver\usr\servers\veProfile
```

Open the bootstrap.properties file to view the eWAS port settings. For example:

veProfile.http.port=9080
veProfile.https.port=9081
veProfile.keystore.pswd={xor}KzosK2spOg==

Issue the netstat -a command to view all active connections and ports. Compare the results with the port settings in the bootstrap.properties file. If any of the ports (specified in the bootstrap.properties file) are not open, complete either of the following tasks to resolve this connection issue:
- Open the closed ports.
- Update the bootstrap.properties file with ports that are available.

Restart the GUI web server:

1. Click Start > Control Panel > Administrative Tools > Services.
2. Right-click Data Protection for VMware Web Server Service and click Restart.

Issue the following command as root:

[root@bold4win /]# service webserver restart

Restarting  the WebSphere Liberty Profile ...
Stopping server veProfile.
Server veProfile stopped.
Starting server veProfile.
Server veProfile started with process ID 17894.

Start the Data Protection for VMware vSphere GUI with either of these methods:
- Click the Data Protection for VMware vSphere GUI icon in the Solutions and Applications window of the vSphere Client.
- Open a web browser and go to the GUI web server. For example:
```
https://finapps.mycompany.com:9081/TsmVMwareUI/
```
  Login by using the vCenter user name and password.

Resolving a VM guest (with application protection) backup failure

In this situation, a backup (with application protection) of a guest machine is stopped by the user. When the Tivoli Storage Manager backup-archive client backup process (dsmagent or dsmc) ends in this manner, the cleanup of the application protection does not take place. As a result, the next backup (with application protection) of the same guest machine can be issued only after a 10 minute interval. This interval is the length of time necessary for the process to recognize that the guest machine is not backed up.

To manually clean up application protection without waiting 10 minutes for communication to clear, complete these steps:

Log on to the guest machine with the same user ID and password that was entered when issuing the backup operation.
Open a command prompt window and issue this command:
```
echo %TEMP%
```
Go to the %TEMP% directory, then change to the Tivoli Storage Manager directory. For example:
```
C:\Users\Administrator\AppData\Local\Temp\TSM
```
Delete the BackupHeartBeat.txt file.
Back up the guest machine.

Event log contains `event ID 8194, VSS` message

After a backup of a VM guest with application protection completes, the event log contains the event ID 8194, VSS error message. This cause of this message is an incorrect security setting in the Volume Shadow Copy Service (VSS) writer or requestor process.

To resolve this error, complete these steps:

Log on to the VM guest as an administrator and run the Microsoft dcomcnfg.exe utility in the Start > Run dialog:
```
dcomcnfg.exe
```
Click OK.
The dcomcnfg.exe utility is used to modify registry settings.
In the Component Services interface, go to Component Services > Computers. Right-click My Computer and select Properties.
In the My Computer properties panel, go to COM Security > Access Permissions: Edit Default.
In the Access Permission panel, add the Network Service account with Local Access permission set to Allow.
Apply your changes and close all open Component Services panels.
Restart the VM guest.
Back up the VM guest and verify that the event ID 8194, VSS error message is not issued to the event log.

Data Protection for VMware installation failure: deployment engine initialization

The Data Protection for VMware installation might be stopped due to a deployment engine initialization failure due to .lock files. If the deployment engine interferes with the Data Protection for VMware installation, the following error message is produced:

Deployment Engine failed to initialize.
The installer will now shutdown. Please check with the log files for a more
complete description of the failure.
PRESS ENTER TO CONTINUE:

The cause might be deployment engine .lock files that come from a concurrent installation that is running or from an installation that stopped before completing. If another installation is running, wait until that installation finishes before installing Data Protection for VMware. If there are no other installations running and you encounter this problem, delete any .lock files.

Important: Do not delete any .lock files if there are other Data Protection for VMware installations running.

To delete .lock files on Windows, issue the following command:

cd C:\Program Files\IBM\Common\acsi\logs
del .lock*

To delete .lock files on Linux, issue the following command:

cd /usr/ibm/common/acsi/logs
rm .lock*

After removing these files, restart the installation.

Unsupported characters in VM or datacenter name

Data Protection for VMware does not support backing up VMs or datacenters that contain any of the following characters in their name:

": Double quotation mark
': Single quotation mark
:: Colon
;: Semicolon
*: Asterisk
?: Question mark
,: Comma
<: Less than sign
>: Greater than sign
/: Forward slash
\: Backward slash
|: Vertical bar

Issues encountered after changing the vCenter

After changing the vCenter in the Data Protection for VMware vSphere GUI, the following two issues might occur:

A data center associated with the new vCenter does not appear on the Configuration Status page.
To resolve this issue, manually set the domain for the new vCenter. See Set_domain for details about issuing this command.
In the Restore tab, an ESX host (associated with a previous data center) displays under a new data center within the new vCenter. This issue is a known limitation. There is currently not a workaround for this issue

Consolidating VM backups

After a VM backup, the VM might contain preexisting snapshots even though no snapshots are present in the Snapshot Manager. For example, the VM hard disk points to snapshot VMDK files (for example *-000001.vmdk) instead of regular VMDK files. Although preexisting snapshots might be intentionally retained, Data Protection for VMware does not provide a mechanism to verify whether the VMDK points to a valid snapshot. When snapshots are not consolidated, and a VM with preexisting snapshot files is backed up, Data Protection for VMware might report an incorrect size for the backup on the Tivoli Storage Manager server. Snapshot consolidation also prevents other VMware related issues. As a result, consolidate your snapshots whenever this situation occurs.

To resolve this potential problem, VMware vSphere Client 5.x (or later) notifies you when a VM requires snapshot consolidation. Details about the consolidate option are provided in the following VMware KB article:
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2003638

For vSphere 4.1 (or earlier), no consolidation option is available. However, the equivalent task is to create a snapshot and then complete a Delete All action.

Re-register the Data Protection for VMware vSphere GUI

To use the Data Protection for VMware vSphere GUI as a vSphere Client plug-in, it must be registered to the vCenter. When the IP address changes on the system where the Data Protection for VMware vSphere GUI is installed, you must re-register the Data Protection for VMware vSphere GUI. If you cannot find the Data Protection for VMware vSphere GUI in the plug-in Manager, re-register the Data Protection for VMware vSphere GUI. Use this procedure when the IP address changes or you did not enable GUI access as a plug-in during installation:

Log in to the system where the Data Protection for VMware vSphere GUI is installed.
Go to the following directory:
/opt/tivoli/tsm/tdpvmware/common/scripts

(64-bit) C:\Program Files (x86)\Common Files\Tivoli\TDPVMware\VMwarePlugin\

To register the Data Protection for VMware vSphere GUI, enter the command:
Linux operating systems

../jre/jre/bin/java -jar reg.jar vCenterServer vCenterUser 
vCenterPassword new_guiHostnameShort new_guiHostnameFull 9080

The new_guiHostnameShort value is the name that is shown for the Data Protection for VMware vSphere GUI icon in the vCenter.
The new_guiHostnameFull value is either the full DNS hostname or the numeric IP address. For example:

../jre/jre/bin/java -jar reg.jar vctr.archive.mycompany.com administrator 
pass4admin myguihost myguihost.mydomain.mycompany.com 9080

register_vcenter.cmd vCenterServer vCenterUser 
vCenterPassword 9080

A config.xml file is created.

Copy the config.xml file to the following directory:
/opt/tivoli/tsm/tdpvmware/common/webserver/usr/servers/veProfile/apps/tsmVmGUI.war/plugin/

C:\IBM\tivoli\tsm\tdpvmware\webserver\usr\servers\veProfile\apps\tsmVmGUI.war\plugin\
Restart the GUI web server:
- Issue the following command as root:
```
[root@bold4win /]# service webserver restart
```
- 1. Click Start > Control Panel > Administrative Tools > Services.
  2. Right-click Data Protection for VMware Web Server Service and click Restart.
Restart the vSphere Client and confirm that the Data Protection for VMware vSphere GUI is now registered.

An error occurred while making the Web server request (GVM0103E)

In this situation, the Data Protection for VMware vSphere GUI or Data Protection for VMware vCloud GUI shows the following error when you tried to access it:

GVM0103E: An error occurred while making the Web server request. If this error persists,
check the network connection with the Web server and verify that the Web server is running.

To prevent this error, make sure that the following conditions exist before you start the GUI:

The Data Protection for VMware vSphere GUI is installed on a system that meets the operating system prerequisites. It must have network connectivity to the following systems:
- vStorage Backup Server
- Tivoli Storage Manager server
- vCenter Server (Data Protection for VMware vSphere GUI)
- vCloud Director (Data Protection for VMware vCloud GUI)
The Data Protection for VMware vSphere GUI host URL address must be set in your Internet Explorer trusted sites zone. In the Internet Explorer menu bar, go to Tools > Internet Options > Security > Trusted sites. Click Sites and add the host URL address. Make sure to apply your changes. For example:
```
Add this website to the zone:http://myvctrmachine.xyzco.com
```
For vSphere Web Client 5.1, when you are using Internet Explorer on Windows Server 2008 or Windows 2012 Server, Enhanced Security Configuration (IE ESC) must be disabled. Go to the Server Manager and make sure that IE ESC is set to Off.
On Windows Server 2008, if you start the Data Protection for VMware vSphere GUI and then add the address as a trusted site, a JavaScript error denies continued access to the Data Protection for VMware vSphere GUI. To resolve this issue, close and restart the Data Protection for VMware vSphere GUI.

Data Protection for VMware vSphere GUI does not display correctly in the VMware vCenter

In this situation, the Data Protection for VMware vSphere GUI is registered as a vCenter plug-in. However, the Data Protection for VMware vSphere GUI icon does not display in the Solutions and Applications window of the vSphere Client. When you view the Data Protection for VMware vSphere GUI in the vSphere Client Plug-in Manager (Menu > Plug-ins > Manage Plug-ins), the following message is shown:

The following error occured while downloading the script plugin
from http://192.168.66.58:9080/TsmVMwareUI/plugin/config.xml: 
The request failed because of a connection failure.
(Unable to connect to the remote server)

To resolve this error, complete the following steps:

Check the network to verify that there is a connection to the web server host.

If the URL is incorrect, manually register the Data Protection for VMware vSphere GUI as a plug-in:

(x86) Open a command prompt and go to the following directory:

C:\Program Files\Common Files\Tivoli\TDPVMware\VMwarePlugin

Issue the following command:

C:\Windows\System32\\cmd.exe /c register_vcenter.cmd "vcenter hostname or IP" 
"vcenter username" "vcenter password" "web server port" "VE proxy machine IP"

(x64) Open a command prompt and go to the following directory:

C:\Program Files (x86)\Common Files\Tivoli\TDPVMware\VMwarePlugin