Last week I needed to update vRSLCM to version 8.12 because of a vRLI update. After the successful precheck I started the vRSLCM update and it was stuck for hours. Absolutely no progress. So I started troubleshooting.
Troubleshooting
Looking into /var/log/vrlcm/vmware_vrlcm.log I noticed the following reoccurring informational message
2023-04-26 11:03:29.366 INFO [http-nio-8080-exec-9] c.v.v.l.u.ShellExecutor - -- Executing shell command: /opt/vmware/bin/vamicli update --progress --detail
2023-04-26 11:03:29.366 INFO [http-nio-8080-exec-9] c.v.v.l.u.ProcessUtil - -- Execute /opt/vmware/bin/vamicli
2023-04-26 11:03:29.432 INFO [http-nio-8080-exec-9] c.v.v.l.u.ShellExecutor - -- Result: [257 packages have been downloaded.
Installing...].
Looking into /opt/vmware/var/log/vami/vami.log, the following error was visible
01/05/2023 11:45:11 [INFO] Downloaded file. url=/package-pool/zstd-1.4.5-2.ph3.x86_64.rpm
01/05/2023 11:45:11 [INFO] Updating package zstd-devel status in /opt/vmware/var/lib/vami/update/data/update_progress.json
01/05/2023 11:45:11 [INFO] Downloaded file. url=/package-pool/zstd-devel-1.4.5-2.ph3.x86_64.rpm
01/05/2023 11:45:11 [INFO] Creating install script files for updatecli
01/05/2023 11:45:11 [INFO] Updating Phase in /opt/vmware/var/lib/vami/update/data/update_progress.json
01/05/2023 11:45:11 [INFO] Using update pre-install script
01/05/2023 11:45:11 [ERROR] Unable to install updates because package files are missing.
So there is an instant error after the download of the packages, but the webinterface was still showing “installing…”.
Unfortunately I don’t have a screenshot of this, but the above logging (vmware_vrlcm.log) shows the continuously “Installing …” progress.
This error led to this KB article (803107). Which is a really joyful fix. You’ll need to:
- Revert to snapshot
- Enable debug mode
- Run the upgrade again
- Fetch the information about the missing package
- Revert to snapshot
- Enable debug mode
- Get the missing packages:
- Mount ISO / Browse ISO
- Copy files using SCP to the vRSLCM appliance (/data/packages/.
- Prepare a copy command:
cp -a /data/packages/. /opt/vmware/var/lib/vami/update/data/package-pool/package-pool/
- Run the upgrade again
- Fire the copy command during the “download” of the packages phase.
In my case the following package was missing:
user@servername [ ~ ]# grep -i missing /opt/vmware/var/log/vami/vami.log
26/08/2022 15:39:55 [ERROR] Unable to install updates because package files are missing.
01/05/2023 12:18:19 [DEBUG] Stat file, file missing. filename=/opt/vmware/var/lib/vami/update/data/package-pool/package-pool/PyYAML-5.4.1-2.ph3.x86_64.rpm
01/05/2023 12:18:19 [ERROR] Unable to install updates because package files are missing.
Although the workaround / fix is time consuming (with two reverts) the upgrade finally actually started and succeeded after copying the missing package.
Update 20 February 2024
Since there are some questions about where to find the file, here is some additional info about that.
The ISO from my.vmware.com, update file highlighted below.
Default view when mounting in Windows Explorer (8.3. format), might be confusing:
When using archiving software (7zip):
We are unable to get the mentioned file in ISO image. From where did you get it? Can you please help with the source to obtain this file.
Hi Pratik,
When you’ve downloaded the ISO from VMware support. Mount it and the (rpm) files will be in the update\package-pool folder.
Hi,
I am facing the same issue as well however the PyYAML-5.4.1-2.ph3.x86_64.rpm file is not found even after I mount the iso?
Can I check how were you able to get it?
Hi Terence,
Don’t know if you’re still looking for an answer. But as said it is within the ISO, in the update\package-pool folder. If you’ll use the default Windows Explorer to extract / open the file, than all files might be shown in the 8.3 format and that could be confusing. Use 7zip or other extraction /archiving tool to view the proper file names within the ISO.
What troubleshooting steps can be taken when encountering the “installing…” progress error message in the VMware web interface, as described in the logging file (vmware_vrlcm.log)? Regard Telkom University
Have a look into: /opt/vmware/var/log/vami/vami.log