Recently we added a couple of hosts to our VMware environment and noticed that 10 out of 21 datastores would not mount upon reboot.
VMware gave us the following solution.
run esxcli storage vmfs snapshot mount -l <datastore_name>The -l in this command stands for Persistent-mount which according to VMwares documentation means mount persistently across reboots
This was not the case for us as rebooting still resulted in these 10 datastores from being seen and VMware had no solution to the problem other than taking downtime to perform datastore resignatures which ended up to be the only viable option after almost 2 months waiting for VMware to fix their persistent-mount command. THE SOLUTION WAS AS FOLLOWS:
The final solution was to resignature the VMFS volumes using the following command.esxcli storage vmfs snapshot resignature -l=<datastore_name>
PLEASE NOTE - this command needs to be run on a datastore with NO RUNNING VMS!
1. Shutdown VMs running on datastore
2. On problem host run esxcli storage vmfs snapshot list
....to verify it is seen as a snapshot volumeExample:
Volume Name: ESXSshare-06
VMFS UUID: 49708f15-345694dc-877a-00215e521038
Can mount: true
Reason for un-mountability:
Can resignature: true
Reason for non-resignaturability:
Unresolved Extent Count: 1
3. Remove any VMs from inventory
4. Unmount datastore from ALL hosts FIRST before performing maintainance
5. Perform resignature
1. Example - esxcli storage vmfs snapshot resignature -l=ESXSshare-06
6. In vSphere datastore view rename the datastore back to its original name
7. Rescan for VMFS volumes on ALL hosts
8. Validate datastore is seen from ALL hosts tab in the datastore viewNow this is where it gets a little tricky.
The datastore will now automatically mount but every single VM that was removed from inventory now has pointers to the disks old signature in their .VMX file. To fix this you need to browse the datastore and download the .VMX file locally. Open it with Notepad or Notepad++ (preferred utility) and change the path to the vmdk files.Example:
In the .VMX file find the following line(s)......
sched.swap.derivedName = "/vmfs/volumes/49708cc0-68198024-4582-00215e521038
Change the above UUID in the filepath to the NEW UUID that was changed. This can be found a few ways but I like to SSH to the host and cd to the /volumes directory then ls -l to list the volumes and their mounts, the mounts are shown as DATASTORE_NAME->UUID.
lrwxr-xr-x 1 root root 35 Apr 8 18:58ESXFshare-01 -> 551c95f2-989d58c6-d057-40f2e987aa40
lrwxr-xr-x 1 root root 35 Apr 8 18:58 ESXFshare-02 -> 551c94a1-1ade708e-24ce-40f2e987aa40
lrwxr-xr-x 1 root root 35 Apr 8 18:58 ESXFshare-03 -> 551c95b0-82cded88-fe13-40f2e987aa40
Once this step is carefully completed, move onto step #9.
9. Add back VMs to inventory
10. Bring up VMs that were originally shutdown
If none of the above steps work please respond to this post. If you are not a member join already, ITS FREE!