A few weeks ago we had an ISCSI problem on our OLVM cluster affecting only one node - a SHE storage migration went wrong. As a result there were intermittent outages to one of our ISCSI hosts which affected one Storage Domain.
During this time our backup software Commvault created a number of snapshots. Well it instructed OLVM to do so. OLVM gladly created all the metadata but was unable to create the actual storage.
Now is comes time to delete the snapshots OLVM cannot since the storage isn't there. So the postgres entries are put in illegal status.
There only appear to be two options: Clean up the metadata or recreate the storage.
Both I am hesitant to do. First attempt at option 1 did not end well.
Some guidance would be appreciated.
AY