I am running Solaris 11 x86. I had a zpool which was running on 2 mirrored 3TB disks. Ok, the short story is, some of zfs file systems on this pool have permanent errors, including the root zfs filesystem, which means I'v lost the whole pool, about 2TB data.
The following is the status of my pool:
root@solaris:~# zpool status -v dps
pool: dps
state: DEGRADED
status: One or more devices has experienced an error resulting in data
corruption. Applications may be affected.
action: Restore the file in question if possible. Otherwise restore the
entire pool from backup.
see: http://support.oracle.com/msg/ZFS-8000-8A
scan: resilvered 442K in 0h0m with 551 errors on Thu Sep 12 00:11:57 2013
config:
NAME STATE READ WRITE CKSUM
dps DEGRADED 0 0 12
c4t1d0 DEGRADED 0 0 24
device details:
c4t1d0 DEGRADED too many errors
status: FMA has degraded this device.
action: Run 'fmadm faulty' for more information. Clear the errors
using 'fmadm repaired'.
see: http://support.oracle.com/msg/ZFS-8000-GH for recovery
errors: Permanent errors have been detected in the following files:
dps/Sharepoint/VirtualDisks:<0x0>
dps:<0x0>
dps:/VirtualBoxDisks/xppro64.vdi
dps/Media:<0x0>
dps/Sharepoint:<0x0>
And here is the long story, one day my server hung and I had to force it powered off and when it started up again, I saw an error said one of the file on that zfs has permanent error on one of the disks. Since I had mirror disks, I tried to fix it by resilvering the disks. And the pool was still in service during the resilvering. After a few hours, I found that the resilvering seemed got stuck and more errors appeared. I then detached one disk from the mirror in order to preserve a copy from more damaging. But later, the detached disk became completely useless, "zpool import" couldn't find any pool info on that disk. I tried my best to recover it by exporting-importing the pool, rebooting the system, etc. But unfortunately thing get worse and worse, and end up with loosing the whole pool.
Does any one here knows if this pool is still recoverable fully or partially? Any suggestion what I should do next? Any input would be greatly appreciated.