I wanted to create a new VM. Then I saw that I can't connect via the vSphere Client.
After checking via SSH I restarted the server.
Then I can connect to the machine, but after few machines started its really slow.
So I going to debug it.
[root@static:~] dmesg
2016-04-08T13:58:41.969Z cpu5:32921)<6>ahci_scsi_abort: cmd 0x28 (0x439d8182b840), entering...
2016-04-08T13:58:41.969Z cpu5:32921)<7>ata2: ahci_port_reset, entering...
2016-04-08T13:58:44.207Z cpu5:32921)<6>ata2: ahci_port_reset, hard reseting port
2016-04-08T13:58:58.961Z cpu2:32809)ScsiDeviceIO: 4889: Waited for completion for all issued commands for partition t10.ATA_____ST3000DM0012D9YN166__________________________________Z1F0YF30:1, for 120 secs. 3 completions still awaited. Giving up !
2016-04-08T13:59:06.489Z cpu2:32922)VSCSI: 2966: Retry 20 on handle 8192 still in progress after 62 seconds
2016-04-08T13:59:07.489Z cpu6:32797)NMP: nmp_ThrottleLogForDevice:3178: Cmd 0x12 (0x439d8065a040, 0) to dev "t10.ATA_____ST3000DM0012D9YN166__________________________________Z1F0YF30" on path "vmhba32:C0:T0:L0" Failed: H:0x5 D:0x0 P:0x0 Possible sense d$
2016-04-08T13:59:07.489Z cpu6:32797)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "t10.ATA_____ST3000DM0012D9YN166__________________________________Z1F0YF30" state in doubt; requested fast path state update...
2016-04-08T13:59:07.489Z cpu6:32797)ScsiDeviceIO: 2646: Cmd(0x439d8065a040) 0x12, CmdSN 0xa8c from world 0 to dev "t10.ATA_____ST3000DM0012D9YN166__________________________________Z1F0YF30" failed H:0x5 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
2016-04-08T13:59:07.489Z cpu6:36458)VMW_SATP_LOCAL: satp_local_updatePathStates:458: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry
2016-04-08T13:59:47.491Z cpu6:32797)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "t10.ATA_____ST3000DM0012D9YN166__________________________________Z1F0YF30" state in doubt; requested fast path state update...
2016-04-08T13:59:47.491Z cpu6:32797)ScsiDeviceIO: 2646: Cmd(0x439d8065a040) 0x12, CmdSN 0xa8d from world 0 to dev "t10.ATA_____ST3000DM0012D9YN166__________________________________Z1F0YF30" failed H:0x5 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
2016-04-08T13:59:48.489Z cpu1:37826)VMW_SATP_LOCAL: satp_local_updatePathStates:458: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry
2016-04-08T13:59:49.689Z cpu5:32921)<6>ata2: ahci_port_reset: FAILED
2016-04-08T13:59:49.689Z cpu5:32921)<6>ahci_scsi_abort: cmd 0x28 (0x439d8182b840), FAILED
2016-04-08T13:59:49.689Z cpu5:32921)WARNING: LinScsi: SCSILinuxAbortCommands:1837: Failed, Driver ahci, for vmhba32
2016-04-08T13:59:49.689Z cpu5:32921)WARNING: VSCSIFs: 352: handle 8192(vscsi4:0):Failed reset of virtual target 1442670, (initiator 0x43025c9893c0, 0): Failure
2016-04-08T13:59:49.689Z cpu5:32921)VSCSI: 2841: handle 8192(vscsi4:0):Reset [Retries: 21/0] from (vmm0:Router-VM)
2016-04-08T13:59:49.689Z cpu5:32921)<6>ahci_scsi_abort: cmd 0x28 (0x439d8182b840), entering...
2016-04-08T13:59:49.689Z cpu5:32921)<7>ata2: ahci_port_reset, entering...
2016-04-08T13:59:51.929Z cpu0:32921)<6>ata2: ahci_port_reset, hard reseting port
2016-04-08T13:59:57.489Z cpu3:33151)NMP: nmp_ResetDeviceLogThrottling:3345: last error status from device t10.ATA_____ST3000DM0012D9YN166__________________________________Z1F0YF30 repeated 1 times
2016-04-08T14:00:07.489Z cpu6:32922)VSCSI: 2966: Retry 21 on handle 8192 still in progress after 56 seconds
2016-04-08T14:00:29.489Z cpu0:38005)VMW_SATP_LOCAL: satp_local_updatePathStates:458: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry
2016-04-08T14:00:57.367Z cpu1:32921)<6>ata2: ahci_port_reset: FAILED
2016-04-08T14:00:57.367Z cpu1:32921)<6>ahci_scsi_abort: cmd 0x28 (0x439d8182b840), FAILED
2016-04-08T14:00:57.367Z cpu1:32921)WARNING: LinScsi: SCSILinuxAbortCommands:1837: Failed, Driver ahci, for vmhba32
2016-04-08T14:00:57.367Z cpu1:32921)WARNING: VSCSIFs: 352: handle 8192(vscsi4:0):Failed reset of virtual target 1442670, (initiator 0x43025c9893c0, 0): Failure
2016-04-08T14:00:57.367Z cpu1:32921)VSCSI: 2841: handle 8192(vscsi4:0):Reset [Retries: 22/0] from (vmm0:Router-VM)
2016-04-08T14:00:57.367Z cpu1:32921)<6>ahci_scsi_abort: cmd 0x28 (0x439d8182b840), entering...
2016-04-08T14:00:57.367Z cpu1:32921)<7>ata2: ahci_port_reset, entering...
2016-04-08T14:00:59.605Z cpu1:32921)<6>ata2: ahci_port_reset, hard reseting port
2016-04-08T14:01:01.179Z cpu2:32809)ScsiDeviceIO: 4889: Waited for completion for all issued commands for partition t10.ATA_____ST3000DM0012D9YN166__________________________________Z1F0YF30:1, for 120 secs. 3 completions still awaited. Giving up !
2016-04-08T14:01:07.491Z cpu7:32922)VSCSI: 2966: Retry 22 on handle 8192 still in progress after 48 seconds
2016-04-08T14:02:05.075Z cpu4:32921)<6>ata2: ahci_port_reset: FAILED
2016-04-08T14:02:05.075Z cpu4:32921)<6>ahci_scsi_abort: cmd 0x28 (0x439d8182b840), FAILED
2016-04-08T14:02:05.075Z cpu4:32921)WARNING: LinScsi: SCSILinuxAbortCommands:1837: Failed, Driver ahci, for vmhba32
2016-04-08T14:02:05.075Z cpu4:32921)WARNING: VSCSIFs: 352: handle 8192(vscsi4:0):Failed reset of virtual target 1442670, (initiator 0x43025c9893c0, 0): Failure
2016-04-08T14:02:05.075Z cpu4:32921)VSCSI: 2841: handle 8192(vscsi4:0):Reset [Retries: 23/0] from (vmm0:Router-VM)
2016-04-08T14:02:05.075Z cpu4:32921)<6>ahci_scsi_abort: cmd 0x28 (0x439d8182b840), entering...
2016-04-08T14:02:05.075Z cpu4:32921)<7>ata2: ahci_port_reset, entering...
2016-04-08T14:02:07.315Z cpu0:32921)<6>ata2: ahci_port_reset, hard reseting port
2016-04-08T14:02:08.491Z cpu3:32922)VSCSI: 2966: Retry 23 on handle 8192 still in progress after 41 seconds
2016-04-08T14:03:03.509Z cpu0:32809)ScsiDeviceIO: 4889: Waited for completion for all issued commands for partition t10.ATA_____ST3000DM0012D9YN166__________________________________Z1F0YF30:1, for 120 secs. 3 completions still awaited. Giving up !
2016-04-08T14:03:09.491Z cpu7:32922)VSCSI: 2966: Retry 23 on handle 8192 still in progress after 102 seconds
2016-04-08T14:03:12.757Z cpu6:32921)<6>ata2: ahci_port_reset: FAILED
2016-04-08T14:03:12.757Z cpu6:32921)<6>ahci_scsi_abort: cmd 0x28 (0x439d8182b840), FAILED
2016-04-08T14:03:12.757Z cpu6:32921)WARNING: LinScsi: SCSILinuxAbortCommands:1837: Failed, Driver ahci, for vmhba32
2016-04-08T14:03:12.757Z cpu6:32921)WARNING: VSCSIFs: 352: handle 8192(vscsi4:0):Failed reset of virtual target 1442670, (initiator 0x43025c9893c0, 0): Failure
2016-04-08T14:03:12.757Z cpu6:32921)VSCSI: 2841: handle 8192(vscsi4:0):Reset [Retries: 24/0] from (vmm0:Router-VM)
2016-04-08T14:03:12.757Z cpu6:32921)<6>ahci_scsi_abort: cmd 0x28 (0x439d8182b840), entering...
2016-04-08T14:03:12.757Z cpu6:32921)<7>ata2: ahci_port_reset, entering...
2016-04-08T14:03:14.997Z cpu6:32921)<6>ata2: ahci_port_reset, hard reseting port
2016-04-08T14:04:07.489Z cpu5:36458)VMW_SATP_LOCAL: satp_local_updatePathStates:458: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry
2016-04-08T14:04:07.489Z cpu6:32996)ScsiCore: 97: Stopping taskMgmt handler world 329962
2016-04-08T14:04:07.489Z cpu6:32797)NMP: nmp_ThrottleLogForDevice:3178: Cmd 0x12 (0x439d8060c840, 0) to dev "t10.ATA_____ST3000DM0012D9YN166__________________________________Z1F0YF30" on path "vmhba32:C0:T0:L0" Failed: H:0x5 D:0x0 P:0x0 Possible sense d$
2016-04-08T14:04:07.489Z cpu6:32797)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "t10.ATA_____ST3000DM0012D9YN166__________________________________Z1F0YF30" state in doubt; requested fast path state update...
2016-04-08T14:04:07.489Z cpu6:32797)ScsiDeviceIO: 2646: Cmd(0x439d8060c840) 0x12, CmdSN 0xa9c from world 0 to dev "t10.ATA_____ST3000DM0012D9YN166__________________________________Z1F0YF30" failed H:0x5 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
2016-04-08T14:04:10.489Z cpu6:32922)VSCSI: 2966: Retry 24 on handle 8192 still in progress after 95 seconds
2016-04-08T14:04:20.467Z cpu4:32921)<6>ata2: ahci_port_reset: FAILED
2016-04-08T14:04:20.467Z cpu4:32921)<6>ahci_scsi_abort: cmd 0x28 (0x439d8182b840), FAILED
2016-04-08T14:04:20.467Z cpu4:32921)WARNING: LinScsi: SCSILinuxAbortCommands:1837: Failed, Driver ahci, for vmhba32
2016-04-08T14:04:20.467Z cpu4:32921)WARNING: VSCSIFs: 352: handle 8192(vscsi4:0):Failed reset of virtual target 1442670, (initiator 0x43025c9893c0, 0): Failure
2016-04-08T14:04:20.467Z cpu4:32921)VSCSI: 2841: handle 8192(vscsi4:0):Reset [Retries: 25/0] from (vmm0:Router-VM)
2016-04-08T14:04:20.467Z cpu4:32921)<6>ahci_scsi_abort: cmd 0x28 (0x439d8182b840), entering...
2016-04-08T14:04:20.467Z cpu4:32921)<7>ata2: ahci_port_reset, entering...
2016-04-08T14:04:22.707Z cpu3:32921)<6>ata2: ahci_port_reset, hard reseting port
2016-04-08T14:04:37.489Z cpu5:39922)ScsiCore: 62: Starting taskmgmt handler world 39922/3
2016-04-08T14:04:39.489Z cpu1:39923)ScsiCore: 62: Starting taskmgmt handler world 39923/4
2016-04-08T14:04:39.489Z cpu2:32793)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "t10.ATA_____ST3000DM0012D9YN166__________________________________Z1F0YF30" state in doubt; requested fast path state update...
2016-04-08T14:04:39.489Z cpu2:32793)ScsiDeviceIO: 2646: Cmd(0x439d80696d00) 0x9e, CmdSN 0xa9d from world 0 to dev "t10.ATA_____ST3000DM0012D9YN166__________________________________Z1F0YF30" failed H:0x5 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
2016-04-08T14:04:47.491Z cpu0:39923)ScsiDeviceIO: 2646: Cmd(0x439d8060c840) 0x12, CmdSN 0xa9e from world 0 to dev "t10.ATA_____ST3000DM0012D9YN166__________________________________Z1F0YF30" failed H:0x5 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
2016-04-08T14:04:47.491Z cpu6:39302)VMW_SATP_LOCAL: satp_local_updatePathStates:458: Failed to update path "vmhba32:C0:T0:L0" state. Status=Transient storage condition, suggest retry
But ok. In the rescue mode I can mount it and copy it to another place if it's broken. And I have backups for the case of cases.
After searching how to fix ESXi problems I found this community. There is a tool called "voma" but I can't fix something with this - so that I not used this now.
Here is the hard disk list.