Hi,
I have a datacenter consisting of 12 hosts out of which 1 host is not showing the LUNs and hence the datastores. While performing the rescan function in the storage adapters in vcenter, I'm getting the following error in vmkernel logs.
Nov 26 23:26:51 vmwarehost11 vmkernel: 22:05:14:47.499 cpu7:4103)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe: NMP device "naa.60050768018e82d71800000000000221" state in doubt; requested fast path state update...
Nov 26 23:26:51 vmwarehost11 vmkernel: 22:05:14:47.499 cpu7:4103)ScsiDeviceIO: 747: Command 0x28 to device "naa.60050768018e82d71800000000000221" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
Nov 26 23:26:51 vmwarehost11 vmkernel: 22:05:14:47.803 cpu7:4103)NMP: nmp_CompleteCommandForPath: Command 0x28 (0x4100071ead40) to NMP device "naa.60050768018e82d71800000000000221" failed on physical path "vmhba1:C0:T0:L5" H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
Nov 26 23:26:51 vmwarehost11 vmkernel: 22:05:14:47.803 cpu7:4103)ScsiDeviceIO: 747: Command 0x28 to device "naa.60050768018e82d71800000000000221" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
Nov 26 23:26:52 vmwarehost11 vmkernel: 22:05:14:48.107 cpu7:4103)NMP: nmp_CompleteCommandForPath: Command 0x28 (0x4100071ead40) to NMP device "naa.60050768018e82d71800000000000221" failed on physical path "vmhba1:C0:T0:L5" H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
Nov 26 23:26:52 vmwarehost11 vmkernel: 22:05:14:48.107 cpu7:4103)ScsiDeviceIO: 747: Command 0x28 to device "naa.60050768018e82d71800000000000221" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
Nov 26 23:26:52 vmwarehost11 vmkernel: 22:05:14:48.410 cpu7:4103)NMP: nmp_CompleteCommandForPath: Command 0x28 (0x4100071ead40) to NMP device "naa.60050768018e82d71800000000000221" failed on physical path "vmhba1:C0:T0:L5" H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
Nov 26 23:26:52 vmwarehost11 vmkernel: 22:05:14:48.410 cpu7:4103)ScsiDeviceIO: 747: Command 0x28 to device "naa.60050768018e82d71800000000000221" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
Nov 26 23:26:52 vmwarehost11 vmkernel: 22:05:14:48.714 cpu7:4103)NMP: nmp_CompleteCommandForPath: Command 0x28 (0x4100071ead40) to NMP device "naa.60050768018e82d71800000000000221" failed on physical path "vmhba1:C0:T0:L5" H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
Nov 26 23:26:52 vmwarehost11 vmkernel: 22:05:14:48.714 cpu7:4103)ScsiDeviceIO: 747: Command 0x28 to device "naa.60050768018e82d71800000000000221" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
Nov 26 23:26:53 vmwarehost11 vmkernel: 22:05:14:49.018 cpu7:4103)NMP: nmp_CompleteCommandForPath: Command 0x28 (0x4100071ead40) to NMP device "naa.60050768018e82d71800000000000221" failed on physical path "vmhba1:C0:T0:L5" H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
I have checked the various commands. their outputs are pasted below:
[root@vmwarehost11 log]# esxcli nmp device list
naa.60050768018e82d7180000000000021b
Device Display Name: IBM Fibre Channel Disk (naa.60050768018e82d7180000000000021b)
Storage Array Type: VMW_SATP_SVC
Storage Array Type Device Config:
Path Selection Policy: VMW_PSP_FIXED
Path Selection Policy Device Config: {preferred=vmhba1:C0:T0:L2;current=vmhba1:C0:T0:L2}
Working Paths: vmhba1:C0:T0:L2
naa.60050768018e82d7180000000000021c
Device Display Name: IBM Fibre Channel Disk (naa.60050768018e82d7180000000000021c)
Storage Array Type: VMW_SATP_SVC
Storage Array Type Device Config:
Path Selection Policy: VMW_PSP_FIXED
Path Selection Policy Device Config: {preferred=vmhba1:C0:T0:L3;current=vmhba1:C0:T0:L3}
Working Paths: vmhba1:C0:T0:L3
naa.60050768018e82d7180000000000021e
Device Display Name: IBM Fibre Channel Disk (naa.60050768018e82d7180000000000021e)
Storage Array Type: VMW_SATP_SVC
Storage Array Type Device Config:
Path Selection Policy: VMW_PSP_FIXED
Path Selection Policy Device Config: {preferred=vmhba1:C0:T0:L4;current=vmhba1:C0:T0:L4}
Working Paths: vmhba1:C0:T0:L4
naa.60050768018e82d71800000000000210
Device Display Name: IBM Fibre Channel Disk (naa.60050768018e82d71800000000000210)
Storage Array Type: VMW_SATP_SVC
Storage Array Type Device Config:
Path Selection Policy: VMW_PSP_FIXED
Path Selection Policy Device Config: {preferred=vmhba1:C0:T0:L1;current=vmhba1:C0:T0:L1}
Working Paths: vmhba1:C0:T0:L1
naa.60050768018e82d71800000000000221
Device Display Name: IBM Fibre Channel Disk (naa.60050768018e82d71800000000000221)
Storage Array Type: VMW_SATP_SVC
Storage Array Type Device Config:
Path Selection Policy: VMW_PSP_FIXED
Path Selection Policy Device Config: {preferred=vmhba1:C0:T0:L5;current=vmhba1:C0:T0:L5}
Working Paths: vmhba1:C0:T0:L5
naa.60050768018e82d71800000000000222
Device Display Name: IBM Fibre Channel Disk (naa.60050768018e82d71800000000000222)
Storage Array Type: VMW_SATP_SVC
Storage Array Type Device Config:
Path Selection Policy: VMW_PSP_FIXED
Path Selection Policy Device Config: {preferred=vmhba1:C0:T0:L6;current=vmhba1:C0:T0:L6}
Working Paths: vmhba1:C0:T0:L6
naa.60050768018e82d71800000000000215
Device Display Name: IBM Fibre Channel Disk (naa.60050768018e82d71800000000000215)
Storage Array Type: VMW_SATP_SVC
Storage Array Type Device Config:
Path Selection Policy: VMW_PSP_FIXED
Path Selection Policy Device Config: {preferred=vmhba1:C0:T0:L0;current=vmhba1:C0:T0:L0}
Working Paths: vmhba1:C0:T0:L0
naa.60050768018e82d71800000000000223
Device Display Name: IBM Fibre Channel Disk (naa.60050768018e82d71800000000000223)
Storage Array Type: VMW_SATP_SVC
Storage Array Type Device Config:
Path Selection Policy: VMW_PSP_FIXED
Path Selection Policy Device Config: {preferred=vmhba1:C0:T0:L7;current=vmhba1:C0:T0:L7}
Working Paths: vmhba1:C0:T0:L7
naa.600508e000000000cf40420d3d12f109
Device Display Name: LSILOGIC Serial Attached SCSI Disk (naa.600508e000000000cf40420d3d12f109)
Storage Array Type: VMW_SATP_LOCAL
Storage Array Type Device Config:
Path Selection Policy: VMW_PSP_FIXED
Path Selection Policy Device Config: {preferred=vmhba0:C1:T3:L0;current=vmhba0:C1:T3:L0}
Working Paths: vmhba0:C1:T3:L0
[root@vmwarehost11 log]# esxcfg-mpath -b
naa.60050768018e82d7180000000000021b : IBM Fibre Channel Disk (naa.60050768018e82d7180000000000021b)
vmhba2:C0:T0:L2 LUN:2 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3f WWPN: 21:00:00:24:ff:32:f7:3f Target: WWNN: 50:05:07:68:01:00:5d:76 WWPN: 50:05:07:68:01:10:5d:76
vmhba2:C0:T1:L2 LUN:2 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3f WWPN: 21:00:00:24:ff:32:f7:3f Target: WWNN: 50:05:07:68:01:00:5a:e3 WWPN: 50:05:07:68:01:40:5a:e3
vmhba1:C0:T1:L2 LUN:2 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3e WWPN: 21:00:00:24:ff:32:f7:3e Target: WWNN: 50:05:07:68:01:00:5d:76 WWPN: 50:05:07:68:01:20:5d:76
vmhba1:C0:T0:L2 LUN:2 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3e WWPN: 21:00:00:24:ff:32:f7:3e Target: WWNN: 50:05:07:68:01:00:5a:e3 WWPN: 50:05:07:68:01:30:5a:e3
naa.60050768018e82d7180000000000021c : IBM Fibre Channel Disk (naa.60050768018e82d7180000000000021c)
vmhba2:C0:T0:L3 LUN:3 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3f WWPN: 21:00:00:24:ff:32:f7:3f Target: WWNN: 50:05:07:68:01:00:5d:76 WWPN: 50:05:07:68:01:10:5d:76
vmhba2:C0:T1:L3 LUN:3 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3f WWPN: 21:00:00:24:ff:32:f7:3f Target: WWNN: 50:05:07:68:01:00:5a:e3 WWPN: 50:05:07:68:01:40:5a:e3
vmhba1:C0:T1:L3 LUN:3 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3e WWPN: 21:00:00:24:ff:32:f7:3e Target: WWNN: 50:05:07:68:01:00:5d:76 WWPN: 50:05:07:68:01:20:5d:76
vmhba1:C0:T0:L3 LUN:3 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3e WWPN: 21:00:00:24:ff:32:f7:3e Target: WWNN: 50:05:07:68:01:00:5a:e3 WWPN: 50:05:07:68:01:30:5a:e3
naa.60050768018e82d7180000000000021e : IBM Fibre Channel Disk (naa.60050768018e82d7180000000000021e)
vmhba2:C0:T1:L4 LUN:4 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3f WWPN: 21:00:00:24:ff:32:f7:3f Target: WWNN: 50:05:07:68:01:00:5a:e3 WWPN: 50:05:07:68:01:40:5a:e3
vmhba2:C0:T0:L4 LUN:4 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3f WWPN: 21:00:00:24:ff:32:f7:3f Target: WWNN: 50:05:07:68:01:00:5d:76 WWPN: 50:05:07:68:01:10:5d:76
vmhba1:C0:T1:L4 LUN:4 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3e WWPN: 21:00:00:24:ff:32:f7:3e Target: WWNN: 50:05:07:68:01:00:5d:76 WWPN: 50:05:07:68:01:20:5d:76
vmhba1:C0:T0:L4 LUN:4 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3e WWPN: 21:00:00:24:ff:32:f7:3e Target: WWNN: 50:05:07:68:01:00:5a:e3 WWPN: 50:05:07:68:01:30:5a:e3
naa.60050768018e82d71800000000000210 : IBM Fibre Channel Disk (naa.60050768018e82d71800000000000210)
vmhba2:C0:T1:L1 LUN:1 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3f WWPN: 21:00:00:24:ff:32:f7:3f Target: WWNN: 50:05:07:68:01:00:5a:e3 WWPN: 50:05:07:68:01:40:5a:e3
vmhba2:C0:T0:L1 LUN:1 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3f WWPN: 21:00:00:24:ff:32:f7:3f Target: WWNN: 50:05:07:68:01:00:5d:76 WWPN: 50:05:07:68:01:10:5d:76
vmhba1:C0:T1:L1 LUN:1 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3e WWPN: 21:00:00:24:ff:32:f7:3e Target: WWNN: 50:05:07:68:01:00:5d:76 WWPN: 50:05:07:68:01:20:5d:76
vmhba1:C0:T0:L1 LUN:1 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3e WWPN: 21:00:00:24:ff:32:f7:3e Target: WWNN: 50:05:07:68:01:00:5a:e3 WWPN: 50:05:07:68:01:30:5a:e3
naa.60050768018e82d71800000000000221 : IBM Fibre Channel Disk (naa.60050768018e82d71800000000000221)
vmhba2:C0:T0:L5 LUN:5 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3f WWPN: 21:00:00:24:ff:32:f7:3f Target: WWNN: 50:05:07:68:01:00:5d:76 WWPN: 50:05:07:68:01:10:5d:76
vmhba2:C0:T1:L5 LUN:5 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3f WWPN: 21:00:00:24:ff:32:f7:3f Target: WWNN: 50:05:07:68:01:00:5a:e3 WWPN: 50:05:07:68:01:40:5a:e3
vmhba1:C0:T1:L5 LUN:5 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3e WWPN: 21:00:00:24:ff:32:f7:3e Target: WWNN: 50:05:07:68:01:00:5d:76 WWPN: 50:05:07:68:01:20:5d:76
vmhba1:C0:T0:L5 LUN:5 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3e WWPN: 21:00:00:24:ff:32:f7:3e Target: WWNN: 50:05:07:68:01:00:5a:e3 WWPN: 50:05:07:68:01:30:5a:e3
naa.60050768018e82d71800000000000222 : IBM Fibre Channel Disk (naa.60050768018e82d71800000000000222)
vmhba2:C0:T0:L6 LUN:6 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3f WWPN: 21:00:00:24:ff:32:f7:3f Target: WWNN: 50:05:07:68:01:00:5d:76 WWPN: 50:05:07:68:01:10:5d:76
vmhba1:C0:T1:L6 LUN:6 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3e WWPN: 21:00:00:24:ff:32:f7:3e Target: WWNN: 50:05:07:68:01:00:5d:76 WWPN: 50:05:07:68:01:20:5d:76
vmhba2:C0:T1:L6 LUN:6 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3f WWPN: 21:00:00:24:ff:32:f7:3f Target: WWNN: 50:05:07:68:01:00:5a:e3 WWPN: 50:05:07:68:01:40:5a:e3
vmhba1:C0:T0:L6 LUN:6 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3e WWPN: 21:00:00:24:ff:32:f7:3e Target: WWNN: 50:05:07:68:01:00:5a:e3 WWPN: 50:05:07:68:01:30:5a:e3
naa.60050768018e82d71800000000000215 : IBM Fibre Channel Disk (naa.60050768018e82d71800000000000215)
vmhba2:C0:T1:L0 LUN:0 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3f WWPN: 21:00:00:24:ff:32:f7:3f Target: WWNN: 50:05:07:68:01:00:5a:e3 WWPN: 50:05:07:68:01:40:5a:e3
vmhba2:C0:T0:L0 LUN:0 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3f WWPN: 21:00:00:24:ff:32:f7:3f Target: WWNN: 50:05:07:68:01:00:5d:76 WWPN: 50:05:07:68:01:10:5d:76
vmhba1:C0:T1:L0 LUN:0 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3e WWPN: 21:00:00:24:ff:32:f7:3e Target: WWNN: 50:05:07:68:01:00:5d:76 WWPN: 50:05:07:68:01:20:5d:76
vmhba1:C0:T0:L0 LUN:0 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3e WWPN: 21:00:00:24:ff:32:f7:3e Target: WWNN: 50:05:07:68:01:00:5a:e3 WWPN: 50:05:07:68:01:30:5a:e3
naa.60050768018e82d71800000000000223 : IBM Fibre Channel Disk (naa.60050768018e82d71800000000000223)
vmhba2:C0:T1:L7 LUN:7 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3f WWPN: 21:00:00:24:ff:32:f7:3f Target: WWNN: 50:05:07:68:01:00:5a:e3 WWPN: 50:05:07:68:01:40:5a:e3
vmhba2:C0:T0:L7 LUN:7 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3f WWPN: 21:00:00:24:ff:32:f7:3f Target: WWNN: 50:05:07:68:01:00:5d:76 WWPN: 50:05:07:68:01:10:5d:76
vmhba1:C0:T1:L7 LUN:7 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3e WWPN: 21:00:00:24:ff:32:f7:3e Target: WWNN: 50:05:07:68:01:00:5d:76 WWPN: 50:05:07:68:01:20:5d:76
vmhba1:C0:T0:L7 LUN:7 state:active fc Adapter: WWNN: 20:00:00:24:ff:32:f7:3e WWPN: 21:00:00:24:ff:32:f7:3e Target: WWNN: 50:05:07:68:01:00:5a:e3 WWPN: 50:05:07:68:01:30:5a:e3
naa.600508e000000000cf40420d3d12f109 : LSILOGIC Serial Attached SCSI Disk (naa.600508e000000000cf40420d3d12f109)
vmhba0:C1:T3:L0 LUN:0 state:active sas Adapter: 5005076b0887dba0 Target: 9f1123d0d4240cf
[root@vmwarehost11 ~]# esxcfg-scsidevs -l
naa.60050768018e82d71800000000000210
Device Type: Direct-Access
Size:
Display Name: IBM Fibre Channel Disk (naa.60050768018e82d71800000000000210)
Plugin: NMP
Console Device: /dev/sdi
Devfs Path: /vmfs/devices/disks/naa.60050768018e82d71800000000000210
Vendor: IBM Model: 2145 Revis: 0000
SCSI Level: 6 Is Pseudo: false Status: on
Is RDM Capable: true Is Removable: false
Is Local: false
Other Names:
vml.020001000060050768018e82d71800000000000210323134352020
naa.60050768018e82d71800000000000215
Device Type: Direct-Access
Size:
Display Name: IBM Fibre Channel Disk (naa.60050768018e82d71800000000000215)
Plugin: NMP
Console Device: /dev/sdj
Devfs Path: /vmfs/devices/disks/naa.60050768018e82d71800000000000215
Vendor: IBM Model: 2145 Revis: 0000
SCSI Level: 6 Is Pseudo: false Status: on
Is RDM Capable: true Is Removable: false
Is Local: false
Other Names:
vml.020000000060050768018e82d71800000000000215323134352020
naa.60050768018e82d7180000000000021b
Device Type: Direct-Access
Size:
Display Name: IBM Fibre Channel Disk (naa.60050768018e82d7180000000000021b)
Plugin: NMP
Console Device: /dev/sdh
Devfs Path: /vmfs/devices/disks/naa.60050768018e82d7180000000000021b
Vendor: IBM Model: 2145 Revis: 0000
SCSI Level: 6 Is Pseudo: false Status: on
Is RDM Capable: true Is Removable: false
Is Local: false
Other Names:
vml.020002000060050768018e82d7180000000000021b323134352020
naa.60050768018e82d7180000000000021c
Device Type: Direct-Access
Size:
Display Name: IBM Fibre Channel Disk (naa.60050768018e82d7180000000000021c)
Plugin: NMP
Console Device: /dev/sdg
Devfs Path: /vmfs/devices/disks/naa.60050768018e82d7180000000000021c
Vendor: IBM Model: 2145 Revis: 0000
SCSI Level: 6 Is Pseudo: false Status: on
Is RDM Capable: true Is Removable: false
Is Local: false
Other Names:
vml.020003000060050768018e82d7180000000000021c323134352020
naa.60050768018e82d7180000000000021e
Device Type: Direct-Access
Size:
Display Name: IBM Fibre Channel Disk (naa.60050768018e82d7180000000000021e)
Plugin: NMP
Console Device: /dev/sdf
Devfs Path: /vmfs/devices/disks/naa.60050768018e82d7180000000000021e
Vendor: IBM Model: 2145 Revis: 0000
SCSI Level: 6 Is Pseudo: false Status: on
Is RDM Capable: true Is Removable: false
Is Local: false
Other Names:
vml.020004000060050768018e82d7180000000000021e323134352020
naa.60050768018e82d71800000000000221
Device Type: Direct-Access
Size:
Display Name: IBM Fibre Channel Disk (naa.60050768018e82d71800000000000221)
Plugin: NMP
Console Device: /dev/sde
Devfs Path: /vmfs/devices/disks/naa.60050768018e82d71800000000000221
Vendor: IBM Model: 2145 Revis: 0000
SCSI Level: 6 Is Pseudo: false Status: on
Is RDM Capable: true Is Removable: false
Is Local: false
Other Names:
vml.020005000060050768018e82d71800000000000221323134352020
naa.60050768018e82d71800000000000222
Device Type: Direct-Access
Size:
Display Name: IBM Fibre Channel Disk (naa.60050768018e82d71800000000000222)
Plugin: NMP
Console Device: /dev/sdd
Devfs Path: /vmfs/devices/disks/naa.60050768018e82d71800000000000222
Vendor: IBM Model: 2145 Revis: 0000
SCSI Level: 6 Is Pseudo: false Status: on
Is RDM Capable: true Is Removable: false
Is Local: false
Other Names:
vml.020006000060050768018e82d71800000000000222323134352020
naa.60050768018e82d71800000000000223
Device Type: Direct-Access
Size:
Display Name: IBM Fibre Channel Disk (naa.60050768018e82d71800000000000223)
Plugin: NMP
Console Device: /dev/sdc
Devfs Path: /vmfs/devices/disks/naa.60050768018e82d71800000000000223
Vendor: IBM Model: 2145 Revis: 0000
SCSI Level: 6 Is Pseudo: false Status: on
Is RDM Capable: true Is Removable: false
Is Local: false
Other Names:
vml.020007000060050768018e82d71800000000000223323134352020
naa.600508e000000000cf40420d3d12f109
Device Type: Direct-Access
Size:
Display Name: LSILOGIC Serial Attached SCSI Disk (naa.600508e000000000cf40420d3d12f109)
Plugin: NMP
Console Device: /dev/sda
Devfs Path: /vmfs/devices/disks/naa.600508e000000000cf40420d3d12f109
Vendor: LSILOGIC Model: Logical Volume Revis: 3000
SCSI Level: 2 Is Pseudo: false Status: on
Is RDM Capable: true Is Removable: false
Is Local: false
Other Names:
vml.0200000000600508e000000000cf40420d3d12f1094c6f67696361
I would like to ask why only this host out of 12 hosts is having this problem such that no datastores are visible
Waiting for a helpful feedback. Thanks