Hi all,
I have a setup that doesn't get the performance I expected. The current datastore that I have created is dedicated to my testing VM. I test the performance with the following command:
fio --time_based --name=/dev/sdb --size=100G --runtime=30 --ioengine=libaio --randrepeat=0 --iodepth=32 --direct=1 --invalidate=1 --verify=0 --verify_fatal=0 --numjobs=4 --rw=randread --blocksize=4k --group_reporting
The following hardware is used:
- Dell H740p Adapter - 8x Intel S3710 SATA SSD in RAID10 (2 disks in 4 stripes)
The following software is used:
- Vmware 6.5U1
- Driver lsi-mr3 version 7.700.50.00-1OEM
This results 153k IOPS average, but it should be up to 500k IOPS at average. I've increased the blocksize to 16k and the IOPS didn't change much getting 128k IOPS at average. Checking my adapters iodepth leads to:
ADAPTR PATH AQLEN
vmhba3 - 4040
This should be sufficient, so switching to disk view I get the following:
DEVICE PATH/WORLD/PARTITION DQLEN WQLEN ACTV QUED %USD LOAD CMDS/s READS/s WRITES/s MBREAD/s MBWRTN/s DAVG/cmd KAVG/cmd GAVG/cm
naa.61866da0976aec00212af32636c0c7e9 - 64 - 0 0 0 0.00 4.09 3.12 0.58 0.01 0.00 0.16 0.01 0.1
Strange enough other devices are showing load and active command queues, but even during the fio benchmark not much is changing here. It looks like this device is also stuck at DQLEN of 64. Listing this device retrieves the following information:
naa.61866da0976aec00212af32636c0c7e9
Display Name: Local DELL Disk (naa.61866da0976aec00212af32636c0c7e9)
Has Settable Display Name: true
Size: 1523712
Device Type: Direct-Access
Multipath Plugin: NMP
Devfs Path: /vmfs/devices/disks/naa.61866da0976aec00212af32636c0c7e9
Vendor: DELL
Model: PERC H740P Adp
Revision: 5.00
SCSI Level: 5
Is Pseudo: false
Status: on
Is RDM Capable: true
Is Local: true
Is Removable: false
Is SSD: true
Is VVOL PE: false
Is Offline: false
Is Perennially Reserved: false
Queue Full Sample Size: 0
Queue Full Threshold: 0
Thin Provisioning Status: unknown
Attached Filters:
VAAI Status: unknown
Other UIDs: vml.020000000061866da0976aec00212af32636c0c7e9504552432048
Is Shared Clusterwide: false
Is Local SAS Device: true
Is SAS: true
Is USB: false
Is Boot USB Device: false
Is Boot Device: false
Device Max Queue Depth: 64
No of outstanding IOs with competing worlds: 48
Drive Type: logical
RAID Level: RAID1_0
Number of Physical Drives: 8
Protection Enabled: false
PI Activated: false
PI Type: 0
PI Protection Mask: NO PROTECTION
Supported Guard Types: NO GUARD SUPPORT
DIX Enabled: false
DIX Guard Type: NO GUARD SUPPORT
Emulated DIX/DIF Enabled: false
It seems this device max queue depth is set to 64, which I can't increase:
# esxcli storage core device set -d naa.61866da0976aec00212af32636c0c7e9 -O 255
Unable to set device's sched-num-req-outstanding. Error was:Cannot set device queue depth parameter. sched-num-req-outstanding should be <= 64
Anybody got a clue how to increase the DQLEN?