Sunday, January 29, 2012

Varying SCSI queue depth for VMware PVSCSI block devices.

I was toying with the block subsystem a bit in a Linux virtual machine running under ESX 5.0, when I realized
I could not change the SCSI queue depth. It turns out that the driver simply didn't implement the interface! It was, however, pretty easy to fix this.

https://github.com/andreiw/andreiw-wip/blob/master/linux/3.2/0001-VMW_PVSCSI-Allow-TCQ-depth-change-through-sysfs.patch

Now you can do something like the following:
# for i in {a..z}; do  eval 'echo 1 > /sys/block/sd$i/device/queue_depth'; done

The patch should make it to mainline, when the current PVSCSI maintainer, Arvind Kumar, gets it integrated.

Why would you care? Because /sys/block/sda/device/queue_depth is quite different from /sys/block/sda/queue/nr_requests. nr_requests controls the request flow before the I/O scheduler, while queue_depth controls the flow of actual dispatch on I/O device. You might be interested in either of those if you run with multiple VMDKs, have an intensive I/O workload on one disk, and notice starvation on others.

1 comment:

  1. What about ring size of 32? To go with QD=255?

    VSA appliances need both.

    Can this be managed with ESXi 5.5 vmtools now?

    Anyone ever done a pepsi challenge (ESXi 5.5 versus Hyper-v 2012R2) to see how far each can push ?

    ReplyDelete