A few months ago, our vSphere Data Protection Appliance backups started failing and I discovered that our disk performance had tanked (write speeds as slow as 2MB/s). This is NON-CACHED disk performance I am referring to here. Operations impacted are backups, cold migrations, etc which all exhaust any cache you have very quickly. VM operations are all nominal (presumably because caching is doing its job).
We run three vSphere/ESXi 5.5 hosts connected by two redundant 10Gb/s switches. Management is via a vCenter Server Appliance. Our primary storage is Virtual SAN but we have local VMFS datastores on each host and a FreeNAS that all exhibit the same slowness. The condition presents from within VMs right down to the ESXi CLI so it feels like some ESXi parameter that has changed but we have done no updates to the environment this year. Found Disk.BandwidthCap and Disk.ThroughputCap but these are all still set at default (2^32). Likewise, all the VMkernal adapters are running with traffic shaping disabled as they have been since we brought the environment up in 2015. Just thought I would through this out there before I open a ticket with VMware support...