ZFS, at least on Solaris, has issue with many multiple readers of the same file, blocking after ~31 simultaneous readers (even when there are NO writers). Ran into this with a third party library which reads a large TTF to produce business PDF documents. The hundreds of reporting processes all slowed to a crawl when accessing the 20Mb Chinese TTF for reporting because ZFS was blocking.
I can't change the code since it is third party. The only way I saw to easily fix it was on system startup to copy the fonts under a new subdir in /tmp (so in tmpfs, ie RAM, no ZFS at all there ) and then softlink the dir the product was expecting to the new dir off of /tmp, eliminating the ZFS high-volume multiple-reader bottleneck.
Never had this problem with the latest EXT filesystems on my volume groups on my Linux VMs with the same 3rd party library and same volume of throughput.
If you recreate the bug on Linux and have something like a simple reproduder I guess that the ZoL devs are more than happy to fix or at least understand it:
From reading the pull requests and issues on that repo I've got the impression that the next release 0.7.0 will be quite step forward and there seems to be quite sophisticasted work to tackle performance issues.
I can't change the code since it is third party. The only way I saw to easily fix it was on system startup to copy the fonts under a new subdir in /tmp (so in tmpfs, ie RAM, no ZFS at all there ) and then softlink the dir the product was expecting to the new dir off of /tmp, eliminating the ZFS high-volume multiple-reader bottleneck.
Never had this problem with the latest EXT filesystems on my volume groups on my Linux VMs with the same 3rd party library and same volume of throughput.