Infrastructure Software

We noticed an issue with read performance while trying to keep our LTO5 tape fed at the speed they recommend (100mb/s). Our average speed was about 90mb/s and it bounced around a lot. We have 6gb/s 10K drives in a 4 drive raid 0+1. Each drive should be able to sustain well over 90mb/s and when reading from a raid 10 of those drives we expected about 400mb/s.

The odd part is that we can write to the drives at around 370mb/s which is about right.. And writes are normally slower. So I am confused why reads are stuck below 100mb/s.

We watched the iostat of the drives and noticed that when writing a file, all four drives light up to their max write speed apx 160 to 190mb/s. Giving us the combines speeds near around 370mb/s. However when reading a file, iostat only shows one drive at a time being accessed at a slow rate.. Given that we are striped across 2 mirrors, you would think at least two drives would be lit minimum.. Yet it seems like it reads a little from drive 1, then a little from drive 2 ect..

In testing we disabled every attribute we could think of in zfs to increase speed, checksum, log, compression, ect.. None increased read.

This is not a slow machine, Core I7 with 64gigs of ram. In testing on several other solaris machines, all 11.1 boxes exhibited this issue. (I don't have a non 11.1 to test).

Anyone else seen this behavior?

Thanks,
Tom