macprouser
Hi again.
Regarding the above, I have discovered the problem after a long process of trying every kind of configuration, benchmarking and observation. The answer was staring at me all the time in the System Profiler.
The SM951's on chip controller cannot negotiate a full speed link in a Mac Pro's x16 slot. It can do so in the x4 slot. The XP941 however can negotiate a full speed link in any slot. So until a firmware upgrade ever comes out you cannot create a full speed raid with two SM951s. The best you can get right now is 1500 MB/s read and write.
To make a faster RAID you can either configure a RAID from two XP941s, stripe an XP941 and SM951 together (for some reason slower than the first option), or the best option I had was to stripe an XP941 with two SM951s for 2300 MB/s read / 2100 MB/s write.
macprouser, regarding your third setup, was the SM951 in slot 3, with the two XP941s in slots 2 and 4? I know we are just testing, but in the real world, I don't know that it would be worth it to give up that other slot for 2300/2100 if you were getting the 2231/1750 BareFeats reported with the XP941s in RAID 0 in slots 2 and 3. Very disappointing news about the SM951, as I was waiting for it to be more commonly available with the idea of putting in a RAID 0 that was even faster than the XP941 rig in BareFeats.