Impossible to tell without trying different CPUs or trying the CPUs in a different dual socket motherboard. I can tell you beyond any doubt that is not normal, though, so you might want to start looking into ways to get refund before it's too late (if it's not already too late).

That is very odd indeed. If it works at 5.8 that implies a stability issue, rather than an outright fault. Things I would start with are:
1) Make sure it's not a RAM compatibility issue. If you have some 4GB ECC RDIMMs, try with those. 8GB DIMMs are unsupported on the SR-2 and although many of us have various 8GB modules working it is not always a sure thing even when only running 6 of them. Also, what is your RAM voltage set to? The spec sheet for your DIMMs says 1.5V (and the spec sheet lists it as 1.425 - 1.575V, so you may want to bump the voltage up a bit there and see if it helps).
2) Check your IOH voltage. Both of my SR-2s are stable at 166 bclk and 96GB of RAM at 1.25V, but anything up to 1.35V should be safe, so it might be worth trying to bump that up, unless it is already at 1.35V.
 
Did you try swapping the RAM between the two CPUs, just to see if the fault follows the RAM?
 
If none of that makes any difference, there is a good chance you have a duff component on the motherboard somewhere around CPU1. Capacitors these days are quite good (most manufacturers learned the lesson to not use cheap capacitors), but inductors still often fail, and on the SR-2 there are 8 for each CPU socket on the SR-2, but the end two in each set are not covered by the heatsinks (the middle 6 are, the end 2 are not) for some unfathomable reason.

1) Specifically try 2T command rate.
And yes, the board is full of design faults. The CPUs (where the MCH is) can handle 192GB of RAM each, yet the board struggles badly with 96GB of RAM between the two CPUs. I think that illustrates the extent of the fail quite well, and that's far from the only problem on boards that are working exactly as designed.
 
Having said all that, I have not observed the specific issue you seem to be having on my SR-2s. What I have observed is that QPI destabilizes very, very shortly after 6.4GT, which means that OC-ing potential hits a wall once you start to exceed 177 bclk (because 4.8GT/s at 177 bclk actually equals 6.4GT/s since default bclk is 133). And at above 180 bclk or so very weird clock spikes, both high and low start to occur, all of which snowballs with crashtastic results very quickly.
 
2) That kind of makes sense since CPUs "uncore" runs off VTT voltage so if there is a marginal/aged component I can see how it might help. Just out of interest, have you checked the state of thermal pads and paste under the heatsinks? If that was the problem I'd expect over-volting to make things worse, but it's worth checking if you run out of options.
 
If all else fails (or when you decide your time is worth something and you've wasted to much of it on the SR-2), can can always get a similar SuperMicro X8 series motherboard and reuse your CPUs and RAM. The X8 series boards are going for a pittance on ebay most of the time. The X8DTH line is the most similar to the SR-2 (7x PCIe x16 slots, although IIRC each is wired only for x8, not that it makes a damn worth of difference to anything). X8DAH line has only 3 PCIe x16 slots (of which two are wired for full x16) and a few x8 slots, and a tonne of RAM sockets, so it's worth a look as well (particularly depending on what's gong cheap on ebay on the day you're looking). They might not have OC-ing features, but with a pair of X5690s you don't exactly need those desperately like you might with lower end CPUs.
 
And note that I am saying that as one of a handful of people that have finally defeated the SR-2 and made it work for most of my requirements. The amount of time and effort it has taken to figure out the workarounds (getting 96GB of RAM to work), write the patches (e.g. for Xen HVM loader to avoid the IOMMU/VT-d bugs on NF200 bridges) and figuring out avoiding all the unfixable issues (every SAS controller I tried on my boards at best causes random lock-ups as soon as I start using VT-d with PCI passthrough for VMs, and some end up just not showing any disks attached) to make it all work passably well was so great that if I'd spent the same time working I could have just bought a top of the line off the shelf workstation filled with Nvidia Quadros and still ended up ahead. Remember that the total advantage from the SR-2 in terms of OC-ing the X5690s is going to be about 15% over stock (4GHz vs. 3.46GHz). Even with lesser CPUs, and if you manage to get it stable at 180 bclk (where clock stability falls off a cliff), you are only looking at a maximum 35% OC and that's as good as it gets with any pretence of long term stability. As one of my friends once wisely said, it takes great wisdom to recognize that it would be a good idea to quite while you're behind.

It sounds increasingly like a faulty motherboard that just happened to be running passably stably with the exact configuration and OS the previous owner was running.
 
When OC-ing:
QPI to 5.8GT/s will actually be 6.4GT/s at 146MHz bclk.
QPI to 4.8GT/s will actually be 6.4GT/s at 177Mhz bclk (this is the main (but not the only) reason why you generally hit a stability wall at about 180MHz bclk on these).
 
For the sake of being thorough, you could set QPI to 5.8GT/s and crank the bclk to 146, adjust the CPU multiplier down to be running default or lower maximum CPU speed, and see if that destabilizes.
 
Just out of interest, what FSB strap do you have set? Auto? Might be worth explicitly setting 1333 or 1600 just to eliminate the possibility of some internal timings being out when set to auto.

I was talking about internal MCH timings which most tools don't report.
An over-aged component due to over-volting is plausible. The Westmere Xeons are specced for a maximum of 1.35V core and 1.30V VTT. Any more and you are pretty much guaranteed to be damaging something and I seem to remember reading more than one post implying that equivalent i7s were expiring even without exceeding the spec, although whether that is due to motherboards overvolting and/or sensor under-reading is difficult to say. I'm running at well below those limits at 4GHz with 166 bclk.
 
Every once in a while I spend a few seconds thinking about trying to bump up the bclk to 177 but I'd end up having to reduce the core and uncore multipliers, re-test the memory timings, and bump the core, uncore and IOH voltages right up to the limit, which seems like a huge waste of time for an extra 2.5% core speed and maybe 6% extra memory I/O at most, and that's assuming I don't have to loosen the memory timings to achieve it. OC-ing is worthwhile with an X5650 and below, but with an X5690 there is no worthwhile benefit. The only reason I got a pair (my other system uses X5650s) was to explore the limits of the Westmere cores and help establishing the bclk limits of the SR-2, In the end I stuck with them because they seem to require slightly less voltage than my X5650s and thus run slightly cooler, but that could easily just be due to them having been manufactured later on than my original X5650s. Which all leads by to my earlier point - if you have X5690s, get a decent non-OC-ing SuperMicro board, as they'll boost to 3.60GHz on all cores and 3.73GHz on 2 cores without any OC-ing. That is precisely what I'll be looking into if it turns out that Nvidia Maxwells don't play well with VT-d on my configuration, as similar issues have already been reported.

X5690 has all core boost frequency of x27, so you can get to 4GHz at 148 bclk, which means if you set QPI to 5.8 it will yield 6.46GT/s which should certainly be within the OC tolerances. If he couldn't get it stable above 4.8 QPI something's not quite right.

Very interesting indeed. But I thought you said you tried both CPUs in socket 1, and they both exhibited the problem in socket 1, but both worked OK in socket 0. Was that a misinterpretation of what you had said on my part?
 
In case it wasn't a misinterpretation and you can definitely confirm that both X5690s have problems in socket 1, then the thing to consider is that X5570 is a 4-core and X5690 is a 6-core, which means some pins on the socket are not used when X5570 is installed. Which in turn implies that the fault could be dodgy contact or RF noise or a short somewhere relating to one of those pins/traces that don't do anything on an X5570.
 
So if you haven't tested both of your 6-cores independently in socket 1 with only one CPU installed, now would probably be a good time to try that for cross-check.

That sounds like a marginal pin or related trace on socket 1 to me, on one of the pins not used by the 4-core but used by the 6-core CPUs.
If you really that determined to rule out motherboard issues, you could look into getting an X5650. They are going for next to nothing on ebay these days, and it is a 6-core 6.4GT QPI chip. If that doesn't work properly in socket 1 either but works in socket 0, then that would remove any trace of remaining doubt, if there still is any, that the fault is with the motherboard rather than the CPUs.
The QPI connections are from each CPU to the NB (PCIe root hub) and from each CPU to the other CPU.
That is why W series Xeons are single socket (only one QPI link) and X series are dual socket (two QPIs).

mathf
Ok! Long story short... 
I got in touch with EVGA Europe techsupport, and they RMA'ed my board.
I now got a E1 refurbished SR-2.

 
Lucky you. They told me they were out of stock with no expected time they'll have more. I think given my RMA history they figured I'd find any faults on it within a day if they exist, and they couldn't afford the couriering costs.
 

mathf
With this board, I am facing a very weird issue.
I am getting an immediate "FF" code when both CPU are physically installed with 2 "80 Plus Platinum" PSUs: 
 - Corsair AX1200i
 - Seasonic SS-1200XP3
 
The symptoms are the same if I disable the CPU1 socket via jumpers.
What it also worth mentioning, is that, when this occurs, the southbridge fan completly stops spinnig (CPU fans are still spinning), and it resumes spinning for 3-4 seconds once the power is shut down.
 
-> Funny part: if I connect a good-old Corsais HX1000W, everything works properly. The only minor point to mention is that I have to mannualy disable C6 state when booting from CPU1 socket only. Otherwise, Windows 7 freezes right after the boot logo animation.

 
Hmm... Dodgy. Sounds like marginal power circuitry on socket #2
 

mathf
I sent a ticket to EVGA techsupport once again, and they are starting to claim that both Platinum PSUs might not be compatible with the SR-2... I kinda find this quite odd.

 
The thing you need to remember is that there is no such thing as a new replacement SR-2 any more, and hasn't been for at least a couple of years. All replacement are refurbs, which means they were faulty and were supposedly repaired. Unfortunately, I have never seen a fully properly working refurb motherboard - once it duffs out, it'll never work properly again. If they offer you to replace it with a different new motherboard, I suggest you take them up on it, sell your Xeons on ebay and buy a CPU that fits the new board. In the long term, if your time is worth anything, it will be by far the most sane thing to do.