Hi,
 
just bought SR-2 in auction, having 2x INTEL XEON L5639 2.13GHz. Any thoughts about what frequency can be achieved using those cpus, I already found some very positive posts, but would like to heard real SR-2 user experience.
I am now considering also the RAM. I red this post from the very beginning and it is really useful! Well it looks to me that the system is limited somehow by bios implementation even I would expect that the board will support as much memory as CPU supports, the controller is on CPU, but it looks there are some issues what the bios is able to read.
 
Hm on the other hand I found this thread at AVADirect
This is about 192GB support, but on the SR-X. So younger brother support this, but older brother SR-2 not?
 
Well, in my case the 96 is for my purpose still enough. It is for GPU machine, so I am happy with 7 PCI slots, 6x for R9 290x and one dual-port Infiniband HCA 4X QDR card used as TCPIP wrapper.
 
One more question, I am already aware of the fact that when all RAM slots are taken, it is very good practice to low donw the memory frequency to 800 to not getting stuck with some buggy behavior. Is that correct?
 
Thanks folks.
 
Ladislav

PSU: Underspecified. I wouldn't want to run a setup like this with less than 1000W for heavy load usage.
 
RAM: Get ECC RAM and save yourself a whole world of stability troubleshooting time wasting. For better stability at higher clocks speeds, get RDIMMs. For best performance you will want 6 DIMMs, not 8. 6x8GB will by and large just work, with both single-ranked and dual ranked DIMMs. Dual rank DIMMs seem to work a little more reliably in 6x8GB configuration. If you plan to go to 12x8GB at some point, make sure you get dual rank DIMMs, SR ones won't get detected reliably past 48GB, in most cases won't even POST. I have always had best  results with Crucial RAM. The speed they are rated at doesn't matter past 1800MHz since you won't achieve that on the SR-2 stably anyway, and most Crucial 1600 RAM easily hits 1800 with next to no timing tweaks at standard voltage. (Note: SR-2 will over-volt DIMMs by default when set to "auto".
 
So, to summarize - your best bet is to get 6x8GB dual rank ECC RDIMMs. There is no advantage to getting DIMMs rated for more than 1800MHz since this won't be stably attainable anyway.
 
I am using 12x8GB=96GB in my SR-2s. If you plan to go past 48GB (the only number bigger than 48GB that makes sense is 96GB) there are a few settings you will need to manually set to ensure the machine POSTs reliably. There is another, more recent thread dedicated to info regarding running 8GB DIMMs and 96GB in total, search for it, I'm sure you'll find it.
 
4GHz is attainable _stably_ on air, but you will need extremely good heatsinks. I just upgraded my heatsinks this evening, and I have to say I am awed by the Thermalright HR-22. They are _huge_ and _heay_, but they will fit in a Lian Li PC-P80N. I was running some cheap Igloo H46 HSFs because they were cheap and available locally and I needed some HSFs that day. They do reasonably well in reviews, even compared to much more expensive HSFs. With 12 cores loaded to the maximum with minerd for testing (the harshest test I have found, harsher than OCCT), the hottest core was touching over 90C (T-junction, T-case was getting to 80-81C which is the Intel specified thermal limit for the highest heat rated Westmere 6-core Xeon).
 
I just fitted the HR-22s, ducted directly to exhaust fans (CPU0 to the 140mm top, CPU1 to the 120mm rear), with 120mm fans off the H46s attached as pushers. I had to make a ducting extension from some cardboard to make the ducting reach between the top exhaust fan to the HS on CPU0.
 
Result? 10C cooler at idle, and a truly shocking 30C cooler at full load (I cannot get past 60C on any core now!). I am running 166 bclk with x24 multiplier, which is 4GHz almost exactly (3.99 to 3 significant figures accuracy). The main downside is that getting to the first 3 DIMM slots on CPU1 will mean removing the HS, but I guess you shouldn't need to do that very often once you have it all tested.

1) Disable all OC. Test at standard clock speeds and ratio multipliers, everything on "auto".
2) If that still doesn't work, set command rate to 2T, that might help with 8GB DIMMs
 
I am running with 12x8GB Crucial ECC RDIMMs, dual ranked, x4, and mine takes a few reset loops to POST, but it almost always POSTs within 30 seconds. When it doesn't POST, it doesn't get as far as the splash screen before it loops back to retry. BCLK=166 works fine for me, but above about 170 bclk it will not POST, ever, with 12x8GB. It is worth noting that all my voltages are on "auto" or manually selected to lowest possible, e.g. 1.35V DIMM voltage - boosting DIMM voltages above what they are specced for is completely pointless as far as getting them to POST in the SR-2 is concerned). The only thing I had to do to get it to POST with 96GB of RAM is set the command rate to 2T. That seems to be the key ingredient, but you will inevitably pay the price in memory bandwidth and latency.
 
I also tried 6x8GB Crucial ECC RDIMMs, single ranked, x4, and that almost always POSTs first time every time, but putting them all on the same bank with a single CPU installed never POSTs, so single ranked DIMMs are obviously more problematic with this board.
 
I don't believe quad-ranked RDIMMs have been tried yet, but according to Intel MCH specs that most definitely requires downclocking the DIMMs, which may well cause POST-ing problems. For example, my board posts with DIMMs set to 1333 (at 166 bclk that means 1660), but if I reduce the memory speed multiplier, it will not POST, ever.
 
So to summarize:
1) Everything on "auto", stock clock speeds
2) Memory speed manually set to 1333, MCH strap to 1600
4) Memory command rate to 2T
 
If you have already tried all of the above, you may just have a duff SR-2 (a not trivial fraction are just outright faulty AFAICT).

Now that you mention it, a duff DIMM can actually produce these kinds of symptoms. EDAC error reporting should show it up if they are ECC DIMMs.
But you are definitely doing the right thing sticking with the Supermicro board and dumping the SR-2.

If you mean repairing the DIMM slot, you might get lucky if the board is till in warranty.

If he didn't upload it to EVGA when he registered for the extended 10 year warranty, you are out of luck.

The other thing to consider is that the SR-2 was never tested by EVGA with DIMMs bigger than 4GB, and BIOS is at best buggy when handling bigger modules. There is a good chance that it would work OK.
 
Also, what CPUs are you using? While 6-core CPUs seem to handle 8GB DIMMs reasonably well, lower end CPUs sometimes don't. Also, the later the CPU, the better success rate seems to have been reported with 8GB DIMMs and to a similar extent, 96GB of total RAM. For example, I'm running top of the line X5690 CPUs in mine and it works OK. Having a very good PSU that provides noise free power to the motherboard also probably helps to some extent (I am using a 1500W CoolerMaster).

It's not weird at all. That actually implies you have a duff CPU, not a defect that is on the motherboard. It sounds like a part of the MCH in the CPU is faulty, or you have pad on the faulty CPU that isn't making proper contact. Carefully rub down the contact pads on the underside of the CPU with a decent quality pencil eraser that doesn't leave greasy residue, refit it, and see if the problem goes away.

That duff CPU works with 48gb ram in my supermicro board.
 
 I'll try with that eraser when I get home, it seems resonable enough to try. Boy oh boy it does take a lot of time stuff like this.
 
 The same thing actually applied when using two E5506. I already tried looking a few hundred times at the socket pins, and adjusting just a few of them.
 

Its sitting on my desk at home. It has small effective waterblocks. No heavyduty stuff
 
 I'll try the plastic card approach with the sokkel pins also, you never know if one is a tad lower than the rest, I did refit cpu's alot of times.

I got it to post with 48gb, but only with two E5506. With X5677 and E5620 it will only boot if I remove one stick of 8 gb.
 
Edit: Thanks for the replys Gordan, your remarks gave me hope.
 
 Just got it to post with 48gb using two x5677, i honestly dont know what did it? It must have been a loose connection at a motherboard socket pin.
  
You can see the mess my gf had to put up with, she's alright that garl. (can't post pictures or url, so sad cause I haven't been here for five days.
 
Right now I'm so happy. But my fear is, when I put the waterblock back on, and fit the entire motherboard in it's "cage" that it will go back to memory-waldo state.