EVGA

SR-2 - 96GB Secret Sauce

Page: < 1234 > Showing page 2 of 4
Author
Post
Essentials Only Full Version
gordan79
SSC Member
  • Total Posts : 531
  • Reward points : 0
  • Joined: 2013/01/27 00:17:36
  • Status: offline
  • Ribbons : 3
Re: SR-2 - 96GB Secret Sauce 2014/03/06 02:42:34 (permalink)
It will work, but you will have no way of telling whether any instability you encounter (and you will encounter it) is due to CPU or memory errors. I would very, very strongly advise against using non-ECC memory in anything.
 
Also note that all of us using 96GB are using registered DIMMs which all have ECC. Non-registered RAM is likely to be worse in both stability and OC-ability because the signalling lines aren't buffered.
 
And your questions are now completely off topic for this thread. Start another thread unless you have a query specifically relating to operating an SR-2 with 96GB of RAM.

Supermicro X8DTH-6, 2x X5690
Crucial 12x 8GB x4 DR 1.35V DDR3-1600 ECC RDIMMs (96GB)
3x GTX 1080Ti
Triple-Seat Virtualized With VGA Passthrough (KVM)
#31
gordan79
SSC Member
  • Total Posts : 531
  • Reward points : 0
  • Joined: 2013/01/27 00:17:36
  • Status: offline
  • Ribbons : 3
Re: SR-2 - 96GB Secret Sauce 2014/03/07 05:40:52 (permalink)
Another minor update on running with 96GB with the Crucial DIMMs I linked previously:
 
It turns out most of my problems were down to a duff DIMM (first duff Crucial DIMM I have ever seen!). With good, properly working DIMMs there is no need whatsoever to push vDIMM past 1.35V (or whatever your DIMMs are rated at. I'm currently running 11x8GB pending replacement DIMM arriving, and it comes up every time from cold start with no problems, no missing RAM.
 
There is also no need to boost any other voltages if running at stock speeds, just manually select them. The key seems to be manually selecting memory timings and voltages, and make sure you set the memory command rate to 2T.
 
I am quite surprised how trouble free it is now. I also cranked the memory scrub rate to the maximum of 2GB/s per MCH, and no errors at all any more with a heavy memory load and near full occupancy.
 
I do believe the 96GB issue is hereby resolved.

Supermicro X8DTH-6, 2x X5690
Crucial 12x 8GB x4 DR 1.35V DDR3-1600 ECC RDIMMs (96GB)
3x GTX 1080Ti
Triple-Seat Virtualized With VGA Passthrough (KVM)
#32
killissa
New Member
  • Total Posts : 2
  • Reward points : 0
  • Joined: 2014/07/25 01:33:52
  • Status: offline
  • Ribbons : 0
Re: SR-2 - 96GB Secret Sauce 2014/07/25 01:43:19 (permalink)
Hi gordan79,
please could you write which bios version you are using?
 
I´ve also tried to use 96GB and 192GB RAM and its only possible after bios reset. If I change bios settings as u suggested as well as with some further changes for test, the system will not boot up. I´m using bios 58 and I´m wondering if you are using the same or a different one.
Cheers
#33
gordan79
SSC Member
  • Total Posts : 531
  • Reward points : 0
  • Joined: 2013/01/27 00:17:36
  • Status: offline
  • Ribbons : 3
Re: SR-2 - 96GB Secret Sauce 2014/07/25 02:22:44 (permalink)
I'm running A58. I never managed to get 16GB DIMMs to work at all, even when only using 6. 12x8GB ECC DR RDIMMs work for me.
What RAM are you using? How many ranks? I couldn't get it working with 12x8GB SR ECC RDIMMs, but 12x8GB DR ECC RDIMMs. 6x8GB SR ECC RDIMMs work, though. All I do is manually set all the timings and DIMM voltage (1.35V). The most important thing to set manually is the 2T commant rate (which defaults to 1T).

Supermicro X8DTH-6, 2x X5690
Crucial 12x 8GB x4 DR 1.35V DDR3-1600 ECC RDIMMs (96GB)
3x GTX 1080Ti
Triple-Seat Virtualized With VGA Passthrough (KVM)
#34
killissa
New Member
  • Total Posts : 2
  • Reward points : 0
  • Joined: 2014/07/25 01:33:52
  • Status: offline
  • Ribbons : 0
Re: SR-2 - 96GB Secret Sauce 2014/07/25 03:33:54 (permalink)
The 8GB modules are dual-rank:
Kingston KVR1333D3D4R9S
The 16GB module are quad-rank (as far as I understood the 5520 chipset specification this is necessary, dual-rank shouldn´t work here):
Samsung M393B2G70BH0
I set all timinigs manually and the voltages as well, but no chance to boot after saving bios. After CMOS reset boot success is 100%, so there should be a way to get it running...
#35
gordan79
SSC Member
  • Total Posts : 531
  • Reward points : 0
  • Joined: 2013/01/27 00:17:36
  • Status: offline
  • Ribbons : 3
Re: SR-2 - 96GB Secret Sauce 2014/07/25 04:04:45 (permalink)
QR modules are only necessary if you have 12 DIMMs/CPU, which obviously isn't possible on the SR-2. QR modules also necessitate down-clocking the memory, which is also problematic - I have found that if I keep memory at 1333 it works with 12x8GB, but if I reduce the RAM clock speed to 1066 or 800 it doesn't.
 
Requirement for QR is in no way related to DIMM size (other than that you cannot get SR or DR DIMMs over a certain size due to chip size limits), it is to do with the number of DIMMs/channel. This is somewhat counter-intuitive because more ranks means more interleaving, and each DIMM slot on a channel is effectively an extra rank (having 1x DR DIMM on a channel is electrically near-identical to have 2x SR DIMMs on a channel), but there are also timing and interference details to consider.
 
Also, RAM has nothing to do with the 5520 chipset - memory controllers are inside CPUs now, and the NB (5520) doesn't do that much more than provide PCIe connectivity.
 
I only use Crucial RAM so I don't know about compatibility of other manufacturers' modules, but if you set the command rate to 2T it should work. It can still take a minute or two to complete the POST from a cold boot (you'll notice several cycles on the display on the motherboard where the POST cycles to d8, d9, then resets and retries), but it should come up after a little while. On regular resets it comes up without an extended POST-ing delay.
 

Supermicro X8DTH-6, 2x X5690
Crucial 12x 8GB x4 DR 1.35V DDR3-1600 ECC RDIMMs (96GB)
3x GTX 1080Ti
Triple-Seat Virtualized With VGA Passthrough (KVM)
#36
jjnunes
New Member
  • Total Posts : 9
  • Reward points : 0
  • Joined: 2010/05/02 17:34:26
  • Status: offline
  • Ribbons : 0
Re: SR-2 - 96GB Secret Sauce 2014/07/28 11:44:44 (permalink)
I run 6x8GB non registered ram on an SR-2 (x5680 CPU) and only have had problems trying to OC at the ragged edge. It's 2133MHz G Skill and I run it @ 180 bclk X10 with timings 9 9 9 26. (11 11 11 31 @ the rated 2133 spd)
 
This board typically not recognizing 16GB registered sticks is a bizarre problem. I'm glad to see it's not insurmountable.
post edited by jjnunes - 2014/07/28 15:58:49
#37
falco81
New Member
  • Total Posts : 2
  • Reward points : 0
  • Joined: 2014/08/24 07:02:55
  • Status: offline
  • Ribbons : 0
Re: SR-2 - 96GB Secret Sauce 2014/08/25 17:09:06 (permalink)
Hey everyone - just wanted to share my experience with my Sr-2 and let you all know I have 96gb running perfectly. it's slightly overclocked @ about 1900mhz-1950mhz.  Using G-skill Trident 2133 Cl9 1.65v 8GB memory sticks and dual Xeon E-5645's.
 
I did have some frustrating experiences with it only registering 80gb out of the 96gb that was installed, but this has been resolved with the A58 bios and some modifications to BIOS settings.
 
FINALLY I have this box running stable with ALL 96gb and I thought that if anyone else dealt with even a small percentage of the time/money/and frustration I spent on this motherboard, I ought to make an effort to share the knowledge!
 
Let me know if anyone would like some more information about my setup and bIOS settings and Pics... 
 
#38
mpogr
New Member
  • Total Posts : 34
  • Reward points : 0
  • Joined: 2007/01/26 01:55:44
  • Status: offline
  • Ribbons : 0
Re: SR-2 - 96GB Secret Sauce 2014/08/25 17:20:41 (permalink)
By now I think it's pretty clear the main ingredient to the 96GB configuration success is the CPUs. Everyone struggling has older/dodgier (ES) CPUs. Newer CPUs seem to fare significantly better, regardless of memory type and/or settings.
@falco81: please, do share your settings, including CPU exact model/steppings/production site/date.
#39
jjnunes
New Member
  • Total Posts : 9
  • Reward points : 0
  • Joined: 2010/05/02 17:34:26
  • Status: offline
  • Ribbons : 0
Re: SR-2 - 96GB Secret Sauce 2014/08/25 18:18:43 (permalink)
It's well known that the CPU makes a big difference. For example, the i980x/i990x desktop CPU's handle high memory loads much better than it's underlings. I ran the same 48GB of RAM at full 2133 MHz with no problems whatsoever with my i980x, despite the published figure being a max of 24GB. There address lines are there. The Xeon x5680/90 are essentially the same CPU as the desktop ones earlier mentioned, with the only differences being the QPI ports so dual CPU's will talk to one another and also the support to run registered ECC sticks. The only other difference is the max memory multiplier being 10x on the Xeon's and the desktop versions having 14x. Somebody please correct me if I'm wrong, but the oft said thing about Xeons being made from "higher binned silicon" appears to just be hearsay. Is there actually documentation from Intel that indicates this? Or is it private conversations with engineers in the inside know?
 
If you look at Supermicro's socket 1366 boards, even they don't recommend more than 24GB of unregistered RAM per CPU.
 
I haven't tried sticking all 48GB of unregistered RAM to be controlled by only 1 CPU on my SR-2 as I'm done fooling with it. But it wouldn't surprise me to find out that it would work. 
#40
gordan79
SSC Member
  • Total Posts : 531
  • Reward points : 0
  • Joined: 2013/01/27 00:17:36
  • Status: offline
  • Ribbons : 3
Re: SR-2 - 96GB Secret Sauce 2014/08/25 23:58:35 (permalink)
The key ingredient is hard-setting all of the memory options and disabling all auto-detection, especially the command rate which must be set to 2T.
 
In general, Intel specs tend to be based on what the chipset is known to be capable of. If the Intel spec sheet says 24GB, then your chances of getting more than 24GB working are pretty close to 0. Note that what the Intel spec sheet says and what the motherboard manufacturer's spec sheet says are completely different things. For example, intel spec sheet says Xeon X56xx CPUs can handle 192GB of RAM, which means the SR-2 should be able to handle 384GB, which just plainly isn't going to happen due to the BIOS deficiencies.
 
AMD, OTOH, tend to put in their spec what they have tested, so if biggest DIMMs available at the time were, say, 16GB, the spec will say that the chip will support that. If larger DIMMs become available later, they usually work, too. A prime example is the HP G7 Microserver. When they were released biggest DDR3 ECC UDIMMs were 4GB so the machine was rated for 8GB. But now that 8GB ECC UDIMMs are available, they work just fine in the G7 Microservers. Similar with my old Phenom motherboard, it was rated for 8GB (4x2GB), but with 4GB DIMMs it works just fine with 16GB.
 
Xeons aren't different in manufacturing quality per se. They differ in features (e.g. no ECC on desktop CPUs).
 
The key word in what you are quoting forSupermicro motherboards is "unregistered". I have only ever tried registered ECC RAM in the SR-2. Why would you even bother with anything else?

Supermicro X8DTH-6, 2x X5690
Crucial 12x 8GB x4 DR 1.35V DDR3-1600 ECC RDIMMs (96GB)
3x GTX 1080Ti
Triple-Seat Virtualized With VGA Passthrough (KVM)
#41
jjnunes
New Member
  • Total Posts : 9
  • Reward points : 0
  • Joined: 2010/05/02 17:34:26
  • Status: offline
  • Ribbons : 0
Re: SR-2 - 96GB Secret Sauce 2014/08/26 23:27:13 (permalink)
It's what I had and it does the job. Mine isn't a machine running scientific models, where a random drop of one bit can possibly skew the results, but a music practice machine.
 
"You can get away with murder in music!"
 
- Anon
#42
bbasbagill
New Member
  • Total Posts : 1
  • Reward points : 0
  • Joined: 2011/12/20 18:01:25
  • Status: offline
  • Ribbons : 0
Re: SR-2 - 96GB Secret Sauce 2015/04/24 06:11:38 (permalink)
Sorry to keep such an old thread on life support, but my google-fu keeps turning this up for going 96gb on the SR-2.  The information in this thread seems to be the furthest down the rabbit hole people have gone.  I was recently able to go 96gb with my SR-2 and wanted to share the experience in case anyone else is still wrestling with one of these beasts.
 
I started off with the SR-2 running two Xeon E5645 SLBWZ CPU's and 32GB of G.SKILL ripjaws non-ECC.  Everything ran smooth but I wanted to take the 96gb plunge.  I had a hard time finding the Crucial memory people keep reporting success with at a reasonable price.  Instead, I found a bargain on 12 sticks of Samsung M393B1K70CH0-YH9 (12x8GB, DDR3-1333 ECC CAS9) and pulled the trigger.
 
With that configuration I would always get stuck in the b8/b9/CF bootloop that most of us are familiar with unless I cleared the CMOS.  I reflashed the latest firmware, manually set the timings, and fiddled with different bios settings based on forum posts from around the internet to no avail.  The trick of hitting <DEL> for BIOS, then <ESC> and <Enter> to boot without saving any changes.. instead of hitting the clear CMOS button all the time... made the system usable but then the clock was reset all the time (which gave me false filesystem and certificate errors until the NTP updated the clock) and I couldn't enable any of the nicer options in the BIOS.
 
I then found a bargain on a pair of Xeon X5690 SLBVX CPUS's from Costa Rica and upgraded the configuration.  Now everything works well.  I don't have to clear the CMOS anymore, BIOS settings are saved, and I can enable AHCI, Full speed USB, disable the logo, etc.  I will admit posting is slow and sometimes it goes through the b8/b9/CF loop once or twice but the system always posts within 30-45 seconds (which can seem like forever sometimes).
 
This seems to be consistent with the theory that there is some sort of BIOS bug interfering with the memory controllers on the Xeon's.  Newer CPU's work better, look for the costa rican chips, etc.
 
Did an upgrade to Ubuntu 15.04 yesterday.. 24 cores @ 3.46Ghz (w/HT), 96gb ram, and all 6 displays come up on my Eyefinity6 card.  Life is good.
#43
gordan79
SSC Member
  • Total Posts : 531
  • Reward points : 0
  • Joined: 2013/01/27 00:17:36
  • Status: offline
  • Ribbons : 3
Re: SR-2 - 96GB Secret Sauce 2015/04/24 06:34:53 (permalink)
bbasbagill
Sorry to keep such an old thread on life support, but my google-fu keeps turning this up for going 96gb on the SR-2.

 
That's because there's only about 3 of us that have done it. :-p
 
bbasbagill The information in this thread seems to be the furthest down the rabbit hole people have gone.  I was recently able to go 96gb with my SR-2 and wanted to share the experience in case anyone else is still wrestling with one of these beasts.

 
Make that 4 of us. :-)
 
bbasbagillI started off with the SR-2 running two Xeon E5645 SLBWZ CPU's and 32GB of G.SKILL ripjaws non-ECC.  Everything ran smooth but I wanted to take the 96gb plunge.  I had a hard time finding the Crucial memory people keep reporting success with at a reasonable price.  Instead, I found a bargain on 12 sticks of Samsung M393B1K70CH0-YH9 (12x8GB, DDR3-1333 ECC CAS9) and pulled the trigger.

 
Right, registered ECC x4. Seems pretty equivalent to the Crucial DIMMs I'm using.
 
bbasbagillWith that configuration I would always get stuck in the b8/b9/CF bootloop that most of us are familiar with unless I cleared the CMOS.  I reflashed the latest firmware, manually set the timings, and fiddled with different bios settings based on forum posts from around the internet to no avail.  The trick of hitting <DEL> for BIOS, then <ESC> and <Enter> to boot without saving any changes.. instead of hitting the clear CMOS button all the time... made the system usable but then the clock was reset all the time (which gave me false filesystem and certificate errors until the NTP updated the clock) and I couldn't enable any of the nicer options in the BIOS.
 
 
Did you hard-set the command rate to 2T? It will default to 1T and I found it is absolutely crucial to hard-set this to 2T to get it to POST.
 
bbasbagillI then found a bargain on a pair of Xeon X5690 SLBVX CPUS's from Costa Rica and upgraded the configuration.  Now everything works well.  I don't have to clear the CMOS anymore, BIOS settings are saved, and I can enable AHCI, Full speed USB, disable the logo, etc.  I will admit posting is slow and sometimes it goes through the b8/b9/CF loop once or twice but the system always posts within 30-45 seconds (which can seem like forever sometimes).

 
That is as good as it gets. I have it working on two different motherboards with all of my CPUs, including X5690s and X5650s.
 
bbasbagillThis seems to be consistent with the theory that there is some sort of BIOS bug interfering with the memory controllers on the Xeon's.  Newer CPU's work better, look for the costa rican chips, etc.

 
I am still reasonably convinced it is a simple timeout issue. BIOS simply doesn't give the MCH enough time to check and initialize the RAM registers. A BIOS developer probably erroneously thought they were being clever and shaved a second or two off the POST time by shortening the timeout without actually considering the consequences. It would probably be a trivial fix if they could be bothered, but there are far more serious and more obvious bugs in the SR-2 BIOS (and hardware for that matter) that this particular issue is pretty far down the list of criticality of bugs that will never get fixed.
 
 
bbasbagillDid an upgrade to Ubuntu 15.04 yesterday.. 24 cores @ 3.46Ghz (w/HT), 96gb ram, and all 6 displays come up on my Eyefinity6 card.  Life is good.

 
Sweet. Mine are running as multi-seat VM workstations with Nvidia cards.

Supermicro X8DTH-6, 2x X5690
Crucial 12x 8GB x4 DR 1.35V DDR3-1600 ECC RDIMMs (96GB)
3x GTX 1080Ti
Triple-Seat Virtualized With VGA Passthrough (KVM)
#44
jayjay1986
New Member
  • Total Posts : 9
  • Reward points : 0
  • Joined: 2015/02/11 06:05:14
  • Status: offline
  • Ribbons : 0
Re: SR-2 - 96GB Secret Sauce 2015/06/02 02:44:12 (permalink)
lol and i am getting troubles to get my 48 gb to work , 6x 8gb hynix 1333 ecc
 
i am only getting 32 gb
 
3 red slots cpu 1
1 red slot cpu 0
 
2 slots  (dim 3 and dim 5)on cpu 0 site dont seem to work (ram is tested and ok)
 
2x 5680's at default no overclocks
 
 
#45
gordan79
SSC Member
  • Total Posts : 531
  • Reward points : 0
  • Joined: 2013/01/27 00:17:36
  • Status: offline
  • Ribbons : 3
Re: SR-2 - 96GB Secret Sauce 2015/06/03 02:13:09 (permalink)
Have you done the basic testing, one DIMM at a time in each of the red sockets to verify that the sockets all work?

Supermicro X8DTH-6, 2x X5690
Crucial 12x 8GB x4 DR 1.35V DDR3-1600 ECC RDIMMs (96GB)
3x GTX 1080Ti
Triple-Seat Virtualized With VGA Passthrough (KVM)
#46
jayjay1986
New Member
  • Total Posts : 9
  • Reward points : 0
  • Joined: 2015/02/11 06:05:14
  • Status: offline
  • Ribbons : 0
Re: SR-2 - 96GB Secret Sauce 2015/06/04 14:27:53 (permalink)
I didnt do that yet , first trying t2 command line setting for ram , can anyone tell me the right settings Out full in this thread
#47
gordan79
SSC Member
  • Total Posts : 531
  • Reward points : 0
  • Joined: 2013/01/27 00:17:36
  • Status: offline
  • Ribbons : 3
Re: SR-2 - 96GB Secret Sauce 2015/06/06 07:13:18 (permalink)
I only ever used registered ECC RAM on my SR-2s, so cannot comment on whether 8GB non-registered DIMMs would work.

Supermicro X8DTH-6, 2x X5690
Crucial 12x 8GB x4 DR 1.35V DDR3-1600 ECC RDIMMs (96GB)
3x GTX 1080Ti
Triple-Seat Virtualized With VGA Passthrough (KVM)
#48
jjnunes
New Member
  • Total Posts : 9
  • Reward points : 0
  • Joined: 2010/05/02 17:34:26
  • Status: offline
  • Ribbons : 0
Re: SR-2 - 96GB Secret Sauce 2015/06/25 19:19:01 (permalink)
gordan79
I only ever used registered ECC RAM on my SR-2s, so cannot comment on whether 8GB non-registered DIMMs would work.




I've got it running with 12x8 non-registered DIMMS. See one of my posts from last August for the brand and model.
 
I made no changes from running 6x8, except changing MCH strap to 1600 and command rate to 2T. With 8x6, I did not need to modify those from the defaults. 
#49
Mzilikazi
New Member
  • Total Posts : 2
  • Reward points : 0
  • Joined: 2015/10/12 15:28:07
  • Location: London
  • Status: offline
  • Ribbons : 0
Re: SR-2 - 96GB Secret Sauce 2016/01/16 14:50:59 (permalink)
I realise I'm a bit late to the party but, has anyone ever tried large capacity Quad ranked LRDIMMs such as CT16G3ELSLQ8160B (16GB), CT3K32G3ELSLQ41339 (3 x 32GB kit) or even KTD-PE318LQ/32G, which is also a 32GB reduced load stick?

EVGA Classified SR-2
24GB OCZ3G1333LV12GK 1333MHz 
GeForce GTX 480 1536MB
HyperX Savage SHSS37A240G
Lian Li PC-V2120B

[image]F:\Google Drive\EVGA Classified SR-2\setup experiments\16-10-2015\CoreTemp-Scr.png[/image]
#50
Dr_Decon
New Member
  • Total Posts : 1
  • Reward points : 0
  • Joined: 2016/01/16 15:42:37
  • Status: offline
  • Ribbons : 0
Re: SR-2 - 96GB Secret Sauce 2016/01/16 15:49:59 (permalink)
Don't exactly know where to posts this, but I recently acquired a NIB SR-2 to replace my X8DAH+-F-O and am encountering issues with running 6*8gb ECC Reg DDr3.  This was also an issue on the SM board.  Occasionally on boot one stick is ignored but the system works fine.  Power cycling fixes the issue. Any idea what's wrong here? I'm assuming this isn't an issue with the SR-2 since it affected the SM board as well.  ATM I'm guessing it's either 2-way QPI (never get this issue when just running one CPU) or a bad MCH.  Any idea what's causing this?
 
Im running:
 
EVGA SR-2 (duh)
E5645s X2 (OC'd 180 BCLK, OC disabled ATM)
Dual Noctua 1366 Coolers
6*8gb ECC DDR3 Sammies
GTX 580
GTX 750ti (physx)
 
Sorry for posting this in the 96gb thread but i didn't know where else to put it
#51
gordan79
SSC Member
  • Total Posts : 531
  • Reward points : 0
  • Joined: 2013/01/27 00:17:36
  • Status: offline
  • Ribbons : 3
Re: SR-2 - 96GB Secret Sauce 2016/01/17 06:53:01 (permalink)
Mzilikazi
I realise I'm a bit late to the party but, has anyone ever tried large capacity Quad ranked LRDIMMs such as CT16G3ELSLQ8160B (16GB), CT3K32G3ELSLQ41339 (3 x 32GB kit) or even KTD-PE318LQ/32G, which is also a 32GB reduced load stick?

 
LR DIMMs for DDR3 are essentially the same thing as Fully Buffered DIMMs were back in the DDR2 days. Officially, SR-2 only support unbuffered and registered memory, not LR memory, but AFAIK nobody has actually tried LR memory. If you have some handy, I'm sure there are some of us here who would be interested in your results.
 
AFAIK nobody has managed to get 16GB DIMMs working on the SR-2. 96GB with 8GB RDIMMs works, but it can take a while to POST (the BIOS tries to initialize the memory, fails, tries again, wash, rinse, repeat, until eventually succeeds which can take 30 seconds, or it can take many minutes depending largely on luck. CPUs with later manufacture dates seem to do better. My X5690s POST within 30 seconds 99% of the time with 96GB.
 
I don't recall if anybody has tried quad ranked RDIMMs. In general, that requires downclocking, according to Intel spec. Whether the SR-2 BIOS will do that or just fail to POST, I don't know. "Ranks" behave almost the same as multiple sockets on the same channel, so electrically a single dual ranked DIMM is roughly equivalent to two single ranked DIMMs in the two sockets for the same memory channel.

Dr_Decon
Don't exactly know where to posts this, but I recently acquired a NIB SR-2 to replace my X8DAH+-F-O and am encountering issues with running 6*8gb ECC Reg DDr3.  This was also an issue on the SM board.

 
I dare say you have some duff DIMMs if that is the case. Replacing the SM board with the SR-2 will be a giant leap backward in just about every way.
 
Dr_Decon
Occasionally on boot one stick is ignored but the system works fine.  Power cycling fixes the issue. Any idea what's wrong here?

 
My bet would be on a marginal DIMM. Have you tracked it down to a specific DIMM that is being intermittent? You should be able to check the EDAC registers for which module isn't showing up.
 
Dr_Decon
I'm assuming this isn't an issue with the SR-2 since it affected the SM board as well.  ATM I'm guessing it's either 2-way QPI (never get this issue when just running one CPU) or a bad MCH.  Any idea what's causing this?

 
Bad QPI would make the 2nd CPU not show up either. You need to do more testing to establish whether the missing DIMM follows a particular CPU or a particular DIMM.
 
Dr_Decon
Sorry for posting this in the 96gb thread but i didn't know where else to put it

 
The correct thing to do would be to start your own thread on the subject.

Supermicro X8DTH-6, 2x X5690
Crucial 12x 8GB x4 DR 1.35V DDR3-1600 ECC RDIMMs (96GB)
3x GTX 1080Ti
Triple-Seat Virtualized With VGA Passthrough (KVM)
#52
MagicManRed
New Member
  • Total Posts : 1
  • Reward points : 0
  • Joined: 2016/03/21 19:42:34
  • Status: offline
  • Ribbons : 0
Re: SR-2 - 96GB Secret Sauce 2016/03/21 19:48:04 (permalink)
naimc
Thank you  bygordan79 .
 
I followed your steps and got my SR-2 with dual x5560's working with 96 GB of ram using 6 x 16GB Dual Rank DIMMS (Kingston KVR13LR9D4K3/48,  () 
 
Excellent.


This must have been a joke.

I bought the same exact ram.. Updated my bios for good measure.
Even reset my bios... Can't post at all. Stuck at b9 post code.
What a waste... Now I have to fork over a 25% restocking fee.

Would have been more helpful if you posted all of your config info.

Has anyone even gotten 48gb to work without a headache?
This is pathetic that this board has these kind of issues.
#53
gordan79
SSC Member
  • Total Posts : 531
  • Reward points : 0
  • Joined: 2013/01/27 00:17:36
  • Status: offline
  • Ribbons : 3
Re: SR-2 - 96GB Secret Sauce 2016/03/22 03:11:06 (permalink)
48GB using 4GB DIMMs should work perfectly. If it doesn't you have either duff CPUs or a duff board.
 
The thing you need to note is that not all motherboards and CPUs are the same. I have already observed that newer CPUs (in terms of manufacturing date, not just stepping) POST more reliably with 96GB of RAM. Additionally, CPUs with fewer cores may do better, too.
 
My limited testing indicates that 96GB using dual-rank 8GB DIMMs works, but using single-rank 8GB DIMMs does not.
 
Long story short, if you want to use more than 96GB of RAM get a decent motherboard, such as one of the similarly specced Supermicro models.

Supermicro X8DTH-6, 2x X5690
Crucial 12x 8GB x4 DR 1.35V DDR3-1600 ECC RDIMMs (96GB)
3x GTX 1080Ti
Triple-Seat Virtualized With VGA Passthrough (KVM)
#54
Aiwandia
New Member
  • Total Posts : 37
  • Reward points : 0
  • Joined: 2015/02/03 08:26:20
  • Status: offline
  • Ribbons : 0
Re: SR-2 - 96GB Secret Sauce 2016/03/22 16:29:28 (permalink)
gordan79 
I can completely relate to your disappointment and exhaustion from this motherboard - it really is a product that was designed for claiming bragging rights but turned out to be an embarrassment instead.
 

This thread again, .
E-Bay has been busy with SR-2's this past month, both new and used.
5 year old embarrassment of a platform holds it's value remarkably.
Where else do you get to overclock dual CPU and try H2O on 8 GPU?
I'd like to run x4 Kingston KVR16E11K3/24 = 12x8GB = 96GB, DDR3 1600
12800 CL11 ECC 240 Pin UDIMM, but secret sauce didn't do the trick.
It's 100% Cu spreaders until I get the H2O loop completed.
Does UDIMM rule out 96GB or do I try it again?
 
 
 
 
  
   
#55
Aiwandia
New Member
  • Total Posts : 37
  • Reward points : 0
  • Joined: 2015/02/03 08:26:20
  • Status: offline
  • Ribbons : 0
Re: SR-2 - 96GB Secret Sauce 2016/03/23 09:45:23 (permalink)
MagicManRed
naimc
Thank you  bygordan79 .
 
I followed your steps and got my SR-2 with dual x5560's working with 96 GB of ram using 6 x 16GB Dual Rank DIMMS (Kingston KVR13LR9D4K3/48,  () 
 
Excellent.

Has anyone even gotten 48gb to work without a headache?
This is pathetic that this board has these kind of issues.

48GB is not any difficulty for me.
48GB Corsair Vengeance DDR3 1600 no problem.
48GB Kingston ECC DDR3 1600 no problem.
96GB Kingston ECC DDR3 1600 would be awesome
as the SR-2 was originally marketed as being overclockable with non-ECC RAM(only).
Is that to say x16GB Dual Rank DIMMS 96GB non-ECC with dual 4 core 5560 works
but with dual 6 core Xeon it may be more problematic?   
 
#56
gordan79
SSC Member
  • Total Posts : 531
  • Reward points : 0
  • Joined: 2013/01/27 00:17:36
  • Status: offline
  • Ribbons : 3
Re: SR-2 - 96GB Secret Sauce 2016/03/23 09:49:02 (permalink)
I am running 96GB as 12x8GB of ECC Registered dual-rank RAM @ 1660 (166 bclk), with a pair of X5690 CPUs (6 cores each). As per the signature.

Supermicro X8DTH-6, 2x X5690
Crucial 12x 8GB x4 DR 1.35V DDR3-1600 ECC RDIMMs (96GB)
3x GTX 1080Ti
Triple-Seat Virtualized With VGA Passthrough (KVM)
#57
Aiwandia
New Member
  • Total Posts : 37
  • Reward points : 0
  • Joined: 2015/02/03 08:26:20
  • Status: offline
  • Ribbons : 0
Re: SR-2 - 96GB Secret Sauce 2016/03/30 12:06:44 (permalink)
2x48GB(6x8GB) DDR3 PC3-10600 1.5v 1333MHz ECC REG. 2Rx4 Dural SDRAM Hynix
Is this 96GB suitable?
I do hope it does not depend on which particular SR-2 it is deployed on.
Should work with either 5670 or 5690 Xeons, no? 
#58
gordan79
SSC Member
  • Total Posts : 531
  • Reward points : 0
  • Joined: 2013/01/27 00:17:36
  • Status: offline
  • Ribbons : 3
Re: SR-2 - 96GB Secret Sauce 2016/03/31 07:15:16 (permalink)
It should be OK, but you may have better luck with 1.35V DIMMs.
The most important things is to put in 6 DIMMs first, manually set all the timings to the auto-detected values, set command rate to 2T, then add the other 6DIMMs.

Supermicro X8DTH-6, 2x X5690
Crucial 12x 8GB x4 DR 1.35V DDR3-1600 ECC RDIMMs (96GB)
3x GTX 1080Ti
Triple-Seat Virtualized With VGA Passthrough (KVM)
#59
raminux
Superclocked Member
  • Total Posts : 124
  • Reward points : 0
  • Joined: 2012/04/02 11:24:07
  • Status: offline
  • Ribbons : 0
Re: SR-2 - 96GB Secret Sauce 2016/05/30 21:55:10 (permalink)
I have ordered 96gb of Samsung 1600mhz ECC Registered dual-rank modules and they are on the way. I hope it is going to work out smooth.

Asus Z10PE-D16 WS, 2x Xeon E5-2666v3, 2x Scythe Ninja 4, 64GB Samsung 2133MHz ECC LRDIMM, Nvidia Quadro M4000, Samsung 256GB 950 Pro nvme system drive, 2x 1TB Samsung 850 Pro in Raid 0, Seasonic X-1250, Mountain Mods U2-UFO, Dell Ultrasharp UP3216Q, Windows 10 Professional 64bit, Linux Ubuintu 64bit in VmWare; Heatware: http://www.heatware.com/eval.php?id=80690
#60
Page: < 1234 > Showing page 2 of 4
Jump to:
  • Back to Mobile