CZ_MaWa

I am not "CPU guru" so I cant tell for sure, but when i look at the intel specs sheets here http://ark.intel.com/prod...-6_40-GTs-Intel-QPI%29 and here http://ark.intel.com/prod...GHz-6_40-GTs-Intel-QPI it looks like x5650 and x5690 are totally same CPUs except frequency.

X5650 has highest multiplier x21 i think so if you would get 12x 8GB RAMs at about 170 BCLK, that would mean 170 x 21 = 3 570 MHz.

 
Thank you! Well you have the experience, guru or not. Guru status probably is in the eye of the one asking for assistance!  I haven't got the details on the set up, but I assume the guy that got stable 4.0Ghz with 12x4GB NON ECC RAM and the X5650s must be using a higher multiplier. So, I guess that would be because he's using gear that can handle 190 BLCK, or so. So I may have to decide wether to go for the clocking, or the RAM... since the X5650 is limited.
 
BTW your memories are OC to 1700, but are 1333 rated. scsi2man uses 1600 ones. Why go for the 1333? (Price is obviously one, for me), lower latency, or same oc ability?

tived

Damn, I just reread this thread and noted that SCSI2MAN mentioned that he could not get 16GB sticks running in the SR-2 :-) I thought for a moment I could be sneeky :-) but no, just as well I am sitting with my finger on the BUY button.
So its back to the Crucials 12 x 8gb...hmmm

thanks guys
Henrik

How sure are you that the Crucials are going to make a difference? It seems jon98064 got them at it wasn't working for him (or perhaps just didn't wait long enough during booting?).

Gadgety1


Thank you for the rapid reply! I'm in Sweden. I agree 100% if the price difference is that tiny. Would know if your CZ supplier ships here? Could you either post here or PM me?

I need to state I'm not experiencing any problems, other than budget and decision making, as I have not yet got my unit up and running. Just ordered the case, PSU (starting with only one Silverstone Strider 1500W). Got the GPUs. I'm straining my budget and even went with the wrong color on the case because it's 65€ cheaper, so if the Kingstons you have work, and are that much cheaper, I don't think the Crucials would be worth that much of a premium.

 
I have jsut checked some Sweden online shops and you are right, in the Sweden ECC kits are extremely expensive, at least what I found - 4x 8GB non ecc kingstons for 1199 SEK, 2x 8GB ECC kingstons for 1299 SEK - thats more than double . Really strange. Of course I can send them from Czech republic to you, but i think you can get it somewhere near you, in example Germany http://geizhals.de/428793

Gadgety1

BTW your memories are OC to 1700, but are 1333 rated. scsi2man uses 1600 ones. Why go for the 1333? (Price is obviously one, for me), lower latency, or same oc ability?

 
Here in CZ i can buy CL9 1333MHz ECC or CL10 1600MHz ECC Kingstons. The price is almost the same, because the sticks are totally same, just another SPD in them. I didnt tried any CL9 1600MHz ECC, so i dont know if it would be possible to OC them higher tham 180 BCLK. I dont need it, 4.42GHz is perfectly OK for me.
I think that 6x 8GB over 180 BCLK is possible. I can try it with my own machine, but not sooner than in first January week...

scsi2man

echrei

scsi2man
That depends on if you use ECC non registered or ECC Registered. The ECC Registered does add one cycle of latency due to the register on the stick. ECC registered is recommended only if stability is of utmost importance. It is generally more expensive than non ecc memory, but lately the prices for both have been fluctuating a lot. Since you are already using non ECC memory, I would suggest you stick with that as it would be the least expensive option at this point. As far as I know and have tested, the non ECC non registered 8GB sticks do not work in this board. I have only been able to get the ECC registered 8GB sticks to work in this board. The 8GB non ECC sticks will work in the newer Sandy Bridge boards, however. As far as memory expansion, it also depends on if you use 4GB sticks, or 2GB sticks, and how many slots you use. If you aren't planning to go above 48GB, I suggest going with 4GB non ECC sticks. Depending on how high you plan to overclock (if you do) I'd go with the 1600 or 1866 sticks. You can find lots of them on Newegg, which is where I have bought most of my memory.  

So people have tried RAM kits such as the following with the SR-2 and they do not work? I was looking at keeping it to 3 DIMMs per CPU since I didn't want to incur the memory multiplier reduction. This would require 8GB DIMMs if I wanted to move to 48GB.

http://www.newegg.com/Pro...x?Item=N82E16820231359

I'm not sure what you mean by the memory multiplier reduction. That does not occur with the 5600 series xeons, only the 5500 series. If you put more than 1 dimm per channel on a 5500 series xeon it will downclock. But you can have 2 per channel on the 5600 series and it won't downclock. So this memory multiplier situation you are referring to only exists with the 5500 series xeons. I have referenced for you from Intel's own website about the additions that the 5600 series xeons have over the 5500 series.
http://www.intel.com/cd/channel/reseller/asmo-na/eng/products/server/intelligentxeon/440799.htm

So as you can see, people claiming that 2 dimms per channel will downclock is not exactly accurate. For the 5500 series, yes. I run my memory with the 1333 memory strap and it runs it perfectly. And at the default bclk of 133, my memory runs at 1333. But since I've overclocked it to 160, it runs at 1600.  I've also tried and have used regular non-ecc non-registered ddr3 1600, 1866, and 2000 memory in both channels per cpu and have NOT experienced any downclocking at at all. The dimms I used were from Corsair, Crucial, Kingston, Gskill, Samsung, and Hynix. Also, if you read the motherboard manuals from various manufacturers on their boards that use the 5520 chipset like this board, you will also find in the documentation  that 2 dimms per channel do not downclock the memory.

One of the other 5520 based boards I've used is the Supermicro X8DA3, X8DAi, and X8DAH+. If you look  on pages 2-8 and 2-9 of the manual for this board, you will find that it explains very thoroughly the memory arrangement for both the 5500 and 5600 series xeons. 
http://www.supermicro.com/manuals/motherboard/5500/MNL-1031.pdf

You will find using the same non-ecc non-registered ddr3 sticks in this board will give you the identical memory configurations to the SR-2. How do I know? I've tried them and have done it on numerous occasions when I have tested memory for both boards. 

http://www.supermicro.com/manuals/motherboard/5500/MNL-1060.pdf
You will find though on the motherboards that have 18 memory slots instead of 12, that when you have 3 dimms per channel for the 5600 series it WILL downclock the memory. For this, I reference pages 2-10, 2-11, and 2-12 of the X8DAH+ manual listed above as MNL-1060.pdf. There you will also find the same information as stated in the manual above with the exception of the addition of a 3rd memory slot configuration. 

This is also true of the Asus Z8PE-D12 as I own that board as well. Same scenario as with the EVGA and Supermicro boards: 2 dimms per channel for the 5600 series xeons does NOT downclock the memory if it is single or dual rank. If it is QUAD rank, then it will downclock for 2 dimms per channel. But being that I have not seen regular DDR3 that is quad rank, I don't see where this would be an issue. 
Because all these boards have the same chipsets, they also follow the same rules for memory arrangement as far as dimms per channel goes. I suggest for those that are having issues with the memory to make sure they are supported. While ECC Registered memory generally is not particularly good at overclocking, you can still overclock your motherboard should you choose to use them by simply changing the memory divider from 1333 to 1066, and from there you can increase the bclk while still keeping your memory within rated spec. Once again, it all depends on what memory you use and from which manufacturer. Hope this helps.

A really useful post. The Intel link provides superficial data, as I have to be a reseller to access profound stuff. In my search elsewhere I found this on IBM's servers when populating their 5520 based servers with 5500 and 5600 series CPUs, and the ins and outs of memory performance, channels, dual-rank, singular-rank populating all slots or not, theoretical effects, effects on actual performance depending on tasks etc. I found it a really good read, although of course the SR-2 board is a different animal.
ftp://public.dhe.ibm.com/...5usen/XSW03075USEN.PDF

Gadgety1
Looking into the 8GB ECC units, the manager went into data that are inaccessible to normal users, and recommended the  following:

CT102472BD160B   240-pin DIMM  DDR3 PC3-12800 • CL=11 • Unbuffered • ECC • DDR3-1600 • 1.35V
or the sister module PC3-10600, DDR-1333, CL-9, so slower speed, lower latency, CT102472BD1339. Both are ~€101, or €202 for the equivalent 16GB, that is almost 90% more expensive. All prices include 20%VAT.

I got these exact memory modules (in kits of 3) and more than 6 of them won't POST with the board, no matter what I tried. As I had previously with my Kingston ECC Registered modules, the board POSTs ONLY after CMOS reset, but any settings saved into the BIOS (even the default ones) make it endlessly cycling through CF .. 68 69 codes.

With only two kits (6 modules, 48GB of memory in total) you will be fine, the problems with POST begin when you try more than 6 of these modules. Besides, I think your modules are different (BB vs. BD).

Gadgety1
Thanks. I got these with the possibility of adding more ram in the future, which is why I went with 8GB modules. I went with these rather than the ones suggested by the Crucial manager because I had later seen that its better to get registered ones.

You will still most probably be fine with 6 modules only, regardless of whether you use unbuffered or registered modules. I tried 3 different kinds (Kingston KVR13R9D4/8I, which is 1333MHz 9-9-9 ECC registered, Crucial CT3KIT102472BB160B, which is exactly what you've got, and Crucial CT3KIT102472BD160B, which only differs in a sense that it's unbuffered and rated at 1.35V rather than 1.5V), and all of them behaved exactly the same way:

• 6 modules work completely fine, and overclock roughly the same
• Any amount above 6 boots fine after clearing CMOS. However, as soon as any BIOS settings are saved (even the defaults), it doesn't boot anymore, endlessly cycling through CF ... 68 69 codes. I opened a thread on this subject and looks like there is at least one additional person (strangely enough, also from Australia) who experienced exactly the same behaviour so far.

Gadgety1 
What do I know, I have so much to learn and theres a lot of confusing info out there. As for BB and BD - I can't remember what the difference is, do you know? I chatted with a Crucial employee in the states who said that if BD works BB should work etc.

BD is unbuffered and BB is registered. And what that Crucial employee told you is most definitely not true. I have 2 additional systems (based on ASUS P6T WS and ASUS P8C mobos) which work only with ECC unbuffered, but not ECC registered modules. This is clearly stated in their manuals and I verified it myself as well.

Vapochilled

Hi,

I can also confirm i am using Samsung 8 GB * 6 (total 48GB) and the system boots with success. However with OC memory is gone i only see 32Gb :(

 
put more voltage to RAM modules and it will be OK. on 1.65V our boards detect sometimes 80GB, sometimes 96GB. with 1.7 or 1.75v it detects every time al capacity, 96GB. also depending on the exact modules type and cpu type, you may need to be sure that they are not working over 1333MHz or 1066MHz...so use the proper divider.

CZ_MaWa
with 1.7 or 1.75v it detects every time al capacity, 

Doesn't using above 1.65V put in danger the CPU?

CZ_MaWa 
96GB

 
I'm trying to get these working desperately without much luck so far, tried almost everything imaginable, but not raising VDIMM above 1.65V (it's been reported as 1.67V in the BIOS). Kind of scary raising the voltages even more than that...

mpogr

CZ_MaWa 
how long does for you mean "endlessly" ? in example, 96GB with our boards, everything on default, BIOS reset, then boots, in bios loaded defaults, restart - cycling 68, 69 about five times, than it will boot. but when we OC it, it last 10 or 15 cycles until it will boot. which last something about 8 or 9 minutes. but if you put everything correct in bios, it WILL finally boot and all rams will be detected, memtest stable, totally stable. so its definitelly NOT about luck, its about knowing what to do and where to do and its about patience

I did leave it overnight (well above 8 hours), still no luck...

 
which bios ?
do you have twelve exactly same RAMs with same part number (and the best would be if bought together in one time) ? which exactly ?
what cpus you have ?
does it boot with all memory when you reset bios to default ?

put there a56
put there the kingstons - i dont know this exact product type but it looks like they are about the same spec as my kits (1333, 1.5v, CL9).
reset bios
boot into bios
disable all controllers - usb3, esata, lan, everything.
qpi link 4.8G
memory ddr3-1333
mch strap 1867
voltage cpu - without vdroop, 1.3v an both cpus, eventual 1.3v on both cpus
voltage vtt - 1.3v
voltage memory - 1.7 or 1.75
voltage ioh 1.4
 
save and restart and it HAS TO BOOT. but really, it can last up to 10 minutes with 68, 69 cycling through. but after that d4, d5 and voila...
 
if it wont boot, you have defective motherboard, incompatible cpus (for some strange reason) or incompatible RAM sticks. but i am pretty sure that by design the board itself is capable of 96GB ram...stable, forever, every time.
 
btw what does it do when you put four modules for each cpu ? not six, not three, but four ? does it hang too ?

So then your CPUs cant handle it or your rams are incompatible. Dont ask me why. Because i have six 5690 cpus (two of them two years old, four of them three months old), three sr-2 boards (one of them two years old, two of them three months old) and totally 36 pieces of kingston 8GB rams (six two years old, 30 three months old) here and no matter which cpu i put in which board and with which RAMs installed, its doing every time exactly the same (if i use the settings above), it detects all 96GB after few 68 69 cycles through, memtest stable. So i am really confident that i can say, that this whole problem is not motherboard fault. Unless your board is older than two years - maybe it has some different revision in that days, but i doubt it.
 
all i can do for you - send me mb, rams and cpu and i will figure it out very quickly - i will try my rams and cpu in your board and your ram and cpu in my boards.

hmm yes, thats kinda too far away  i thought you are from europe somewhere...

I read this whole thread and there's so much details and confusing parts, my head has come to absolutely 0 conclusions of any kind!
 
What's the fastest memory for a 48GB configuration?
And which will allow for the board to overclock the best.
And which can be purchased on Newegg/Amazon/Etc.
And will use 6 slots only.
 
Thanks!
Really