Helpful ReplyCooling 6 GTX580?

Author
MySchizoBuddy
New Member
  • Total Posts : 21
  • Reward points : 0
  • Joined: 2012/02/27 15:51:40
  • Status: offline
  • Ribbons : 0
2012/02/27 20:57:51 (permalink)
How do you liquid cool 6 or 7 GTX580 Hydro copper?
Can you even fit 7 of them on the Classified SR2 and connect them via 4way and 3way barbs.
Do you still use water or a different liquid?
What should be the flow rate?
What should be the radiator size and at what fan speed?
 
Is there a set of formulas i can use to calculate these parameters. 
 
PS: this setup is for CUDA/OpenCL, not gaming.
 
post edited by MySchizoBuddy - 2012/02/27 21:23:09
#1
malVizio
CLASSIFIED Member
  • Total Posts : 2392
  • Reward points : 0
  • Joined: 2011/09/26 10:24:30
  • Location: Affiliate code: A83U9ZY5VO
  • Status: offline
  • Ribbons : 7
Re:Cooling 6 GTX590? 2012/02/27 21:18:19 (permalink)
SLI it's possible only for at most 4 GPU's...
sincerelly i didn't know it's possible to run more for other purpose...

Case:Corsair Obsidian 650D
M.B.:ASRock Z68 Ext7 Gen3/PSU: Seasonic-X 850W Gold
GPU:2xEVGA GTX680 2GB/RAM:16GB DDR3 Corsair
CPU:Intel i5 2500k@4.6Ghz+Zalman CNPS9900 MAX
FANS:3xBitFenix Spectre PRO RedLed
SSD/HDD:OCZ Agility 3 60gb/Samsung 840 250gb/WD Raptor 300gb/Seagate Barracuda 500mb
MICE:Corsair M60/KB:Razer BlackWidow/HP:Roccat Kave 5.1/Shure SRH440
Screen:BenQ XL2420T+Asus PA238Q
#2
EliteGeek91
Omnipotent Enthusiast
  • Total Posts : 10669
  • Reward points : 0
  • Joined: 2011/02/10 09:40:19
  • Location: Censored!
  • Status: offline
  • Ribbons : 44
Re:Cooling 6 GTX590? 2012/02/27 21:19:29 (permalink)
malVizio

SLI it's possible only for at most 4 GPU's...

He might not be "SLI'ing" them. Maybe he wants raw GPU power for folding, crunching, etc.
 
I would help, but I don't have that much knowledge man. Someone on here can surely help you though!

#3
MySchizoBuddy
New Member
  • Total Posts : 21
  • Reward points : 0
  • Joined: 2012/02/27 15:51:40
  • Status: offline
  • Ribbons : 0
Re:Cooling 6 GTX590? 2012/02/27 21:22:26 (permalink)
malVizio
SLI it's possible only for at most 4 GPU's...
sincerelly i didn't know it's possible to run more for other purpose...

Not using SLI at all. it's purely for high performance computing. CUDA all the way. I know cuda has no problem accessing thousands of GPU cards. This setup is now used in Super computers. 
#4
tpb7463
ACX Member
  • Total Posts : 437
  • Reward points : 0
  • Joined: 2010/08/21 16:51:59
  • Location: Utica, NY
  • Status: offline
  • Ribbons : 9
Re:Cooling 6 GTX590? 2012/02/27 21:39:59 (permalink)
If I recall correctly, even though the slot config would support up to 7, 5 is the limit. 6 is bootable, but the system runs slowly with that many installed. (I believe one of the EVGA techs mentioned this) Honestly, I'd stick to 4 and save yourself the headaches.
 
The minimum cooling solution I'd attempt on a setup like that is a 4x120 rad on their own loop with an MCP655 pump and 1500rpm fans (push/pull). The MCP655 is more than enough for a loop like that.
 
Although, you've probably got the cash, so either a (3x120)x2 or a (4x120 + 2x120) is what you should really be shooting for.
 
Distilled water with a kill coil if you don't run nickel-plated blocks.   Oh right, hydrocoppers.
Distilled water with biocide if you have nickel in your system.
post edited by tpb7463 - 2012/02/27 21:43:27

SR-2 Buildlog: http://www.evga.com/forums/tm.aspx?m=1018539
EVGA SR-2 | 2 x 5650's @ 4 GHz | 2 x EVGA GTX590's | GTS 450
#5
EliteGeek91
Omnipotent Enthusiast
  • Total Posts : 10669
  • Reward points : 0
  • Joined: 2011/02/10 09:40:19
  • Location: Censored!
  • Status: offline
  • Ribbons : 44
Re:Cooling 6 GTX590? 2012/02/27 21:41:53 (permalink)
Heck or maybe even a SR1 560mm rad for all those...

#6
EVGATech_MattM
EVGA Tech Support
  • Total Posts : 1598
  • Reward points : 0
  • Joined: 2009/11/18 09:34:00
  • Status: offline
  • Ribbons : 3
Re:Cooling 6 GTX590? 2012/02/28 03:41:27 (permalink) ☄ Helpful
tpb7463

If I recall correctly, even though the slot config would support up to 7, 5 is the limit. 6 is bootable, but the system runs slowly with that many installed. (I believe one of the EVGA techs mentioned this) Honestly, I'd stick to 4 and save yourself the headaches. 

 
We get this question a lot and tpb7463 is correct.  5 max recommended.  It will run 6 but starts acting funny and 7+ will fit but the board will run out of resources and just not boot.
 
Please note that the PCIe lanes for the slots behave this way:
 
  • PCIe slot 1: 16x (8x if slot 2 is filled)
  • PCIe slot 2: 8x
  • PCIe slot 3: 16x (8x if slot 4 is filled)
  • PCIe slot 4: 8x
  • PCIe slot 5: 16x (8x if slot 6 is filled)
  • PCIe slot 6: 8x
  • PCIe slot 7: 16x
For high end cards the best performance is running 4 cards in slots 1, 3, 5, 7.
#7
MySchizoBuddy
New Member
  • Total Posts : 21
  • Reward points : 0
  • Joined: 2012/02/27 15:51:40
  • Status: offline
  • Ribbons : 0
Re:Cooling 6 GTX590? 2012/02/28 06:29:11 (permalink)
will this change in the newer SRX that was shown in CES2012.
 
@tpb7463 how did you get to the conclusion of 4x120 rads with the MCP655 pump? Is it from experience or there is a way to calculate it. I'm going to have 4 node cluster so I need one 4x120 rad for each node as minimum. Plus additional rads for the 8CPUs. 
 
Is there a liquid with better heat rejection capability than water so i can decrease the size of the radiators. 
post edited by MySchizoBuddy - 2012/02/28 06:31:41
#8
Sleinous
EVGA Forum Moderator
  • Total Posts : 7459
  • Reward points : 0
  • Joined: 2010/06/12 12:22:01
  • Location: Paris, France
  • Status: offline
  • Ribbons : 31
Re:Cooling 6 GTX580? 2012/02/28 06:50:45 (permalink)
MySchizoBuddy

How do you liquid cool 6 or 7 GTX580 Hydro copper?
Can you even fit 7 of them on the Classified SR2 and connect them via 4way and 3way barbs.
Do you still use water or a different liquid?
What should be the flow rate?
What should be the radiator size and at what fan speed?

Is there a set of formulas i can use to calculate these parameters. 

PS: this setup is for CUDA/OpenCL, not gaming.


Hello, yes it is just about possible in terms of physical space; take a look at my mods rigs, but you need to have true single slot waterblocks, for linking them up just use bitspower sli fittings (the small ones) with the connectors that are to be used with them.
 
But as mentioned above, serious issues in booting up!
post edited by Sleinous - 2012/02/28 06:55:02

My Affiliate Code: GL1IAAFWJF
#9
voklskier4452
CLASSIFIED Member
  • Total Posts : 4001
  • Reward points : 0
  • Joined: 2008/08/27 15:34:29
  • Location: VT
  • Status: offline
  • Ribbons : 8
Re:Cooling 6 GTX580? 2012/02/28 07:05:23 (permalink)
You could look into working with the P6t7 Supercomputer like the fastra II team did.
http://en.wikipedia.org/wiki/Fastra_II
 
They got a special bios from Asus last I checked so that they could run that many GPUs.

Asrock Z77E-ITX | Intel i5 3570K @ 4.5ghz | 8gb Gskill Ripjaws X 1600mhz | Corsair H80i
Asus GTX 670 DCU II | Sandisk Extreme 120gb | Silverstone ST45SF-G | Silverstone SG-05
#10
MySchizoBuddy
New Member
  • Total Posts : 21
  • Reward points : 0
  • Joined: 2012/02/27 15:51:40
  • Status: offline
  • Ribbons : 0
Re:Cooling 6 GTX580? 2012/02/28 07:08:36 (permalink)
what are the chances that EVGA has a special 64bit BIOS.Hopefully the new SRX with LGA 2011 will allow something special.
post edited by MySchizoBuddy - 2012/02/28 07:19:01
#11
tpb7463
ACX Member
  • Total Posts : 437
  • Reward points : 0
  • Joined: 2010/08/21 16:51:59
  • Location: Utica, NY
  • Status: offline
  • Ribbons : 9
Re:Cooling 6 GTX580? 2012/02/28 07:55:18 (permalink) ☄ Helpful
Thanks for confiming, Matt, I'm sure the OP appreciates it.
 
Well, my suggestion comes from a little of both.
The MCP655, is simply a very good, standard pump.
 
I'm cooling a pair of GTX590's and they really needed at least 4x120 worth of radiator space to be cooled properly. This is why I said a minimum--because obviously a quartet of 580's will output more heat.
 
I'm not an expert in water cooling, but you can (roughly) compute how large of a radiator by doing something like this:
1. Calculate the TDP of all the components you want to cool.
2. Multiply that number by 0.9 (electronics are roughly 10% efficient) this number is what you're aiming to dissipate in watts.
3. http://skinneelabs.com/2011-2012-radiator-comparison/3/ has a very nice comparison of Fan RPM to Watts dissipated at set flow rate. Use that chart in determining exactly how much "rad" you need.
 
I'll do some of the legwork myself:
The TDP of a single GTX 580 is roughly 250W. 
You have four, but when accounting for efficiency, you get 900W.
Looking at that chart I linked, each 3x120 rad is capable of dissipating roughly 450W @ (1500RPM/1.6GPM)
As you can see, my earlier suggestion of 2x(3x120) is pretty close.
 
But, the moment you mentioned nodes in a cluster, I realize you are not the typical SR-2 customer...you sound like you have experience in complex server setups. Not to mention, this isn't going to be a workstation where someone will have to listen to the fans blowing full bore. So if we take that into account, then according to the same chart, you could potentially get away with a single 3x120 rad for all 4 GPU's, but you would need 4000RPM fans.
 
I hope that takes the guesswork out of things. 
 
EDIT:
I seemed to have neglected the concept of delta. Well, I could make my post even more absurdly long, but it's easier just to provide the linkage: 
http://www.overclockers.com/guide-deltat-water-cooling/
post edited by tpb7463 - 2012/02/28 08:12:29

SR-2 Buildlog: http://www.evga.com/forums/tm.aspx?m=1018539
EVGA SR-2 | 2 x 5650's @ 4 GHz | 2 x EVGA GTX590's | GTS 450
#12
MySchizoBuddy
New Member
  • Total Posts : 21
  • Reward points : 0
  • Joined: 2012/02/27 15:51:40
  • Status: offline
  • Ribbons : 0
Re:Cooling 6 GTX580? 2012/02/28 12:01:55 (permalink)
thanks that answered my question. One last one. can you even place them at single slot spacing. Will the fittings allow them or you still stuck with using single slot cards at dual slot spacing.
#13
Tt-Shannon
Thermaltake Support
  • Total Posts : 57
  • Reward points : 0
  • Joined: 2012/02/28 08:50:58
  • Status: offline
  • Ribbons : 1
Re:Cooling 6 GTX580? 2012/02/28 12:36:12 (permalink)
I have seen some fittings that were threaded on both sides to join 2 blocks together very closely,   where I found them escapes me at the moment but if I find them again I will post up.
 
basically they would have to be threaded in and all of the cards would have to be inserted as a single unit into the slots all at once,   and also need space to insert a small pick or screwdriver to hit latches on PCIe slots if they need to be removed.
 
the Lack of orom space will leave performance problematic with too many gpus and will require at the least a special BIOS setup specifically for that purpose.  I know it has been done on the Asus boards but not sure about others.
 
EDIT:  heres one I found with quick search,  can be used but would require some hand yoga to get the gpus tightened together....   and possibly spinning the card a lil.
post edited by Tt-Enthusiasts - 2012/02/28 12:38:14

Thermaltake Enthusiasts Support
Thermaltake Support Email: Support@thermaltakeusa.com
Support Phone Number: (800) 988-1088 *option 1
Asus Rampage V Extreme
i7 5960X @ 4700 MHz 24/7  
Custom Tt LCS Liquid cooling
16TB SSD array
48TB Mechanical SAS12 Array
64GB Crucial Ballistix Elite DDR4 
3-way TITAN X
 
#14
MySchizoBuddy
New Member
  • Total Posts : 21
  • Reward points : 0
  • Joined: 2012/02/27 15:51:40
  • Status: offline
  • Ribbons : 0
Re:Cooling 6 GTX580? 2012/02/28 13:03:24 (permalink)
you forgot the link. Can you tell me what was your search term
#15
Tt-Shannon
Thermaltake Support
  • Total Posts : 57
  • Reward points : 0
  • Joined: 2012/02/28 08:50:58
  • Status: offline
  • Ribbons : 1
Re:Cooling 6 GTX580? 2012/02/28 13:13:44 (permalink)
I guess im too new and cannot send links :(
 
google this and you will get idea..
 
ADT-XMM-BK

Thermaltake Enthusiasts Support
Thermaltake Support Email: Support@thermaltakeusa.com
Support Phone Number: (800) 988-1088 *option 1
Asus Rampage V Extreme
i7 5960X @ 4700 MHz 24/7  
Custom Tt LCS Liquid cooling
16TB SSD array
48TB Mechanical SAS12 Array
64GB Crucial Ballistix Elite DDR4 
3-way TITAN X
 
#16
Halo_003
Omnipotent Enthusiast
  • Total Posts : 12780
  • Reward points : 0
  • Joined: 2009/03/20 18:18:10
  • Status: offline
  • Ribbons : 50
Re:Cooling 6 GTX580? 2012/02/28 13:31:41 (permalink)
Why not get some 590s? Would be cheaper and cooler, not to mention more bang for buck/slot.

Desktop: i7 3820 - ASUS X79 Sabertooth TUF - 4x4GB G.Skill TridentX 2400C9 - EVGA GTX 580 Classified ULTRA Hydro Copper
Microsoft Surface Pro 4: i7 6650U - 16GB RAM - 256GB SSD - Sennheiser HD 598 - DualShock 4 Controller
#17
Tt-Shannon
Thermaltake Support
  • Total Posts : 57
  • Reward points : 0
  • Joined: 2012/02/28 08:50:58
  • Status: offline
  • Ribbons : 1
Re:Cooling 6 GTX580? 2012/02/28 13:37:38 (permalink)
Halo is correct.   then you could run 3x 590's and a single 580 all with plenty of spacing.
 
the board will post with a max of 7 GPU's so you should be safe there too,   however due to allocation you still could see some slowdown issues as well.

Thermaltake Enthusiasts Support
Thermaltake Support Email: Support@thermaltakeusa.com
Support Phone Number: (800) 988-1088 *option 1
Asus Rampage V Extreme
i7 5960X @ 4700 MHz 24/7  
Custom Tt LCS Liquid cooling
16TB SSD array
48TB Mechanical SAS12 Array
64GB Crucial Ballistix Elite DDR4 
3-way TITAN X
 
#18
EliteGeek91
Omnipotent Enthusiast
  • Total Posts : 10669
  • Reward points : 0
  • Joined: 2011/02/10 09:40:19
  • Location: Censored!
  • Status: offline
  • Ribbons : 44
Re:Cooling 6 GTX580? 2012/02/28 13:39:45 (permalink)
I'd get a X79, and a 3930x, and atleast OC it to 5.0GHz to handle all those cards...

#19
willdearborn
CLASSIFIED Member
  • Total Posts : 2228
  • Reward points : 0
  • Joined: 2008/01/04 18:54:42
  • Status: offline
  • Ribbons : 4
Re:Cooling 6 GTX590? 2012/02/28 13:59:46 (permalink)
tpb7463

Distilled water with a kill coil if you don't run nickel-plated blocks.   Oh right, hydrocoppers.
Distilled water with biocide if you have nickel in your system.

 
You can use a silver killcoil in a loop that has nickel plated blocks. Biocide will work as well, but you certainly can mix nickel with a killcoil. I think this misconception comes from EK's earlier model nickel plated blocks (non-EN). SOME people's blocks had nickel flaking off into the loop when a killcoil was present, but this was due to an inferior nickel plating the EK used. In fact it did not happen to all of these blocks, as I had a nickel plated (non-EN) Supreme HF in a loop with only a killcoil for about a year and had zero flaking or corrosion. But the newer EK nickel blocks (EN), as well as blocks from other manufacturers will be fine with only a killcoil present in the loop.

2x EVGA GTX 1080 SLi • 43" 4K Wasabi Mango UHD430 • 3x 30" HP LP3065s
ASUS Rampage IV Extreme • Intel Core i7 4820k • 16GB Corsair Vengeance Pro
Lian Li PC D600WB • Corsair AX1200 • 2x Samsung 850 EVO 500GB SSDs in RAID 0
#20
MySchizoBuddy
New Member
  • Total Posts : 21
  • Reward points : 0
  • Joined: 2012/02/27 15:51:40
  • Status: offline
  • Ribbons : 0
Re:Cooling 6 GTX590? 2012/02/29 12:58:01 (permalink)
yes 4 GTX590 will give me 8GPUs total, if this board or the newer SR-X board can handle four GTX590s. 
 
#21
hoserx
CLASSIFIED Member
  • Total Posts : 2869
  • Reward points : 0
  • Joined: 2007/02/15 21:00:31
  • Location: Ohio
  • Status: offline
  • Ribbons : 4
Re:Cooling 6 GTX590? 2012/02/29 14:03:50 (permalink)
Have you considered some of the Tesla products if youre just doing cuda stuff?

Asus Rampage IV Formula x79/ Core i7 3820 /16gb G Skill ddr3-2133mhz  / 2x EVGA GTX 680 /   Intel X-25m 80gb G2 + 120gb G2  / Corsair Force GT 120gb   /WD Caviar black 1TB  /WD Caviar green 500gb /Sound Blaster X-fi titanium fatal1ty / Corsair AX1200 /NZXT Switch 810 (white)/Corsair H100/ HP ZR30W 
 
#22
MySchizoBuddy
New Member
  • Total Posts : 21
  • Reward points : 0
  • Joined: 2012/02/27 15:51:40
  • Status: offline
  • Ribbons : 0
Re:Cooling 6 GTX590? 2012/02/29 15:06:57 (permalink)
tesla is outrageously expensive and doesn't come in dual GPU conf
post edited by MySchizoBuddy - 2012/02/29 16:34:08
#23
Johnny_Utah
CLASSIFIED Member
  • Total Posts : 4330
  • Reward points : 0
  • Joined: 2008/02/13 16:26:04
  • Status: offline
  • Ribbons : 5
Re:Cooling 6 GTX590? 2012/02/29 15:21:46 (permalink) ☄ Helpful
As for the cooling, I like the MCP655, one of the best out there, if not THE best pump.  For connecting the cards on your loop, you have several options.  You could just cut a piece of tubing and run that between cards, or go with this type of fitting (has worked great for me):
 

 
For radiators, I tend to go for MORE than I probably need, but my temps stay VERY low, even when folding.  With 3 x 590's (if on same loop as CPU) I would go with at least 2 x 120.3 rads. 
post edited by Johnny_Utah - 2012/02/29 15:24:58

5930k (H20)  //  ASUS Rampage V Extreme
EVGA Titan SC x 4 (H20)
Gskill Ripjaws 2400
 EVGA 1600G2 //  DD Torture Rack
 
 
#24
Johnny_Utah
CLASSIFIED Member
  • Total Posts : 4330
  • Reward points : 0
  • Joined: 2008/02/13 16:26:04
  • Status: offline
  • Ribbons : 5
Re:Cooling 6 GTX590? 2012/02/29 16:27:42 (permalink)
I just took this one of my 470's.  I just use tubing and 2 x compression fittings:


5930k (H20)  //  ASUS Rampage V Extreme
EVGA Titan SC x 4 (H20)
Gskill Ripjaws 2400
 EVGA 1600G2 //  DD Torture Rack
 
 
#25
Bruno747
CLASSIFIED Member
  • Total Posts : 3127
  • Reward points : 0
  • Joined: 2010/01/13 11:00:12
  • Location: Looking on google to see what Nvidia is going to o
  • Status: offline
  • Ribbons : 3
Re:Cooling 6 GTX590? 2012/02/29 18:11:27 (permalink)
you will also need one heck of a plumbing system, but rather than get the hydrocoppers, get normal ones and put aquacomputer blocks on them then you can use their delrin block for tying 7 of them together with only 2 fittings.
#26
MySchizoBuddy
New Member
  • Total Posts : 21
  • Reward points : 0
  • Joined: 2012/02/27 15:51:40
  • Status: offline
  • Ribbons : 0
Re:Cooling 6 GTX590? 2012/02/29 18:14:10 (permalink)
you are talking about " twinConnect kit for GTX 580" that allows 7 cards to be connect together. It allows 4 GTX 590 as they are dual slot cards. I like this solution so far. 

post edited by MySchizoBuddy - 2012/02/29 19:10:57
#27
MySchizoBuddy
New Member
  • Total Posts : 21
  • Reward points : 0
  • Joined: 2012/02/27 15:51:40
  • Status: offline
  • Ribbons : 0
Re:Cooling 6 GTX590? 2012/02/29 18:46:14 (permalink)
comment removed by me
post edited by MySchizoBuddy - 2012/02/29 18:57:14
#28
rottenmutt
SSC Member
  • Total Posts : 569
  • Reward points : 0
  • Joined: 2005/10/29 21:33:19
  • Status: offline
  • Ribbons : 0
Re:Cooling 6 GTX590? 2012/02/29 19:37:27 (permalink)
tpb7463

If I recall correctly, even though the slot config would support up to 7, 5 is the limit. 6 is bootable, but the system runs slowly with that many installed. (I believe one of the EVGA techs mentioned this) Honestly, I'd stick to 4 and save yourself the headaches.

 
to much miss information on the internet...  the rig below has four 590 cards which is 8 GPUs
http://setiathome.berkele...ail.php?hostid=5890623

EVGA X58 Classified|i7-920 @ 4.2GHz on H20|Tri SLI GTX 680|CORSAIR 6GB DDR3 CM3X2G1600C8D|SB Xfi|ThoughPower 1200W 
EVGA X58 Classified|i7-975 @ 4.2GHz on H20|2x GTX 275|CORSAIR 6GB DDR3 CM3X2G1600C8D
EVGA X58 Rev 1.0|i7-930 @ 4.00GHz|GTX 580, GTX 470|6GB Kingston PC3-10600
Skulltrail|2x E5450 @ 3.6GHz on H2O|2X GTX 480|SB Xfi|ThoughPower 1200W
Skulltrail|2x E5430 @ 3.2GHz on H2O|GTX 580 3GB|SB Xfi|ThoughPower 1200W
H8DCE|2x Opteron 270|4GB|2X GTX 565ti
KFN32D-SAS|2x Opteron 2354|GTX 480, GTX 470


#29
MySchizoBuddy
New Member
  • Total Posts : 21
  • Reward points : 0
  • Joined: 2012/02/27 15:51:40
  • Status: offline
  • Ribbons : 0
Re:Cooling 6 GTX590? 2012/02/29 20:03:41 (permalink)
this information helps me other boards or future boards that will allow 7. Tyan has one that allows 8 full x16, bootable and stable. Nevertheless this is good information to have. 
#30
Jump to:
  • Back to Mobile