utterchaos
New Member
- Total Posts : 4
- Reward points : 0
- Joined: 2016/08/28 22:36:28
- Status: offline
- Ribbons : 0
So I have an odd question and up until now I've been attacking this problem from the MOBO side but maybe there's a way to attack it from the GPU side? I am currently running a 10 GPU setup with 9 980 ti's and 1 980. It's for CG rendering with Octane. It's running a dual PROC board with 2 xeon 2670s and I've got 8 of those 10 GPUs coming off of CPU1's PCIe lanes... Theoretically if PCIe lanes were the only resource I'd have to worry about I could get up to 16 GPUs like you can with the 700 series cards... For some reason though, and it's probably due to increased PCIe bandwidth on the 900 series, I run out of "PCIe resources" when I add an 11th card. SO my question is, can the BIOS on the 980 ti be hacked to use less IO memory space on the MOBO's BIOS? I know this'll stunt the card for gaming but when rendering there isn't much PCIe traffic so hopefully no effect!  
|
Cool GTX
EVGA Forum Moderator
- Total Posts : 31000
- Reward points : 0
- Joined: 2010/12/12 14:22:25
- Location: Folding for the Greater Good
- Status: online
- Ribbons : 122
Re: Hack the GPU BIOS to use less MOBO BIOS memory?
2016/08/29 11:25:55
(permalink)
Welcome to the forums utterchaos Above my pay grade Wicked project though  Why not evenly split the GPUs between CPUs ? Maybe Bond two servers of 5 GPUs? How many PSUs do you have connected ?
post edited by Cool GTX - 2016/08/29 11:28:17
|
bcavnaugh
The Crunchinator
- Total Posts : 38977
- Reward points : 0
- Joined: 2012/09/18 17:31:18
- Location: USA Affiliate E5L3CTGE12 Associate 9E88QK5L7811G3H
- Status: offline
- Ribbons : 282
Re: Hack the GPU BIOS to use less MOBO BIOS memory?
2016/08/29 11:38:04
(permalink)
Even more so Cool GTX what is the purpose of even doing this?
|
utterchaos
New Member
- Total Posts : 4
- Reward points : 0
- Joined: 2016/08/28 22:36:28
- Status: offline
- Ribbons : 0
Re: Hack the GPU BIOS to use less MOBO BIOS memory?
2016/08/29 11:50:13
(permalink)
Thanks for the welcome Cool GTX! The configuration of which CPU the GPUs are on doesn't effect the memory allocation issue I'm currently dealing with. As long as there is less than 10 GPUs on 1 CPU then it'll boot no problem. I have 2 1300W and 1 1600W. Its actually a bit overkill since most of the GPUs use about 75% of their full wattage under 100% load based on stats spit out by nvidia-smi.
There are many purposes of this bcavnaugh! First, traditional CPU rendering nodes are costly and scaling requires you to buy more full systems, RAM, MOBO, CPU, PSU, HDD, etc!! So with GPU rendering if I wanted to double my performacne I just get 2x more GPUs and I don't have to scale the rest of the machine! Not to mention GPU rendering is significantly faster than traditional CPU. Sure the current VRAM is a limitation but it'll get better soonish and with smart project management like instancing you can work around the limit pretty easily!
|
bcavnaugh
The Crunchinator
- Total Posts : 38977
- Reward points : 0
- Joined: 2012/09/18 17:31:18
- Location: USA Affiliate E5L3CTGE12 Associate 9E88QK5L7811G3H
- Status: offline
- Ribbons : 282
Re: Hack the GPU BIOS to use less MOBO BIOS memory?
2016/08/29 12:36:52
(permalink)
utterchaos
Thanks for the welcome Cool GTX! The configuration of which CPU the GPUs are on doesn't effect the memory allocation issue I'm currently dealing with. As long as there is less than 10 GPUs on 1 CPU then it'll boot no problem. I have 2 1300W and 1 1600W. Its actually a bit overkill since most of the GPUs use about 75% of their full wattage under 100% load based on stats spit out by nvidia-smi.
There are many purposes of this bcavnaugh! First, traditional CPU rendering nodes are costly and scaling requires you to buy more full systems, RAM, MOBO, CPU, PSU, HDD, etc!! So with GPU rendering if I wanted to double my performacne I just get 2x more GPUs and I don't have to scale the rest of the machine! Not to mention GPU rendering is significantly faster than traditional CPU. Sure the current VRAM is a limitation but it'll get better soonish and with smart project management like instancing you can work around the limit pretty easily!
Yes we know, but for what? BOINC, Folding, making Movies, what are you Rendering?
|
utterchaos
New Member
- Total Posts : 4
- Reward points : 0
- Joined: 2016/08/28 22:36:28
- Status: offline
- Ribbons : 0
Re: Hack the GPU BIOS to use less MOBO BIOS memory?
2016/08/29 12:44:27
(permalink)
Well octane is a path tracer so anything cg really: movies, commericals, just fun art, etc. Check out my behance for some examples: vfxPatton on behance. Guess links aren't allowed for me, sorry!
post edited by utterchaos - 2016/08/29 12:47:45
|
Leosmart116
New Member
- Total Posts : 1
- Reward points : 0
- Joined: 2016/08/28 20:51:58
- Status: offline
- Ribbons : 0
Re: Hack the GPU BIOS to use less MOBO BIOS memory?
2016/09/12 07:32:53
(permalink)
|
bcavnaugh
The Crunchinator
- Total Posts : 38977
- Reward points : 0
- Joined: 2012/09/18 17:31:18
- Location: USA Affiliate E5L3CTGE12 Associate 9E88QK5L7811G3H
- Status: offline
- Ribbons : 282
Re: Hack the GPU BIOS to use less MOBO BIOS memory?
2016/09/12 08:21:57
(permalink)
Leosmart116
h
Like because it is a Mountain and you have to climb it? Or just having fun and playing around with what you can do for the fun of it.
|
cdc-951
SSC Member
- Total Posts : 520
- Reward points : 0
- Joined: 2012/04/27 02:26:30
- Status: offline
- Ribbons : 1
Re: Hack the GPU BIOS to use less MOBO BIOS memory?
2016/09/12 12:44:57
(permalink)
Wow... sorry I can't help with that, but just wanted to complement your system. I was like holy $#1! Someone here might be able to help, but if not then I would try overclockers.net
|
utterchaos
New Member
- Total Posts : 4
- Reward points : 0
- Joined: 2016/08/28 22:36:28
- Status: offline
- Ribbons : 0
Re: Hack the GPU BIOS to use less MOBO BIOS memory?
2016/09/12 12:56:15
(permalink)
Thanks man!!! Ill try on overclockers!!!
|
ebirck
New Member
- Total Posts : 1
- Reward points : 0
- Joined: 2017/05/09 10:20:46
- Status: offline
- Ribbons : 0
Re: Hack the GPU BIOS to use less MOBO BIOS memory?
2017/05/09 10:28:30
(permalink)
Hello,
@utterchaos did you ever solve this problem? I am attempting a very similar setup and will likely run into the same issue.
Thanks,
-Eric
|