http://www.game-debate.com/news/?news=16302&game=None&title=Pascals%20NVLink%20Will%20Provide%20Support%20For%208-Way%20Multi-GPU%20Configurations
 
Let me know if anyone knows when mobos will have NVLink to support 8-way SLI.  The earliest implementation I can see is either Skylake-E or Cannonlake.  Any later and the cards will be out before a mobo that supports it.

Brad_Hawthorne
As was brought up looooong ago with gen-2 SLI, it gets to a point where SLI overhead probably would actually start having negative results for performance because of the associated computational overhead. I have my doubts about 8-way anything based on that unless it is truly a pipeline directly between all GPUs without any negotiation happening and it acts like a large single GPU. We all know current gen SLI does not behave transparently and negotiation is involved both within hardware and software drivers.

The farthest I have, and will ever go is 2-way.  I prefer having a single beastly card, such as my GTX970FTW.  It seems Pascal will have 3D stacked memory, NVIDIA is really pushing forward fast.  Also Brad, check out the paragraph right above the pick of the Pascal card next to a pen.  It talks about what you are suggesting as introducing a new version of SLI: http://www.digitalversus....ushed-back-n33779.html

lehpron
I interpreted Brad's comment to be more about the CPU, rather than the capabilities of the GPU.
 
With one graphics card, most games are set with 2-4 cores, and maybe a marginal overclock (covered by a boosting frequency).  Add a graphic card and it makes CPU overclocking more beneficial to a point.  I believe due to the limits of DX9-DX11 where games weren't coded openly for as many cores as you had available, it just so happened that getting more cores for SLI didn't scale well, suggesting 6-core on up was for prosumer and not gamer.  With DX12 however, it is possible that more GPUs will need more CPU cores to feed, that games (for instance) will be coded more openly.  Because of this, 4-way SLI could be a scaling limiter against future high-end CPU iterations by both AMD and Intel of 8-core and beyond if the scaling for multi-GPU is indeed better for going beyond a quad-core.  That said, the stalling of core count progress due to a lack of competition could have been the catch 22 for how the programs themselves were written; one affects the other, how to break the cycle and restore progress?

We have yet to see, waiting for DX12 and Pascal to bear their fruit, but this is my theory; it will render multi-GPU scaling beyond 2 cards on mainstream AMD/Intel platforms obsolete and forcing demand for higher-core count with greater GPU scaling.  Of course the price structure could change as well, single-GPU won't be dominant below $1000 in the future.

Though, in my mind, I figure the growing progression of IGP by AMD and Intel is hitting nVidia's low-end, these Titans are just an attempt to recoup lost revenue.  In the future nVidia would need extreme multi-GPU scaling as the revenue/profit restorer to the original price structure when IGPs were laughable.

It is possible that Skylake's flagship IGP with its 72-eu and the eDRAM cache running DDR4 could reach the equivalent of a single GTX750 Ti-- it is not even one-fifth the power of a Titan-X.  Ten years ago, Intel IGP was 40 times slower than the single-GPU flagship of the day (Radeon X1800 or GeForce 7800), that's not the case anymore.   There could be a point when future IGP actually matches the high-end dedicated card, so a company like nVidia would have to enable scaling beyond 4-way just to differentiate themselves.  Of course by then, because of IGP stigmata, no one will see the single-GPU flagship as high-end anymore.

I think the GPUs are categorized low end to high end.  SLI has not applied to the rating system so far.  You have 2-way or 3-way SLI of high end or middle end GPUs, not a 2-way SLI of these cards is rated as high end.  But maybe you are right and things will change to rating your system as high end by the number of GPUs.  
 
Things are still not adding up for me as far as Intel CPU releases and Pascal having NVLink which details make it seem that it will be a replacement for PCI-E used as a slot for NVIDIA cards.  But motherboards are putting out PCI-E 4.0 like NVLink isnt even coming.  They are hiding that detail very well, but releasing it on cannonlake is too last minute IMO. How am I supposed to do 8-way SLI when I have all these damn PCI-E 4.0 slots.  Here is an article: http://www.electronics-ee...&news_id=222920599 
 
I can only imagine the joy of spending 8 grand on 8x Titans w/ Pascal architecture and then to find out a year later they are only worth 4 or 5 grand.  Sell them and put up another 3 or4 grand to have 8 grand to buy 8x Titans w/ Volta architecture.  Pretty safe estimate in my opinion.  4 or 5 grand always down on the cards and 3 to 4 grand a year to stay on top with 8x Titans.  Go for it!
 
 

boylerya
Things are still not adding up for me as far as Intel CPU releases and Pascal having NVLink which details make it seem that it will be a replacement for PCI-E used as a slot for NVIDIA cards.  But motherboards are putting out PCI-E 4.0 like NVLink isnt even coming.  They are hiding that detail very well.

Just like they hid Shield, seemingly every time.  In my mind 'duh' since console anything isn't popular to the PC community, so some leaks aren't being sought and spread.  Those anonymous leakers only spread rumors that they want to talk about, they aren't going to objectively leak everything as if there was no bias.
 

Therefore, unless NVLink boards are Tegra-based to push even more proprietary, those boards are not coming with Pascal.  I think NVLink is just a name for the bandwidth they provide between the GPU and HBM, I don't think the comparison with PCIe Express was to require an entirely new format and standard that neither AMD/Intel have built in NVLink controllers (unless nVIdia makes a chipset like nF200 that acts like a bridge chipset).  

But I'm betting the first version of Pascal will still come in the standard PCIe format.

I read the article, but my issue is with the needed infrastructure to enable a brand new slot type and everything that goes along with that.  I think the form we first saw Pascal in was just prototype and not mass-production, it will go into any slot, MXM or PCIe 3/4.

But because NVLink is proprietary, it doesn't sound like a standard that AMD/Intel would have made controllers for soon enough for nVidia to debut Pascal within a year; especially when it steps all over the PCISIG standard.  It is almost as if we're looking at nVidia's Mantle, acting as competitive push to make the next PCIe 4.1 faster or bringing PCIe 5.0 earlier than originally planned, based on the demand than nVidia can invoke.  Quite possible with their market share.

Getting back to making Pascal practical in the short term, if the intent was to enable Pascal 8-way scaling on modern x86 systems, a chipset must be made to bridge between the NVLink controller and the primary PCIe controllers.  Except doing so, however, will be a bottleneck by nearly an order of magnitude and defeating NVLink's purpose.  I can accept that first generation cards from Pascal may not hit a bottleneck, but it would defeat the purpose of having NVLink enabled.  Might as well make those cards PCIe 3.0 and PCIe 4.0 and allow 5-way on X99 due to 40-lanes at x8 each.

My thinking at this point regarding the efficiency of the overhead bandwidth of NVLink and eliminating the need of an external NVLink bridge chipset, it would be better if nVidia made the whole platform from-the-ground-up, i.e. via Tegra SoC ARM superchip with built-in NVLink controllers.  This way they can make up the board and slots without worrying about the castration of PCI Express in both AMD/Intel platforms.  If nVidia can have complete control of the infrastructure, then it is easier for me to believe it (judging by the Shield console secrecy).  It would also make sense to embed HBM as system RAM for the processor (given the news about stacking dozens of layers on the cheap, and AMD is already following suit if their intent is to put HBM on their APUs after demonstrating proof-of-concept it on 390X this summer).  The platform essentially becomes a console with embedded CPU + RAM but with up to 8 slots for graphics cards; but we can install whatever OS and games or programs.  I wouldn't be surprised if nVidia marketed this thing as a GRID node for use with their Shield consoles, so the end-user isn't the only customer.

But how many of us would ditch x86 just to experience 8-way scaling?  DX12 is supposed to reduce CPU overhead, so it would hypothetically make ARM relevant for what once required x86 high-performance.  Especially if Windows 10 is being marketed as a cross-platform OS.

I'd prefer that NVLink will utilize PCIe standards for the slot, electrically and physically, just so there is compatibility with both Pascal ARM platforms and any x86 ATX motherboard, a simply swap to another platform and add cards.  This way no one has to buy different cards just for 8-way scaling versus 4-way x86.  That said, Intel makes customers buy entirely different processors and boards just to scale beyond one, i.e. Xeon -EP and -EX versus i7.  But those customers don't mind, their purchase pays for itself eventually.

Given the nature of professional and corporate uses of Quadro and Tesla cards, respectively, I wonder if Pascal 8-way scaling on proprietary boards will go to those markets first; meaning, GeForce versions remain PCIe/x86 and limit to traditional 4-way SLI.  Their focus with Deep Learning at GTC2015 has me thinking; talking about gaming is what we want to hear, but we all know these cards can do much more, thanks to CUDA.

Would AMD follow this madness with their own version, possibly without requiring a proprietary standard?  Sure, but I'll bet they'll discuss with PCISIG regarding the capabilities of the next standard iteration so both AMD and Intel end up with a version of NVLink.  Maybe have Intel's X290 platforms with 96-128 PCIe lanes controllers on die, I think Skylake-E is too soon.

But I've been wrong before.

lehpron
I read the article, but my issue is with the needed infrastructure to enable a brand new slot type and everything that goes along with that.  I think the form we first saw Pascal in was just prototype and not mass-production, it will go into any slot, MXM or PCIe 3/4.

But because NVLink is proprietary, it doesn't sound like a standard that AMD/Intel would have made controllers for soon enough for nVidia to debut Pascal within a year; especially when it steps all over the PCISIG standard.  It is almost as if we're looking at nVidia's Mantle, acting as competitive push to make the next PCIe 4.1 faster or bringing PCIe 5.0 earlier than originally planned, based on the demand than nVidia can invoke.  Quite possible with their market share.

Getting back to making Pascal practical in the short term, if the intent was to enable Pascal 8-way scaling on modern x86 systems, a chipset must be made to bridge between the NVLink controller and the primary PCIe controllers.  Except doing so, however, will be a bottleneck by nearly an order of magnitude and defeating NVLink's purpose.  I can accept that first generation cards from Pascal may not hit a bottleneck, but it would defeat the purpose of having NVLink enabled.  Might as well make those cards PCIe 3.0 and PCIe 4.0 and allow 5-way on X99 due to 40-lanes at x8 each.

My thinking at this point regarding the efficiency of the overhead bandwidth of NVLink and eliminating the need of an external NVLink bridge chipset, it would be better if nVidia made the whole platform from-the-ground-up, i.e. via Tegra SoC ARM superchip with built-in NVLink controllers.  This way they can make up the board and slots without worrying about the castration of PCI Express in both AMD/Intel platforms.  If nVidia can have complete control of the infrastructure, then it is easier for me to believe it (judging by the Shield console secrecy).  It would also make sense to embed HBM as system RAM for the processor (given the news about stacking dozens of layers on the cheap, and AMD is already following suit if their intent is to put HBM on their APUs after demonstrating proof-of-concept it on 390X this summer).  The platform essentially becomes a console with embedded CPU + RAM but with up to 8 slots for graphics cards; but we can install whatever OS and games or programs.  I wouldn't be surprised if nVidia marketed this thing as a GRID node for use with their Shield consoles, so the end-user isn't the only customer.

But how many of us would ditch x86 just to experience 8-way scaling?  DX12 is supposed to reduce CPU overhead, so it would hypothetically make ARM relevant for what once required x86 high-performance.  Especially if Windows 10 is being marketed as a cross-platform OS.

I'd prefer that NVLink will utilize PCIe standards for the slot, electrically and physically, just so there is compatibility with both Pascal ARM platforms and any x86 ATX motherboard, a simply swap to another platform and add cards.  This way no one has to buy different cards just for 8-way scaling versus 4-way x86.  That said, Intel makes customers buy entirely different processors and boards just to scale beyond one, i.e. Xeon -EP and -EX versus i7.  But those customers don't mind, their purchase pays for itself eventually.

Given the nature of professional and corporate uses of Quadro and Tesla cards, respectively, I wonder if Pascal 8-way scaling on proprietary boards will go to those markets first; meaning, GeForce versions remain PCIe/x86 and limit to traditional 4-way SLI.  Their focus with Deep Learning at GTC2015 has me thinking; talking about gaming is what we want to hear, but we all know these cards can do much more, thanks to CUDA.

Would AMD follow this madness with their own version, possibly without requiring a proprietary standard?  Sure, but I'll bet they'll discuss with PCISIG regarding the capabilities of the next standard iteration so both AMD and Intel end up with a version of NVLink.  Maybe have Intel's X290 platforms with 96-128 PCIe lanes controllers on die, I think Skylake-E is too soon.

But I've been wrong before.

If thats the case, they will definitely implement NVLink for only the high-end cards given the extra expense of putting a Tegra chip on the card.  So much for power efficiency on cards having to power two chips, or how about a Titan Z equivalent.  It would have 2 GPUs and 2 Tegra chips to power, or maybe a Titan Z equivalent will be out of the picture for awhile.  Cant wait to see what will happen.  All I care is that NVLink has compatibility with Skylake, since Broadwell is outa the picture for my next upgrade from an i7-860.  

boylerya
If thats the case, they will definitely implement NVLink for only the high-end cards given the extra expense of putting a Tegra chip on the card.

I meant a future variant of Tegra as the primary CPU embedded on its own motherboard with built-in NVLink controlelrs and NVLink compatible slots, I didn't mean having a Tegra on the graphics card PCB.  I meant that a fully-optimized NVLink system won't make sense on an x86 computer any time soon, and any attempt to do so too early is inefficient by the fact that NVLink is proposed to be 5-12 times faster than the PCIe controllers in the x86 CPU die.

Like I said, it would be easier for me to believe that nVidia invents their own platform in order to shift x86 dominance to ARM-ville over the course of a decade than seeing NVLink slots anytime soon on an AMD/Intel platform, with or without a bridge chipset.  The supposed secret they are holding so well is only possible if no other companies are involved, i.e. just like their Shield ecosystem, an in-house project that no one leaks because it won't follow PC gaming status quos.  Too many anonymous rumor generators and leakers are Intel fans, why report on something that makes you doubt your favorites?

But going by the nVidia link you gave:

And it will feature NVLink – NVIDIA’s high-speed interconnect, which links together two or more GPUs — that will lead to a total 10X improvement in deep learning.

NVLink is more like an SLI interface than one connecting to the CPU through the mainboard.  I don't get the impression of a brand new platform with NVLink slots.

boylerya
All I care is that NVLink has compatibility with Skylake, since Broadwell is out a the picture for my next upgrade from an i7-860.

It won't happen that soon, if it does it will be castrated, as mentioned above.  Simply put, AMD and Intel will not put NVLink controllers in their processors, they'll stick to PCIe.  Therefore, I suspect the first Pascal Titans will use PCI Express and be faster than a Maxwell Titan by the nature of how many CUDA cores they pack in a 14nm die.  Pascal cards won't need a new slot.

The simple truth of technology is someone's gotta do it, whether or not anyone wants it.