sparetimepc
So are those cards all linked and running sli with the Nvidia driver mod and without an nvlink adapter?
Love the RTX line, one thing that surprised me however is that they no longer offer peer-to-peer through pcie - unlike several generations of previous GeForce cards which DO offer a pcie p2p bus link. see: Unified Memory Architecture.
That’s a bit of a big deal for compute-centric uses. Even if you didn’t have the speed of an infiniband or NVLink - on previous gen cards you still have the ability to use a single merged memory space for a large DL model at slightly slower speeds. Hence, on Linux (not Windows) every previous gen GTX card could access each others page space in pcie - without going through the cpu memcopy - which is very unstable along with slow.
The RTX cards offer, at most, 2 cards in UMA through NVLink... but no option whatsoever for pcie memory sharing for additional cards. So other cards, outside of the 2 connected by NVLink, are cut off from a single merged UMA altogether.
pcie used to be a rather nice fall-back when NVLink wasn’t there for above stated reasons.
NVLink >> pcie >>>>>> memcopy through cpu