The release of the R256 series of GeForce drivers, beginning with GeForce 257.15, brings with it an unprecedented degree of configurability in both GPU and PhysX support, taking things right down to the user level. Ever the naturally curious one, I wanted to put my rig through its paces with the recently released PhysX FluidMark, having purposely designed the system around the concept of configurability and the ability to adapt to both environments and applications to-hand. Outfitted with four GPUs, one of which fulfills a dedicated PhysX PPU role, I was able to reproduce eleven differing configurations from single, stand-alone GTX 260 operation all the way to 3-Way GTX 260 SLI w/GT 240 PhysX support
 

 
Taken as individual results grouped by configuration, it would appear that there's only about two types of results to be expected: decent and very good. The final scoring results, however, don't seem to differentiate much of a seperation between configurations with dedicated support and self-supported roles. It would appear that this breakdown, because of a GPU's near-constant graphics ability within this test, merely highlights the strength of the PhysX engine itself and not the ability to support the graphics engines in their rendering roles as to timing and coordination requirements when PhysX is enabled and used.
 
To be fair, between the new driver and the new release of the testing application, I did not investigate the app's inability to set various levels of graphics intensity even though the (apparently non-functional) controls to accomplish this are provided. I was, quite frankly, more intrested in seeing engine performance numbers anyway rather than trying to troubleshoot the controls themselves on a 32-bit application running a new driver structure in a 64-bit environment. I simply made the assumption that without added graphics loads on the cards I could use the results in a manner similar to the "Crash and Burn" PhysX test found within FutureMark's 3DVantage benchmark.
To that end, I decided to "stack" all of the results by configuration in order to possibly highlight and identify trends within groups of results:
 

 
Indeed, after doing this it became clearer that there are actually three levels of support demonstrated - GTX 260-based results, GT 240-based results, and anything using the Number 1 engine in any multi-GPU configuration (which also seems to include a single GTX 260 working entirely by itself). And since there was a clear seperation of capabilities by engine rather than by configuration without a load placed on the renderers, I decided that this would be an excellent opportunity to test the constantly-debated issue of dedicated PhysX PPU support. And, by taking the test to currently-enabled PhysX applications, I could also get a better idea of the levels of support provided and their effects on output when the rendering cards are fully loaded.
 
The testing rig:
 
ASUS P7P55 Workstation Supercomputer Motherboard

Intel P55 Chipset
nVidia NF200 for PCIe augmentation
Seven (7) available PCI Express physical x16 slots

x8-x8-x8-x8 PCIe ver 2 electrical configuration running 3-Way w/dedicated PPU

Intel Core i7 X3460 Xeon Lynnfield Socket LGA1156

Default 133.3 MHz FSB overclocked to 146.4 MHz for 10% frequency increase
Available 21x-26x CPU multipliers - 25x maximum dual-core, 26x maximum single-core
All CPU features enabled save ECC memory support and Spread Spectrums

4 GB Mushkin Dual-Channel DDR3-1600 SDRAM

DRAM:FSB Ratio 5:1 for 1464 MHz support
JEDEC SPD-derived 9-10-10-28 CR1 Timings (AUTO)

3x GeForce GTX 260

192-Core G200 GPU w/896 MB 448-bit GDDR3
eVGA FTW models released at 666 MHz Core/1404 MHz Shaders/1007 Mhz VRAM

BIOS reset to 648 MHz Core/1404 MHz Shaders/1188 MHz VRAM
Voltages reset in BIOS for 1.0000V Idle, 1.0375V 2D, 1.0500V 3D, 1.1500V Boost
Fans reset in BIOS for 65%, AUTO ramp 68C to 84C at 100%

GeForce GT 240

96-Core GT215 GPU w/1GB 128-bit GDDR5
PNY XLR8 model released at 550 MHz Core/1340 MHz Shaders/1700 MHz VRAM

Microsoft WIndows 7 Professional x64

Superfetch, Search and Index Services disabled

2x Western Digital VelociRaptor HDDs running RAID0 on SATA ports 4+5 (logical), 5+6 (physical)
 
Samsung SyncMaster P2350 23" Widescreen 1920*1080p LCD Monitor
 
Logitech X-530 5.1 Surround Sound Speakers
 
Coolermaster UCP 1100 Power Supply
 
 
Applications used during testing:
 

PhysX FluidMark

 Release version 1.2.0
 Initial PhysX configuration tests and data generation

Batman: Arkham Asylum

 PhysX Patch applied for version 1.1
 In-Game Benchmark used for data generation

Cryostasis: Sleep of Reason

 Tech Demo used for data generation

Darkest of Days

 Patched to ver 1.05, PhysX included in patch
 In-Game PhysX Benchmark used for data generation

Dark Void

 PhysX Demo version used for data generation
 

FRAPS

 In-Game Screen Captures and Frame Times data generation

Microsoft Image Composer 1.5

 Image file conversions to .png and resizing for posting

Microsoft Excel 2000

 Data collection, analysis and graph generation
 

nVidia Control Panel

 Used to configure rendering and PhysX support as required
 

Windows 7 Device Manager

 Third GTX 260 disabled when not used for data generation
 

The Tests:
 
Batman: Arkham Asylum
 

The decision to purchase the GT 240 for dedicated PhysX support was heavily influenced by the conclusions drawn by ChrisRay in his excellent examination of PhysX support in Batman: Arkham Asylum. One of the conclusions drawn from it was the size of the PhysX engine, as it seems an engine with half the Shader count of the type card it's supporting is sufficient unto that goal. Since my GTXs have 192 Shaders, the GT 240 fits that advice. Making it even more appealing is its compliment of a full gigabyte of GDDR5 VRAM - even more capable than the GTX 260s themselves and certainly more than enough "head room" for calculation-intensive tasks. The "clincher" for this particular card is its very low power draw, making a single-slot solution not only feasible by more than entirely possible.
 
Since the R256 GeForce and its in-built configurability weren't available when ChrisRay made his initial determinations, I decided to start there to gauge the effects on those conclusions. I made two runs this way, both without Anti-Aliasing enabled and with the full 16xQ available in-game (that is, after all, my primary reason for building a 3-Way SLI rig).
 

 
Without AA enabled, the results break down similarly to the FluidMark results, except that a couple of the multi-engine configurations are beginning to show a slight lag. The GTX SLI w/dedicated GTX 260 rigging is the winner here, with the two higher multi-card SLIs w/GT 240 support close on its heels. The advantage in the SLI 260 w/260 support comes from the stronger engine in the GTX 260, and the similarity of the two GT 240 SLI results would indicate that in an unfettered environment the GT 240 itself is the bottleneck. The slight lag experienced by the straight 3-Way GTX 260 rigging is most likely explained by the timing and coordination requirements of self-support.
 

 
Once you start placing more and more of a load on the renderers, the difference begins to show. In this case, all but the two straight, dedicated GTX 260 configurations fall far behind the riggings with a dedicated GT 240 PPU. Looking closer, though, tells you that the only real difference lies in the maximum frame rates - the minimum and average frame rates clearly show that the GT 240 can keep pace with its GTX 260 counterpart when the GTX 260 is in a self-supporting role. From there, it would seem that adding more of a load, through Transparency AA and a higher level of Texture quality - or even higher levels of AA available through apps such as nHancer - would highlight this advantage even further.
 
But even with this workload there's a hidden advantage to having dedicated support. On the next level of testing for this game, I decided to use FRAPS to time the frame generation across the PhysX demo. Unfortunately, I could not graph the entire run without a great deal of work converting the data into a format similar to the SVG results used in Far Cry 2's benchmarks, and my copy of Excel 200 is limited to a maximum of 254 columns. Instead, I ran each and simply grabbed representative 250-frame samples from the same point within each run of the three highest scorers in the 16xQ AA PhysX benchmark from within Batman:
 

 

 

 

 
 
 
Of the next three games tested, Dark Void comes closest to duplicating the results of the FluidMark Tests. The reason is not hard to determine when you look at the graphics involved.

 
Dark Void
 
 

 

 
 
 

 
Cryostasis: Sleep of Reason
 
This game is a bit more graphically intensive, and displays results similar to those of Batman. The advantage to the GT 240 though comes from a greater capability to render the graphics, to include features like Ambient Occlusion.
 

 

 
 
 
 

 
Darkest of Days
 
Here we come to the most intensive use of PhysX I've found to date. Once the battlefield starts filling with running people and horses, explosions, smoke, artillery fire, debris and even blowing leaves, there's a considerably larger number of items to be rendered and tracked in the scene generation.
 
There can be no doubt in this game as to the efficacy of a dedicated PhysX PPU as well as a multi-GPU environment to produce the necessary graphics.
 
And it would do for most reviewers on the WEB to consider this game alongside Crysis for graphics when it comes to generating a load on your graphics+PhysX...
 

 

 

Nice guide! BR worthy

_Nite_ Nice guide! BR worthy

And I do Thank You for the recommendation
 
As well as its results!

Wow this makes me want to cry. Correct me if I am wrong but does this mean if I took out my 9800gt PPU and just used two 260's for graphical rendering and the middle one for Physx I would get better performance?!

bg8780

Wow this makes me want to cry. Correct me if I am wrong but does this mean if I took out my 9800gt PPU and just used two 260's for graphical rendering and the middle one for Physx I would get better performance?!

If you had two GTX 480's in games that need both PhysX and graphical power, you would have gotten better performance. In your case, no. Taking out your 9800 GT will leave you with dual GTX 260s. It will usually result in lower FPS.

jaafa, the reason why Batman sucks on multi-GTX 260 is that it needs a lot of GPU power and turning on PhysX break SLI. It's not because of the "game code bla bla ". The charts say it all. You were getting choppy frames on each setup. You had to turn down details. Your cards would be sufficient for indoor parts of the game, but it'd lag on the outdoors of asylum at those settings.

Did you look at SLI bar on dual GTX 260 SLI when you enabled PhysX on it in Batman? It will stop scaling and the green part will get stuck. That means SLI is gone.

It does, try the config I told you by looking at SLI bar again and the performance. Then enable PhysX on GT 240. In fact, it shows that in the benches that you averaged 50 FPS.

Oh and both cards are loaded because one card is rendering the scene, where the other one is doing PhysX.

Yup, I got it. That's why I said "will leave you with dual GTX 260" to render the scene. GTX 260 Tri-SLI + 9800 GT PhysX > GTX 260 Two-way SLI + GTX 260 Physx. A lot of people say PhysX doesn't even make full use of a 9600 GT. I believe a faster card can make it a little bit better, as Eidos recommends a 9800 GTX+ for Physx. Your 9800 GT is on sweet spot and it's the perfect card for PhysX. By taking it out, you sacrifice one of your GTX 260s. That's why you will get lower performance.

yourmom It does, try the config I told you by looking at SLI bar again and the performance. Then enable PhysX on GT 240. In fact, it shows that in the benches that you averaged 50 FPS.

Oh and both cards are loaded because one card is rendering the scene, where the other one is doing PhysX.

Ah, no.
 
What you're seeing is a different phenomenon. SLI isn't being "broken" — it's the effects of trying to coordinate PhysX with rendering on Alternate Frame Rendering when multiple cards are teamed. It's just difficult to discern because "GPU1", "GPU2", and "GPU3" don't necessarily correspond to what you perceive as their physical order on your MOBO. And that's why the results differ so much once PhysX is taken off the renderers altogether.
 
Look at the results for GPU1 PhysX in any configuration as compared to the others, with or without AA enabled (but more visible with). The consistently low score there is a result of being the lead card in AFR - taking on the lion's share of coordination - while handling the PhysX timing as well. Take PhysX off the Primary and it just coordinates the team's timing while the Secondary shifts in and out of its compute config, and the timing load is slightly more distributed. Shift it to a third teamed card and the lack of scaling gives that Tertiary card a hair bit more "breathng room" - but not much, granted, even when the frame rate is CPU limited with no AA. And you'll see this pattern shift if you change your 3D Settings to either GPU2 AFR or GPU3 AFR.
 
Were SLI actually "broken" so that one card from the team could be dedicated solely to PhysX, then all configurations with the same number of cards would be consistent across all the results - min, avg and max. There certainly wouldn't be a discernible difference in just delineating any one particular card as seen in the charts.
 

...That's why I said "will leave you with dual GTX 260" to render the scene. GTX 260 Tri-SLI + 9800 GT PhysX > GTX 260 Two-way SLI + GTX 260 Physx...

That might be the case with the 112 shaders present in the 9800 GT, but certainly wasn't with the GT 240's 96. With these charts I was judging the advice of having half the shader count more than contrasting levels of shader count support.
 
 
 
Would've returned a response sooner, but I went to the Riverbend Festival for a while...

yourmom It does, try the config I told you by looking at SLI bar again and the performance. Then enable PhysX on GT 240. In fact, it shows that in the benches that you averaged 50 FPS.

Oh and both cards are loaded because one card is rendering the scene, where the other one is doing PhysX.

jaafaman Ah, no.
 
What you're seeing is a different phenomenon. SLI isn't being "broken" — it's the effects of trying to coordinate PhysX with rendering on Alternate Frame Rendering when multiple cards are teamed. It's just difficult to discern because "GPU1", "GPU2", and "GPU3" don't necessarily correspond to what you perceive as their physical order on your MOBO. And that's why the results differ so much once PhysX is taken off the renderers altogether.
 
Were SLI actually "broken" so that one card from the team could be dedicated solely to PhysX, then all configurations with the same number of cards would be consistent across all the results - min, avg and max. There certainly wouldn't be a discernible difference in just delineating any one particular card as seen in the charts.

SLI is completely broken in my selection, 2X GTX 260 SLI. On 3 GPUs, 1 goes to PhysX and others keep processing graphics. You don't lose it %100 but it goes down from Tri-way to 2 GPUs.

Look at the results of GTX 260 SLI at Batman. See how every result is nearly the same? Whether SLI is enabled or not? They all top about 70 and average to 50 FPS. What do you understand from that? lol.
 

...That's why I said "will leave you with dual GTX 260" to render the scene. GTX 260 Tri-SLI + 9800 GT PhysX > GTX 260 Two-way SLI + GTX 260 Physx...

jaafamanThat might be the case with the 112 shaders present in the 9800 GT, but certainly wasn't with the GT 240's 96. With these charts I was judging the advice of having half the shader count more than contrasting levels of shader count support.
 
Would've returned a response sooner, but I went to the Riverbend Festival for a while...

No, it won't make a huge difference. GT 240 is mostly enough for PhysX. Watch GPU usage on it while you're using medium level PhysX. It's going to be %50-85. Ageia's PPU only had the potential of that card. And it was what they officially released with shaders to do PhysX.

That's no problem, have your life rather then sitting in front of a monitor all day.

yourmom SLI is completely broken in my selection, 2X GTX 260 SLI. On 3 GPUs, 1 goes to PhysX and others keep processing graphics. You don't lose it %100 but it goes down from Tri-way to 2 GPUs.

Look at the results of GTX 260 SLI at Batman. See how every result is nearly the same? Whether SLI is enabled or not? They all top about 70 and average to 50 FPS. What do you understand from that? lol.

Ah, but balance that against the 3-Way configs with either GPU1 or GPU3 chosen as the engine. The other two don't seem to provide the straight rendering either - quite anomolous to the use of GPU2. Does that imply a total break-down of SLI in particular configs and not others?
 
Which is why I suggested comparison against the non-AA results, wheresome 3-Way configurations are even worse than the two-card/single results.
 
It could also be a case similar to the initial release of the ATHENA, where quite a few reviewers dogged the PPU as slowing down a system by attributing the drag to the PPU rather than the more correct conclusion that the systems were simply being overwhelmed by the number of particles and objects the CPU had to then track and the GPUs render and process.
 
In all honesty, I wouldn't put too much stock in these initial results being dogmatic one way or the other. These drivers and their configurations are new, SLI, and particularly 3-Way SLI, has been suffering as of late in many games, both in coordination and consistent frame times. Emphasis from nVidia on the last half-dozen driver releases has been mostly toward their newly-released Fermi microarchitecture. Many DX-9 games suffer with AA outside of in-game settings, and even B:AA won't let you get away with so much as AA Enhancement, much less overriding the settings.
 
That list of variables goes on a good measure, for sure. And chief among the reasons I tried to illustrate several games. Particularly Darkest of Days, which seems to present the same type of situation the ATHENA did on its release. The entire review wasn't intended to provide a thorough analysis of each individual situation - just illustrate what the gamer can expect and to show that you cannot depend on any singular barometer. Hence the recommendations to try each and see how each responds.
Were we to take the time for such an examination in detail in all cases, the time invested for the return wouldn't necessarily make it worthwhile either if changes are made to the drivers to accomodate as the drivers develop fauther.

...No, it won't make a huge difference. GT 240 is mostly enough for PhysX. Watch GPU usage on it while you're using medium level PhysX. It's going to be %50-85. Ageia's PPU only had the potential of that card. And it was what they officially released with shaders to do PhysX.

That potential never seems to have reached fruition. One of the engineers that transferred from AGEIA to nVidia claimed, at best, 9600-level support (64 shaders), but I gave up my ATHENA two years ago when I moved to GTX 260s and my old 8800 GTS (640)s trumped it by a considerable margin.
 

yourmom No, I never talked about a CPU bottleneck or the clock speed of your GPU. That edit was for a grammatical correction. And yes, I have been talking about results with AA from the beginning.

Considering I've been jumping in and out of two Forums and three threads on the topic, it's entirely possible I owe you an apology and someone else the "rundown".
 
Stick around and you'll see me make plenty of those types of gaffs - I'm getting old and senile..

yourmom You have done a great work on rest of your review. But you should have realized that in Batman, which is the first game people think of when PhysX is mentioned.
 
...In the end, having a dedicated card to do PhysX is better in your situation. You see that in your latest Batman benches too. PhysX does break SLI, you should have the GT 240 do PhysX. Even if it stays at good FPS in the bench, it won't perform like that near the buildings of Arkham Asylum. You'll need to turn down details or close PhysX on GTX 260 SLI. That's where GT 240 comes in and saves you. ...

I have no problems with that at all, or with your conclusions - just so you know.
 
And the original intent of the review, as stated, was to test the usefulness of not only a dedicated PhysX engine and the recommendation as to its size in shader count. As with the statements made concerning both the new version of FluidMark in conjunction with a new driver set, to show the results is not necessarily to determine the exact cause of the response. Do we agree that this point has at least been shown in a positive light, and we can therefore move on from here?...