GTXJackBauer

Sajin
The reason the cuda cores are so high is because the ampere cores can do 2 operations per clock. 10496/2 = 5248

I'm wonder with that kind of advertising, that might create some legal trouble.  So basically they're doing what Intel does but instead, they're advertising them as all physical cores, no?  Why not do something similar that intel does and say 5248 Physical Cores and 10496 Virtual Cores?  I'm kind of disappointed they took this route of advertising making many consumers think it's packed with that many physical cores.  At least they had fooled me at first.

Well it's not "technically" incorrect since they aren't just calling a "hyper threading" virtual core a core per se. What Nvidia has done with Ampere is allow the shader core to execute INT32 and FP32 instructions in parallel with independent thread scheduling instead of having the sharder only be able to execute one or the other at a time. There are 2 separate physical compute units per shader core and now that the shader cores have independent thread scheduling they can execute instructions in parallel. That's how they doubled their FP32 compute performance. Previously as seen in Turing these same compute units are present but they lack independent thread scheduling so can only ever be doing one or the other, not both at the same time per shader core as they can with Ampere. 
 
This is quite a bit different than the way hyper threading or simultaneous multiprocessing work in CPUs. The way CPUs process threads in parallel isn't "really" simultaneous since the CPU core only works on each instruction set during waiting cycles in each thread being worked on. So for instance a CPU that has 2 tasks to work on it will only work on one at a time and while waiting for memory calls or instructions to execute after issuing a command to one task it will work on the other one. This is also why 8 physical cores is always faster than 4 physical cores processing 8 threads in a multithreaded workload. 

After further research it appears that the card does actually have 10496 cuda cores (fp32 cores), but the int32 cores don't match that of the fp32 cores. That is why the card is faster overall, but isn't double the speed of 2080 ti.

Sajin
After further research it appears that the card does actually have 10496 cuda cores (fp32 cores), but the int32 cores don't match that of the fp32 cores. That is why the card is faster overall, but isn't double the speed of 2080 ti.

Ok I'm still learning here, are the CUDA cores indicative of total core count, including RT cores? Because 3000 series isn't simply 50% faster tier for tier vs their predecessor, they do RT 1.9x faster to top it off. 
 
Edit: 
 
Getting ahead of myself, we still don't know how Ampere overclocks, it may only do another 20% making it only 40% faster. But even only another 10% overclock and Ampere is still 30% faster tier for tier. 

Sajin

vulcan1978

Sajin
After further research it appears that the card does actually have 10496 cuda cores (fp32 cores), but the int32 cores don't match that of the fp32 cores. That is why the card is faster overall, but isn't double the speed of 2080 ti.

Ok I'm still learning here, are the CUDA cores indicative of total core count, including RT cores? Because 3000 series isn't simply 50% faster tier for tier vs their predecessor, they do RT 1.9x faster to top it off. 

No.

Wow, so what exactly is the difference in cores between Turing and Ampere? I need to watch GN's Ampere break-down. 

The original video is clearly adding in the frame rate counter. You hear 65% improvement to 85% improvement in the video but the frame counter that is added shows it as being only a 15% improvement. The original video shows only a 2080Ti frame rate which seems to be locked at 60FPS for making a smoother video for Youtube to upload. Even based on just 60FPS for Shadow of the Tomb Raider the 65% improvement would take you to 99FPS which is a nice improvement 85% improvement which is the max we saw would be going from 60FPS to 111FPS. Below I posted the original video that hasn't been edited so people can see the original post is clickbait.
 
https://www.youtube.com/watch?v=cWD01yUQdVA&feature=emb_logo

IMWork87
Turing: 1 SM -> 64 FP32-ALUs and 64 INT32-ALUs
Ampere: 1 SM -> same as Turing but additional 64 ALUs that can also calculate FPs and INTs but not simultaneously.
 
While Turing is able to handle 64 FP32 and 64 INT32 calculations at the same time, Ampere can either handle 128 FP32 or 64FP32 and 64INT32 calculations at the same time, depending on the workload.
 
So if a game engine uses both, FP and INT, Ampere has the same performance as Turing (worst case). If an engine uses primarly FP, though, Ampere has twice the compute power compared to Turing.
That's the reason why Nvidia is talking about twice the CUDA-Cores.
 
I'd assume that the games used in the Digital Foundry Video primarly used FP calculations to show Ampere's benefits. Don't expect to see a performance gain like this in all your games.

Fantastic reply, thanks for adding to the discussion! I did some research and have mostly concluded the same thing, I also found a thread that really breaks things down and shows that 1 Ampere TFLOP is actually .7 TFLOP compared to a Turing TFLOP. But reading the technical aspects of this, apparently only 26% of an average game instruction stream is INT32, meaning, 74% is actually FP32 and would benefit from the new doubling of FP32 performance. 
 
Ampere also has doubled the shared memory and L1 cache performance per SM:
 
"Doubling math throughput required doubling the data paths supporting it, which is why the Ampere SM also doubled the shared memory and L1 cache performance for the SM. (128 bytes/clock per Ampere SM versus 64 bytes/clock in Turing). Total L1 bandwidth for GeForce RTX 3080 is 219 GB/sec versus 116 GB/sec for GeForce RTX 2080 Super."
 
"

• • Could you elaborate a little on these doubling of CUDA cores?
  
  [Tony Tamasi] One of the key design goals for the Ampere 30-series SM was to achieve twice the throughput for FP32 operations compared to the Turing SM. To accomplish this goal, the Ampere SM includes new datapath designs for FP32 and INT32 operations. One datapath in each partition consists of 16 FP32 CUDA Cores capable of executing 16 FP32 operations per clock. Another datapath consists of both 16 FP32 CUDA Cores and 16 INT32 Cores. As a result of this new design, each Ampere SM partition is capable of executing either 32 FP32 operations per clock, or 16 FP32 and 16 INT32 operations per clock. All four SM partitions combined can execute 128 FP32 operations per clock, which is double the FP32 rate of the Turing SM, or 64 FP32 and 64 INT32 operations per clock.
  Doubling the processing speed for FP32 improves performance for a number of common graphics and compute operations and algorithms. Modern shader workloads typically have a mixture of FP32 arithmetic instructions such as FFMA, floating point additions (FADD), or floating point multiplications (FMUL), combined with simpler instructions such as integer adds for addressing and fetching data, floating point compare, or min/max for processing results, etc. Performance gains will vary at the shader and application level depending on the mix of instructions. Ray tracing denoising shaders are good examples that might benefit greatly from doubling FP32 throughput.
  Doubling math throughput required doubling the data paths supporting it, which is why the Ampere SM also doubled the shared memory and L1 cache performance for the SM. (128 bytes/clock per Ampere SM versus 64 bytes/clock in Turing). Total L1 bandwidth for GeForce RTX 3080 is 219 GB/sec versus 116 GB/sec for GeForce RTX 2080 Super.
  Like prior NVIDIA GPUs, Ampere is composed of Graphics Processing Clusters (GPCs), Texture Processing Clusters (TPCs), Streaming Multiprocessors (SMs), Raster Operators (ROPS), and memory controllers.
  The GPC is the dominant high-level hardware block with all of the key graphics processing units residing inside the GPC. Each GPC includes a dedicated Raster Engine, and now also includes two ROP partitions (each partition containing eight ROP units), which is a new feature for NVIDIA Ampere Architecture GA10x GPUs. More details on the NVIDIA Ampere architecture can be found in NVIDIA’s Ampere Architecture White Paper, which will be published in the coming days.
  
  

https://wccftech.com/nvidia-details-geforce-rtx-30-series-graphics-cards-reddit/
 
https://www.nvidia.com/en-us/geforce/news/rtx-30-series-community-qa/
 
Nvidia Ampere teraflops and how you cannot compare them to Turing
 
https://neogaf.com/threads/nvidia-ampere-teraflops-and-how-you-cannot-compare-them-to-turing.1564257/
 
"

• #1TL;DR 1 Ampere TF = 0.72 Turing TF, or 30TF (Ampere) = 21.6TF (Turing)
  
  Reddit Q&A
  
  To accomplish this goal, the Ampere SM includes new datapath designs for FP32 and INT32 operations. One datapath in each partition consists of 16 FP32 CUDA Cores capable of executing 16 FP32 operations per clock. Another datapath consists of both 16 FP32 CUDA Cores and 16 INT32 Cores. As a result of this new design, each Ampere SM partition is capable of executing either 32 FP32 operations per clock, or 16 FP32 and 16 INT32 operations per clock. All four SM partitions combined can execute 128 FP32 operations per clock, which is double the FP32 rate of the Turing SM, or 64 FP32 and 64 INT32 operations per clock.
  
  
  A reminder from the Turing whitepaper:
  First, the Turing SM adds a new independent integer datapath that can execute instructions concurrently with the floating-point math datapath. In previous generations, executing these instructions would have blocked floating-point instructions from issuing.
  
  
  So, Turing GPU can execute 64INT32 + 64FP32 ops per clock per SM.
  Ampere GPU can either execute 64INT32 + 64FP32 or 128FP32 ops per clock per SM.
  
  Which means if a game executes 0 (zero) INT32 instructions then Ampere = 2xTuring
  And if game executes 50/50 INT32 and FP32 then Ampere = Turing exactly.
  
  So how many INT32 are there on average?
  According to Nvidia:
  
  we typically see about 36 additional integer pipe instructions for every 100 floating point instructions
  
  
  Some math: 36 / (100+36) = 26%, i.e. in an average game instruction stream 26% are INT32
  
  So we can now calculate what will happen to both Ampere and Turing when 26% INT32 + 74% FP32 instruction streams are used.
  I have written a simple software to do that. But you can calculate an analytical upper bound easily: 74%/50% = 1.48 or +48%
  My software shows a slightly smaller number +44% (and that's because of the edge cases where you cannot distribute the last INT32 ops in a batch equally, as only one pipeline can issue INT32 per each block of 16 cores)
  So the theoretical absolute max is +48%, in practice the absolute achievable max is +44%
  
  Thus each 2TF of Ampere have only 1.44TF of Turing performance.
  
  Let's check the actual data Nvidia gave us:
  3080 = 30TF (ampere) = 21.6TF (turing) = 2.14x 2080 (10.07TF turing)
  Nvidia is even more conservative than that and gives us: 3080 = 2x2080
  3070 = 20.4TF (ampere) = 14.7TF (turing) = 1.86x 2070 (7.88TF turing)
  Nvidia is massively more conservative here giving us: 3070 = 1.6x2070
  Actually if we average the two max numbers that Nvidia gives us (they explicitly say "up to") we get to even lower theoretical max of 1 Ampere TF = 0.65 Turing TF
  Which suggests that maybe these new FP32/INT32 mixed pipelines cannot execute FP32 at full speed (or cannot execute all the instructions).
  We do know that Turing had reduced register file access in INT32 (64 vs 256 for FP32) if it's the same (and everything suggests that Ampere is just a Turing facelift) then obviously not all FP32 instruction sequences can run on these pipelines.
  
  Anyway a TF table:
  
   Ampere TFTuring TF (me)Turing TF (NV)3080 (Ampere)3021.619.53070 (Ampere)20.414.713.32080Ti (Turing)18.75 (me) or 20.7 (NV)13.513.52080 (Turing)14 (me) or 15.5 (NV)10.110.12070 (Turing)10.4 (me) or 11.5 (NV)7.57.5
  
  Bonus round: RDNA1 TF
  RDNA1 has no INT32 pipeline, all the INT32 instructions are handled in the main stream. Thus it's essentially almost exactly the same as Ampere, but it has no skew in the last instruction thus +48% theoretical max applies here (Ampere +2.3%)
  
   Ampere TFTuring TF (me)Turing TF (NV)5700XT (RDNA1)10.017.2?
  
  Amusingly enough 5700XT actual performance is pretty similar to 2070 and these adjusted TF numbers show exactly that (10TF vs 10-11TF)"

He was invited by Nvidia to have the first look and hands on with the card. 

Hoggle
The original video is clearly adding in the frame rate counter. You hear 65% improvement to 85% improvement in the video but the frame counter that is added shows it as being only a 15% improvement. The original video shows only a 2080Ti frame rate which seems to be locked at 60FPS for making a smoother video for Youtube to upload. Even based on just 60FPS for Shadow of the Tomb Raider the 65% improvement would take you to 99FPS which is a nice improvement 85% improvement which is the max we saw would be going from 60FPS to 111FPS. Below I posted the original video that hasn't been edited so people can see the original post is clickbait.
 
https://www.youtube.com/watch?v=cWD01yUQdVA&feature=emb_logo

The only things that can do ~60fps in that benchmark, as shown by the partially obscured FPS counter in the upper left of the DF video are an overclocked 2080 Ti, the 3080 or something comparable.  Not a 2080, stock 2080 Ti or 1080 Ti.  The framerate is not locked.
 
https://www.youtube.com/watch?v=X5CrHwlCItg
https://youtu.be/aznxnYrZxKY
https://youtu.be/MoRAxF06_Ck
 
The FPS is partially obscured in the upper left.  It's not added in, it's part of the benchmark.
 
https://youtu.be/cWD01yUQdVA?t=412
 
The 2080 being compared does 30-45 FPS in the benchmark at those settings.