Some PPS (Sieve) v1.39 (cudaPPSsieve) Runs
https://www.primegrid.com/results.php?userid=273570&offset=0&show_names=0&state=4&appid=9
GTX 1080 Ti 417.01
Run time (sec) 440.76 CPU time (sec) 67.36 https://www.primegrid.com/result.php?resultid=950392347
GTX 1080 Ti 390.65
Run time (sec) 431.49 CPU time (sec) 73.53 https://www.primegrid.com/result.php?resultid=950396221
 
Genefer 15 v3.19 (OCLcudaGFN15) Runs
https://www.primegrid.com/results.php?userid=273570&offset=0&show_names=0&state=0&appid=22
GTX 1080 Ti 417.01
Run time (sec) 81.31 CPU time (sec) 0.88 https://www.primegrid.com/result.php?resultid=950392347
GTX 1080 Ti 390.65
Run time (sec) 72.13 CPU time (sec) 5.98 https://www.primegrid.com/result.php?resultid=950387379
 
Genefer 16 v3.19 (OCLcudaGFN16) Runs
https://www.primegrid.com/results.php?userid=273570&offset=0&show_names=0&state=0&appid=23
GTX 1080 Ti 417.01
Run time (sec) 113.82 CPU time (sec) 0.63 https://www.primegrid.com/result.php?resultid=950384706
GTX 1080 Ti 390.65
Run time (sec) 126.11 CPU time (sec) 0.38 https://www.primegrid.com/result.php?resultid=950385703
 
Genefer 17 Low v3.19 (OCLcudaGFN17LOW) https://www.primegrid.com/results.php?
userid=273570&offset=0&show_names=0&state=0&appid=24
GTX 1080 Ti 417.01
Run time (sec) 315.87 CPU time (sec) 1.27 https://www.primegrid.com/result.php?resultid=950183317
GTX 1080 Ti 390.65
Run time (sec) 316.44 CPU time (sec) 0.70 https://www.primegrid.com/result.php?resultid=950183852
 
Genefer 17 Mega v3.19 (OCLcudaGFN17MEGA) https://www.primegrid.com/results.php?userid=273570&offset=0&show_names=0&state=0&appid=25
GTX 1080 Ti 417.01
Run time (sec) 353.36 CPU time (sec) 1.27 https://www.primegrid.com/result.php?resultid=950343326
GTX 1080 Ti 390.65
Run time (sec) 344.25 CPU time (sec) 0.78 https://www.primegrid.com/result.php?resultid=950342176
 
Genefer 18 v3.19 (OCLcudaGFN18) https://www.primegrid.com/results.php?userid=273570&offset=0&show_names=0&state=0&appid=26
GTX 1080 Ti 417.01
Run time (sec) 1,045.81 CPU time (sec) 4.80 https://www.primegrid.com/result.php?resultid=950319533
GTX 1080 Ti 390.65
Run time (sec) 1,039.76 CPU time (sec) 1.72 https://www.primegrid.com/result.php?resultid=950322804
 
Genefer 19 v3.19 (OCLcudaGFN19) https://www.primegrid.com/results.php?userid=273570&offset=0&show_names=0&state=0&appid=27
GTX 1080 Ti 417.01
Run time (sec) 2,942.28 CPU time (sec) 17.59 https://www.primegrid.com/result.php?resultid=950164479
GTX 1080 Ti 390.65
Run time (sec) 2,768.33 CPU time (sec) 10.05 https://www.primegrid.com/result.php?resultid=950162061
 
 

Did 417.01 correct all of the issues you were having?

All of your numbers above are very close together.

Gamma-ray pulsar binary search #1 on GPUs v1.20 () windows_x86_64 https://einsteinathome.org/account/tasks/0/0
GTX 1080 Ti 417.01
Run time (sec) 454 CPU time (sec) 452 https://einsteinathome.org/task/807077400
GTX 1080 Ti 390.65
Run time (sec) 449 CPU time (sec) 446 https://einsteinathome.org/task/807074529
Links above may or may not work.

MilkyWay @ Home v1.46 (opencl_nvidia_101) https://milkyway.cs.rpi.edu/milkyway/results.php?userid=1016454
GTX 1080 Ti 417.01
Run time (sec) 198.39 CPU time (sec) 109.53 https://milkyway.cs.rpi.edu/milkyway/result.php?resultid=75107090
GTX 1080 Ti 390.65
Run time (sec) 189.30 CPU time (sec) 104.50 https://milkyway.cs.rpi.edu/milkyway/result.php?resultid=75099677
 

Distributed.net Client v1.05 (opencl_nvidia_101) windows_intelx86 https://moowrap.net/results.php?userid=115955
GTX 1080 Ti 417.01
Run time (sec) 331.10 CPU time (sec) 320.45 https://moowrap.net/result.php?resultid=84673493
 
GTX 1080 Ti 390.65
Run time (sec) 322.40 CPU time (sec) 311.91 https://moowrap.net/result.php?resultid=84679318

Wonderful news! NVidia is reproducing the issue in house, and they are investigating now. The post don NVidia:

Robert Crovella
A widespread performance degradation should have been caught by our QA processes, so if this issue is mostly localized to a single app, that isn't necessarily surprising.

Also, contrary to this thread title, the development team has categorized this as an issue with OpenCL, not CUDA. I assume this means, in spite of the title of the executable, that the underlying (GPU) code is written in OpenCL, not CUDA, however I have not attempted to confirm this myself. The distinction isn't terribly important with respect to issue resolution. Merely a point of clarification for others who might read this thread and wonder if it applies to them.

The issue has been reproduced internally at NVIDIA, based on information provided so far via the bug report. There is a (fairly standard) plan in place to attempt to identify underlying root cause. I don't have any further information to share at this time, and won't be able to respond to requests for more information, most other questions, or any sort of inquiry about what the current status or state of the issue is, until there is sufficient forward progress on the analysis of the bug. At that time, I will do my best to be proactive and provide an update here. Until then, I'm unlikely to respond to requests for more information.

I don't expect the issue to be sorted out rapidly. Working with a compiled binary (as opposed to having source code and active participation from the developer) generally results in a slower progress of issue resolution (using the "standard" plan I referred to. If that plan doesn't yield useful info, progress can be even slower).

And of course, like all issues, resolution of this issue is subject to assessed priority as well as competing priorities in a resource-constrained environment.

 
 Topic: GeForce Drivers 4xx.xx drop more than 2/3 in OpenCL Performance from the 3xx.xx Drivers

---

There is no point in testing newer drivers; I don't expect any changes in this respect. Changes are required in the application if they want to restore performance with the newer drivers.
Current Scenario in ap26 app:
1. App queries CL_KERNEL_WORK_GROUP_SIZE in order to decide local work group size of either 1024 (seems optimal) or 64 (sub-optimal). If app gets value for query <1024 it reduces local work group size to 64 assuming device doesn't support 1024.
2. Nvidia OpenCL Driver changed return value for CL_KERNEL_WORK_GROUP_SIZE from 1024 to 256.
3. App is not using CL_KERNEL_WORK_GROUP_SIZE returned by driver as is, but just choosing a non-optimal local work-group size (64) based on this query.
What should developers do:
• Query CL_KERNEL_WORK_GROUP_SIZE to get just hint about work group size from driver and use it to launch kernel with that specific value. It need not be optimal for all kernels.
• App is free to choose any value from range [1 , CL_DEVICE_MAX_WORK_GROUP_SIZE] to get best possible work group size for different kernels, irrespective of CL_KERNEL_WORK_GROUP_SIZE returned by driver.
Suggestions specific to ap26:
• App can query CL_DEVICE_MAX_WORK_GROUP_SIZE and set work group size accordingly instead of using CL_KERNEL_WORK_GROUP_SIZE.
• Simplest solution for ap26 would be to use 1024 work group size directly if it comes in range [1 , CL_DEVICE_MAX_WORK_GROUP_SIZE].
I don't know how to best communicate the above information to the developers. If there is a good way to do that, please advise.

OK, but can you say that 3 times really fast ...

Even with this BR Driver it is still undermining the GTX 1080 Cards.