EVGA

Multiple Tasks on GPU for HCC

Author
yodap
CLASSIFIED Member
  • Total Posts : 4642
  • Reward points : 0
  • Joined: 2011/05/15 06:13:40
  • Location: NY, Upstate
  • Status: offline
  • Ribbons : 8
2012/12/06 06:43:21 (permalink)
I'm thinking that this topic needs to be "un-buried" from the Christmas Race thread. More eyes on it may mean more work done.
I have 2 460's and a 560ti to set up for this. Just waiting for tasks in the queue to finish. I'll have questions but rj's instructional post seems quite complete. Maybe it should be copied to this thread.
One of my concerns was that I'd need to sacrifice too many cpu threads but after rereading rj's post again it looks like it will be fine.
 
Can someone give an opinion on what a "good bang for the buck" AMD/ATI card would be for these tasks? I'm thinking of asking Santa for one.


 

 
#1

13 Replies Related Threads

    planetclown
    FTW Member
    • Total Posts : 1692
    • Reward points : 0
    • Joined: 2010/03/02 07:59:51
    • Status: offline
    • Ribbons : 15
    Re:Multiple Tasks on GPU for HCC 2012/12/06 08:44:14 (permalink)
    I bought a used 5850 to get some double-precision firepower on a budget.  However, I believe these tasks aren't DP and I've found the HCC GPU tasks take almost twice as long to complete when running two of them concurrently.  So ask the big man for something newer than a 5850 (real helpful huh? )

    EVGA Associates code: OLY2307BPM0MXZU Click for a discount on your next purchase from EVGA.com


    #2
    linuxrouter
    Omnipotent Enthusiast
    • Total Posts : 8043
    • Reward points : 0
    • Joined: 2008/02/28 14:47:45
    • Status: offline
    • Ribbons : 104
    Re:Multiple Tasks on GPU for HCC 2012/12/06 10:03:37 (permalink)
    How much are you looking to spend on an AMD card? The 7950 starts at around $290 and is not too far off from a 7970 for single precision computational performance.
     
    I have found that with this particular project, my AMD cards are much faster than my NVIDIA cards at processing many tasks at once if there are enough CPU resources to backup the GPU. I am running 8-tasks per GPU but I think the card could do even more than that especially if I had more CPU resources. My Kepler card on the other hand does not seem to like to run more than 1 HCC task at once. It will run, but explorer becomes near unresponsive.
     
    There are several things that I have found that can help the AMD card out. One is CPU resources. The more cores the better. Even with a CPU supporting 12-threads, my CPU load will run full load intermittently while running 16 tasks total. Also, I see significant bandwidth saturation on the PCI-E bus which is likely from all the data moving back and forth between the CPU and GPU for all running tasks. I suspect the more bandwidth available the better. Also, AMD has made very significant performance improvements with OpenCL via driver 12.11 beta. I would recommend running this driver version. My system saw a 25% increase in performance over driver 12.10 via another project I tested with.
    post edited by linuxrouter - 2012/12/06 10:06:43

    CaseLabs M-S8 - ASRock X99 Pro - Intel 5960x 4.2 GHz - XSPC CPU WC - EVGA 980 Ti Hybrid SLI - Samsung 950 512GB - EVGA 1600w Titanium
    Affiliate Code: OZJ-0TQ-41NJ
    #3
    kody7839
    CLASSIFIED Member
    • Total Posts : 3132
    • Reward points : 0
    • Joined: 2011/02/01 17:15:29
    • Location: Affiliate Code: 7P5KPZC7F5
    • Status: offline
    • Ribbons : 17
    Re:Multiple Tasks on GPU for HCC 2012/12/10 08:41:14 (permalink)
    Just to add to this thread, I've had 1 680 (i7-2600K@4.8) and 1 570 (i7-2600K@4.2) running 4 Milkway@home projects at a time with little to no impact on my day to day usage.  

    Here is the app_info.xml I'm using:
     
    <app> 
    <name>milkyway</name>
    </app>
    <file_info>
    <name>milkyway_separation_1.02_windows_x86_64__opencl_nvidia.exe</name>
    <executable/>
    </file_info>
    <app_version>
    <app_name>milkyway</app_name>
    <version_num>102</version_num>
    <flops>1.0e11</flops>
    <avg_ncpus>0.05</avg_ncpus>
    <max_ncpus>1</max_ncpus>
    <plan_class>cuda</plan_class>
    <coproc>
    <type>NVIDIA</type>
    <count>0.25</count>
    </coproc>
    <cmdline>
    </cmdline>
    <file_ref>
    <file_name>milkyway_separation_1.02_windows_x86_64__opencl_nvidia.exe</file_name>
    <main_program/>
    </file_ref>
    </app_version>
    </app_info>

     
    Don't mean to steer this away from HCC projects, but just letting people know it's possible with other projects as well. As soon as SETI gets their servers back stable, I plan to try the same with them as well. 
     

    My computer finds cures for diseases and searches for aliens when I'm not gaming...what does yours do?
     
    Ryzen 1700, ASRock Fatal1ty X370 Professional, EVGA GTX1080Ti FTW3, 16GB Corsair LPX, Noctua D14, EVGA SuperNOVA 650 P2, Overlord Tempest X270X Glossy, LG 34UM94 Ultrawide 
     
    #4
    rjbelans
    CLASSIFIED ULTRA Member
    • Total Posts : 6223
    • Reward points : 0
    • Joined: 2009/11/20 20:23:25
    • Location: N.F., ON, CA
    • Status: offline
    • Ribbons : 31
    Re:Multiple Tasks on GPU for HCC 2012/12/10 14:17:11 (permalink)
    I'm running a 7870 and a 7970 with 20 units on each of them and no problems. I finish a work unit every minute or so on each of them (7970 is a little faster, but not by much). The limiting factor is the CPU. I have 8 cores/threads dedicated for each of them and staggered the start of each unit so that the CPU isn't trying to do 20 units at a time at the beginning and end of each image.
     
    What you choose really depends on your budget.
     
    Anybody could have copied this over, I would have been ok with it.
     
    rjbelans
    To run more than 1 unit on a card at a time, you need an app_info.xml file. I have a couple here, but they will limit you to just doing GPU units. You can copy the appropriate text below and paste it into a new Notepad file and save as app_info.xml into the www.worlcommunitygrid.org directory under the projects directory.

    Be sure to have ONLY HCC1 GPU work units in the queue before adding the app_info file. Pause queued units (and currently running unit(s) if you don't mind starting them over) and exit BOINC completely (on some of my rigs I need to go into Task Manager to end BOINC from running in the background). Restart BOINC and resume the paused tasks. You should now show in the Status column of BOINC Manager that multiple units are running on the same "device" number.

    NVIDIA
    <app_info> 
    <app>
    <name>hcc1</name>
    <user_friendly_name>Help Conquer Cancer</user_friendly_name>
    </app>
    <file_info>
    <name>wcg_hcc1_img_7.05_windows_intelx86__nvidia_hcc1</name>
    <executable/>
    </file_info>
    <file_info>
    <name>hcckernel.cl.7.05</name>
    <executable/>
    </file_info>
    <app_version>
    <app_name>hcc1</app_name>
    <version_num>705</version_num>
    <platform>windows_intelx86</platform>
    <plan_class>nvidia_hcc1</plan_class>
    <avg_ncpus>1</avg_ncpus>
    <max_ncpus>1</max_ncpus>
    <coproc>
    <type>CUDA</type>
    <count>0.5</count>
    </coproc>
    <file_ref>
    <file_name>wcg_hcc1_img_7.05_windows_intelx86__nvidia_hcc1</file_name>
    <main_program/>
    </file_ref>
    <file_ref>
    <file_name>hcckernel.cl.7.05</file_name>
    <open_name>hcckernel.cl</open_name>
    </file_ref>
    </app_version>
    </app_info>


    AMD/ATI
    <app_info> 
    <app>
    <name>hcc1</name>
    <user_friendly_name>Help Conquer Cancer</user_friendly_name>
    </app>
    <file_info>
    <name>wcg_hcc1_img_7.05_windows_intelx86__ati_hcc1</name>
    <executable/>
    </file_info>
    <file_info>
    <name>hcckernel.cl.7.05</name>
    <executable/>
    </file_info>
    <app_version>
    <app_name>hcc1</app_name>
    <version_num>705</version_num>
    <platform>windows_intelx86</platform>
    <plan_class>ati_hcc1</plan_class>
    <avg_ncpus>0.5</avg_ncpus>
    <max_ncpus>1.0</max_ncpus>
    <coproc>
    <type>ATI</type>
    <count>0.33</count>
    </coproc>
    <file_ref>
    <file_name>wcg_hcc1_img_7.05_windows_intelx86__ati_hcc1</file_name>
    <main_program/>
    </file_ref>
    <file_ref>
    <file_name>hcckernel.cl.7.05</file_name>
    <open_name>hcckernel.cl</open_name>
    </file_ref>
    </app_version>
    </app_info>


    Some info to use when adjusting for your own use.

    There are two key lines to look at when setting how many units to run; <avg_ncpus>#.##</avg_ncpus> and <count>#.##</count>. Both of these numbers tell BOINC Manager how much of a CPU or GPU core you would like to dedicate to each unit. Adjust these numbers to allow your hardware to run the number of units you want it to.
    NOTE!: You [<font]cannot run NVIDIA cards with less than one full CPU core/thread or everything will bog down regardless of how many CPU cores/threads you have. So, just leave <avg_ncpus>1.0</avg_ncpus> alone for NVIDIA cards.

    Some handy numbers (in units per core or GPU) to help when changing these settings (leading and trailing 0s are not necessary, and you don't have to be any more accurate than 3 decimals to get the desired effect):
    1.000 ---> 1  unit
    0.500 ---> 2  units
    0.330 ---> 3  units
    0.250 ---> 4  units
    0.200 ---> 5  units
    0.166 ---> 6  units
    0.142 ---> 7  units
    0.125 ---> 8  units
    0.111 ---> 9  units
    0.100 ---> 10 units
    0.090 ---> 11 units
    0.083 ---> 12 units
    0.076 ---> 13 units
    0.071 ---> 14 units
    0.066 ---> 15 units
    0.062 ---> 16 units
    0.058 ---> 17 units
    0.055 ---> 18 units
    0.052 ---> 19 units
    0.049 ---> 20 units  ( I know, it should be 0.050, but for some reason I only got 19 units until I changed it to 0.049)

    Note: I went up to 20 because I know first hand that high end AMD cards will do 20 each and I haven't heard of anyone going over 20 yet.

    #5
    Punchy
    CLASSIFIED Member
    • Total Posts : 2969
    • Reward points : 0
    • Joined: 2010/02/06 09:33:05
    • Status: offline
    • Ribbons : 14
    Re:Multiple Tasks on GPU for HCC 2012/12/16 19:08:02 (permalink)
    So how do I figure out how many GPU tasks are optimal for an HD 6450?  With a single task, it's taking about 13 minutes elapsed time.
     
    And why does my results status on the WCG site show elapsed/CPU time as 0.74 hours when BOINC Manager shows them as 13 minutes?
    post edited by Punchy - 2012/12/16 19:22:31


    #6
    rjbelans
    CLASSIFIED ULTRA Member
    • Total Posts : 6223
    • Reward points : 0
    • Joined: 2009/11/20 20:23:25
    • Location: N.F., ON, CA
    • Status: offline
    • Ribbons : 31
    Re:Multiple Tasks on GPU for HCC 2012/12/17 04:29:09 (permalink)
    I figure out the number of tasks by trial and error. If the average time to complete 1 unit stays below the previous trial, it is better. So, if you try 1 unit and get 13 min, you want to see if 2 units will finish in less than 26 min, and so on until you start seeing errors (I've only had my NVIDIA cards start to error with too many units) or your CPU starts to get overwhelmed. Typically, you will see the time go from 13 min to 14 or 15 min when you go from 1 to 2 tasks on the GPU. So when you go to 3 units, you want to see times under 21 min. I haven't really found the tipping point on the 7870 or 7970 due to CPU saturation, but they each run 20 units ata time and complete them in around 20 min, which means submitting a unit about every minute.
     
    WCG has two different numbers that it tracks, the elapsed time and the CPU time. The elapsed time is from download to upload, whereas the CPU time is the actual processing time. I keep a cache, so I get elapsed times in 0.5+ hours, but CPU time more like 0.08 hours (when running 20 units at a time).
    #7
    Punchy
    CLASSIFIED Member
    • Total Posts : 2969
    • Reward points : 0
    • Joined: 2010/02/06 09:33:05
    • Status: offline
    • Ribbons : 14
    Re:Multiple Tasks on GPU for HCC 2012/12/17 06:02:18 (permalink)
    It went up to over 20 minutes for two tasks, so I think it's already near saturation at 1.  Still, for a fanless card running at 38 degrees C, and not many watts, it's producing about the same as the 12 cores of that system would running the CPU version.
    rj, are you saying your 7870 and 7970 produce about the same, even though the 7970 series seems to have nearly twice the stream processors of the 7870s?


    #8
    rjbelans
    CLASSIFIED ULTRA Member
    • Total Posts : 6223
    • Reward points : 0
    • Joined: 2009/11/20 20:23:25
    • Location: N.F., ON, CA
    • Status: offline
    • Ribbons : 31
    Re:Multiple Tasks on GPU for HCC 2012/12/17 12:15:58 (permalink)
    Very nearly the same. The 7870 @ 1.050GHz does take closer to 22 min to finish each unit, which is 1:06 per unit instead of 1:00 that I get on the 7970 @ 1.100 GHz. The bigger factor might be the supporting CPU. The 7870 is with an i7 920, with a bit of an OC I believe, whereas the 7970 is with an FX-8350, at stock. I haven't taken the time to play around with different setups to see how much of a difference there is between each piece of hardware. After Christmas, i will likely take a look at what the cards can do with my Hexa-Core CPUs.
    #9
    Col323
    Superclocked Member
    • Total Posts : 115
    • Reward points : 0
    • Joined: 2011/11/28 09:07:30
    • Status: offline
    • Ribbons : 2
    Re:Multiple Tasks on GPU for HCC 2012/12/21 06:21:07 (permalink)
    Anyone crunching with an AMD card on Linux?  If so, any tricks or hurdles to get up and going?  I have space in my Ubuntu 12.04 box, and well, Christmas is coming.  

    None of us is as strong as all of us.

      
     


     

      

     
    #10
    linuxrouter
    Omnipotent Enthusiast
    • Total Posts : 8043
    • Reward points : 0
    • Joined: 2008/02/28 14:47:45
    • Status: offline
    • Ribbons : 104
    Re:Multiple Tasks on GPU for HCC 2012/12/21 18:17:01 (permalink)
    Col323
    Anyone crunching with an AMD card on Linux?  If so, any tricks or hurdles to get up and going?  I have space in my Ubuntu 12.04 box, and well, Christmas is coming.  

     
    I have a extra SSD with Linux and the AMD drivers on it. Getting up and running was not too bad. In addition to the AMD drivers, the APP SDK is necessary. The SDK will add OpenCL support. I am running Slackware64 14.0 currently but did manage to get the AMD cards going in BOINC via Ubuntu 12.10 for a short while. I had to install the development tools (GCC) and 32-bit libraries in Ubuntu. I also had to uninstall the factory NVIDIA drivers that come with Ubuntu.
     
    If you plan to run more than one AMD card, there are two things to keep in mind.
     
    When running aticonfig to generate the xorg.conf, you want to run this command for all adapters to be detected and configured:
     
    aticonfig --adapter=all --initial
     
    For BOINC to see both cards, you want to set the following in xterm before starting the BOINC client:
     
    export DISPLAY=:0
     
    Without setting the DISPLAY, BOINC will only be able to use one card.
     
    I would recommend running the 12.11 beta drivers with APP SDK 2.7 or 2.8 due to a very significant increase in performance for OpenCL. Overclocking is supported in Linux using aticonfig. I found that memory frequency with the AMD cards does not give any performance gain with the projects I have run so far. Based on that, I reduce my memory frequency to a lower value to save on power.
     
    At the moment, I am temporarily running Windows for the HCC application and the WCG challenge but I believe the project has released a Linux version of the HCC application. I have not tried the Linux HCC application yet but it is on my list of things to do.
    post edited by linuxrouter - 2012/12/21 18:18:31
    #11
    Col323
    Superclocked Member
    • Total Posts : 115
    • Reward points : 0
    • Joined: 2011/11/28 09:07:30
    • Status: offline
    • Ribbons : 2
    Re:Multiple Tasks on GPU for HCC 2012/12/22 07:46:28 (permalink)
    Excellent information.  This is exactly what I was looking for.  Thanks a ton!

    None of us is as strong as all of us.

      
     


     

      

     
    #12
    linuxrouter
    Omnipotent Enthusiast
    • Total Posts : 8043
    • Reward points : 0
    • Joined: 2008/02/28 14:47:45
    • Status: offline
    • Ribbons : 104
    Re:Multiple Tasks on GPU for HCC 2012/12/22 11:44:25 (permalink)
    I just got the WCG HCC GPU application running in Linux. The application is compiled as 64-bit which should make it easy to get up and running on a 64-bit distribution.
     
    I made this app_info.xml file for running multiple tasks per AMD GPU. The count is set to .25 for running 4 tasks per GPU. You can adjust this value to run more or less tasks at once. The file only has the GPU application in it. Adding the CPU applications is doable but will take some time as there are so many of them.
     
    Edit:
     
    It turns out that in Linux, the WCG project servers are not allowing for anonymous platform when trying to download or upload HCC tasks. However, I found that with BOINC 7.0.40 and greater, there is a new option for creating a app_config.xml in the project directory which will allow for running multiple tasks per GPU.
     
    # Sample with two GPU running four concurrent tasks each
    File Location: BOINC/projects/www.worldcommunitygrid.org/app_config.xml
     
    <app_config>
    <app>
    <name>hcc1</name>
    <max_concurrent>8</max_concurrent>
    <gpu_versions>
    <gpu_usage>.25</gpu_usage>
    <cpu_usage>.5</cpu_usage>
    </gpu_versions>
    </app>
    </app_config>
     
    post edited by linuxrouter - 2012/12/22 14:02:24
    #13
    RHMash
    FTW Member
    • Total Posts : 1117
    • Reward points : 0
    • Joined: 2009/12/08 13:56:43
    • Status: offline
    • Ribbons : 6
    Re:Multiple Tasks on GPU for HCC 2013/03/05 17:50:54 (permalink)

     
    thought this was pretty awesome....



    #14
    Jump to:
  • Back to Mobile