TheWolf
Here is something I found that may help others when there not seeing a full 100% load across all cores.
Say its only in the 90 to 95% load range on all cores. In my case its 20 cores.

I can open Task Manager right click on the VirtualBox go to Set Affinity uncheck the last 10 cores, click ok to set, this will put the other 10 cores up to a full 100% load.

I leave this for about 15 to 30 seconds then click the box at the top to have all cores included again. Once done and clicking ok it now loads up all 20 cores at 98 to 100% load.

I have no idea why this works or forces more of a load, but it does hold an I see better times per frame.
Edit: also I might add that the time drop in my post above this one is when I found this.  Did a 3rd regular smp
and it yield the same 3:55 times.  Now testing the same on a 8101 and its frame times are looking better than ever.

Resurrecting this post because I've been attempting to solve what many Folders have seen with VBox...getting the CPUs (in greater than 8-thread configs) to run at or near 100% usage.  Seems like if you are running a single quad/8-thread CPU, the problem doesn't surface.  But when you move to an SR-2 with 16, 20, or 24-threads the problem often arises.
 
I've been back playing with VBox and like many others, I can't get my CPU usage levels above 85% on my SR-2.  I'm running Mint13 as my Virtual OS.  I've tried Wolf's suggestion above plus a bunch of different permutations of it and at best I can affect about half my CPUs with it.  I can't get all of them to run at or near 100% and the average has been 85%. 
 
I'm wondering what to do next?  I've searched the web for people that are Folding and trying to run full out in VBox (as we do in native Windows/Liunux).  But, I've had no success in locating a solution to the problem.  In my situation, VBox is a great alternative that allows me to run Win7 and Fold in Linux.  But the 85% max CPU resources is hurting my Folding production.
 
Anyone have an idea as to what I could do next?

Sorry for not specifying, yes it is Mint13 GUI running in a "Gnome Classic (No effects)" user session.  I've considered whether I should revert back to a non-GUI form of Linux to see if that has any effect.  But I could swear that I didn't have this issue before with previous Linux GUI distros.  Maybe it is Mint13 because that is a recent change as I had used Ubuntu 10.10 GUI in the past. 
 
The Windows Task Manager CPU Usage (when Folding Linux in VBox) has never been that rock solid 100% line that we see when Folding in native Windows.  But in the past it was a ripple-type line that ran in the 98-100% range.  Now, I can't seem to get above 85% overall CPU Usage.  Some CPU/threads that will run up toward 95% and the other half want to hover at 85% and even less for some of the CPU/threads.  I had "the Kraken" running too and it was not able to overcome the problem from within Linux.  Seems that in the transition back over to Windows, VBox (even though I have the "Processor Execution Cap" setting to 100%) is not allowing the CPUs to run at max CPU Usage (as seen in Task Manager).

There is another option that may help edit the Grub to use terminal.
 

Once you have everything set up, there is no need to boot into the GUI.
This will let you machine boot to a command line.
1. Open a terminal and type: Code: sudo nano /etc/default/grub
Change this line: Code: GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
to this: Code: GRUB_CMDLINE_LINUX_DEFAULT="quiet splash text"
Now, update grub: Code: sudo update-grub
The next time you reboot, you will boot to a command line.
To start the GUI, type: Code: sudo service gdm start
To stop the GUI, type:Code: sudo service gdm stop

Ok after about 5% complete on a standard SMP WU #6947, here's where I'm at:

• Running Linurouter's image in Terminal mode now.
• Windows Task Mgr is now indicating ~92-95% (fluctuates between the two levels).  That is an improvement from the 85% level that I was seeing in the VBox/Mint 13 GUI installation.
• TPFs comparing this WU (6947) in progress under Linux64_FAH/VBox vs a similar WU (6944) in native Windows is:
  
  • Native Windows TPF =1:18
  • Linux64_FAH in VBox = 1:25
• I have "NUMA" disabled in the SR-2 BIOS
• I do not have the Kraken or BFS running in the Linux64_FAH VBox setting.  From what I've read over at [H], it appears that BFS was recommended, but NUMA needed to be disabled and the Kraken was having no impact.  http://hardforum.com/show...p;highlight=virtualbox

• Many of the responses in that [H] thread above said the best they could get in CPU Usage was what I'm now seeing (92-95%).
• Lastly, I can't seem to figure out how to install other things (like the Kraken) using Linuxrouter's image in Terminal mode.  I can fire up FAH, list directories, etc.  But if I try to install something else, I get a lot of "not found" or "can't do it" type of responses.  Also, when I type in that Terminal, there is no keyboard echo and I can't verify what I'm typing as I type.  How do I get around that situation?

Anyway, this is where I am right now and don't really know what (if anything) to do next.  Is this as good as it gets or am I missing something?

Tex a question why not do you use the qemu or kvm module to run a windows os inside a virtual machine?.Try to enable NUMA according to another apps i use/have used not for folding only.What version of the kernel you are running ? if that is a v3.0 read this
http://lkml.indiana.edu/h...rnel/1212.0/03076.html according to this NUMA was improved at version 3 of the kernel so expect no big diff between turning it ON or OFF but you can do some test to confirm that because it was some long time ago when i tried on non folding apps.
If not try to run windows 8 with an hyper-v Linux OS guest.

NerdGZ
Tex a question why not do you use the qemu or kvm module to run a windows os inside a virtual machine?.Try to enable NUMA according to another apps i use/have used not for folding only.What version of the kernel you are running ? if that is a v3.0 read this
http://lkml.indiana.edu/h...rnel/1212.0/03076.html according to this NUMA was improved at version 3 of the kernel so expect no big diff between turning it ON or OFF but you can do some test to confirm that because it was some long time ago when i tried on non folding apps.
If not try to run windows 8 with an hyper-v Linux OS guest.

I have considered that very thing (install the Windows OS virtually from within native Linux), but it seems that it would be a lot more complicated that running Vbox within Windows.  I could be wrong because I haven't tried it, but it sure feels that it could be even more complicated. 
 
I have no idea what version of the kernal that I'm running using Linurouter's Linux64_FAH installation.  I'd be happy to check that, but don't know how to do it.  I'm running at the edge of my experience with this stuff right now, so some of the things you guys know off the top of your head is still "greek" to me. 

Excellent help NerdGZ!  I will need to take time and watch those videos in a bit.  But I can answer your Kernel question (thanks to your information).
 
The Linux Kernel shows on-screen as the following from that "uname -a" command:

• Linux FAH 3.4.4-corei7-gen1 #6 SMP Sun Jul 1 16:12:23 MST 2012 x86_64 GNU/Linux

 
This whole thread probably needs to moved over to the Help section as some sort of a Virtualbox Guide! 
 
Thanks!

I've searched for Linux education (self taught stuff) and never ran across that IBM site that you shared.  That looks real good and I will spend some time there.  I just ran across a free book titled "The Linux Command Line" by William Shots.  He sells it on Amazon, but also makes it available in a PDF format for free via his website.
http://linuxcommand.org/tlcl.php
 
On that website (for any interested) look for the sentence below that has a link to where you can download the book for free:
Released under a Creative Commons license, this book is available for free download in PDF format. Download it here.
 
 

Thanks Wolf, your post got me going with the Kraken installation.  Where I was going wrong was not putting the "./" in front of "thekraken" command.  Once I typed it as you indicated, the Kraken took off and wrapped my Core.
 
So, I fired up a fresh standard SMP WU, saw that the Kraken also started up with the WU and then I used the "Top" command to verify the Kraken was in the running processes list.
 
Back to your tweak/experiment with the Affinity setting on the Virtualbox.exe process, I tried it again as you had done.  Unfortunately in my situation, the CPU usage levels remained where they were (an average of 92%).  But that is still better than it was at 85%.  So, your suggestion of going back to Linuxrouter's Terminal only Linux installation was a success.  Thanks for taking time to help me today!
 
PS:  I still can't figure out how to get what I type to echo on the screen.  Its started working OK at one point today, but now is back to not showing what I'm typing.  So I've been having to "type in the dark" as it were 'cause the characters I'm typing are getting through.  I've tried starting another Terminal window, switching windows, etc, but that hasn't triggered being able to see what I'm typing.  I'd sure like to solve that because it's annoying not being able to see what I'm typing. 

Yes, I had done several "flavors" of affinity settings and I can't seem to change the way the CPU Usage is working.  I have not gone back and enabled NUMA becuase I'd seen where the guys over at [H] were saying that it was having worse effect upon TPFs in this Virtualbox setting.  I can always enable it to see what happens though.
 
Using that Linux Command Line book that I mentioned above, I figured out how to make the FAHlog.txt file visible across the network so I could read it with HFM.  At the command prompt I typed:
chmod 775 FAHlog.txt
 
I'm still having to get used to needing to type things exactly as they are in Linux.  For example, if I typed fahlog.txt in the chmod example above, it would result in a "no such file" response from the shell.  Can I say that I really love this stuff...ummmm...nope.  But it is serving a purpose, so I'm trying to learn to "swim with the sharks". 

Yes, closing down the Virtual window, saving the state ans restarting it solved the problem with my typing not echoing on-screen.  Thanks!  I don't know what triggers it to stop showing what I'm typing...maybe a bug of some kind.
 
I have one of those lengthy 8559 WUs Folding right now, but should be able to fire up a Bigadv tomorrow.  My main interest was in getting the CPU usage as high as I could get it before launching a Bigadv.  I think it may be about as good as it will get, but will try the NUMA enable just to see if it helps.

I think enabling NUMA has helped.  I just shut down, enabled it and then restarted that WU that I was processing.  I have picked up another 3% now with an average of 95% CPU Usage in Task Manager. 
 
Yes, I'll be launching a Bigadv tomorrow early and (fingers crossed) I don't start off with an 8101.  Things are in much better shape for me thanks to your help/advice today!    I'm winding down here and will see ya at the big March Madness rally tomorrow. 

I did try that last night after enabling NUMA in the BIOS.  What I found was that I could start off with CPUs 0-3 selected in the Affinity screen and all other CPUs not selected.  Going back over to the CPU Usage screen, the selected (4) CPUs were running pretty solid lines right at or near 100%.  I then added/selected (4) more CPUs at a time until I got to the last (4).  Up until adding the last (4) CPUs, all the other CPUs were holding pretty well at or near 100% with just a slight amount of occasional ripple in the graph lines.  Adding the last (4) CPUs put me right back to where I was with all CPUs running at ~95% with noticeable ripple in the graphs.  I experimented with this further and found that this actually happens when I add/select the last (2) CPUs to the Affinity selection. 
 
So, the bottom-line was that I could have most of the CPUs showing at ~100% with (2) CPUs reading ~0% or have all CPUs at 95%.  I just went ahead and selected/enabled all CPUs to the Virtualbox.exe Process because I really couldn't see the benefit of excluding the last (2) CPUs.
 
From what I can tell, Affinity is simply the ability to select individual CPUs/Threads and assign them to a given Process for load balancing/distribution.  I think the reason why we are seeing the increase in CPU Usage levels when we deselect some of the CPUs is because the other CPUs (that are assigned to the Process) are picking up more work.  Sort of like squeezing the work across less CPUs increases the CPU Usage levels across the CPUs that are assigned to a given Process.  This is especially visible with an application like Folding that puts a large load on all available processors.
 
That's my 2-cents on this thing.

Thanks NerdGZ...always appreciate your help and insight too! 
 
Back to the less than 100% CPU Usage thing using Virtualbox to Fold, I think the real questions for me are:

1. Is Virtualbox holding back/preventing FAH from being able to access every ounce of CPU processing in Windows?
2. Or, is this some issue with Windows Task Manager reporting things differently and VBox is not preventing anything?

I recall when we used to be able to run Bigadvs from native Windows and the CPU Usage was always flat-lined at 100% whenever Folding.  With Virtualbox running and managing Folding from within a Linux Virtual OS, Windows 7 still thinks I have 5% headroom on every CPU.  5% waste across all those processors would add up to increased WU TPFs in my mind.  In testing standard SMP WUs, I've been able to see that TPF difference when Folding in native Windows vs within Virtualbox .  On a standard SMP WU, that difference of ~7 seconds isn't really a big deal.  With a Bigadv that spans days of processing time, it would have a more significant impact. 
 
So, I tend to think that Windows is calling it the way it is and that Virtualbox (for some reason) is not able to take advantage of full CPU processing power in Windows 7 (when Folding).  I'm no expert either, just guessing based on what I've seen so far.

NerdGZ, here's a screen capture of the Top Process window.  I thought a visual would be better than my trying to interpret for you.  I see that theKraken process says it is using 2,398% which seemed odd to me.  I would have thought 100% would be the maximum amount of CPU usage that could be shown for a given Process.  But this is Linux and my expertise lacks...