VMware && Linux bigadv folding

Page: << < ..1617181920.. > >> Showing page 17 of 32
Author
linuxrouter
Omnipotent Enthusiast
  • Total Posts : 8043
  • Reward points : 0
  • Joined: 2008/02/28 14:47:45
  • Status: offline
  • Ribbons : 102
Re:VMWare 3.0 && bigadv folding (Image available for testing) 2010/01/24 10:52:21 (permalink)
leftygof

upgraded to the newest 07 build LR....all seems great (as usual)

One question: with 8 VMs running should I number them on the network to fah1..2..and so on? (using dhcp)

I am going to switch all to LRs stuff (6-smp and 2 -bigadv)
 


You can number then like that. That is what I do also. I have mine numbered fah0 - 3. It makes it easier to identify which system is which on the network. You can specify the hostname for each one in the web page.

CaseLabs M-S8 - ASRock X99 Pro - Intel 5960x - XSPC WC - EVGA 980 Ti Hybrid - Samsung 950 512GB - EVGA 1600w Titanium
Affiliate Code: OZJ-0TQ-41NJ
linuxrouter
Omnipotent Enthusiast
  • Total Posts : 8043
  • Reward points : 0
  • Joined: 2008/02/28 14:47:45
  • Status: offline
  • Ribbons : 102
Re:VMWare 3.0 && bigadv folding (Image available for testing) 2010/01/24 13:36:12 (permalink)
geokilla
They're added to the VMX file that you open in notepad. It's the file that you double click to open up your image.



What I can do is setup several extra VMX files to cover each processor configuration: dual, quad, and 8-core.

CaseLabs M-S8 - ASRock X99 Pro - Intel 5960x - XSPC WC - EVGA 980 Ti Hybrid - Samsung 950 512GB - EVGA 1600w Titanium
Affiliate Code: OZJ-0TQ-41NJ
linuxrouter
Omnipotent Enthusiast
  • Total Posts : 8043
  • Reward points : 0
  • Joined: 2008/02/28 14:47:45
  • Status: offline
  • Ribbons : 102
Re:VMWare 3.0 && bigadv folding (Image available for testing) 2010/01/24 14:31:39 (permalink)
I found an issue with the VMX file that limits some of the memory available to the VM when you set more than 3600MB. If you are setting more than 3600MB, I would recommend changing the following in the Linux64_FAH.vmx file: 

virtualHW.version = "6" 

to 

virtualHW.version = "7" 

This will make it so the full memory allocation is available to Linux.

CaseLabs M-S8 - ASRock X99 Pro - Intel 5960x - XSPC WC - EVGA 980 Ti Hybrid - Samsung 950 512GB - EVGA 1600w Titanium
Affiliate Code: OZJ-0TQ-41NJ
Foldmiester
ACX Member
  • Total Posts : 321
  • Reward points : 0
  • Joined: 2009/11/20 21:31:29
  • Location: Northern Minnesota
  • Status: offline
  • Ribbons : 0
Re:VMWare 3.0 && bigadv folding (Image available for testing) 2010/01/24 15:11:04 (permalink)
linuxrouter

It looks like Stanford released a new client version 6.29 today:

http://foldingforum.org/viewtopic.php?f=24&t=12997


It has a bug fix and support for the new SMP2 cores.

I plan to add it to the image. However, if you want to install into your existing image, these are the commands to do that:

wget http://www.stanford.edu/group/pandegroup/folding/release/FAH6.29-Linux.tgz
mkdir old
cp fah6 mpiexec old
tar xzf FAH6.29-Linux.tgz

I fired it up on one of my bigadv rigs and seems to be running okay so far.


LR....When do you think you will have 6.29 added to the fah image?

 
Be careful what you wish for, you might just get it all...and then some you don't want.

Dual quad E5335 Xeon's @ 2.4/ GTX 260/216 (x2) 
leftygof
FTW Member
  • Total Posts : 1755
  • Reward points : 0
  • Joined: 2007/11/08 17:16:59
  • Location: Amarillo, TX
  • Status: offline
  • Ribbons : 5
Re:VMWare 3.0 && bigadv folding (Image available for testing) 2010/01/24 15:14:41 (permalink)
Development  
 
Linux FAH Image - Dev (Last Updated 01-24-10) - Includes folding client 6.29


leftygof
FTW Member
  • Total Posts : 1755
  • Reward points : 0
  • Joined: 2007/11/08 17:16:59
  • Location: Amarillo, TX
  • Status: offline
  • Ribbons : 5
Re:VMWare 3.0 && bigadv folding (Image available for testing) 2010/01/24 15:19:40 (permalink)
linuxrouter

I found an issue with the VMX file that limits some of the memory available to the VM when you set more than 3600MB. If you are setting more than 3600MB, I would recommend changing the following in the Linux64_FAH.vmx file: 

virtualHW.version = "6" 

to 

virtualHW.version = "7" 

This will make it so the full memory allocation is available to Linux.


with a shutdown now?
edit the vmx file and restart?

I start shaking and crying when I mess with a unit in midstream :)


Foldmiester
ACX Member
  • Total Posts : 321
  • Reward points : 0
  • Joined: 2009/11/20 21:31:29
  • Location: Northern Minnesota
  • Status: offline
  • Ribbons : 0
Re:VMWare 3.0 && bigadv folding (Image available for testing) 2010/01/24 15:19:47 (permalink)
leftygof

Development  

Linux FAH Image - Dev (Last Updated 01-24-10) - Includes folding client 6.29

Thanks for the link Lefty...I am downloading now.

 
Be careful what you wish for, you might just get it all...and then some you don't want.

Dual quad E5335 Xeon's @ 2.4/ GTX 260/216 (x2) 
linuxrouter
Omnipotent Enthusiast
  • Total Posts : 8043
  • Reward points : 0
  • Joined: 2008/02/28 14:47:45
  • Status: offline
  • Ribbons : 102
Re:VMWare 3.0 && bigadv folding (Image available for testing) 2010/01/24 15:28:26 (permalink)
leftygof
with a shutdown now? 
edit the vmx file and restart?

I start shaking and crying when I mess with a unit in midstream :)



I would wait until you finish up the work units first just to be safe. After finishing up the work units, you can run shutdown -h now, edit the VMX and then start up again.

CaseLabs M-S8 - ASRock X99 Pro - Intel 5960x - XSPC WC - EVGA 980 Ti Hybrid - Samsung 950 512GB - EVGA 1600w Titanium
Affiliate Code: OZJ-0TQ-41NJ
leftygof
FTW Member
  • Total Posts : 1755
  • Reward points : 0
  • Joined: 2007/11/08 17:16:59
  • Location: Amarillo, TX
  • Status: offline
  • Ribbons : 5
Re:VMWare 3.0 && bigadv folding (Image available for testing) 2010/01/24 15:29:50 (permalink)
Foldmiester

leftygof

Development  

Linux FAH Image - Dev (Last Updated 01-24-10) - Includes folding client 6.29

Thanks for the link Lefty...I am downloading now.


no problem...that version has been rock solid for me.....don't forget to change the hardware version in the vmx file as mentioned above

I need LR to reassure me how to do this in mid-unit without "screwing the pooch"


leftygof
FTW Member
  • Total Posts : 1755
  • Reward points : 0
  • Joined: 2007/11/08 17:16:59
  • Location: Amarillo, TX
  • Status: offline
  • Ribbons : 5
Re:VMWare 3.0 && bigadv folding (Image available for testing) 2010/01/24 15:31:18 (permalink)
linuxrouter

leftygof
with a shutdown now? 
edit the vmx file and restart?

I start shaking and crying when I mess with a unit in midstream :)



I would wait until you finish up the work units first just to be safe. After finishing up the work units, you can run shutdown -h now, edit the VMX and then start up again.


thanks dude......new dev build is running great seems more consistent in it's ppd...not as many peaks and valleys


linuxrouter
Omnipotent Enthusiast
  • Total Posts : 8043
  • Reward points : 0
  • Joined: 2008/02/28 14:47:45
  • Status: offline
  • Ribbons : 102
Re:VMWare 3.0 && bigadv folding (Image available for testing) 2010/01/24 19:59:54 (permalink)
leftygof
thanks dude......new dev build is running great seems more consistent in it's ppd...not as many peaks and valleys 



Thanks. Glad to hear it is working out well for you.

CaseLabs M-S8 - ASRock X99 Pro - Intel 5960x - XSPC WC - EVGA 980 Ti Hybrid - Samsung 950 512GB - EVGA 1600w Titanium
Affiliate Code: OZJ-0TQ-41NJ
linuxrouter
Omnipotent Enthusiast
  • Total Posts : 8043
  • Reward points : 0
  • Joined: 2008/02/28 14:47:45
  • Status: offline
  • Ribbons : 102
Re:VMWare 3.0 && bigadv folding (Image available for testing) 2010/01/25 18:53:47 (permalink)
I put up a new version that includes the new 6.29 client. There are a few other updates from 1.0:

- Updated the version info in the VMX which makes it so the full memory allocation is available when more than 3600MB is set. I also set the default memory allocation to 4600MB since VMware only uses what it needs for memory and this might help with bigadv stability.
- Switched to JFS for the file system for less delay between work units and in hopes of minimizing work unit failures from shutting down the client before finishing the work unit. JFS has been working great on my VM's so far. I have not lost a work unit since switching.
- Added options to the web interface for setting the workgroup and selecting the default kernel (Intel or AMD).
- Added support for as few as 2-cores via the web interface to cover dual-cores. SMP2 supports running 2-threads or more. 
- Updated the console and web info pages including adding HFM.NET setup instructions to the console info that appears on bootup.

If you downloaded the dev version from the last couple days, then you probably already have most if not all of these new updates. Let me know if you have any problems.

post edited by linuxrouter - 2010/01/25 19:03:58

CaseLabs M-S8 - ASRock X99 Pro - Intel 5960x - XSPC WC - EVGA 980 Ti Hybrid - Samsung 950 512GB - EVGA 1600w Titanium
Affiliate Code: OZJ-0TQ-41NJ
Alex2190
SSC Member
  • Total Posts : 504
  • Reward points : 0
  • Joined: 2009/05/19 23:21:39
  • Location: Maryland, USA
  • Status: offline
  • Ribbons : 0
Re:VMWare 3.0 && bigadv folding (Image available for testing) 2010/01/25 22:55:20 (permalink)
just started attempting bigadv on windows SMP, getting a calc of about 14k PPD...sound good? its eating my processor lol

EVGA X58 SLI (758) Rev 1.1
EVGA GTX 480 SC+
Intel Core i7 920 C0 3.8GHz ~ 1.33v
Intel X25-M 80GB G2
WD 500GB
12 GB RAM 1600MHz
Corsair HX750W  
Antec 300 Case
ajr23
FTW Member
  • Total Posts : 1315
  • Reward points : 0
  • Joined: 2009/04/04 11:34:18
  • Location: Issaquah, Wa
  • Status: offline
  • Ribbons : 4
Re:VMWare 3.0 && bigadv folding (Image available for testing) 2010/01/25 23:11:54 (permalink)
Alex2190

just started attempting bigadv on windows SMP, getting a calc of about 14k PPD...sound good? its eating my processor lol


what is your time per frame?
TheCrazyCanuck
CLASSIFIED Member
  • Total Posts : 2191
  • Reward points : 0
  • Joined: 2006/04/16 12:54:44
  • Location: Texas Yee-Haw!
  • Status: offline
  • Ribbons : 4
Re:VMWare 3.0 && bigadv folding (Image available for testing) 2010/01/25 23:20:02 (permalink)
I max out at 12k PPD running at 4GHz so ya that sounds really good.  The pointers per day is nothing compared to the bonus you get though.

Main rig:  4GHz i7-5960x, 3xTitan X, 32GB RAM, RAID 0 SSDs
1080p bedroom TV rig:  3.5GHz i7-4770k, 16GB RAM, couple SSDs
4k TV rig:  i7-4770, 16GB RAM, SSD
Laptop:  i7-4900MQ, 680m, 3xSSD RAID 0, connected to 4k TV (I don't mess around)


 
Alex2190
SSC Member
  • Total Posts : 504
  • Reward points : 0
  • Joined: 2009/05/19 23:21:39
  • Location: Maryland, USA
  • Status: offline
  • Ribbons : 0
Re:VMWare 3.0 && bigadv folding (Image available for testing) 2010/01/25 23:47:54 (permalink)
3 min 30 sec

EVGA X58 SLI (758) Rev 1.1
EVGA GTX 480 SC+
Intel Core i7 920 C0 3.8GHz ~ 1.33v
Intel X25-M 80GB G2
WD 500GB
12 GB RAM 1600MHz
Corsair HX750W  
Antec 300 Case
linuxrouter
Omnipotent Enthusiast
  • Total Posts : 8043
  • Reward points : 0
  • Joined: 2008/02/28 14:47:45
  • Status: offline
  • Ribbons : 102
Re:VMWare 3.0 && bigadv folding (Image available for testing) 2010/01/25 23:54:45 (permalink)
Alex2190

just started attempting bigadv on windows SMP, getting a calc of about 14k PPD...sound good? its eating my processor lol



Sounds good for PPD. However, I do not think bigadv has been ported to SMP2 yet. I just tried the -bigadv flag with Windows SMP and got a project 6014, which is the same project I got without the -bigadv flag on another system.

CaseLabs M-S8 - ASRock X99 Pro - Intel 5960x - XSPC WC - EVGA 980 Ti Hybrid - Samsung 950 512GB - EVGA 1600w Titanium
Affiliate Code: OZJ-0TQ-41NJ
TheCrazyCanuck
CLASSIFIED Member
  • Total Posts : 2191
  • Reward points : 0
  • Joined: 2006/04/16 12:54:44
  • Location: Texas Yee-Haw!
  • Status: offline
  • Ribbons : 4
Re:VMWare 3.0 && bigadv folding (Image available for testing) 2010/01/26 18:38:05 (permalink)
Alex2190

3 min 30 sec


With that time then it cannot be a bigadv work unit (normally it's 28+ minutes on an i7)

Main rig:  4GHz i7-5960x, 3xTitan X, 32GB RAM, RAID 0 SSDs
1080p bedroom TV rig:  3.5GHz i7-4770k, 16GB RAM, couple SSDs
4k TV rig:  i7-4770, 16GB RAM, SSD
Laptop:  i7-4900MQ, 680m, 3xSSD RAID 0, connected to 4k TV (I don't mess around)


 
kougar
FTW Member
  • Total Posts : 1003
  • Reward points : 0
  • Joined: 2006/05/08 10:11:19
  • Location: Texas
  • Status: offline
  • Ribbons : 3
Re:VMWare 3.0 && bigadv folding (Image available for testing) 2010/01/29 09:23:06 (permalink)
Thanks for the continual updates Linuxrouter!   Just updated my clients and everything's folding along just fine.


linuxrouter
Omnipotent Enthusiast
  • Total Posts : 8043
  • Reward points : 0
  • Joined: 2008/02/28 14:47:45
  • Status: offline
  • Ribbons : 102
Re:VMWare 3.0 && bigadv folding (Image available for testing) 2010/01/29 10:38:54 (permalink)
kougar

Thanks for the continual updates Linuxrouter!   Just updated my clients and everything's folding along just fine.



Thanks for the feedback. Glad to hear the updated image is working out for you.

CaseLabs M-S8 - ASRock X99 Pro - Intel 5960x - XSPC WC - EVGA 980 Ti Hybrid - Samsung 950 512GB - EVGA 1600w Titanium
Affiliate Code: OZJ-0TQ-41NJ
TheWolf
CLASSIFIED Member
  • Total Posts : 3841
  • Reward points : 0
  • Joined: 2007/11/14 16:05:23
  • Location: Pascagoula, Ms
  • Status: offline
  • Ribbons : 9
Re:VMWare 3.0 && bigadv folding (Image available for testing) 2010/01/29 17:05:29 (permalink)
linuxrouter

Alex2190

just started attempting bigadv on windows SMP, getting a calc of about 14k PPD...sound good? its eating my processor lol



Sounds good for PPD. However, I do not think bigadv has been ported to SMP2 yet. I just tried the -bigadv flag with Windows SMP and got a project 6014, which is the same project I got without the -bigadv flag on another system.


Thought I post that I just got my 1st -bigadv WU using the Dev Image from post #484, were I was getting only A3 WU's.
But I just added the -bigadv flag before the last WU was done, so I'm not sure if I was able to the big WU's before now
or if Dev image was able to all along?

EVGA Affiliate Code ZHKWRJB9D4 My HeatWare 
 
linuxrouter
Omnipotent Enthusiast
  • Total Posts : 8043
  • Reward points : 0
  • Joined: 2008/02/28 14:47:45
  • Status: offline
  • Ribbons : 102
Re:VMWare 3.0 && bigadv folding (Image available for testing) 2010/01/29 18:09:40 (permalink)
TheWolf
Thought I post that I just got my 1st -bigadv WU using the Dev Image from post #484, were I was getting only A3 WU's. 
But I just added the -bigadv flag before the last WU was done, so I'm not sure if I was able to the big WU's before now
or if Dev image was able to all along?



If you add -bigadv, this flag should override -advmethods. With the -bigadv flag the client will attempt to download the 25K work units that run via Core a2 currently. If no bigadv work is available, it should revert back to Core a3. If no Core a3 work is available, then the client should try and get Core a2 work. -advmethods enables the possibility of getting Core a3 work. If you leave out both -advmethods and -bigadv, then the client will try and get Core a2 1920-pt work units.
post edited by linuxrouter - 2010/01/29 18:10:46

CaseLabs M-S8 - ASRock X99 Pro - Intel 5960x - XSPC WC - EVGA 980 Ti Hybrid - Samsung 950 512GB - EVGA 1600w Titanium
Affiliate Code: OZJ-0TQ-41NJ
touche
CLASSIFIED Member
  • Total Posts : 4390
  • Reward points : 0
  • Joined: 2008/02/09 05:35:16
  • Status: offline
  • Ribbons : 11
Re:VMWare 3.0 && bigadv folding (Image available for testing) 2010/01/30 06:34:07 (permalink)
Hey guys!

I got VMWare folding set up with BigAdv, but FahMon tells me I'm only getting 1277PPD out of it.  So far, I've finished 2 WUs with it over a span of 12 hours, and the current WU looks like it is worth 470 points... and its 7 hours in.

I've got it set up on 7 cores, affinity set to 0-6, and GPU folding on 7.


This isn't normal, is it?

EVGA X58 SLI
Intel Core i7 920 (OC, 3.8GHz)
12GB RAM (3x 2GB Corsair, 3x 2GB Patriot) 1600MHz
EVGA 760GTX 2GB
Corsair 850W
Antec 900
Full Metal Folder
FTW Member
  • Total Posts : 1232
  • Reward points : 0
  • Joined: 2008/07/14 16:31:55
  • Status: offline
  • Ribbons : 4
Re:VMWare 3.0 && bigadv folding (Image available for testing) 2010/01/30 07:56:09 (permalink)
touche

Hey guys!

I got VMWare folding set up with BigAdv, but FahMon tells me I'm only getting 1277PPD out of it.  So far, I've finished 2 WUs with it over a span of 12 hours, and the current WU looks like it is worth 470 points... and its 7 hours in.

I've got it set up on 7 cores, affinity set to 0-6, and GPU folding on 7.


This isn't normal, is it?

I have the same problem, even the a1 1920 point wu's are only using 30% of my CPU.
It takes to long to finish any smp WU's PPD is down like 60% using -smp 8  no bigadv.
 
Any ideas?
thanks
post edited by mech581 - 2010/01/30 07:57:58
touche
CLASSIFIED Member
  • Total Posts : 4390
  • Reward points : 0
  • Joined: 2008/02/09 05:35:16
  • Status: offline
  • Ribbons : 11
Re:VMWare 3.0 && bigadv folding (Image available for testing) 2010/01/30 08:05:33 (permalink)
Well, I can see on the Task Manager that I have 7 cores at 100% - it's just that I'm getting very little points for them.

EVGA X58 SLI
Intel Core i7 920 (OC, 3.8GHz)
12GB RAM (3x 2GB Corsair, 3x 2GB Patriot) 1600MHz
EVGA 760GTX 2GB
Corsair 850W
Antec 900
TheWolf
CLASSIFIED Member
  • Total Posts : 3841
  • Reward points : 0
  • Joined: 2007/11/14 16:05:23
  • Location: Pascagoula, Ms
  • Status: offline
  • Ribbons : 9
Re:VMWare 3.0 && bigadv folding (Image available for testing) 2010/01/30 08:33:04 (permalink)
touche

Well, I can see on the Task Manager that I have 7 cores at 100% - it's just that I'm getting very little points for them.


Those are not Bigadv WU's, there the core A3 WU, they work faster, show less PPD in Fahmon, but there is a bonus.
You can pick up HFM monitoring program here: http://code.google.com/p/hfm-net/ it will show the bonus points.

EVGA Affiliate Code ZHKWRJB9D4 My HeatWare 
 
TheWolf
CLASSIFIED Member
  • Total Posts : 3841
  • Reward points : 0
  • Joined: 2007/11/14 16:05:23
  • Location: Pascagoula, Ms
  • Status: offline
  • Ribbons : 9
Re:VMWare 3.0 && bigadv folding (Image available for testing) 2010/01/30 08:38:53 (permalink)
mech581

touche

Hey guys!

I got VMWare folding set up with BigAdv, but FahMon tells me I'm only getting 1277PPD out of it.  So far, I've finished 2 WUs with it over a span of 12 hours, and the current WU looks like it is worth 470 points... and its 7 hours in.

I've got it set up on 7 cores, affinity set to 0-6, and GPU folding on 7.


This isn't normal, is it?

I have the same problem, even the a1 1920 point wu's are only using 30% of my CPU.
It takes to long to finish any smp WU's PPD is down like 60% using -smp 8  no bigadv.
 
Any ideas?
thanks

I haven't researched it, but looks like Stanford is handing our some WU's that are using like a 2 CPU core load.
Weather there bad WU's or just a flaw in the programing I don't know. I've seen quite a few of these over the past weeks.
Adding the -forceasm flag seemed to help a little, making it put a slite more load across all cores.

EVGA Affiliate Code ZHKWRJB9D4 My HeatWare 
 
Full Metal Folder
FTW Member
  • Total Posts : 1232
  • Reward points : 0
  • Joined: 2008/07/14 16:31:55
  • Status: offline
  • Ribbons : 4
Re:VMWare 3.0 && bigadv folding (Image available for testing) 2010/01/30 08:52:18 (permalink)
TheWolf

mech581

touche

Hey guys!

I got VMWare folding set up with BigAdv, but FahMon tells me I'm only getting 1277PPD out of it.  So far, I've finished 2 WUs with it over a span of 12 hours, and the current WU looks like it is worth 470 points... and its 7 hours in.

I've got it set up on 7 cores, affinity set to 0-6, and GPU folding on 7.


This isn't normal, is it?

I have the same problem, even the a1 1920 point wu's are only using 30% of my CPU.
It takes to long to finish any smp WU's PPD is down like 60% using -smp 8  no bigadv.
 
Any ideas?
thanks

I haven't researched it, but looks like Stanford is handing our some WU's that are using like a 2 CPU core load.
Weather there bad WU's or just a flaw in the programing I don't know. I've seen quite a few of these over the past weeks.
Adding the -forceasm flag seemed to help a little, making it put a slite more load across all cores.

 
Thanks for the reply
I'll give it shot.
hotwired
ACX Member
  • Total Posts : 342
  • Reward points : 0
  • Joined: 2010/01/30 08:48:49
  • Status: offline
  • Ribbons : 0
Re:VMWare 3.0 && bigadv folding (Image available for testing) 2010/01/30 09:11:37 (permalink)
linuxrouter thanks .  rocking along at 28 min frames

920 @ 20x200 @1.3
fpo/batch#3938B024  
kingston @ 2000  
cosair hx 1000 ps  
gtx480 875/2001  
         


 
linuxrouter
Omnipotent Enthusiast
  • Total Posts : 8043
  • Reward points : 0
  • Joined: 2008/02/28 14:47:45
  • Status: offline
  • Ribbons : 102
Re:VMWare 3.0 && bigadv folding (Image available for testing) 2010/01/30 11:21:57 (permalink)
touche

Hey guys!

I got VMWare folding set up with BigAdv, but FahMon tells me I'm only getting 1277PPD out of it.  So far, I've finished 2 WUs with it over a span of 12 hours, and the current WU looks like it is worth 470 points... and its 7 hours in.

I've got it set up on 7 cores, affinity set to 0-6, and GPU folding on 7.


This isn't normal, is it?
 


Those are the new Core a3 work units. If you have a passkey setup and have completed 10-work units by the preferred deadline, then you should qualify for bonus points. The base point value is low on these work units, but the bonus points can be fairly high depending on processor and smp configuration. HFM will report work unit and PPD values including both the base and bonus points.


Do you know what TPF you are getting and what project # you are running?



CaseLabs M-S8 - ASRock X99 Pro - Intel 5960x - XSPC WC - EVGA 980 Ti Hybrid - Samsung 950 512GB - EVGA 1600w Titanium
Affiliate Code: OZJ-0TQ-41NJ
Page: << < ..1617181920.. > >> Showing page 17 of 32
Jump to:
  • Back to Mobile