256 core server courtesy of IBM

Please confine these topics to things that would be of general interest to those who are interested in FAH which don't fall into any other category.

Moderator: Site Moderators

Michael_McCord,_M.D.
Posts: 43
Joined: Tue Dec 04, 2007 3:40 am
Location: Beaumont, TX

256 core server courtesy of IBM

Post by Michael_McCord,_M.D. »

I would love to fire up the SMP client with the flag -smp 256 hahahaha:

http://www.maximumpc.com/article/home/i ... nix_server
Proud to still be on Page One World!
Team MPC- in relentless pursuit of the cure.
P5-133XL
Posts: 2948
Joined: Sun Dec 02, 2007 4:36 am
Hardware configuration: Machine #1:

Intel Q9450; 2x2GB=8GB Ram; Gigabyte GA-X48-DS4 Motherboard; PC Power and Cooling Q750 PS; 2x GTX 460; Windows Server 2008 X64 (SP1).

Machine #2:

Intel Q6600; 2x2GB=4GB Ram; Gigabyte GA-X48-DS4 Motherboard; PC Power and Cooling Q750 PS; 2x GTX 460 video card; Windows 7 X64.

Machine 3:

Dell Dimension 8400, 3.2GHz P4 4x512GB Ram, Video card GTX 460, Windows 7 X32

I am currently folding just on the 5x GTX 460's for aprox. 70K PPD
Location: Salem. OR USA

Re: 256 core server courtesy of IBM

Post by P5-133XL »

It will be a cluster of cores, and folding does not yet support clusters. It's also Power-7 which is also a problem.

I'd just be happy with an octo MB with 12 core mangy-cours CPU's i.e. -smp 96
Image
Zagen30
Posts: 823
Joined: Tue Mar 25, 2008 12:45 am
Hardware configuration: Core i7 3770K @3.5 GHz (not folding), 8 GB DDR3 @2133 MHz, 2xGTX 780 @1215 MHz, Windows 7 Pro 64-bit running 7.3.6 w/ 1xSMP, 2xGPU

4P E5-4650 @3.1 GHz, 64 GB DDR3 @1333MHz, Ubuntu Desktop 13.10 64-bit

Re: 256 core server courtesy of IBM

Post by Zagen30 »

P5-133XL wrote:I'd just be happy with an octo MB with 12 core mangy-cours CPU's i.e. -smp 96
*Floods house with drool*

Sometimes I wish I'd win lotto so I could build crazy stuff like that. Course, I'd have to actually play lotto to win it, but that's beside the point.
Image
codysluder
Posts: 1024
Joined: Sun Dec 02, 2007 12:43 pm

Re: 256 core server courtesy of IBM

Post by codysluder »

I wonder if there's a way to adapt the PS3 client which uses the cell to this architecture.
AZBrandon
Posts: 42
Joined: Sat Jan 17, 2009 1:43 am

Re: 256 core server courtesy of IBM

Post by AZBrandon »

Unless I'm terribly mis-understanding the Supermicro pricing, you can build a 4-way 48-core Magny-cours box for maybe $15,000 and run linux on it, do maybe 180,000 ppd in a nice little 1 or 2U rackmount system. These IBM megaservers tend to be up in the half to full million dollar range. At that cost you could just fill up an entire server rack with cheap linux 4-way AMD boxes instead and still probably spend less money. Assuming a 42U rack, and dedicating 6U to networking and console servers, you could do 18 2U AMD servers. 18 * 4 * 48 = 3456 cores in a rack. If each box does 180k ppd, that's over 3 million ppd for maybe $300,000 plus whatever you pay for a dedicated rack, power, and internet at a colocation datacenter. Now THAT would be your ultimate lotto folder!
Nathan_P
Posts: 1164
Joined: Wed Apr 01, 2009 9:22 pm
Hardware configuration: Asus Z8NA D6C, 2 [email protected] Ghz, , 12gb Ram, GTX 980ti, AX650 PSU, win 10 (daily use)

Asus Z87 WS, Xeon E3-1230L v3, 8gb ram, KFA GTX 1080, EVGA 750ti , AX760 PSU, Mint 18.2 OS

Not currently folding
Asus Z9PE- D8 WS, 2 [email protected] Ghz, 16Gb 1.35v Ram, Ubuntu (Fold only)
Asus Z9PA, 2 Ivy 12 core, 16gb Ram, H folding appliance (fold only)
Location: Jersey, Channel islands

Re: 256 core server courtesy of IBM

Post by Nathan_P »

AZBrandon wrote:Unless I'm terribly mis-understanding the Supermicro pricing, you can build a 4-way 48-core Magny-cours box for maybe $15,000 and run linux on it, do maybe 180,000 ppd in a nice little 1 or 2U rackmount system. These IBM megaservers tend to be up in the half to full million dollar range. At that cost you could just fill up an entire server rack with cheap linux 4-way AMD boxes instead and still probably spend less money. Assuming a 42U rack, and dedicating 6U to networking and console servers, you could do 18 2U AMD servers. 18 * 4 * 48 = 3456 cores in a rack. If each box does 180k ppd, that's over 3 million ppd for maybe $300,000 plus whatever you pay for a dedicated rack, power, and internet at a colocation datacenter. Now THAT would be your ultimate lotto folder!
You could go for the older istanbuls and do each server for $10000 and get the same PPD
Image
ParrLeyne
Posts: 40
Joined: Sun Dec 14, 2008 10:59 pm
Location: Toronto, Canada

Re: 256 core server courtesy of IBM

Post by ParrLeyne »

AZBrandon wrote:Unless I'm terribly mis-understanding the Supermicro pricing, you can build a 4-way 48-core Magny-cours box for maybe $15,000
Actually, I figure you can build a 4 x AMD 6176 SE (48 Core @ 2.3 Ghz) box for $10,000 USD.
AZBrandon wrote:Assuming a 42U rack, and dedicating 6U to networking and console servers, you could do 18 2U AMD servers. 18 * 4 * 48 = 3456 cores in a rack.
6U for networking and console server is a little much, Gbe switches and IP KVM modules can be mounted back to back in a rack (the switches/modules aren't that deep).
  • 1 rack with 20 servers, only needs 1U for networking/console servers.
  • 2 racks with 41 servers, only needs 2U for networking/console servers.
  • 3 racks with 62 servers, only needs 2U for networking/console servers.
  • 4 racks with 82 servers, only needs 3U for networking/console servers.
ParrLeyne
Posts: 40
Joined: Sun Dec 14, 2008 10:59 pm
Location: Toronto, Canada

Re: 256 core server courtesy of IBM

Post by ParrLeyne »

Nathan_P wrote:You could go for the older istanbuls and do each server for $10000 and get the same PPD
:?: :!:
I can't believe that the older CPUs, running at the same speed, would run as fast as the new ones -- ie. get same PPD. :!:
HaloJones
Posts: 906
Joined: Thu Jul 24, 2008 10:16 am

Re: 256 core server courtesy of IBM

Post by HaloJones »

You can't actually put that much into a single rack without a serious amount of floor strength, some major power capabilities and industrial cooling. And don't ask about the sound proofing around it because I won't be able to hear you.
single 1070

Image
AZBrandon
Posts: 42
Joined: Sat Jan 17, 2009 1:43 am

Re: 256 core server courtesy of IBM

Post by AZBrandon »

ParrLeyne wrote:
Nathan_P wrote:You could go for the older istanbuls and do each server for $10000 and get the same PPD
:?: :!:
I can't believe that the older CPUs, running at the same speed, would run as fast as the new ones -- ie. get same PPD. :!:
Well you could run the old 6-core units in an 8-way configuration, for a total of 48 cores, plus they were up to 2.6 or 2.8ghz. Now with the 12-core chips you're limited to 2.3ghz and 4-way, but they have more memory bandwidth and a little more efficient architecture. The energy usage should be a good deal lower with the Magny-Cours route, which would let you stack more servers for any given power/cooling budget. As a bonus, they can be replaced with 16-core Bulldozer chips next year when they are released. Ah, daydreams about what you could do with an excess of funds.
theteofscuba
Posts: 96
Joined: Wed Dec 05, 2007 7:15 am
Hardware configuration: PS3, Phenom II X4, QX9775, HD 8570
Contact:

Re: 256 core server courtesy of IBM

Post by theteofscuba »

these websites that benchmark hardware with various games and software should try to include some rough folding@home measurements in the future. I am a little disappointed to not see such a test when reading the magazines or their online articles.
Qinsp
Posts: 216
Joined: Sun Oct 17, 2010 2:34 pm

Re: 256 core server courtesy of IBM

Post by Qinsp »

Anyone figure out what that user Awachs is running?

Over 1.3 million PPD? Holy smokes!
Quality Inspection - Corona, CA, USA
Dimensional Inspection Laboratory
Pat McSwain, President
bruce
Posts: 20824
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: 256 core server courtesy of IBM

Post by bruce »

Qinsp wrote:Anyone figure out what that user Awachs is running?

Over 1.3 million PPD? Holy smokes!
From the official stats,
awachs
Active clients (within 50 days) 151
Active clients (within 7 days) 62
Qinsp
Posts: 216
Joined: Sun Oct 17, 2010 2:34 pm

Re: 256 core server courtesy of IBM

Post by Qinsp »

Since the average PPD is recorded in a week, that means each client is ~20K PPD. That can't be GPU, right? It can barely be SMP on normal desktops?

I'm thinking it's an array of multi-CPU servers each CPU running an instance?

Or somebody who collects i7 computers as a hobby.:D
Quality Inspection - Corona, CA, USA
Dimensional Inspection Laboratory
Pat McSwain, President
nibbs
Posts: 7
Joined: Fri Dec 28, 2007 12:38 pm
Hardware configuration: Q6600 @ 3.4 GHz; 2 GB DDR2-800; ATI Sapphire HD 4870 @ stock (not used for Folding atm)
Location: Germany, Berlin

Re: 256 core server courtesy of IBM

Post by nibbs »

An SMP client with -bigadv can earn that very easily thru bonuses.
Image
Post Reply