[MITgcm-support] multicore scaling

[Constantinos Evangelinos]

On Mon 14 Apr 2008, Michael A. Spall wrote:

> I have been told that the MITgcm does not scale well with
> multicore processors.  I am thinking about say 16 processors on
> 2 or 4 nodes (2 processors per node, dual or quad processors) with
> infiniband.  Does anyone have experience with this?

So far in our testing at least dual core scales OKish - ie. on both Intel and 
AMD systems we're scaling great with 2 cores out of 4 running user code but 
see less than 3 speedup going to all 4 cores. I have not tried to test 
running on 3 out of 4 cores on a node yet but I doubt it would be greatly 
worth it because the problem is with insufficient memory bandwidth for memory 
streaming at the socket level for AMD architectures and at the FSB per socket 
level for the Intel architectures.

Given that, quad cores are unlikely to be worth it for Intel systems. AMD 
Barcelona might be a somewhat better option according to published memory 
performance numbers but I have not run on it yet - Chris has. However I don't 
expect to see more than a speedup of 3 within each quad core processor or 6 
in a dual socket node.

On the other hand you'll soon probably be out of options as all processors are 
going to be quad core or more and dual core "HPC" systems may not catch on 
because of lack of volume.

Constantinos
-- 
Dr. Constantinos Evangelinos
Department of Earth, Atmospheric and Planetary Sciences
Massachusetts Institute of Technology