[MITgcm-support] Minimal computational requirements to run MITgcm

Wed Jul 20 20:59:23 EDT 2011

Hi Luis

Just to give a slightly different angle,
and perhaps complement what Ryan said.

We ran the MITgcm here on several generations of Beowulf class clusters.
Beowulf clusters are Linux servers with standard off the shelf PC 
hardware, and an interconnect for MPI message passing, which can be 
Gigabit Ethernet, Infiniband, Myrinet, etc.
In general they also have an additional Gigabit Ethernet network for 
control,login, and perhaps I/O.

We never did coupled experiments with MITgcm, only ocean,
but I suppose the hardware we have would support coupled integrations.
We do run other coupled climate models here, though.

The older cluster (10yrs) has 32 compute nodes with two single core AMD 
Athlon processors, and 512MB RAM per core/processor. The interconnect is
Myrinet-2000, now obsolete. (Myrinet has faster products these days.) 
An astonishing 1.5TB RAID array on a separate file server provides 
storage over 100Mb/s Ethernet and NFS.
(It was huge when we bought it ...)
This cluster supported several low/mid resolution experiments
(4-1 degree, global) running on 4-32 nodes.

The new clusters (2.5 and < 1yr old, respectively)
have Infinband interconnect
(the older is DDR/20Gb/s, the newer QDR/40Gb/s) for MPI communication.
They have respectively 34 and 16 compute nodes with dual
AMD Opteron processors (quad-core, and eight-core respectively)
and the RAM is 2GB/core and 4GB/core each.
(Note RAM *per core* not per node or per processor.)
It is hard to buy less RAM-per-core these days, and you
probably don't need more than that to run MITgcm integrations either.
These clusters have 8TB and 20TB RAID storage nodes, respectively.
You can run NFS over Infiniband or Gigabit Ethernet for I/O,
or a fancier parallel file system.
Each of these clusters fits on a single rack tower,
and uses a single Infiniband and a single Gigabit Ethernet switch, 
making them relatively cheap, manageable,
and requiring relatively modest power,
ups and air-conditioning infrastructure.

These two clusters can run, say, a few to several 1 degree
resolution experiments simultaneously, for instance,
and presumably could also run coupled experiments.
However, for eddy-resolving experiments,
I guess we would need more hardware than this,
either a multiple-tower cluster with more complex/expensive
interconnect network, requiring more power/ac/ups than
our computer room can currently support,
or to use a public supercomputer.

The software stack is mostly open source (Linux, Gnu, gcc, g++, 
gfortran, MPICH2, OpenMPI, Torque/Maui queuing system, etc).
The exceptions are commercial Fortran compilers (Intel and PGI),
and Matlab (essential for MITgcm data analysis, although you could
get away with the free Octave).

Of course, some people here run (small) MITgcm integrations
on their laptops, and as Ryan said,
mobile phones may be getting there too.

I hope this helps,
Gus Correa

luis.aimola at vale.com wrote:
> 
> Ryan:
> 
> I´m very grateful for your answer.
> You gave a very good perspective to begin my work and make plans for 
> future experiments.
> Thank you very much.
> All the best for you.
> 
> 
> Luis Aímola, PhD
> Pesquisador Associado/Associate Researcher
> Grupo de Mudanças Climáticas/Climate Change Group
> 
> Instituto Tecnológico Vale - Desenvolvimento Sustentável
> Vale Technological Institute - Sustainable Development
> 
> Travessa Dr. Moraes, nº 78, 3º andar (Nazaré)
> 66035-080 Belém, Pará, Brasil
> Telefone: +5591-32152405 Fax: +5591-32152492
> 
> 
> 
> *Ryan Abernathey <rpa at MIT.EDU>*
> 
> 20/07/2011 19:16
> Favor responder a
> <mitgcm-support at mitgcm.org>
> 
> 
> 	
> Para
> 	<mitgcm-support at mitgcm.org>
> cc
> 	
> Assunto
> 	Re: [MITgcm-support] Minimal computational requirements to run       
>  MITgcm
> 
> 
> 	
> 
> 
> 
> 
> 
> Luis,
> 
> I don't know if anyone has gotten back to you about this... Here is a 
> response from my limited knowledge.
> 
> MITgcm is a very flexible model. It can run on almost any hardware. Some 
> users run it on the most powerful supercomputers in the world using 
> thousands of nodes, and generating terabytes of data. On the other side, 
> I would not be surprised if someone has it running on their mobile 
> phone. You can find a de-facto list of the supported platforms in the 
> tools/build_options/ folder.
> 
> There are no "minimal requirements" in general. The requirements are 
> determined by the scale of your problem. The speed of execution will 
> depend on your hardware. For long, complex, high-resolution simulations, 
> the only efficient way to run the model is in a parallel environment 
> such as on a cluster using MPI.
> 
> A good way to get started is to compile some of the basic tutorials on 
> your personal computer to get a feel for how it works and how fast it runs.
> 
> Hope this helps,
> 
> Ryan
> 
> 
> 
> On Jul 18, 2011, at 4:27 PM, _luis.aimola at vale.com_ 
> <mailto:luis.aimola at vale.com> wrote:
> 
> 
> Dears colleagues:
> 
> I´m a newcomer to this mailing list.
> I´m a researcher in the Climate Change Laboratory in the Vale 
> Technological Institute, Belém, Brazil.
> We want to install and run the MITgcm model here to make many 
> experiments in several time and space scales using the complete coupled 
> model (Oceam-Ice-Atmosphere-Land).
> 
> We look in the manual about the minimal computer requirements to run 
> this model, but don´t got any detailed information.
> (Processors or nodes, RAM memory, hard disk capacity, etc).
> Which are the minimal requirements for a low, middle and high running 
> performance for that?
> 
> So, if someone can indicate me where I find this information, or could 
> give it to us, I will be very grateful.
> Regards.
> 
> 
> Luis Aímola, PhD
> Pesquisador Associado/Associate Researcher
> Grupo de Mudanças Climáticas/Climate Change Group
> 
> Instituto Tecnológico Vale - Desenvolvimento Sustentável
> Vale Technological Institute - Sustainable Development
> 
> Travessa Dr. Moraes, nº 78, 3º andar (Nazaré)
> 66035-080 Belém, Pará, Brasil
> Telefone: +5591-32152405 Fax: +5591-32152492
> AVISO LEGAL "As informações existentes nesta mensagem e nos arquivos 
> anexados são para uso restrito. A utilização, divulgação, cópia ou 
> distribuição dessa mensagem por qualquer pessoa diferente do 
> destinatário é proibida. Se essa mensagem foi recebida por engano, favor 
> excluí-la e informar ao remetente pelo endereço eletrônico 
> acima.******************************************************************" 
> DISCLAIMER "This email and its attachments may contain privileged and/or 
> confidential information. Use, disclosure, copying or distribution of 
> this message by anyone other than the intended recipient is strictly 
> prohibited. If you have received this email in error, please notify the 
> sender by reply email and destroy all copies of this message." 
>  _______________________________________________
> MITgcm-support mailing list_
> __MITgcm-support at mitgcm.org_ <mailto:MITgcm-support at mitgcm.org>
> http://mitgcm.org/mailman/listinfo/mitgcm-support
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-support
> 
> AVISO LEGAL "As informações existentes nesta mensagem e nos arquivos 
> anexados são para uso restrito. A utilização, divulgação, cópia ou 
> distribuição dessa mensagem por qualquer pessoa diferente do 
> destinatário é proibida. Se essa mensagem foi recebida por engano, favor 
> excluí-la e informar ao remetente pelo endereço eletrônico acima." 
> *******************************************************************DISCLAIMER 
> "This email and its attachments may contain privileged and/or 
> confidential information. Use, disclosure, copying or distribution of 
> this message by anyone other than the intended recipient is strictly 
> prohibited. If you have received this email in error, please notify the 
> sender by reply email and destroy all copies of this message." 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-support