[MITgcm-support] Minimal computational requirements to run MITgcm

Wed Jul 20 21:19:19 EDT 2011

Dear Gus:

Very nice your technical descriptions and come to compliment very much the 
Ray informations.
I gonna analyse carefully your observations and decide what to do in my 
short, middle and long term research.
I will leave you know about our progress.
We keep in touch.
Thank you very much.
Regards,

Luis Aímola, PhD
Pesquisador Associado/Associate Researcher
Grupo de Mudanças Climáticas/Climate Change Group
Instituto Tecnológico Vale - Desenvolvimento Sustentável
Vale Technological Institute - Sustainable Development

Travessa Dr. Moraes, nº 78, 3º andar (Nazaré)
66035-080 Belém, Pará, Brasil
Telefone: +5591-32152405 Fax: +5591-32152492

Gus Correa <gus at ldeo.columbia.edu> 
20/07/2011 21:59
Favor responder a
<mitgcm-support at mitgcm.org>

Para
MITgcm Support <mitgcm-support at mitgcm.org>
cc

Assunto
Re: [MITgcm-support] Minimal computational requirements to      run MITgcm

Hi Luis

Just to give a slightly different angle,
and perhaps complement what Ryan said.

We ran the MITgcm here on several generations of Beowulf class clusters.
Beowulf clusters are Linux servers with standard off the shelf PC 
hardware, and an interconnect for MPI message passing, which can be 
Gigabit Ethernet, Infiniband, Myrinet, etc.
In general they also have an additional Gigabit Ethernet network for 
control,login, and perhaps I/O.

We never did coupled experiments with MITgcm, only ocean,
but I suppose the hardware we have would support coupled integrations.
We do run other coupled climate models here, though.

The older cluster (10yrs) has 32 compute nodes with two single core AMD 
Athlon processors, and 512MB RAM per core/processor. The interconnect is
Myrinet-2000, now obsolete. (Myrinet has faster products these days.) 
An astonishing 1.5TB RAID array on a separate file server provides 
storage over 100Mb/s Ethernet and NFS.
(It was huge when we bought it ...)
This cluster supported several low/mid resolution experiments
(4-1 degree, global) running on 4-32 nodes.

The new clusters (2.5 and < 1yr old, respectively)
have Infinband interconnect
(the older is DDR/20Gb/s, the newer QDR/40Gb/s) for MPI communication.
They have respectively 34 and 16 compute nodes with dual
AMD Opteron processors (quad-core, and eight-core respectively)
and the RAM is 2GB/core and 4GB/core each.
(Note RAM *per core* not per node or per processor.)
It is hard to buy less RAM-per-core these days, and you
probably don't need more than that to run MITgcm integrations either.
These clusters have 8TB and 20TB RAID storage nodes, respectively.
You can run NFS over Infiniband or Gigabit Ethernet for I/O,
or a fancier parallel file system.
Each of these clusters fits on a single rack tower,
and uses a single Infiniband and a single Gigabit Ethernet switch, 
making them relatively cheap, manageable,
and requiring relatively modest power,
ups and air-conditioning infrastructure.

These two clusters can run, say, a few to several 1 degree
resolution experiments simultaneously, for instance,
and presumably could also run coupled experiments.
However, for eddy-resolving experiments,
I guess we would need more hardware than this,
either a multiple-tower cluster with more complex/expensive
interconnect network, requiring more power/ac/ups than
our computer room can currently support,
or to use a public supercomputer.

The software stack is mostly open source (Linux, Gnu, gcc, g++, 
gfortran, MPICH2, OpenMPI, Torque/Maui queuing system, etc).
The exceptions are commercial Fortran compilers (Intel and PGI),
and Matlab (essential for MITgcm data analysis, although you could
get away with the free Octave).

Of course, some people here run (small) MITgcm integrations
on their laptops, and as Ryan said,
mobile phones may be getting there too.

I hope this helps,
Gus Correa

luis.aimola at vale.com wrote:
> 
> Ryan:
> 
> I´m very grateful for your answer.
> You gave a very good perspective to begin my work and make plans for 
> future experiments.
> Thank you very much.
> All the best for you.
> 
> 
> Luis Aímola, PhD
> Pesquisador Associado/Associate Researcher
> Grupo de Mudanças Climáticas/Climate Change Group
> 
> Instituto Tecnológico Vale - Desenvolvimento Sustentável
> Vale Technological Institute - Sustainable Development
> 
> Travessa Dr. Moraes, nº 78, 3º andar (Nazaré)
> 66035-080 Belém, Pará, Brasil
> Telefone: +5591-32152405 Fax: +5591-32152492
> 
> 
> 
> *Ryan Abernathey <rpa at MIT.EDU>*
> 
> 20/07/2011 19:16
> Favor responder a
> <mitgcm-support at mitgcm.org>
> 
> 
> 
> Para
>                <mitgcm-support at mitgcm.org>
> cc
> 
> Assunto
>                Re: [MITgcm-support] Minimal computational requirements 
to run 
>  MITgcm
> 
> 
> 
> 
> 
> 
> 
> 
> Luis,
> 
> I don't know if anyone has gotten back to you about this... Here is a 
> response from my limited knowledge.
> 
> MITgcm is a very flexible model. It can run on almost any hardware. Some 

> users run it on the most powerful supercomputers in the world using 
> thousands of nodes, and generating terabytes of data. On the other side, 

> I would not be surprised if someone has it running on their mobile 
> phone. You can find a de-facto list of the supported platforms in the 
> tools/build_options/ folder.
> 
> There are no "minimal requirements" in general. The requirements are 
> determined by the scale of your problem. The speed of execution will 
> depend on your hardware. For long, complex, high-resolution simulations, 

> the only efficient way to run the model is in a parallel environment 
> such as on a cluster using MPI.
> 
> A good way to get started is to compile some of the basic tutorials on 
> your personal computer to get a feel for how it works and how fast it 
runs.
> 
> Hope this helps,
> 
> Ryan
> 
> 
> 
> On Jul 18, 2011, at 4:27 PM, _luis.aimola at vale.com_ 
> <mailto:luis.aimola at vale.com> wrote:
> 
> 
> Dears colleagues:
> 
> I´m a newcomer to this mailing list.
> I´m a researcher in the Climate Change Laboratory in the Vale 
> Technological Institute, Belém, Brazil.
> We want to install and run the MITgcm model here to make many 
> experiments in several time and space scales using the complete coupled 
> model (Oceam-Ice-Atmosphere-Land).
> 
> We look in the manual about the minimal computer requirements to run 
> this model, but don´t got any detailed information.
> (Processors or nodes, RAM memory, hard disk capacity, etc).
> Which are the minimal requirements for a low, middle and high running 
> performance for that?
> 
> So, if someone can indicate me where I find this information, or could 
> give it to us, I will be very grateful.
> Regards.
> 
> 
> Luis Aímola, PhD
> Pesquisador Associado/Associate Researcher
> Grupo de Mudanças Climáticas/Climate Change Group
> 
> Instituto Tecnológico Vale - Desenvolvimento Sustentável
> Vale Technological Institute - Sustainable Development
> 
> Travessa Dr. Moraes, nº 78, 3º andar (Nazaré)
> 66035-080 Belém, Pará, Brasil
> Telefone: +5591-32152405 Fax: +5591-32152492
> AVISO LEGAL "As informações existentes nesta mensagem e nos arquivos 
> anexados são para uso restrito. A utilização, divulgação, cópia ou 
> distribuição dessa mensagem por qualquer pessoa diferente do 
> destinatário é proibida. Se essa mensagem foi recebida por engano, favor 

> excluí-la e informar ao remetente pelo endereço eletrônico 
> 
acima.******************************************************************" 
> DISCLAIMER "This email and its attachments may contain privileged and/or 

> confidential information. Use, disclosure, copying or distribution of 
> this message by anyone other than the intended recipient is strictly 
> prohibited. If you have received this email in error, please notify the 
> sender by reply email and destroy all copies of this message." 
>  _______________________________________________
> MITgcm-support mailing list_
> __MITgcm-support at mitgcm.org_ <mailto:MITgcm-support at mitgcm.org>
> http://mitgcm.org/mailman/listinfo/mitgcm-support
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-support
> 
> AVISO LEGAL "As informações existentes nesta mensagem e nos arquivos 
> anexados são para uso restrito. A utilização, divulgação, cópia ou 
> distribuição dessa mensagem por qualquer pessoa diferente do 
> destinatário é proibida. Se essa mensagem foi recebida por engano, favor 

> excluí-la e informar ao remetente pelo endereço eletrônico acima." 
> 
*******************************************************************DISCLAIMER 

> "This email and its attachments may contain privileged and/or 
> confidential information. Use, disclosure, copying or distribution of 
> this message by anyone other than the intended recipient is strictly 
> prohibited. If you have received this email in error, please notify the 
> sender by reply email and destroy all copies of this message." 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-support

_______________________________________________
MITgcm-support mailing list
MITgcm-support at mitgcm.org
http://mitgcm.org/mailman/listinfo/mitgcm-support

AVISO LEGAL "As informações existentes nesta mensagem e nos arquivos anexados são para uso restrito. A utilização, divulgação, cópia ou distribuição dessa mensagem por qualquer pessoa diferente do destinatário é proibida. Se essa mensagem foi recebida por engano, favor excluí-la e informar ao remetente pelo endereço eletrônico acima.******************************************************************" DISCLAIMER "This email and its attachments may contain privileged and/or confidential information. Use, disclosure, copying or distribution of this message by anyone other than the intended recipient is strictly prohibited. If you have received this email in error, please notify the sender by reply email and destroy all copies of this message."  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mitgcm.org/pipermail/mitgcm-support/attachments/20110720/d491680c/attachment-0001.htm>