[MITgcm-devel] Very large memory machine available with evaluation compilers

[Constantinos Evangelinos]

Announcing a new machine with 16GB of memory that is installed and running in 
the lab:

It's called batsi.mit.edu (it is not pronounced the same way as "kisses" in 
Italian as the stress in batsi is on the "i" and not the "a" :-)

It has dual AMD Opteron 240 (1.4GHz) processors with 1MB of L2 cache each, 
16GB of 266MHz DDR SDRAM split equally among the processors and directly 
addressable in 64-bit mode from either one. 

The machine is dual-boot: Linux (Fedora Core 3) and Solaris 10. For evaluation 
purposes it is expected to rotate O/S from time to time. 

The Solaris side of things is expected to be ready in a few days, along with a 
2-month evaluation installation of Sun Studio 10 for Solaris 10/x86 which 
includes all the optimizing compilers and other software development, 
debugging and tuning tools and libraries you'd expect to find on Solaris for 
Sparc. There is a possibility that large memory parallel (shared memory 
OpenMP) jobs might do better under Solaris/Studio 10 than under Linux.

The Linux side of things is already installed and should function fine. Please 
report any problems to admin at ocean.mit.edu and myself (regarding development 
tools, libraries etc.) in case we've missed something.

On the Linux side we have:
a) The Intel 8.1 compilers icc, icpc and ifort and debugger idb
b) The Portland Group 5.2 compilers pgcc,pgCC,pgf77,pgf90 with a 15-day 
evaluation license so you need to hurry if you plan to test them.
c) The Pathscale compilers (which apparently offer the best performance on 
this system) with a 30-day evaluation license. If you are interested in 
testing these compilers you need to e-mail me so I can make arrangements to 
add you to the license file (should take at most a day).
d) The system GNU compilers versions 3.4.2 and 3.3.4
e) Sun Java 1.4.2_06

To compile code that uses really large arrays with (b) and (d) you would need 
to add the "-mcmodel=medium" flag to your compile lines for all your objects.
The Portland Group compiler would also require "-Mlarge_arrays" for the case 
of really large arrays.

BLAS (and part of LAPACK) libraries installed:
a) GOTO BLAS (32 and 64 bit, serial and parallel) in /opt/goto
b) ATLAS BLAS and part of LAPACK (3 different versions for 32bit, 64bit and 
parallel 64bit) in /opt/atlas
c) ACML BLAS and large parts of LAPACK and ScaLAPACK as well as FFT in 
i) /opt/acml2.5.0 for the GNU and Portland Group compilers
ii) /opt/acml2.5.1 for the Pathscale compilers
iii) builtin with the evaluation version of the Portland Group compilers
d) Netlib BLAS and LAPACK in /usr/lib and /usr/lib64 for 32 and 64-bit 
environments respectively

NetCDF 3.5.1 is installed in /usr/lib64.

A 2-month evaluation version of Sun Studio 10 (run as "sunstudio") for Linux 
(with a very nice suite of debugging and performance tuning tools) and 
NetBeans 3.5V (run as "runide.sh")  are installed in /opt/sun.

To take advantage of the two processors on the system:
i) Compilers (a),(b) and (c) have support for OpenMP ((c) only for the case of 
Fortran). 
ii) Compilers (a) and (b) also have autoparallelizing capabilities.
iii) MPI (LAM 7.0.6) and PVM 3.4.4 are installed on the system for use with 
the GNU compilers.

All of the above should be in one's path when logging in to the system.

This is an ideal machine for large memory postprocessing work, SVD or 
eigenvalue calculations etc. It lives on the Ocean network and has access to 
all your ocean network home directories as well as other data and scratch 
directories which should be useful as the onboard disk capacity is currently 
limited.

Happy testing and number crunching,

Constantinos
-- 
Dr. Constantinos Evangelinos
Department of Earth, Atmospheric and Planetary Sciences
Massachusetts Institute of Technology