[MITgcm-support] IO problems with large numbers of CPUs

[Dr. Christopher L. Wolfe]

Hi All,

I'm trying to get the MITgcm to run on 1024 CPUs on a Cray XT4  
(franklin.nersc.gov). Per-tile IO works fine, but produces an utterly  
unmanageable number of files. However, if I turn on SingleCpuIo or  
GlobalFiles *and* the diagnostics package, my jobs die with an out of  
memory error. SingleCpuIo or GlobalFiles works fine without the  
diagnostics package and the diagnostics package works fine without  
SingleCpuIo or GlobalFiles. Changing these options shouldn't have that  
much of an impact on memory usage, since nearly all MITgcm's memory is  
statically allocated. I figure that SingleCpuIo might be overflowing  
my MPI message buffers, but I can't see why GlobalFiles would if each  
process writes to the global file independently. If anyone has any  
idea what's going on here, I'd love to hear your input.

Thanks,
Christopher

-----------------------------------------------------------
Dr. Christopher L. Wolfe              	   858-534-4560
Physical Oceanography Research Division    OAR 357
Scripps Institution of Oceanography, UCSD  clwolfe at ucsd.edu
-----------------------------------------------------------