[MITgcm-support] domain size limit?
Oliver Jahn
jahn at MIT.EDU
Tue Dec 3 14:13:42 EST 2013
Try -mcmodel=medium (and no -fPIC)?
On 2013-12-03 13:46, Ryan Abernathey wrote:
> Ok, so now I am really confused. In order to eliminate the possibility
> of a compiler mismatch, I upgraded to the latest set of gcc / gfortran
> (4.8) from the mac hpc website:
> http://hpc.sourceforge.net/
> I stuck them all in a special directory (/usr/local/gcc-4.8)
> $ file /usr/local/gcc-4.8/bin/gcc
> /usr/local/gcc-4.8/bin/gcc: Mach-O 64-bit executable x86_64
> $ file /usr/local/gcc-4.8/bin/gfortran
> /usr/local/gcc-4.8/bin/gfortran: Mach-O 64-bit executable x86_64
> I am no longer using the gcc that comes with Xcode.
>
> I put this in my optfile
> BASEDIR="/usr/local/gcc-4.8"
> CC="$BASEDIR/bin/gcc"
> FC="$BASEDIR/bin/gfortran"
> F90C=$FC
> LINK=$FC
> CPP="$BASEDIR/bin/cpp -traditional -P"
> # and libaries / includes
> INCLUDEDIR="$BASEDIR/include"
> INCLUDES="-I$BASEDIR/include"
> LIBDIR="$BASEDIR/lib"
> LIBS="-L$BASEDIR/lib"
> FFLAGS="$FFLAGS -arch x86_64"
> CFLAGS="-arch x86_64"
>
> I am *still* getting the same error about:
> ld: 32-bit RIP relative reference out of range (2149180116 max is
> +/-4GB): from _find_alpha_ (0x1001ADD60) to _rholoc.3148 (0x18034E880)
> in '_find_alpha_' from find_alpha.o for architecture x86_64
>
> Full optfile is attached. It is a modified version of the
> darwin_amd64_gfortran.
>
> -Ryan
>
>
>
>
>
>
> On Tue, Dec 3, 2013 at 1:39 PM, Gus Correa <gus at ldeo.columbia.edu
> <mailto:gus at ldeo.columbia.edu>> wrote:
>
> Hi Ryan
>
> Sorry, I meant:
>
> $ file /usr/bin/gcc
>
> or
>
> $ file `which gcc`
>
> This should tell if it is 32-bit or 64-bit
>
> Cheers,
> Gus
>
> On 12/03/2013 01:25 PM, Ryan Abernathey wrote:
>
> $ which gcc
> /usr/bin/gcc
>
>
> On Tue, Dec 3, 2013 at 1:21 PM, Gus Correa
> <gus at ldeo.columbia.edu <mailto:gus at ldeo.columbia.edu>
> <mailto:gus at ldeo.columbia.edu <mailto:gus at ldeo.columbia.edu>>__>
> wrote:
>
> Hi Ryan
>
> Because of the output of 'gcc -v' that you sent.
>
> It says:
> > $ gcc -v
> > Using built-in specs.
> > Target: i686-apple-darwin11\
> ...
> > --program-prefix=i686-apple-______darwin11-
>
> --host=x86_64-apple-darwin11
> > --target=i686-apple-darwin11
>
> All the "i686" refer to 32-bit Pentium-type (and later)
> processors.
> Normally 64 bit shows as "x86_64" instead.
>
> If you want to make sure, please send the output of
>
> file `which gcc`
>
> [please note the backquotes]
> That would be the best check.
>
> This will tell what type of executable your default gcc
> 4.2.1 is.
>
> Cheers,
> Gus
>
>
> On 12/03/2013 01:11 PM, Ryan Abernathey wrote:
>
> Gus,
>
> How do you know that my gcc is not 64 bit? This gcc
> 4.2.1 is the
> latest
> version available through Apple and, according to this
> documentation,
> should be able to compile 64 bit no problem:
>
> https://developer.apple.com/____library/mac/documentation/____Darwin/Conceptual/____64bitPorting/building/____building.html
> <https://developer.apple.com/__library/mac/documentation/__Darwin/Conceptual/__64bitPorting/building/__building.html>
>
>
> <https://developer.apple.com/__library/mac/documentation/__Darwin/Conceptual/__64bitPorting/building/__building.html
> <https://developer.apple.com/library/mac/documentation/Darwin/Conceptual/64bitPorting/building/building.html>>
>
> I don't have gcc 4.7.1 installed.
>
> -Ryan
>
>
> On Tue, Dec 3, 2013 at 12:59 PM, Gus Correa
> <gus at ldeo.columbia.edu <mailto:gus at ldeo.columbia.edu>
> <mailto:gus at ldeo.columbia.edu <mailto:gus at ldeo.columbia.edu>>
> <mailto:gus at ldeo.columbia.edu
> <mailto:gus at ldeo.columbia.edu> <mailto:gus at ldeo.columbia.edu
> <mailto:gus at ldeo.columbia.edu>>__>__>
>
> wrote:
>
> Hi Ryan
>
> Your gfortran is 64-bit and seems to have been
> built with
> gcc 4.7.1.
> However, that is not the gcc 4.2.1, which is 32-bit,
> and which seems to take precedence in your PATH.
>
> I wonder if you can locate your gcc 4.7.1, and
> modify your
> PATH to
> point to it (Would this break other Mac functionality?
> Maybe ...),
> and/or point to gcc 4.7.1 full path in the MITgcm
> build_options file.
> Hopefully this will take care of the link error.
>
> I hope this helps,
> Gus Correa
>
>
> On 12/03/2013 09:13 AM, Ryan Abernathey wrote:
>
> Thanks guys for the quick response.
>
> The full error is
>
> Creating mitgcmuv ...
> gfortran -o mitgcmuv -fsecond-underscore
> -fconvert=big-endian
> -Wunused
> -Wuninitialized -O3 -funroll-loops chksum_tiled.o
> [...all the
> objects...] timer_stats.o -L/usr/local/lib
> final section layout:
> __TEXT/__text addr=0x100001620,
> size=0x0029268D,
> fileOffset=0x00001620, type=1
> __TEXT/__text_startup addr=0x100293CB0,
> size=0x00000029,
> fileOffset=0x00293CB0, type=1
> __TEXT/__stubs addr=0x100293CDA,
> size=0x0000013E,
> fileOffset=0x00293CDA, type=27
> __TEXT/__stub_helper addr=0x100293E18,
> size=0x00000222,
> fileOffset=0x00293E18, type=31
> __TEXT/__cstring addr=0x100294040,
> size=0x00002840,
> fileOffset=0x00294040, type=12
> __TEXT/__const addr=0x100296880,
> size=0x00018440,
> fileOffset=0x00296880, type=0
> __TEXT/__eh_frame addr=0x1002AECC0,
> size=0x00018330,
> fileOffset=0x002AECC0, type=18
> __DATA/__program_vars addr=0x1002C7000,
> size=0x00000028,
> fileOffset=0x002C7000, type=29
> __DATA/__got addr=0x1002C7028,
> size=0x00000198,
> fileOffset=0x002C7028, type=28
> __DATA/__nl_symbol_ptr addr=0x1002C71C0,
> size=0x00000010,
> fileOffset=0x002C71C0, type=28
> __DATA/__la_symbol_ptr addr=0x1002C71D0,
> size=0x000001A8,
> fileOffset=0x002C71D0, type=26
> __DATA/__data addr=0x1002C7380,
> size=0x00000078,
> fileOffset=0x002C7380, type=0
> __DATA/__bss addr=0x1002C73F8,
> size=0x00000010,
> fileOffset=0x00000000, type=24
> __DATA/__bss4 addr=0x1002C7410,
> size=0x00000024,
> fileOffset=0x00000000, type=24
> __DATA/__bss5 addr=0x1002C7440,
> size=0x00007238,
> fileOffset=0x00000000, type=24
> __DATA/__bss3 addr=0x1002CE678,
> size=0x00000018,
> fileOffset=0x00000000, type=24
> __DATA/__common addr=0x1002CE6A0,
> size=0x00152F90,
> fileOffset=0x00000000, type=24
> __DATA/__huge addr=0x100421640,
> size=0x15D3D3B88,
> fileOffset=0x00000000, type=24
> ld: 32-bit RIP relative reference out of range
> (2162237024 <tel:%282162237024> <tel:%282162237024>
> <tel:%282162237024> max is
> +/-4GB): from _find_beta__ (0x100227200) to
> _rhoden.4514
> (0x18103AE80)
> in '_find_beta__' from find_alpha.o for
> architecture x86_64
> collect2: error: ld returned 1 exit status
> make[1]: *** [mitgcmuv] Error 1
> make: *** [fwd_exe_target] Error 2
>
> My detailed compiler versions.
>
> $ gcc -v
> Using built-in specs.
> Target: i686-apple-darwin11
> Configured with:
>
>
> /private/var/tmp/llvmgcc42/______llvmgcc42-2336.11~182/src/______configure
> --disable-checking --enable-werror
>
>
> --prefix=/Applications/Xcode.______app/Contents/Developer/usr/______llvm-gcc-4.2
> --mandir=/share/man
> --enable-languages=c,objc,c++,______obj-c++
> --program-prefix=llvm-
>
> --program-transform-name=/^[______cg][^.-]*$/s/$/-4.2/
> --with-slibdir=/usr/lib --build=i686-apple-darwin11
>
>
> --enable-llvm=/private/var/______tmp/llvmgcc42/llvmgcc42-2336.______11~182/dst-llvmCore/____Developer/__usr/local
> --program-prefix=i686-apple-______darwin11-
> --host=x86_64-apple-darwin11
> --target=i686-apple-darwin11
> --with-gxx-include-dir=/usr/______include/c++/4.2.1
>
>
> Thread model: posix
> gcc version 4.2.1 (Based on Apple Inc. build 5658)
> (LLVM build
> 2336.11.00)
> localhost:build rpa$
>
> $ gfortran -v
> Using built-in specs.
> COLLECT_GCC=gfortran
>
>
> COLLECT_LTO_WRAPPER=/usr/______local/libexec/gcc/x86_64-______apple-darwin11.4.0/4.7.1/lto-______wrapper
>
>
> Target: x86_64-apple-darwin11.4.0
> Configured with: ../gcc-4.7.1/configure
> --enable-languages=fortran
> Thread model: posix
> gcc version 4.7.1 (GCC)
>
>
> On Tue, Dec 3, 2013 at 8:53 AM, fancer lancer
> <fancer.lancer at gmail.com
> <mailto:fancer.lancer at gmail.com>
> <mailto:fancer.lancer at gmail.__com <mailto:fancer.lancer at gmail.com>>
> <mailto:fancer.lancer at gmail.
> <mailto:fancer.lancer at gmail.>____com
> <mailto:fancer.lancer at gmail.__com <mailto:fancer.lancer at gmail.com>>>
> <mailto:fancer.lancer at gmail
> <mailto:fancer.lancer at gmail>. <mailto:fancer.lancer at gmail
> <mailto:fancer.lancer at gmail>.>______com
>
>
> <mailto:fancer.lancer at gmail.
> <mailto:fancer.lancer at gmail.>____com
> <mailto:fancer.lancer at gmail.__com
> <mailto:fancer.lancer at gmail.com>>>>> wrote:
>
> Martin,
> Is it the only error produced by the
> compiler/linker or
> there are
> some additional warnings or messages? If
> so could you,
> please, post
> them here? It can be useful.
>
> Sincerely,
> -Sergey
>
>
> On Tue, Dec 3, 2013 at 4:50 PM, Martin Losch
> <Martin.Losch at awi.de <mailto:Martin.Losch at awi.de>
> <mailto:Martin.Losch at awi.de <mailto:Martin.Losch at awi.de>>
> <mailto:Martin.Losch at awi.de <mailto:Martin.Losch at awi.de>
> <mailto:Martin.Losch at awi.de <mailto:Martin.Losch at awi.de>>>
> <mailto:Martin.Losch at awi.de <mailto:Martin.Losch at awi.de>
> <mailto:Martin.Losch at awi.de <mailto:Martin.Losch at awi.de>>
> <mailto:Martin.Losch at awi.de <mailto:Martin.Losch at awi.de>
> <mailto:Martin.Losch at awi.de <mailto:Martin.Losch at awi.de>>>>__>
> wrote:
>
> Hi Sergey,
>
> The compiler flags are the ones that
> you can
> find in
>
> tools/build_options/darwin_______amd64_gfortran,
>
> so no
>
> 32bit related
> parameters (-arch i386 is commented out).
>
> I saw that thread earlier today, but I
> cannot
> extract
> solution
> for the MITgcm. I have no idea about
> C, and as
> far as I
> know/can
> see, including headers is always the first
> stuff in all
> files.
>
> Martin
>
> On Dec 3, 2013, at 1:38 PM, fancer lancer
> <fancer.lancer at gmail.com
> <mailto:fancer.lancer at gmail.com>
> <mailto:fancer.lancer at gmail.__com <mailto:fancer.lancer at gmail.com>>
> <mailto:fancer.lancer at gmail.
> <mailto:fancer.lancer at gmail.>____com
> <mailto:fancer.lancer at gmail.__com <mailto:fancer.lancer at gmail.com>>>
> <mailto:fancer.lancer at gmail
> <mailto:fancer.lancer at gmail>. <mailto:fancer.lancer at gmail
> <mailto:fancer.lancer at gmail>.>______com
>
>
> <mailto:fancer.lancer at gmail.
> <mailto:fancer.lancer at gmail.>____com
> <mailto:fancer.lancer at gmail.__com
> <mailto:fancer.lancer at gmail.com>>>>> wrote:
>
> > Hi Martin,
> > Ok, you use x86_64 version of compiler. And I suppose you
> don't specify any additional
> parameters to the
> compiler
> like '-m32'.
> > Alas I don't have an Apple computer nearby to check the
> actual cause of the problem. But I
> guess the
> clue to
> solve the
> issue may be found in the thread:
> >
>
> http://lists.apple.com/______archives/xcode-users/2012/Jun/______msg00118.html
> <http://lists.apple.com/____archives/xcode-users/2012/Jun/____msg00118.html>
>
> <http://lists.apple.com/____archives/xcode-users/2012/Jun/____msg00118.html
> <http://lists.apple.com/__archives/xcode-users/2012/Jun/__msg00118.html>>
>
>
>
> <http://lists.apple.com/____archives/xcode-users/2012/Jun/____msg00118.html
> <http://lists.apple.com/__archives/xcode-users/2012/Jun/__msg00118.html>
>
> <http://lists.apple.com/__archives/xcode-users/2012/Jun/__msg00118.html
> <http://lists.apple.com/archives/xcode-users/2012/Jun/msg00118.html>>>
> > And additional explanation is here:
> >
>
> http://lists.apple.com/______archives/xcode-users/2012/Jun/______msg00126.html
> <http://lists.apple.com/____archives/xcode-users/2012/Jun/____msg00126.html>
>
> <http://lists.apple.com/____archives/xcode-users/2012/Jun/____msg00126.html
> <http://lists.apple.com/__archives/xcode-users/2012/Jun/__msg00126.html>>
>
>
>
> <http://lists.apple.com/____archives/xcode-users/2012/Jun/____msg00126.html
> <http://lists.apple.com/__archives/xcode-users/2012/Jun/__msg00126.html>
>
> <http://lists.apple.com/__archives/xcode-users/2012/Jun/__msg00126.html
> <http://lists.apple.com/archives/xcode-users/2012/Jun/msg00126.html>>>
> >
> > Sincerely,
> > -Sergey
> >
> >
> >
> >
> > On Tue, Dec 3, 2013 at 3:43 PM, Martin Losch
> <Martin.Losch at awi.de <mailto:Martin.Losch at awi.de>
> <mailto:Martin.Losch at awi.de <mailto:Martin.Losch at awi.de>>
> <mailto:Martin.Losch at awi.de <mailto:Martin.Losch at awi.de>
> <mailto:Martin.Losch at awi.de <mailto:Martin.Losch at awi.de>>>
> <mailto:Martin.Losch at awi.de <mailto:Martin.Losch at awi.de>
> <mailto:Martin.Losch at awi.de <mailto:Martin.Losch at awi.de>>
>
> <mailto:Martin.Losch at awi.de <mailto:Martin.Losch at awi.de>
> <mailto:Martin.Losch at awi.de <mailto:Martin.Losch at awi.de>>>>__>
> wrote:
> > Hi Sergey,
> >
> > I don’t know if Ryan’s compiler is the same (probably
> similar), but here’s mine:
> >
> > csysm15::build> gfortran -v
> > Using built-in specs.
> > COLLECT_GCC=gfortran
> >
>
>
>
> COLLECT_LTO_WRAPPER=/usr/______local/libexec/gcc/x86_64-______apple-darwin13.0.0/4.9.0/lto-______wrapper
>
>
> > Target: x86_64-apple-darwin13.0.0
> > Configured with: ../gcc-4.9-20130929/configure
> --enable-languages=c,c++,______fortran
>
>
> > Thread model: posix
> > gcc version 4.9.0 20130929 (experimental) (GCC)
> >
> > Martin
> >
> > On Dec 3, 2013, at 9:48 AM, fancer lancer
> <fancer.lancer at gmail.com
> <mailto:fancer.lancer at gmail.com>
> <mailto:fancer.lancer at gmail.__com <mailto:fancer.lancer at gmail.com>>
> <mailto:fancer.lancer at gmail.
> <mailto:fancer.lancer at gmail.>____com
> <mailto:fancer.lancer at gmail.__com <mailto:fancer.lancer at gmail.com>>>
> <mailto:fancer.lancer at gmail
> <mailto:fancer.lancer at gmail>. <mailto:fancer.lancer at gmail
> <mailto:fancer.lancer at gmail>.>______com
>
>
> <mailto:fancer.lancer at gmail.
> <mailto:fancer.lancer at gmail.>____com
> <mailto:fancer.lancer at gmail.__com
> <mailto:fancer.lancer at gmail.com>>>>> wrote:
> >
> > > Hi Ryan,
> > >
> > > Could you please, show the result of the command
> "gfortran
> -v". I suppose, that you used the 32-bit
> version of the
> compiler
> that is why you overcame the limit.
> > >
> > > Sincerely,
> > > -Sergey
> > >
> > >
> > > On Tue, Dec 3, 2013 at 12:29 PM, Martin Losch
> <Martin.Losch at awi.de <mailto:Martin.Losch at awi.de>
> <mailto:Martin.Losch at awi.de <mailto:Martin.Losch at awi.de>>
> <mailto:Martin.Losch at awi.de <mailto:Martin.Losch at awi.de>
> <mailto:Martin.Losch at awi.de <mailto:Martin.Losch at awi.de>>>
> <mailto:Martin.Losch at awi.de <mailto:Martin.Losch at awi.de>
> <mailto:Martin.Losch at awi.de <mailto:Martin.Losch at awi.de>>
>
> <mailto:Martin.Losch at awi.de <mailto:Martin.Losch at awi.de>
> <mailto:Martin.Losch at awi.de <mailto:Martin.Losch at awi.de>>>>__>
> wrote:
> > > Hi Ryan,
> > >
> > > I can only reproduce the problem with
> tutorial_barotropic_gyre (and
> gcc/gfortran 4.9
> on OSX
> 10.9).
> > >
> > > By how much do you have to repduce the domain size
> to make
> it work?
> > >
> > > The simplest solution that I found: instead of
> sNy=1600 and
> nSy = 1 use sNy=800 and nSy = 2. It’s
> a little
> bit of
> cheating,
> isn’t it?
> > >
> > > Martin
> > >
> > > On Dec 3, 2013, at 6:00 AM, Ryan Abernathey
> <ryan.abernathey at gmail.com
> <mailto:ryan.abernathey at gmail.com>
> <mailto:ryan.abernathey at gmail.__com
> <mailto:ryan.abernathey at gmail.com>>
> <mailto:ryan.abernathey at gmail.
> <mailto:ryan.abernathey at gmail.>____com
> <mailto:ryan.abernathey at gmail.__com
> <mailto:ryan.abernathey at gmail.com>>>
> <mailto:ryan.abernathey at gmail
> <mailto:ryan.abernathey at gmail>.
> <mailto:ryan.abernathey at gmail
> <mailto:ryan.abernathey at gmail>.__>____com
>
>
> <mailto:ryan.abernathey at gmail.
> <mailto:ryan.abernathey at gmail.>____com
> <mailto:ryan.abernathey at gmail.__com
> <mailto:ryan.abernathey at gmail.com>>>>>
>
> wrote:
> > >
> > > > Dear Super Modelers,
> > > >
> > > > I found a weird error when trying to compile a large
> domain (500 x 1600 x 1) on a single
> processor.
> On the
> final link
> stage of the build, I get:
> > > > ld: 32-bit RIP relative reference out of range
> (2162237024 <tel:%282162237024> <tel:%282162237024>
> <tel:%282162237024>
> <tel:%282162237024> max is
>
> +/-4GB): from
>
> _find_beta__ (0x100227200) to _rhoden.4514
> (0x18103AE80) in
> '_find_beta__' from find_alpha.o for architecture x86_64
> > > > The error disappears if I just reduce sNx * sNy.
> I guess
> this has something to do with running
> out of
> memory address
> space? Is it weird that "32-bit RIP"
> is being
> used even
> though
> my platform is 64 bit?
> > > >
> > > > My one weak attempt to overcome this was by
> adding -fPIC
> to my FFLAGS. No success.
> > > >
> > > > Is there any way to overcome this problem, or am I
> hitting some kind of fundamental limit
> to the
> domain size?
> > > >
> > > > Thanks for your suggestions,
> > > > Ryan
> > > >
> > > >
> > > > My platform is Mac OX 10.8.5, Intel Core i7, gcc
> version
> 4.2.1 (via xcode), gfortran GNU
> Fortran (GCC)
> 4.7.1. My
> optfile
> is the default darwin_amd64_gfortran.
> > > >
> > > > From my SIZE.h:
> > > > & sNx = 500,
> > > > & sNy = 1600,
> > > > & OLx = 4,
> > > > & OLy = 4,
> > > > & nSx = 1,
> > > > & nSy = 1,
> > > > & nPx = 1,
> > > > & nPy = 1,
> > > > & Nx = sNx*nSx*nPx,
> > > > & Ny = sNy*nSy*nPy,
> > > > & Nr = 1)
> > > >
> > > >
> > > >
> > > >
> > > > _____________________________________________________
>
>
> > > > MITgcm-support mailing list
> > > > MITgcm-support at mitgcm.org
> <mailto:MITgcm-support at mitgcm.org>
> <mailto:MITgcm-support at mitgcm.__org
> <mailto:MITgcm-support at mitgcm.org>>
> <mailto:MITgcm-support at mitgcm.
> <mailto:MITgcm-support at mitgcm.>____org
> <mailto:MITgcm-support at mitgcm.__org
> <mailto:MITgcm-support at mitgcm.org>>>
> <mailto:MITgcm-support at mitgcm
> <mailto:MITgcm-support at mitgcm>.
> <mailto:MITgcm-support at mitgcm
> <mailto:MITgcm-support at mitgcm>.__>____org
> <mailto:MITgcm-support at mitgcm.
> <mailto:MITgcm-support at mitgcm.>____org
> <mailto:MITgcm-support at mitgcm.__org
> <mailto:MITgcm-support at mitgcm.org>>>>
>
> > > >
> http://mitgcm.org/mailman/______listinfo/mitgcm-support
> <http://mitgcm.org/mailman/____listinfo/mitgcm-support>
> <http://mitgcm.org/mailman/____listinfo/mitgcm-support
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support>>
> <http://mitgcm.org/mailman/____listinfo/mitgcm-support
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support>
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support
> <http://mitgcm.org/mailman/listinfo/mitgcm-support>>>
> > >
> > >
> > > _____________________________________________________
>
>
> > > MITgcm-support mailing list
> > > MITgcm-support at mitgcm.org
> <mailto:MITgcm-support at mitgcm.org>
> <mailto:MITgcm-support at mitgcm.__org
> <mailto:MITgcm-support at mitgcm.org>>
> <mailto:MITgcm-support at mitgcm.
> <mailto:MITgcm-support at mitgcm.>____org
> <mailto:MITgcm-support at mitgcm.__org
> <mailto:MITgcm-support at mitgcm.org>>>
> <mailto:MITgcm-support at mitgcm
> <mailto:MITgcm-support at mitgcm>.
> <mailto:MITgcm-support at mitgcm
> <mailto:MITgcm-support at mitgcm>.__>____org
> <mailto:MITgcm-support at mitgcm.
> <mailto:MITgcm-support at mitgcm.>____org
> <mailto:MITgcm-support at mitgcm.__org
> <mailto:MITgcm-support at mitgcm.org>>>>
>
> > >
> http://mitgcm.org/mailman/______listinfo/mitgcm-support
> <http://mitgcm.org/mailman/____listinfo/mitgcm-support>
> <http://mitgcm.org/mailman/____listinfo/mitgcm-support
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support>>
> <http://mitgcm.org/mailman/____listinfo/mitgcm-support
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support>
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support
> <http://mitgcm.org/mailman/listinfo/mitgcm-support>>>
> > >
> > > _____________________________________________________
>
>
> > > MITgcm-support mailing list
> > > MITgcm-support at mitgcm.org
> <mailto:MITgcm-support at mitgcm.org>
> <mailto:MITgcm-support at mitgcm.__org
> <mailto:MITgcm-support at mitgcm.org>>
> <mailto:MITgcm-support at mitgcm.
> <mailto:MITgcm-support at mitgcm.>____org
> <mailto:MITgcm-support at mitgcm.__org
> <mailto:MITgcm-support at mitgcm.org>>>
> <mailto:MITgcm-support at mitgcm
> <mailto:MITgcm-support at mitgcm>.
> <mailto:MITgcm-support at mitgcm
> <mailto:MITgcm-support at mitgcm>.__>____org
> <mailto:MITgcm-support at mitgcm.
> <mailto:MITgcm-support at mitgcm.>____org
> <mailto:MITgcm-support at mitgcm.__org
> <mailto:MITgcm-support at mitgcm.org>>>>
>
> > >
> http://mitgcm.org/mailman/______listinfo/mitgcm-support
> <http://mitgcm.org/mailman/____listinfo/mitgcm-support>
> <http://mitgcm.org/mailman/____listinfo/mitgcm-support
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support>>
> <http://mitgcm.org/mailman/____listinfo/mitgcm-support
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support>
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support
> <http://mitgcm.org/mailman/listinfo/mitgcm-support>>>
> >
> >
> > _____________________________________________________
>
>
> > MITgcm-support mailing list
> > MITgcm-support at mitgcm.org
> <mailto:MITgcm-support at mitgcm.org>
> <mailto:MITgcm-support at mitgcm.__org
> <mailto:MITgcm-support at mitgcm.org>>
> <mailto:MITgcm-support at mitgcm.
> <mailto:MITgcm-support at mitgcm.>____org
> <mailto:MITgcm-support at mitgcm.__org
> <mailto:MITgcm-support at mitgcm.org>>>
> <mailto:MITgcm-support at mitgcm
> <mailto:MITgcm-support at mitgcm>.
> <mailto:MITgcm-support at mitgcm
> <mailto:MITgcm-support at mitgcm>.__>____org
> <mailto:MITgcm-support at mitgcm.
> <mailto:MITgcm-support at mitgcm.>____org
> <mailto:MITgcm-support at mitgcm.__org
> <mailto:MITgcm-support at mitgcm.org>>>>
>
> >
> http://mitgcm.org/mailman/______listinfo/mitgcm-support
> <http://mitgcm.org/mailman/____listinfo/mitgcm-support>
> <http://mitgcm.org/mailman/____listinfo/mitgcm-support
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support>>
> <http://mitgcm.org/mailman/____listinfo/mitgcm-support
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support>
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support
> <http://mitgcm.org/mailman/listinfo/mitgcm-support>>>
> >
> > _____________________________________________________
>
>
> > MITgcm-support mailing list
> > MITgcm-support at mitgcm.org
> <mailto:MITgcm-support at mitgcm.org>
> <mailto:MITgcm-support at mitgcm.__org
> <mailto:MITgcm-support at mitgcm.org>>
> <mailto:MITgcm-support at mitgcm.
> <mailto:MITgcm-support at mitgcm.>____org
> <mailto:MITgcm-support at mitgcm.__org
> <mailto:MITgcm-support at mitgcm.org>>>
> <mailto:MITgcm-support at mitgcm
> <mailto:MITgcm-support at mitgcm>.
> <mailto:MITgcm-support at mitgcm
> <mailto:MITgcm-support at mitgcm>.__>____org
> <mailto:MITgcm-support at mitgcm.
> <mailto:MITgcm-support at mitgcm.>____org
> <mailto:MITgcm-support at mitgcm.__org
> <mailto:MITgcm-support at mitgcm.org>>>>
>
> >
> http://mitgcm.org/mailman/______listinfo/mitgcm-support
> <http://mitgcm.org/mailman/____listinfo/mitgcm-support>
> <http://mitgcm.org/mailman/____listinfo/mitgcm-support
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support>>
> <http://mitgcm.org/mailman/____listinfo/mitgcm-support
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support>
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support
> <http://mitgcm.org/mailman/listinfo/mitgcm-support>>>
>
>
>
> _____________________________________________________
>
>
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> <mailto:MITgcm-support at mitgcm.org>
> <mailto:MITgcm-support at mitgcm.__org
> <mailto:MITgcm-support at mitgcm.org>>
> <mailto:MITgcm-support at mitgcm.
> <mailto:MITgcm-support at mitgcm.>____org
> <mailto:MITgcm-support at mitgcm.__org
> <mailto:MITgcm-support at mitgcm.org>>>
> <mailto:MITgcm-support at mitgcm
> <mailto:MITgcm-support at mitgcm>.
> <mailto:MITgcm-support at mitgcm
> <mailto:MITgcm-support at mitgcm>.__>____org
> <mailto:MITgcm-support at mitgcm.
> <mailto:MITgcm-support at mitgcm.>____org
> <mailto:MITgcm-support at mitgcm.__org
> <mailto:MITgcm-support at mitgcm.org>>>>
>
> http://mitgcm.org/mailman/______listinfo/mitgcm-support
> <http://mitgcm.org/mailman/____listinfo/mitgcm-support>
> <http://mitgcm.org/mailman/____listinfo/mitgcm-support
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support>>
> <http://mitgcm.org/mailman/____listinfo/mitgcm-support
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support>
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support
> <http://mitgcm.org/mailman/listinfo/mitgcm-support>>>
>
>
>
>
> _____________________________________________________
>
>
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> <mailto:MITgcm-support at mitgcm.org>
> <mailto:MITgcm-support at mitgcm.__org
> <mailto:MITgcm-support at mitgcm.org>>
> <mailto:MITgcm-support at mitgcm.
> <mailto:MITgcm-support at mitgcm.>____org
> <mailto:MITgcm-support at mitgcm.__org
> <mailto:MITgcm-support at mitgcm.org>>>
> <mailto:MITgcm-support at mitgcm
> <mailto:MITgcm-support at mitgcm>.
> <mailto:MITgcm-support at mitgcm
> <mailto:MITgcm-support at mitgcm>.__>____org
> <mailto:MITgcm-support at mitgcm.
> <mailto:MITgcm-support at mitgcm.>____org
> <mailto:MITgcm-support at mitgcm.__org
> <mailto:MITgcm-support at mitgcm.org>>>>
>
> http://mitgcm.org/mailman/______listinfo/mitgcm-support
> <http://mitgcm.org/mailman/____listinfo/mitgcm-support>
> <http://mitgcm.org/mailman/____listinfo/mitgcm-support
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support>>
> <http://mitgcm.org/mailman/____listinfo/mitgcm-support
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support>
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support
> <http://mitgcm.org/mailman/listinfo/mitgcm-support>>>
>
>
>
>
>
> _____________________________________________________
>
>
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> <mailto:MITgcm-support at mitgcm.org>
> <mailto:MITgcm-support at mitgcm.__org
> <mailto:MITgcm-support at mitgcm.org>>
> <mailto:MITgcm-support at mitgcm.
> <mailto:MITgcm-support at mitgcm.>____org
> <mailto:MITgcm-support at mitgcm.__org
> <mailto:MITgcm-support at mitgcm.org>>>
> http://mitgcm.org/mailman/______listinfo/mitgcm-support
> <http://mitgcm.org/mailman/____listinfo/mitgcm-support>
> <http://mitgcm.org/mailman/____listinfo/mitgcm-support
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support>>
> <http://mitgcm.org/mailman/____listinfo/mitgcm-support
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support>
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support
> <http://mitgcm.org/mailman/listinfo/mitgcm-support>>>
>
>
>
> _____________________________________________________
>
>
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> <mailto:MITgcm-support at mitgcm.org>
> <mailto:MITgcm-support at mitgcm.__org
> <mailto:MITgcm-support at mitgcm.org>>
> <mailto:MITgcm-support at mitgcm.
> <mailto:MITgcm-support at mitgcm.>____org
> <mailto:MITgcm-support at mitgcm.__org
> <mailto:MITgcm-support at mitgcm.org>>>
> http://mitgcm.org/mailman/______listinfo/mitgcm-support
> <http://mitgcm.org/mailman/____listinfo/mitgcm-support>
> <http://mitgcm.org/mailman/____listinfo/mitgcm-support
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support>>
>
>
> <http://mitgcm.org/mailman/____listinfo/mitgcm-support
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support>
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support
> <http://mitgcm.org/mailman/listinfo/mitgcm-support>>>
>
>
>
>
> ___________________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> <mailto:MITgcm-support at mitgcm.org>
> <mailto:MITgcm-support at mitgcm.__org
> <mailto:MITgcm-support at mitgcm.org>>
> http://mitgcm.org/mailman/____listinfo/mitgcm-support
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support>
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support
> <http://mitgcm.org/mailman/listinfo/mitgcm-support>>
>
>
>
> ___________________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org <mailto:MITgcm-support at mitgcm.org>
> <mailto:MITgcm-support at mitgcm.__org
> <mailto:MITgcm-support at mitgcm.org>>
> http://mitgcm.org/mailman/____listinfo/mitgcm-support
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support>
> <http://mitgcm.org/mailman/__listinfo/mitgcm-support
> <http://mitgcm.org/mailman/listinfo/mitgcm-support>>
>
>
>
>
> _________________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org <mailto:MITgcm-support at mitgcm.org>
> http://mitgcm.org/mailman/__listinfo/mitgcm-support
> <http://mitgcm.org/mailman/listinfo/mitgcm-support>
>
>
>
> _________________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org <mailto:MITgcm-support at mitgcm.org>
> http://mitgcm.org/mailman/__listinfo/mitgcm-support
> <http://mitgcm.org/mailman/listinfo/mitgcm-support>
>
>
>
>
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-support
>
More information about the MITgcm-support
mailing list