<html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Hi An<div class=""><br class=""></div><div class="">In my experience this is always due to requesting too much memory (or a node <i class="">thinking</i> you are requesting too much memory). Does this every happen when you run smaller setups? (Or, perhaps, when you run the same setup but with half the vertical levels?)</div><div class=""><br class=""></div><div class="">-Matt</div><div class=""><br class=""></div><div class=""> </div><div class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Feb 27, 2018, at 5:41 PM, An Nguyen <<a href="mailto:antnguyen13@gmail.com" class="">antnguyen13@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html charset=us-ascii" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">hi Jean-Michel,<div class=""><br class=""></div><div class="">Thank you for the suggestion, no I did not run with debugMode=.TRUE. in eedata, I might try that now. I will use -convert big_endian and -assume byterecl, and -mcmodel=large.</div><div class=""><br class=""></div><div class="">I think the problem we're encountering here is that it's "not working", and we just can not figure out where the mitgcmuv fails, whether it is the mitgcmuv itself or the infrastructure of the computing node that fails. We do not get any message at all, just "exit" message by the nodes , no STDOUT or STDERR or .err or .out files to even understand where the problem is. So I would like to run the mitgcmuv (and primarily its _ad version where we have the issue) in the debugging mode to see if I can go line-by-line until it crashes to try to narrow down where the problem is.</div><div class=""><br class=""></div><div class="">Thanks,</div><div class="">An</div><div class=""><br class=""></div><div class=""><br class=""><div class=""><blockquote type="cite" class=""><div class="">On Feb 27, 2018, at 8:09 PM, Jean-Michel Campin <<a href="mailto:jmc@mit.edu" class="">jmc@mit.edu</a>> wrote:</div><br class="Apple-interchange-newline"><div class="">Hi An,<br class=""><br class="">I am not sure I understand correctly:<br class="">Trying to use the simplest CFLAGS, FFLAGS, FOPTIM setting might not be <br class="">the best to compile and run. <br class="">For instance, without:<br class=""> -convert big_endian -assume byterecl<br class="">it will not be abble to read any "big-endian" binary, which could be a problem.<br class="">Or without:<br class=""> -mcmodel=medium<br class="">it might not run if the memory footprint is too large.<br class=""><br class="">I found that, generally, once I load the right module and set env.variable: <br class="">MPI_INC_DIR to correct location,<br class="">I can use one of the 3 "standard" optfile:<br class=""> linux_amd64_gfortran<br class=""> linux_amd64_ifort11<br class=""> linux_amd64_ifort+impi<br class="">and it just works.<br class="">And the improvement that I would get by fine tuning of some of the compiler <br class="">options is not very significant (and in my case, not worth spending too much time).<br class=""><br class="">And the nice thing about these ones is that you could use "-ieee" or "-devel"<br class="">genmake2 option to turn off all optimisation level (-ieee) or even turn on<br class="">all debug option (-devel).<br class=""><br class="">Now in your case, are you running with debugMode=.TRUE., in eedata ?<br class=""><br class="">Cheers,<br class="">Jean-Michel<br class=""><br class="">On Tue, Feb 27, 2018 at 04:24:04PM -0500, An Nguyen wrote:<br class=""><blockquote type="cite" class="">Hello mitgcm gurus,<br class=""><br class="">For the last 2 years I've been having problem with the mitgcm crashing right out of the gate without producing any STDOUT or STDERR or any message at all on the various computing nodes we have at UT Austin, and the only help tech-support can provide us so far is a message saying that we need to do our own debugging. <br class=""><br class="">I'd like to ask for some help with the flags needed to remove all optimization options when compiling the mitgcm (with ifort) in order to enter debug mode such as dbg or this ddt tool they suggested (<a href="https://portal.tacc.utexas.edu/software/ddt" class="">https://portal.tacc.utexas.edu/software/ddt</a> <<a href="https://portal.tacc.utexas.edu/software/ddt" class="">https://portal.tacc.utexas.edu/software/ddt</a>>) . The genmake2 various flags are very elaborate with various options for CFLAGS, FFLAGS, FOPTIM that are really beyond my comprehension (I spent 5 hours last night fiddling around and didn't succeed), so I'm hoping to get some suggestions on what I need to set: would stripping ALL CFLAGS and FFLAGS and FOPTIM and only use <br class=""><br class="">FFLAGS='-g'<br class="">FOPTIM='-O0'<br class="">CFLAGS=''<br class=""><br class="">an option? Any suggestion, including a sampled very-stripped-down optfile would be very helpful for me. I can provide more information (within my comprehension) if needed.<br class=""><br class="">Many thanks,<br class="">An<br class=""><br class="">An T Nguyen, Ph.D. <br class="">Institute of Computational Engineering and Sciences <br class="">The University of Texas at Austin, <br class="">201 East 24th Street, POB 4.234, <br class="">Austin, TX 78712, USA <br class="">Phone: (512) 471-4207 <br class=""><<a href="mailto:atnguyen@ices.utexas.edu" class="">atnguyen@ices.utexas.edu</a>><br class=""><br class=""></blockquote><br class=""><blockquote type="cite" class="">_______________________________________________<br class="">MITgcm-devel mailing list<br class=""><a href="mailto:MITgcm-devel@mitgcm.org" class="">MITgcm-devel@mitgcm.org</a><br class=""><a href="http://mailman.mitgcm.org/mailman/listinfo/mitgcm-devel" class="">http://mailman.mitgcm.org/mailman/listinfo/mitgcm-devel</a><br class=""></blockquote><br class="">_______________________________________________<br class="">MITgcm-devel mailing list<br class=""><a href="mailto:MITgcm-devel@mitgcm.org" class="">MITgcm-devel@mitgcm.org</a><br class=""><a href="http://mailman.mitgcm.org/mailman/listinfo/mitgcm-devel" class="">http://mailman.mitgcm.org/mailman/listinfo/mitgcm-devel</a><br class=""></div></blockquote></div><br class=""><div apple-content-edited="true" class="">
<div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class="">An T Nguyen, Ph.D. </div><div class="">Institute of Computational Engineering and Sciences </div><div class="">and Jackson School for Geosciences, </div><div class="">The University of Texas at Austin, </div><div class="">201 East 24th Street, POB 4.234, </div><div class="">Austin, TX 78712, USA </div><div class="">Phone: (512) 471-4207 </div><div class=""><<a href="mailto:atnguyen@ices.utexas.edu" class="">atnguyen@ices.utexas.edu</a>></div></div></div>
</div>
<br class=""></div></div>_______________________________________________<br class="">MITgcm-devel mailing list<br class=""><a href="mailto:MITgcm-devel@mitgcm.org" class="">MITgcm-devel@mitgcm.org</a><br class="">http://mailman.mitgcm.org/mailman/listinfo/mitgcm-devel<br class=""></div></blockquote></div><br class=""></div></body></html>