Using m1qn3, and having made these changes
<https://github.com/dorugeber/MITgcm/commit/0a236266f0bffdfabb55338c26bf07ed1b87eb30>
to data.optim, is the attached file reasonable output for the first
iteration?
In particular, I'm a bit concerned by line 132:
pathei: ff 5.3046703247717040E+180
Thanks,
Andrew
On 24 April 2018 at 18:06, Matthew Mazloff <mmazloff at ucsd.edu> wrote:
> Hi Andrew
>
> I would make sure your data.optim is compatible with optim_m1qn3. I have
> pasted an example here:
>
> more ../../SOCCOM/Optim/data.optim
> #
> # ********************************
> # Off-line optimization parameters
> # ********************************
> &OPTIM
> optimcycle=124,
> numiter=1000,
> nfunc=2,
> fmin= 2000000000.0,
> iprint=10,
> nupdate=3,
> /
> #
> &M1QN3
> coldstart = .FALSE.,
> /
> #
>
> And then I wouldn’t use a script that takes many steps at once. Try
> stepping through one step at a time and see where it fails.
>
> Matt
>
>
>
> On Apr 24, 2018, at 2:22 AM, Andrew McRae <andrew.mcrae at physics.ox.ac.uk>
> wrote:
>
> What changes do I need to make to get this working with, e.g.,
> tutorial_global_oce_optim?
>
> I have added
>
> &M1QN3
> &end
>
> to data.optim, and have updated the cycsh script as the filenames there
> seem outdated:
>
> 11c10
> < cp -p OPTIM/ctrlhflux_tut_MITgcm.opt0$ii .
> ---
> > cp -p OPTIM/ecco_ctrl_MIT_CE_000.opt0$ii .
> 27,28c26,27
> < mv ctrlhflux_tut_MITgcm.opt0$ii $fich $direc
> < mv costhflux_tut_MITgcm.opt0$ii OPTIM/
> ---
> > mv ecco_ctrl_MIT_CE_000.opt0$ii $fich $direc
> > mv ecco_cost_MIT_CE_000.opt0$ii OPTIM/
>
> but, following the instructions from 3.18.5 of the .pdf manual, the
> optimisation seems to fail from the second iteration onwards:
>
> Beginning of iteration 001
> Running mitcgm_ad: iteration 001
> STOP NORMAL END
> Cleaning
> rm: No match.
> mkdir: cannot create directory ‘runtutorial001’: File exists
> Line-search: iteration 001
> optimcycle=1,
> Beginning of iteration 002
> Running mitcgm_ad: iteration 002
> STOP NORMAL END
> Cleaning
> rm: No match.
> mkdir: cannot create directory ‘runtutorial002’: File exists
> Line-search: iteration 002
> optimcycle=2,
> STOP ABNORMAL in S/R OPTIM_SUB
> Beginning of iteration 003
> Running mitcgm_ad: iteration 003
> [Ctrl+C]
>
> The file m1qn3_output.txt is attached. I'm suspicious that lines 28-29
> and 49-50 are the same; it looks like the step was never taken?
>
> Thanks,
> Andrew
>
> On 16 April 2018 at 11:24, Andrew McRae <andrew.mcrae at physics.ox.ac.uk>
> wrote:
>
>> Thanks, this seems to work (at least, build, and the testbed files run).
>>
>> Hopefully this 5-year old code replaces the 10-year old code in master at
>> some point!
>>
>> Andrew
>>
>> On 20 March 2018 at 19:37, Matthew Mazloff <mmazloff at ucsd.edu> wrote:
>>
>>> Hello
>>>
>>> Lately I have been using:
>>> http://wwwcvs.mitgcm.org/viewvc/MITgcm/MITgcm_contrib/mlosch
>>> /optim_m1qn3/?pathrev=HEAD
>>>
>>> This is an alternative search algorithm code that I find easier to use
>>> (Thanks again Martin!). You may want to give it a try
>>>
>>> -Matt
>>>
>>>
>>> On Mar 20, 2018, at 11:46 AM, Andrew McRae <
>>> andrew.mcrae at physics.ox.ac.uk> wrote:
>>>
>>> Are these run as part of the daily builds? I can't get it to build.
>>>
>>> Following part 3.18.4.2 of the .pdf manual (I don't think this section
>>> has been ported online yet)...
>>>
>>> In lsopt/, the supplied blas libraries are 10+ years old and my compiler
>>> says the Pentium 4 blas1 library is incompatible.
>>>
>>> In optim/, "make depend" doesn't work (makedepend undefined).
>>>
>>> "make" obviously causes the compiler to complain about -lblas1 not
>>> existing.
>>>
>>> After changing the makefile to use -lblas instead of -lblas1, "make"
>>> gives the output
>>>
>>> f77 -o optim.x optim_main.o optim_sub.o optim_numbmod.o optim_initmod.o
>>> optim_readdata.o optim_writedata.o optim_write_control.o xerbla.o simul.o
>>> -L. -L../lsopt/ -llsopt_ecco -lblas
>>> /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/crt1.o: In
>>> function `_start':
>>> (.text+0x20): undefined reference to `main'
>>> optim_numbmod.o: In function `optim_numbmod_':
>>> optim_numbmod.f:(.text+0x1ff1): undefined reference to `ilnblnk_'
>>> optim_numbmod.f:(.text+0x2478): undefined reference to `ilnblnk_'
>>> collect2: error: ld returned 1 exit status
>>> Makefile:78: recipe for target 'optim.x' failed
>>> make: *** [optim.x] Error 1
>>>
>>> And this just looks like broken code rather than anything BLAS-related.
>>> An undefined reference to main, and two undefined references to ilnblnk.
>>>
>>> ilnblnk is defined with
>>> c == external ==
>>> integer ilnblnk
>>>
>>> Then used twice as
>>> il = max(ilnblnk(record),1)
>>>
>>> Any ideas?
>>> _______________________________________________
>>> MITgcm-support mailing list
>>> MITgcm-support at mitgcm.org
>>> http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support
>>>
>>>
>>>
>>
> <m1qn3_output.txt>_______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.mitgcm.org/pipermail/mitgcm-support/attachments/20180427/06df154d/attachment-0001.html>
-------------- next part --------------
==================================================
Large Scale Optimization with off-line capability.
==================================================
Version 3.1.0
OPTIM_READPARMS: Control options have been read.
OPTIM_READPARMS: Minimization options have been read.
pathei-lsopt in optim_readdata
OPTIM_READDATA: Reading cost function and
gradient of cost function
for optimization cycle: 0
opened file ecco_cost_MIT_CE_000.opt0000
pathei: nvartype 1
pathei: nvarlength 2315
pathei: yctrlid MIT_CE_000
pathei: filenopt 0
pathei: fileff 6.2002322818232924
pathei: fileiG 1
pathei: filejG 1
pathei: filensx 2
pathei: filensy 2
pathei: nWetcGlobal 2315 2315 2254 2215 2178 2142 2114 2076 2048 1999 1948 1850 1655 1372 828
pathei: nWetsGlobal 2149 2149 2102 2058 2027 1987 1959 1918 1887 1831 1771 1653 1404 1118 623
pathei: nWetwGlobal 2206 2206 2146 2104 2070 2029 2004 1959 1925 1869 1808 1705 1458 1164 671
pathei: nWetvGlobal 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
pathei: ncvarindex -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 124 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
pathei: ncvarrecs 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
pathei: ncvarxmax 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 45 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
pathei: ncvarymax 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 20 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
pathei: ncvarnrmax 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
pathei: ncvargrd ???????????????????????c????????????????????????????????????
OPTIM_READPARMS: Iteration number = 0
number of control variables = 2315
cost function value in ecco_ctrl = 6.2002322818232924
expected cost function minimum = 5.7400000000000002
expected cost function decrease = 0.46023228182329223
Data will be read from the following file: ecco_ctrl_MIT_CE_000.opt0000
OPTIM_SUB: Calling m1qn3_optim for iteration: 0
OPTIM_SUB: with nn, REAL_BYTE = 2315 4
OPTIM_SUB: read model state
pathei-lsopt in optim_readdata
OPTIM_READDATA: Reading control vector
for optimization cycle: 0
opened file ecco_ctrl_MIT_CE_000.opt0000
pathei: nvartype 1
pathei: nvarlength 2315
pathei: yctrlid MIT_CE_000
pathei: filenopt 0
pathei: fileff 6.2002322818232924
pathei: fileiG 1
pathei: filejG 1
pathei: filensx 2
pathei: filensy 2
pathei: nWetcGlobal 2315 2315 2254 2215 2178 2142 2114 2076 2048 1999 1948 1850 1655 1372 828
pathei: nWetsGlobal 2149 2149 2102 2058 2027 1987 1959 1918 1887 1831 1771 1653 1404 1118 623
pathei: nWetwGlobal 2206 2206 2146 2104 2070 2029 2004 1959 1925 1869 1808 1705 1458 1164 671
pathei: nWetvGlobal 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
pathei: ncvarindex -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 124 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
pathei: ncvarrecs 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
pathei: ncvarxmax 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 45 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
pathei: ncvarymax 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 20 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
pathei: ncvarnrmax 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
pathei: ncvargrd ???????????????????????c????????????????????????????????????
pathei-lsopt in optim_readdata
OPTIM_READDATA: Reading cost function and
gradient of cost function
for optimization cycle: 0
opened file ecco_cost_MIT_CE_000.opt0000
pathei: nvartype 1
pathei: nvarlength 2315
pathei: yctrlid MIT_CE_000
pathei: filenopt 0
pathei: fileff 6.2002322818232924
pathei: fileiG 1
pathei: filejG 1
pathei: filensx 2
pathei: filensy 2
pathei: nWetcGlobal 2315 2315 2254 2215 2178 2142 2114 2076 2048 1999 1948 1850 1655 1372 828
pathei: nWetsGlobal 2149 2149 2102 2058 2027 1987 1959 1918 1887 1831 1771 1653 1404 1118 623
pathei: nWetwGlobal 2206 2206 2146 2104 2070 2029 2004 1959 1925 1869 1808 1705 1458 1164 671
pathei: nWetvGlobal 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
pathei: ncvarindex -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 124 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
pathei: ncvarrecs 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
pathei: ncvarxmax 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 45 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
pathei: ncvarymax 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 20 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
pathei: ncvarnrmax 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
pathei: ncvargrd ???????????????????????c????????????????????????????????????
OPTIM_SUB after reading ecco_ctrl and ecco_cost:
OPTIM_SUB nn = 2315
OPTIM_SUB objf = 6.2002322818232924
OPTIM_SUB xx(1) = 0.0000000000000000
OPTIM_SUB adxx(1) = -6.7879882408306003E-005
OPTIM_SUB: cold start, optimcycle = 0
OPTIM_SUB: call m1qn3_offline ........
OPTIM_SUB: ...........................
OPTIM_SUB: returned from m1qn3_offline
OPTIM_SUB: nn = 2315
OPTIM_SUB: xx(1) = 0.38550331655968251 0.54187150690779251
OPTIM_SUB: adxx(1) = -6.7879882408306003E-005 -9.5413379312958568E-005
OPTIM_SUB: omode = -1
OPTIM_SUB: niter = 1
OPTIM_SUB: nsim = 2000
OPTIM_SUB: reverse = 1
OPTIM_SUB: mean(xx) = 0.12147501041947409
OPTIM_SUB: max(xx) = 3.4534949169843263
OPTIM_SUB: min(xx) = -6.9274397317483585
OPTIM_SUB: std(xx) = 8.6027463877452739
OPTIM_STORE_M1QN3: saving the state of m1qn3 in OPWARM.opt0001
OPTIM_SUB: writing 2315 sized control to file ecco_ctrl
OPTIM_WRITEDATA: Writing new control vector to file(s)
for optimization cycle: 1
pathei: nvartype 1
pathei: nvarlength 2315
pathei: yctrlid MIT_CE_000
pathei: nopt 1
pathei: ff 5.3046703247717040E+180
pathei: iG 1
pathei: jG 1
pathei: nsx 2
pathei: nsy 2
pathei: nWetcGlobal 2315 2315 2254 2215 2178 2142 2114 2076 2048 1999 1948 1850 1655 1372 828
pathei: nWetsGlobal 2149 2149 2102 2058 2027 1987 1959 1918 1887 1831 1771 1653 1404 1118 623
pathei: nWetwGlobal 2206 2206 2146 2104 2070 2029 2004 1959 1925 1869 1808 1705 1458 1164 671
pathei: nWetvGlobal 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
pathei: ncvarindex -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 124 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
pathei: ncvarrecs 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
pathei: ncvarxmax 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 45 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
pathei: ncvarymax 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 20 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
pathei: ncvarnrmax 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
pathei: ncvargrd ???????????????????????c????????????????????????????????????
end of optim_writedata: icvoffset 2315
======================================
Large Scale Optimization run finished.
======================================