[Mitgcm-support] More: 2) Calibrating Process Noise Q

Wed Jul 9 15:57:05 EDT 2003

HyperNews notification of new message.  See: 
  http://escher.JPL.NASA.GOV:2000/HyperNews/get/forums/assim/112/1.html

     =======================================================================     

Assessing Self-Consistency of the K-filter Assimilation: 
(or figuring out what's wrong with the K-filter) 
-------------------------------------------------------------

2) Calibrating Process Noise Q
------------------------------

SYNOPSIS: Process noise needs to be calibrated so that model errors
          are consistent with actual model-data differences.  Model
          errors must taper to zero at open cell boundaries to account
          for representation error. 

Error covariances used in the assimilation must be comparable with
actual model-data differences.  Otherwise, the estimate is
inconsistent with its assumptions.  In particular,
        Sum_(simulation - T/P)^2       .......................... (1) 
and 
        Sum_(simulation - T/P)^2  - Sum_(dynup - T/P)^2 ......... (2) 
that is used as a measure of skill, must be comparable with 
        H*P_sim*H^T + R  ........................................ (3) 
and 
        H*P_sim*H^T - H*P_dynup*H^T ............................. (4) 
respectively, where 
      H: observation matrix (sea level)
      P_sim : state error covariance of simulation
      P_dynup : state error covariance of dynup
      R: data constraint error

The (1) vs (3) agreement can be achieved by covariance matching; i.e.,
calibration of Q by comparing simulation residuals with prior
simulation error estimate.  The (2) vs (4) comparison is done by trial
and error using different Q.

(1) vs (3) (Calibrating Q)
--------------------------

Here we compare H*P_sim*H^T with its estimate from simulation-data
differences described in the 3-27-00 posting[1] on prior error
estimation.

The simulation sea level error estimates (HPHt; H*P*H^T) corresponding
to process noise (Q) used in kf021c (first trial of barotropic filter)
and kf023b3 (first trial of baroclinic filter) are shown here[2].  kf021c
estimate (top left; P1206a) is the barotropic sea level error and
kf023b3 estimate (middle left; P1213) is first baroclinic mode sea
level error.  Both use NCEP wind covariance as Q with
time-decorrelation of 1-day (kf021c) and 3-days (kf023b3).  These are
to be compared with prior error estimates described 3-27-00[3] and
plotted here[4].  The first baroclinic mode part of the baroclinic sea
level error is estimated here (top right)[5].  [Based on the ratio of
total baroclinic sea level to 1st baroclinic mode sea level.]

The kf021c barotropic simulation error estimate (P1206a) is only
slightly smaller than the prior estimate, but the kf023b3 1st
baroclinic mode simulation error estimate (P1213) is way too small;
i.e., the Q used in kf023b3 is inconsistent with actual model-data
residuals.  Process noise were re-calibrated by scaling Q on a
grid-by-grid basis on the coarse grids.  Resulting simulation error
estimates are shown in the previous figure[6]; barotropic (top right;
P1208), first baroclinic (middle right; P1219d).  These calibrated
simulation error estimates are now more comparable with prior
estimates.

(2) vs (4) (Checking and adjusting Q)
-------------------------------------

The skill of the filters (simulation residual variance minus dynup
residual variance) are shown here[7].  [All assimilation based on
detrended sea level, as described in parent posting[8].]  This
corresponds to (2).  Upper rows are barotropic filters (kf021c,
kf022e), lower rows are baroclinic filters (kf023b3, kf024f).  The
left columns use the uncalibrated Q; The right columns use the
calibrated Q described above.

The expected skill of each filter is shown here[9].  These correspond to
(4).  There is a zero order consistency between the overall magnitude
of (2) and (4) for assimilation using the calibrated Q (right column).
However, there are some discrepancies in geographic structure,
indicating room for further improvement.  Some of this difference may
be due to temporal variability; There is temporal variability in
skill, which the errors do not take into consideration.

As evident in the skill plot[10], the calibrated barotropic filter
(kf022e) shows modest improvement in the Southern Ocean.  It does
worse than uncalibrated Q (kf021c) in the Argentine Basin.
Nevertheless, the global averaged skill of the calibrated kf022e is
better than non-calibrated kf021c.

The calibrated baroclinic filter (kf024f) also has better skill than
the uncalibrated run (kf023b3).  The difference in overall skill is
surprisingly small given the large differences in prior simulation
error estimates.  However, although simulation errors are so
different, the measurement updated error have similar structure and
more similar values.  This figure[11] compares the simulation errors (top
row) and the measurement update errors (bottom row) for kf023b3 (left
column; P1213) and kf024f (right column; P1219d).

Although the overall skill of kf024f is higher than kf023b3, there is
once again degradation off Taiwan in kf024f.  This degradation seems
to be due to inaccurate coarse state updates along near the northern
boundary of the baroclinic K-filter cell.  This figure[12] compares the
differences between kf024f and kf023b3 off Taiwan (130E-140E,
23N-27N); The curves are sea level (blue) and time-integrated sea
level updates by the filter in this area (black).  The black curve
would be the actual sea level change (blue) if there were no dynamics;
i.e., change in sea level doesn't evolve.  The magenta and red curves
are the contributions to the black curve by two of the westernmost
coarse grid points of the northernmost and second from the north rows,
respectively; i.e., coarse grids 106-107 and 120-121.  (See coarse
grid here[13].)  The green curve is the sum of magenta and red; i.e.,
contributions from coarse grids 106, 107, 120, 121.

Contributions from 120-121 (red) is comparable to those of 106-107
(magenta), and the sum of the two (green) account for most of the
Kalman updates (black).  There is coherence between the blue curve and
the rest (except toward the end, likely due to dynamics).  The
similarity indicates that changes off Taiwan, and consequently the
degradation, may mostly be due to Kalman updates at the northwest
corner.  Change estimates along the open boundary are the least
accurate because the cellular approximation lacks the physics there.
The error estimate along open boundaries should be small and
corresponding Kalman filter updates be negligible.  Therefore, the
degradation may be due to too large Kalman updates near the open
boundary due to too large an error estimate.

Process noise may be re-adjusted so that simulation error at open
boundaries are negligible.  Alternatively, estimated model errors may
be adjusted manually to effect the correction.  This figure[14] shows the
skill of assimilation with the latter approximation (kf024i; lower
left) by zeroing any Kalman updates along the north and south coarse
grid boundaries.  Indeed the degradation off Taiwan is much reduced
from an non-corrected Q (kf024f;upper right), closer to our previous
best result (kf023b3; upper left), while maintaining the superior
performance elsewhere.

There is still room for improving kf022e (barotropic) and kf024i
(baroclinic) by tuning their assumed process noise, Q; e.g.,
   a) Taper down baroclinic Q more rapidly toward the open boundaries
      to make degradation off Taiwan and Australia smaller. 
   b) Reduce barotropic Q in Argentine basin, south of Australia-New
      Zealand, Indian Ocean, Gulf Stream extension. 

However, such tuning will likely be cosmetic, without fundamental
changes in estimated circulation.  We probably should do the other
things first;
   1) Analyze the circulation
   2) Make the analysis available on the web
   3) Implement baroclinic filter with more modes.
   4) Implement other regional baroclinic filters.
   5) Assimilate other data types. 

=======================================================================     

-----------------------------------------------------------------------
[1] http://escher.JPL.NASA.GOV:2000/HyperNews/get/forums/assim/83.html
[2] http://escher.JPL.NASA.GOV:2000/hosts/escher/escher2/medea/if/forum/t73g.ps
[3] http://escher.JPL.NASA.GOV:2000/HyperNews/get/forums/assim/83.html
[4] http://escher.JPL.NASA.GOV:2000/hosts/escher/escher2/medea/if/forum/t36e5_h2.ps
[5] http://escher.JPL.NASA.GOV:2000/hosts/escher/escher2/medea/if/forum/t36e5_h6.ps
[6] http://escher.JPL.NASA.GOV:2000/hosts/escher/escher2/medea/if/forum/t73g.ps
[7] http://escher.JPL.NASA.GOV:2000/hosts/escher/escher2/medea/if/forum/t73f5g_b.ps
[8] http://escher.JPL.NASA.GOV:2000/HyperNews/get/forums/assim/112.html
[9] http://escher.JPL.NASA.GOV:2000/hosts/escher/escher2/medea/if/forum/t73g_4.ps
[10] http://escher.JPL.NASA.GOV:2000/hosts/escher/escher2/medea/if/forum/t73f5g_b.ps
[11] http://escher.JPL.NASA.GOV:2000/hosts/escher/escher2/medea/if/forum/t73g_3.ps
[12] http://escher.JPL.NASA.GOV:2000/hosts/escher/escher2/medea/if/forum/t79e.ps
[13] http://escher.JPL.NASA.GOV:2000/hosts/escher/escher2/medea/if/forum/cgrid_bc3_plt.ps
[14] http://escher.JPL.NASA.GOV:2000/hosts/escher/escher2/medea/if/forum/t73f5g_b_2.ps