[MITgcm-support] Single reduction CG solver

[Martin Losch]

Hi Christopher,

that sounds very good. Do you think that the effect will be similar  
for cg3d?

Are the results between the two cg2d's different? If so, by how much?

Martin
On Nov 19, 2009, at 1:45 AM, Christopher L. Wolfe wrote:

>
> Hi all,
>
> I've written an implementation of d'Azevedo, Eijkhout, and Romine's  
> (1999) single reduction CG solver for the MITgcm. This method uses a  
> rearrangement of the standard conjugate gradient method so that the  
> required scalars can be determined by a single call to  
> MPI_Allreduce. For problems running on a large number of processors,  
> the decreased MPI overhead can significantly increase the efficiency  
> of the conjugate gradient solver.
>
> The attached figure shows the scaling of the 2D CG solver for a  
> fixed problem size of 1024x1024 as the number of processors increase  
> on the Cray XT4 "Franklin." For less than 2^6 processors, the  
> original cg2d solver is slightly more efficient than the single  
> reduction solver since the latter requires slightly more matrix- 
> vector multiplications. However, for processor counts over 2^7, the  
> single reduction solver's performance is significantly better than  
> the original cg2d.
>
> I've attached a tarball of the new CG routine cg2d_sr plus the  
> modified files CG2D.h, ini_cg2d.F, and solve_for_pressure.F. To use  
> the single reduce solver, simply drop these files into an experiment  
> directory and compile with "ALLOW_CG2D_SR" defined. I've also  
> included at benchmarking suite, which can also be compiled like a  
> standard MITgcm experiment.
>
> I've only implemented the single reduce solver for the 2D case since  
> I just use the hydrostatic model, but the 3D implementation should  
> be straightforward. A paper deriving the single reduction method can  
> be found at http://www.netlib.org/lapack/lawnspdf/lawn56.pdf
>
> Feel free to contact me with any questions or comments.
>
> Cheers,
> Christopher
>
> -----------------------------------------------------------
> Dr. Christopher L. Wolfe              	   858-534-4560
> Climate, Atmospheric Science, and Physical Oceanography
> Scripps Institution of Oceanography, UCSD  clwolfe at ucsd.edu
> -----------------------------------------------------------
>
> <strong.eps>
>
>
> <cg2d_sr.tar.gz>
> <cg2d_sr_bench.tar.gz>
>
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-support