[MITgcm-support] "multiple tests" for testreport

[Chris Hill]

All,

Sounds good - but we should be a little more structured:

We should use a tree structure for this otherwise it will get illegible and
like Dimitris I want code variants too. The final structure should be
something like.

 input/
 input_alt/
 input_alt/sea_mount_bathy
 input_alt/shelf_bathy
 input_alt/complex_forcing

 code/
 code_alt/
 code_alt/s192t_8x4
 code_alt/s176t_8x4
 code_alt/sea_mount_bathy

 results/
 results_alt/sea_mount_bathy
 results_alt/s192t_8x4/


 If we don't do this at the top of an experiment level we will end up with
too many directories at the top and it will look messy. 

 **NOTE** if we allow code_alt and input_alt arbitrarily together we will
have a combinatorial explostion! The combinations exercised would therefore
be limited to {"matching_NAME_pair:matching_NAME_pair", "nomatch_NAME:base",
"base:nomatch_NAME"}, where NAME is a subdirectory entry from
[input_alt|results_alt] and "a:b" are selections from input_alt:code_alt
respectively. This means that in results_alt/ we only need one name. 

Chris
> -----Original Message-----
> From: mitgcm-support-bounces at mitgcm.org 
> [mailto:mitgcm-support-bounces at mitgcm.org] On Behalf Of Ed Hill
> Sent: Thursday, March 11, 2004 10:14 PM
> To: MITgcm-support
> Subject: [MITgcm-support] "multiple tests" for testreport
> 
> 
> Hi folks,
> 
> Based on discussions with JMC and AJA, I've modified testreport so 
> that one can create multiple (similar) tests for each directory within 
> verification.  The idea is that input files can be shared between 
> similar tests.
> 
> The way it works is that directories within verification will have 
> multiple tests run if they meet the following criteria:
> 
>   1) the test dir must contain a "build" directory
> 
>   2) the test dir must contain one or more extra "input" 
>      directories each with names of the format "input.*" 
>      (eg. input.alt_bathy, input.alt_forcing, etc.)
> 
>   3) the test dir must contain one or more extra "output" 
>      files in the format "results/output.txt.*" where the 
>      extension on each file matches each of the extensions 
>      on the input.* dir names
> 
> So an example would be:
> 
>   exp0/input                  <== standard/shared input files 
>   exp0/input.ex               <== extra and/or modif inputs
>   exp0/results/output.txt.ex  <== results for the ".ex" case
> 
> Please let me know if you have any ideas for improvements.  
> JMC said he would soon check in some examples to demonstrate it.  And 
> attached is an example of the output it generates.
> 
> Ed
> 
> 
> --
> Edward H. Hill III, PhD
> office:  MIT Dept. of EAPS;  Room 54-1424;  77 Massachusetts Ave.
>             Cambridge, MA 02139-4307
> email:   eh3 at mit.edu,  ed at eh3.com
> URL:     http://web.mit.edu/eh3/
> phone:   617-253-0098
> fax:     617-253-4464
>