Changeset 4806


Ignore:
Timestamp:
Mar 10, 2009, 3:02:46 PM (15 years ago)
Author:
Nicklas Nordborg
Message:

References #1251: Improvements of the 'performance' test program

1-4 should now be fixed, as well as several other things discovered during the process to make things work smoothly.

Location:
trunk
Files:
9 edited

Legend:

Unmodified
Added
Removed

  • trunk/build.xml

    r4778 r4806  
    14071407      <fileset dir="doc" includes="licenses/**/*" />
    14081408      <fileset dir="doc" includes="historical/**/*" />
    1409       <fileset dir="doc" includes="test/roles/**/*" />
     1409      <fileset dir="doc" includes="test/**/*" />
    14101410      <!--fileset dir="doc" includes="user/**/*" /-->
    14111411      <fileset dir="doc" includes="*.*" />

  • trunk/doc/test/performance/index.html

    r4509 r4806  
    2525  <head>
    2626    <title>BASE - Performance testing of BASE</title>
    27   <link rel=stylesheet type="text/css" href="../../styles.css">
     27  <link rel=stylesheet type="text/css" href="../../historical/styles.css">
    2828  </head>
    2929<body>
    … …  
    3131<div class="navigation">
    3232  <a href="../../index.html">BASE</a>
    33   <img src="../../next.gif">
     33  <img src="../../historical/next.gif">
    3434  <a href="../index.html">Test procedures</a>
    35   <img src="../../next.gif">
     35  <img src="../../historical/next.gif">
    3636  Performance
    3737</div>
    … …  
    4646      a BASE version 1 server must be done by hand. For later BASE
    4747      versions, there is a test program. The current test program
    48       is a prototype which includes the following tests:
     48      includes the following tests:
    4949    </p>
    5050   
    … …  
    5454    <li>Create a filtered bioassayset</li>
    5555    <li>Create a normalized bioassayset with LOWESS</li>
     56    <li>Export bioassay set data in Serial and Matrix BASEFile format and MeV format</li>
    5657    </ul>
    5758   
    … …  
    6061    <ol>
    6162    <li><a href="#program">Running the test program</a></li>
     63    <li><a href="#typical">A typical test sequence</a></li>
    6264    </ol>
    6365   
    … …  
    6870
    6971 
    70   <a name="summary"></a>
     72  <a name="program"></a>
    7173  <h2>1. Running the test pgoram</h2>
    7274
    … …  
    8991    <li>Change to the <code>build/test/</code> directory.</li>
    9092
    91     <li>Run test programs: <code>./test.sh performace &lt;options&gt; &lt;cmds&gt;</code>
    92       where <code>&lt;options&gt;</code> is may be one or more of the
    93       following:
     93    <li>Run test program:
     94      <pre class="code">
     95./test.sh performace &lt;options&gt; &lt;cmd&gt;
     96</pre>
     97  </ol>
     98
     99     
     100      <p>
     101      <b>Test program options</b>
     102
    94103      <ul>
    95       <li><code>-n</code>: Number of raw data sets to import or use in the root bioassayset.
    96         The default value is 4.
    97       <li><code>-u</code>: Username to run the test as. The default value is "root".
    98       <li><code>-p</code>: Password for the user. The default value is "root".
    99       </ul>
    100      
    101       and <code>&lt;cmds&gt;</code> is one or more of the following:
    102       <ul>
    103       <li><code>all</code>: run all tests (except clean)</li>
    104       <li><code>raw</code>: import raw data</li>
    105       <li><code>root</code>: create a root bioassayset</li>
    106       <li><code>filter</code>: create a filtered bioassayset</li>
    107       <li><code>lowess</code>: create a lowess normalized bioassayset</li>
    108       <li><code>clean</code>: delete all test data, except reporters</li>
    109       </ul>
    110     </li>
    111    
    112   </ol>
     104      <li>global options:
     105        <ul>
     106        <li><code>-u</code>: Username to run the test as. The default value is "root".
     107        <li><code>-p</code>: Password for the user. The default value is "root".
     108        <li><code>-x</code>: If present, the tests are executed immediately. Otherwise
     109          they are added as jobs to the job queue.
     110        </ul>
     111        <p>
     112       
     113      <li><code>raw</code>: import raw data
     114        <ul>
     115        <li><code>-i</code>: Number of raw bioassays to create.
     116          The default value is 1.
     117        <li><code>-c</code>: Name prefix for the created raw bioassays. A
     118          running number is appended to the name automatically. The
     119          default value is "Raw".
     120        </ul>
     121        <p>
     122     
     123      <li><code>root</code>: create root bioassay sets
     124        <ul>
     125        <li><code>-s</code>: Name prefix of raw bioassays that should be used.
     126          If no value is given, all raw bioassays in the experiment are used.
     127        <li><code>-c</code>: Name prefix for the created bioassay sets. The
     128          iteration number and number of raw data sets is appended to the name
     129          automatically. The default value is "Root".
     130        <li><code>-i</code>: Number of root bioassay sets to create in each
     131          iteration. The default value is 1.
     132        <li><code>-mi</code>: Number of source raw bioassays to use in the first
     133          iteration. The default value is 1. The number of source raw bioassays
     134          are double in each iteration.
     135        <li><code>-mx</code>: Number of source raw bioassays to use in the last
     136          iteration. The default value is 1.
     137        </ul>
     138        Note! With the -i, -mi and -mx parameters it is possible to specify that
     139        multiple root bioassay sets are created. Here is an example:
     140        <code>-i 2 -mi 10 -mx 80</code>. This creates 2 root bioassays with
     141        10, 20, 40 and 80 source raw bioassays each.
     142        <p>
     143       
     144      <li><code>filter</code>: create filtered bioassay sets
     145        <ul>
     146        <li><code>-s</code>: Name prefix of bioassay sets that should be
     147          filtered. The default value is "Root".
     148        <li><code>-c</code>: Name prefix for the created bioassay sets.
     149          The default value is "Filtered".
     150        </ul>
     151        <p>
     152     
     153      <li><code>lowess</code>: create normalized bioassay sets using lowess
     154        <ul>
     155        <li><code>-s</code>: Name prefix of bioassay sets that should be
     156          normalized. The default value is "Filtered".
     157        <li><code>-c</code>: Name prefix for the created bioassay sets.
     158          The default value is "Lowess".
     159        </ul>
     160        <p>
     161     
     162      <li><code>export</code>: export bioassay set data
     163        <ul>
     164        <li><code>-s</code>: Name prefix of bioassay sets that should be
     165          exported. There is no default value.
     166        </ul>
     167        <p>
     168 
     169      <li><code>clean</code>: delete all test data, except reporters
     170        <p>
     171     
     172    </ul>
     173    </ol>
    113174
    114175  <p>
    … …  
    125186  </p>
    126187 
     188  <a name="typical"></a>
     189  <h2>2. A typical test sequence</h2>
     190 
     191  <p>
     192    In this case we assume a fresh, empty database.
     193  </p>
     194 
     195  <pre class="code">
     196# Import raw data to 10 raw bioassays.
     197./test.sh performace -i 10 raw
     198
     199# Create 2 root bioassay sets each with 1, 2, 4, and 8
     200# raw bioassays
     201./test.sh performance -i 2 -mi 1 -mx 8 root
     202
     203# Filter all root bioassay sets
     204./test.sh performance -s Root -c Filtered filter
     205
     206# Normalize all filtered bioassay sets
     207./test.sh performance -s Filtered -c Lowess lowess
     208
     209# Export all root bioassay sets
     210./test.sh performance -s Root export
     211
     212# Export all filtered bioassay sets
     213./test.sh performance -s Filtered export
     214</pre>
     215
    127216 
    128217</body>

  • trunk/src/test/net/sf/basedb/test/performance/ExportTest.java

    r4803 r4806  
    4040import net.sf.basedb.core.RawDataTypes;
    4141import net.sf.basedb.core.StringParameterType;
     42import net.sf.basedb.test.PluginUtil;
    4243import net.sf.basedb.test.TestUtil;
     44import net.sf.basedb.util.Values;
    4345
    4446/**
    … …  
    5759    Run the tests in this class.
    5860  */
    59   public static boolean run(String user, String password, String bioAssaySet)
     61  public static boolean run(String user, String password, String sourcePrefix, boolean execute)
    6062  {
    6163    DbControl dc = null;
    … …  
    6567      TestUtil.login(user, password, false);
    6668
    67       // Create raw bioassays
     69      // Configuration options
     70      TestUtil.write("---------------------------------------------\n");
     71      TestUtil.write("Source prefix=" + sourcePrefix + "\n");
     72      TestUtil.write("      Execute=" + execute + "\n");
     73      TestUtil.write("---------------------------------------------\n");
     74     
     75      // Load source bioassay sets
    6876      dc = TestUtil.getDbControl();
    69      
    70       // Load prepared items
    7177      Experiment experiment = PrepareTest.getExperiment(dc);
    7278      String expression = Config.getFilterExpression();
    73       List<BioAssaySet> allBas = new ArrayList<BioAssaySet>(Util.findAllBioAssaySets(dc, experiment, bioAssaySet));
    74       if (allBas.size() == 0)
    75       {
    76         throw new ItemNotFoundException("Bioassay set[name like " + bioAssaySet + "]");
     79      List<BioAssaySet> sources = new ArrayList<BioAssaySet>(
     80          Util.findAllBioAssaySets(dc, experiment, sourcePrefix + "%"));
     81      int numSources = sources.size();
     82      if (numSources == 0)
     83      {
     84        throw new ItemNotFoundException("BioAssaySet[name like " + sourcePrefix + "]");
    7785      }
    7886      dc.close();
    7987
     88      // Various export options
     89      // ch1 + ch2
    8090      List<String> ch12 = Arrays.asList(new String[] {"intensity1", "intensity2"});
     91      // Reporter#externalId
    8192      List<String> externalId = Arrays.asList(new String[] { "rep.externalId" });
     93      // All reporter properties
    8294      List<String> allReporter = new ArrayList<String>();
    8395      for (ExtendedProperty ep : ExtendedProperties.getProperties("ReporterData"))
    … …  
    8597        allReporter.add("rep." + ep.getName());
    8698      }
     99      // ch1 + ch2 + FGMean of both channels
    87100      List<String> ch12FgMean = new ArrayList<String>();
    88101      ch12FgMean.addAll(ch12);
    89102      ch12FgMean.add("raw.ch1FgMean");
    90103      ch12FgMean.add("raw.ch2FgMean");
     104      // ch1 + ch2 + all raw data properties
    91105      List<String> ch12allRaw = new ArrayList<String>();
    92106      ch12allRaw.addAll(ch12);
    … …  
    96110      }
    97111
    98       TestUtil.write("---------------------------------------------\n");
    99       for (BioAssaySet bas : allBas)
    100       {
    101         exportBioAssaySet(bas, "S1", "basefile.serial", false, ch12, null);
    102         exportBioAssaySet(bas, "M1", "basefile.matrix", false, ch12, null);
     112      for (BioAssaySet bas : sources)
     113      {
     114        exportBioAssaySet(bas, "S1", "basefile.serial", false, ch12, null, execute);
     115        exportBioAssaySet(bas, "M1", "basefile.matrix", false, ch12, null, execute);
    103116       
    104         exportBioAssaySet(bas, "S2", "basefile.serial", false, ch12, externalId);
    105         exportBioAssaySet(bas, "M2", "basefile.matrix", false, ch12, externalId);
    106  
    107         exportBioAssaySet(bas, "S3", "basefile.serial", false, ch12, allReporter);
    108         exportBioAssaySet(bas, "M3", "basefile.matrix", false, ch12, allReporter);
    109         exportBioAssaySet(bas, "MeV", "mev.tdms", false, null, null);
    110  
    111         exportBioAssaySet(bas, "S4", "basefile.serial", false, ch12FgMean, null);
    112         exportBioAssaySet(bas, "M4", "basefile.matrix", false, ch12FgMean, null);
    113  
    114         exportBioAssaySet(bas, "S5", "basefile.serial", false, ch12allRaw, null);
    115         exportBioAssaySet(bas, "M5", "basefile.matrix", false, ch12allRaw, null);
    116  
    117         exportBioAssaySet(bas, "S6", "basefile.serial", false, ch12allRaw, allReporter);
    118         exportBioAssaySet(bas, "M6", "basefile.matrix", false, ch12allRaw, allReporter);
     117        exportBioAssaySet(bas, "S2", "basefile.serial", false, ch12, externalId, execute);
     118        exportBioAssaySet(bas, "M2", "basefile.matrix", false, ch12, externalId, execute);
     119 
     120        exportBioAssaySet(bas, "S3", "basefile.serial", false, ch12, allReporter, execute);
     121        exportBioAssaySet(bas, "M3", "basefile.matrix", false, ch12, allReporter, execute);
     122        exportBioAssaySet(bas, "MeV", "mev.tdms", false, null, null, execute);
     123 
     124        exportBioAssaySet(bas, "S4", "basefile.serial", false, ch12FgMean, null, execute);
     125        exportBioAssaySet(bas, "M4", "basefile.matrix", false, ch12FgMean, null, execute);
     126 
     127        exportBioAssaySet(bas, "S5", "basefile.serial", false, ch12allRaw, null, execute);
     128        exportBioAssaySet(bas, "M5", "basefile.matrix", false, ch12allRaw, null, execute);
     129 
     130        exportBioAssaySet(bas, "S6", "basefile.serial", false, ch12allRaw, allReporter, execute);
     131        exportBioAssaySet(bas, "M6", "basefile.matrix", false, ch12allRaw, allReporter, execute);
    119132      }
    120133      TestUtil.write("---------------------------------------------\n");
    … …  
    138151    Create a job for exporting the bioassay set.
    139152  */
    140   public static void exportBioAssaySet(BioAssaySet bioAssaySet,
     153  public static long exportBioAssaySet(BioAssaySet bioAssaySet,
    141154    String jobPrefix, String fileFormat,
    142     boolean mergeOnReporter, List<String> fields, List<String> reporters)
     155    boolean mergeOnReporter, List<String> fields, List<String> reporters,
     156    boolean execute)
    143157    throws Exception
    144158  {
    145159    String name = bioAssaySet.getName();
    146     TestUtil.write("--Add job for export " + jobPrefix + ": " + name + " to " + fileFormat + "\n");
     160    if (execute)
     161    {
     162      TestUtil.write("--Export " + jobPrefix + ": " + name + " to " + fileFormat + "\n");
     163    }
    147164    DbControl dc = TestUtil.getDbControl();
    148165    try
    … …  
    166183      dc.saveItem(job);
    167184      dc.commit();
     185      long time = 0;
     186      if (execute)
     187      {
     188        time = System.currentTimeMillis();
     189        PluginUtil.executeJob(job);
     190        time = System.currentTimeMillis() - time;
     191        TestUtil.write(job.getName() + "\t" + Values.formatTime(time / 1000) + "\n");
     192      }
     193      else
     194      {
     195        TestUtil.write("Export job created: " + job.getName() + "\n");
     196      }
     197      return time;
     198     
    168199    }
    169200    finally

  • trunk/src/test/net/sf/basedb/test/performance/FilterTest.java

    r4514 r4806  
    2222package net.sf.basedb.test.performance;
    2323
     24
     25import java.util.ArrayList;
     26import java.util.List;
    2427
    2528import net.sf.basedb.core.BioAssaySet;
    … …  
    4750  public static boolean ok = true;
    4851
     52  public static final String DEFAULT_SOURCE_PREFIX = "Root";
     53  public static final String DEFAULT_CHILD_PREFIX = "Filtered";
     54 
    4955  /**
    5056    Run the tests in this class.
    5157  */
    52   public static boolean run(String user, String password)
     58  public static boolean run(String user, String password,
     59    String sourcePrefix, String childPrefix, boolean execute)
    5360  {
    5461    DbControl dc = null;
    … …  
    5764    {
    5865      TestUtil.login(user, password, false);
     66     
     67      // Configuration options
     68      String expression = Config.getFilterExpression();
     69      if (sourcePrefix == null) sourcePrefix = DEFAULT_SOURCE_PREFIX;
     70      if (childPrefix == null) childPrefix = DEFAULT_CHILD_PREFIX;
    5971
    60       // Create raw bioassays
     72      TestUtil.write("---------------------------------------------\n");
     73      TestUtil.write("   Expression=" + expression + "\n");
     74      TestUtil.write("Source prefix=" + sourcePrefix + "\n");
     75      TestUtil.write(" Child prefix=" + childPrefix + "\n");
     76      TestUtil.write("      Execute=" + execute + "\n");
     77      TestUtil.write("---------------------------------------------\n");
     78
     79      // Load source bioassay sets
    6180      dc = TestUtil.getDbControl();
     81      Experiment experiment = PrepareTest.getExperiment(dc);
     82      List<BioAssaySet> sources = new ArrayList<BioAssaySet>(
     83          Util.findAllBioAssaySets(dc, experiment, sourcePrefix + "%"));
     84      int numSources = sources.size();
     85      if (numSources == 0)
     86      {
     87        throw new ItemNotFoundException("BioAssaySet[name like " + sourcePrefix + "]");
     88      }
     89      dc.close();
    6290     
    63       // Load prepared items
    64       Experiment experiment = PrepareTest.getExperiment(dc);
    65       String expression = Config.getFilterExpression();
    66       BioAssaySet root = Util.findBioAssaySet(dc, experiment, "Root%");
    67       long numBioAssays = root.getBioAssays().count(dc);
    68       if (root == null)
     91      // Run filter tests
     92      long minTime = Long.MAX_VALUE;
     93      long maxTime = 0;
     94      long totalTime = 0;
     95      for (int i = 0; i < numSources; ++i)
    6996      {
    70         throw new ItemNotFoundException("Root bioassayset");
     97        BioAssaySet source = sources.get(i);
     98        String filteredName = source.getName().replace(sourcePrefix, childPrefix);
     99        long time = createFilteredBioAssaySet(source, filteredName, expression, execute) / 1000;
     100        if (execute)
     101        {
     102          totalTime += time;
     103          if (time < minTime) minTime = time;
     104          if (time > maxTime) maxTime = time;
     105          TestUtil.write(filteredName + "\t" + Values.formatTime(time) + "\n");
     106        }
     107        else
     108        {
     109          TestUtil.write("--Filter job created for: " + source.getName() + "-->" +
     110              filteredName + "\n");
     111        }
    71112      }
    72113     
    73       long num = experiment.getBioAssaySets().count(dc) + 1;
    74       dc.close();
    75 
    76       long time = createFilteredBioAssaySet(root, "Filtered #" + num, expression) / 1000;
    77      
     114      if (execute)
     115      {
     116        TestUtil.write("---------------------------------------------\n");
     117        TestUtil.write("Filtered " + numSources + " bioassay sets\n");
     118        TestUtil.write("Max time: " + Values.formatTime(maxTime) + "\n");
     119        TestUtil.write("Min time: " + Values.formatTime(minTime) + "\n");
     120        TestUtil.write("Mean time: " + Values.formatTime(totalTime / numSources) + "\n");
     121      }   
    78122      TestUtil.write("---------------------------------------------\n");
    79       TestUtil.write("Source: " + root.getName() + "\n");
    80       TestUtil.write("  BioAssasys: " + numBioAssays + "\n");
    81       TestUtil.write("  Spots: " + root.getNumSpots() + "\n");
    82       TestUtil.write("  Reporters: " + root.getNumReporters() + "\n");
    83       TestUtil.write("Expression=" + expression + "\n");
    84       TestUtil.write("Create filtered bioassayset from " + root.getName() + ": " +
    85           Values.formatTime(time) + "\n");
    86       TestUtil.write("---------------------------------------------\n");
    87    
    88123      TestUtil.logout();
    89124    }
    … …  
    104139    Create a filtered bioassay set.
    105140  */
    106   public static long createFilteredBioAssaySet(BioAssaySet bioAssaySet, String name, String expression)
     141  public static long createFilteredBioAssaySet(BioAssaySet bioAssaySet,
     142    String name, String expression, boolean execute)
    107143    throws Exception
    108144  {
    109     TestUtil.write("--Create filtered bioassay set: " + name + "\n");
     145    if (execute) TestUtil.write("--Create filtered bioassay set: " + name + "\n");
    110146    DbControl dc = TestUtil.getDbControl();
    111147    try
    112     {   
     148    {
    113149      PluginDefinition plugin = PluginDefinition.getByClassName(dc, "net.sf.basedb.plugins.FormulaFilter");
     150      bioAssaySet = BioAssaySet.getById(dc, bioAssaySet.getId());
    114151      Job job = Job.getNew(dc, plugin, null, bioAssaySet.getExperiment());
    115       job.setName("PerformanceTest: Create filtered bioassay set: " + name);
     152      job.setName("PerformanceTest->Filter: " + name);
    116153      job.setSendMessage(false);
    117154      job.setRemoveJobWhenFinished(true);
    … …  
    124161      dc.saveItem(job);
    125162      dc.commit();
    126       long time = System.currentTimeMillis();
    127       PluginUtil.executeJob(job);
    128       return  System.currentTimeMillis() - time;
     163      long time = 0;
     164      if (execute)
     165      {
     166        time = System.currentTimeMillis();
     167        PluginUtil.executeJob(job);
     168        time = System.currentTimeMillis() - time;
     169      }
     170      return time;
    129171    }
    130172    finally

  • trunk/src/test/net/sf/basedb/test/performance/LowessTest.java

    r4514 r4806  
    2222package net.sf.basedb.test.performance;
    2323
     24
     25import java.util.ArrayList;
     26import java.util.List;
    2427
    2528import net.sf.basedb.core.BioAssaySet;
    … …  
    4952  public static boolean ok = true;
    5053
     54  public static final String DEFAULT_SOURCE_PREFIX = "Filtered";
     55  public static final String DEFAULT_CHILD_PREFIX = "Lowess";
     56
     57 
    5158  /**
    5259    Run the tests in this class.
    5360  */
    54   public static boolean run(String user, String password)
     61  public static boolean run(String user, String password,
     62    String sourcePrefix, String childPrefix, boolean execute)
    5563  {
    5664    DbControl dc = null;
    … …  
    6068      TestUtil.login(user, password, false);
    6169
    62       // Create raw bioassays
    63       dc = TestUtil.getDbControl();
    64      
    65       // Load prepared items
    66       Experiment experiment = PrepareTest.getExperiment(dc);
     70      // Configuration options
    6771      Float fitFraction = Values.getFloat(Config.get("lowess.fitFraction"), 0.33f);
    6872      Float delta = Values.getFloat(Config.get("lowess.delta"), 0.1f);
    6973      Integer iterations = Values.getInteger(Config.get("lowess.iterations"), 4);
    70       Integer blockGroup = Values.getInteger(Config.get("lowess.blockGroup"), 12);
    71       BioAssaySet filtered = Util.findBioAssaySet(dc, experiment, "Filtered%");
    72       long numBioAssays = filtered.getBioAssays().count(dc);
    73       if (filtered == null)
     74      Integer blockGroup = Values.getInteger(Config.get("lowess.blockGroup"), 0);
     75      if (sourcePrefix == null) sourcePrefix = DEFAULT_SOURCE_PREFIX;
     76      if (childPrefix == null) childPrefix = DEFAULT_CHILD_PREFIX;
     77     
     78      TestUtil.write("---------------------------------------------\n");
     79      TestUtil.write(" Fit fraction=" + fitFraction + "\n");
     80      TestUtil.write("        Delta=" + delta + "\n");
     81      TestUtil.write("   Iterations=" + iterations + "\n");
     82      TestUtil.write("  Block group=" + blockGroup + "\n");
     83      TestUtil.write("Source prefix=" + sourcePrefix + "\n");
     84      TestUtil.write(" Child prefix=" + childPrefix + "\n");
     85      TestUtil.write("      Execute=" + execute + "\n");
     86      TestUtil.write("---------------------------------------------\n");
     87
     88      // Load source bioassay sets
     89      dc = TestUtil.getDbControl();
     90      Experiment experiment = PrepareTest.getExperiment(dc);
     91      List<BioAssaySet> sources = new ArrayList<BioAssaySet>(
     92          Util.findAllBioAssaySets(dc, experiment, sourcePrefix + "%"));
     93      int numSources = sources.size();
     94      if (numSources == 0)
    7495      {
    75         throw new ItemNotFoundException("Filtered bioassayset");
     96        throw new ItemNotFoundException("BioAssaySet[name like " + sourcePrefix + "]");
    7697      }
    77      
    78      
    79       long num = experiment.getBioAssaySets().count(dc) + 1;
    8098      dc.close();
    8199
    82       long time = createNormalizedBioAssaySet(filtered, "LOWESS #" + num, fitFraction, delta, iterations, blockGroup) / 1000;
    83      
     100      // Run Lowess tests
     101      long minTime = Long.MAX_VALUE;
     102      long maxTime = 0;
     103      long totalTime = 0;
     104      for (int i = 0; i < numSources; ++i)
     105      {
     106        BioAssaySet source = sources.get(i);
     107        String lowessName = source.getName().replace(sourcePrefix, childPrefix);
     108        long time = createNormalizedBioAssaySet(source, lowessName,
     109            fitFraction, delta, iterations, blockGroup, execute) / 1000;
     110        if (execute)
     111        {
     112          totalTime += time;
     113          if (time < minTime) minTime = time;
     114          if (time > maxTime) maxTime = time;
     115          TestUtil.write(lowessName + "\t" + Values.formatTime(time) + "\n");
     116        }
     117        else
     118        {
     119          TestUtil.write("--Lowess job created for: " + source.getName() + "-->" +
     120              lowessName + "\n");
     121        }
     122      }
     123
     124      if (execute)
     125      {
     126        TestUtil.write("---------------------------------------------\n");
     127        TestUtil.write("Max time: " + Values.formatTime(maxTime) + "\n");
     128        TestUtil.write("Min time: " + Values.formatTime(minTime) + "\n");
     129        TestUtil.write("Mean time: " + Values.formatTime(totalTime / numSources) + "\n");
     130      }
    84131      TestUtil.write("---------------------------------------------\n");
    85       TestUtil.write("Source: " + filtered.getName() + "\n");
    86       TestUtil.write("  BioAssasys: " + numBioAssays + "\n");
    87       TestUtil.write("  Spots: " + filtered.getNumSpots() + "\n");
    88       TestUtil.write("  Reporters: " + filtered.getNumReporters() + "\n");
    89       TestUtil.write("Fit fraction=" + fitFraction + "\n");
    90       TestUtil.write("Delta=" + delta + "\n");
    91       TestUtil.write("Iterations=" + iterations + "\n");
    92       TestUtil.write("Block group=" + blockGroup + "\n");
    93       TestUtil.write("Create LOWESS bioassayset from " + filtered.getName() + ": " +
    94           Values.formatTime(time) + "\n");
    95       TestUtil.write("---------------------------------------------\n");
    96    
    97132      TestUtil.logout();
    98133    }
    … …  
    114149  */
    115150  public static long createNormalizedBioAssaySet(BioAssaySet bioAssaySet, String name,
    116     Float fitFraction, Float delta, Integer iterations, Integer blockGroup)
     151    Float fitFraction, Float delta, Integer iterations, Integer blockGroup, boolean execute)
    117152    throws Exception
    118153  {
    119     TestUtil.write("--Create LOWESS bioassay set: " + name + "\n");
     154    if (execute) TestUtil.write("--Create LOWESS bioassay set: " + name + "\n");
    120155    DbControl dc = TestUtil.getDbControl();
    121156    try
    122157    {   
    123158      PluginDefinition plugin = PluginDefinition.getByClassName(dc, "net.sf.basedb.plugins.LowessNormalization");
     159      bioAssaySet = BioAssaySet.getById(dc, bioAssaySet.getId());
    124160      Job job = Job.getNew(dc, plugin, null, bioAssaySet.getExperiment());
    125       job.setName("PerformanceTest: Create LOWESS bioassay set: " + name);
     161      job.setName("PerformanceTest->Lowess: " + name);
    126162      job.setSendMessage(false);
    127163      job.setRemoveJobWhenFinished(true);
    … …  
    136172      dc.saveItem(job);
    137173      dc.commit();
    138       long time = System.currentTimeMillis();
    139       PluginUtil.executeJob(job);
    140       return  System.currentTimeMillis() - time;
     174      long time = 0;
     175      if (execute)
     176      {
     177        time = System.currentTimeMillis();
     178        PluginUtil.executeJob(job);
     179        time = System.currentTimeMillis() - time;
     180      }
     181      return  time;
    141182    }
    142183    finally

  • trunk/src/test/net/sf/basedb/test/performance/RawDataTest.java

    r4805 r4806  
    5252  public static boolean ok = true;
    5353
     54  public static final String DEFAULT_CHILD_PREFIX = "Raw";
     55 
    5456  /**
    5557    Run the tests in this class.
    5658  */
    57   public static boolean run(String user, String password, int numIterations)
     59  public static boolean run(String user, String password, String namePrefix,
     60      int numIterations, boolean execute)
    5861  {
    5962    DbControl dc = null;
    … …  
    7780      long startNum = experiment.getRawBioAssays().count(dc);
    7881      RawBioAssay[] rba = new RawBioAssay[numIterations];
     82      if (namePrefix == null) namePrefix = DEFAULT_CHILD_PREFIX;
    7983      for (int i = 0; i < numIterations; ++i)
    8084      {
    8185        String rbaNum = Long.toString(startNum + i);
    8286        rbaNum = MD5.leftPad(rbaNum, '0', 3);
    83         rba[i] = createRawBioAssay(dc, generic, genepix, "PerformanceTest #" + rbaNum, design);
     87        rba[i] = createRawBioAssay(dc, generic, genepix, namePrefix + "#" + rbaNum, design);
    8488      }
    8589      dc.commit();
    … …  
    9195      for (int i = 0; i < numIterations; i += 2)
    9296      {
    93         TestUtil.gc();
    94         long time = importRawData(rawDataImporter, rawData1, rba[i]) / 1000;
    95         totalTime += time;
    96         if (time < minTime) minTime = time;
    97         if (time > maxTime) maxTime = time;
    98         TestUtil.write("Import #" + i + "\t" + Values.formatTime(time) + "\n");
    99 
    100         time = importRawData(rawDataImporter, rawData2, rba[i+1]) / 1000;
    101         totalTime += time;
    102         if (time < minTime) minTime = time;
    103         if (time > maxTime) maxTime = time;
    104         TestUtil.write("Import #" + (i + 1) + "\t" + Values.formatTime(time) + " s\n");
     97        long time = importRawData(rawDataImporter, rawData1, rba[i], execute) / 1000;
     98        if (execute)
     99        {
     100          totalTime += time;
     101          if (time < minTime) minTime = time;
     102          if (time > maxTime) maxTime = time;
     103          TestUtil.write("Import #" + i + "\t" + Values.formatTime(time) + "\n");
     104        }
     105        else
     106        {
     107          TestUtil.write("--Import job created for: " + rba[i].getName() + "\n");
     108        }
     109       
     110        time = importRawData(rawDataImporter, rawData2, rba[i+1], execute) / 1000;
     111        if (execute)
     112        {
     113          totalTime += time;
     114          if (time < minTime) minTime = time;
     115          if (time > maxTime) maxTime = time;
     116          TestUtil.write("Import #" + (i + 1) + "\t" + Values.formatTime(time) + " s\n");
     117        }
     118        else
     119        {
     120          TestUtil.write("--Import job created for: " + rba[i+1].getName() + "\n");
     121        }
    105122      }
    106123      TestUtil.write("---------------------------------------------\n");
    107       TestUtil.write("Imported " + numIterations + " raw data sets\n");
    108       TestUtil.write("Max time: " + Values.formatTime(maxTime) + "\n");
    109       TestUtil.write("Min time: " + Values.formatTime(minTime) + "\n");
    110       TestUtil.write("Mean time: " + Values.formatTime(totalTime / numIterations) + "\n");
    111       TestUtil.write("---------------------------------------------\n");
     124      if (execute)
     125      {
     126        TestUtil.write("Imported " + numIterations + " raw data sets\n");
     127        TestUtil.write("Max time: " + Values.formatTime(maxTime) + "\n");
     128        TestUtil.write("Min time: " + Values.formatTime(minTime) + "\n");
     129        TestUtil.write("Mean time: " + Values.formatTime(totalTime / numIterations) + "\n");
     130        TestUtil.write("---------------------------------------------\n");
     131      }
    112132     
    113133      // Add raw bioassays to experiment
    … …  
    120140      }
    121141      dc.commit();
    122      
    123142      TestUtil.logout();
    124143    }
    … …  
    150169
    151170  /**
    152     Import raw data.
     171    Import raw data. The job is only executed if 'execute=true'.
     172    @return The time (in milliseconds) used to execute the job, or
     173      0 if the job wasn't executed
    153174  */
    154   public static long importRawData(PluginConfiguration config, File file, RawBioAssay rba)
     175  public static long importRawData(PluginConfiguration config, File file,
     176    RawBioAssay rba, boolean execute)
    155177    throws Exception
    156178  {
    157     TestUtil.write("--Importing raw data from file: " + file.getName() + "\n");
     179    if (execute) TestUtil.write("--Importing raw data from file: " + file.getName() + "\n");
    158180    DbControl dc = TestUtil.getDbControl();
    159181    try
    … …  
    161183      config = PluginConfiguration.getById(dc, config.getId());
    162184      Job job = Job.getNew(dc, config.getPluginDefinition(), config, null);
    163       job.setName("PerformanceTest: Import data to raw bioassay: " + rba.getName());
     185      job.setName("PerformanceTest->Import: " + rba.getName());
    164186      job.setSendMessage(false);
    165187      job.setRemoveJobWhenFinished(true);
    … …  
    171193      dc.saveItem(job);
    172194      dc.commit();
    173       long time = System.currentTimeMillis();
    174       PluginUtil.executeJob(job);
    175       return  System.currentTimeMillis() - time;
     195      long time = 0;
     196      if (execute)
     197      {
     198        time = System.currentTimeMillis();
     199        if (execute) PluginUtil.executeJob(job);
     200        time = System.currentTimeMillis() - time;
     201      }
     202      return time;
    176203    }
    177204    finally

  • trunk/src/test/net/sf/basedb/test/performance/RootTest.java

    r4514 r4806  
    2222package net.sf.basedb.test.performance;
    2323
     24import java.util.ArrayList;
    2425import java.util.List;
    2526
    … …  
    2728import net.sf.basedb.core.Experiment;
    2829import net.sf.basedb.core.Formula;
     30import net.sf.basedb.core.ItemNotFoundException;
    2931import net.sf.basedb.core.ItemParameterType;
    3032import net.sf.basedb.core.ItemQuery;
    … …  
    3436import net.sf.basedb.core.RawDataType;
    3537import net.sf.basedb.core.StringParameterType;
     38import net.sf.basedb.core.query.Expressions;
     39import net.sf.basedb.core.query.Hql;
     40import net.sf.basedb.core.query.Orders;
     41import net.sf.basedb.core.query.Restrictions;
    3642import net.sf.basedb.test.PluginUtil;
    3743import net.sf.basedb.test.TestUtil;
    … …  
    4753public class RootTest
    4854{
    49 
    5055  public static boolean ok = true;
    51    
     56  public static final String DEFAULT_CHILD_PREFIX = "Root";
     57 
    5258  /**
    5359    Run the tests in this class.
    5460  */
    55   public static boolean run(String user, String password, int numItems)
     61  public static boolean run(String user, String password,
     62      String sourcePrefix, String childPrefix,
     63      int iterations, int numSourcesStart, int numSourcesEnd, boolean execute)
    5664  {
    5765    DbControl dc = null;
    … …  
    6169      TestUtil.login(user, password, false);
    6270
    63       // Create raw bioassays
    6471      dc = TestUtil.getDbControl();
    6572     
    66       // Load prepared items
     73      // Configuration options
     74      if (childPrefix == null) childPrefix = DEFAULT_CHILD_PREFIX;
     75
    6776      Experiment experiment = PrepareTest.getExperiment(dc);
    6877      RawDataType rawDataType = experiment.getRawDataType();
    6978      Formula formula = Util.findFormula(dc, rawDataType,
    70         Formula.Type.INTENSITY_EXPRESSION, Config.getIntensityFormula(rawDataType.getId()));
    71       List<String> expressions = formula.getFormulas();
    72       long num = experiment.getBioAssaySets().count(dc) + 1;
     79          Formula.Type.INTENSITY_EXPRESSION, Config.getIntensityFormula(rawDataType.getId()));
     80        List<String> expressions = formula.getFormulas();
     81      TestUtil.write("---------------------------------------------\n");
     82      TestUtil.write("      formula = " + formula.getName()+"\n");
     83      TestUtil.write("        ch(1) = " + expressions.get(0) + "\n");
     84      TestUtil.write("        ch(2) = " + expressions.get(1) + "\n");
     85      TestUtil.write("Source prefix = " + sourcePrefix + "\n");
     86      TestUtil.write(" Child prefix = " + childPrefix + "\n");
     87      TestUtil.write("      Execute = " + execute + "\n");
     88      TestUtil.write("---------------------------------------------\n");
     89     
     90      // Load source raw bioassays
     91      ItemQuery<RawBioAssay> rawQuery = experiment.getRawBioAssays();
     92      if (sourcePrefix != null)
     93      {
     94        // Only use raw bioassay sets starting with 'sourcePrefix'
     95        rawQuery.restrict(Restrictions.like(
     96            Hql.property("name"), Expressions.string(sourcePrefix + "%")));
     97      }
     98      rawQuery.order(Orders.asc(Hql.property("name")));
     99      List<RawBioAssay> sources = new ArrayList<RawBioAssay>(rawQuery.list(dc));
     100      int numSources = sources.size();
     101      if (numSources == 0)
     102      {
     103        throw new ItemNotFoundException("RawBioAssay set[name like " + sourcePrefix + "]");
     104      }
    73105      dc.close();
    74 
    75       long time = createRootBioAssaySet(experiment, formula, "Root #" + num, numItems) / 1000;
    76      
    77       TestUtil.write("---------------------------------------------\n");
    78       TestUtil.write("formula=" + formula.getName()+"\n");
    79       TestUtil.write("  ch(1)=" + expressions.get(0) + "\n");
    80       TestUtil.write("  ch(2)=" + expressions.get(1) + "\n");
    81       TestUtil.write("Create root bioassayset with " + numItems + " raw data sets: " +
    82           Values.formatTime(time) + "\n");
     106     
     107      // Run tests
     108      long minTime = Long.MAX_VALUE;
     109      long maxTime = 0;
     110      long totalTime = 0;
     111      int firstIndex = 0;
     112      for (int i = 1; i <= iterations; ++i)
     113      {
     114        int useSources = numSourcesStart;
     115        while (useSources <= numSourcesEnd && useSources <= sources.size())
     116        {
     117          String childName = childPrefix + "." + i + "#" + useSources;
     118          if (firstIndex + useSources >= numSources)
     119          {
     120            firstIndex = 0;
     121          }
     122          List<RawBioAssay> subSources = sources.subList(firstIndex, firstIndex + useSources);
     123          long time = createRootBioAssaySet(experiment, subSources, formula, childName, execute) / 1000;
     124          firstIndex += useSources;
     125          useSources *= 2;
     126          if (execute)
     127          {
     128            totalTime += time;
     129            if (time < minTime) minTime = time;
     130            if (time > maxTime) maxTime = time;
     131            TestUtil.write(childName + "\t" + Values.formatTime(time) + "\n");
     132          }
     133          else
     134          {
     135            TestUtil.write("--Job created for: " + childName + "\n");
     136          }
     137        }
     138      }
     139     
     140      if (execute)
     141      {
     142        TestUtil.write("---------------------------------------------\n");
     143        TestUtil.write("Max time: " + Values.formatTime(maxTime) + "\n");
     144        TestUtil.write("Min time: " + Values.formatTime(minTime) + "\n");
     145      }
    83146      TestUtil.write("---------------------------------------------\n");
    84147   
    … …  
    101164    Create a root bioassay set.
    102165  */
    103   public static long createRootBioAssaySet(Experiment experiment, Formula formula, String name, int numRaw)
     166  public static long createRootBioAssaySet(Experiment experiment, List<RawBioAssay> sources,
     167    Formula formula, String name, boolean execute)
    104168    throws Exception
    105169  {
    106     TestUtil.write("--Create root bioassay set: " + name + "\n");
     170    if (execute) TestUtil.write("--Create root bioassay set: " + name + "\n");
    107171    DbControl dc = TestUtil.getDbControl();
    108172    try
    109173    {
    110       ItemQuery<RawBioAssay> sources = experiment.getRawBioAssays();
    111       sources.setMaxResults(numRaw);
    112      
    113174      PluginDefinition plugin = PluginDefinition.getByClassName(dc, "net.sf.basedb.plugins.IntensityCalculatorPlugin");
    114175      Job job = Job.getNew(dc, plugin, null, experiment);
    115       job.setName("PerformanceTest: Create root bioassay set: " + name);
     176      job.setName("PerformanceTest->Intensity: " + name);
    116177      job.setSendMessage(false);
    117       job.setRemoveJobWhenFinished(true);
    118       job.setParameterValue("name",
    119         new StringParameterType(255, null, true), name);
    120       job.setParameterValue("experiment",
    121         new ItemParameterType<Experiment>(Experiment.class, null), experiment);
    122       job.setParameterValues("rawBioAssays",
    123         new ItemParameterType<RawBioAssay>(RawBioAssay.class, null, true, 0, null),
    124         sources.list(dc));
    125       job.setParameterValue("formula",
    126         new ItemParameterType<Formula>(Formula.class, null), formula);
     178      job.setRemoveJobWhenFinished(false);
     179      job.setParameterValue("name", new StringParameterType(255, null, true),
     180        name);
     181      job.setParameterValue("experiment", new ItemParameterType<Experiment>(Experiment.class, null),
     182        experiment);
     183      job.setParameterValues("rawBioAssays", new ItemParameterType<RawBioAssay>(RawBioAssay.class, null, true, 0, null),
     184        sources);
     185      job.setParameterValue("formula", new ItemParameterType<Formula>(Formula.class, null),
     186        formula);
    127187      dc.saveItem(job);
    128188      dc.commit();
    129       long time = System.currentTimeMillis();
    130       PluginUtil.executeJob(job);
    131       time = System.currentTimeMillis() - time;
     189      long time = 0;
     190      if (execute)
     191      {
     192        time = System.currentTimeMillis();
     193        PluginUtil.executeJob(job);
     194        time = System.currentTimeMillis() - time;
     195      }
    132196      return time;
    133197    }

  • trunk/src/test/net/sf/basedb/test/performance/Run.java

    r4803 r4806  
    2222package net.sf.basedb.test.performance;
    2323
    24 import java.util.Arrays;
    25 import java.util.HashSet;
    26 import java.util.Set;
    27 
    2824import net.sf.basedb.test.CmdLine;
    2925import net.sf.basedb.test.TestUtil;
    … …  
    4339   
    4440    CmdLine cmdLine = new CmdLine(args);
     41    String cmd = cmdLine.getCmd();
    4542
    46     Set<String> cmds = new HashSet<String>(Arrays.asList(args));
    47     boolean all = cmds.contains("all");
    48    
    4943    String user = cmdLine.getOption("-u", "root");
    5044    String password = cmdLine.getOption("-p", "root");
    51     int numItems = Values.getInt(cmdLine.getOption("-n"), 4);
     45    boolean execute = cmdLine.hasOption("-x");
     46    int numIterations = Values.getInt(cmdLine.getOption("-i"), 1);
     47    String sourceNamePrefix = cmdLine.getOption("-s", null);
     48    String childNamePrefix = cmdLine.getOption("-c", null);
    5249
    5350    int tests = 0;
    5451    TestUtil.begin(false, true);
    5552    boolean ok = PrepareTest.run(user, password);
    56     if (ok && (all || cmds.contains("raw")))
     53    if ("raw".equals(cmd))
    5754    {
    58       ok = RawDataTest.run(user, password, numItems);
     55      ok = RawDataTest.run(user, password, childNamePrefix, numIterations, execute);
    5956      tests++;
    6057    }
    61     if (ok && (all || cmds.contains("root")))
     58    else if ("root".equals(cmd))
    6259    {
    63       ok = RootTest.run(user, password, numItems);
     60      int numSourcesStart = Values.getInt(cmdLine.getOption("-mi"), 1);
     61      int numSourcesEnd = Values.getInt(cmdLine.getOption("-mx"), 1);
     62      ok = RootTest.run(user, password, sourceNamePrefix, childNamePrefix, numIterations,
     63          numSourcesStart, numSourcesEnd, execute);
    6464      tests++;
    6565    }
    66     if (ok && (all || cmds.contains("filter")))
     66    else if ("filter".equals(cmd))
    6767    {
    68       ok = FilterTest.run(user, password);
     68      ok = FilterTest.run(user, password, sourceNamePrefix, childNamePrefix, execute);
    6969      tests++;
    7070    }
    71     if (ok && (all || cmds.contains("lowess")))
     71    else if ("lowess".equals(cmd))
    7272    {
    73       ok = LowessTest.run(user, password);
     73      ok = LowessTest.run(user, password, sourceNamePrefix, childNamePrefix, execute);
    7474      tests++;
    7575    }
    76     if (ok && (all || cmds.contains("export")))
     76    else if ("export".equals(cmd))
    7777    {
    78       String bioAssaySet = cmdLine.getOption("-b", null);
    79       if (bioAssaySet != null)
    80       {
    81         ok = ExportTest.run(user, password, bioAssaySet);
    82         tests++;
    83       }
    84       else
    85       {
    86         TestUtil.write("No name for bioassay set specified. Use -b option!\n\n");
    87       }
     78      ok = ExportTest.run(user, password, sourceNamePrefix, execute);
     79      tests++;
    8880    }
    89     if (ok && cmds.contains("clean"))
     81    else if ("clean".equals(cmd))
    9082    {
    9183      ok = PrepareTest.cleanUp(user, password);
    … …  
    10496  {
    10597    TestUtil.write(
    106       "test.sh performance [-n xxx] [-u user] [-p password] [-b bioassayset] <cmd>\n" +
    107       "     -n = number of items to import; default = 100\n" +
    108       "     -u = username; default = root\n" +
    109       "     -p = password; default = root\n" +
    110       "     -b = Name of bioassay set to export; no default\n" +
    111       "     cmd = the tests to run; all, raw\n\n");
     98      "test.sh performance [-option value] <cmd>\n" +
     99      "    cmd = the test to run; raw, root, filter, lowess, export, clean\n\n" +
     100      "global options:\n" +
     101      "     -u = username (default=root)\n" +
     102      "     -p = password (default=root)\n" +
     103      "     -x = if present, jobs are executed immediately\n" +
     104      "     -s = Source prefix; default varies by cmd\n" +
     105      "     -c = Child prefix; default varies by cmd\n" +
     106      "raw:\n" +
     107      "     -i = Number of raw data sets to import\n"+
     108      "     -c = Name prefix for created raw bioassays (default=Raw)\n"+
     109      "root:\n"+
     110      "     -s = Only use raw bioassays that matches the prefix (default=null)\n" +
     111      "     -c = Prefix to use for created bioassay sets (default=Root)\n" +
     112      "     -i = Number of root:s to to create for each iteration (default=1)\n"+
     113      "    -mi = Number of raw bioassays to use in the first iteration (default=1)\n"+
     114      "    -mx = Number of raw bioassay to use in the last iteration (default=1)\n"+
     115      "          Each iteration doubles the number of raw bioassays\n"+
     116      "filter:\n"+
     117      "     -s = Filter all bioassay sets that matches the prefix (default=Root)\n"+
     118      "     -c = Prefix to use for created bioassay sets (default=Filtered)\n"+
     119      "lowess:\n" +
     120      "     -s = Normalize all bioassay sets that matches the prefix (default=Filtered)\n"+
     121      "     -c = Prefix to use for created bioassay sets (default=Lowess)\n"+
     122      "export:\n"+
     123      "     -s = Export all bioassay sets that matches the prefix (no default)\n"+
     124      "\n"
     125    );
    112126   
    113127  }

  • trunk/src/test/net/sf/basedb/test/performance/Util.java

    r4803 r4806  
    129129  public static List<BioAssaySet> findAllBioAssaySets(DbControl dc, Experiment experiment, String name)
    130130  {
    131     TestUtil.write("--Loading bioassaysets: " + name);
     131    TestUtil.write("--Loading bioassaysets: " + name +"\n");
    132132    ItemQuery<BioAssaySet> query = experiment.getBioAssaySets();
    133133    query.restrict(Restrictions.like(Hql.property("name"), Expressions.string(name)));
Note: See TracChangeset for help on using the changeset viewer.