Changeset 2165

Timestamp:

Dec 9, 2013, 12:23:59 PM (9 years ago)

Author:

olle

Message:

Refs #228. Refs #541. Quantile normalization updated to untransform logarithmic data before normalization, and then transform it back to logarithmic format before results are stored. Default averaging method will always be set to Formula.AverageMethod.GEOMETRIC_MEAN:

1. Class/file QuantileNormalizer.java in src/net/sf/basedb/plugins/ in package net.sf.basedb.normalizers updates:
   a. Private method RequestInformation getConfiguredJobParameters() updated to always set the default averaging method to Formula.AverageMethod.GEOMETRIC_MEAN.
   b. Private method BioAssaySet normalize(DbControl dc, BioAssaySet source, Job job, ProgressReporter progress) updated to call public method double transform(double value) in class IntensityTransform to transform the normalized result back before storing it. Error message when number of spots differ between two BioAssay sets updated to report the names of the latter and the number of spots in each one. Also minor updates in order to increase clarity of code.
2. Class/file AbstractNormalizationPlugin.java in src/net/sf/basedb/plugins/ in package net.sf.basedb.normalizers updated in help text for selecting averaging method by avoiding reference to the format data is stored in, since averaging is performed on untransformed data.
3. XML files extensions.xml in META-INF in package net.sf.basedb.normalizers updated in help text for Quantile normalization by adding information that data stored in logarithmic format will be untransformed before averaging, and then transformed back to logarithmic format before results are stored.

Location:

plugins/base2/net.sf.basedb.normalizers/trunk

Files:

• META-INF/extensions.xml (modified) (5 diffs)
• src/net/sf/basedb/plugins/AbstractNormalizationPlugin.java (modified) (1 diff)
• src/net/sf/basedb/plugins/QuantileNormalization.java (modified) (8 diffs)

plugins/base2/net.sf.basedb.normalizers/trunk/META-INF/extensions.xml

@@ -11 +11 @@
     </description>
     <version>1.1-beta</version>
-    <min-base-version>3.0.0</min-base-version>
+    <min-base-version>3.2.4</min-base-version>
     <copyright>BASE development team</copyright>
     <email>basedb-users@lists.sourceforge.net</email>
@@ -30 +30 @@
         
         The new expression values will become "S" times the original 
-        expression value. Background subtraction and proper filtration 
+        expression value. Background subtraction and proper filtration 
         have to be done before running this plug-in. 
         
@@ -52 +52 @@
         values are replaced with the row average value. Finally, each assay 
         is reordered into its original order to retain a standard 
-        expression matrix were each row represents one probe. Assays are 
+        expression matrix where each row represents one probe. Assays are 
         not mixed.
         
@@ -58 +58 @@
         bioassay set before running this plug-in. The bioassay set must not 
         contain any missing values.
+        
+        Data stored in logarithmic format will be untransformed before averaging,
+        and then transformed back to logarithmic format before results are stored.
         
         This plug-in supports 1-channel and 2-channel data.
@@ -111 +114 @@
       <name>Rank invariant normalization</name>
       <description>
-        The development of this plug-in is still in progress
+        The development of this plug-in is still in progress.
+        
+        This plug-in currently only supports 1-channel data.
       </description>      
     </about>    

plugins/base2/net.sf.basedb.normalizers/trunk/src/net/sf/basedb/plugins/AbstractNormalizationPlugin.java

@@ -71 +71 @@
     "averageMethod",
     "Average calculation method",
-    "Select which method to use when calculating averages.\n" +
-    geometricOption +
-    " is default for non-logarithmic values and " + arithmeticOption +
-    " is default for logarithmic values.",
+    "Select which method to use when calculating averages.",
     null
   );  

plugins/base2/net.sf.basedb.normalizers/trunk/src/net/sf/basedb/plugins/QuantileNormalization.java

@@ -70 +70 @@
    values are replaced with the row average value. Finally, each assay
    is reordered into its original order to retain a standard
-   expression matrix were each row represents one probe. Assays are
+   expression matrix where each row represents one probe. Assays are
    not mixed.
 
@@ -252 +252 @@
         
         // Average normalization options
-        Formula.AverageMethod defaultAverageMethod = Formula.AverageMethod.ARITHMETIC_MEAN;
-        IntensityTransform transform = bas.getIntensityTransform();
-        if (transform == IntensityTransform.NONE)
-        {
-          defaultAverageMethod = Formula.AverageMethod.GEOMETRIC_MEAN;
-        }
+        // Default is always set to geometric mean, since averaging is performed on untransformed data 
+        Formula.AverageMethod defaultAverageMethod = Formula.AverageMethod.GEOMETRIC_MEAN;
         StringParameterType spt = new StringParameterType
         (
@@ -301 +297 @@
     int noOfChannels = source.getRawDataType().getChannels();
     long normalizedSpots = 0;
+    IntensityTransform transform = source.getIntensityTransform();
         
     // Get query to recieve the spot data.
@@ -325 +322 @@
     String configuredAverageMethod = (String)job.getParameterValue(averageMethodParameter.getName());
     int spotsPerAssay = -1;
+    String refAssayName = "";
     
     for (BioAssay assay : assays)
@@ -332 +330 @@
       // Control that the number of spots per assay is the same in all assays
       if (spotsPerAssay > -1 && spotsPerAssay != assay.getNumSpots())
-        throw new BaseException("The number of spots are not equal between the dispal\n " +
+      {
+        throw new BaseException("The number of spots for assay '" + assay.getName() + "' (" + assay.getNumSpots() + ") is not equal to that for assay '" + refAssayName + "' (" + spotsPerAssay + ")\n " +
             "The normalization can not be done.");
+      }
       else if (spotsPerAssay == -1)
+      {
+        // First BioAssay, save data for check and possible error message
         spotsPerAssay = assay.getNumSpots();
+        refAssayName = assay.getName();
+      }
       
       // Get spot data and sort it ascending
@@ -342 +346 @@
       
       // Write spot data to file and calculate row summaries
-      if (rowCalculators == null) rowCalculators = new AverageCalculator[data.size()];
+      if (rowCalculators == null)
+      {
+        rowCalculators = new AverageCalculator[data.size()];
+      }
       FileWriter fw = null;
       try
@@ -352 +359 @@
         {
           AbstractSpotData spot = data.get(i);
-          if (rowCalculators[i] == null) rowCalculators[i] = new AverageCalculator(configuredAverageMethod);
+          if (rowCalculators[i] == null)
+          {
+            rowCalculators[i] = new AverageCalculator(configuredAverageMethod);
+          }
           rowCalculators[i].addNumber(spot.getNormalizableData());
           
@@ -413 +423 @@
           AbstractSpotData spot = getSpotDataFromString(lineInfo);
           spot.setNormalizableData(rowCalculators[rowIndex].getAverage());
-          batcher.insert(columnNo, spot.getPosition(), spot.getChannelData());
+          // Average is calculated for untransformed data, transform channel data before storing
+          float[] channelDataArr = spot.getChannelData();
+          float[] transformedChannelDataArr = new float[channelDataArr.length];
+          for (int i=0; i < channelDataArr.length; i++)
+          {
+            // d1 = untransormed value, d2 = transformed value (0 if d1 is out of range)
+            float d1 = channelDataArr[i];
+            float d2 = 0F;
+            if (!Float.isNaN(d1) && !Float.isInfinite(d1))
+            {
+              d2 = (float) transform.transform(d1);
+            }
+            transformedChannelDataArr[i] = d2;
+          }
+          batcher.insert(columnNo, spot.getPosition(), transformedChannelDataArr);
           normalizedSpots++;
           rowIndex++;
         }
+        child.setIntensityTransform(transform);
         // Clean up
         input.close();