source: trunk/doc/src/docbook/appendix/incompatible.xml @ 5060

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE appendix PUBLIC 
    "-//Dawid Weiss//DTD DocBook V3.1-Based Extension for XML and graphics inclusion//EN" 
    "../../../../lib/docbook/preprocess/dweiss-docbook-extensions.dtd">
<!--
  $Id: incompatible.xml 5060 2009-08-19 07:02:11Z nicklas $
  
  Copyright (C) 2007 Peter Johansson, Nicklas Nordborg
  
  This file is part of BASE - BioArray Software Environment.
  Available at http://base.thep.lu.se/
  
  BASE is free software; you can redistribute it and/or
  modify it under the terms of the GNU General Public License
  as published by the Free Software Foundation; either version 3
  of the License, or (at your option) any later version.
  
  BASE is distributed in the hope that it will be useful,
  but WITHOUT ANY WARRANTY; without even the implied warranty of
  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
  GNU General Public License for more details.
  
  You should have received a copy of the GNU General Public License
  along with BASE. If not, see <http://www.gnu.org/licenses/>.
-->

<appendix id="appendix.incompatible">
  <title>API changes that may affect backwards compatibility</title>
  <para>
    In this document we list all changes to code in the <emphasis>Public API</emphasis>
    that may be backwards incompatible with existing client applications
    and or plug-ins. See <xref linkend="api_overview.public_api" /> for more
    information about what we mean with the <emphasis>Public API</emphasis>
    and backwards compatible.
  </para>
  
  <note>
    <para>
    There is no history for releases prior to BASE 2.2 because
    we did not actively keep track of them. We believe that if such
    changes exists, they are minor and does not affect many plug-ins
    since in those days very few 3rd-party plug-ins existed.
    </para>
  </note>
  
  <sect1 id="appendix.incompatible.2.13">
    <title>BASE 2.13 release</title>

    <bridgehead>
      Reporters, Raw data, Features and Array design blocks are now proxied
    </bridgehead>
    
    <para>
      This was previously not possible due to a Hibernate problem with
      stateless sessions. See <ulink 
      url="http://opensource.atlassian.com/projects/hibernate/browse/HHH-3528"
      >http://opensource.atlassian.com/projects/hibernate/browse/HHH-3528</ulink>
      for more information about this problem.
    </para>
    
    <para>
      This change means that items linking to reporter, raw data, feature or array design
      blocks will no longer load the linked items automatically. This is usually 
      not a problem since the proxies will be initialised if needed. The exception
      is when a stateless session was used to create the proxy since the stateless
      can't initialise proxies. In BASE, stateless sessions are only used by <classname
      docapi="net.sf.based.core">DataQuery</classname> instances, eg. queries that
      returns reporter, raw data or features. When this type of query is used and
      when linked items are used in a way that causes proxy initialization the linked 
      item must be explicitely FETCH JOIN-ed by the query. Here is an example:
    </para>
    
    <programlisting language="java"><![CDATA[ 
RawBioAssay rba = ...
DataQuery<RawData> rawQuery = rba.getRawData();
rawQuery.join(
  Hql.leftJoin(null, "reporter", Item.REPORTER.getAlias(), null, true));
// NOTE! Last parameter is 'true' to FETCH JOIN the reporter!!
...
DbControl dc = ...
DataResultIterator<RawData> rawData = rawQuery.iterate(dc);
while (rawData.hasNext())
{
  RawData rd = rawData.next();
  ReporterData reporter = rd.getReporter();
  int reporterId = reporter.getId();
  // Always safe since getId() doesn't cause proxy initialization

  reporter.getName();
  // The above statement will fail in BASE 2.13 if 
  // the FETCH JOIN is no included
  ....
}]]>
</programlisting>
  
    <para>
      The error message to look out for is:
      <code> 
      org.hibernate.SessionException: proxies cannot be fetched by a stateless session
      </code>
    </para>

    <bridgehead>
      Re-attaching a detached item no longer assumes that it has been modified
    </bridgehead>
    
    <para>
      The <code>DbControl.reattachItem(BasicItem)</code> method has been deprected
      and replaced with <code>DbControl.reattachItem(BasicItem, boolean)</code>.
      The boolean parameter is used to tell BASE if the item has been modified 
      or not while it was detached from the session. The previous behaviour of
      <code>DbControl.reattachItem(BasicItem)</code> was equivalent to
      <code>DbControl.reattachItem(BasicItem, true)</code>, but this is now
      changed to <code>DbControl.reattachItem(BasicItem, false)</code> since
      in most cases there are really no changes to the item. The problem with
      the old behaviour was seen in the new (in BASE 2.13) change history 
      functionality which would create UPDATE events even when there was no 
      change.
    </para>

  </sect1>
  
  <sect1 id="appendix.incompatible.2.12">
    <title>BASE 2.12 release</title>
    
    <bridgehead>
      Log-2 and log-10 transformed spot intensity data is now allowed
    </bridgehead>
    <para>
      Prior versions of BASE only allowed unlogged spot intensity values.
      Analysis plug-ins that operate on spot data should be updated to
      check the kind of values that are present in the source bioassay set
      and either:
    </para>
    <itemizedlist>
    <listitem>
      <para>
      Use an appropriate algorithm if it encounters logged data
      </para>
    </listitem>
    <listitem>
      <para>
      Give an error message that says that it requires unlogged data
      </para>
    </listitem>
    </itemizedlist>
    
    <para>
      Plug-ins that are not aware of the type of data may produce unexpected
      results if they are applied on logged data. The core plug-ins that are
      shipped with BASE has been fixed and they should work with any kind
      of data. The <classname>Base1PluginExecuter</classname> that is used
      for executing BASE 1 plug-ins can be configured to work with only
      a specific kind of data. After upgrading to BASE 2.12 a server admin
      should manually update the configuration of all registered BASE 1 plug-ins
      with information about what kind of source data that is required and
      what kind of result data the plug-in produces. The default setting is that
      a plug-in works with any kind of data and produces the same kind of data
      used as source.
    </para>
    
    <para>
      This change also affects formulas, which now has two additional properties:
      source and result intensity transform. The source intensity transform property
      tells BASE what kind of source data that the formula can be used with. If this
      property is not specified the formula can be used with any kind of data. The
      result intensity transform property tells BASE what kind of result the forumla
      generates. If this property is not specified the formula is expected to 
      generate the same kind of data as the source data. All existing user-defined
      forumlas will not have any of the properties set. After upgrading to BASE 2.12
      user should check their formulas and set appropriate values for the source
      and result intensity transform attributes.
    </para>
    
    <note>
      <title>The ch() function automatically converts logged intensities to unlogged</title>
      <para>
      In order to maintain as much backwards compatibility as possible the ch() function
      will automatically convert logged data back to unlogged. This means that many formulas
      will continue to work unmodified, but some may create unneccesary complex formulas.
      Consider, for example, the log-ratio formula: <code>log2(ch(1) / ch(2))</code>, which
      will be converted to: <code>log2(2 ^ rawCh(1) / 2 ^ rawCh(2))</code> if it is applied on
      logged values. A better re-write is: <code>rawCh(1) - rawCh(2)</code>. 
      </para>
    </note>
  </sect1>

  <sect1 id="appendix.incompatible.2.11">
    <title>BASE 2.11 release</title>
    
    <bridgehead>
      Biomaterial batch importers uses a different coordinate system 
      to target biowells
    </bridgehead>
    <para>
      The batch importers previously used the same coordinate system 
      for locating biowells on a plate that BASE uses internally. A 
      0-based numerical coordinate pair. This has now been changed to use 
      the more logical alphanumeric 1-based coordinate system typically found on plates. 
      As an example files should now specify A1, B2, C3 instead of [0,0],
      [1,1] or [2,2]. Files that use the "old" coordinate system must
      be updated to the new coordinate system, or the imported data will
      end up in incorrect wells or in no well at all. The change affects three 
      batch importers:
    </para>
    <itemizedlist>
      <listitem>
      <para><classname docapi="net.sf.basedb.plugins.batchimport">SampleImporter</classname></para>
      </listitem>
      <listitem>
      <para><classname docapi="net.sf.basedb.plugins.batchimport">ExtractImporter</classname></para>
      </listitem>
      <listitem>
      <para><classname docapi="net.sf.basedb.plugins.batchimport">LabeledExtractImporter</classname></para>
      </listitem>
    </itemizedlist>
    
    <note>
      It is still possible to use purely numerical coordinates, but they must
      be 1-based and not 0-based as before!
    </note>
    
    <warning>
      The new coordinate system only affects the batch importers. The BASE
      web client will still display the internal 0-based coordinate values.
      BASE users should be aware of this, particularly if they use the table
      exporter to generate template files intended for import at a later
      time. In this case the coordinates in the template file needs to be 
      edited before importing.
    </warning>
  </sect1>
  
  <sect1 id="appendix.incompatible.2.10">
    <title>BASE 2.10 release</title>
    
    <bridgehead>Added 'entryDate' property to a lot of items</bridgehead>
    <para>
      Including reporters and users. Since those two items are extendable
      by the server admin this addition may break a custom property with
      the same name.
    </para>
  </sect1>
  
  <sect1 id="appendix.incompatible.2.9">
    <title>BASE 2.9 release</title>
    
    <bridgehead>Must use a database that supports UTF-8</bridgehead>
    <para>
      If you have been running BASE on a database that is not using
      UTF-8 as the character set you need to convert the database 
      as part of the update. Read <xref linkend="update.useUTF8" />
      for more information.
    </para>
  </sect1>
  
  
  <sect1 id="appendix.incompatible.2.7.1">
    <title>BASE 2.7.1 release</title>
    
    <bridgehead>Type.BOOLEAN.parseString()</bridgehead>
    <para>
      This method now converts the empty string, "no", "0", and "false" (ignoring case)
      to boolean <constant>false</constant>. A <constant>null</constant>
      value as input still returns <constant>null</constant> as output.
      All other values are converted to <constant>true</constant>.
      Previously, all values except <constant>null</constant> and the string 
      "true", was converted to <constant>false</constant>. The change was made
      to make this method behave the same as other string conversion methods.
    </para>
  </sect1>
  
  <sect1 id="appendix.incompatible.2.7">
    <title>BASE 2.7 release</title>
    
    <bridgehead>Tomcat 6 or higher is required</bridgehead>
    <para>
      If you have been running BASE with Tomcat 5.5 you need
      to upgrade your Tomcat installation before installing
      BASE 2.7.
    </para>
  </sect1>
  
  <sect1 id="appendix.incompatible.2.6">
    <title>BASE 2.6 release</title>
    
    <bridgehead>Feature identification methods</bridgehead>
    <para>
      Array design features can now be identified by three different methods:
      COORDINATES, POSITION and FEATURE_ID. The coordinates method was the
      only one supported earlier. 
    </para>
    
    <itemizedlist>
    <listitem>
      <para>
      Client and plug-in code that depends on features having unique COORDINATES 
      may no longer work if used with an array design using a different identification method.
      Code that is, directly or indirectly, using a 
      <classname docapi="net.sf.basedb.core">FeatureBatcher</classname> or 
      <classname docapi="net.sf.basedb.core">RawDataBatcher</classname> are 
      probably affected by this. For example, a raw data importer which 
      doesn't know how to set the position or the feature ID can't import data to
      an array design that is using one of the new identification methods.
      </para>
    </listitem>
    
    <listitem>
      <para>
      The POSITION identification method will require a unique position number. 
      This value used to be an auto-generated sequence starting at 1. The other
      identification methods will still do that, but when using the POSITION identification
      method the only requirement is that the position value is a unique positive integer. 
      Client or plug-in code that depends on the position being a number between
      1 and the number of features may fail if used with an array design using 
      the POSITION identification method.
      </para>
    </listitem>   
    </itemizedlist>
    
    <bridgehead>Data file types</bridgehead>
    <para>
      The <methodname>DataFileType.setValidatorClass()</methodname> and
      <methodname>DataFileType.setMetadataReaderClass()</methodname> no longer
      verifies that the specified class names exists and implements the required interfaces.
      This validation will instead happen at commit time. The reason for this change is
      the added support for having the validator/meta data reader in external JAR files.
      This means that the validation can't happen until both the class names and JAR paths
      has been set. If a client application need validation before commit time it should use
      <methodname>DataFileType.getValidator()</methodname> and 
      <methodname>DataFileType.getMetadataReader()</methodname> after the class names and JAR
      paths has been set.
    </para>

    <bridgehead>Job.Status.ABORTING</bridgehead>
    <para>
      Added the <constant>ABORTING</constant> option to the <classname>Job.Status</classname>
      enumeration. This option is used when the <constant>ABORT</constant> signal has been 
      sent to an executing job, but the job has not yet responded to it.
      Code that uses the job's status to decide what action to take may fail
      since this is a new option. In most cases, it should be handled in the same way
      as if the job is <constant>EXECUTING</constant>.
    </para>
    
    <bridgehead>Hybridization to Labeled extracts link</bridgehead>
    <para>
      This link can now hold information about which sub-array a labeled
      extract belongs to on a multi-array hybridization. Code that is
      unaware of the concept of sub-arrays may find hybridizations 
      where the number of labeled extracts doesn't match the number
      channels in the platform used, and that more than one labeled
      extract has the same label. This was previously considered as
      an error condition by the experiment validator. With the
      new scheme the validation has to be done on a sub-array basis instead
      of on the complete hybridization.
    </para>
    
    <para>
      A similar issue arises when inheriting annotations to a raw bioassay
      which stems from a specific sub-array on a multi-array hybridization.
      This raw bioassay should only inherit annotations from the labeled
      extracts that are part of the same sub-array. For API compatibility
      reasons the <methodname>Hybridization.getAnnotatableParents()</methodname>
      will still return <emphasis>all</emphasis> labeled extracts. Code
      that is calling this method in order to find the parents to
      a raw bioassay should instead call the new method,
      <methodname>Hybridizations.getAnnotatableParents(int)</methodname>,
      where the <code>int</code> is the sub-array index value for
      the raw bioassay.
    </para>
    
    <para>
      A related issue arise when querying labeled extracts and joining
      the source events collection to use the linked hybridizations in
      the query. Here is an example:
    </para>
    
    <programlisting language="java">
<![CDATA[
// Find all labeled extracts on a given hybridization
Hybridization current = ...
ItemQuery<LabeledExtract> labeledExtractQuery = LabeledExtract.getQuery();

// This no longer works
// labeledExtractQuery.join(Hql.innerJoin("sourceEvents", "evt"));

// Replace the above line with these two line
labeledExtractQuery.join(Hql.innerJoin("sourceEvents", "srcevt"));
labeledExtractQuery.join(Hql.innerJoin("srcevt", "event", "evt"));

labeledExtractQuery.restrict(
  Restrictions.eq(
    Hql.property("evt", "hybridization"), 
    Expressions.integer(current.getId())
  )
);
]]>
</programlisting>

    <para>
      The good new is that the modifications makes it possible to filter
      the query on used quantity and sub-array. See the Javadoc for
      <classname docapi="net.sf.basedb.core">Hybridization</classname>
      to get more information.
    </para>
    
  </sect1>  
  
  <sect1 id="appendix.incompatible.2.5">
    <title>BASE 2.5 release</title>
    
    <bridgehead>Raw data types</bridgehead>
    <para>
      The <sgmltag class="attribute">storage</sgmltag> attribute of the
      <sgmltag class="starttag">raw-data-type</sgmltag> has been deprecated
      for the <filename>raw-data-types.xml</filename> file. BASE will refuse
      to start if it finds this attribute. Raw data types which doesn't use
      the database for storing data should be registered as <classname docapi="net.sf.basedb.core">Platform</classname>:s
      instead.
    </para>
    
    <para>
      Applications or plug-ins that filters on the <property>rawDataType</property>
      property for <classname docapi="net.sf.basedb.core">RawBioAssay</classname> or <classname docapi="net.sf.basedb.core">Experiment</classname>
      may need to change. The ID given to raw data types that doesn't use the
      database for storing data are prefixed with "<constant>platform.</constant>".
      The ID for the Affymetrix platform has changed from <constant>affymetrix</constant>
      to <constant>platform.affymetrix</constant>.
    </para>
    
    <bridgehead>Raw bioassays</bridgehead>
    <para>
      The property <property>spots</property> which used to hold the number
      of spots in the database OR in the file(s) has been split into two
      properties:
    </para>
    <itemizedlist>
    <listitem>
      <para>
      <property>spots</property>: Now only holds the number of database spots
      </para>
    </listitem>
    <listitem>
      <para>
      <property>numFileSpots</property>: Holds the number of spots that are stored in files
      </para>
    </listitem>
    </itemizedlist>
    
    <para>
      Applications or plug-ins that filters on the <property>spots</property> property
      may no longer work as expected for file-only platforms, since this value is now
      always 0. They should use the <property>numFileSpots</property> property instead.
    </para>
    
    <bridgehead>Array designs</bridgehead>
    <para>
      The <methodname>ArrayDesign.isAffyChip()</methodname> method has
      been deprecated. Applications or plug-ins that filter on the <property>affyChip</property>
      property may no longer work as expected. The applications should
      instead filter on the <property>platform.externalId</property> and
      look for the value given by the constant <constant>Platform.AFFYMETRIX</constant>.
    </para>
    
    <programlisting language="java">
// This may no longer work
query.restrict(
   Restrictions.eq(
      Hql.property("affyChip"), 
      Expressions.parameter("affyChip", true, Type.BOOLEAN)
   )
);

// Use this instead
query.restrict(
   Restrictions.eq(
      Hql.property("platform.externalId"), 
      Expressions.string(Platform.AFFYMETRIX)
   )
);
</programlisting>
    
    
  </sect1>
  
  <sect1 id="appendix.incompatible.2.4">
    <title>BASE 2.4 release</title>
    
    <bridgehead>Plug-in API</bridgehead>
    <para>
      The <methodname>InteractivePlugin.isInContext()</methodname>
      method may now throw exceptions to indicate error-level
      messages. Messages that are returned by the method are
      considered as a warning message and are by default no longer
      shown to the users. 
      See <xref linkend="plugin_developer.api.interfaces.interactive" />
      and <xref linkend="webclient.configuration.preferences.plugins" />.
    </para>
    
    <bridgehead>Custom JSP pages for plug-ins</bridgehead>
    <para>
      Plug-ins using custom JSP pages for parameter input are recommended
      to pass along the <varname>requestId</varname> parameter. The parameter
      is optional, but if used it will prevent data from two different
      plug-ins configurations to get mixed up. See 
      <xref linkend="plugin_developer.api.jspparameters" />.
    </para>
    
    <bridgehead>JEP parser</bridgehead>
    <para>
      The <methodname>Jep.nodeToExpression()</methodname>
      and <methodname>Jep.nodeToString()</methodname> methods
      return <constant>NULL</constant> if they find an unqouted
      <constant>NULL</constant> string in the expression. This
      allows JEP to convert expressions like <code>ch(1) == NULL</code>
      into a Query API expression testing for null. To get the
      old behaviour use <code>ch(1) == 'NULL'</code>.
    </para>
    
    <bridgehead>Parsing strings into numeric values</bridgehead>
    <para>
      The <methodname>Type.parseString(String)</methodname> method for 
      <constant>Type.FLOAT</constant> and <constant>Type.DOUBLE</constant>
      has changed it's behaviour for <constant>NaN</constant>
      and <constant>Infinity</constant> values. The methods used to
      return <constant>Float.NaN</constant>, <constant>Float.NEGATIVE_INFINITY</constant>,
      <constant>Float.POSITIVE_INFINITY</constant> or the corresponding
      <classname>Double</classname> values. Since databases doesn't like
      these special values and eventually most values will go into the database,
      the <methodname>parseString</methodname> method now returns <constant>null</constant> 
      instead.
    </para>
    
    <bridgehead>Extended properties and raw data types</bridgehead>
    <para>
      We have added validation code to check for invalid values. If you
      have modified the <filename>extended-properties.xml</filename>
      or the <filename>raw-data-types.xml</filename> file and they
      contain invalid values, you may not be able to start BASE until 
      they are fixed. The validation is rather strict and things that may
      have worked before (because you were lucky or the because the database
      has been forgiving) may no longer work. Here is an overview of the most
      important validation rules:
    </para>
    
    <itemizedlist>
    <listitem>
      <para>
      Names and identifiers for extended properties and raw data type
      can only contain letters, digits and underscores. They must not
      start with a digit.
      </para>
    </listitem>
    <listitem>
      <para>
      Names of database tables and columns can only contain letters, 
      digits and underscores. They must not start with a digit.
      </para>
    </listitem>
    <listitem>
      <para>
      There mustn't be any duplicate tables, columns, properties, etc.
      for a given context. For example, no duplicate tables in the
      database, no duplicate columns in a table, and no duplicate
      properties for a raw data type.
      </para>
    </listitem>
    </itemizedlist>
    
  </sect1>
  
  <sect1 id="appendix.incompatible.2.3">
    <title>BASE 2.3 release</title>
    
    <bridgehead>FlatFileParser</bridgehead>
    <para>
      The <methodname>hasMoreData()</methodname> method has changed the 
      order of checks made to determine the line type. The checks are
      now made in the following order:
      
      <orderedlist>
      <listitem>
        <para>
        Check if the line should be ignored.
        </para>
      </listitem>
      <listitem>
        <para>
        Check if the line is a data footer.
        </para>
      </listitem>
      <listitem>
        <para>
        Check if the line is the start of a new section.
        </para>
      </listitem>
      <listitem>
        <para>
        Check if the line is a data line.
        </para>
      </listitem>
      </orderedlist>
      The data line check has been moved to the last since it was difficult
      to create settings that made sure section and data footer lines where
      matched correctly.
    </para>

    <bridgehead>BASE 1 Plug-in executer</bridgehead>
    
    <para>
      Changed to store information about plug-in parameters as XML in the
      database instead of in the original BASE version 1 plug-in definition file.
      Existing BASE version 1 plug-ins must be re-configured before they can be 
      used. To do this:
    </para>
    
    <orderedlist>
    <listitem>
      <para>
      Go to 
      <menuchoice>
        <guimenu>Administrate</guimenu>
        <guisubmenu>Plugins</guisubmenu>
        <guimenuitem>Configurations</guimenuitem>
      </menuchoice>
      </para>
    </listitem>
    <listitem>
      <para>
      Step through the configuration wizard for all BASE version 1 plug-in
      configurations. You do not need to change any parameters. 
      Just click on the <guibutton>Next</guibutton> button
      until the configuration is complete.
      </para>
    </listitem>
    </orderedlist>
  </sect1>

  <sect1 id="appendix.incompatible.2.2">
    <title>BASE 2.2 release</title>
    
    <bridgehead>BASE 1 Plug-in executer</bridgehead>
    
    <para>
      No longer provides a default mapping between BASE version 1 and
      BASE version 2 raw data columns. To solve this add
      a <guilabel>Formula</guilabel> item with the same name as the
      BASE version 1 column name and an expression that picks the BASE
      version 2 raw data property. For example:
    </para>
    
<literallayout>
<guilabel>Name</guilabel>          BCh1Mean
<guilabel>Type</guilabel>          Column expression
<guilabel>Raw data type</guilabel> GenePix
<guilabel>Channels</guilabel>      2
<guilabel>Expressions</guilabel>   1: raw('ch1BgMean')
</literallayout>

  </sect1>

</appendix>