source: trunk/doc/src/docbook/developerdoc/plugin_developer.xml @ 4974

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE chapter PUBLIC 
    "-//Dawid Weiss//DTD DocBook V3.1-Based Extension for XML and graphics inclusion//EN" 
    "../../../../lib/docbook/preprocess/dweiss-docbook-extensions.dtd">
<!--
  $Id: plugin_developer.xml 4974 2009-06-15 09:05:06Z nicklas $:
  
  Copyright (C) 2007 Johan Enell, Peter Johansson, Nicklas Nordborg, Martin Svensson
  
  This file is part of BASE - BioArray Software Environment.
  Available at http://base.thep.lu.se/
  
  BASE is free software; you can redistribute it and/or
  modify it under the terms of the GNU General Public License
  as published by the Free Software Foundation; either version 3
  of the License, or (at your option) any later version.
  
  BASE is distributed in the hope that it will be useful,
  but WITHOUT ANY WARRANTY; without even the implied warranty of
  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
  GNU General Public License for more details.
  
  You should have received a copy of the GNU General Public License
  along with BASE. If not, see <http://www.gnu.org/licenses/>.
-->

<chapter id="plugin_developer">
  <?dbhtml dir="plugin_developer"?>
  <title>Plug-in developer</title>
  <sect1 id="plugin_developer.organize">
    <title>How to organize your plug-in project</title>
    
    <sect2 id="plugin_developer.organize.ant">
      <title>Using Ant</title>
      <para>
        Here is a simple example of how you might organize your project using ant 
        (<ulink url="http://ant.apache.org">http://ant.apache.org</ulink>) as the build 
        tool. This is just a recommendation that we have found to be working
        well. You may choose to do it another way.
      </para>

      <sect3 id="plugin_developer.organize.ant.directories">
        <title>Directory layout</title>
        <para>
        
          Create a directory on your computer where you want
          to store your plug-in project. This directory is the 
          <filename class="directory"><replaceable>pluginname</replaceable>/</filename>
          directory in the listing below. You should also create some subdirectories:
          
          <literallayout>
<filename class="directory"><replaceable>pluginname</replaceable>/</filename>
<filename class="directory"><replaceable>pluginname</replaceable>/bin/</filename>
<filename class="directory"><replaceable>pluginname</replaceable>/lib/</filename>
<filename class="directory"><replaceable>pluginname</replaceable>/src/<replaceable>org/company</replaceable>/</filename>
<filename class="directory"><replaceable>pluginname</replaceable>/META-INF/</filename>
          </literallayout>
          The
          <filename class="directory">bin/</filename>
          directory is empty to start with. It will contain the compiled code. 
          In the <filename class="directory">lib/</filename>
          directory you should put <filename>BASE2Core.jar</filename>
          and other library files your plug-in depends on. The
          <filename class="directory">src/</filename>
          directory contains your source code. In this directory you should create
          subdirectories corresponding to the package name of your plug-in
          class(es). See <ulink url="http://en.wikipedia.org/wiki/Java_package"
          >http://en.wikipedia.org/wiki/Java_package</ulink> for information
          about conventions for naming packages. The 
          <filename class="directory">META-INF</filename> directory contains metadata
          about the plug-in and are needed for best functionality.
        </para>
      </sect3>
    
      <sect3 id="plugin_developer.organize.ant.buildfile">
        <title>The build file</title>
        <para>
          In the root of your directory, create the build file:
          <filename>build.xml</filename>.
          Here is an example that will compile your plug-in and put it in a JAR file.
        </para>
        <example id="plugin_developer.organize.build.file">
          <title>A simple build file</title>
<programlisting language="xml">&lt;?xml version="1.0" encoding="UTF-8"?&gt;
&lt;project 
   name="MyPlugin" 
   default="build.plugin" 
   basedir="."
   &gt;

   &lt;!-- variables used --&gt;
   &lt;property name="plugin.name" value="MyPlugin" /&gt;
   &lt;property name="src" value="src" /&gt;
   &lt;property name="bin" value="bin" /&gt;

   &lt;!-- set up classpath for compiling --&gt;
   &lt;path id="classpath"&gt;
      &lt;fileset dir="lib"&gt;
         &lt;include name="**/*.jar"/&gt;
      &lt;/fileset&gt;
   &lt;/path&gt;

    
   &lt;!-- main target --&gt;
   &lt;target 
      name="build.plugin"  
      description="Compiles the plug-in and put in jar"
      &gt;
      &lt;javac 
         encoding="ISO-8859-1"
         srcdir="${src}"
         destdir="${bin}"
         classpathref="classpath"&gt;
      &lt;/javac&gt;
      &lt;jar 
         jarfile="${plugin.name}.jar" 
         basedir="bin"
         manifest="META-INF/MANIFEST.MF"
         &gt;
         &lt;!--Include this to add required files for auto registration wizard--&gt;
         &lt;metainf file="META-INF/base-plugins.xml"&gt;&lt;/metainf&gt;
         &lt;metainf file="META-INF/base-configurations.xml"&gt;&lt;/metainf&gt;        
      &lt;/jar&gt;

    &lt;/target&gt;
&lt;/project&gt; </programlisting>
        </example>
        <para>
          If your plug-in depends on other JAR files than the
          <filename>BASE2Core.jar</filename> you must create a file called
          <filename>MANIFEST.MF</filename> in the project <filename>META-INF</filename>
          directory. List the other JAR files as in the following example.
          If your plug-in does not depend on other JAR files, you may remove the 
          <sgmltag class="attribute">manifest</sgmltag> 
          attribute of the <sgmltag class="starttag">jar</sgmltag> tag.
<programlisting>Manifest-Version: 1.0
Class-Path: OtherJar.jar ASecondJar.jar</programlisting>
        </para>
        <para>
          If your plug-in should support registration with the auto-installation 
          wizard it is a good idea to add the <sgmltag class="element">metainf</sgmltag>
          tags. This will add the two files <filename>META-INF/base-plugins.xml</filename>
          and <filename>META-INF/base-configurations.xml</filename> to the 
          <filename>META-INF</filename> directory in the JAR file. The two files
          contains information about your plug-in and BASE can automatically 
          find and extract information from those. See
          <xref linkend="plugin_developer.organize.autoinstallcompatible" />
          for get more information about this feature.
        </para>
      </sect3>
        
      <sect3 id="plugin_developer.organize.ant.build">
        <title>Building the plug-in</title>
        <para>
          Compile the plug-in simply by typing
          <command>ant</command>
          in the console window. If all went well the
          <filename>MyPlugin.jar</filename>
          will be created in the same directory.
        </para>
        <para>
          To install the plug-in copy the JAR file to the server including the dependent JAR
          files (if any). Place all files together in the same directory. For more
          information read
          <xref linkend="plugins.installation" />.
        </para>
      </sect3>
    </sect2>
    
    <sect2 id="plugin_developer.organize.eclipse">
      <title>With Eclipse</title>
      <para>
        If somebody is willing to add information to this chapter please send us a note or
        some written text to put here. Otherwise, this chapter will be removed.
      </para>
    </sect2>

    <sect2 id="plugin_developer.organize.autoinstallcompatible">
      <title>Make the plug-in compatible with the auto-installation wizard</title>
      <para>
        BASE has support for automatically detecting new plug-ins with
        the auto-installation wizard. The wizard makes it very easy for a
        server administrator to install new plug-ins. See 
        <xref linkend="plugins.autoinstall" />.
      </para>
      
      <para>
        The auto-install feature requires that a plug-in provides some information
        about itself. The wizard looks in all JAR file for the file
        <filename>META-INF/base-plugins.xml</filename>. This file contains
        some information about the plug-in(s). Here is an example:
      </para>
  
      <programlisting language="xml">
&lt;?xml version="1.0" encoding="UTF-8"?&gt;
&lt;!DOCTYPE plugins SYSTEM "base-plugins.dtd" &gt;
&lt;plugins jarname="jarfile.jar"&gt;
  &lt;pluginclass classname="se.lu.thep.PluginClass"&gt;
    &lt;minbaseversion&gt;2.4&lt;/minbaseversion&gt;
    &lt;hasconfigurations/&gt;
    &lt;/pluginclass&gt;
    .
    .
    .
&lt;/plugins&gt;
</programlisting>
      <para>
        The first two lines should be the same in all <filename>base-plugins.xml</filename> files.
        The rest of the tags are described in following list.
        <variablelist>
          <varlistentry>
            <term><sgmltag class="starttag">plugins</sgmltag></term>
            <listitem>
              <para>
                This is the root element in the XML-file and must have the
                attribute
                <sgmltag class="attribute">jarname</sgmltag>
                set. The value must be the same as name of the JAR file
                the plug-in is located in or the auto-installation wizard
                will fail with an error.
              </para>
            </listitem>
          </varlistentry>
          <varlistentry>
            <term>
              <sgmltag class="starttag">pluginclass</sgmltag>
            </term>
            <listitem>
              <para>
                This tag defines information about a plug-in in the JAR file. The
                <sgmltag class="attribute">classname</sgmltag>
                attribute needs to have the full classname of the plug-in's main
                class. There can be one or several of this element inside the
                <sgmltag class="starttag">plugins</sgmltag>
                tag, which means that there should be one element for
                each plug-in in the JAR file.
              </para>
            </listitem>
          </varlistentry>
          <varlistentry>
            <term>
              <sgmltag class="starttag">minbaseversion</sgmltag>
            </term>
            <listitem>
              <para>
                A required child element in
                <sgmltag class="starttag">pluginclass</sgmltag>.
                This defines from which version of BASE the plug-in can be used.
                The plug-in will not be included in auto installer if the
                BASE-version is lower then the value of this tag. The format of the
                value in this tag should be (numeric)
                <emphasis>majorversion</emphasis>.<emphasis>minorversion</emphasis>.
                It is not possible to check the bugfix or revision numbers.
              </para>
            </listitem>
          </varlistentry>
          <varlistentry>
            <term>
              <sgmltag class="starttag">hasconfigurations</sgmltag>             
            </term>
            <listitem>
              <para>
                A required child element in
                <sgmltag class="starttag">pluginclass</sgmltag>.
                This should tell if there are plug-in configurations
                shipped with the plug-in. Setting the value to
                <emphasis>yes</emphasis>
                will indicate that there are available configurations in the
                JAR file. This can be left as an empty tag if no
                configurations are available for import.
              </para>
              <para>
                Configurations shipped with a JAR file should be exported with the
                plug-in configuration exporter in BASE. The exported file must be
                given the name
                <filename>base-configurations.xml</filename>
                and be placed in
                <filename class="directory">META-INF/</filename>.
              </para>
            </listitem>
          </varlistentry>
        </variablelist>
      </para>
    </sect2>
    
  </sect1>

  <sect1 id="plugin_developer.api">
    <title>The Plug-in API</title>
    <para>
    </para>
    
    <sect2 id="plugin_developer.api.interfaces">
    
      <title>The main plug-in interfaces</title>
      <para>
        The Base2 core defines two interfaces and one
        abstract class that are vital for implementing plug-ins:
        <itemizedlist spacing="compact">
          <listitem>
            <simpara>
              <interfacename docapi="net.sf.basedb.core.plugin">net.sf.basedb.core.plugin.Plugin</interfacename>
            </simpara>
          </listitem>
          <listitem>
            <simpara>
              <interfacename docapi="net.sf.basedb.core.plugin">net.sf.basedb.core.plugin.InteractivePlugin</interfacename>
            </simpara>
          </listitem>
          <listitem>
            <simpara>
              <classname docapi="net.sf.basedb.core.plugin">net.sf.basedb.core.plugin.AbstractPlugin</classname>
            </simpara>
          </listitem>
        </itemizedlist>
        
        A plug-in must always implement the 
        <interfacename docapi="net.sf.basedb.core.plugin">Plugin</interfacename> interface. 
        The <interfacename docapi="net.sf.basedb.core.plugin">InteractivePlugin</interfacename>
        interface is optional, and is only needed if you want user interaction.
        The <classname docapi="net.sf.basedb.core.plugin">AbstractPlugin</classname> is a useful base class
        that your plug-in can use as a superclass. It provides default implementations
        for some of the interface methods and also has utility methods for
        validating and storing job and configuration parameter values. Another
        reason to use this class as a superclass is that it will shield your 
        plug-in from future changes to the Plug-in API. For example, if
        we decide that a new method is needed in the <interfacename docapi="net.sf.basedb.core.plugin">Plugin</interfacename> 
        interface we will also try to add a default implementation in 
        the <classname docapi="net.sf.basedb.core.plugin">AbstractPlugin</classname> class.
      </para>
  
      <important>
        <para>
        A plug-in must also have public no-argument contructor. Otherwise, BASE will not
        be able to create new instances of the plug-in class.
        </para>
      </important>
      
      <sect3 id="plugin_developer.api.interfaces.plugin">
        <title>The net.sf.basedb.core.plugin.Plugin interface</title>
        <para>
          This interface defines the following methods and must be implemented 
          by all plug-ins.
        </para>
        <variablelist>
          <varlistentry>
            <term>
              <methodsynopsis language="java">
                <modifier>public</modifier>
                <type>About</type>
                <methodname>getAbout</methodname>
                <void />
              </methodsynopsis>
            </term>
            <listitem>
              <para>
                Return information about the plug-in, i.e. the name, version, and a short
                description about what the plug-in does. The
                <classname docapi="net.sf.basedb.core.plugin">About</classname>
                object also has fields for naming the author and various other contact
                information. The returned information is copied by the core at
                installation time into the database. The only required information is
                the name of the plug-in. All other fields may have null values.
              </para>
              <example id="net.sf.basedb.core.plugin.Plugin.getAbout">
                <title>A typical implementation stores this information in a static field</title>
<programlisting language="java">
private static final About about = new AboutImpl
(
   "Spot images creator",
   "Converts a full-size scanned image into smaller preview jpg " +
     "images for each individual spot.",
   "2.0",
   "2006, Department of Theoretical Physics, Lund University",
   null,
   "base@thep.lu.se",
   "http://base.thep.lu.se"
);
  
public About getAbout()
{
   return about;
}</programlisting>
              </example>
            </listitem>
          </varlistentry>
          <varlistentry>
            <term>
              <methodsynopsis language="java">
                <modifier>public</modifier>
                <type>Plugin.MainType</type>
                <methodname>getMainType</methodname>
                <void />
              </methodsynopsis>
            </term>
            <listitem>
              <para>
                Return information about the main type of plug-in. The
                <classname docapi="net.sf.basedb.core.plugin">Plugin.MainType</classname>
                is an enumeration with five possible values:
                <itemizedlist>
                  <listitem>
                    <para>
                      <constant>ANALYZE</constant>: 
                      An analysis plug-in
                    </para>
                  </listitem>
                  <listitem>
                    <para>
                      <constant>EXPORT</constant>:
                      A plug-in that exports data
                    </para>
                  </listitem>
                  <listitem>
                    <para>
                      <constant>IMPORT</constant>:
                      A plug-in that imports data
                    </para>
                  </listitem>
                  <listitem>
                    <para>
                      <constant>INTENSITY</constant>:
                      A plug-in that calculates the original spot intensities
                      from raw data
                    </para>
                  </listitem>
                  <listitem>
                    <para>
                      <constant>OTHER</constant>:
                      Any other type of plug-in
                    </para>
                  </listitem>
                </itemizedlist>
                The returned value is stored in the database but is otherwise not used
                by the core. Client applications (such as the web client) will probably
                use this information to group the plug-ins, i.e., a button labeled 
                &gbExport;
                will let you select among the export plug-ins.
              </para>
              <example id="net.sf.basedb.core.plugin.Plugin.getMainType">
                <title>A typical implementation just return one of the values</title>
<programlisting language="java">
public Plugin.MainType getMainType()
{
   return Plugin.MainType.OTHER;
}</programlisting>
              </example>
            </listitem>
          </varlistentry>
          <varlistentry>
            <term>
              <methodsynopsis language="java">
                <modifier>public</modifier>
                <type>boolean</type>
                <methodname>supportsConfigurations</methodname>
                <void />
              </methodsynopsis>
            </term>
            <listitem>
              <para>
                If this method returns true, the plug-in can have different
                configurations, (i.e.
                <classname docapi="net.sf.basedb.core">PluginConfiguration</classname>).
                Note that this method may return true even if the
                <interfacename docapi="net.sf.basedb.core.plugin">InteractivePlugin</interfacename>
                interface is not implemented. The
                <classname docapi="net.sf.basedb.core.plugin">AbstractPlugin</classname>
                returns true for this method, which is the old way before the
                introduction of this method.
              </para>
            </listitem>
          </varlistentry>
          <varlistentry>
            <term>
              <methodsynopsis language="java">
                <modifier>public</modifier>
                <type>boolean</type>
                <methodname>requiresConfiguration</methodname>
                <void />
              </methodsynopsis>
            </term>
            <listitem>
              <para>
                If this method returns true, the plug-in must have a configuration
                to be able to run. For example, some of the core import plug-ins 
                must have information about the file format, to be able to import
                any data. 
                The
                <classname docapi="net.sf.basedb.core.plugin">AbstractPlugin</classname>
                returns false for this method, which is the old way before the
                introduction of this method.
              </para>
            </listitem>
          </varlistentry>
          <varlistentry>
            <term>
              <methodsynopsis language="java">
                <modifier>public</modifier>
                <type>Collection&lt;Permissions&gt;</type>
                <methodname>getPermissions</methodname>
                <void />
              </methodsynopsis>
            </term>
            <listitem>
              <para>
                Return a collection of permissions that the plug-in needs
                to be able to function as expected. This method may return null
                or an empty collection. In this case the plug-in permission system
                is not used and the plug-in always gets the same permissions as the
                logged in user. If permissions are specified the plug-in should
                list all permissions it requires. Permissions that are not listed
                are denied.
              </para>
              <note>
                <para>
                The final assignment of permissions to a plug-in is always
                at the hands of a server administrator. He/she may decide to
                disable the plug-in permission system or revoke some of the
                requested permissions. The permissions returned by this method is
                only a recommendation that the server administrator may or may not
                accept. See <xref linkend="plugins.permissions" />
                for more information about plug-in permissions.
                </para>
              </note>
            </listitem>
          </varlistentry>
          <varlistentry>
            <term>
              <methodsynopsis language="java">
                <modifier>public</modifier>
                <void />
                <methodname>init</methodname>
                <methodparam>
                  <type>SessionControl</type>
                  <parameter>sc</parameter>
                </methodparam>
                <methodparam>
                  <type>ParameterValues</type>
                  <parameter>configuration</parameter>
                </methodparam>
                <methodparam>
                  <type>ParameterValues</type>
                  <parameter>job</parameter>
                </methodparam>
                <exceptionname>BaseException</exceptionname>
              </methodsynopsis>
            </term>
            <listitem>
              <para>
                Prepare the plug-in for execution or configuration. If the plug-in needs
                to do some initialization this is the place to do it. A typical
                implementation however only stores the passed parameters in instance
                variables for later use. Since it is not possible what the user is going to
                do at this stage, we recommend lazy initialisation of all other resources.
              </para>
              <para>
                The parameters passed to this method has vital information that is
                needed to execute the plug-in. The
                <classname docapi="net.sf.basedb.core">SessionControl</classname>
                is a central core object holding information about the logged in
                user and is used to create
                <classname docapi="net.sf.basedb.core">DbControl</classname>
                objects which allows a plug-in to connect to the database to read, add or
                update information. The two
                <classname docapi="net.sf.basedb.core.plugin">ParameterValues</classname>
                objects contain information about the configuration and job 
                parameters to the plug-in. 
                The <varname>configuration</varname> object holds all parameters stored
                together with a <classname docapi="net.sf.basedb.core">PluginConfiguration</classname>
                object in the database. If the plug-in is started without
                a configuration this object is null.
                The <varname>job</varname> object holds all parameters that are
                stored together with a <classname docapi="net.sf.basedb.core">Job</classname> object in the 
                database. This object is null if the plug-in is started without a job.
              </para>
              <para>
                The difference between a configuration parameter and a job parameter is
                that a configuration is usually something an administrator sets up,
                while a job is an actual execution of a plug-in. For example, a
                configuration for an import plug-in holds the regular expressions needed
                to parse a text file and find the headers, sections and data lines,
                while the job holds the file to parse.
              </para>
              <para>
                The
                <classname docapi="net.sf.basedb.core.plugin">AbstractPlugin</classname>
                contains an implementation of this method that saves the passed objects
                in protected instance variables. If you override this method 
                we recommend that you also call <code>super.init()</code>.
              </para>
              <example id="net.sf.basedb.core.plugin.Plugin.init">
                <title>The <classname docapi="net.sf.basedb.core.plugin">AbstractPlugin</classname> implementation of Plugin.init()</title>
<programlisting language="java">
protected SessionControl sc = null;
protected ParameterValues configuration = null;
protected ParameterValues job = null;
/**
   Store copies of the session control, plug-in and job configuration. These
   are available to subclasses in the {@link #sc}, {@link #configuration}
   and {@link #job} variables. If a subclass overrides this method it is 
   recommended that it also calls super.init(sc, configuration, job).
*/
public void init(SessionControl sc, 
   ParameterValues configuration, ParameterValues job)
   throws BaseException
{
   this.sc = sc;
   this.configuration = configuration;
   this.job = job;
}</programlisting>
              </example>
            </listitem>
          </varlistentry>
          <varlistentry>
            <term>
              <methodsynopsis language="java">
                <modifier>public</modifier>
                <void />
                <methodname>run</methodname>
                <methodparam>
                  <type>Request</type>
                  <parameter>request</parameter>
                </methodparam>
                <methodparam>
                  <type>Response</type>
                  <parameter>response</parameter>
                </methodparam>
                <methodparam>
                  <type>ProgressReporter</type>
                  <parameter>progress</parameter>
                </methodparam>
              </methodsynopsis>
            </term>
            <listitem>
              <para>
                Run the plug-in.
              </para>
              <para>
                The <varname>request</varname>
                parameter is of historical interest only. It has no useful information 
                and can be ignored.
              </para>
              <para>
                The <varname>progress</varname> parameter
                can be used by a plug-in to report its progress back to the core. The
                core will usually send the progress information to the database, which
                allows users to see exactly how the plug-in is progressing from the web
                interface. This parameter can be null, but if it is not we recommend all
                plug-ins to use it. However, it should be used sparingly, since each call
                to set the progress results in a database update. If the execution
                involves several thousands of items it is a bad idea to update the
                progress after processing each one of them. A good starting point is to
                divide the work into 100 pieces each representing 1% of the work, i.e.,
                if the plug-in should export 100 000 items it should report progress
                after every 1000 items.
              </para>
              <para>
                The <varname>response</varname>
                parameter is used to tell the core if the plug-in was successful or
                failed. Not setting a response is considered a failure by the core. From
                the run method it is only allowed to use the
                <methodname>Response.setDone()</methodname>
                or the
                <methodname>Response.setError()</methodname>
                methods.
              </para>
              
              <important>
                It is also considered bad practice to let exceptions escape
                out from this method. Always use <code>try...catch</code>
                to catch exceptions and use <code>Response.setError()</code>
                to report the error back to the core.
              </important>
              
              <example id="net.sf.basedb.core.plugin.Plugin.run">
                <title>
                  Here is a skeleton that we recommend each plug-in to use in its
                  implementation of the
                  <methodname>run()</methodname>
                  method
                </title>
<programlisting language="java">
public void run(Request request, Response response, ProgressReporter progress)
{
   // Open a connection to the database
   // sc is set by init() method
   DbControl dc = sc.newDbControl();
   try
   {
      // Insert code for plug-in here

      // Commit the work
      dc.commit();
      response.setDone("Plug-in ended successfully");
   }
   catch (Throwable t)
   {
      // All exceptions must be catched and sent back 
      // using the response object
      response.setError(t.getMessage(), Arrays.asList(t));
   }
   finally
   {
      // IMPORTANT!!! Make sure opened connections are closed
      if (dc != null) dc.close();
   }
}</programlisting>
              </example>
            </listitem>
          </varlistentry>
          <varlistentry>
            <term>
              <methodsynopsis language="java">
                <modifier>public</modifier>
                <void />
                <methodname>done</methodname>
                <void />
              </methodsynopsis>
            </term>
            <listitem>
              <para>
                Clean up all resources after executing the plug-in. This method must not
                throw any exceptions.
              </para>
              <example id="net.sf.basedb.core.plugin.Plugin.done">
                <title>
                  The
                  <classname docapi="net.sf.basedb.core.plugin">AbstractPlugin</classname>
                  contains an implementation of the <methodname>done()</methodname>
                  method simply sets the
                  parameters passed to the
                  <methodname>init()</methodname>
                  method to null
                </title>
<programlisting language="java">
/**
   Clears the variables set by the init method. If a subclass 
   overrides this method it is recommended that it also calls super.done().
*/
public void done()
{
   configuration = null;
   job = null;
   sc = null;
}</programlisting>
              </example>
            </listitem>
          </varlistentry>
        </variablelist>
      </sect3>
  
      <sect3 id="plugin_developer.api.interfaces.interactive">
        <title>The net.sf.basedb.core.plugin.InteractivePlugin interface</title>
        <para>
          If you want the plug-in to be able to interact with the user you must also implement
          this interface. This is probably the case for most plug-ins. Among the core plug-ins
          shipped with BASE the
          <classname docapi="net.sf.basedb.plugins">SpotImageCreator</classname>
          is one plug-in that does not interact with the user. Instead, the web client has
          special JSP pages that handles all the interaction, creates a job for it and sets
          the parameters. This, kind of hardcoded, approach can also be used for other 
          plug-ins, but then it usually requires modification of the client application 
          as well.
        </para>
        <para>
          The
          <interfacename docapi="net.sf.basedb.core.plugin">InteractivePlugin</interfacename>
          has three main tasks:
          
          <orderedlist>
          <listitem>
            <para>
            Tell a client application where the plug-in should be plugged
            in.
            </para>
          </listitem>
          <listitem>
            <para>
            Ask the users for configuration and job parameters.
            </para>
          </listitem>
          
          <listitem>
            <para>
            Validate parameter values entered by the user and store those in the
            database.
            </para>
          </listitem>
          </orderedlist>
          This requires that the following methods are 
          implemented.
        </para>
        
        <variablelist>
          <varlistentry>
            <term>
              <methodsynopsis language="java">
                <modifier>public</modifier>
                <type>Set&lt;GuiContext&gt;</type>
                <methodname>getGuiContexts</methodname>
                <void />
              </methodsynopsis>
            </term>
            <listitem>
              <para>
                Return information about where the plug-in should be plugged in. Each
                place is identified by a
                <classname docapi="net.sf.basedb.core.plugin">GuiContext</classname>
                object, which is an
                <classname docapi="net.sf.basedb.core">Item</classname>
                and a
                <classname docapi="net.sf.basedb.core">Type</classname>.
                The item is one of the objects defined by the
                <classname docapi="net.sf.basedb.core">Item</classname>
                enumeration and the type is either
                <constant>Type.LIST</constant>
                or
                <constant>Type.ITEM</constant>, which corresponde to the
                list view and the single-item view in the web client.
              </para>
              <para>
                For example, the
                <varname>GuiContext</varname> = 
                (<constant>Item.REPORTER</constant>, 
                <constant>Type.LIST</constant>)
                tells a client application that this plug-in can be plugged in whenever a
                list of reporters is displayed. The
                <varname>GuiContext</varname> =
                (<constant>Item.REPORTER</constant>,
                <constant>Type.ITEM</constant>)
                tells a client application that this plug-in can be plugged in whenever
                a single reporter is displayed. The first case may be appropriate for a
                plug-in that imports or exports reporters. The second case may be used by
                a plug-in that updates the reporter information from an external source
                (well, it may make sense to use this in the list case as well).
              </para>
              <para>
                The returned information is copied by the core at installation time to
                make it easy to ask for all plug-ins for a certain
                <classname docapi="net.sf.basedb.core.plugin">GuiContext</classname>.
              </para>
              <para>
                A typical implementation creates a static unmodifiable
                <classname>Set</classname>
                which is returned by this method. It is important that the returned set
                cannot be modified. It may be a security issue if a misbehaving
                client application does that.
              </para>
              <example id="net.sf.basedb.core.plugin.InteractivePlugin.getGuiContexts">
                <title>
                  A typical implementation of
                  <methodname>getGuiContexts</methodname>
                </title>
<programlisting language="java">
// From the net.sf.basedb.plugins.RawDataFlatFileImporter plug-in
private static final Set&lt;GuiContext&gt; guiContexts = 
   Collections.singleton(new GuiContext(Item.RAWBIOASSAY, GuiContext.Type.ITEM));

public Set&lt;GuiContext&gt; getGuiContexts()
{
   return guiContexts;
}</programlisting>
              </example>
            </listitem>
          </varlistentry>
          <varlistentry>
            <term>
              <methodsynopsis language="java">
                <modifier>public</modifier>
                <type>String</type>
                <methodname>isInContext</methodname>
                <methodparam>
                  <type>GuiContext</type>
                  <parameter>context</parameter>
                </methodparam>
                <methodparam>
                  <type>Object</type>
                  <parameter>item</parameter>
                </methodparam>
              </methodsynopsis>
            </term>
            <listitem>
              <para>
                This method is called to check if a particular item is usable for the
                plug-in. This method is invoked to check if a plug-in can be used
                in a given context. If invoked from a list context the <parameter>item</parameter>
                parameter is <constant>null</constant>.  
                The plug-in should return <constant>null</constant> if it
                finds that it can be used. If the plug-in can't be used it
                must decide if the reason should be a warning or an error condition.
              </para>
              
              <para>
                A warning is issued by returning a string with the warning
                message. It should be used when the plug-in can't be used because
                it is unrelated to the current task. For example, a plug-in for
                importing Genepix data should return a warning when somebody wants
                to import data to an Agilent raw bioassay.
              </para>
              
              <para>
                An error message is issued by throwing an exception. This 
                should be used when the plug-in is related to the current task
                but still can't do what it is supposed to do. For example,
                trying to import raw data if the logged in user doesn't have
                write permission to the raw bioassay.
              </para>
              
              <para>
                As a rule of thumb, if there is a chance that another plug-in
                might be able to perform the same task a warning should be used.
                If it is guaranteed that no other plug-in can do it an error
                message should be used.
              </para>
              
              <note>
                <para>
                The contract of this method was changed in in BASE 2.4
                to allow warning and error level message. Prior to BASE
                2.4 all messages were treated as error message. We recommend
                that existing plug-ins are updated to throw exception to indicate
                error-level messages since the default is to not show 
                warning messages to users.
                </para>
              </note>
              
              <para>
                Here is a real example from the
                <classname docapi="net.sf.basedb.plugins">RawDataFlatFileImporter</classname>
                plug-in which imports raw data to a
                <classname docapi="net.sf.basedb.core">RawBioAssay</classname>. 
                
                Thus,
                <varname>GuiContext</varname> = 
                (<constant>Item.RAWBIOASSAY</constant>,
                <constant>Type.ITEM</constant>), 
                
                but the plug-in can only import data if the logged in user has write permission,
                there is no data already, and
                if the raw bioassay has the same raw data type as the plug-in has been
                configured for.
              </para>
              <example id="net.sf.basedb.core.plugin.InteractivePlugin.isInContext">
                <title>
                  A realistic implementation of the
                  <methodname>isInContext()</methodname> method
                </title>
<programlisting language="java">
/**
   Returns null if the item is a {@link RawBioAssay} of the correct
   {@link RawDataType} and doesn't already have spots.
   @throws PermissionDeniedException If the raw bioasssay already has raw data
   or if the logged in user doesn't have write permission
*/
public String isInContext(GuiContext context, Object item)
{
   String message = null;
   if (item == null)
   {
      message = "The object is null";
   }
   else if (!(item instanceof RawBioAssay))
   {
      message = "The object is not a RawBioAssay: " + item;
   }
   else
   {
      RawBioAssay rba = (RawBioAssay)item;
      String rawDataType = (String)configuration.getValue("rawDataType");
      RawDataType rdt = rba.getRawDataType();
      if (!rdt.getId().equals(rawDataType))
      {
         // Warning
         message = "Unsupported raw data type: " + rba.getRawDataType().getName();
      }
      else if (!rdt.isStoredInDb())
      {
         // Warning
         message = "Raw data for raw data type '" + rdt + "' is not stored in the database";
      }
      else if (rba.hasData())
      {
         // Error
         throw new PermissionDeniedException("The raw bioassay already has data.");
      }
      else
      {
         // Error
         rba.checkPermission(Permission.WRITE);
      }
   }
   return message;    
}</programlisting>
              </example>
            </listitem>
          </varlistentry>
          <varlistentry>
            <term>
              <methodsynopsis language="java">
                <modifier>public</modifier>
                <type>RequestInformation</type>
                <methodname>getRequestInformation</methodname>
                <methodparam>
                  <type>GuiContext</type>
                  <parameter>context</parameter>
                </methodparam>
                <methodparam>
                  <type>String</type>
                  <parameter>command</parameter>
                </methodparam>
                <exceptionname>BaseException</exceptionname>
              </methodsynopsis>
            </term>
            <listitem>
              <para>
                Ask the plug-in for parameters that need to be entered by the user. The
                <classname docapi="net.sf.basedb.core.plugin">GuiContext</classname>
                parameter is one of the contexts returned by the
                <methodname>getGuiContexts</methodname>
                method. The command is a string telling the plug-in what command was
                executed. There are two predefined commands but as you will see the
                plug-in may define its own commands. The two predefined commands are
                defined in the
                <classname docapi="net.sf.basedb.core.plugin">net.sf.basedb.core.plugin.Request</classname>
                class.
                <variablelist>
                  <varlistentry>
                    <term>
                      <constant>Request.COMMAND_CONFIGURE_PLUGIN</constant>
                    </term>
                    <listitem>
                      <para>
                        Used when an administrator is initiating a configuration
                        of the plug-in.
                      </para>
                    </listitem>
                  </varlistentry>
                  <varlistentry>
                    <term>
                      <constant>Request.COMMAND_CONFIGURE_JOB</constant>
                    </term>
                    <listitem>
                      <para>
                        Used when a user has selected the plug-in for running a
                        job.
                      </para>
                    </listitem>
                  </varlistentry>
                </variablelist>
                Given this information the plug-in must return a
                <classname docapi="net.sf.basedb.core">RequestInformation</classname>
                object. This is simply a title, a description, and a list of parameters.
                Usually the title will end up as the input form title and the
                description as a help text for the entire form. Do not put information
                about the individual parameters in this description, since each
                parameter has a description of its own.
              </para>
              <example id="net.sf.basedb.core.plugin.InteractivePlugin.getRequestInformation">
                <title>
                  When running an import plug-in it needs to ask for the file to import
                  from and if existing items should be updated or not
                </title>
<programlisting language="java">
// The complete request information
private RequestInformation configure Job;

// The parameter that asks for a file to import from
private PluginParameter&lt;File&gt; file Parameter;

// The parameter that asks if existing items should be updated or not
private PluginParameter&lt;Boolean&gt; updateExistingParameter;

public RequestInformation getRequestInformation(GuiContext context, String command)
   throws BaseException
{
   RequestInformation requestInformation = null;
   if (command.equals(Request.COMMAND_CONFIGURE_PLUGIN))
   {
      requestInformation = getConfigurePlugin();
   }
   else if (command.equals(Request.COMMAND_CONFIGURE_JOB))
   {
      requestInformation = getConfigureJob();
   }
   return requestInformation;
}

/**
   Get (and build) the request information for starting a job.
*/
private RequestInformation getConfigureJob()
{
   if (configureJob == null)
   {
      // A file is required
      fileParameter = new PluginParameter&lt;File&gt;(
         "file",
         "File",
         "The file to import the data from",
         new FileParameterType(null, true, 1)
      ); 
      
      // The default value is 'false'
      updateExistingParameter = new PluginParameter&lt;Boolean&gt;(
         "updateExisting",
         "Update existing items",
         "If this option is selected, already existing items will be updated " +
         " with the information in the file. If this option is not selected " +
         " existing items are left untouched.",
         new BooleanParameterType(false, true)
      );

      List&lt;PluginParameter&lt;?&gt;&gt; parameters = 
         new ArrayList&lt;PluginParameter&lt;?&gt;&gt;(2);
      parameters.add(fileParameter);
      parameters.add(updateExistingParameter);
      
      configureJob = new RequestInformation
      (
         Request.COMMAND_CONFIGURE_JOB,
         "Select a file to import items from",
         "Description",
         parameters
      );
   }
   return configureJob;
}</programlisting>
              </example>
              <para>
                As you can see it takes a lot of code to put together a
                <classname docapi="net.sf.basedb.core">RequestInformation</classname>
                object. For each parameter you need one
                <classname docapi="net.sf.basedb.core">PluginParameter</classname>
                object and one
                <classname docapi="net.sf.basedb.core">ParameterType</classname>
                object. To make life a little easier, a
                <classname docapi="net.sf.basedb.core">ParameterType</classname>
                can be reused for more than one
                <classname docapi="net.sf.basedb.core">PluginParameter</classname>.
              </para>
              
<programlisting language="java">
StringParameterType stringPT = new StringParameterType(255, null, true);
PluginParameter one = new PluginParameter("one", "One", "First string", stringPT);
PluginParameter two = new PluginParameter("two", "Two", "Second string", stringPT);
// ... and so on
</programlisting>
              <para>
                The
                <classname docapi="net.sf.basedb.core">ParameterType</classname>
                is an abstract base class for several subclasses each implementing a
                specific type of parameter. The list of subclasses may grow in the
                future, but here are the most important ones currently implemented.
              </para>
              <note>
                <para>
                  Most parameter types include support for supplying a predefined list
                  of options to select from. In that case the list will be displayed
                  as a drop-down list for the user, otherwise a free input field is
                  used.
                </para>
              </note>
              <variablelist>
                <varlistentry>
                  <term>
                    <classname docapi="net.sf.basedb.core">StringParameterType</classname>
                  </term>
                  <listitem>
                    <para>
                      Asks for a string value. Includes an option for
                      specifying the maximum length of the string.
                    </para>
                  </listitem>
                </varlistentry>
                <varlistentry>
                  <term>
                    <classname docapi="net.sf.basedb.core">FloatParameterType</classname>, 
                    <classname docapi="net.sf.basedb.core">DoubleParameterType</classname>, 
                    <classname docapi="net.sf.basedb.core">IntegerParameterType</classname>, 
                    <classname docapi="net.sf.basedb.core">LongParameterType</classname>
                  </term>
                  <listitem>
                    <para>
                      Asks for numerical values. Includes options for
                      specifying a range (min/max) of allowed values.
                    </para>
                  </listitem>
                </varlistentry>
                <varlistentry>
                  <term>
                    <classname docapi="net.sf.basedb.core">BooleanParameterType</classname>
                  </term>
                  <listitem>
                    <para>Asks for a boolean value.
                    </para>
                  </listitem>
                </varlistentry>
                <varlistentry>
                  <term>
                    <classname docapi="net.sf.basedb.core">DateParameterType</classname>
                  </term>
                  <listitem>
                    <para>Asks for a date.
                    </para>
                  </listitem>
                </varlistentry>
                <varlistentry>
                  <term>
                    <classname docapi="net.sf.basedb.core">FileParameterType</classname>
                  </term>
                  <listitem>
                    <para>Asks for a file item.
                    </para>
                  </listitem>
                </varlistentry>
                <varlistentry>
                  <term>
                    <classname docapi="net.sf.basedb.core">ItemParameterType</classname>
                  </term>
                  <listitem>
                    <para>
                      Asks for any other item. This parameter type requires
                      that a list of options is supplied, except when the item
                      type asked for matches the current <classname docapi="net.sf.basedb.core.plugin">GuiContext</classname>, in which
                      case the currently selected item is used as the
                      parameter value.
                    </para>
                  </listitem>
                </varlistentry>
                <varlistentry>
                  <term>
                    <classname docapi="net.sf.basedb.core">PathParameterType</classname>
                  </term>
                  <listitem>
                    <para>
                      Ask for a path to a file or directory. The path may be
                      non-existing and should be used when a plug-in needs an
                      output destination, i.e., the file to export to, or a
                      directory where the output files should be placed.
                    </para>
                  </listitem>
                </varlistentry>
              </variablelist>
              <para>
                You can also create a
                <classname docapi="net.sf.basedb.core">PluginParameter</classname>
                with a null name and
                <classname docapi="net.sf.basedb.core">ParameterType</classname>.
                In this case, the web client will not ask for input from the user, instead
                it is used as a section header, allowing you to group parameters into
                different sections which increase the readability of the input
                parameters page.
              </para>
<programlisting language="java">
PluginParameter firstSection = new PluginParameter(null, "First section", null, null);
PluginParameter secondSection = new PluginParameter(null, "Second section", null, null);
// ...

parameters.add(firstSection);
parameters.add(firstParameterInFirstSection);
parameters.add(secondParameteInFirstSection);

parameters.add(secondSection);
parameters.add(firstParameterInSecondSection);
parameters.add(secondParameteInSecondSection);
</programlisting>
            </listitem>
          </varlistentry>
          <varlistentry>
            <term>
              <methodsynopsis language="java">
                <modifier>public</modifier>
                <void />
                <methodname>configure</methodname>
                <methodparam>
                  <type>GuiContext</type>
                  <parameter>context</parameter>
                </methodparam>
                <methodparam>
                  <type>Request</type>
                  <parameter>request</parameter>
                </methodparam>
                <methodparam>
                  <type>Response</type>
                  <parameter>response</parameter>
                </methodparam>
              </methodsynopsis>
            </term>
            <listitem>
              <para>
                Sends parameter values entered by the user for processing by the plug-in.
                The plug-in must validate that the parameter values are
                correct and then store them in database.
              </para>
              
              <important>
                <para>
                No validation is done by the core, except converting the input to the
                correct object type, i.e. if the plug-in asked for a
                <classname>Float</classname>
                the input string is parsed and converted to a 
                <classname>Float</classname>. If you have extended the
                <classname docapi="net.sf.basedb.core.plugin">AbstractPlugin</classname>
                class it is very easy to validate the parameters with the
                <methodname>AbstractPlugin.validateRequestParameters()</methodname>
                method. This method takes the same list of
                <classname docapi="net.sf.basedb.core">PluginParameter</classname>:s
                as used in the
                <classname docapi="net.sf.basedb.core">RequestInformation</classname>
                object and uses that information for validation. It returns null or a
                list of
                <exceptionname>Throwable</exceptionname>:s that can be 
                given directly to the <code>response.setError()</code>
                methods.
                </para>
              </important>
              <para>
                When the parameters have been validated, they need to be stored
                in the database. Once again, it is very easy, if you use one of the
                <methodname>AbstractPlugin.storeValue()</methodname>
                or
                <methodname>AbstractPlugin.storeValues()</methodname>
                methods.
              </para>
              <para>
                The configure method works much like the <methodname>Plugin.run()</methodname> 
                method. It must return the result in the <classname docapi="net.sf.basedb.core.plugin">Response</classname> object, 
                and should not throw any exceptions.
              </para>
              
              <example id="net.sf.basedb.core.plugin.InteractivePlugin.configure">
                <title>
                  Configuration implementation building on the examples above
                </title>
<programlisting language="java">
public void configure(GuiContext context, Request request, Response response)
{
   String command = request.getCommand();
   try
   {
      if (command.equals(Request.COMMAND_CONFIGURE_PLUGIN))
      {
         // TODO
      }
      else if (command.equals(Request.COMMAND_CONFIGURE_JOB))
      {
         // Validate user input
         List&lt;Throwable&gt; errors = 
            validateRequestParameters(getConfigureJob().getParameters(), request);
         if (errors != null)
         {
            response.setError(errors.size() +
               " invalid parameter(s) were found in the request", errors);
            return;
         }
         
         // Store user input
         storeValue(job, request, fileParameter);
         storeValue(job, request, updateExistingParameter);
         
         // We are happy and done
         response.setDone("Job configuration complete", Job.ExecutionTime.SHORT);
         // TODO - check file size to make a better estimate of execution time
      }
   }
   catch (Throwable ex)
   {
      response.setError(ex.getMessage(), Arrays.asList(ex));
   }
}</programlisting>
              </example>
              
              <para>
                Note that the call to
                <methodname>response.setDone()</methodname>
                has a second parameter
                <constant>Job.ExecutionTime.SHORT</constant>. It is an indication 
                about how long time it will take to execute the
                plug-in. This is of interest for job queue managers which probably
                does not want to start too many long-running jobs at the same time
                blocking the entire system. Please try to use this parameter wisely and
                not use <constant>Job.ExecutionTime.SHORT</constant>
                out of old habit all the time.
              </para>
              <para>
                The <classname docapi="net.sf.basedb.core.plugin">Response</classname> class also has a
                <methodname>setContinue()</methodname>
                method which tells the core that the plug-in needs more parameters,
                i.e. the core will then call
                <methodname>getRequestInformation()</methodname>
                again with the new command, let the user enter values, and then call
                <methodname>configure()</methodname>
                with the new values. This process is repeated until the plug-in
                reports that it is done or an error occurs.
              </para>
              <tip>
                <para>
                You do not have to store all values the plug-in asked for in the
                first place. You may even choose to store different values than
                those that were entered. For example, you might ask for the mass
                and height of a person and then only store the body mass index,
                which is calculated from those values.
                </para>
              </tip>
              <para>
                An important note is that during this iteration it is the same instance
                of the plug-in that is used. However, no parameter values are stored in
                the database until the plugin sends a 
                <methodname>response.setDone()</methodname>.
                After that, the plug-in instance is usually discarded, and a job is placed
                in the job queue. The execution of the plug-in happens in a new instance 
                and maybe on a different server. This means that a plug-in can't store
                state from the configuration phase internally and expect it to be there
                in the execution phase. Everything the plug-in needs to do it's job must
                be stored as parameters in the database.
              </para>
              <para>
                The only exception to the above rule is if the plug-in answers with
                <methodname>Response.setExecuteImmediately()</methodname>
                or <methodname>Response.setDownloadImmediately()</methodname>.
                Doing so bypasses the entire job queue system and requests that the
                job is started immediately. This is a permission that has to
                be granted to each plug-in by the server administrator. If the 
                plug-in has this permission, the same object instance that
                was used in the configuration phase is also used in the execution
                phase. This is the only case where a plug-in can retain internal 
                state between the two phases.
              </para>
              
            </listitem>
          </varlistentry>
        </variablelist>
      </sect3>
    
    </sect2>
  
    <sect2 id="plugin_developer.api.callsequence">
      <title>How the BASE core interacts with the plug-in when...</title>
      
      <para>
        This section describes how the BASE core interacts with the
        plug-in in a number of use cases. We will outline the 
        order the methods are invoked on the plug-in.
      </para>
      
      <sect3 id="plugin_developer.api.callsequence.install">
        <title>Installing a plug-in</title>
        
        <para>
          When a plug-in is installed the core is eager to find out
          information about the plug-in. To do this it calls the
          following methods in this order:
        </para>
        
        <orderedlist>
        <listitem>
          <para>
          A new instance of the plug-in class is created. The plug-in must
          have a public no-argument constructor.
          </para>
        </listitem>
        
        <listitem>
          <para>
          Calls are made to <methodname>Plugin.getMainType()</methodname>,
          <methodname>Plugin.supportsConfigurations()</methodname>,
          <methodname>Plugin.requiresConfiguration()</methodname> and
          <methodname>Plugin.getAbout()</methodname> to find out information
          about the plug-in. This is the only time these methods are called.
          The information that is returned by them are copied and stored in 
          the database for easy access.
          </para>
          
          <note>
            <para>
            The <methodname>Plugin.init()</methodname> method is
            never called during plug-in installation.
            </para>
          </note>
        </listitem>
        
        <listitem>
          <para>
          If the plug-in implements the <interfacename docapi="net.sf.basedb.core.plugin">InteractivePlugin</interfacename>
          interface the <methodname>InteractivePlugin.getGuiContexts()</methodname>
          method is called. This is the only time this method is called and the information it
          returns are copied and stored in the database.
          </para>
        </listitem>
        
        <listitem>
          <para>
          If the server admin decided to use the plug-in permission system, the
          <methodname>Plugin.getPermissions()</methodname> method is called.
          The returned information is copied and stored in the database.
          </para>
        </listitem>
        
        </orderedlist>
      </sect3>
    
      <sect3 id="plugin_developer.api.callsequence.configure">
        <title>Configuring a plug-in</title>
        
        <para>
          The plug-in must implement the <interfacename docapi="net.sf.basedb.core.plugin">InteractivePlugin</interfacename>
          interface and the <methodname>Plugin.supportsConfigurations()</methodname> method
          must return <constant>TRUE</constant>. The configuration is done with
          a wizard-like interface (see <xref linkend="plugins.configuration.wizard" />). 
          The same plug-in instance is used throughout the entire configuration sequence.
        </para>
        
        <orderedlist>
        <listitem>
          <para>
          A new instance of the plug-in class is created. The plug-in must
          have a public no-argument constructor.
          </para>
        </listitem>
        
        <listitem>
          <para>
          The <methodname>Plugin.init()</methodname> method is called. The 
          <varname>job</varname> parameter is null.
          </para>
        </listitem>
        
        <listitem id="plugin_developer.api.callsequence.configure.requestinformation">
          <para>
          The <methodname>InteractivePlugin.getRequestInformation()</methodname> 
          method is called. The <varname>context</varname> parameter is <constant>null</constant>
          and the <varname>command</varname> is the value of the string 
          constant <constant>Request.COMMAND_CONFIGURE_PLUGIN</constant> 
          (<constant>_config_plugin</constant>).
          </para>
        </listitem>
        
        <listitem id="plugin_developer.api.callsequence.configure.form">
          <para>
          The web client process the returned information and displays a form for user
          input. The plug-in will have to wait some time while the user enters 
          data.
          </para>
        </listitem>
        
        <listitem>
          <para>
          The <methodname>InteractivePlugin.configure()</methodname> method
          is called. The <varname>context</varname> parameter is still 
          <constant>null</constant> and the <varname>request</varname>
          parameter contains the parameter values entered by the user.
          </para>
        </listitem>
        
        <listitem>
          <para>
          The plug-in must validate the values and decide whether they should be
          stored in the database or not. We recommend that you use the
          methods in the <classname docapi="net.sf.basedb.core.plugin">AbstractPlugin</classname> class for this.
          </para>
        </listitem>
        
        <listitem>
          <para>
          The plug-in can choose between three different respones:
          
          <itemizedlist>
          <listitem>
            <para>
            <methodname>Response.setDone()</methodname>: The configuration
            is complete. The core will write any configuation changes to the
            database, call the <methodname>Plugin.done()</methodname> method and 
            then discard the plug-in instance.
            </para>
          </listitem>
          
          <listitem>
            <para>
            <methodname>Response.setError()</methodname>: There was one or more
            errors. The web client will display the error messages for the user and
            allow the user to enter new values. The process continues with 
            step <xref linkend="plugin_developer.api.callsequence.configure.form" />.
            </para>
          </listitem>
          
          <listitem>
            <para>
            <methodname>Response.setContinue()</methodname>: The parameters are correct
            but the plug-in wants more parameters. The process continues with
            step <xref linkend="plugin_developer.api.callsequence.configure.requestinformation" />
            but the <varname>command</varname> has the value that was passed to the 
            <methodname>setContinue()</methodname> method.
            </para>
          </listitem>
          </itemizedlist>
          
          </para>
        </listitem>
        </orderedlist>

      </sect3>    
    
      <sect3 id="plugin_developer.api.callsequence.context">
        <title>Checking if a plug-in can be used in a given context</title>
        
        <para>
          If the plug-in is an <interfacename docapi="net.sf.basedb.core.plugin">InteractivePlugin</interfacename>
          it has specified in which contexts it can be used by the 
          information returned from <methodname>InteractivePlugin.getGuiContexts()</methodname>
          method. The web client uses this information to decide whether,
          for example, a <guibutton>Run plugin</guibutton>
          button should be displayed on a page or not. However, this is not
          always enough to know whether the plug-in can be used or not.
          For example, a raw data importer plug-in cannot be used to
          import raw data if the raw bioassay already has data.
          So, when the user clicks the button, the web client will
          load all plug-ins that possibly can be used in the given context
          and let each one of them check whether they can be used or not.
        </para>
        
      
        <orderedlist>
        <listitem>
          <para>
          A new instance of the plug-in class is created. The plug-in must
          have a public no-argument constructor.
          </para>
        </listitem>
        
        <listitem>
          <para>
          The <methodname>Plugin.init()</methodname> method is called.
          The <varname>job</varname> parameter is <constant>null</constant>. 
          The <varname>configuration</varname> parameter is <constant>null</constant>
          if the plug-in does not have any configuration parameters.
          </para>
        </listitem>
        
        <listitem>
          <para>
          The <methodname>InteractivePlugin.isInContext()</methodname>
          is called. If the context is a list context, the <varname>item</varname>
          parameter is null, otherwise the current item is passed. The plug-in 
          should return <constant>null</constant> if it can be used under the 
          current circumstances, or a message explaining why not.
          </para>
        </listitem>
        
        <listitem>
          <para>
            After this, <methodname>Plugin.done()</methodname> is called and 
            the plug-in instance is discarded. If there are
            several configurations for a plug-in, this procedure is repeated
            for each configuration.
          </para>
        </listitem>
        </orderedlist>
      </sect3>
    
      <sect3 id="plugin_developer.api.callsequence.job">
        <title>Creating a new job</title>
        
        <para>
          If the web client found that the plug-in could be
          used in a given context and the user selected the plug-in,
          the job configuration sequence is started. It is a wizard-like interface
          identical to the configuration wizard. In fact, the same JSP pages,
          and calling sequence is used. See <xref linkend="plugin_developer.api.callsequence.configure" />.
          We do not repeat everything here. There are a few differences:
        </para>
        
        <itemizedlist>
        <listitem>
          <para>
          The <varname>job</varname> parameter is not null, but it does not contain
          any parameter values to start with. The plug-in should use this
          object to store job-related parameter values. The 
          <varname>configuration</varname> parameter is <constant>null</constant> 
          if the plug-in is started without configuration. In any case,
          the configuration values are write-protected and cannot be modified.
          </para>
        </listitem>
        
        <listitem>
          <para>
          The first call to <methodname>InteractivePlugin.getRequestInformation()</methodname>
          is done with <constant>Request.COMMAND_CONFIGURE_JOB</constant> (<constant>_configjob</constant>)
          as the command. The <varname>context</varname> parameter reflects the 
          current context.
          </para>
        </listitem>
        
        <listitem>
          <para>
          When calling <methodname>Response.setDone()</methodname> the plug-in
          should use the variant that takes an estimated execution time.
          If the plug-in has support for immediate execution or download 
          (export plug-ins only), it can also respond with 
          <methodname>Response.setExecuteImmediately()</methodname> or
          <methodname>Response.setDownloadImmediately()</methodname>.
          </para>
          
          <para>
          If the plug-in requested and was granted immediate execution or 
          download the same plug-in instance is used to execute the plug-in.
          This may be done with the same or a new thread. Otherwise, a new
          job is added to the job queue, the parameter value are saved
          and the plug-in instance is discarded after calling the
          <methodname>Plugin.done()</methodname> method.
          </para>
        </listitem>
        </itemizedlist>
      </sect3>
    
      <sect3 id="plugin_developer.api.callsequence.execute">
        <title>Executing a job</title>
        
        <para>
          Normally, the creation of a job and the execution of it are
          two different events. The execution may as well be done on a
          different server. See <xref linkend="installation_upgrade.jobagents" />.
          This means that the execution takes place in a different instance
          of the plug-in class than what was used for creating the job.
          The exception is if a plug-in supports immediate execution or download.
          In this case the same instance is used, and it is, of course,
          always executed on the web server.
        </para>
        
        <orderedlist>
        <listitem>
          <para>
          A new instance of the plug-in class is created. The plug-in must
          have a public no-argument constructor.
          </para>
        </listitem>
        
        <listitem>
          <para>
          The <methodname>Plugin.init()</methodname> method is called.
          The <varname>job</varname> parameter contains the job
          configuration parameters. The <varname>configuration</varname> parameter 
          is <constant>null</constant> if the plug-in does not have any 
          configuration parameters.
          </para>
        </listitem>
        
        <listitem>
          <para>
          The <methodname>Plugin.run()</methodname> method is called. 
          It is finally time for the plug-in to do the work it has been
          designed for. This method should not throw any exceptions.
          Use the <methodname>Response.setDone()</methodname>
          method to report success or the <methodname>Response.setError()</methodname>
          to report errors.
          </para>
        </listitem>
        
        <listitem>
          <para>
          In both cases the <methodname>Plugin.done()</methodname>
          method is called and the plug-in instance is discarded.
          </para>
        </listitem>
        
        </orderedlist>
      </sect3>
    
    </sect2>
  
    <sect2 id="plugin_developer.api.jspparameters">
      <title>Using custom JSP pages for parameter input</title>

        <para>
          This is an advanced option for plug-ins that require a different interface for
          specifying plug-in parameters than the default list showing one parameter at a
          time. This feature is used by setting the 
          <methodname>RequestInformation.getJspPage()</methodname>
          property when constructing the request information object. If this property has
          a non-null value, the web client will send the browser to the specified JSP page
          instead of to the generic parameter input page.
        </para>
        <para>
          When setting the JSP page you can either specify an absolute path
          of only the filename of the JSP file. If only the filename is specified, 
          the JSP file is expected to be located in a special location, generated
          from the package name of your plug-in. If the plug-in is located in the package
          <classname>org.company</classname> the JSP file must be located in
          <filename class="directory">&lt;base-dir&gt;/www/plugins/org/company/</filename>.
        </para>
        
        <para>
          An absolute path starts with '/' and may or may not include the
          root directory of the BASE installation. If, for example, BASE is intalled to
          <userinput>http://your.base.server.com/base</userinput>, the following absolute 
          paths are equivalent  <filename>/base/path/to/file.jsp</filename>, 
          <filename>/path/to/file.jsp</filename>.
        </para>
        <para>
          In both cases, please note that the browser still thinks that it is showing 
          the regular parameter input page at the usual location:
          <filename class="directory">&lt;base-dir&gt;/www/common/plugin/index.jsp</filename>.
          All links in your JSP page should be relative to that directory.
        </para>
        <para>
          Even if you use your own JSP page we recommend that you use the built-in
          facility for passing the parameters back to the plug-in. For this to work you
          must:
        </para>
        <itemizedlist spacing="compact">
          <listitem>
            <simpara>
            Generate the list of <classname docapi="net.sf.basedb.core">PluginParameter</classname> 
            objects as usual.
            </simpara>
          </listitem>
          <listitem>
            <simpara>
              Name all your input fields in the JSP like:
              <parameter>
                parameter:<replaceable>name-of-parameter</replaceable>
              </parameter>
            </simpara>
<programlisting language="java:nogutter">
// Plug-in generate PluginParameter
StringParameterType stringPT = new StringParameterType(255, null, true);
PluginParameter one = new PluginParameter("one", "One", "First string", stringPT);
PluginParameter two = new PluginParameter("two", "Two", "Second string", stringPT);

// JSP should name fields as:
First string: &lt;input type="text" name="parameter:one"&gt;&lt;br&gt;
Second string: &lt;input type="text" name="parameter:two"&gt;
</programlisting>
          </listitem>
          <listitem>
          <simpara>
            Send the form to
            <filename>index.jsp</filename>
            with the <varname>ID</varname>, 
            <varname>cmd</varname> and <varname>requestId</varname> 
            parameters as shown below.
          </simpara>
<programlisting language="xml">
&lt;form action="index.jsp" method="post"&gt;
&lt;input type="hidden" name="ID" value="&lt;%=ID%&gt;"&gt;
&lt;input type="hidden" name="requestId" value="&lt;%=request.getParameter("requestId")%&gt;"&gt;
&lt;input type="hidden" name="cmd" value="SetParameters"&gt;
...
&lt;/form&gt;
</programlisting>
          <simpara>
            The <varname>ID</varname> is the session ID for the logged
            in user and is required. The <varname>requestId</varname>
            is the ID for this particular plug-in/job configuration
            sequence. It is optional, but we recommend that you use it
            since it protects your plug-in from getting mixed up with 
            other plug-in configuration wizards. The <varname>cmd</varname>
            tells BASE to send the parameters to the plug-in for validation
            and saving.
          </simpara>

          <simpara>
            Values are sent as strings to BASE that converts them to the
            proper value type before they are passed on to your plug-in. 
            However, there is one case that can't be
            accurately represented with custom JSP pages, namely 'null' values.
            A null value is sent by not sending any value at all. This is not
            possible with a fixed form. It is of course possible to add some custom
            JavaScript that adds and removes form elements as needed, but it is
            also possible to let the empty string represent null. Just include a 
            hidden parameter like this if you want an empty value for the 'one' 
            parameter converted to null:
          </simpara>
<programlisting language="xml">
&lt;input type="hidden" name="parameter:one:emptyIsNull" value="1"&gt;
</programlisting>
          </listitem>
          </itemizedlist>
          
          <para>
            If you want a &gbCancel; button to abort the configuration
            you should reload the page with with the url:
            <uri>index.jsp?ID=&lt;%=ID%&gt;&amp;cmd=CancelWizard</uri>. This
            allows BASE to clean up resources that has been put in global
            session variables.
          </para>
          
            <para>
              In your JSP page you will probably need to access some information like the
              <classname docapi="net.sf.basedb.core">SessionControl</classname>, <classname docapi="net.sf.basedb.core">Job</classname>
              and possible even the <classname docapi="net.sf.basedb.core">RequestInformation</classname>
              object created by your plug-in.
            </para>
<programlisting language="java">
// Get session control and its ID (required to post to index.jsp)
final SessionControl sc = Base.getExistingSessionControl(pageContext, true);
final String ID = sc.getId();

// Get information about the current request to the plug-in
PluginConfigurationRequest pcRequest = 
   (PluginConfigurationRequest)sc.getSessionSetting("plugin.configure.request");
PluginDefinition plugin = 
   (PluginDefinition)sc.getSessionSetting("plugin.configure.plugin");
PluginConfiguration pluginConfig = 
   (PluginConfiguration)sc.getSessionSetting("plugin.configure.config");
PluginDefinition job = 
   (PluginDefinition)sc.getSessionSetting("plugin.configure.job");
RequestInformation ri = pcRequest.getRequestInformation();
</programlisting>

    </sect2>
  </sect1>

  <sect1 id="plugin_developer.import">
    <title>Import plug-ins</title>

    <para>
      A plugin becomes an import plugin simply by returning 
      <constant>Plugin.MainType.IMPORT</constant>
      from the <methodname>Plugin.getMainType()</methodname> method.
    </para>
    
    <sect2 id="plugin_developer.import.autodetect">
      <title>Autodetect file formats</title>
      <para>
        BASE has built-in functionality for autodetecting file formats. 
        Your plug-in can be part of that feature if it reads it data
        from a single file. It must also implement the 
        <interfacename docapi="net.sf.basedb.core.plugin">AutoDetectingImporter</interfacename>
        interface. 
      </para>

      <sect3 id="plugin_developer.api.interfaces.autodetecting">
        <title>The net.sf.basedb.core.plugin.AutoDetectingImporter interface</title>

        <variablelist>
        <varlistentry>
          <term>
            <methodsynopsis language="java">
              <modifier>public</modifier>
              <type>boolean</type>
              <methodname>isImportable</methodname>
              <methodparam>
                <type>InputStream</type>
                <parameter>in</parameter>
              </methodparam>
              <exceptionname>BaseException</exceptionname>
            </methodsynopsis>
          </term>
          <listitem>
            <para>
              Check the input stream if it seems to contain data that can be imported by
              the plugin. Usually it means scanning a few lines for some header 
              mathing a predefined string or a regexp.
            </para>
            <para>
              The <classname docapi="net.sf.basedb.plugins">AbstractFlatFileImporter</classname> implements this method 
              by reading the headers from the input stream and checking if 
              it stopped at an unknown type of line or not:
                <programlisting language="java">
public final boolean isImportable(InputStream in)
   throws BaseException
{
   FlatFileParser ffp = getInitializedFlatFileParser();
   ffp.setInputStream(in);
   try
   {
      ffp.nextSection();
      FlatFileParser.LineType result = ffp.parseHeaders();
      if (result == FlatFileParser.LineType.UNKNOWN)
      {
         return false;
      }
      else
      {
         return isImportable(ffp);
      }
   }
   catch (IOException ex)
   {
      throw new BaseException(ex);
   }
}
</programlisting>
            </para>
            <para>
              Note that the input stream doesn't have to be a text file. 
              It can be any type of file, for example a binary or an XML file. 
              In the case of an XML file you would need to validate the entire 
              input stream in order to be a 100% sure that it is a valid
              xml file, but we recommend that you only check the first few XML tags, 
              for example, the &lt;!DOCTYPE &gt; declaration and/or the root element 
              tag.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <methodsynopsis language="java">
              <modifier>public</modifier>
              <void/>
              <methodname>doImport</methodname>
              <methodparam>
                <type>InputStream</type>
                <parameter>in</parameter>
              </methodparam>
              <methodparam>
                <type>ProgressReporter</type>
                <parameter>progress</parameter>
              </methodparam>
              <exceptionname>BaseException</exceptionname>
            </methodsynopsis>
          </term>
          <listitem>
            <para>
              Parse the input stream and import all data that is found. 
              This method is of course only called if the 
              <methodname>isImportable()</methodname> has returned true. Note 
              however that the input stream is reopened at the start of the 
              file. It may even be the case that the <methodname>isImportable()</methodname>
              method is called on one instance of the plugin and the 
              <methodname>doImport()</methodname> method is called on another.
              Thus, the <methodname>doImport()</methodname> can't rely on any state set 
              by the <methodname>isImportable()</methodname> method.
            </para>
          </listitem>
        </varlistentry>
        </variablelist>
        
        <tip>
          <title>Try casting to ImportInputStream</title>
          <para>
            As of BASE 2.9 the auto-detect functionality uses a
            <classname docapi="net.sf.basedb.core.plugin">ImportInputStream</classname>
            as the <varname>in</varname> parameter. This class contains some
            metadata about the file the input stream is originating from. The most
            useful feature is the possibility to get information about the
            character set used in the file. This makes it possible to open text
            files using the correct character set.
          </para>
          <programlisting language="java">
String charset = Config.getCharset(); // Default value
if (in instanceof ImportInputStream)
{
   ImportInputStream iim = (ImportInputStream)in;
   if (iim.getCharacterSet() != null) charset = iim.getCharacterSet();
}
Reader reader = new InputStreamReader(in, Charset.forName(charset)));
</programlisting>
        </tip>        
      </sect3>
      
      <sect3 id="plugin_developer.import.autodetect.callsequence">
        <title>Call sequence during autodetection</title>
        
        <para>
          The call sequence for autodetection resembles the call sequence for 
          checking if the plug-in can be used in a given context.
        </para>

        <orderedlist>
        <listitem>
          <para>
          A new instance of the plug-in class is created. The plug-in must
          have a public no-argument constructor.
          </para>
        </listitem>
        
        <listitem>
          <para>
          The <methodname>Plugin.init()</methodname> method is called.
          The <varname>job</varname> parameter is <constant>null</constant>. 
          The <varname>configuration</varname> parameter is <constant>null</constant>
          if the plug-in does not have any configuration parameters.
          </para>
        </listitem>
        
        <listitem>
          <para>
          If the plug-in is interactive the the <methodname>InteractivePlugin.isInContext()</methodname>
          is called. If the context is a list context, the <varname>item</varname>
          parameter is null, otherwise the current item is passed. The plug-in 
          should return <constant>null</constant> if it can be used under the 
          current circumstances, or a message explaining why not.
          </para>
        </listitem>
        
        <listitem>
          <para>
          If the plug-in can be used the <methodname>AutoDetectingImporter.isImportable()</methodname>
          method is called to check if the selected file is importable or not.
          </para>
        </listitem>
        
        <listitem>
          <para>
          After this, <methodname>Plugin.done()</methodname> is called and 
          the plug-in instance is discarded. If there are
          several configurations for a plug-in, this procedure is repeated
          for each configuration. If the plug-in can be used without 
          a configuration the procedure is also repeated without
          configuration parameters.
          </para>
        </listitem>
        
        <listitem>
          <para>
          If a single plug-in was found the user is taken to the regular
          job configuration wizard. A new plug-in instance is created for
          this. If more than one plug-in was found the user is presented
          with a list of the plug-ins. After selecting one of them the
          regular job configuration wizard is used with a new plug-in instance.
          </para>
        </listitem>
        
        </orderedlist>
        
      </sect3>

    </sect2>
    
    <sect2 id="plugin_developer.import.abstractflatfileimporter">
      <title>The AbstractFlatFileImporter superclass</title>
      <para>
        The <classname docapi="net.sf.basedb.plugins">AbstractFlatFileImporter</classname> is a very useful abstract 
        class to use as a superclass for your own import plug-ins. It can be used
        if your plug-in uses regular text files that can be parsed by an instance of the 
        <classname docapi="net.sf.basedb.util">net.sf.basedb.util.FlatFileParser</classname> class. This class parses a file
        by checking each line against a few regular expressions. Depending on which regular
        expression matches the line, it is classified as a header line, a section line, 
        a comment, a data line, a footer line or unknown. Header lines are inspected as a group,
        but data lines individually, meaning that it consumes very little memory since only 
        a few lines at a time needs to be loaded.
      </para>
      
      <para>
        The <classname docapi="net.sf.basedb.plugins">AbstractFlatFileImporter</classname> defines 
        <classname docapi="net.sf.basedb.core">PluginParameter</classname> objects
        for each of the regular expressions and other parameters used by the parser. It also
        implements the <methodname>Plugin.run()</methodname> method and does most of 
        the ground work for instantiating a <methodname>FlatFileParser</methodname> and 
        parsing the file. What you have to do in your plugin is to put together the 
        <classname docapi="net.sf.basedb.core">RequestInformation</classname> objects
        for configuring the plugin and creating a job and implement the 
        <methodname>InteractivePlugin.configure()</methodname> method for validating and 
        storing the parameters. You should also implement or override some methods 
        defined by <classname>AbstractFlatFileImporter</classname>.
      </para>

      <para>
      Here is what you need to do:
      </para>

      <itemizedlist>
      <listitem>
        <para>
        Implement the <methodname>Plugin.getAbout()</methodname> method. See
        <xref linkend="plugin_developer.api.interfaces.plugin" /> for more information.
        </para>
      </listitem>
      
      <listitem>
        <para>
        Implement the <interfacename docapi="net.sf.basedb.core.plugin">InteractivePlugin</interfacename> methods.
        See <xref linkend="plugin_developer.api.interfaces.interactive" /> for more information. Note that the 
        <classname docapi="net.sf.basedb.plugins">AbstractFlatFileImporter</classname>
        has defined many parameters for regular expressions used by the parser
        already. You should just pick them and put in your <classname docapi="net.sf.basedb.core">RequestInformation</classname>
        object.
        </para>
        
        <programlisting language="java">
// Parameter that maps the items name from a column
private PluginParameter&lt;String&gt; nameColumnMapping;

// Parameter that maps the items description from a column
private PluginParameter&lt;String&gt; descriptionColumnMapping;

private RequestInformation getConfigurePluginParameters(GuiContext context)
{
   if (configurePlugin == null)
   {
      // To store parameters for CONFIGURE_PLUGIN
      List&lt;PluginParameter&lt;?&gt;&gt; parameters = 
         new ArrayList&lt;PluginParameter&lt;?&gt;&gt;();

      // Parser regular expressions - from AbstractFlatFileParser
      parameters.add(parserSection);
      parameters.add(headerRegexpParameter);
      parameters.add(dataHeaderRegexpParameter);
      parameters.add(dataSplitterRegexpParameter);
      parameters.add(ignoreRegexpParameter);
      parameters.add(dataFooterRegexpParameter);
      parameters.add(minDataColumnsParameter);
      parameters.add(maxDataColumnsParameter);

      // Column mappings
      nameColumnMapping = new PluginParameter&lt;String&gt;(
         "nameColumnMapping",
         "Name",
         "Mapping that picks the items name from the data columns",
         new StringParameterType(255, null, true)
      );
    
      descriptionColumnMapping = new PluginParameter&lt;String&gt;(
        "descriptionColumnMapping",
        "Description",
        "Mapping that picks the items description from the data columns",
        new StringParameterType(255, null, false)
      );

      parameters.add(mappingSection);
      parameters.add(nameColumnMapping);
      parameters.add(descriptionColumnMapping);
      
      configurePlugin = new RequestInformation
      (
         Request.COMMAND_CONFIGURE_PLUGIN,
         "File parser settings",
         "",
         parameters
      );

   }
   return configurePlugin;
}
</programlisting>
      </listitem>
      
      <listitem>
        <para>
        Implement/override some of the methods defined by 
        <classname>AbstractFlatFileParser</classname>. The most important
        methods are listed below.
        </para>
      </listitem>
      
      </itemizedlist>

      <variablelist>
      <varlistentry>
        <term>
          <methodsynopsis language="java">
            <modifier>protected</modifier>
            <type>FlatFileParser</type>
            <methodname>getInitializedFlatFileParser</methodname>
            <exceptionname>BaseException</exceptionname>
          </methodsynopsis>
        </term>
        <listitem>
          <para>
          The method is called to create a <classname docapi="net.sf.basedb.util.parser">FlatFileParser</classname>
          and set the regular expressions that should be used for parsing the file.
          The default implementation assumes that your plug-in has used the built-in
          <classname docapi="net.sf.basedb.core">PluginParameter</classname> objects and has stored the values
          at the configuration level. You should override this method if you need to
          initialise the parser in a different way. See for example the
          code for the <classname docapi="net.sf.basedb.plugins">PrintMapFlatFileImporter</classname> plug-in which
          has a fixed format and doesn't use configurations.
          </para>
          <programlisting language="java">
@Override
protected FlatFileParser getInitializedFlatFileParser()
   throws BaseException
{
   FlatFileParser ffp = new FlatFileParser();
   ffp.setSectionRegexp(Pattern.compile("\\[(.+)\\]"));
   ffp.setHeaderRegexp(Pattern.compile("(.+)=,(.*)"));
   ffp.setDataSplitterRegexp(Pattern.compile(","));
   ffp.setDataFooterRegexp(Pattern.compile(""));
   ffp.setMinDataColumns(12);
   return ffp;
}
</programlisting>
        </listitem>
      </varlistentry>
      
      <varlistentry>
        <term>
          <methodsynopsis language="java">
            <modifier>protected</modifier>
            <type>boolean</type>
            <methodname>isImportable</methodname>
            <methodparam>
              <type>FlatFileParser</type>
              <parameter>ffp</parameter>
            </methodparam>
            <exceptionname>IOException</exceptionname>
          </methodsynopsis>
        </term>
        <listitem>
          <para>
          This method is called from the <methodname>isImportable(InputStream)</methodname>
          method, AFTER <methodname>FlatFileParser.nextSection()</methodname> and
          <methodname>FlatFileParser.parseHeaders()</methodname> has been called
          a single time and if the <methodname>parseHeaders</methodname> method didn't
          stop on an unknown line. The default implementation of this method always returns
          TRUE, since obviously some data has been found. A subclass may override this method
          if it wants to do more checks, for example, make that a certain header is present
          with a certain value. It may also continue parsing the file. Here is a code example from
          the <classname docapi="net.sf.basedb.plugins">PrintMapFlatFileImporter</classname> which checks if a 
          <constant>FormatName</constant> header is present and contains either 
          <constant>TAM</constant> or <constant>MwBr</constant>.
          </para>
          
          <programlisting language="java">
/**
   Check that the file is a TAM or MwBr file.
   @return TRUE if a FormatName header is present and contains "TAM" or "MwBr", FALSE
      otherwise
*/
@Override
protected boolean isImportable(FlatFileParser ffp)
{
   String formatName = ffp.getHeader("FormatName");
   return formatName != null &amp;&amp; 
      (formatName.contains("TAM") || formatName.contains("MwBr"));
}
</programlisting>
        </listitem>
      </varlistentry>
      
      <varlistentry>
        <term>
          <methodsynopsis language="java">
            <modifier>protected</modifier>
            <void/>
            <methodname>begin</methodname>
            <methodparam>
              <type>FlatFileParser</type>
              <parameter>ffp</parameter>
            </methodparam>
            <exceptionname>BaseException</exceptionname>
          </methodsynopsis>
        </term>
        <listitem>
          <para>
          This method is called just before the parsing of the file
          begins. Override this method if you need to initialise some
          internal state. This is, for example, a good place to open 
          a <classname docapi="net.sf.basedb.core">DbControl</classname> object, read parameters from the 
          job and configuration and put them into more useful variables. The default 
          implementation does nothing, but we recommend that 
          <methodname>super.begin()</methodname> is always called.
          </para>
          <programlisting language="java">
// Snippets from the RawDataFlatFileImporter class
private DbControl dc;
private RawDataBatcher batcher;
private RawBioAssay rawBioAssay;
private Map&lt;String, String&gt; columnMappings;
private int numInserted;

@Override
protected void begin()
   throws BaseException
{
   super.begin();

   // Get DbControl
   dc = sc.newDbControl();
   rawBioAssay = (RawBioAssay)job.getValue(rawBioAssayParameter.getName());

   // Reload raw bioassay using current DbControl
   rawBioAssay = RawBioAssay.getById(dc, rawBioAssay.getId());
   
   // Create a batcher for inserting spots
   batcher = rawBioAssay.getRawDataBatcher();

   // For progress reporting
   numInserted = 0;
}         
</programlisting>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term>
          <methodsynopsis language="java">
            <modifier>protected</modifier>
            <void/>
            <methodname>handleHeader</methodname>
            <methodparam>
              <type>FlatFileParser.Line</type>
              <parameter>line</parameter>
            </methodparam>
            <exceptionname>BaseException</exceptionname>
          </methodsynopsis>
        </term>
        <listitem>
          <para>
          This method is called once for every header line that is found in 
          the file. The <varname>line</varname> parameter contains information
          about the header. The default implementation of this method does
          nothing.
          </para>
          <programlisting language="java">
@Override
protected void handleHeader(Line line) 
   throws BaseException
{
   super.handleHeader(line);
   if (line.name() != null &amp;&amp; line.value() != null)
   {
      rawBioAssay.setHeader(line.name(), line.value());
   }
}
</programlisting>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term>
          <methodsynopsis language="java">
            <modifier>protected</modifier>
            <void/>
            <methodname>handleSection</methodname>
            <methodparam>
              <type>FlatFileParser.Line</type>
              <parameter>line</parameter>
            </methodparam>
            <exceptionname>BaseException</exceptionname>
          </methodsynopsis>
        </term>
        <listitem>
          <para>
            This method is called once for each section that is found in the file.
            The <varname>line</varname> parameter contains information
            about the section. The default implementation of this method does
            nothing.
          </para>
        </listitem>
      </varlistentry>
      
      <varlistentry>
        <term>
          <methodsynopsis language="java">
            <modifier>protected abstract</modifier>
            <void/>
            <methodname>beginData</methodname>
            <exceptionname>BaseException</exceptionname>
          </methodsynopsis>
        </term>
        <listitem>
          <para>
          This method is called after the headers has been parsed, but before
          the first line of data. This is a good place to add code that 
          depends on information in the headers, for example, put
          together column mappings.
          </para>
          
          <programlisting language="java">
private Mapper reporterMapper;
private Mapper blockMapper;
private Mapper columnMapper;
private Mapper rowMapper;
// ... more mappers

@Override
protected void beginData()
{
   boolean cropStrings = ("crop".equals(job.getValue("stringTooLongError")));

   // Mapper that always return null; used if no mapping expression has been entered
   Mapper nullMapper = new ConstantMapper((String)null);
   
   // Column mappers
   reporterMapper = getMapper(ffp, (String)configuration.getValue("reporterIdColumnMapping"), 
      cropStrings ? ReporterData.MAX_EXTERNAL_ID_LENGTH : null, nullMapper);
   blockMapper = getMapper(ffp, (String)configuration.getValue("blockColumnMapping"), 
      null, nullMapper);
   columnMapper = getMapper(ffp, (String)configuration.getValue("columnColumnMapping"), 
      null, nullMapper);
   rowMapper = getMapper(ffp, (String)configuration.getValue("rowColumnMapping"), 
      null, nullMapper);
   // ... more mappers: metaGrid coordinate, X-Y coordinate, extended properties
   // ...
}
</programlisting>
          
        </listitem>
      </varlistentry>
        
      <varlistentry>
        <term>
          <methodsynopsis language="java">
            <modifier>protected abstract</modifier>
            <void/>
            <methodname>handleData</methodname>
            <methodparam>
              <type>FlatFileParser.Data</type>
              <parameter>data</parameter>
            </methodparam>
            <exceptionname>BaseException</exceptionname>
          </methodsynopsis>
        </term>
        <listitem>
          <para>
          This method is abstract and must be implemented by all subclasses. 
          It is called once for every data line in the the file.
          </para>

          <programlisting language="java">
// Snippets from the RawDataFlatFileImporter class
@Override
protected void handleData(Data data)
   throws BaseException
{
   // Create new RawData object
   RawData raw = batcher.newRawData();

   // External ID for the reporter
   String externalId = reporterMapper.getValue(data);
   
   // Block, row and column numbers
   raw.setBlock(blockMapper.getInt(data));
   raw.setColumn(columnMapper.getInt(data));
   raw.setRow(rowMapper.getInt(data));
   // ... more: metaGrid coordinate, X-Y coordinate, extended properties
   
   // Insert raw data to the database
   batcher.insert(raw, externalId);
   numInserted++;
}
</programlisting> 
          
        </listitem>
      </varlistentry>
      
      <varlistentry>
        <term>
          <methodsynopsis language="java">
            <modifier>protected</modifier>
            <void/>
            <methodname>end</methodname>
            <methodparam>
              <type>boolean</type>
              <parameter>success</parameter>
            </methodparam>
          </methodsynopsis>
        </term>
        <listitem>
          <para>
            Called when the parsing has ended, either because the end of
            file was reached or because an error has occurred. The subclass
            should close any open resources, ie. the <classname docapi="net.sf.basedb.core">DbControl</classname>
            object. The <varname>success</varname> parameter is <constant>true</constant>
            if the parsing was successful, <constant>false</constant> otherwise.
            The default implementation does nothing.
          </para>
          
          <programlisting language="java">
@Override
protected void end(boolean success)
   throws BaseException
{
   try
   {
      // Commit if the parsing was successful
      if (success)
      {
         batcher.close();
         dc.commit();
      }
   }
   catch (BaseException ex)
   {
      // Well, now we got an exception
      success = false;
      throw ex;
   }
   finally
   {
      // Always close... and call super.end()
      if (dc != null) dc.close();
      super.end(success);
   }
}     
</programlisting>         
        </listitem>
      </varlistentry>
      
      <varlistentry>
        <term>
          <methodsynopsis language="java">
            <modifier>protected</modifier>
            <type>String</type>
            <methodname>getSuccessMessage</methodname>
            <void/>
          </methodsynopsis>
        </term>
        <listitem>
          <para>
          This is the last method that is called, and it is only called if
          everything went suceessfully. This method allows a subclass to generate
          a short message that is sent back to the database as a final progress
          report. The default implementation returns null, which means that no
          message will be generated.
          </para>
          <programlisting language="java">
@Override
protected String getSuccessMessage()
{
   return numInserted + " spots inserted";
}
</programlisting>
        </listitem>
      </varlistentry>
      </variablelist>

      <para>
        The <classname docapi="net.sf.basedb.plugins">AbstractFlatFileImporter</classname> has a lot of
        other methods that you may use and/or override in your own plug-in.
        Check the javadoc for more information. 
      </para>

    </sect2>
  </sect1>

  <sect1 id="plugin_developer.export">
    <title>Export plug-ins</title>
    
    <para>
      Export plug-ins are plug-ins that takes data from BASE, and 
      prepares it for use with some external entity. Usually this
      means that data is taken from the database and put into a file
      with some well-defined file format.
      An export plug-in should return <constant>MainType.EXPORT</constant> from
      the <methodname>Plugin.getMainType()</methodname> method. 
    </para>
    
    <sect2 id="plugin_developer.export.download">
      <title>Immediate download of exported data</title>
      <para>
        An export plug-in may want to give the user a choice
        between saving the exported data in the BASE file system
        or to download it immediately to the client computer. With the basic 
        plug-in API the second option is not possible. The 
        <interfacename docapi="net.sf.basedb.core.plugin">ImmediateDownloadExporter</interfacename> is an
        interface that extends the <interfacename docapi="net.sf.basedb.core.plugin">Plugin</interfacename>
        interface to provide this functionality. If your export
        plug-in wants to provide immediate download functionality it must
        implement the <interfacename docapi="net.sf.basedb.core.plugin">ImmediateDownloadExporter</interfacename>
        interface.
      </para>
      
      <sect3 id="plugin_developer.export.immediatedownloadexporter">
      <title>The ImmediateDownloadExporter interface</title>
      <variablelist>
      <varlistentry>
        <term>
          <methodsynopsis language="java">
            <modifier>public</modifier>
            <void/>
            <methodname>doExport</methodname>
            <methodparam>
              <type>ExportOutputStream</type>
              <parameter>out</parameter>
            </methodparam>
            <methodparam>
              <type>ProgressReporter</type>
              <parameter>progress</parameter>
            </methodparam>
          </methodsynopsis>
        </term>
        <listitem>
          <para>
          Perform the export. The plug-in should write the
          exported data to the <varname>out</varname> stream.
          If the <varname>progress</varname> parameter is not null, 
          the progress should be reported at regular interval in the
          same manner as in the <methodname>Plugin.run()</methodname>
          method.
          </para>
        </listitem>
      </varlistentry>
      </variablelist>
      
      </sect3>
      
      <sect3 id="plugin_developer.export.exportoutputstream">
      <title>The ExportOutputStream class</title>
      
      <para>
        The <classname docapi="net.sf.basedb.core.plugin">ExportOutputStream</classname> is 
        an extension to the
        <classname>java.io.OutputStream</classname>. Use the regular 
        <methodname>write()</methodname> methods to write data to it.
        It also has some additional methods, which are used for setting 
        metadata about the generated file. These methods are useful, for
        example, when generating HTTP response headers.
      </para>
      
      <note>
        <para>
        These methods must be called before starting to write data to
        the <varname>out</varname> stream.
        </para>
      </note>
      
      <variablelist>
      <varlistentry>
        <term>
          <methodsynopsis language="java">
            <modifier>public</modifier>
            <void/>
            <methodname>setContentLength</methodname>
            <methodparam>
              <type>long</type>
              <parameter>contentLength</parameter>
            </methodparam>
          </methodsynopsis>
        </term>
        <listitem>
          <para>
          Set the total size of the exported data. Don't call this method if the
          total size is not known.
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term>
          <methodsynopsis language="java">
            <modifier>public</modifier>
            <void/>
            <methodname>setMimeType</methodname>
            <methodparam>
              <type>String</type>
              <parameter>mimeType</parameter>
            </methodparam>
          </methodsynopsis>
        </term>
        <listitem>
          <para>
          Set the MIME type of the file that is being generated.
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term>
          <methodsynopsis language="java">
            <modifier>public</modifier>
            <void/>
            <methodname>setCharacterSet</methodname>
            <methodparam>
              <type>String</type>
              <parameter>charset</parameter>
            </methodparam>
          </methodsynopsis>
        </term>
        <listitem>
          <para>
          Sets the character set used in text files. For example,
          UTF-8 or ISO-8859-1.
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term>
          <methodsynopsis language="java">
            <modifier>public</modifier>
            <void/>
            <methodname>setFilename</methodname>
            <methodparam>
              <type>String</type>
              <parameter>filename</parameter>
            </methodparam>
          </methodsynopsis>
        </term>
        <listitem>
          <para>
          Set a suggested name of the file that is being
          generated.
          </para>
        </listitem>
      </varlistentry>
      </variablelist>
      
      </sect3>
      
      <sect3 id="plugin_developer.export.callsequence">
        <title>Call sequence during immediate download</title>

      <para>
        Supporting immediate download also means that the method call
        sequence is a bit altered from the standard sequence described
        in <xref linkend="plugin_developer.api.callsequence.execute" />.
      </para>
      
      <itemizedlist>
      <listitem>
        <para>
        The plug-in must call <methodname>Response.setDownloadImmediately()</methodname>
        instead of <methodname>Response.setDone()</methodname> in <methodname>Plugin.configure()</methodname>
        to end the job configuration wizard. This requests that the core starts 
        an immediate download.
        </para>
        
        <note>
          <para>
          Even if an immediate download is requested by the plug-in this feature
          may have been disabled by the server administrator. If so, the plug-in
          can choose if the job should be added to job queue or if this is an
          error condition.
          </para>
        </note>
      </listitem>
      
      <listitem>
        <para>
        If immediate download is granted the web client will keep the
        same plug-in instance and call <methodname>ImmediateDownloadExporter.doExport()</methodname>.
        In this case, the <methodname>Plugin.run()</methodname> is never called.
        After the export, <methodname>Plugin.done()</methodname> is called as
        usual.
        </para>
        
      </listitem>

      <listitem>
        <para>
        If immediate download is not granted and the job is added to the job queue
        the regular job execution sequence is used.
        </para>
      </listitem>
      </itemizedlist>
      
      </sect3>

    </sect2>
    
    <sect2 id="plugin_developer.export.abstractexporter">
      <title>The AbstractExporterPlugin class</title>
    
      <para>
        This is an abstract superclass that will make it easier
        to implement export plug-ins that support immediate 
        download. It defines <classname docapi="net.sf.basedb.core">PluginParameter</classname>
        objects for asking a user about a path where the exported
        data should be saved and if existing files should be overwritten or not.
        If the user leaves the path empty the immediate download functionality
        should be used. It also contains implementations of both the 
        <methodname>Plugin.run()</methodname> method and the
        <methodname>ImmediateDownloadExporter.doExport()</methodname> method.
        Here is what you need to do in your own plug-in code (code examples are
        taken from the <classname docapi="net.sf.basedb.plugins">HelpExporter</classname>):
      </para>
      
      <itemizedlist>
      <listitem>
        <para>
          Your plug-in should extend the <classname docapi="net.sf.basedb.core.plugin">AbstractExporterPlugin</classname>
          class:
          <programlisting language="java">
public class HelpExporter
  extends AbstractExporterPlugin
  implements InteractivePlugin
</programlisting>
        </para>
      </listitem>
      
      <listitem>
        <para>
          You need to implement the 
          <methodname>InteractivePlugin.getRequestInformation()</methodname>
          method. Use the <methodname>getSaveAsParameter()</methodname>
          and <methodname>getOverwriteParameter()</methodname> methods defined in the 
          superclass to create plug-in parameters that asks for the file name to save
          to and if existing files can be overwritten or not.
          You should also check if the administrator has enabled the immediate execution
          functionality for your plug-in. If not, the only option is to 
          export to a file in the BASE file system and the filename is a
          required parameter.
          
          <programlisting language="java">
// Selected parts of the getRequestConfiguration() method
...
List&lt;PluginParameter&lt;?&gt;&gt; parameters = 
   new ArrayList&lt;PluginParameter&lt;?&gt;&gt;();
...
PluginDefinition pd = job.getPluginDefinition();
boolean requireFile = pd == null ? 
   false : !pd.getAllowImmediateExecution();

parameters.add(getSaveAsParameter(null, null, defaultPath, requireFile));
parameters.add(getOverwriteParameter(null, null));

configureJob = new RequestInformation
(
   Request.COMMAND_CONFIGURE_JOB,
   "Help exporter options",
   "Set Client that owns the helptexts, " +
     "the file path where the export file should be saved",
   parameters
);
....
return configureJob;
</programlisting>
        </para>
      </listitem>

      <listitem>
        <para>
        You must also implement the <methodname>configure()</methodname>
        method and check the parameters. If no filename has been given,
        you should check if immediate exection is allowed and set an
        error if it is not. If a filename is present, use the 
        <methodname>pathCanBeUsed()</methodname> method to check if 
        it is possible to save the data to a file with that name. If the
        file already exists it can be overwritten if the <varname>OVERWRITE</varname>
        is <constant>TRUE</constant> or if the file has been flagged for removal.
        Do not forget to store the parameters with the <methodname>storeValue()</methodname>
        method.

        <programlisting language="java">
// Selected parts from the configure() method
if (request.getParameterValue(SAVE_AS) == null)
{
   if (!request.isAllowedImmediateExecution())
   {
      response.setError("Immediate download is not allowed. " + 
         "Please specify a filename.", null);
      return;
   }
   Client client = (Client)request.getParameterValue("client");
   response.setDownloadImmediately("Export help texts for client application " + 
     client.getName(), ExecutionTime.SHORTEST, true);
}
else
{
   if (!pathCanBeUsed((String)request.getParameterValue(SAVE_AS), 
      (Boolean)request.getParameterValue(OVERWRITE)))
   {
      response.setError("File exists: " + 
         (String)request.getParameterValue(SAVE_AS), null);
      return;
   }
   storeValue(job, request, ri.getParameter(SAVE_AS));
   storeValue(job, request, ri.getParameter(OVERWRITE));
   response.setDone("The job configuration is complete", ExecutionTime.SHORTEST);
}
</programlisting>
        
        </para>
      </listitem>
      
      <listitem>
        <para>
        Implement the <methodname>performExport()</methodname> method.
        This is defined as abstract in the <classname docapi="net.sf.basedb.core.plugin">AbstractExporterPlugin</classname>
        class. It has the same parameters as the <methodname>ImmediateDownloadExporter.doExport()</methodname>
        method and they have the same meaning. The only difference is that the 
        <varname>out</varname> stream can be linked to a file in the BASE filesystem
        and not just to the HTTP response stream.
        </para>
      </listitem>
      
      <listitem>
        <para>
        Optionally, implement the <methodname>begin()</methodname>,
        <methodname>end()</methodname> and <methodname>getSuccessMessage()</methodname>
        methods. Theese methods do nothing by default. 
        </para>
      </listitem>
      </itemizedlist>
      
      <para>
        The call sequence for plug-ins extending <classname docapi="net.sf.basedb.core.plugin">AbstractExporterPlugin</classname>
        is:
      </para>
      
      <orderedlist>
      <listitem>
        <para>
        Call <methodname>begin()</methodname>.
        </para>
      </listitem>
      <listitem>
        <para>
        Call <methodname>performExport()</methodname>.
        </para>
      </listitem>
      <listitem>
        <para>
        Call <methodname>end()</methodname>.
        </para>
      </listitem>
      <listitem>
        <para>
        Call <methodname>getSuccessMessage()</methodname> if running as a regular
        job. This method is never called when doing an immediate download since there
        is no place to show the message.
        </para>
      </listitem>
      </orderedlist>
      
    </sect2>
    
  </sect1>
  
  <sect1 id="plugin_developer.analyse">
    <title>Analysis plug-ins</title>
    <para>
      A plug-in becomes an analysis plug-in simply by returning
      <constant>Plugin.MainType.ANALYZE</constant> from the
      <methodname>Plugin.getMainType()</methodname> method. The information returned from
      <methodname>InteractivePlugin.getGuiContexts()</methodname>
      must include: [<constant>Item.BIOASSAYSET</constant>, <constant>Type.ITEM</constant>] 
      since this is the main place where the web client looks for analysis plug-ins. If
      the plug-in can work on a subset of the bioassays it may also include
      [<constant>Item.BIOASSAY</constant>, <constant>Type.LIST</constant>]
      among the contexts. This will make it possible for a user to select
      bioassays from the list and then invoke the plug-in.
    </para>
    
<programlisting language="java">
private static final Set&lt;GuiContext&gt; guiContexts = 
   Collections.singleton(new GuiContext(Item.BIOASSAYSET, GuiContext.Type.ITEM));

public Set&lt;GuiContext&gt; getGuiContexts()
{
   return guiContexts;
}</programlisting>

    <para>
    If the plugin depends on a specific raw data type or on the number of
    channels, it should check that the current bioassayset is of the
    correct type in the <methodname>InteractivePlugin.isInContext()</methodname> 
    method. It is also a good idea to check if the current user has permission
    to use the current experiment. This permission is needed to create new bioassaysets or
    other data belonging to the experiment.
    </para>
  
    <programlisting language="java">
public boolean isInContext(GuiContext context, Object item)
{
   if (item == null)
   {
      message = "The object is null";
   }
   else if (!(item instanceof BioAssaySet))
   {
      message = "The object is not a BioAssaySet: " + item;
   }
   else
   {
      BioAssaySet bas = (BioAssaySet)item;
      int channels = bas.getRawDataType().getChannels();
      if (channels != 2)
      {
         message = "This plug-in requires 2-channel data, not " + channels + "-channel.";
      }
      else
      {
         Experiment e = bas.getExperiment();
         e.checkPermission(Permission.USE);
      }
   }
}
</programlisting>

    <para>
    The plugin should always include a parameter asking for the current 
    bioassay set when the <methodname>InteractivePlugin.getRequestInformation()</methodname>
    is called with <literal>command = Request.COMMAND_CONFIGURE_JOB</literal>.
    </para> 

    <programlisting language="java">
private static final RequestInformation configurePlugin;
private RequestInformation configureJob;
private PluginParameter&lt;BioAssaySet&gt; bioAssaySetParameter;

public RequestInformation getRequestInformation(GuiContext context, String command) 
   throws BaseException
{
   RequestInformation requestInformation = null;
   if (command.equals(Request.COMMAND_CONFIGURE_PLUGIN))
   {
      requestInformation = getConfigurePlugin(context);
   }
   else if (command.equals(Request.COMMAND_CONFIGURE_JOB))
   {
      requestInformation = getConfigureJob(context);
   }
   return requestInformation;
}

private RequestInformation getConfigureJob(GuiContext context)
{
   if (configureJob == null)
   {
      bioAssaySetParameter; = new PluginParameter&lt;BioAssaySet&gt;(
         "bioAssaySet",
         "Bioassay set",
         "The bioassay set used as the source for this analysis plugin",
         new ItemParameterType&lt;BioAssaySet&gt;(BioAssaySet.class, null, true, 1, null)
      );

      List&lt;PluginParameter&lt;?&gt;&gt; parameters = new ArrayList&lt;PluginParameter&lt;?&gt;&gt;();
      parameters.add(bioAssaySetParameter);
      // Add more plug-in-specific parameters here...
    
      configureJob = new RequestInformation(
         Request.COMMAND_CONFIGURE_JOB,
         "Configure job",
         "Set parameter for plug-in execution",
         parameters
      );
   }
   return configureJob;
}
</programlisting>

    <para>
    Of course, the <methodname>InteractivePlugin.configure()</methodname> method needs 
    to validate and store the bioassay set parameter as well:
    </para>
  
    <programlisting language="java">
public void configure(GuiContext context, Request request, Response response)
{
   String command = request.getCommand();
   try
   {
      if (command.equals(Request.COMMAND_CONFIGURE_PLUGIN))
      {
         // Validate and store configuration parameters
         response.setDone("Plugin configuration complete");
      }
      else if (command.equals(Request.COMMAND_CONFIGURE_JOB))
      {
         List&lt;Throwable&gt; errors = 
            validateRequestParameters(configureJob.getParameters(), request);
         if (errors != null)
         {
            response.setError(errors.size() +
               " invalid parameter(s) were found in the request", errors);
            return;
         }
         storeValue(job, request, bioAssaySetParameter);
         // Store other plugin-specific parameters
      
         response.setDone("Job configuration complete", Job.ExecutionTime.SHORT);
      }
   }
   catch (Throwable ex)
   {
      // Never throw exception, always set response!
      response.setError(ex.getMessage(), Arrays.asList(ex));
   }
}
</programlisting>

    <para>
    Now, the typical <methodname>Plugin.run()</methodname> method loads the specfied bioassay set
    and its spot data. It may do some filtering and recalculation of the spot
    intensity value(s). In most cases it will store the result as a child bioassay
    set with one bioassay for each bioassay in the parent bioassay set.
    Here is an example, which just copies the intensity values, while
    removing those with a negative value in either channel.
    </para>
  
    <programlisting language="java">
public void run(Request request, Response response, ProgressReporter progress)
{
   DbControl dc = sc.newDbControl();
   try
   {
      BioAssaySet source = (BioAssaySet)job.getParameter("bioAssaySet");
      // Reload with current DbControl
      source = BioAssaySet.getById(dc, source.getId());
      int channels = source.getRawDataType().getChannels();
      
      // Create transformation and new bioassay set
      Job j = Job.getById(dc, job.getId());
      Transformation t = source.newTransformation(j);
      t.setName("Copy spot intensities &gt;= 0");
      dc.saveItem(t);

      BioAssaySet result = t.newProduct(null, "new", true);
      result.setName("After: Copying spot intensities");
      dc.saveItem(result);

      // Get query for source data
      DynamicSpotQuery query = source.getSpotData();
      
      // Do not return spots with intensities &lt; 0
      for (int ch = 1; ch &lt;= channels; ++ch)
      {
         query.restrict(
            Restrictions.gteq(
               Dynamic.column(VirtualColumn.channel(ch)),
               Expressions.integer(0)
            )
         );
      }
      
      // Create batcher and copy data
      SpotBatcher batcher = result.getSpotBatcher();
      int spotsCopied = batcher.insert(query);
      batcher.close();
      
      // Commit and return
      dc.commit();
      response.setDone("Copied " + spotsCopied + " spots.");
   }
   catch (Throwable t)
   {
      response.setError(t.getMessage(), Arrays.asList(t));
   }
   finally
   {
      if (dc != null) dc.close();
   }
}
</programlisting>

    <para>
    See <xref linkend="api_overview.dynamic_and_batch_api" />
    for more examples of using the analysis API.
    </para>

    <sect2 id="plugin_developer.analyse.abstractanalysis">
      <title>The AbstractAnalysisPlugin class</title>
      
      <para>
        This class is an abstract base class. It is a useful
        class for most analysis plug-ins to inherit from. Its main 
        purpose is to define <classname docapi="net.sf.basedb.core">PluginParameter</classname>
        objects that are commonly used in analysis plug-ins. This includes:
      </para>
      
      <itemizedlist>
      <listitem>
        <para>
        The source bioassay set: 
          <methodname>getSourceBioAssaySetParameter()</methodname>,
          <methodname>getCurrentBioAssaySet()</methodname>,
          <methodname>getSourceBioAssaySet()</methodname>
        </para>
      </listitem>
      <listitem>
        <para>
        The optional restriction of which bioassays to use. 
        All bioassays in a bioassay set will be used if this 
        parameter is empty. This is useful when the plugin only 
        should run on a subset of bioassays in a bioassay set: 
          <methodname>getSourceBioAssaysParameter()</methodname>,
          <methodname>getSourceBioAssays()</methodname>
        </para>
      </listitem>
      <listitem>
        <para>
        The name and description of the child bioassay set that
        is going to be created by the plug-in:
        <methodname>getChildNameParameter()</methodname>,
        <methodname>getChildDescriptionParameter()</methodname>
        </para>
      </listitem>
      <listitem>
        <para>
        The name and description of the transformation that
        represents the execution of the plug-in:
        <methodname>getTransformationNameParameter()</methodname>,
        <methodname>getTransformationName()</methodname>
        </para>
      </listitem>
      </itemizedlist>
      
    </sect2>
    
    <sect2 id="plugin_developer.analyse.filterplugin">
      <title>The AnalysisFilterPlugin interface</title>
      
      <para>
        The <interfacename docapi="net.sf.basedb.core.plugin">net.sf.basedb.core.plugin.AnalysisFilterPlugin</interfacename> 
        is a tagging interface, with no methods, that all analysis plug-ins that only filters
        data should implement. The benefit is that they will be linked from the
        <guibutton>Filter bioassay set</guibutton> button and not just
        the <guibutton>Run analysis</guibutton> button. They will also get
        a different icon in the experiment outline to make filtering 
        transformations appear different from other transformations.
      </para>
      
      <para>
        The interface exists purely for making the user interaction better. There is
        no harm in not implementing it since the plug-in will always appear in
        from the <guibutton>Run analysis</guibutton> button. On the other hand,
        it doesn't cost anything to implement the interface since it doesn't
        have any methods.
      </para>
    
    </sect2>

  </sect1>
  
  <sect1 id="plugin_developer.other">
    <title>Other plug-ins</title>
    <para></para>
    
    <sect2 id="plugin_developer.other.authentication">
      <title>Authentication plug-ins</title>
      
      <para>
        BASE provides a plug-in mechanism for authenticating users 
        (validating the username and password) when they are logging in.
        This plug-in mechanism is not the same as the regular plug-in API.
        That is, you do not have worry about user interaction or implementing the 
        <interfacename docapi="net.sf.basedb.core.plugin">Plugin</interfacename> interface.
      </para>
      
      <sect3 id="plugin_developer.other.authentication.internal_external">
        <title>Internal vs. external authentation</title>
      
        <para>
          BASE can authenticate users in two ways. Either it uses the internal 
          authentication or the external authentication. With internal 
          authentication BASE stores logins and passwords in its own database. 
          With external authentication this is handled by some external 
          application. Even with external authentication it is possible to 
          let BASE cache the logins/passwords. This makes it possible to login to 
          BASE if the external authentication server is down.
        </para>
        
        <note>
          <para>
          An external authentication server can only be used to grant or deny 
          a user access to BASE. It cannot be used to give a user permissions,
          or put a user into groups or different roles inside BASE.
          </para>
        </note>

        <para>
          The external authentication service is only used when a user logs in. 
          Now, one or more of several things can happen:
          
          <itemizedlist>
          <listitem>
            <para>
              The ROOT user is logging on. Internal authentication is always 
              used for the root user and the authenticator plug-in is never 
              used.
            </para>
          </listitem>

          <listitem>
            <para>
              The login is correct and the user is already known to BASE. 
              If the plug-in supports extra information (name, email, phone, etc.)
              and the <property>auth.synchronize</property> setting
              is <constant>TRUE</constant> the extra information is copied to 
              the BASE server.
            </para>
          </listitem>
          
          <listitem>
            <para>
              The login is correct, but the user is not known to BASE. This happens
              the first time a user logs in. BASE will create
              a new user account. If the driver supports extra information, it 
              is copied to the BASE server (even if <property>auth.synchronize</property> 
              is not set). The new user account will get the default quota 
              and be added to the all roles and groups which has been
              marked as <emphasis>default</emphasis>.
              
              <note>
              <para>
                Prior to BASE 2.4 it was hardcoded to add the new user to the
                <emphasis>Users</emphasis> role only.
              </para>
              </note>
            </para>
          </listitem>

          <listitem>
            <para>
              If password caching is enabled, the password is copied to BASE. 
              If an expiration timeout has been set, an expiration date
              will be calculated and set on the user account. The expiration
              date is only checked when the external authentication server is 
              down.
            </para>
          </listitem>

          <listitem>
            <para>
              The authentication server says that the login is invalid or 
              the password is incorrect. The user will not be logged in. 
              If a user account with the specified login already exists in 
              BASE, it will be disabled.
            </para>
          </listitem>
          
          <listitem>
            <para>
              The authentication driver says that something else is wrong. 
              If password caching is enabled, internal authentication will be 
              used. Otherwise the user will not be logged in. An already 
              existing account is not modified or disabled.
            </para>
          </listitem>
          </itemizedlist>
          
          <note>
            <para>
            The <guilabel>Encrypt password</guilabel> option that is
            available on the login page does not work with external 
            authentication. The simple reason is that the password is
            encrypted with a one-way algorithm making it impossible to
            call <methodname>Authenticator.authenticate()</methodname>.
            </para>
          </note>
          
        </para>
      </sect3>
      
      <sect3 id="plugin_developer.other.authentication.authenticator">
        <title>The Authenticator interface</title>
        
        <para>
          To be able to use external authentication you must create a class 
          that implements the 
          <interfacename docapi="net.sf.basedb.core.authentication">net.sf.based.core.authentication.Authenticator</interfacename> 
          interface. Specify the name of the class in the <property>auth.driver</property> 
          setting in <filename>base.config</filename> and 
          its initialisation parameters in the <property>auth.init</property> setting.
        </para>
        
        <para>
          Your class must have a public no-argument constructor. The BASE 
          application will create only one instance of the class for lifetime 
          of the BASE server. It must be thread-safe since it may be invoked by 
          multiple threads at the same time. Here are the methods that you must 
          implement
        </para>
        
        <variablelist>
        <varlistentry>
          <term>
            <methodsynopsis language="java">
              <modifier>public</modifier>
              <void/>
              <methodname>init</methodname>
              <methodparam>
                <type>String</type>
                <parameter>settings</parameter>
              </methodparam>
              <exceptionname>AuthenticationException</exceptionname>
            </methodsynopsis>
          </term>
          <listitem>
            <para>
            This method is called just after the object has been created with its argument 
            taken from the <property>auth.init</property> setting in your <filename>base.config</filename> 
            file. This method is only called once for an instance of the object. The syntax and meaning of 
            the parameter is driver-dependent and should be documented by the plug-in. 
            It is irrelevant for the BASE core.
            </para>
          </listitem>
        </varlistentry>
        
        <varlistentry>
          <term>
            <methodsynopsis language="java">
              <modifier>public</modifier>
              <type>boolean</type>
              <methodname>supportsExtraInformation</methodname>
            </methodsynopsis>
          </term>
          <listitem>
            <para>
            This method should simply return <constant>TRUE</constant> or <constant>FALSE</constant> 
            depending on if the plug-in supports extra user information or not. The only required 
            information about a user is a unique ID and the login. Extra information includes 
            name, address, phone, email, etc.
            </para>
          </listitem>       
        </varlistentry>

        <varlistentry>
          <term>
            <methodsynopsis language="java">
              <modifier>public</modifier>
              <type>AuthenticationInformation</type>
              <methodname>authenticate</methodname>
              <methodparam>
                <type>String</type>
                <parameter>login</parameter>
              </methodparam>
              <methodparam>
                <type>String</type>
                <parameter>password</parameter>
              </methodparam>
              <exceptionname>UnknownLoginException</exceptionname>
              <exceptionname>InvalidPasswordException</exceptionname>
              <exceptionname>AuthenticationException</exceptionname>
            </methodsynopsis>
          </term>
          <listitem>
            <para>
            Try to authenticate a login/password combination. The plug-in should return 
            an <classname docapi="net.sf.basedb.core.authentication">AuthenticationInformation</classname> object if the 
            authentication is successful or throw an exception if not. 
            
            There are three exceptions to choose from:
            
            <itemizedlist>
            <listitem>
              <para>
              <exceptionname>UnknownLoginException</exceptionname>:
              This exception should be thrown if the login is not known to the 
              external authentication system.
              </para>
            </listitem>

            <listitem>
              <para>
              <exceptionname>InvalidPasswordException</exceptionname>:
              This exception should be thrown if the login is known but the 
              password is invalid. In case it is considered a security issue 
              to reveal that a login exists, the plugin may throw an 
              <exceptionname>UnknowLoginException</exceptionname> instead.
              </para>
            </listitem>
            
            <listitem>
              <para>
              <exceptionname>AuthenticationException</exceptionname>:
              In case there is another problem, such as the authentication service 
              being down. This exception triggers the use of cached passwords
              if caching has been enabled.
              </para>
            </listitem>
            </itemizedlist>
            </para>
          </listitem>       
        </varlistentry>
        </variablelist> 
      </sect3>
      
      <sect3 id="plugin_developer.other.authentication.settings">
        <title>Configuration settings</title>
      
        <para>
          The configuration settings for the authentication driver are located 
          in the <filename>base.config</filename> file. 
          Here is an overview of the settings. For more information read 
          <xref linkend="appendix.base.config.authentication" />.
        </para>
        
        <variablelist>
        <varlistentry>
          <term><property>auth.driver</property></term>
          <listitem>
            <para>
            The class name of the authentication plug-in.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term><property>auth.init</property></term>
          <listitem>
            <para>
            Initialisation parameters sent to the plug-in when calling the 
            <methodname>Authenticator.init()</methodname> method. 
            </para>
          </listitem>
        </varlistentry>
        
        <varlistentry>
          <term><property>auth.synchronize</property></term>
          <listitem>
            <para>
            If extra user information is synchronized at login time or not. 
            This setting is ignored if the driver does not support extra information. 
            </para>
          </listitem>
        </varlistentry>
        
        <varlistentry>
          <term><property>auth.cachepasswords</property></term>
          <listitem>
            <para>
              If passwords should be cached by BASE or not. If the passwords are 
              cached a user may login to BASE even if the external authentication 
              server is down.
            </para>
          </listitem>
        </varlistentry>
        
        <varlistentry>
          <term><property>auth.daystocache</property></term>
          <listitem>
            <para>
              How many days to cache the passwords if caching has been enabled. 
              A value of 0 caches the passwords for ever.     
            </para>
          </listitem>
        </varlistentry>
        </variablelist>
      </sect3>
      
    </sect2>
    
    <sect2 id="plugin_developer.other.secondary">
      <title>Secondary file storage plugins</title>
      
      <sect3 id="plugin_developer.other.secondary.vsprimary">
        <title>Primary vs. secondary storage</title>
        <para>
          BASE has support for storing files in two locations, the primary storage and 
          the secondary storage. The primary storage is always disk-based and must be 
          accessible by the BASE server as a path on the file system. The path to the 
          primary storage is configured by the <varname>userfiles</varname> setting in the 
          <filename>base.config</filename> file. The primary storage is internal to 
          the core. Client applications don't get access to read or manipulate the 
          files directly from the file system.
        </para>
        
        <para>
          The secondary storage can be anything that can store files. It could, for 
          example, be another directory, a remote FTP server, or a tape based archiving
          system. A file located in the secondary storage is not accessible by the 
          core, client applications or plug-ins. The secondary storage can only be accessed 
          by the secondary storage controller. The core (and client) applications uses 
          flags on the file items to handle the interaction with the secondary storage.
        </para>
        
        <para>
          Each file has an <property>action</property> attribute which default's to 
          <constant>File.Action.NOTHING</constant>. It can take two other values:
        </para>
        
        <orderedlist>
        <listitem>
          <para>
          <constant>File.Action.MOVE_TO_SECONDARY</constant>
          </para>
        </listitem>
        <listitem>
          <para>
          <constant>File.Action.MOVE_TO_PRIMARY</constant>
          </para>
        </listitem>
        </orderedlist>
        
        <para>
          All files with the action attribute set to <constant>MOVE_TO_SECONDARY</constant> 
          should be moved to the secondary storage by the controller, and all files 
          with the action attribute set to <constant>MOVE_TO_PRIMARY</constant> should be 
          brought back to primary storage.
        </para>
        
        <para>
          The moving of files between primary and secondary storage doesn't happen 
          immediately. It is up to the server administrator to configure how often and 
          at what times the controller should check for files that should be moved. 
          This is configured by the <varname>secondary.storage.interval</varname> 
          and <varname>secondary.storage.time</varname> settings in the 
          <filename>base.config</filename> file.        
        </para>
      </sect3>
      
      <sect3 id="plugin_developer.other.secondary.interface">
        <title>The SecondaryStorageController interface</title>
        
        <para>
          All you have to do to create a secondary storage controller is to 
          create a class that implements the 
          <interfacename docapi="net.sf.basedb.core">net.sf.basedb.core.SecondaryStorageController</interfacename>
          interface. In your <filename>base.config</filename> file you then specify the
          class name in the <varname>secondary.storage.driver</varname> setting and its 
          initialisation parameters in the <varname>secondary.storage.init</varname> setting.
        </para>

        <para>
          Your class must have a public no-argument constructor. 
          The BASE application will create only one instance of the class for 
          lifetime of the BASE server. Here are the methods that you must implement: 
        </para>
        
        <variablelist>
        <varlistentry>
          <term>
          <methodsynopsis language="java">
            <modifier>public</modifier>
            <void />
            <methodname>init</methodname>
            <methodparam>
              <type>String</type>
              <parameter>settings</parameter>
            </methodparam>
          </methodsynopsis>
          </term>
          <listitem>
            <para>
            This method is called just after the object has been created with its argument 
            taken from the <varname>secondary.storage.init</varname> setting in your 
            <filename>base.config</filename> file. This method is only called once for 
            an object.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
          <methodsynopsis language="java">
            <modifier>public</modifier>
            <void />
            <methodname>run</methodname>
          </methodsynopsis>
          </term>
          <listitem>
            <para>
            This method is called whenever the core thinks it is time to do some 
            management of the secondary storage. How often the <methodname>run()</methodname>
            method is called is controlled by the <varname>secondary.storage.interval</varname>
            and <varname>secondary.storage.time</varname> settings in the 
            <filename>base.config</filename> file.
            When this method is called the controller should:
            </para>
            
            <itemizedlist>
            <listitem>
              <para>
              Move all files which has <constant>action=MOVE_TO_SECONDARY</constant> to 
              the secondary storage. When the file has been moved call 
              <methodname>File.setLocation(Location.SECONDARY)</methodname> to tell the 
              core that the file is now in the secondary storage. You should also call 
              <methodname>File.setAction(File.Action.NOTHING)</methodname> to reset the 
              action attribute.
              </para>
            </listitem>
            
            <listitem>
              <para>
              Restore all files which has <constant>action=MOVE_TO_PRIMARY</constant>. 
              The core will set the location attribute automatically, but you should 
              call <methodname>File.setAction(File.Action.NOTHING)</methodname> to reset 
              the action attribute.
              </para>
            </listitem>
            
            <listitem>
              <para>
              Delete all files from the secondary storage that are not present 
              in the database with <constant>location=Location.SECONDARY</constant>. 
              This includes files which has been deleted and files that have been 
              moved offline or re-uploaded.
              </para>
            </listitem>
            
            </itemizedlist>
            
            <para>
              As a final act the method should send a message to each user owning 
              files that has been moved from one location to the other. The message 
              should include a list of files that has been moved to the secondary 
              storage and a list of files moved from the secondary storage and a 
              list of files that has been deleted due to some of the reasons above. 
            </para>
          </listitem>
        </varlistentry>       
        
        <varlistentry>
          <term>
          <methodsynopsis language="java">
            <modifier>public</modifier>
            <void />
            <methodname>close()</methodname>
          </methodsynopsis>
          </term>
          <listitem>
            <para>
            This method is called when the server is closing down. After this the object 
            is never used again.
            </para>
          </listitem>
        </varlistentry>
        </variablelist>
      </sect3>
      
      <sect3 id="plugin_developer.other.secondary.settings">
        <title>Configuration settings</title>
        
        <para>
        The configuration settings for the secondary storage controller is located in the 
        <filename>base.config</filename> file. Here is an overview of the settings. 
        For more information read <xref linkend="appendix.base.config" />.
        </para>
        
        <variablelist>
        <varlistentry>
          <term><property>secondary.storage.driver</property></term>
          <listitem>
            <para>
            The class name of the secondary storage plug-in.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term><property>secondary.storage.init</property></term>
          <listitem>
            <para>
             Initialisation parameters sent to the plug-in by calling the 
             <methodname>init()</methodname> method.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term><property>secondary.storage.interval</property></term>
          <listitem>
            <para>
             Interval in seconds between each execution of the secondary storage 
             controller plug-in.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term><property>secondary.storage.time</property></term>
          <listitem>
            <para>
              Time points during the day when the secondary storage controller plugin 
              should be executed.
            </para>
          </listitem>
        </varlistentry>
        </variablelist>
      </sect3>
    </sect2>
    
    <sect2 id="plugin_developer.other.unpacker">
      <title>File unpacker plug-ins</title>
      <para>
        The BASE web client has integrated support for unpacking of 
        compressed files. See <xref linkend="file_system.handling.upload" />.
        Behind the scenes, this support is provided by plug-ins. The standard 
        BASE distribution comes with support for ZIP files 
        (<classname docapi="net.sf.basedb.plugins">net.sf.basedb.plugins.ZipFileUnpacker</classname>)
        and TAR files (<classname docapi="net.sf.basedb.plugins">net.sf.basedb.plugins.TarFileUnpacker</classname>).
      </para>
      <para>
        To add support for additional compressed formats you have to create a plug-in that
        implements the <interfacename docapi="net.sf.basedb.util.zip">net.sf.basedb.util.zip.FileUnpacker</interfacename>
        interface. The best way to do this is to extend the 
        <classname docapi="net.sf.basedb.util.zip">net.sf.basedb.util.zip.AbstractFileUnpacker</classname> which 
        implements all methods in the <interfacename docapi="net.sf.basedb.core.plugin">Plugin</interfacename>
        and <interfacename docapi="net.sf.basedb.core.plugin">InteractivePlugin</interfacename>
        interfaces except <methodname>Plugin.getAbout()</methodname>. This leaves
        you with the actual unpacking of the files as the only thing to implement.
      </para>
      
      <note>
        <title>No support for configurations</title>
        The integrated upload in the web interface only works with plug-ins that
        does not require a configuration to run.
      </note>
      
      <variablelist>
        <title>Methods in the <interfacename docapi="net.sf.basedb.util.zip">FileUnpacker</interfacename> interface</title>
        <varlistentry>
          <term>
            <methodsynopsis language="java">
              <modifier>public</modifier>
              <type>String</type>
              <methodname>getFormatName</methodname>
            </methodsynopsis>
          </term>
          <listitem>
            <para>
            Return a short string naming the file format. For example:
            <constant>ZIP files</constant> or <constant>TAR files</constant>.
            </para>
          </listitem>
        </varlistentry>
            
        <varlistentry>
          <term>
            <methodsynopsis language="java">
              <modifier>public</modifier>
              <type>Set&lt;String&gt;</type>
              <methodname>getExtensions</methodname>
            </methodsynopsis>
          </term>
          <listitem>
            <para>
            Return a set of strings with the file extensions that
            are most commonly used with the compressed file format.
            For example: <constant>[zip, jar]</constant>. Do not include
            the dot in the extensions. The web client and the 
            <methodname>AbstractFlatFileUnpacker.isInContext()</methodname> method
            will use this information to automatically guess which plug-in to 
            use for unpacking the files.
            </para>
          </listitem>
        </varlistentry>
        
        <varlistentry>
          <term>
            <methodsynopsis language="java">
              <modifier>public</modifier>
              <type>Set&lt;String&gt;</type>
              <methodname>getMimeTypes</methodname>
            </methodsynopsis>
          </term>
          <listitem>
            <para>
            Return a set of string with the MIME types that commonly used with 
            the compressed file format. For example: 
            <constant>[application/zip, application/java-archive]</constant>. 
            This information is used by the 
            <methodname>AbstractFlatFileUnpacker.isInContext()</methodname> 
            method to automatically guess which plug-in to use for unpacking 
            the files.
            </para>
          </listitem>
        </varlistentry>
        
        <varlistentry>
          <term>
            <methodsynopsis language="java">
              <modifier>public</modifier>
              <type>int</type>
              <methodname>unpack</methodname>
              <methodparam>
                <type>DbControl</type>
                <parameter>dc</parameter>
              </methodparam>
              <methodparam>
                <type>Directory</type>
                <parameter>dir</parameter>
              </methodparam>
              <methodparam>
                <type>InputStream</type>
                <parameter>in</parameter>
              </methodparam>
              <methodparam>
                <type>boolean</type>
                <parameter>overwrite</parameter>
              </methodparam>
              <methodparam>
                <type>AbsoluteProgressReporter</type>
                <parameter>progress</parameter>
              </methodparam>
              <exceptionname>IOException</exceptionname>
              <exceptionname>BaseException</exceptionname>
            </methodsynopsis>
          </term>
          <listitem>
            <para>
            Unpack the files and store them in the BASE file system. 
            
            <itemizedlist>
            <listitem>
              <para>
              Do not <methodname>close()</methodname> or 
              <methodname>commit()</methodname> the 
              <classname docapi="net.sf.basedb.core">DbControl</classname> passed to this method.
              This is done automatically by the 
              <classname docapi="net.sf.basedb.util.zip">AbstractFileUnpacker</classname> or by the web client.
              </para>
            </listitem>
            
            <listitem>
              <para>
              The <varname>dir</varname> parameter is the root directory where
              the unpacked files should be placed. If the compressed file
              contains subdirectories the plug-in must create those subdirectories
              unless they already exists.
              </para>
            </listitem>
            
            <listitem>
              <para>
              If the <varname>overwrite</varname> parameter is 
              <constant>FALSE</constant> no existing file should be overwritten
              unless the file is <systemitem>OFFLINE</systemitem>.
              </para>
            </listitem>
            
            <listitem>
              <para>
              The <varname>in</varname> parameter is the stream
              containing the compressed data. The stream may come
              directly from the web upload or from an existing
              file in the BASE file system.
              </para>
            </listitem>
            
            <listitem>
              <para>
              The <varname>progress</varname> parameter, if not 
              <constant>null</constant>, should be used to report the
              progress back to the calling code. The plug-in should count
              the number of bytes read from the <varname>in</varname>
              stream. If it is not possible by other means the stream can
              be wrapped by a <classname docapi="net.sf.basedb.util">net.sf.basedb.util.InputStreamTracker</classname>
              object which has a <methodname>getNumRead()</methodname> method.
              </para>
            </listitem>
            </itemizedlist>
            
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
      
      <para>
        When the compressed file is uncompressed during the file upload
        from the web interface, the call sequence to the plug-in is slightly
        altered from the standard call sequence described in 
        <xref linkend="plugin_developer.api.callsequence.execute" />.
        
        <itemizedlist>
        <listitem>
          <para>
          After the plug-in instance has been created, the 
          <methodname>Plugin.init()</methodname> method is called with <constant>null</constant>
          values for both the <varname>configuration</varname> and <varname>job</varname>
          parameters.
          </para>
        </listitem>
        
        <listitem>
          <para>
          Then, the <methodname>unpack()</methodname> method is called. The
          <methodname>Plugin.run()</methodname> method is never called in this case.
          </para>
        </listitem>
        
        </itemizedlist>
        
      </para>
      
    </sect2>
    
    <sect2 id="plugin_developer.other.packer">
      <title>File packer plug-ins</title>
    
      <para>
        BASE has support for compressing and downloading a set of selected files and/or
        directories. This functionality is provided by a plug-in, the 
        <classname docapi="net.sf.basedb.plugins">PackedFileExporter</classname>. This plug-in doesn't do the actual
        packing itself. This is delegated to classes implementing the 
        <interfacename docapi="net.sf.basedb.util.zip">net.sf.basedb.util.zip.FilePacker</interfacename> interface.
      </para>
      
      <para>
        BASE ships with a number of packing methods, including ZIP and TAR. To
        add support for other methods you have to provide an implementation
        of the <interfacename docapi="net.sf.basedb.util.zip">FilePacker</interfacename>
        interface. Then, create a new configuration for the <classname docapi="net.sf.basedb.plugins">PackedFileExporter</classname>
        and enter the name of your class in the configuration wizard.
      </para>
      
      <para>
        The <interfacename docapi="net.sf.basedb.util.zip">FilePacker</interfacename> interface is not a regular
        plug-in interface (ie. it is not a subinterface to 
        <interfacename docapi="net.sf.basedb.core.plugin">Plugin</interfacename>). This means that you don't have to
        mess with configuration or job parameters. Another difference is that your
        class must be installed in Tomcat's classpath (ie. in one of the 
        <filename>WEB-INF/classes</filename> or <filename>WEB-INF/lib</filename>
        folders).
      </para>
      
      <variablelist>
        <title>Methods in the <interfacename docapi="net.sf.basedb.util.zip">FilePacker</interfacename> interface</title>
        <varlistentry>
          <term>
            <methodsynopsis language="java">
              <modifier>public</modifier>
              <type>String</type>
              <methodname>getDescription</methodname>
            </methodsynopsis>
          </term>
          <listitem>
            <para>
            Return a short description the file format that is suitable for use
            in dropdown lists in client applications. For example:
            <constant>Zip-archive (.zip)</constant> or <constant>TAR-archive (.tar)</constant>.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <methodsynopsis language="java">
              <modifier>public</modifier>
              <type>String</type>
              <methodname>getFileExtension</methodname>
            </methodsynopsis>
          </term>
          <listitem>
            <para>
            Return the default file extension of the packed format. The returned
            value should not include the dot. For example:
            <constant>zip</constant> or <constant>tar</constant>.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <methodsynopsis language="java">
              <modifier>public</modifier>
              <type>String</type>
              <methodname>getMimeType</methodname>
            </methodsynopsis>
          </term>
          <listitem>
            <para>
            Return the standard MIME type of the packed file format.
            For example: 
            <constant>application/zip</constant> or <constant>application/x-tar</constant>.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <methodsynopsis language="java">
              <modifier>public</modifier>
              <void />
              <methodname>setOutputStream</methodname>
              <methodparam>
                <type>OutputStream</type>
                <parameter>out</parameter>
              </methodparam>
              <exceptionname>IOException</exceptionname>
            </methodsynopsis>
          </term>
          <listitem>
            <para>
            Sets the outputstream that the packer should write the packed
            files to.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <methodsynopsis language="java">
              <modifier>public</modifier>
              <void />
              <methodname>pack</methodname>
              <methodparam>
                <type>String</type>
                <parameter>entryName</parameter>
              </methodparam>
              <methodparam>
                <type>InputStream</type>
                <parameter>in</parameter>
              </methodparam>
              <methodparam>
                <type>long</type>
                <parameter>size</parameter>
              </methodparam>
              <methodparam>
                <type>long</type>
                <parameter>lastModified</parameter>
              </methodparam>
              <exceptionname>IOException</exceptionname>
            </methodsynopsis>
          </term>
          <listitem>
            <para>
            Add another file or directory to the packed file. The 
            <parameter>entryName</parameter> is the name of the new entry, including
            path information. The <parameter>in</parameter> is the stream to read
            the file data from. If <parameter>in</parameter> is <constant>null</constant>
            then the entry denotes a directory. The <parameter>size</parameter> parameter
            gives the size in bytes of the file (zero for empty files or directories). 
            The <parameter>lastModified</parameter>
            is that time the file was last modified or 0 if not known.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <methodsynopsis language="java">
              <modifier>public</modifier>
              <void />
              <methodname>close</methodname>
              <exceptionname>IOException</exceptionname>
            </methodsynopsis>
          </term>
          <listitem>
            <para>
            Finish the packing. The packer should release any resources, flush
            all data and close all output streams, including the <varname>out</varname> stream
            set in the <methodname>setOutputStream</methodname> method.
            </para>
          </listitem>
        </varlistentry>
        
      </variablelist>
      
    </sect2>
    
    <sect2 id="plugin_developer.other.datafiles">
      <title>File validator and metadata reader plug-ins</title>
    
      <itemizedlist>
        <title>See also</title>
        <listitem><xref linkend="core_api.data_in_files" /></listitem>
        <listitem><xref linkend="data_api.platforms" /></listitem>
      </itemizedlist>
    
    
      <para>
        In those cases where files are used to store data instead
        of importing it to the database, BASE can use plug-ins to 
        check that the supplied files are valid and also to extract
        metadata from the files. For example, the 
        <classname docapi="net.sf.basedb.core.filehandler">net.sf.basedb.core.filehandler.CelFileHandler</classname>
        is used to check if a file is a valid Affymetrix CEL file and
        to extract data headers and the number of spots from it.
      </para>
      
      <para>
        The validator and metadata reader plug-ins are not regular plug-ins
        (ie. they don't have to implement the <interfacename docapi="net.sf.basedb.core.plugin">Plugin</interfacename>
        interface). This means that you don't have to mess with configuration or 
        job parameters.
      </para>
      
      <para>
        Validator plug-ins must implement the 
        <interfacename docapi="net.sf.basedb.core.filehandler">net.sf.basedb.core.filehandler.DataFileHandler</interfacename>
        and <interfacename docapi="net.sf.basedb.core.filehandler">net.sf.basedb.core.filehandler.DataValidator</interfacename>
        interfaces. Metadata reader plug-ins should implement the 
        <interfacename docapi="net.sf.basedb.core.filehandler">net.sf.basedb.core.filehandler.DataFileHandler</interfacename>
        and <interfacename docapi="net.sf.basedb.core.filehandler">net.sf.basedb.core.filehandler.DataFileMetadataReader</interfacename>
        interfaces.
      </para>

      <note>
        <para>
        Meta data extraction can only be done if the file has first been 
        validated. We recommend that metadata reader plug-ins also takes the role as 
        validator plug-ins. This will make BASE re-use the same object instance
        and the file doesn't have to be parsed twice. 
        </para>
      </note>

      <important>
        <title>
          Always extend the 
          <classname docapi="net.sf.basedb.core.filehandler">net.sf.basedb.core.filehandler.AbstractDataFileHandler</classname>
          class
        </title>
        <para>
          We consider the mentioned interface to be part of the public API only
          from the caller side, not from the implementor side. Thus, we may
          add methods to those interfaces in the future without prior notice.
          The <classname docapi="net.sf.basedb.core.filehandler">AbstractDataFileHandler</classname> will provide default
          implementations of the new methods in order to not break existing
          plug-ins.
        </para>
      </important>
      
      <variablelist>
        <title>Methods in the <interfacename docapi="net.sf.basedb.core.filehandler">DataFileHandler</interfacename> interface</title>
        <varlistentry>
          <term>
            <methodsynopsis language="java">
              <modifier>public</modifier>
              <void/>
              <methodname>setFile</methodname>
              <methodparam>
                <type>FileSetMember</type>
                <parameter>member</parameter>
              </methodparam>
            </methodsynopsis>
          </term>
          <listitem>
            <para>
            Sets the file that is going to be validated or used for metadata
            extraction. If the same plug-in can be used for validating
            more than one type of file, this method will be called
            one time for each file that is present in the file set.
            </para>
          </listitem>
        </varlistentry>
        
        <varlistentry>
          <term>
            <methodsynopsis language="java">
              <modifier>public</modifier>
              <void/>
              <methodname>setItem</methodname>
              <methodparam>
                <type>FileStoreEnabled</type>
                <parameter>item</parameter>
              </methodparam>
            </methodsynopsis>
          </term>
          <listitem>
            <para>
            Sets the item that the files belong to. This method is only
            called once.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
      
      <variablelist>
        <title>Methods in the <interfacename docapi="net.sf.basedb.core.filehandler">DataFileValidator</interfacename> interface</title>
        <varlistentry>
          <term>
            <methodsynopsis language="java">
              <modifier>public</modifier>
              <void/>
              <methodname>validate</methodname>
              <methodparam>
                <type>DbControl</type>
                <parameter>dc</parameter>
              </methodparam>
              <exceptionname>InvalidDataException</exceptionname>
              <exceptionname>InvalidRelationException</exceptionname>
            </methodsynopsis>
          </term>
          <listitem>
            <para>
              Validate the file. The file is valid if this method returns
              sucessfully. If the file is not valid an 
              <exceptionname>InvalidDataException</exceptionname> should be
              thrown. Note that BASE will still accept the file, but will indicate 
              the failure with a flag and also keep the message of the exception in the
              database to remind the user of the failure.
            </para>
            <para>
              The <exceptionname>InvalidRelationException</exceptionname>
              should be used to indicate a partial success/partial failure,
              where the file as such is a valid file, but in relation to
              other files it is not. For example, we may assign a valid CEL 
              file to a raw bioassay, but the chip type doesn't match
              the chip type of the CDF file of the related array design.
              This exception will also allow metadata to be extracted from
              the file.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>

      <variablelist>
        <title>Methods in the <interfacename docapi="net.sf.basedb.core.filehandler">DataFileMetadataReader</interfacename> interface</title>
        <varlistentry>
          <term>
            <methodsynopsis language="java">
              <modifier>public</modifier>
              <void/>
              <methodname>extractMetadata</methodname>
              <methodparam>
                <type>DbControl</type>
                <parameter>dc</parameter>
              </methodparam>
            </methodsynopsis>
          </term>
          <listitem>
            <para>
              Extract metadata from the file. It is up to the plug-in
              to decide what to extract and how to store it.
              The <classname docapi="net.sf.basedb.core.filehandler">CelFileHandler</classname> will, for
              example, extract headers and the number of spots from the file
              and store it with the raw bioassay. 
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <methodsynopsis language="java">
              <modifier>public</modifier>
              <void/>
              <methodname>resetMetadata</methodname>
              <methodparam>
                <type>DbControl</type>
                <parameter>dc</parameter>
              </methodparam>
            </methodsynopsis>
          </term>
          <listitem>
            <para>
              Remove all metadata that the plug-in usually can extract.
              This method is called if a file is unlinked from an item
              or if the validation fails. It is important that the
              plug-in cleans up everything so that data from a
              previous file doesn't remain in the database.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>

      <variablelist>
        <title>Methods in the <classname docapi="net.sf.basedb.core.filehandler">AbstractDataFileHandler</classname> class</title>
        <varlistentry>
          <term>
            <methodsynopsis language="java">
              <modifier>public</modifier>
              <type>FileStoreEnabled</type>
              <methodname>getItem</methodname>
            </methodsynopsis>
          </term>
          <listitem>
            <para>
              Get the item that was previously added to
              <methodname>setItem()</methodname>
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <methodsynopsis language="java">
              <modifier>public</modifier>
              <type>FileSetMember</type>
              <methodname>getMember</methodname>
              <methodparam>
                <type>String</type>
                <parameter>dataFileTypeId</parameter>
              </methodparam>
            </methodsynopsis>
          </term>
          <listitem>
            <para>
              Get a file that was previously added to 
              <methodname>setFile()</methodname>. The 
              <parameter>dataFileTypeId</parameter> is
              the external ID of the <classname docapi="net.sf.basedb.core">DataFileType</classname>.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>

    </sect2>
    
  </sect1>
  
  <sect1 id="plugin_developer.signals">
    <title>Enable support for aborting a running a plug-in</title>
    
    <para>
      BASE includes a simple signalling system that can be used to send
      signals to plug-ins. The system was primarly developed to allow a user
      to kill a plug-in when it is executing. Therfore, the focus of this chapter
      will be how to implement a plug-in to make it possible to kill it
      during it's execution.
    </para>
    
    <para>
      Since we don't want to do this by brute force such as destroying the
      process or stopping thread the plug-in executes in, cooperation is needed
      by the plug-in. First, the plug-in must implement the 
      <interfacename docapi="net.sf.basedb.core.signal">SignalTarget</interfacename>
      interface. From this, a <interfacename 
      docapi="net.sf.basedb.core.signal">SignalHandler</interfacename> can be created.
      A plug-in may choose to implement it's own signal handler or use an existing
      implementation. BASE, for example, provides the <classname docapi="net.sf.basedb.core.signal"
      >ThreadSignalHandler</classname> implementation that supports the <constant>ABORT</constant> signal.
      This is a simple implementation that just calls <code>Thread.interrupt()</code>
      on the plug-in worker thread. This may cause two different effects:
    </para>
    
    <itemizedlist>
    <listitem>
      <para>
      The <code>Thread.interrupted()</code> flag is set. The plug-in must check this
      at regular intervals and if the flag is set it must cleanup, rollback 
      open transactions and exit as soon as possible.
      </para>
    </listitem>
    
    <listitem>
      <para>
      If the plug-in is waiting in a blocking call that is interruptable, for
      example <code>Thread.sleep()</code>, an <classname>InterruptedException</classname>
      is thrown. This should cause the same actions as if the flag was set to happen.
      </para>
      
      <warning>
        <title>Not all blocking calls are interruptable</title>
        <para>
        For example calling <code>InputStream.read()</code> may leave the
        plug-in waiting in a non-interruptable state. In this case there
        is nothing BASE can do to wake it up again.
        </para>
      </warning>
    </listitem>
    </itemizedlist>
    
    <para>
      Here is a general outline for a plug-in that uses the 
      <classname>ThreadSignalHandler</classname>.
    </para>
    <programlisting language="java">
<![CDATA[
private ThreadSignalHandler signalHandler;
public SignalHandler getSignalHandler()
{
   signalHandler = new ThreadSignalHandler();
   return signalHandler;
}

public void run(Request request, Response response, ProgressReporter progress)
{
   if (signalHandler != null) signalHandler.setWorkerThread(null);
   beginTransaction();
   boolean done = false;
   boolean interrupted = false;
   while (!done && !interrupted)
   {
      try
      {
         done = doSomeWork(); // NOTE! This must not take forever!
         interrupted = Thread.interrupted();
      }
      catch (InterruptedException ex)
      {
         // NOTE! Try-catch is only needed if thread calls 
         // a blocking method that is interruptable 
         interrupted = true;
      }
   }
   if (interrupted)
   {
      rollbackTransaction();
      response.setError("Aborted by user", null);
   }
   else
   {
      commitTransaction();
      response.setDone("Done");
   }
}
]]>
</programlisting>
    
    <para>
      Another signal handler implementation is the
      <classname docapi="net.sf.basedb.core.signal">ProgressReporterSignalHandler</classname>.
      See that javadoc for information about how to use it.
      For more information about the signalling system as a whole,
      see <xref linkend="core_api.signals" />.
    </para>
    
  </sect1>
  
  <sect1 id="plugin_developer.classload">
    <title>How BASE load plug-in classes</title>
    
    <para>
      We recommend that plug-in JAR files are installed outside the web server's 
      classpath. If you are using Tomcat this means that you should not
      install the plug-in in the <filename>&lt;base-dir&gt;/www/WEB-INF/lib</filename>
      directory or any other directory where the web server keeps it's classes.
      The rest of the information in this section only applies to plug-ins that
      have been installed following this restriction.
    </para>
    
    <para>
      If the above recommendation has been followed BASE will use it's
      own classloader to load the plug-in classes. This have several
      benefits:
    </para>
    
    <itemizedlist>
    <listitem>
      <para>
      New plug-ins can be installed and existing plug-ins can be updated
      without restarting the web server. If the <property>plugins.autounload</property>
      setting in <filename>base.config</filename> has been enabled allyou have to
      do to update a plug-in is to replace the JAR file with a new version.
      BASE will automatically load the new classes the next time the plug-in
      is used. If the option isn't enabled, the server admin has to manually
      unload the old code from the web interface first.
      </para>
    </listitem>
    
    <listitem>
      <para>
      Plug-ins may use it's own 3-rd party libraries without interfering
      with other plug-ins. This may be important because a plug-in may depend on
      a certain version of a library while another plug-in may depend on
      a different version. Since BASE is using different class-loaders
      for different plug-ins this is not a problem.
      </para>
    </listitem>
      
    </itemizedlist>
    
    <para>
      The classloading scheme used by BASE also means plug-in
      developers must pay attention to a few things:
    </para>
    
    <itemizedlist>
    <listitem>
      <para>
      A plug-in can only access/use classes from it's own JAR file, BASE core
      classes, Java system classes and from JAR files listed in the plug-in's
      <filename>MANIFEST.MF</filename> file. See <xref linkend="plugin_developer.organize" />.
      </para>
    </listitem>
    
    <listitem>
      <para>
      A plug-in can also access other plug-ins, but only via the methods and
      interfaces defined in BASE. In the following example we assume that 
      there are two plug-ins, <classname>ex.MyPlugin</classname> 
      and <classname>ex.MyOtherPlugin</classname>,
      located in two different JAR files. The code below is executing
      in the <classname>ex.MyPlugin</classname>:
      </para>
      
      <programlisting language="java">
// Prepare to load MyOtherPlugin
SessionControl sc = ...
DbControl dc = ...
PluginDefinition def = PluginDefinition.getByClassName(dc, "ex.MyOtherPlugin");

// Ok
Plugin other = def.newInstance(Plugin.class, null, sc, null, null);

// Not ok; fails with ClassCastException
MyOtherPlugin other = def.newInstance(MyOtherPlugin.class, null, sc, null, null);
</programlisting>
      
      <para>
      The reason that the second call fails is that BASE uses a different
      classloader to load the <classname>ex.MyOtherPlugin</classname> class. This 
      class is not (in Java terms) the same as the <classname>ex.MyOtherPlugin</classname> 
      class loaded by the classloader that loaded the <classname>ex.MyPlugin</classname>
      class. If, on the other hand, both plug-ins are located in the same
      JAR file BASE uses the same classloader and the second call will succeed.
      </para>
      
      <para>
      The first call always succeeds because it uses the <interface>Plugin</interface>
      interface which is defined by BASE. This class is loaded by the web servers class
      loaded and is the same for all plug-ins.
      </para>
      
      <para>
      A third option is that the <classname>ex.MyPlugin</classname>
      lists the JAR file where <classname>ex.MyOtherPlugin</classname> is
      located in it's <filename>MANIFEST.MF</filename> file. Then, the following
      code can be used: <code>MyOtherPlugin other = new MyOtherPlugin();</code>
      </para>
      
    </listitem>
    </itemizedlist>
    
    <para>
      Tomcat includes a good document describing how classloading is implemented
      in Tomcat: <ulink url="http://tomcat.apache.org/tomcat-5.5-doc/class-loader-howto.html"
        >http://tomcat.apache.org/tomcat-5.5-doc/class-loader-howto.html</ulink>.
      
      BASE's classloading scheme isn't as complex as Tomcat's, but it
      very similar to how Tomcat loads different web applications. 
      The figure on the linked document could be extended with another level
      with separate classloaders for each plug-in as child
      classloaders to the web application classloaders.  
    </para>
  </sect1>
  
  <sect1 id="plugin_developer.example">
    <title>Example plug-ins (with download)</title>
    <para>
      We have created some example plug-ins which demonstrates how to
      use the plug-in system and how to create an interactive plug-in that
      can ask a user for one or more parameters.
    
      You can <ulink url="http://baseplugins.thep.lu.se/wiki/net.sf.basedb.examples.plugins">download 
      a tar file with the source and compiled plug-in code</ulink> from the BASE plug-ins website.
    </para>
    
  </sect1>
</chapter>