source: trunk/doc/src/docbook/developerdoc/plugin_developer.xml @ 4974

Last change on this file since 4974 was 4974, checked in by Nicklas Nordborg, 13 years ago

Merged patch release 2.12.1 to the trunk.

  • Property svn:eol-style set to native
  • Property svn:keywords set to Id
File size: 166.0 KB
Line 
1<?xml version="1.0" encoding="UTF-8"?>
2<!DOCTYPE chapter PUBLIC
3    "-//Dawid Weiss//DTD DocBook V3.1-Based Extension for XML and graphics inclusion//EN"
4    "../../../../lib/docbook/preprocess/dweiss-docbook-extensions.dtd">
5<!--
6  $Id: plugin_developer.xml 4974 2009-06-15 09:05:06Z nicklas $:
7 
8  Copyright (C) 2007 Johan Enell, Peter Johansson, Nicklas Nordborg, Martin Svensson
9 
10  This file is part of BASE - BioArray Software Environment.
11  Available at http://base.thep.lu.se/
12 
13  BASE is free software; you can redistribute it and/or
14  modify it under the terms of the GNU General Public License
15  as published by the Free Software Foundation; either version 3
16  of the License, or (at your option) any later version.
17 
18  BASE is distributed in the hope that it will be useful,
19  but WITHOUT ANY WARRANTY; without even the implied warranty of
20  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
21  GNU General Public License for more details.
22 
23  You should have received a copy of the GNU General Public License
24  along with BASE. If not, see <http://www.gnu.org/licenses/>.
25-->
26
27<chapter id="plugin_developer">
28  <?dbhtml dir="plugin_developer"?>
29  <title>Plug-in developer</title>
30  <sect1 id="plugin_developer.organize">
31    <title>How to organize your plug-in project</title>
32   
33    <sect2 id="plugin_developer.organize.ant">
34      <title>Using Ant</title>
35      <para>
36        Here is a simple example of how you might organize your project using ant
37        (<ulink url="http://ant.apache.org">http://ant.apache.org</ulink>) as the build
38        tool. This is just a recommendation that we have found to be working
39        well. You may choose to do it another way.
40      </para>
41
42      <sect3 id="plugin_developer.organize.ant.directories">
43        <title>Directory layout</title>
44        <para>
45       
46          Create a directory on your computer where you want
47          to store your plug-in project. This directory is the
48          <filename class="directory"><replaceable>pluginname</replaceable>/</filename>
49          directory in the listing below. You should also create some subdirectories:
50         
51          <literallayout>
52<filename class="directory"><replaceable>pluginname</replaceable>/</filename>
53<filename class="directory"><replaceable>pluginname</replaceable>/bin/</filename>
54<filename class="directory"><replaceable>pluginname</replaceable>/lib/</filename>
55<filename class="directory"><replaceable>pluginname</replaceable>/src/<replaceable>org/company</replaceable>/</filename>
56<filename class="directory"><replaceable>pluginname</replaceable>/META-INF/</filename>
57          </literallayout>
58          The
59          <filename class="directory">bin/</filename>
60          directory is empty to start with. It will contain the compiled code.
61          In the <filename class="directory">lib/</filename>
62          directory you should put <filename>BASE2Core.jar</filename>
63          and other library files your plug-in depends on. The
64          <filename class="directory">src/</filename>
65          directory contains your source code. In this directory you should create
66          subdirectories corresponding to the package name of your plug-in
67          class(es). See <ulink url="http://en.wikipedia.org/wiki/Java_package"
68          >http://en.wikipedia.org/wiki/Java_package</ulink> for information
69          about conventions for naming packages. The
70          <filename class="directory">META-INF</filename> directory contains metadata
71          about the plug-in and are needed for best functionality.
72        </para>
73      </sect3>
74   
75      <sect3 id="plugin_developer.organize.ant.buildfile">
76        <title>The build file</title>
77        <para>
78          In the root of your directory, create the build file:
79          <filename>build.xml</filename>.
80          Here is an example that will compile your plug-in and put it in a JAR file.
81        </para>
82        <example id="plugin_developer.organize.build.file">
83          <title>A simple build file</title>
84<programlisting language="xml">&lt;?xml version="1.0" encoding="UTF-8"?&gt;
85&lt;project
86   name="MyPlugin"
87   default="build.plugin"
88   basedir="."
89   &gt;
90
91   &lt;!-- variables used --&gt;
92   &lt;property name="plugin.name" value="MyPlugin" /&gt;
93   &lt;property name="src" value="src" /&gt;
94   &lt;property name="bin" value="bin" /&gt;
95
96   &lt;!-- set up classpath for compiling --&gt;
97   &lt;path id="classpath"&gt;
98      &lt;fileset dir="lib"&gt;
99         &lt;include name="**/*.jar"/&gt;
100      &lt;/fileset&gt;
101   &lt;/path&gt;
102
103   
104   &lt;!-- main target --&gt;
105   &lt;target
106      name="build.plugin" 
107      description="Compiles the plug-in and put in jar"
108      &gt;
109      &lt;javac
110         encoding="ISO-8859-1"
111         srcdir="${src}"
112         destdir="${bin}"
113         classpathref="classpath"&gt;
114      &lt;/javac&gt;
115      &lt;jar
116         jarfile="${plugin.name}.jar"
117         basedir="bin"
118         manifest="META-INF/MANIFEST.MF"
119         &gt;
120         &lt;!--Include this to add required files for auto registration wizard--&gt;
121         &lt;metainf file="META-INF/base-plugins.xml"&gt;&lt;/metainf&gt;
122         &lt;metainf file="META-INF/base-configurations.xml"&gt;&lt;/metainf&gt;       
123      &lt;/jar&gt;
124
125    &lt;/target&gt;
126&lt;/project&gt; </programlisting>
127        </example>
128        <para>
129          If your plug-in depends on other JAR files than the
130          <filename>BASE2Core.jar</filename> you must create a file called
131          <filename>MANIFEST.MF</filename> in the project <filename>META-INF</filename>
132          directory. List the other JAR files as in the following example.
133          If your plug-in does not depend on other JAR files, you may remove the
134          <sgmltag class="attribute">manifest</sgmltag> 
135          attribute of the <sgmltag class="starttag">jar</sgmltag> tag.
136<programlisting>Manifest-Version: 1.0
137Class-Path: OtherJar.jar ASecondJar.jar</programlisting>
138        </para>
139        <para>
140          If your plug-in should support registration with the auto-installation
141          wizard it is a good idea to add the <sgmltag class="element">metainf</sgmltag>
142          tags. This will add the two files <filename>META-INF/base-plugins.xml</filename>
143          and <filename>META-INF/base-configurations.xml</filename> to the
144          <filename>META-INF</filename> directory in the JAR file. The two files
145          contains information about your plug-in and BASE can automatically
146          find and extract information from those. See
147          <xref linkend="plugin_developer.organize.autoinstallcompatible" />
148          for get more information about this feature.
149        </para>
150      </sect3>
151       
152      <sect3 id="plugin_developer.organize.ant.build">
153        <title>Building the plug-in</title>
154        <para>
155          Compile the plug-in simply by typing
156          <command>ant</command>
157          in the console window. If all went well the
158          <filename>MyPlugin.jar</filename>
159          will be created in the same directory.
160        </para>
161        <para>
162          To install the plug-in copy the JAR file to the server including the dependent JAR
163          files (if any). Place all files together in the same directory. For more
164          information read
165          <xref linkend="plugins.installation" />.
166        </para>
167      </sect3>
168    </sect2>
169   
170    <sect2 id="plugin_developer.organize.eclipse">
171      <title>With Eclipse</title>
172      <para>
173        If somebody is willing to add information to this chapter please send us a note or
174        some written text to put here. Otherwise, this chapter will be removed.
175      </para>
176    </sect2>
177
178    <sect2 id="plugin_developer.organize.autoinstallcompatible">
179      <title>Make the plug-in compatible with the auto-installation wizard</title>
180      <para>
181        BASE has support for automatically detecting new plug-ins with
182        the auto-installation wizard. The wizard makes it very easy for a
183        server administrator to install new plug-ins. See
184        <xref linkend="plugins.autoinstall" />.
185      </para>
186     
187      <para>
188        The auto-install feature requires that a plug-in provides some information
189        about itself. The wizard looks in all JAR file for the file
190        <filename>META-INF/base-plugins.xml</filename>. This file contains
191        some information about the plug-in(s). Here is an example:
192      </para>
193 
194      <programlisting language="xml">
195&lt;?xml version="1.0" encoding="UTF-8"?&gt;
196&lt;!DOCTYPE plugins SYSTEM "base-plugins.dtd" &gt;
197&lt;plugins jarname="jarfile.jar"&gt;
198  &lt;pluginclass classname="se.lu.thep.PluginClass"&gt;
199    &lt;minbaseversion&gt;2.4&lt;/minbaseversion&gt;
200    &lt;hasconfigurations/&gt;
201    &lt;/pluginclass&gt;
202    .
203    .
204    .
205&lt;/plugins&gt;
206</programlisting>
207      <para>
208        The first two lines should be the same in all <filename>base-plugins.xml</filename> files.
209        The rest of the tags are described in following list.
210        <variablelist>
211          <varlistentry>
212            <term><sgmltag class="starttag">plugins</sgmltag></term>
213            <listitem>
214              <para>
215                This is the root element in the XML-file and must have the
216                attribute
217                <sgmltag class="attribute">jarname</sgmltag>
218                set. The value must be the same as name of the JAR file
219                the plug-in is located in or the auto-installation wizard
220                will fail with an error.
221              </para>
222            </listitem>
223          </varlistentry>
224          <varlistentry>
225            <term>
226              <sgmltag class="starttag">pluginclass</sgmltag>
227            </term>
228            <listitem>
229              <para>
230                This tag defines information about a plug-in in the JAR file. The
231                <sgmltag class="attribute">classname</sgmltag>
232                attribute needs to have the full classname of the plug-in's main
233                class. There can be one or several of this element inside the
234                <sgmltag class="starttag">plugins</sgmltag>
235                tag, which means that there should be one element for
236                each plug-in in the JAR file.
237              </para>
238            </listitem>
239          </varlistentry>
240          <varlistentry>
241            <term>
242              <sgmltag class="starttag">minbaseversion</sgmltag>
243            </term>
244            <listitem>
245              <para>
246                A required child element in
247                <sgmltag class="starttag">pluginclass</sgmltag>.
248                This defines from which version of BASE the plug-in can be used.
249                The plug-in will not be included in auto installer if the
250                BASE-version is lower then the value of this tag. The format of the
251                value in this tag should be (numeric)
252                <emphasis>majorversion</emphasis>.<emphasis>minorversion</emphasis>.
253                It is not possible to check the bugfix or revision numbers.
254              </para>
255            </listitem>
256          </varlistentry>
257          <varlistentry>
258            <term>
259              <sgmltag class="starttag">hasconfigurations</sgmltag>             
260            </term>
261            <listitem>
262              <para>
263                A required child element in
264                <sgmltag class="starttag">pluginclass</sgmltag>.
265                This should tell if there are plug-in configurations
266                shipped with the plug-in. Setting the value to
267                <emphasis>yes</emphasis>
268                will indicate that there are available configurations in the
269                JAR file. This can be left as an empty tag if no
270                configurations are available for import.
271              </para>
272              <para>
273                Configurations shipped with a JAR file should be exported with the
274                plug-in configuration exporter in BASE. The exported file must be
275                given the name
276                <filename>base-configurations.xml</filename>
277                and be placed in
278                <filename class="directory">META-INF/</filename>.
279              </para>
280            </listitem>
281          </varlistentry>
282        </variablelist>
283      </para>
284    </sect2>
285   
286  </sect1>
287
288  <sect1 id="plugin_developer.api">
289    <title>The Plug-in API</title>
290    <para>
291    </para>
292   
293    <sect2 id="plugin_developer.api.interfaces">
294   
295      <title>The main plug-in interfaces</title>
296      <para>
297        The Base2 core defines two interfaces and one
298        abstract class that are vital for implementing plug-ins:
299        <itemizedlist spacing="compact">
300          <listitem>
301            <simpara>
302              <interfacename docapi="net.sf.basedb.core.plugin">net.sf.basedb.core.plugin.Plugin</interfacename>
303            </simpara>
304          </listitem>
305          <listitem>
306            <simpara>
307              <interfacename docapi="net.sf.basedb.core.plugin">net.sf.basedb.core.plugin.InteractivePlugin</interfacename>
308            </simpara>
309          </listitem>
310          <listitem>
311            <simpara>
312              <classname docapi="net.sf.basedb.core.plugin">net.sf.basedb.core.plugin.AbstractPlugin</classname>
313            </simpara>
314          </listitem>
315        </itemizedlist>
316       
317        A plug-in must always implement the
318        <interfacename docapi="net.sf.basedb.core.plugin">Plugin</interfacename> interface.
319        The <interfacename docapi="net.sf.basedb.core.plugin">InteractivePlugin</interfacename>
320        interface is optional, and is only needed if you want user interaction.
321        The <classname docapi="net.sf.basedb.core.plugin">AbstractPlugin</classname> is a useful base class
322        that your plug-in can use as a superclass. It provides default implementations
323        for some of the interface methods and also has utility methods for
324        validating and storing job and configuration parameter values. Another
325        reason to use this class as a superclass is that it will shield your
326        plug-in from future changes to the Plug-in API. For example, if
327        we decide that a new method is needed in the <interfacename docapi="net.sf.basedb.core.plugin">Plugin</interfacename> 
328        interface we will also try to add a default implementation in
329        the <classname docapi="net.sf.basedb.core.plugin">AbstractPlugin</classname> class.
330      </para>
331 
332      <important>
333        <para>
334        A plug-in must also have public no-argument contructor. Otherwise, BASE will not
335        be able to create new instances of the plug-in class.
336        </para>
337      </important>
338     
339      <sect3 id="plugin_developer.api.interfaces.plugin">
340        <title>The net.sf.basedb.core.plugin.Plugin interface</title>
341        <para>
342          This interface defines the following methods and must be implemented
343          by all plug-ins.
344        </para>
345        <variablelist>
346          <varlistentry>
347            <term>
348              <methodsynopsis language="java">
349                <modifier>public</modifier>
350                <type>About</type>
351                <methodname>getAbout</methodname>
352                <void />
353              </methodsynopsis>
354            </term>
355            <listitem>
356              <para>
357                Return information about the plug-in, i.e. the name, version, and a short
358                description about what the plug-in does. The
359                <classname docapi="net.sf.basedb.core.plugin">About</classname>
360                object also has fields for naming the author and various other contact
361                information. The returned information is copied by the core at
362                installation time into the database. The only required information is
363                the name of the plug-in. All other fields may have null values.
364              </para>
365              <example id="net.sf.basedb.core.plugin.Plugin.getAbout">
366                <title>A typical implementation stores this information in a static field</title>
367<programlisting language="java">
368private static final About about = new AboutImpl
369(
370   "Spot images creator",
371   "Converts a full-size scanned image into smaller preview jpg " +
372     "images for each individual spot.",
373   "2.0",
374   "2006, Department of Theoretical Physics, Lund University",
375   null,
376   "base@thep.lu.se",
377   "http://base.thep.lu.se"
378);
379 
380public About getAbout()
381{
382   return about;
383}</programlisting>
384              </example>
385            </listitem>
386          </varlistentry>
387          <varlistentry>
388            <term>
389              <methodsynopsis language="java">
390                <modifier>public</modifier>
391                <type>Plugin.MainType</type>
392                <methodname>getMainType</methodname>
393                <void />
394              </methodsynopsis>
395            </term>
396            <listitem>
397              <para>
398                Return information about the main type of plug-in. The
399                <classname docapi="net.sf.basedb.core.plugin">Plugin.MainType</classname>
400                is an enumeration with five possible values:
401                <itemizedlist>
402                  <listitem>
403                    <para>
404                      <constant>ANALYZE</constant>:
405                      An analysis plug-in
406                    </para>
407                  </listitem>
408                  <listitem>
409                    <para>
410                      <constant>EXPORT</constant>:
411                      A plug-in that exports data
412                    </para>
413                  </listitem>
414                  <listitem>
415                    <para>
416                      <constant>IMPORT</constant>:
417                      A plug-in that imports data
418                    </para>
419                  </listitem>
420                  <listitem>
421                    <para>
422                      <constant>INTENSITY</constant>:
423                      A plug-in that calculates the original spot intensities
424                      from raw data
425                    </para>
426                  </listitem>
427                  <listitem>
428                    <para>
429                      <constant>OTHER</constant>:
430                      Any other type of plug-in
431                    </para>
432                  </listitem>
433                </itemizedlist>
434                The returned value is stored in the database but is otherwise not used
435                by the core. Client applications (such as the web client) will probably
436                use this information to group the plug-ins, i.e., a button labeled
437                &gbExport;
438                will let you select among the export plug-ins.
439              </para>
440              <example id="net.sf.basedb.core.plugin.Plugin.getMainType">
441                <title>A typical implementation just return one of the values</title>
442<programlisting language="java">
443public Plugin.MainType getMainType()
444{
445   return Plugin.MainType.OTHER;
446}</programlisting>
447              </example>
448            </listitem>
449          </varlistentry>
450          <varlistentry>
451            <term>
452              <methodsynopsis language="java">
453                <modifier>public</modifier>
454                <type>boolean</type>
455                <methodname>supportsConfigurations</methodname>
456                <void />
457              </methodsynopsis>
458            </term>
459            <listitem>
460              <para>
461                If this method returns true, the plug-in can have different
462                configurations, (i.e.
463                <classname docapi="net.sf.basedb.core">PluginConfiguration</classname>).
464                Note that this method may return true even if the
465                <interfacename docapi="net.sf.basedb.core.plugin">InteractivePlugin</interfacename>
466                interface is not implemented. The
467                <classname docapi="net.sf.basedb.core.plugin">AbstractPlugin</classname>
468                returns true for this method, which is the old way before the
469                introduction of this method.
470              </para>
471            </listitem>
472          </varlistentry>
473          <varlistentry>
474            <term>
475              <methodsynopsis language="java">
476                <modifier>public</modifier>
477                <type>boolean</type>
478                <methodname>requiresConfiguration</methodname>
479                <void />
480              </methodsynopsis>
481            </term>
482            <listitem>
483              <para>
484                If this method returns true, the plug-in must have a configuration
485                to be able to run. For example, some of the core import plug-ins
486                must have information about the file format, to be able to import
487                any data.
488                The
489                <classname docapi="net.sf.basedb.core.plugin">AbstractPlugin</classname>
490                returns false for this method, which is the old way before the
491                introduction of this method.
492              </para>
493            </listitem>
494          </varlistentry>
495          <varlistentry>
496            <term>
497              <methodsynopsis language="java">
498                <modifier>public</modifier>
499                <type>Collection&lt;Permissions&gt;</type>
500                <methodname>getPermissions</methodname>
501                <void />
502              </methodsynopsis>
503            </term>
504            <listitem>
505              <para>
506                Return a collection of permissions that the plug-in needs
507                to be able to function as expected. This method may return null
508                or an empty collection. In this case the plug-in permission system
509                is not used and the plug-in always gets the same permissions as the
510                logged in user. If permissions are specified the plug-in should
511                list all permissions it requires. Permissions that are not listed
512                are denied.
513              </para>
514              <note>
515                <para>
516                The final assignment of permissions to a plug-in is always
517                at the hands of a server administrator. He/she may decide to
518                disable the plug-in permission system or revoke some of the
519                requested permissions. The permissions returned by this method is
520                only a recommendation that the server administrator may or may not
521                accept. See <xref linkend="plugins.permissions" />
522                for more information about plug-in permissions.
523                </para>
524              </note>
525            </listitem>
526          </varlistentry>
527          <varlistentry>
528            <term>
529              <methodsynopsis language="java">
530                <modifier>public</modifier>
531                <void />
532                <methodname>init</methodname>
533                <methodparam>
534                  <type>SessionControl</type>
535                  <parameter>sc</parameter>
536                </methodparam>
537                <methodparam>
538                  <type>ParameterValues</type>
539                  <parameter>configuration</parameter>
540                </methodparam>
541                <methodparam>
542                  <type>ParameterValues</type>
543                  <parameter>job</parameter>
544                </methodparam>
545                <exceptionname>BaseException</exceptionname>
546              </methodsynopsis>
547            </term>
548            <listitem>
549              <para>
550                Prepare the plug-in for execution or configuration. If the plug-in needs
551                to do some initialization this is the place to do it. A typical
552                implementation however only stores the passed parameters in instance
553                variables for later use. Since it is not possible what the user is going to
554                do at this stage, we recommend lazy initialisation of all other resources.
555              </para>
556              <para>
557                The parameters passed to this method has vital information that is
558                needed to execute the plug-in. The
559                <classname docapi="net.sf.basedb.core">SessionControl</classname>
560                is a central core object holding information about the logged in
561                user and is used to create
562                <classname docapi="net.sf.basedb.core">DbControl</classname>
563                objects which allows a plug-in to connect to the database to read, add or
564                update information. The two
565                <classname docapi="net.sf.basedb.core.plugin">ParameterValues</classname>
566                objects contain information about the configuration and job
567                parameters to the plug-in.
568                The <varname>configuration</varname> object holds all parameters stored
569                together with a <classname docapi="net.sf.basedb.core">PluginConfiguration</classname>
570                object in the database. If the plug-in is started without
571                a configuration this object is null.
572                The <varname>job</varname> object holds all parameters that are
573                stored together with a <classname docapi="net.sf.basedb.core">Job</classname> object in the
574                database. This object is null if the plug-in is started without a job.
575              </para>
576              <para>
577                The difference between a configuration parameter and a job parameter is
578                that a configuration is usually something an administrator sets up,
579                while a job is an actual execution of a plug-in. For example, a
580                configuration for an import plug-in holds the regular expressions needed
581                to parse a text file and find the headers, sections and data lines,
582                while the job holds the file to parse.
583              </para>
584              <para>
585                The
586                <classname docapi="net.sf.basedb.core.plugin">AbstractPlugin</classname>
587                contains an implementation of this method that saves the passed objects
588                in protected instance variables. If you override this method
589                we recommend that you also call <code>super.init()</code>.
590              </para>
591              <example id="net.sf.basedb.core.plugin.Plugin.init">
592                <title>The <classname docapi="net.sf.basedb.core.plugin">AbstractPlugin</classname> implementation of Plugin.init()</title>
593<programlisting language="java">
594protected SessionControl sc = null;
595protected ParameterValues configuration = null;
596protected ParameterValues job = null;
597/**
598   Store copies of the session control, plug-in and job configuration. These
599   are available to subclasses in the {@link #sc}, {@link #configuration}
600   and {@link #job} variables. If a subclass overrides this method it is
601   recommended that it also calls super.init(sc, configuration, job).
602*/
603public void init(SessionControl sc,
604   ParameterValues configuration, ParameterValues job)
605   throws BaseException
606{
607   this.sc = sc;
608   this.configuration = configuration;
609   this.job = job;
610}</programlisting>
611              </example>
612            </listitem>
613          </varlistentry>
614          <varlistentry>
615            <term>
616              <methodsynopsis language="java">
617                <modifier>public</modifier>
618                <void />
619                <methodname>run</methodname>
620                <methodparam>
621                  <type>Request</type>
622                  <parameter>request</parameter>
623                </methodparam>
624                <methodparam>
625                  <type>Response</type>
626                  <parameter>response</parameter>
627                </methodparam>
628                <methodparam>
629                  <type>ProgressReporter</type>
630                  <parameter>progress</parameter>
631                </methodparam>
632              </methodsynopsis>
633            </term>
634            <listitem>
635              <para>
636                Run the plug-in.
637              </para>
638              <para>
639                The <varname>request</varname>
640                parameter is of historical interest only. It has no useful information
641                and can be ignored.
642              </para>
643              <para>
644                The <varname>progress</varname> parameter
645                can be used by a plug-in to report its progress back to the core. The
646                core will usually send the progress information to the database, which
647                allows users to see exactly how the plug-in is progressing from the web
648                interface. This parameter can be null, but if it is not we recommend all
649                plug-ins to use it. However, it should be used sparingly, since each call
650                to set the progress results in a database update. If the execution
651                involves several thousands of items it is a bad idea to update the
652                progress after processing each one of them. A good starting point is to
653                divide the work into 100 pieces each representing 1% of the work, i.e.,
654                if the plug-in should export 100 000 items it should report progress
655                after every 1000 items.
656              </para>
657              <para>
658                The <varname>response</varname>
659                parameter is used to tell the core if the plug-in was successful or
660                failed. Not setting a response is considered a failure by the core. From
661                the run method it is only allowed to use the
662                <methodname>Response.setDone()</methodname>
663                or the
664                <methodname>Response.setError()</methodname>
665                methods.
666              </para>
667             
668              <important>
669                It is also considered bad practice to let exceptions escape
670                out from this method. Always use <code>try...catch</code>
671                to catch exceptions and use <code>Response.setError()</code>
672                to report the error back to the core.
673              </important>
674             
675              <example id="net.sf.basedb.core.plugin.Plugin.run">
676                <title>
677                  Here is a skeleton that we recommend each plug-in to use in its
678                  implementation of the
679                  <methodname>run()</methodname>
680                  method
681                </title>
682<programlisting language="java">
683public void run(Request request, Response response, ProgressReporter progress)
684{
685   // Open a connection to the database
686   // sc is set by init() method
687   DbControl dc = sc.newDbControl();
688   try
689   {
690      // Insert code for plug-in here
691
692      // Commit the work
693      dc.commit();
694      response.setDone("Plug-in ended successfully");
695   }
696   catch (Throwable t)
697   {
698      // All exceptions must be catched and sent back
699      // using the response object
700      response.setError(t.getMessage(), Arrays.asList(t));
701   }
702   finally
703   {
704      // IMPORTANT!!! Make sure opened connections are closed
705      if (dc != null) dc.close();
706   }
707}</programlisting>
708              </example>
709            </listitem>
710          </varlistentry>
711          <varlistentry>
712            <term>
713              <methodsynopsis language="java">
714                <modifier>public</modifier>
715                <void />
716                <methodname>done</methodname>
717                <void />
718              </methodsynopsis>
719            </term>
720            <listitem>
721              <para>
722                Clean up all resources after executing the plug-in. This method must not
723                throw any exceptions.
724              </para>
725              <example id="net.sf.basedb.core.plugin.Plugin.done">
726                <title>
727                  The
728                  <classname docapi="net.sf.basedb.core.plugin">AbstractPlugin</classname>
729                  contains an implementation of the <methodname>done()</methodname>
730                  method simply sets the
731                  parameters passed to the
732                  <methodname>init()</methodname>
733                  method to null
734                </title>
735<programlisting language="java">
736/**
737   Clears the variables set by the init method. If a subclass
738   overrides this method it is recommended that it also calls super.done().
739*/
740public void done()
741{
742   configuration = null;
743   job = null;
744   sc = null;
745}</programlisting>
746              </example>
747            </listitem>
748          </varlistentry>
749        </variablelist>
750      </sect3>
751 
752      <sect3 id="plugin_developer.api.interfaces.interactive">
753        <title>The net.sf.basedb.core.plugin.InteractivePlugin interface</title>
754        <para>
755          If you want the plug-in to be able to interact with the user you must also implement
756          this interface. This is probably the case for most plug-ins. Among the core plug-ins
757          shipped with BASE the
758          <classname docapi="net.sf.basedb.plugins">SpotImageCreator</classname>
759          is one plug-in that does not interact with the user. Instead, the web client has
760          special JSP pages that handles all the interaction, creates a job for it and sets
761          the parameters. This, kind of hardcoded, approach can also be used for other
762          plug-ins, but then it usually requires modification of the client application
763          as well.
764        </para>
765        <para>
766          The
767          <interfacename docapi="net.sf.basedb.core.plugin">InteractivePlugin</interfacename>
768          has three main tasks:
769         
770          <orderedlist>
771          <listitem>
772            <para>
773            Tell a client application where the plug-in should be plugged
774            in.
775            </para>
776          </listitem>
777          <listitem>
778            <para>
779            Ask the users for configuration and job parameters.
780            </para>
781          </listitem>
782         
783          <listitem>
784            <para>
785            Validate parameter values entered by the user and store those in the
786            database.
787            </para>
788          </listitem>
789          </orderedlist>
790          This requires that the following methods are
791          implemented.
792        </para>
793       
794        <variablelist>
795          <varlistentry>
796            <term>
797              <methodsynopsis language="java">
798                <modifier>public</modifier>
799                <type>Set&lt;GuiContext&gt;</type>
800                <methodname>getGuiContexts</methodname>
801                <void />
802              </methodsynopsis>
803            </term>
804            <listitem>
805              <para>
806                Return information about where the plug-in should be plugged in. Each
807                place is identified by a
808                <classname docapi="net.sf.basedb.core.plugin">GuiContext</classname>
809                object, which is an
810                <classname docapi="net.sf.basedb.core">Item</classname>
811                and a
812                <classname docapi="net.sf.basedb.core">Type</classname>.
813                The item is one of the objects defined by the
814                <classname docapi="net.sf.basedb.core">Item</classname>
815                enumeration and the type is either
816                <constant>Type.LIST</constant>
817                or
818                <constant>Type.ITEM</constant>, which corresponde to the
819                list view and the single-item view in the web client.
820              </para>
821              <para>
822                For example, the
823                <varname>GuiContext</varname> =
824                (<constant>Item.REPORTER</constant>,
825                <constant>Type.LIST</constant>)
826                tells a client application that this plug-in can be plugged in whenever a
827                list of reporters is displayed. The
828                <varname>GuiContext</varname> =
829                (<constant>Item.REPORTER</constant>,
830                <constant>Type.ITEM</constant>)
831                tells a client application that this plug-in can be plugged in whenever
832                a single reporter is displayed. The first case may be appropriate for a
833                plug-in that imports or exports reporters. The second case may be used by
834                a plug-in that updates the reporter information from an external source
835                (well, it may make sense to use this in the list case as well).
836              </para>
837              <para>
838                The returned information is copied by the core at installation time to
839                make it easy to ask for all plug-ins for a certain
840                <classname docapi="net.sf.basedb.core.plugin">GuiContext</classname>.
841              </para>
842              <para>
843                A typical implementation creates a static unmodifiable
844                <classname>Set</classname>
845                which is returned by this method. It is important that the returned set
846                cannot be modified. It may be a security issue if a misbehaving
847                client application does that.
848              </para>
849              <example id="net.sf.basedb.core.plugin.InteractivePlugin.getGuiContexts">
850                <title>
851                  A typical implementation of
852                  <methodname>getGuiContexts</methodname>
853                </title>
854<programlisting language="java">
855// From the net.sf.basedb.plugins.RawDataFlatFileImporter plug-in
856private static final Set&lt;GuiContext&gt; guiContexts =
857   Collections.singleton(new GuiContext(Item.RAWBIOASSAY, GuiContext.Type.ITEM));
858
859public Set&lt;GuiContext&gt; getGuiContexts()
860{
861   return guiContexts;
862}</programlisting>
863              </example>
864            </listitem>
865          </varlistentry>
866          <varlistentry>
867            <term>
868              <methodsynopsis language="java">
869                <modifier>public</modifier>
870                <type>String</type>
871                <methodname>isInContext</methodname>
872                <methodparam>
873                  <type>GuiContext</type>
874                  <parameter>context</parameter>
875                </methodparam>
876                <methodparam>
877                  <type>Object</type>
878                  <parameter>item</parameter>
879                </methodparam>
880              </methodsynopsis>
881            </term>
882            <listitem>
883              <para>
884                This method is called to check if a particular item is usable for the
885                plug-in. This method is invoked to check if a plug-in can be used
886                in a given context. If invoked from a list context the <parameter>item</parameter>
887                parameter is <constant>null</constant>
888                The plug-in should return <constant>null</constant> if it
889                finds that it can be used. If the plug-in can't be used it
890                must decide if the reason should be a warning or an error condition.
891              </para>
892             
893              <para>
894                A warning is issued by returning a string with the warning
895                message. It should be used when the plug-in can't be used because
896                it is unrelated to the current task. For example, a plug-in for
897                importing Genepix data should return a warning when somebody wants
898                to import data to an Agilent raw bioassay.
899              </para>
900             
901              <para>
902                An error message is issued by throwing an exception. This
903                should be used when the plug-in is related to the current task
904                but still can't do what it is supposed to do. For example,
905                trying to import raw data if the logged in user doesn't have
906                write permission to the raw bioassay.
907              </para>
908             
909              <para>
910                As a rule of thumb, if there is a chance that another plug-in
911                might be able to perform the same task a warning should be used.
912                If it is guaranteed that no other plug-in can do it an error
913                message should be used.
914              </para>
915             
916              <note>
917                <para>
918                The contract of this method was changed in in BASE 2.4
919                to allow warning and error level message. Prior to BASE
920                2.4 all messages were treated as error message. We recommend
921                that existing plug-ins are updated to throw exception to indicate
922                error-level messages since the default is to not show
923                warning messages to users.
924                </para>
925              </note>
926             
927              <para>
928                Here is a real example from the
929                <classname docapi="net.sf.basedb.plugins">RawDataFlatFileImporter</classname>
930                plug-in which imports raw data to a
931                <classname docapi="net.sf.basedb.core">RawBioAssay</classname>.
932               
933                Thus,
934                <varname>GuiContext</varname> =
935                (<constant>Item.RAWBIOASSAY</constant>,
936                <constant>Type.ITEM</constant>),
937               
938                but the plug-in can only import data if the logged in user has write permission,
939                there is no data already, and
940                if the raw bioassay has the same raw data type as the plug-in has been
941                configured for.
942              </para>
943              <example id="net.sf.basedb.core.plugin.InteractivePlugin.isInContext">
944                <title>
945                  A realistic implementation of the
946                  <methodname>isInContext()</methodname> method
947                </title>
948<programlisting language="java">
949/**
950   Returns null if the item is a {@link RawBioAssay} of the correct
951   {@link RawDataType} and doesn't already have spots.
952   @throws PermissionDeniedException If the raw bioasssay already has raw data
953   or if the logged in user doesn't have write permission
954*/
955public String isInContext(GuiContext context, Object item)
956{
957   String message = null;
958   if (item == null)
959   {
960      message = "The object is null";
961   }
962   else if (!(item instanceof RawBioAssay))
963   {
964      message = "The object is not a RawBioAssay: " + item;
965   }
966   else
967   {
968      RawBioAssay rba = (RawBioAssay)item;
969      String rawDataType = (String)configuration.getValue("rawDataType");
970      RawDataType rdt = rba.getRawDataType();
971      if (!rdt.getId().equals(rawDataType))
972      {
973         // Warning
974         message = "Unsupported raw data type: " + rba.getRawDataType().getName();
975      }
976      else if (!rdt.isStoredInDb())
977      {
978         // Warning
979         message = "Raw data for raw data type '" + rdt + "' is not stored in the database";
980      }
981      else if (rba.hasData())
982      {
983         // Error
984         throw new PermissionDeniedException("The raw bioassay already has data.");
985      }
986      else
987      {
988         // Error
989         rba.checkPermission(Permission.WRITE);
990      }
991   }
992   return message;   
993}</programlisting>
994              </example>
995            </listitem>
996          </varlistentry>
997          <varlistentry>
998            <term>
999              <methodsynopsis language="java">
1000                <modifier>public</modifier>
1001                <type>RequestInformation</type>
1002                <methodname>getRequestInformation</methodname>
1003                <methodparam>
1004                  <type>GuiContext</type>
1005                  <parameter>context</parameter>
1006                </methodparam>
1007                <methodparam>
1008                  <type>String</type>
1009                  <parameter>command</parameter>
1010                </methodparam>
1011                <exceptionname>BaseException</exceptionname>
1012              </methodsynopsis>
1013            </term>
1014            <listitem>
1015              <para>
1016                Ask the plug-in for parameters that need to be entered by the user. The
1017                <classname docapi="net.sf.basedb.core.plugin">GuiContext</classname>
1018                parameter is one of the contexts returned by the
1019                <methodname>getGuiContexts</methodname>
1020                method. The command is a string telling the plug-in what command was
1021                executed. There are two predefined commands but as you will see the
1022                plug-in may define its own commands. The two predefined commands are
1023                defined in the
1024                <classname docapi="net.sf.basedb.core.plugin">net.sf.basedb.core.plugin.Request</classname>
1025                class.
1026                <variablelist>
1027                  <varlistentry>
1028                    <term>
1029                      <constant>Request.COMMAND_CONFIGURE_PLUGIN</constant>
1030                    </term>
1031                    <listitem>
1032                      <para>
1033                        Used when an administrator is initiating a configuration
1034                        of the plug-in.
1035                      </para>
1036                    </listitem>
1037                  </varlistentry>
1038                  <varlistentry>
1039                    <term>
1040                      <constant>Request.COMMAND_CONFIGURE_JOB</constant>
1041                    </term>
1042                    <listitem>
1043                      <para>
1044                        Used when a user has selected the plug-in for running a
1045                        job.
1046                      </para>
1047                    </listitem>
1048                  </varlistentry>
1049                </variablelist>
1050                Given this information the plug-in must return a
1051                <classname docapi="net.sf.basedb.core">RequestInformation</classname>
1052                object. This is simply a title, a description, and a list of parameters.
1053                Usually the title will end up as the input form title and the
1054                description as a help text for the entire form. Do not put information
1055                about the individual parameters in this description, since each
1056                parameter has a description of its own.
1057              </para>
1058              <example id="net.sf.basedb.core.plugin.InteractivePlugin.getRequestInformation">
1059                <title>
1060                  When running an import plug-in it needs to ask for the file to import
1061                  from and if existing items should be updated or not
1062                </title>
1063<programlisting language="java">
1064// The complete request information
1065private RequestInformation configure Job;
1066
1067// The parameter that asks for a file to import from
1068private PluginParameter&lt;File&gt; file Parameter;
1069
1070// The parameter that asks if existing items should be updated or not
1071private PluginParameter&lt;Boolean&gt; updateExistingParameter;
1072
1073public RequestInformation getRequestInformation(GuiContext context, String command)
1074   throws BaseException
1075{
1076   RequestInformation requestInformation = null;
1077   if (command.equals(Request.COMMAND_CONFIGURE_PLUGIN))
1078   {
1079      requestInformation = getConfigurePlugin();
1080   }
1081   else if (command.equals(Request.COMMAND_CONFIGURE_JOB))
1082   {
1083      requestInformation = getConfigureJob();
1084   }
1085   return requestInformation;
1086}
1087
1088/**
1089   Get (and build) the request information for starting a job.
1090*/
1091private RequestInformation getConfigureJob()
1092{
1093   if (configureJob == null)
1094   {
1095      // A file is required
1096      fileParameter = new PluginParameter&lt;File&gt;(
1097         "file",
1098         "File",
1099         "The file to import the data from",
1100         new FileParameterType(null, true, 1)
1101      );
1102     
1103      // The default value is 'false'
1104      updateExistingParameter = new PluginParameter&lt;Boolean&gt;(
1105         "updateExisting",
1106         "Update existing items",
1107         "If this option is selected, already existing items will be updated " +
1108         " with the information in the file. If this option is not selected " +
1109         " existing items are left untouched.",
1110         new BooleanParameterType(false, true)
1111      );
1112
1113      List&lt;PluginParameter&lt;?&gt;&gt; parameters =
1114         new ArrayList&lt;PluginParameter&lt;?&gt;&gt;(2);
1115      parameters.add(fileParameter);
1116      parameters.add(updateExistingParameter);
1117     
1118      configureJob = new RequestInformation
1119      (
1120         Request.COMMAND_CONFIGURE_JOB,
1121         "Select a file to import items from",
1122         "Description",
1123         parameters
1124      );
1125   }
1126   return configureJob;
1127}</programlisting>
1128              </example>
1129              <para>
1130                As you can see it takes a lot of code to put together a
1131                <classname docapi="net.sf.basedb.core">RequestInformation</classname>
1132                object. For each parameter you need one
1133                <classname docapi="net.sf.basedb.core">PluginParameter</classname>
1134                object and one
1135                <classname docapi="net.sf.basedb.core">ParameterType</classname>
1136                object. To make life a little easier, a
1137                <classname docapi="net.sf.basedb.core">ParameterType</classname>
1138                can be reused for more than one
1139                <classname docapi="net.sf.basedb.core">PluginParameter</classname>.
1140              </para>
1141             
1142<programlisting language="java">
1143StringParameterType stringPT = new StringParameterType(255, null, true);
1144PluginParameter one = new PluginParameter("one", "One", "First string", stringPT);
1145PluginParameter two = new PluginParameter("two", "Two", "Second string", stringPT);
1146// ... and so on
1147</programlisting>
1148              <para>
1149                The
1150                <classname docapi="net.sf.basedb.core">ParameterType</classname>
1151                is an abstract base class for several subclasses each implementing a
1152                specific type of parameter. The list of subclasses may grow in the
1153                future, but here are the most important ones currently implemented.
1154              </para>
1155              <note>
1156                <para>
1157                  Most parameter types include support for supplying a predefined list
1158                  of options to select from. In that case the list will be displayed
1159                  as a drop-down list for the user, otherwise a free input field is
1160                  used.
1161                </para>
1162              </note>
1163              <variablelist>
1164                <varlistentry>
1165                  <term>
1166                    <classname docapi="net.sf.basedb.core">StringParameterType</classname>
1167                  </term>
1168                  <listitem>
1169                    <para>
1170                      Asks for a string value. Includes an option for
1171                      specifying the maximum length of the string.
1172                    </para>
1173                  </listitem>
1174                </varlistentry>
1175                <varlistentry>
1176                  <term>
1177                    <classname docapi="net.sf.basedb.core">FloatParameterType</classname>,
1178                    <classname docapi="net.sf.basedb.core">DoubleParameterType</classname>,
1179                    <classname docapi="net.sf.basedb.core">IntegerParameterType</classname>,
1180                    <classname docapi="net.sf.basedb.core">LongParameterType</classname>
1181                  </term>
1182                  <listitem>
1183                    <para>
1184                      Asks for numerical values. Includes options for
1185                      specifying a range (min/max) of allowed values.
1186                    </para>
1187                  </listitem>
1188                </varlistentry>
1189                <varlistentry>
1190                  <term>
1191                    <classname docapi="net.sf.basedb.core">BooleanParameterType</classname>
1192                  </term>
1193                  <listitem>
1194                    <para>Asks for a boolean value.
1195                    </para>
1196                  </listitem>
1197                </varlistentry>
1198                <varlistentry>
1199                  <term>
1200                    <classname docapi="net.sf.basedb.core">DateParameterType</classname>
1201                  </term>
1202                  <listitem>
1203                    <para>Asks for a date.
1204                    </para>
1205                  </listitem>
1206                </varlistentry>
1207                <varlistentry>
1208                  <term>
1209                    <classname docapi="net.sf.basedb.core">FileParameterType</classname>
1210                  </term>
1211                  <listitem>
1212                    <para>Asks for a file item.
1213                    </para>
1214                  </listitem>
1215                </varlistentry>
1216                <varlistentry>
1217                  <term>
1218                    <classname docapi="net.sf.basedb.core">ItemParameterType</classname>
1219                  </term>
1220                  <listitem>
1221                    <para>
1222                      Asks for any other item. This parameter type requires
1223                      that a list of options is supplied, except when the item
1224                      type asked for matches the current <classname docapi="net.sf.basedb.core.plugin">GuiContext</classname>, in which
1225                      case the currently selected item is used as the
1226                      parameter value.
1227                    </para>
1228                  </listitem>
1229                </varlistentry>
1230                <varlistentry>
1231                  <term>
1232                    <classname docapi="net.sf.basedb.core">PathParameterType</classname>
1233                  </term>
1234                  <listitem>
1235                    <para>
1236                      Ask for a path to a file or directory. The path may be
1237                      non-existing and should be used when a plug-in needs an
1238                      output destination, i.e., the file to export to, or a
1239                      directory where the output files should be placed.
1240                    </para>
1241                  </listitem>
1242                </varlistentry>
1243              </variablelist>
1244              <para>
1245                You can also create a
1246                <classname docapi="net.sf.basedb.core">PluginParameter</classname>
1247                with a null name and
1248                <classname docapi="net.sf.basedb.core">ParameterType</classname>.
1249                In this case, the web client will not ask for input from the user, instead
1250                it is used as a section header, allowing you to group parameters into
1251                different sections which increase the readability of the input
1252                parameters page.
1253              </para>
1254<programlisting language="java">
1255PluginParameter firstSection = new PluginParameter(null, "First section", null, null);
1256PluginParameter secondSection = new PluginParameter(null, "Second section", null, null);
1257// ...
1258
1259parameters.add(firstSection);
1260parameters.add(firstParameterInFirstSection);
1261parameters.add(secondParameteInFirstSection);
1262
1263parameters.add(secondSection);
1264parameters.add(firstParameterInSecondSection);
1265parameters.add(secondParameteInSecondSection);
1266</programlisting>
1267            </listitem>
1268          </varlistentry>
1269          <varlistentry>
1270            <term>
1271              <methodsynopsis language="java">
1272                <modifier>public</modifier>
1273                <void />
1274                <methodname>configure</methodname>
1275                <methodparam>
1276                  <type>GuiContext</type>
1277                  <parameter>context</parameter>
1278                </methodparam>
1279                <methodparam>
1280                  <type>Request</type>
1281                  <parameter>request</parameter>
1282                </methodparam>
1283                <methodparam>
1284                  <type>Response</type>
1285                  <parameter>response</parameter>
1286                </methodparam>
1287              </methodsynopsis>
1288            </term>
1289            <listitem>
1290              <para>
1291                Sends parameter values entered by the user for processing by the plug-in.
1292                The plug-in must validate that the parameter values are
1293                correct and then store them in database.
1294              </para>
1295             
1296              <important>
1297                <para>
1298                No validation is done by the core, except converting the input to the
1299                correct object type, i.e. if the plug-in asked for a
1300                <classname>Float</classname>
1301                the input string is parsed and converted to a
1302                <classname>Float</classname>. If you have extended the
1303                <classname docapi="net.sf.basedb.core.plugin">AbstractPlugin</classname>
1304                class it is very easy to validate the parameters with the
1305                <methodname>AbstractPlugin.validateRequestParameters()</methodname>
1306                method. This method takes the same list of
1307                <classname docapi="net.sf.basedb.core">PluginParameter</classname>:s
1308                as used in the
1309                <classname docapi="net.sf.basedb.core">RequestInformation</classname>
1310                object and uses that information for validation. It returns null or a
1311                list of
1312                <exceptionname>Throwable</exceptionname>:s that can be
1313                given directly to the <code>response.setError()</code>
1314                methods.
1315                </para>
1316              </important>
1317              <para>
1318                When the parameters have been validated, they need to be stored
1319                in the database. Once again, it is very easy, if you use one of the
1320                <methodname>AbstractPlugin.storeValue()</methodname>
1321                or
1322                <methodname>AbstractPlugin.storeValues()</methodname>
1323                methods.
1324              </para>
1325              <para>
1326                The configure method works much like the <methodname>Plugin.run()</methodname> 
1327                method. It must return the result in the <classname docapi="net.sf.basedb.core.plugin">Response</classname> object,
1328                and should not throw any exceptions.
1329              </para>
1330             
1331              <example id="net.sf.basedb.core.plugin.InteractivePlugin.configure">
1332                <title>
1333                  Configuration implementation building on the examples above
1334                </title>
1335<programlisting language="java">
1336public void configure(GuiContext context, Request request, Response response)
1337{
1338   String command = request.getCommand();
1339   try
1340   {
1341      if (command.equals(Request.COMMAND_CONFIGURE_PLUGIN))
1342      {
1343         // TODO
1344      }
1345      else if (command.equals(Request.COMMAND_CONFIGURE_JOB))
1346      {
1347         // Validate user input
1348         List&lt;Throwable&gt; errors =
1349            validateRequestParameters(getConfigureJob().getParameters(), request);
1350         if (errors != null)
1351         {
1352            response.setError(errors.size() +
1353               " invalid parameter(s) were found in the request", errors);
1354            return;
1355         }
1356         
1357         // Store user input
1358         storeValue(job, request, fileParameter);
1359         storeValue(job, request, updateExistingParameter);
1360         
1361         // We are happy and done
1362         response.setDone("Job configuration complete", Job.ExecutionTime.SHORT);
1363         // TODO - check file size to make a better estimate of execution time
1364      }
1365   }
1366   catch (Throwable ex)
1367   {
1368      response.setError(ex.getMessage(), Arrays.asList(ex));
1369   }
1370}</programlisting>
1371              </example>
1372             
1373              <para>
1374                Note that the call to
1375                <methodname>response.setDone()</methodname>
1376                has a second parameter
1377                <constant>Job.ExecutionTime.SHORT</constant>. It is an indication
1378                about how long time it will take to execute the
1379                plug-in. This is of interest for job queue managers which probably
1380                does not want to start too many long-running jobs at the same time
1381                blocking the entire system. Please try to use this parameter wisely and
1382                not use <constant>Job.ExecutionTime.SHORT</constant>
1383                out of old habit all the time.
1384              </para>
1385              <para>
1386                The <classname docapi="net.sf.basedb.core.plugin">Response</classname> class also has a
1387                <methodname>setContinue()</methodname>
1388                method which tells the core that the plug-in needs more parameters,
1389                i.e. the core will then call
1390                <methodname>getRequestInformation()</methodname>
1391                again with the new command, let the user enter values, and then call
1392                <methodname>configure()</methodname>
1393                with the new values. This process is repeated until the plug-in
1394                reports that it is done or an error occurs.
1395              </para>
1396              <tip>
1397                <para>
1398                You do not have to store all values the plug-in asked for in the
1399                first place. You may even choose to store different values than
1400                those that were entered. For example, you might ask for the mass
1401                and height of a person and then only store the body mass index,
1402                which is calculated from those values.
1403                </para>
1404              </tip>
1405              <para>
1406                An important note is that during this iteration it is the same instance
1407                of the plug-in that is used. However, no parameter values are stored in
1408                the database until the plugin sends a
1409                <methodname>response.setDone()</methodname>.
1410                After that, the plug-in instance is usually discarded, and a job is placed
1411                in the job queue. The execution of the plug-in happens in a new instance
1412                and maybe on a different server. This means that a plug-in can't store
1413                state from the configuration phase internally and expect it to be there
1414                in the execution phase. Everything the plug-in needs to do it's job must
1415                be stored as parameters in the database.
1416              </para>
1417              <para>
1418                The only exception to the above rule is if the plug-in answers with
1419                <methodname>Response.setExecuteImmediately()</methodname>
1420                or <methodname>Response.setDownloadImmediately()</methodname>.
1421                Doing so bypasses the entire job queue system and requests that the
1422                job is started immediately. This is a permission that has to
1423                be granted to each plug-in by the server administrator. If the
1424                plug-in has this permission, the same object instance that
1425                was used in the configuration phase is also used in the execution
1426                phase. This is the only case where a plug-in can retain internal
1427                state between the two phases.
1428              </para>
1429             
1430            </listitem>
1431          </varlistentry>
1432        </variablelist>
1433      </sect3>
1434   
1435    </sect2>
1436 
1437    <sect2 id="plugin_developer.api.callsequence">
1438      <title>How the BASE core interacts with the plug-in when...</title>
1439     
1440      <para>
1441        This section describes how the BASE core interacts with the
1442        plug-in in a number of use cases. We will outline the
1443        order the methods are invoked on the plug-in.
1444      </para>
1445     
1446      <sect3 id="plugin_developer.api.callsequence.install">
1447        <title>Installing a plug-in</title>
1448       
1449        <para>
1450          When a plug-in is installed the core is eager to find out
1451          information about the plug-in. To do this it calls the
1452          following methods in this order:
1453        </para>
1454       
1455        <orderedlist>
1456        <listitem>
1457          <para>
1458          A new instance of the plug-in class is created. The plug-in must
1459          have a public no-argument constructor.
1460          </para>
1461        </listitem>
1462       
1463        <listitem>
1464          <para>
1465          Calls are made to <methodname>Plugin.getMainType()</methodname>,
1466          <methodname>Plugin.supportsConfigurations()</methodname>,
1467          <methodname>Plugin.requiresConfiguration()</methodname> and
1468          <methodname>Plugin.getAbout()</methodname> to find out information
1469          about the plug-in. This is the only time these methods are called.
1470          The information that is returned by them are copied and stored in
1471          the database for easy access.
1472          </para>
1473         
1474          <note>
1475            <para>
1476            The <methodname>Plugin.init()</methodname> method is
1477            never called during plug-in installation.
1478            </para>
1479          </note>
1480        </listitem>
1481       
1482        <listitem>
1483          <para>
1484          If the plug-in implements the <interfacename docapi="net.sf.basedb.core.plugin">InteractivePlugin</interfacename>
1485          interface the <methodname>InteractivePlugin.getGuiContexts()</methodname>
1486          method is called. This is the only time this method is called and the information it
1487          returns are copied and stored in the database.
1488          </para>
1489        </listitem>
1490       
1491        <listitem>
1492          <para>
1493          If the server admin decided to use the plug-in permission system, the
1494          <methodname>Plugin.getPermissions()</methodname> method is called.
1495          The returned information is copied and stored in the database.
1496          </para>
1497        </listitem>
1498       
1499        </orderedlist>
1500      </sect3>
1501   
1502      <sect3 id="plugin_developer.api.callsequence.configure">
1503        <title>Configuring a plug-in</title>
1504       
1505        <para>
1506          The plug-in must implement the <interfacename docapi="net.sf.basedb.core.plugin">InteractivePlugin</interfacename>
1507          interface and the <methodname>Plugin.supportsConfigurations()</methodname> method
1508          must return <constant>TRUE</constant>. The configuration is done with
1509          a wizard-like interface (see <xref linkend="plugins.configuration.wizard" />).
1510          The same plug-in instance is used throughout the entire configuration sequence.
1511        </para>
1512       
1513        <orderedlist>
1514        <listitem>
1515          <para>
1516          A new instance of the plug-in class is created. The plug-in must
1517          have a public no-argument constructor.
1518          </para>
1519        </listitem>
1520       
1521        <listitem>
1522          <para>
1523          The <methodname>Plugin.init()</methodname> method is called. The
1524          <varname>job</varname> parameter is null.
1525          </para>
1526        </listitem>
1527       
1528        <listitem id="plugin_developer.api.callsequence.configure.requestinformation">
1529          <para>
1530          The <methodname>InteractivePlugin.getRequestInformation()</methodname> 
1531          method is called. The <varname>context</varname> parameter is <constant>null</constant>
1532          and the <varname>command</varname> is the value of the string
1533          constant <constant>Request.COMMAND_CONFIGURE_PLUGIN</constant> 
1534          (<constant>_config_plugin</constant>).
1535          </para>
1536        </listitem>
1537       
1538        <listitem id="plugin_developer.api.callsequence.configure.form">
1539          <para>
1540          The web client process the returned information and displays a form for user
1541          input. The plug-in will have to wait some time while the user enters
1542          data.
1543          </para>
1544        </listitem>
1545       
1546        <listitem>
1547          <para>
1548          The <methodname>InteractivePlugin.configure()</methodname> method
1549          is called. The <varname>context</varname> parameter is still
1550          <constant>null</constant> and the <varname>request</varname>
1551          parameter contains the parameter values entered by the user.
1552          </para>
1553        </listitem>
1554       
1555        <listitem>
1556          <para>
1557          The plug-in must validate the values and decide whether they should be
1558          stored in the database or not. We recommend that you use the
1559          methods in the <classname docapi="net.sf.basedb.core.plugin">AbstractPlugin</classname> class for this.
1560          </para>
1561        </listitem>
1562       
1563        <listitem>
1564          <para>
1565          The plug-in can choose between three different respones:
1566         
1567          <itemizedlist>
1568          <listitem>
1569            <para>
1570            <methodname>Response.setDone()</methodname>: The configuration
1571            is complete. The core will write any configuation changes to the
1572            database, call the <methodname>Plugin.done()</methodname> method and
1573            then discard the plug-in instance.
1574            </para>
1575          </listitem>
1576         
1577          <listitem>
1578            <para>
1579            <methodname>Response.setError()</methodname>: There was one or more
1580            errors. The web client will display the error messages for the user and
1581            allow the user to enter new values. The process continues with
1582            step <xref linkend="plugin_developer.api.callsequence.configure.form" />.
1583            </para>
1584          </listitem>
1585         
1586          <listitem>
1587            <para>
1588            <methodname>Response.setContinue()</methodname>: The parameters are correct
1589            but the plug-in wants more parameters. The process continues with
1590            step <xref linkend="plugin_developer.api.callsequence.configure.requestinformation" />
1591            but the <varname>command</varname> has the value that was passed to the
1592            <methodname>setContinue()</methodname> method.
1593            </para>
1594          </listitem>
1595          </itemizedlist>
1596         
1597          </para>
1598        </listitem>
1599        </orderedlist>
1600
1601      </sect3>   
1602   
1603      <sect3 id="plugin_developer.api.callsequence.context">
1604        <title>Checking if a plug-in can be used in a given context</title>
1605       
1606        <para>
1607          If the plug-in is an <interfacename docapi="net.sf.basedb.core.plugin">InteractivePlugin</interfacename>
1608          it has specified in which contexts it can be used by the
1609          information returned from <methodname>InteractivePlugin.getGuiContexts()</methodname>
1610          method. The web client uses this information to decide whether,
1611          for example, a <guibutton>Run plugin</guibutton>
1612          button should be displayed on a page or not. However, this is not
1613          always enough to know whether the plug-in can be used or not.
1614          For example, a raw data importer plug-in cannot be used to
1615          import raw data if the raw bioassay already has data.
1616          So, when the user clicks the button, the web client will
1617          load all plug-ins that possibly can be used in the given context
1618          and let each one of them check whether they can be used or not.
1619        </para>
1620       
1621     
1622        <orderedlist>
1623        <listitem>
1624          <para>
1625          A new instance of the plug-in class is created. The plug-in must
1626          have a public no-argument constructor.
1627          </para>
1628        </listitem>
1629       
1630        <listitem>
1631          <para>
1632          The <methodname>Plugin.init()</methodname> method is called.
1633          The <varname>job</varname> parameter is <constant>null</constant>.
1634          The <varname>configuration</varname> parameter is <constant>null</constant>
1635          if the plug-in does not have any configuration parameters.
1636          </para>
1637        </listitem>
1638       
1639        <listitem>
1640          <para>
1641          The <methodname>InteractivePlugin.isInContext()</methodname>
1642          is called. If the context is a list context, the <varname>item</varname>
1643          parameter is null, otherwise the current item is passed. The plug-in
1644          should return <constant>null</constant> if it can be used under the
1645          current circumstances, or a message explaining why not.
1646          </para>
1647        </listitem>
1648       
1649        <listitem>
1650          <para>
1651            After this, <methodname>Plugin.done()</methodname> is called and
1652            the plug-in instance is discarded. If there are
1653            several configurations for a plug-in, this procedure is repeated
1654            for each configuration.
1655          </para>
1656        </listitem>
1657        </orderedlist>
1658      </sect3>
1659   
1660      <sect3 id="plugin_developer.api.callsequence.job">
1661        <title>Creating a new job</title>
1662       
1663        <para>
1664          If the web client found that the plug-in could be
1665          used in a given context and the user selected the plug-in,
1666          the job configuration sequence is started. It is a wizard-like interface
1667          identical to the configuration wizard. In fact, the same JSP pages,
1668          and calling sequence is used. See <xref linkend="plugin_developer.api.callsequence.configure" />.
1669          We do not repeat everything here. There are a few differences:
1670        </para>
1671       
1672        <itemizedlist>
1673        <listitem>
1674          <para>
1675          The <varname>job</varname> parameter is not null, but it does not contain
1676          any parameter values to start with. The plug-in should use this
1677          object to store job-related parameter values. The
1678          <varname>configuration</varname> parameter is <constant>null</constant> 
1679          if the plug-in is started without configuration. In any case,
1680          the configuration values are write-protected and cannot be modified.
1681          </para>
1682        </listitem>
1683       
1684        <listitem>
1685          <para>
1686          The first call to <methodname>InteractivePlugin.getRequestInformation()</methodname>
1687          is done with <constant>Request.COMMAND_CONFIGURE_JOB</constant> (<constant>_configjob</constant>)
1688          as the command. The <varname>context</varname> parameter reflects the
1689          current context.
1690          </para>
1691        </listitem>
1692       
1693        <listitem>
1694          <para>
1695          When calling <methodname>Response.setDone()</methodname> the plug-in
1696          should use the variant that takes an estimated execution time.
1697          If the plug-in has support for immediate execution or download
1698          (export plug-ins only), it can also respond with
1699          <methodname>Response.setExecuteImmediately()</methodname> or
1700          <methodname>Response.setDownloadImmediately()</methodname>.
1701          </para>
1702         
1703          <para>
1704          If the plug-in requested and was granted immediate execution or
1705          download the same plug-in instance is used to execute the plug-in.
1706          This may be done with the same or a new thread. Otherwise, a new
1707          job is added to the job queue, the parameter value are saved
1708          and the plug-in instance is discarded after calling the
1709          <methodname>Plugin.done()</methodname> method.
1710          </para>
1711        </listitem>
1712        </itemizedlist>
1713      </sect3>
1714   
1715      <sect3 id="plugin_developer.api.callsequence.execute">
1716        <title>Executing a job</title>
1717       
1718        <para>
1719          Normally, the creation of a job and the execution of it are
1720          two different events. The execution may as well be done on a
1721          different server. See <xref linkend="installation_upgrade.jobagents" />.
1722          This means that the execution takes place in a different instance
1723          of the plug-in class than what was used for creating the job.
1724          The exception is if a plug-in supports immediate execution or download.
1725          In this case the same instance is used, and it is, of course,
1726          always executed on the web server.
1727        </para>
1728       
1729        <orderedlist>
1730        <listitem>
1731          <para>
1732          A new instance of the plug-in class is created. The plug-in must
1733          have a public no-argument constructor.
1734          </para>
1735        </listitem>
1736       
1737        <listitem>
1738          <para>
1739          The <methodname>Plugin.init()</methodname> method is called.
1740          The <varname>job</varname> parameter contains the job
1741          configuration parameters. The <varname>configuration</varname> parameter
1742          is <constant>null</constant> if the plug-in does not have any
1743          configuration parameters.
1744          </para>
1745        </listitem>
1746       
1747        <listitem>
1748          <para>
1749          The <methodname>Plugin.run()</methodname> method is called.
1750          It is finally time for the plug-in to do the work it has been
1751          designed for. This method should not throw any exceptions.
1752          Use the <methodname>Response.setDone()</methodname>
1753          method to report success or the <methodname>Response.setError()</methodname>
1754          to report errors.
1755          </para>
1756        </listitem>
1757       
1758        <listitem>
1759          <para>
1760          In both cases the <methodname>Plugin.done()</methodname>
1761          method is called and the plug-in instance is discarded.
1762          </para>
1763        </listitem>
1764       
1765        </orderedlist>
1766      </sect3>
1767   
1768    </sect2>
1769 
1770    <sect2 id="plugin_developer.api.jspparameters">
1771      <title>Using custom JSP pages for parameter input</title>
1772
1773        <para>
1774          This is an advanced option for plug-ins that require a different interface for
1775          specifying plug-in parameters than the default list showing one parameter at a
1776          time. This feature is used by setting the
1777          <methodname>RequestInformation.getJspPage()</methodname>
1778          property when constructing the request information object. If this property has
1779          a non-null value, the web client will send the browser to the specified JSP page
1780          instead of to the generic parameter input page.
1781        </para>
1782        <para>
1783          When setting the JSP page you can either specify an absolute path
1784          of only the filename of the JSP file. If only the filename is specified,
1785          the JSP file is expected to be located in a special location, generated
1786          from the package name of your plug-in. If the plug-in is located in the package
1787          <classname>org.company</classname> the JSP file must be located in
1788          <filename class="directory">&lt;base-dir&gt;/www/plugins/org/company/</filename>.
1789        </para>
1790       
1791        <para>
1792          An absolute path starts with '/' and may or may not include the
1793          root directory of the BASE installation. If, for example, BASE is intalled to
1794          <userinput>http://your.base.server.com/base</userinput>, the following absolute
1795          paths are equivalent  <filename>/base/path/to/file.jsp</filename>,
1796          <filename>/path/to/file.jsp</filename>.
1797        </para>
1798        <para>
1799          In both cases, please note that the browser still thinks that it is showing
1800          the regular parameter input page at the usual location:
1801          <filename class="directory">&lt;base-dir&gt;/www/common/plugin/index.jsp</filename>.
1802          All links in your JSP page should be relative to that directory.
1803        </para>
1804        <para>
1805          Even if you use your own JSP page we recommend that you use the built-in
1806          facility for passing the parameters back to the plug-in. For this to work you
1807          must:
1808        </para>
1809        <itemizedlist spacing="compact">
1810          <listitem>
1811            <simpara>
1812            Generate the list of <classname docapi="net.sf.basedb.core">PluginParameter</classname> 
1813            objects as usual.
1814            </simpara>
1815          </listitem>
1816          <listitem>
1817            <simpara>
1818              Name all your input fields in the JSP like:
1819              <parameter>
1820                parameter:<replaceable>name-of-parameter</replaceable>
1821              </parameter>
1822            </simpara>
1823<programlisting language="java:nogutter">
1824// Plug-in generate PluginParameter
1825StringParameterType stringPT = new StringParameterType(255, null, true);
1826PluginParameter one = new PluginParameter("one", "One", "First string", stringPT);
1827PluginParameter two = new PluginParameter("two", "Two", "Second string", stringPT);
1828
1829// JSP should name fields as:
1830First string: &lt;input type="text" name="parameter:one"&gt;&lt;br&gt;
1831Second string: &lt;input type="text" name="parameter:two"&gt;
1832</programlisting>
1833          </listitem>
1834          <listitem>
1835          <simpara>
1836            Send the form to
1837            <filename>index.jsp</filename>
1838            with the <varname>ID</varname>,
1839            <varname>cmd</varname> and <varname>requestId</varname> 
1840            parameters as shown below.
1841          </simpara>
1842<programlisting language="xml">
1843&lt;form action="index.jsp" method="post"&gt;
1844&lt;input type="hidden" name="ID" value="&lt;%=ID%&gt;"&gt;
1845&lt;input type="hidden" name="requestId" value="&lt;%=request.getParameter("requestId")%&gt;"&gt;
1846&lt;input type="hidden" name="cmd" value="SetParameters"&gt;
1847...
1848&lt;/form&gt;
1849</programlisting>
1850          <simpara>
1851            The <varname>ID</varname> is the session ID for the logged
1852            in user and is required. The <varname>requestId</varname>
1853            is the ID for this particular plug-in/job configuration
1854            sequence. It is optional, but we recommend that you use it
1855            since it protects your plug-in from getting mixed up with
1856            other plug-in configuration wizards. The <varname>cmd</varname>
1857            tells BASE to send the parameters to the plug-in for validation
1858            and saving.
1859          </simpara>
1860
1861          <simpara>
1862            Values are sent as strings to BASE that converts them to the
1863            proper value type before they are passed on to your plug-in.
1864            However, there is one case that can't be
1865            accurately represented with custom JSP pages, namely 'null' values.
1866            A null value is sent by not sending any value at all. This is not
1867            possible with a fixed form. It is of course possible to add some custom
1868            JavaScript that adds and removes form elements as needed, but it is
1869            also possible to let the empty string represent null. Just include a
1870            hidden parameter like this if you want an empty value for the 'one'
1871            parameter converted to null:
1872          </simpara>
1873<programlisting language="xml">
1874&lt;input type="hidden" name="parameter:one:emptyIsNull" value="1"&gt;
1875</programlisting>
1876          </listitem>
1877          </itemizedlist>
1878         
1879          <para>
1880            If you want a &gbCancel; button to abort the configuration
1881            you should reload the page with with the url:
1882            <uri>index.jsp?ID=&lt;%=ID%&gt;&amp;cmd=CancelWizard</uri>. This
1883            allows BASE to clean up resources that has been put in global
1884            session variables.
1885          </para>
1886         
1887            <para>
1888              In your JSP page you will probably need to access some information like the
1889              <classname docapi="net.sf.basedb.core">SessionControl</classname>, <classname docapi="net.sf.basedb.core">Job</classname>
1890              and possible even the <classname docapi="net.sf.basedb.core">RequestInformation</classname>
1891              object created by your plug-in.
1892            </para>
1893<programlisting language="java">
1894// Get session control and its ID (required to post to index.jsp)
1895final SessionControl sc = Base.getExistingSessionControl(pageContext, true);
1896final String ID = sc.getId();
1897
1898// Get information about the current request to the plug-in
1899PluginConfigurationRequest pcRequest =
1900   (PluginConfigurationRequest)sc.getSessionSetting("plugin.configure.request");
1901PluginDefinition plugin =
1902   (PluginDefinition)sc.getSessionSetting("plugin.configure.plugin");
1903PluginConfiguration pluginConfig =
1904   (PluginConfiguration)sc.getSessionSetting("plugin.configure.config");
1905PluginDefinition job =
1906   (PluginDefinition)sc.getSessionSetting("plugin.configure.job");
1907RequestInformation ri = pcRequest.getRequestInformation();
1908</programlisting>
1909
1910    </sect2>
1911  </sect1>
1912
1913  <sect1 id="plugin_developer.import">
1914    <title>Import plug-ins</title>
1915
1916    <para>
1917      A plugin becomes an import plugin simply by returning
1918      <constant>Plugin.MainType.IMPORT</constant>
1919      from the <methodname>Plugin.getMainType()</methodname> method.
1920    </para>
1921   
1922    <sect2 id="plugin_developer.import.autodetect">
1923      <title>Autodetect file formats</title>
1924      <para>
1925        BASE has built-in functionality for autodetecting file formats.
1926        Your plug-in can be part of that feature if it reads it data
1927        from a single file. It must also implement the
1928        <interfacename docapi="net.sf.basedb.core.plugin">AutoDetectingImporter</interfacename>
1929        interface.
1930      </para>
1931
1932      <sect3 id="plugin_developer.api.interfaces.autodetecting">
1933        <title>The net.sf.basedb.core.plugin.AutoDetectingImporter interface</title>
1934
1935        <variablelist>
1936        <varlistentry>
1937          <term>
1938            <methodsynopsis language="java">
1939              <modifier>public</modifier>
1940              <type>boolean</type>
1941              <methodname>isImportable</methodname>
1942              <methodparam>
1943                <type>InputStream</type>
1944                <parameter>in</parameter>
1945              </methodparam>
1946              <exceptionname>BaseException</exceptionname>
1947            </methodsynopsis>
1948          </term>
1949          <listitem>
1950            <para>
1951              Check the input stream if it seems to contain data that can be imported by
1952              the plugin. Usually it means scanning a few lines for some header
1953              mathing a predefined string or a regexp.
1954            </para>
1955            <para>
1956              The <classname docapi="net.sf.basedb.plugins">AbstractFlatFileImporter</classname> implements this method
1957              by reading the headers from the input stream and checking if
1958              it stopped at an unknown type of line or not:
1959                <programlisting language="java">
1960public final boolean isImportable(InputStream in)
1961   throws BaseException
1962{
1963   FlatFileParser ffp = getInitializedFlatFileParser();
1964   ffp.setInputStream(in);
1965   try
1966   {
1967      ffp.nextSection();
1968      FlatFileParser.LineType result = ffp.parseHeaders();
1969      if (result == FlatFileParser.LineType.UNKNOWN)
1970      {
1971         return false;
1972      }
1973      else
1974      {
1975         return isImportable(ffp);
1976      }
1977   }
1978   catch (IOException ex)
1979   {
1980      throw new BaseException(ex);
1981   }
1982}
1983</programlisting>
1984            </para>
1985            <para>
1986              Note that the input stream doesn't have to be a text file.
1987              It can be any type of file, for example a binary or an XML file.
1988              In the case of an XML file you would need to validate the entire
1989              input stream in order to be a 100% sure that it is a valid
1990              xml file, but we recommend that you only check the first few XML tags,
1991              for example, the &lt;!DOCTYPE &gt; declaration and/or the root element
1992              tag.
1993            </para>
1994          </listitem>
1995        </varlistentry>
1996        <varlistentry>
1997          <term>
1998            <methodsynopsis language="java">
1999              <modifier>public</modifier>
2000              <void/>
2001              <methodname>doImport</methodname>
2002              <methodparam>
2003                <type>InputStream</type>
2004                <parameter>in</parameter>
2005              </methodparam>
2006              <methodparam>
2007                <type>ProgressReporter</type>
2008                <parameter>progress</parameter>
2009              </methodparam>
2010              <exceptionname>BaseException</exceptionname>
2011            </methodsynopsis>
2012          </term>
2013          <listitem>
2014            <para>
2015              Parse the input stream and import all data that is found.
2016              This method is of course only called if the
2017              <methodname>isImportable()</methodname> has returned true. Note
2018              however that the input stream is reopened at the start of the
2019              file. It may even be the case that the <methodname>isImportable()</methodname>
2020              method is called on one instance of the plugin and the
2021              <methodname>doImport()</methodname> method is called on another.
2022              Thus, the <methodname>doImport()</methodname> can't rely on any state set
2023              by the <methodname>isImportable()</methodname> method.
2024            </para>
2025          </listitem>
2026        </varlistentry>
2027        </variablelist>
2028       
2029        <tip>
2030          <title>Try casting to ImportInputStream</title>
2031          <para>
2032            As of BASE 2.9 the auto-detect functionality uses a
2033            <classname docapi="net.sf.basedb.core.plugin">ImportInputStream</classname>
2034            as the <varname>in</varname> parameter. This class contains some
2035            metadata about the file the input stream is originating from. The most
2036            useful feature is the possibility to get information about the
2037            character set used in the file. This makes it possible to open text
2038            files using the correct character set.
2039          </para>
2040          <programlisting language="java">
2041String charset = Config.getCharset(); // Default value
2042if (in instanceof ImportInputStream)
2043{
2044   ImportInputStream iim = (ImportInputStream)in;
2045   if (iim.getCharacterSet() != null) charset = iim.getCharacterSet();
2046}
2047Reader reader = new InputStreamReader(in, Charset.forName(charset)));
2048</programlisting>
2049        </tip>       
2050      </sect3>
2051     
2052      <sect3 id="plugin_developer.import.autodetect.callsequence">
2053        <title>Call sequence during autodetection</title>
2054       
2055        <para>
2056          The call sequence for autodetection resembles the call sequence for
2057          checking if the plug-in can be used in a given context.
2058        </para>
2059
2060        <orderedlist>
2061        <listitem>
2062          <para>
2063          A new instance of the plug-in class is created. The plug-in must
2064          have a public no-argument constructor.
2065          </para>
2066        </listitem>
2067       
2068        <listitem>
2069          <para>
2070          The <methodname>Plugin.init()</methodname> method is called.
2071          The <varname>job</varname> parameter is <constant>null</constant>.
2072          The <varname>configuration</varname> parameter is <constant>null</constant>
2073          if the plug-in does not have any configuration parameters.
2074          </para>
2075        </listitem>
2076       
2077        <listitem>
2078          <para>
2079          If the plug-in is interactive the the <methodname>InteractivePlugin.isInContext()</methodname>
2080          is called. If the context is a list context, the <varname>item</varname>
2081          parameter is null, otherwise the current item is passed. The plug-in
2082          should return <constant>null</constant> if it can be used under the
2083          current circumstances, or a message explaining why not.
2084          </para>
2085        </listitem>
2086       
2087        <listitem>
2088          <para>
2089          If the plug-in can be used the <methodname>AutoDetectingImporter.isImportable()</methodname>
2090          method is called to check if the selected file is importable or not.
2091          </para>
2092        </listitem>
2093       
2094        <listitem>
2095          <para>
2096          After this, <methodname>Plugin.done()</methodname> is called and
2097          the plug-in instance is discarded. If there are
2098          several configurations for a plug-in, this procedure is repeated
2099          for each configuration. If the plug-in can be used without
2100          a configuration the procedure is also repeated without
2101          configuration parameters.
2102          </para>
2103        </listitem>
2104       
2105        <listitem>
2106          <para>
2107          If a single plug-in was found the user is taken to the regular
2108          job configuration wizard. A new plug-in instance is created for
2109          this. If more than one plug-in was found the user is presented
2110          with a list of the plug-ins. After selecting one of them the
2111          regular job configuration wizard is used with a new plug-in instance.
2112          </para>
2113        </listitem>
2114       
2115        </orderedlist>
2116       
2117      </sect3>
2118
2119    </sect2>
2120   
2121    <sect2 id="plugin_developer.import.abstractflatfileimporter">
2122      <title>The AbstractFlatFileImporter superclass</title>
2123      <para>
2124        The <classname docapi="net.sf.basedb.plugins">AbstractFlatFileImporter</classname> is a very useful abstract
2125        class to use as a superclass for your own import plug-ins. It can be used
2126        if your plug-in uses regular text files that can be parsed by an instance of the
2127        <classname docapi="net.sf.basedb.util">net.sf.basedb.util.FlatFileParser</classname> class. This class parses a file
2128        by checking each line against a few regular expressions. Depending on which regular
2129        expression matches the line, it is classified as a header line, a section line,
2130        a comment, a data line, a footer line or unknown. Header lines are inspected as a group,
2131        but data lines individually, meaning that it consumes very little memory since only
2132        a few lines at a time needs to be loaded.
2133      </para>
2134     
2135      <para>
2136        The <classname docapi="net.sf.basedb.plugins">AbstractFlatFileImporter</classname> defines
2137        <classname docapi="net.sf.basedb.core">PluginParameter</classname> objects
2138        for each of the regular expressions and other parameters used by the parser. It also
2139        implements the <methodname>Plugin.run()</methodname> method and does most of
2140        the ground work for instantiating a <methodname>FlatFileParser</methodname> and
2141        parsing the file. What you have to do in your plugin is to put together the
2142        <classname docapi="net.sf.basedb.core">RequestInformation</classname> objects
2143        for configuring the plugin and creating a job and implement the
2144        <methodname>InteractivePlugin.configure()</methodname> method for validating and
2145        storing the parameters. You should also implement or override some methods
2146        defined by <classname>AbstractFlatFileImporter</classname>.
2147      </para>
2148
2149      <para>
2150      Here is what you need to do:
2151      </para>
2152
2153      <itemizedlist>
2154      <listitem>
2155        <para>
2156        Implement the <methodname>Plugin.getAbout()</methodname> method. See
2157        <xref linkend="plugin_developer.api.interfaces.plugin" /> for more information.
2158        </para>
2159      </listitem>
2160     
2161      <listitem>
2162        <para>
2163        Implement the <interfacename docapi="net.sf.basedb.core.plugin">InteractivePlugin</interfacename> methods.
2164        See <xref linkend="plugin_developer.api.interfaces.interactive" /> for more information. Note that the
2165        <classname docapi="net.sf.basedb.plugins">AbstractFlatFileImporter</classname>
2166        has defined many parameters for regular expressions used by the parser
2167        already. You should just pick them and put in your <classname docapi="net.sf.basedb.core">RequestInformation</classname>
2168        object.
2169        </para>
2170       
2171        <programlisting language="java">
2172// Parameter that maps the items name from a column
2173private PluginParameter&lt;String&gt; nameColumnMapping;
2174
2175// Parameter that maps the items description from a column
2176private PluginParameter&lt;String&gt; descriptionColumnMapping;
2177
2178private RequestInformation getConfigurePluginParameters(GuiContext context)
2179{
2180   if (configurePlugin == null)
2181   {
2182      // To store parameters for CONFIGURE_PLUGIN
2183      List&lt;PluginParameter&lt;?&gt;&gt; parameters =
2184         new ArrayList&lt;PluginParameter&lt;?&gt;&gt;();
2185
2186      // Parser regular expressions - from AbstractFlatFileParser
2187      parameters.add(parserSection);
2188      parameters.add(headerRegexpParameter);
2189      parameters.add(dataHeaderRegexpParameter);
2190      parameters.add(dataSplitterRegexpParameter);
2191      parameters.add(ignoreRegexpParameter);
2192      parameters.add(dataFooterRegexpParameter);
2193      parameters.add(minDataColumnsParameter);
2194      parameters.add(maxDataColumnsParameter);
2195
2196      // Column mappings
2197      nameColumnMapping = new PluginParameter&lt;String&gt;(
2198         "nameColumnMapping",
2199         "Name",
2200         "Mapping that picks the items name from the data columns",
2201         new StringParameterType(255, null, true)
2202      );
2203   
2204      descriptionColumnMapping = new PluginParameter&lt;String&gt;(
2205        "descriptionColumnMapping",
2206        "Description",
2207        "Mapping that picks the items description from the data columns",
2208        new StringParameterType(255, null, false)
2209      );
2210
2211      parameters.add(mappingSection);
2212      parameters.add(nameColumnMapping);
2213      parameters.add(descriptionColumnMapping);
2214     
2215      configurePlugin = new RequestInformation
2216      (
2217         Request.COMMAND_CONFIGURE_PLUGIN,
2218         "File parser settings",
2219         "",
2220         parameters
2221      );
2222
2223   }
2224   return configurePlugin;
2225}
2226</programlisting>
2227      </listitem>
2228     
2229      <listitem>
2230        <para>
2231        Implement/override some of the methods defined by
2232        <classname>AbstractFlatFileParser</classname>. The most important
2233        methods are listed below.
2234        </para>
2235      </listitem>
2236     
2237      </itemizedlist>
2238
2239      <variablelist>
2240      <varlistentry>
2241        <term>
2242          <methodsynopsis language="java">
2243            <modifier>protected</modifier>
2244            <type>FlatFileParser</type>
2245            <methodname>getInitializedFlatFileParser</methodname>
2246            <exceptionname>BaseException</exceptionname>
2247          </methodsynopsis>
2248        </term>
2249        <listitem>
2250          <para>
2251          The method is called to create a <classname docapi="net.sf.basedb.util.parser">FlatFileParser</classname>
2252          and set the regular expressions that should be used for parsing the file.
2253          The default implementation assumes that your plug-in has used the built-in
2254          <classname docapi="net.sf.basedb.core">PluginParameter</classname> objects and has stored the values
2255          at the configuration level. You should override this method if you need to
2256          initialise the parser in a different way. See for example the
2257          code for the <classname docapi="net.sf.basedb.plugins">PrintMapFlatFileImporter</classname> plug-in which
2258          has a fixed format and doesn't use configurations.
2259          </para>
2260          <programlisting language="java">
2261@Override
2262protected FlatFileParser getInitializedFlatFileParser()
2263   throws BaseException
2264{
2265   FlatFileParser ffp = new FlatFileParser();
2266   ffp.setSectionRegexp(Pattern.compile("\\[(.+)\\]"));
2267   ffp.setHeaderRegexp(Pattern.compile("(.+)=,(.*)"));
2268   ffp.setDataSplitterRegexp(Pattern.compile(","));
2269   ffp.setDataFooterRegexp(Pattern.compile(""));
2270   ffp.setMinDataColumns(12);
2271   return ffp;
2272}
2273</programlisting>
2274        </listitem>
2275      </varlistentry>
2276     
2277      <varlistentry>
2278        <term>
2279          <methodsynopsis language="java">
2280            <modifier>protected</modifier>
2281            <type>boolean</type>
2282            <methodname>isImportable</methodname>
2283            <methodparam>
2284              <type>FlatFileParser</type>
2285              <parameter>ffp</parameter>
2286            </methodparam>
2287            <exceptionname>IOException</exceptionname>
2288          </methodsynopsis>
2289        </term>
2290        <listitem>
2291          <para>
2292          This method is called from the <methodname>isImportable(InputStream)</methodname>
2293          method, AFTER <methodname>FlatFileParser.nextSection()</methodname> and
2294          <methodname>FlatFileParser.parseHeaders()</methodname> has been called
2295          a single time and if the <methodname>parseHeaders</methodname> method didn't
2296          stop on an unknown line. The default implementation of this method always returns
2297          TRUE, since obviously some data has been found. A subclass may override this method
2298          if it wants to do more checks, for example, make that a certain header is present
2299          with a certain value. It may also continue parsing the file. Here is a code example from
2300          the <classname docapi="net.sf.basedb.plugins">PrintMapFlatFileImporter</classname> which checks if a
2301          <constant>FormatName</constant> header is present and contains either
2302          <constant>TAM</constant> or <constant>MwBr</constant>.
2303          </para>
2304         
2305          <programlisting language="java">
2306/**
2307   Check that the file is a TAM or MwBr file.
2308   @return TRUE if a FormatName header is present and contains "TAM" or "MwBr", FALSE
2309      otherwise
2310*/
2311@Override
2312protected boolean isImportable(FlatFileParser ffp)
2313{
2314   String formatName = ffp.getHeader("FormatName");
2315   return formatName != null &amp;&amp; 
2316      (formatName.contains("TAM") || formatName.contains("MwBr"));
2317}
2318</programlisting>
2319        </listitem>
2320      </varlistentry>
2321     
2322      <varlistentry>
2323        <term>
2324          <methodsynopsis language="java">
2325            <modifier>protected</modifier>
2326            <void/>
2327            <methodname>begin</methodname>
2328            <methodparam>
2329              <type>FlatFileParser</type>
2330              <parameter>ffp</parameter>
2331            </methodparam>
2332            <exceptionname>BaseException</exceptionname>
2333          </methodsynopsis>
2334        </term>
2335        <listitem>
2336          <para>
2337          This method is called just before the parsing of the file
2338          begins. Override this method if you need to initialise some
2339          internal state. This is, for example, a good place to open
2340          a <classname docapi="net.sf.basedb.core">DbControl</classname> object, read parameters from the
2341          job and configuration and put them into more useful variables. The default
2342          implementation does nothing, but we recommend that
2343          <methodname>super.begin()</methodname> is always called.
2344          </para>
2345          <programlisting language="java">
2346// Snippets from the RawDataFlatFileImporter class
2347private DbControl dc;
2348private RawDataBatcher batcher;
2349private RawBioAssay rawBioAssay;
2350private Map&lt;String, String&gt; columnMappings;
2351private int numInserted;
2352
2353@Override
2354protected void begin()
2355   throws BaseException
2356{
2357   super.begin();
2358
2359   // Get DbControl
2360   dc = sc.newDbControl();
2361   rawBioAssay = (RawBioAssay)job.getValue(rawBioAssayParameter.getName());
2362
2363   // Reload raw bioassay using current DbControl
2364   rawBioAssay = RawBioAssay.getById(dc, rawBioAssay.getId());
2365   
2366   // Create a batcher for inserting spots
2367   batcher = rawBioAssay.getRawDataBatcher();
2368
2369   // For progress reporting
2370   numInserted = 0;
2371}         
2372</programlisting>
2373        </listitem>
2374      </varlistentry>
2375      <varlistentry>
2376        <term>
2377          <methodsynopsis language="java">
2378            <modifier>protected</modifier>
2379            <void/>
2380            <methodname>handleHeader</methodname>
2381            <methodparam>
2382              <type>FlatFileParser.Line</type>
2383              <parameter>line</parameter>
2384            </methodparam>
2385            <exceptionname>BaseException</exceptionname>
2386          </methodsynopsis>
2387        </term>
2388        <listitem>
2389          <para>
2390          This method is called once for every header line that is found in
2391          the file. The <varname>line</varname> parameter contains information
2392          about the header. The default implementation of this method does
2393          nothing.
2394          </para>
2395          <programlisting language="java">
2396@Override
2397protected void handleHeader(Line line)
2398   throws BaseException
2399{
2400   super.handleHeader(line);
2401   if (line.name() != null &amp;&amp; line.value() != null)
2402   {
2403      rawBioAssay.setHeader(line.name(), line.value());
2404   }
2405}
2406</programlisting>
2407        </listitem>
2408      </varlistentry>
2409      <varlistentry>
2410        <term>
2411          <methodsynopsis language="java">
2412            <modifier>protected</modifier>
2413            <void/>
2414            <methodname>handleSection</methodname>
2415            <methodparam>
2416              <type>FlatFileParser.Line</type>
2417              <parameter>line</parameter>
2418            </methodparam>
2419            <exceptionname>BaseException</exceptionname>
2420          </methodsynopsis>
2421        </term>
2422        <listitem>
2423          <para>
2424            This method is called once for each section that is found in the file.
2425            The <varname>line</varname> parameter contains information
2426            about the section. The default implementation of this method does
2427            nothing.
2428          </para>
2429        </listitem>
2430      </varlistentry>
2431     
2432      <varlistentry>
2433        <term>
2434          <methodsynopsis language="java">
2435            <modifier>protected abstract</modifier>
2436            <void/>
2437            <methodname>beginData</methodname>
2438            <exceptionname>BaseException</exceptionname>
2439          </methodsynopsis>
2440        </term>
2441        <listitem>
2442          <para>
2443          This method is called after the headers has been parsed, but before
2444          the first line of data. This is a good place to add code that
2445          depends on information in the headers, for example, put
2446          together column mappings.
2447          </para>
2448         
2449          <programlisting language="java">
2450private Mapper reporterMapper;
2451private Mapper blockMapper;
2452private Mapper columnMapper;
2453private Mapper rowMapper;
2454// ... more mappers
2455
2456@Override
2457protected void beginData()
2458{
2459   boolean cropStrings = ("crop".equals(job.getValue("stringTooLongError")));
2460
2461   // Mapper that always return null; used if no mapping expression has been entered
2462   Mapper nullMapper = new ConstantMapper((String)null);
2463   
2464   // Column mappers
2465   reporterMapper = getMapper(ffp, (String)configuration.getValue("reporterIdColumnMapping"),
2466      cropStrings ? ReporterData.MAX_EXTERNAL_ID_LENGTH : null, nullMapper);
2467   blockMapper = getMapper(ffp, (String)configuration.getValue("blockColumnMapping"),
2468      null, nullMapper);
2469   columnMapper = getMapper(ffp, (String)configuration.getValue("columnColumnMapping"),
2470      null, nullMapper);
2471   rowMapper = getMapper(ffp, (String)configuration.getValue("rowColumnMapping"),
2472      null, nullMapper);
2473   // ... more mappers: metaGrid coordinate, X-Y coordinate, extended properties
2474   // ...
2475}
2476</programlisting>
2477         
2478        </listitem>
2479      </varlistentry>
2480       
2481      <varlistentry>
2482        <term>
2483          <methodsynopsis language="java">
2484            <modifier>protected abstract</modifier>
2485            <void/>
2486            <methodname>handleData</methodname>
2487            <methodparam>
2488              <type>FlatFileParser.Data</type>
2489              <parameter>data</parameter>
2490            </methodparam>
2491            <exceptionname>BaseException</exceptionname>
2492          </methodsynopsis>
2493        </term>
2494        <listitem>
2495          <para>
2496          This method is abstract and must be implemented by all subclasses.
2497          It is called once for every data line in the the file.
2498          </para>
2499
2500          <programlisting language="java">
2501// Snippets from the RawDataFlatFileImporter class
2502@Override
2503protected void handleData(Data data)
2504   throws BaseException
2505{
2506   // Create new RawData object
2507   RawData raw = batcher.newRawData();
2508
2509   // External ID for the reporter
2510   String externalId = reporterMapper.getValue(data);
2511   
2512   // Block, row and column numbers
2513   raw.setBlock(blockMapper.getInt(data));
2514   raw.setColumn(columnMapper.getInt(data));
2515   raw.setRow(rowMapper.getInt(data));
2516   // ... more: metaGrid coordinate, X-Y coordinate, extended properties
2517   
2518   // Insert raw data to the database
2519   batcher.insert(raw, externalId);
2520   numInserted++;
2521}
2522</programlisting> 
2523         
2524        </listitem>
2525      </varlistentry>
2526     
2527      <varlistentry>
2528        <term>
2529          <methodsynopsis language="java">
2530            <modifier>protected</modifier>
2531            <void/>
2532            <methodname>end</methodname>
2533            <methodparam>
2534              <type>boolean</type>
2535              <parameter>success</parameter>
2536            </methodparam>
2537          </methodsynopsis>
2538        </term>
2539        <listitem>
2540          <para>
2541            Called when the parsing has ended, either because the end of
2542            file was reached or because an error has occurred. The subclass
2543            should close any open resources, ie. the <classname docapi="net.sf.basedb.core">DbControl</classname>
2544            object. The <varname>success</varname> parameter is <constant>true</constant>
2545            if the parsing was successful, <constant>false</constant> otherwise.
2546            The default implementation does nothing.
2547          </para>
2548         
2549          <programlisting language="java">
2550@Override
2551protected void end(boolean success)
2552   throws BaseException
2553{
2554   try
2555   {
2556      // Commit if the parsing was successful
2557      if (success)
2558      {
2559         batcher.close();
2560         dc.commit();
2561      }
2562   }
2563   catch (BaseException ex)
2564   {
2565      // Well, now we got an exception
2566      success = false;
2567      throw ex;
2568   }
2569   finally
2570   {
2571      // Always close... and call super.end()
2572      if (dc != null) dc.close();
2573      super.end(success);
2574   }
2575}     
2576</programlisting>         
2577        </listitem>
2578      </varlistentry>
2579     
2580      <varlistentry>
2581        <term>
2582          <methodsynopsis language="java">
2583            <modifier>protected</modifier>
2584            <type>String</type>
2585            <methodname>getSuccessMessage</methodname>
2586            <void/>
2587          </methodsynopsis>
2588        </term>
2589        <listitem>
2590          <para>
2591          This is the last method that is called, and it is only called if
2592          everything went suceessfully. This method allows a subclass to generate
2593          a short message that is sent back to the database as a final progress
2594          report. The default implementation returns null, which means that no
2595          message will be generated.
2596          </para>
2597          <programlisting language="java">
2598@Override
2599protected String getSuccessMessage()
2600{
2601   return numInserted + " spots inserted";
2602}
2603</programlisting>
2604        </listitem>
2605      </varlistentry>
2606      </variablelist>
2607
2608      <para>
2609        The <classname docapi="net.sf.basedb.plugins">AbstractFlatFileImporter</classname> has a lot of
2610        other methods that you may use and/or override in your own plug-in.
2611        Check the javadoc for more information.
2612      </para>
2613
2614    </sect2>
2615  </sect1>
2616
2617  <sect1 id="plugin_developer.export">
2618    <title>Export plug-ins</title>
2619   
2620    <para>
2621      Export plug-ins are plug-ins that takes data from BASE, and
2622      prepares it for use with some external entity. Usually this
2623      means that data is taken from the database and put into a file
2624      with some well-defined file format.
2625      An export plug-in should return <constant>MainType.EXPORT</constant> from
2626      the <methodname>Plugin.getMainType()</methodname> method.
2627    </para>
2628   
2629    <sect2 id="plugin_developer.export.download">
2630      <title>Immediate download of exported data</title>
2631      <para>
2632        An export plug-in may want to give the user a choice
2633        between saving the exported data in the BASE file system
2634        or to download it immediately to the client computer. With the basic
2635        plug-in API the second option is not possible. The
2636        <interfacename docapi="net.sf.basedb.core.plugin">ImmediateDownloadExporter</interfacename> is an
2637        interface that extends the <interfacename docapi="net.sf.basedb.core.plugin">Plugin</interfacename>
2638        interface to provide this functionality. If your export
2639        plug-in wants to provide immediate download functionality it must
2640        implement the <interfacename docapi="net.sf.basedb.core.plugin">ImmediateDownloadExporter</interfacename>
2641        interface.
2642      </para>
2643     
2644      <sect3 id="plugin_developer.export.immediatedownloadexporter">
2645      <title>The ImmediateDownloadExporter interface</title>
2646      <variablelist>
2647      <varlistentry>
2648        <term>
2649          <methodsynopsis language="java">
2650            <modifier>public</modifier>
2651            <void/>
2652            <methodname>doExport</methodname>
2653            <methodparam>
2654              <type>ExportOutputStream</type>
2655              <parameter>out</parameter>
2656            </methodparam>
2657            <methodparam>
2658              <type>ProgressReporter</type>
2659              <parameter>progress</parameter>
2660            </methodparam>
2661          </methodsynopsis>
2662        </term>
2663        <listitem>
2664          <para>
2665          Perform the export. The plug-in should write the
2666          exported data to the <varname>out</varname> stream.
2667          If the <varname>progress</varname> parameter is not null,
2668          the progress should be reported at regular interval in the
2669          same manner as in the <methodname>Plugin.run()</methodname>
2670          method.
2671          </para>
2672        </listitem>
2673      </varlistentry>
2674      </variablelist>
2675     
2676      </sect3>
2677     
2678      <sect3 id="plugin_developer.export.exportoutputstream">
2679      <title>The ExportOutputStream class</title>
2680     
2681      <para>
2682        The <classname docapi="net.sf.basedb.core.plugin">ExportOutputStream</classname> is
2683        an extension to the
2684        <classname>java.io.OutputStream</classname>. Use the regular
2685        <methodname>write()</methodname> methods to write data to it.
2686        It also has some additional methods, which are used for setting
2687        metadata about the generated file. These methods are useful, for
2688        example, when generating HTTP response headers.
2689      </para>
2690     
2691      <note>
2692        <para>
2693        These methods must be called before starting to write data to
2694        the <varname>out</varname> stream.
2695        </para>
2696      </note>
2697     
2698      <variablelist>
2699      <varlistentry>
2700        <term>
2701          <methodsynopsis language="java">
2702            <modifier>public</modifier>
2703            <void/>
2704            <methodname>setContentLength</methodname>
2705            <methodparam>
2706              <type>long</type>
2707              <parameter>contentLength</parameter>
2708            </methodparam>
2709          </methodsynopsis>
2710        </term>
2711        <listitem>
2712          <para>
2713          Set the total size of the exported data. Don't call this method if the
2714          total size is not known.
2715          </para>
2716        </listitem>
2717      </varlistentry>
2718      <varlistentry>
2719        <term>
2720          <methodsynopsis language="java">
2721            <modifier>public</modifier>
2722            <void/>
2723            <methodname>setMimeType</methodname>
2724            <methodparam>
2725              <type>String</type>
2726              <parameter>mimeType</parameter>
2727            </methodparam>
2728          </methodsynopsis>
2729        </term>
2730        <listitem>
2731          <para>
2732          Set the MIME type of the file that is being generated.
2733          </para>
2734        </listitem>
2735      </varlistentry>
2736      <varlistentry>
2737        <term>
2738          <methodsynopsis language="java">
2739            <modifier>public</modifier>
2740            <void/>
2741            <methodname>setCharacterSet</methodname>
2742            <methodparam>
2743              <type>String</type>
2744              <parameter>charset</parameter>
2745            </methodparam>
2746          </methodsynopsis>
2747        </term>
2748        <listitem>
2749          <para>
2750          Sets the character set used in text files. For example,
2751          UTF-8 or ISO-8859-1.
2752          </para>
2753        </listitem>
2754      </varlistentry>
2755      <varlistentry>
2756        <term>
2757          <methodsynopsis language="java">
2758            <modifier>public</modifier>
2759            <void/>
2760            <methodname>setFilename</methodname>
2761            <methodparam>
2762              <type>String</type>
2763              <parameter>filename</parameter>
2764            </methodparam>
2765          </methodsynopsis>
2766        </term>
2767        <listitem>
2768          <para>
2769          Set a suggested name of the file that is being
2770          generated.
2771          </para>
2772        </listitem>
2773      </varlistentry>
2774      </variablelist>
2775     
2776      </sect3>
2777     
2778      <sect3 id="plugin_developer.export.callsequence">
2779        <title>Call sequence during immediate download</title>
2780
2781      <para>
2782        Supporting immediate download also means that the method call
2783        sequence is a bit altered from the standard sequence described
2784        in <xref linkend="plugin_developer.api.callsequence.execute" />.
2785      </para>
2786     
2787      <itemizedlist>
2788      <listitem>
2789        <para>
2790        The plug-in must call <methodname>Response.setDownloadImmediately()</methodname>
2791        instead of <methodname>Response.setDone()</methodname> in <methodname>Plugin.configure()</methodname>
2792        to end the job configuration wizard. This requests that the core starts
2793        an immediate download.
2794        </para>
2795       
2796        <note>
2797          <para>
2798          Even if an immediate download is requested by the plug-in this feature
2799          may have been disabled by the server administrator. If so, the plug-in
2800          can choose if the job should be added to job queue or if this is an
2801          error condition.
2802          </para>
2803        </note>
2804      </listitem>
2805     
2806      <listitem>
2807        <para>
2808        If immediate download is granted the web client will keep the
2809        same plug-in instance and call <methodname>ImmediateDownloadExporter.doExport()</methodname>.
2810        In this case, the <methodname>Plugin.run()</methodname> is never called.
2811        After the export, <methodname>Plugin.done()</methodname> is called as
2812        usual.
2813        </para>
2814       
2815      </listitem>
2816
2817      <listitem>
2818        <para>
2819        If immediate download is not granted and the job is added to the job queue
2820        the regular job execution sequence is used.
2821        </para>
2822      </listitem>
2823      </itemizedlist>
2824     
2825      </sect3>
2826
2827    </sect2>
2828   
2829    <sect2 id="plugin_developer.export.abstractexporter">
2830      <title>The AbstractExporterPlugin class</title>
2831   
2832      <para>
2833        This is an abstract superclass that will make it easier
2834        to implement export plug-ins that support immediate
2835        download. It defines <classname docapi="net.sf.basedb.core">PluginParameter</classname>
2836        objects for asking a user about a path where the exported
2837        data should be saved and if existing files should be overwritten or not.
2838        If the user leaves the path empty the immediate download functionality
2839        should be used. It also contains implementations of both the
2840        <methodname>Plugin.run()</methodname> method and the
2841        <methodname>ImmediateDownloadExporter.doExport()</methodname> method.
2842        Here is what you need to do in your own plug-in code (code examples are
2843        taken from the <classname docapi="net.sf.basedb.plugins">HelpExporter</classname>):
2844      </para>
2845     
2846      <itemizedlist>
2847      <listitem>
2848        <para>
2849          Your plug-in should extend the <classname docapi="net.sf.basedb.core.plugin">AbstractExporterPlugin</classname>
2850          class:
2851          <programlisting language="java">
2852public class HelpExporter
2853  extends AbstractExporterPlugin
2854  implements InteractivePlugin
2855</programlisting>
2856        </para>
2857      </listitem>
2858     
2859      <listitem>
2860        <para>
2861          You need to implement the
2862          <methodname>InteractivePlugin.getRequestInformation()</methodname>
2863          method. Use the <methodname>getSaveAsParameter()</methodname>
2864          and <methodname>getOverwriteParameter()</methodname> methods defined in the
2865          superclass to create plug-in parameters that asks for the file name to save
2866          to and if existing files can be overwritten or not.
2867          You should also check if the administrator has enabled the immediate execution
2868          functionality for your plug-in. If not, the only option is to
2869          export to a file in the BASE file system and the filename is a
2870          required parameter.
2871         
2872          <programlisting language="java">
2873// Selected parts of the getRequestConfiguration() method
2874...
2875List&lt;PluginParameter&lt;?&gt;&gt; parameters =
2876   new ArrayList&lt;PluginParameter&lt;?&gt;&gt;();
2877...
2878PluginDefinition pd = job.getPluginDefinition();
2879boolean requireFile = pd == null ?
2880   false : !pd.getAllowImmediateExecution();
2881
2882parameters.add(getSaveAsParameter(null, null, defaultPath, requireFile));
2883parameters.add(getOverwriteParameter(null, null));
2884
2885configureJob = new RequestInformation
2886(
2887   Request.COMMAND_CONFIGURE_JOB,
2888   "Help exporter options",
2889   "Set Client that owns the helptexts, " +
2890     "the file path where the export file should be saved",
2891   parameters
2892);
2893....
2894return configureJob;
2895</programlisting>
2896        </para>
2897      </listitem>
2898
2899      <listitem>
2900        <para>
2901        You must also implement the <methodname>configure()</methodname>
2902        method and check the parameters. If no filename has been given,
2903        you should check if immediate exection is allowed and set an
2904        error if it is not. If a filename is present, use the
2905        <methodname>pathCanBeUsed()</methodname> method to check if
2906        it is possible to save the data to a file with that name. If the
2907        file already exists it can be overwritten if the <varname>OVERWRITE</varname>
2908        is <constant>TRUE</constant> or if the file has been flagged for removal.
2909        Do not forget to store the parameters with the <methodname>storeValue()</methodname>
2910        method.
2911
2912        <programlisting language="java">
2913// Selected parts from the configure() method
2914if (request.getParameterValue(SAVE_AS) == null)
2915{
2916   if (!request.isAllowedImmediateExecution())
2917   {
2918      response.setError("Immediate download is not allowed. " +
2919         "Please specify a filename.", null);
2920      return;
2921   }
2922   Client client = (Client)request.getParameterValue("client");
2923   response.setDownloadImmediately("Export help texts for client application " +
2924     client.getName(), ExecutionTime.SHORTEST, true);
2925}
2926else
2927{
2928   if (!pathCanBeUsed((String)request.getParameterValue(SAVE_AS),
2929      (Boolean)request.getParameterValue(OVERWRITE)))
2930   {
2931      response.setError("File exists: " +
2932         (String)request.getParameterValue(SAVE_AS), null);
2933      return;
2934   }
2935   storeValue(job, request, ri.getParameter(SAVE_AS));
2936   storeValue(job, request, ri.getParameter(OVERWRITE));
2937   response.setDone("The job configuration is complete", ExecutionTime.SHORTEST);
2938}
2939</programlisting>
2940       
2941        </para>
2942      </listitem>
2943     
2944      <listitem>
2945        <para>
2946        Implement the <methodname>performExport()</methodname> method.
2947        This is defined as abstract in the <classname docapi="net.sf.basedb.core.plugin">AbstractExporterPlugin</classname>
2948        class. It has the same parameters as the <methodname>ImmediateDownloadExporter.doExport()</methodname>
2949        method and they have the same meaning. The only difference is that the
2950        <varname>out</varname> stream can be linked to a file in the BASE filesystem
2951        and not just to the HTTP response stream.
2952        </para>
2953      </listitem>
2954     
2955      <listitem>
2956        <para>
2957        Optionally, implement the <methodname>begin()</methodname>,
2958        <methodname>end()</methodname> and <methodname>getSuccessMessage()</methodname>
2959        methods. Theese methods do nothing by default.
2960        </para>
2961      </listitem>
2962      </itemizedlist>
2963     
2964      <para>
2965        The call sequence for plug-ins extending <classname docapi="net.sf.basedb.core.plugin">AbstractExporterPlugin</classname>
2966        is:
2967      </para>
2968     
2969      <orderedlist>
2970      <listitem>
2971        <para>
2972        Call <methodname>begin()</methodname>.
2973        </para>
2974      </listitem>
2975      <listitem>
2976        <para>
2977        Call <methodname>performExport()</methodname>.
2978        </para>
2979      </listitem>
2980      <listitem>
2981        <para>
2982        Call <methodname>end()</methodname>.
2983        </para>
2984      </listitem>
2985      <listitem>
2986        <para>
2987        Call <methodname>getSuccessMessage()</methodname> if running as a regular
2988        job. This method is never called when doing an immediate download since there
2989        is no place to show the message.
2990        </para>
2991      </listitem>
2992      </orderedlist>
2993     
2994    </sect2>
2995   
2996  </sect1>
2997 
2998  <sect1 id="plugin_developer.analyse">
2999    <title>Analysis plug-ins</title>
3000    <para>
3001      A plug-in becomes an analysis plug-in simply by returning
3002      <constant>Plugin.MainType.ANALYZE</constant> from the
3003      <methodname>Plugin.getMainType()</methodname> method. The information returned from
3004      <methodname>InteractivePlugin.getGuiContexts()</methodname>
3005      must include: [<constant>Item.BIOASSAYSET</constant>, <constant>Type.ITEM</constant>]
3006      since this is the main place where the web client looks for analysis plug-ins. If
3007      the plug-in can work on a subset of the bioassays it may also include
3008      [<constant>Item.BIOASSAY</constant>, <constant>Type.LIST</constant>]
3009      among the contexts. This will make it possible for a user to select
3010      bioassays from the list and then invoke the plug-in.
3011    </para>
3012   
3013<programlisting language="java">
3014private static final Set&lt;GuiContext&gt; guiContexts =
3015   Collections.singleton(new GuiContext(Item.BIOASSAYSET, GuiContext.Type.ITEM));
3016
3017public Set&lt;GuiContext&gt; getGuiContexts()
3018{
3019   return guiContexts;
3020}</programlisting>
3021
3022    <para>
3023    If the plugin depends on a specific raw data type or on the number of
3024    channels, it should check that the current bioassayset is of the
3025    correct type in the <methodname>InteractivePlugin.isInContext()</methodname> 
3026    method. It is also a good idea to check if the current user has permission
3027    to use the current experiment. This permission is needed to create new bioassaysets or
3028    other data belonging to the experiment.
3029    </para>
3030 
3031    <programlisting language="java">
3032public boolean isInContext(GuiContext context, Object item)
3033{
3034   if (item == null)
3035   {
3036      message = "The object is null";
3037   }
3038   else if (!(item instanceof BioAssaySet))
3039   {
3040      message = "The object is not a BioAssaySet: " + item;
3041   }
3042   else
3043   {
3044      BioAssaySet bas = (BioAssaySet)item;
3045      int channels = bas.getRawDataType().getChannels();
3046      if (channels != 2)
3047      {
3048         message = "This plug-in requires 2-channel data, not " + channels + "-channel.";
3049      }
3050      else
3051      {
3052         Experiment e = bas.getExperiment();
3053         e.checkPermission(Permission.USE);
3054      }
3055   }
3056}
3057</programlisting>
3058
3059    <para>
3060    The plugin should always include a parameter asking for the current
3061    bioassay set when the <methodname>InteractivePlugin.getRequestInformation()</methodname>
3062    is called with <literal>command = Request.COMMAND_CONFIGURE_JOB</literal>.
3063    </para> 
3064
3065    <programlisting language="java">
3066private static final RequestInformation configurePlugin;
3067private RequestInformation configureJob;
3068private PluginParameter&lt;BioAssaySet&gt; bioAssaySetParameter;
3069
3070public RequestInformation getRequestInformation(GuiContext context, String command)
3071   throws BaseException
3072{
3073   RequestInformation requestInformation = null;
3074   if (command.equals(Request.COMMAND_CONFIGURE_PLUGIN))
3075   {
3076      requestInformation = getConfigurePlugin(context);
3077   }
3078   else if (command.equals(Request.COMMAND_CONFIGURE_JOB))
3079   {
3080      requestInformation = getConfigureJob(context);
3081   }
3082   return requestInformation;
3083}
3084
3085private RequestInformation getConfigureJob(GuiContext context)
3086{
3087   if (configureJob == null)
3088   {
3089      bioAssaySetParameter; = new PluginParameter&lt;BioAssaySet&gt;(
3090         "bioAssaySet",
3091         "Bioassay set",
3092         "The bioassay set used as the source for this analysis plugin",
3093         new ItemParameterType&lt;BioAssaySet&gt;(BioAssaySet.class, null, true, 1, null)
3094      );
3095
3096      List&lt;PluginParameter&lt;?&gt;&gt; parameters = new ArrayList&lt;PluginParameter&lt;?&gt;&gt;();
3097      parameters.add(bioAssaySetParameter);
3098      // Add more plug-in-specific parameters here...
3099   
3100      configureJob = new RequestInformation(
3101         Request.COMMAND_CONFIGURE_JOB,
3102         "Configure job",
3103         "Set parameter for plug-in execution",
3104         parameters
3105      );
3106   }
3107   return configureJob;
3108}
3109</programlisting>
3110
3111    <para>
3112    Of course, the <methodname>InteractivePlugin.configure()</methodname> method needs
3113    to validate and store the bioassay set parameter as well:
3114    </para>
3115 
3116    <programlisting language="java">
3117public void configure(GuiContext context, Request request, Response response)
3118{
3119   String command = request.getCommand();
3120   try
3121   {
3122      if (command.equals(Request.COMMAND_CONFIGURE_PLUGIN))
3123      {
3124         // Validate and store configuration parameters
3125         response.setDone("Plugin configuration complete");
3126      }
3127      else if (command.equals(Request.COMMAND_CONFIGURE_JOB))
3128      {
3129         List&lt;Throwable&gt; errors =
3130            validateRequestParameters(configureJob.getParameters(), request);
3131         if (errors != null)
3132         {
3133            response.setError(errors.size() +
3134               " invalid parameter(s) were found in the request", errors);
3135            return;
3136         }
3137         storeValue(job, request, bioAssaySetParameter);
3138         // Store other plugin-specific parameters
3139     
3140         response.setDone("Job configuration complete", Job.ExecutionTime.SHORT);
3141      }
3142   }
3143   catch (Throwable ex)
3144   {
3145      // Never throw exception, always set response!
3146      response.setError(ex.getMessage(), Arrays.asList(ex));
3147   }
3148}
3149</programlisting>
3150
3151    <para>
3152    Now, the typical <methodname>Plugin.run()</methodname> method loads the specfied bioassay set
3153    and its spot data. It may do some filtering and recalculation of the spot
3154    intensity value(s). In most cases it will store the result as a child bioassay
3155    set with one bioassay for each bioassay in the parent bioassay set.
3156    Here is an example, which just copies the intensity values, while
3157    removing those with a negative value in either channel.
3158    </para>
3159 
3160    <programlisting language="java">
3161public void run(Request request, Response response, ProgressReporter progress)
3162{
3163   DbControl dc = sc.newDbControl();
3164   try
3165   {
3166      BioAssaySet source = (BioAssaySet)job.getParameter("bioAssaySet");
3167      // Reload with current DbControl
3168      source = BioAssaySet.getById(dc, source.getId());
3169      int channels = source.getRawDataType().getChannels();
3170     
3171      // Create transformation and new bioassay set
3172      Job j = Job.getById(dc, job.getId());
3173      Transformation t = source.newTransformation(j);
3174      t.setName("Copy spot intensities &gt;= 0");
3175      dc.saveItem(t);
3176
3177      BioAssaySet result = t.newProduct(null, "new", true);
3178      result.setName("After: Copying spot intensities");
3179      dc.saveItem(result);
3180
3181      // Get query for source data
3182      DynamicSpotQuery query = source.getSpotData();
3183     
3184      // Do not return spots with intensities &lt; 0
3185      for (int ch = 1; ch &lt;= channels; ++ch)
3186      {
3187         query.restrict(
3188            Restrictions.gteq(
3189               Dynamic.column(VirtualColumn.channel(ch)),
3190               Expressions.integer(0)
3191            )
3192         );
3193      }
3194     
3195      // Create batcher and copy data
3196      SpotBatcher batcher = result.getSpotBatcher();
3197      int spotsCopied = batcher.insert(query);
3198      batcher.close();
3199     
3200      // Commit and return
3201      dc.commit();
3202      response.setDone("Copied " + spotsCopied + " spots.");
3203   }
3204   catch (Throwable t)
3205   {
3206      response.setError(t.getMessage(), Arrays.asList(t));
3207   }
3208   finally
3209   {
3210      if (dc != null) dc.close();
3211   }
3212}
3213</programlisting>
3214
3215    <para>
3216    See <xref linkend="api_overview.dynamic_and_batch_api" />
3217    for more examples of using the analysis API.
3218    </para>
3219
3220    <sect2 id="plugin_developer.analyse.abstractanalysis">
3221      <title>The AbstractAnalysisPlugin class</title>
3222     
3223      <para>
3224        This class is an abstract base class. It is a useful
3225        class for most analysis plug-ins to inherit from. Its main
3226        purpose is to define <classname docapi="net.sf.basedb.core">PluginParameter</classname>
3227        objects that are commonly used in analysis plug-ins. This includes:
3228      </para>
3229     
3230      <itemizedlist>
3231      <listitem>
3232        <para>
3233        The source bioassay set:
3234          <methodname>getSourceBioAssaySetParameter()</methodname>,
3235          <methodname>getCurrentBioAssaySet()</methodname>,
3236          <methodname>getSourceBioAssaySet()</methodname>
3237        </para>
3238      </listitem>
3239      <listitem>
3240        <para>
3241        The optional restriction of which bioassays to use.
3242        All bioassays in a bioassay set will be used if this
3243        parameter is empty. This is useful when the plugin only
3244        should run on a subset of bioassays in a bioassay set:
3245          <methodname>getSourceBioAssaysParameter()</methodname>,
3246          <methodname>getSourceBioAssays()</methodname>
3247        </para>
3248      </listitem>
3249      <listitem>
3250        <para>
3251        The name and description of the child bioassay set that
3252        is going to be created by the plug-in:
3253        <methodname>getChildNameParameter()</methodname>,
3254        <methodname>getChildDescriptionParameter()</methodname>
3255        </para>
3256      </listitem>
3257      <listitem>
3258        <para>
3259        The name and description of the transformation that
3260        represents the execution of the plug-in:
3261        <methodname>getTransformationNameParameter()</methodname>,
3262        <methodname>getTransformationName()</methodname>
3263        </para>
3264      </listitem>
3265      </itemizedlist>
3266     
3267    </sect2>
3268   
3269    <sect2 id="plugin_developer.analyse.filterplugin">
3270      <title>The AnalysisFilterPlugin interface</title>
3271     
3272      <para>
3273        The <interfacename docapi="net.sf.basedb.core.plugin">net.sf.basedb.core.plugin.AnalysisFilterPlugin</interfacename> 
3274        is a tagging interface, with no methods, that all analysis plug-ins that only filters
3275        data should implement. The benefit is that they will be linked from the
3276        <guibutton>Filter bioassay set</guibutton> button and not just
3277        the <guibutton>Run analysis</guibutton> button. They will also get
3278        a different icon in the experiment outline to make filtering
3279        transformations appear different from other transformations.
3280      </para>
3281     
3282      <para>
3283        The interface exists purely for making the user interaction better. There is
3284        no harm in not implementing it since the plug-in will always appear in
3285        from the <guibutton>Run analysis</guibutton> button. On the other hand,
3286        it doesn't cost anything to implement the interface since it doesn't
3287        have any methods.
3288      </para>
3289   
3290    </sect2>
3291
3292  </sect1>
3293 
3294  <sect1 id="plugin_developer.other">
3295    <title>Other plug-ins</title>
3296    <para></para>
3297   
3298    <sect2 id="plugin_developer.other.authentication">
3299      <title>Authentication plug-ins</title>
3300     
3301      <para>
3302        BASE provides a plug-in mechanism for authenticating users
3303        (validating the username and password) when they are logging in.
3304        This plug-in mechanism is not the same as the regular plug-in API.
3305        That is, you do not have worry about user interaction or implementing the
3306        <interfacename docapi="net.sf.basedb.core.plugin">Plugin</interfacename> interface.
3307      </para>
3308     
3309      <sect3 id="plugin_developer.other.authentication.internal_external">
3310        <title>Internal vs. external authentation</title>
3311     
3312        <para>
3313          BASE can authenticate users in two ways. Either it uses the internal
3314          authentication or the external authentication. With internal
3315          authentication BASE stores logins and passwords in its own database.
3316          With external authentication this is handled by some external
3317          application. Even with external authentication it is possible to
3318          let BASE cache the logins/passwords. This makes it possible to login to
3319          BASE if the external authentication server is down.
3320        </para>
3321       
3322        <note>
3323          <para>
3324          An external authentication server can only be used to grant or deny
3325          a user access to BASE. It cannot be used to give a user permissions,
3326          or put a user into groups or different roles inside BASE.
3327          </para>
3328        </note>
3329
3330        <para>
3331          The external authentication service is only used when a user logs in.
3332          Now, one or more of several things can happen:
3333         
3334          <itemizedlist>
3335          <listitem>
3336            <para>
3337              The ROOT user is logging on. Internal authentication is always
3338              used for the root user and the authenticator plug-in is never
3339              used.
3340            </para>
3341          </listitem>
3342
3343          <listitem>
3344            <para>
3345              The login is correct and the user is already known to BASE.
3346              If the plug-in supports extra information (name, email, phone, etc.)
3347              and the <property>auth.synchronize</property> setting
3348              is <constant>TRUE</constant> the extra information is copied to
3349              the BASE server.
3350            </para>
3351          </listitem>
3352         
3353          <listitem>
3354            <para>
3355              The login is correct, but the user is not known to BASE. This happens
3356              the first time a user logs in. BASE will create
3357              a new user account. If the driver supports extra information, it
3358              is copied to the BASE server (even if <property>auth.synchronize</property> 
3359              is not set). The new user account will get the default quota
3360              and be added to the all roles and groups which has been
3361              marked as <emphasis>default</emphasis>.
3362             
3363              <note>
3364              <para>
3365                Prior to BASE 2.4 it was hardcoded to add the new user to the
3366                <emphasis>Users</emphasis> role only.
3367              </para>
3368              </note>
3369            </para>
3370          </listitem>
3371
3372          <listitem>
3373            <para>
3374              If password caching is enabled, the password is copied to BASE.
3375              If an expiration timeout has been set, an expiration date
3376              will be calculated and set on the user account. The expiration
3377              date is only checked when the external authentication server is
3378              down.
3379            </para>
3380          </listitem>
3381
3382          <listitem>
3383            <para>
3384              The authentication server says that the login is invalid or
3385              the password is incorrect. The user will not be logged in.
3386              If a user account with the specified login already exists in
3387              BASE, it will be disabled.
3388            </para>
3389          </listitem>
3390         
3391          <listitem>
3392            <para>
3393              The authentication driver says that something else is wrong.
3394              If password caching is enabled, internal authentication will be
3395              used. Otherwise the user will not be logged in. An already
3396              existing account is not modified or disabled.
3397            </para>
3398          </listitem>
3399          </itemizedlist>
3400         
3401          <note>
3402            <para>
3403            The <guilabel>Encrypt password</guilabel> option that is
3404            available on the login page does not work with external
3405            authentication. The simple reason is that the password is
3406            encrypted with a one-way algorithm making it impossible to
3407            call <methodname>Authenticator.authenticate()</methodname>.
3408            </para>
3409          </note>
3410         
3411        </para>
3412      </sect3>
3413     
3414      <sect3 id="plugin_developer.other.authentication.authenticator">
3415        <title>The Authenticator interface</title>
3416       
3417        <para>
3418          To be able to use external authentication you must create a class
3419          that implements the
3420          <interfacename docapi="net.sf.basedb.core.authentication">net.sf.based.core.authentication.Authenticator</interfacename> 
3421          interface. Specify the name of the class in the <property>auth.driver</property> 
3422          setting in <filename>base.config</filename> and
3423          its initialisation parameters in the <property>auth.init</property> setting.
3424        </para>
3425       
3426        <para>
3427          Your class must have a public no-argument constructor. The BASE
3428          application will create only one instance of the class for lifetime
3429          of the BASE server. It must be thread-safe since it may be invoked by
3430          multiple threads at the same time. Here are the methods that you must
3431          implement
3432        </para>
3433       
3434        <variablelist>
3435        <varlistentry>
3436          <term>
3437            <methodsynopsis language="java">
3438              <modifier>public</modifier>
3439              <void/>
3440              <methodname>init</methodname>
3441              <methodparam>
3442                <type>String</type>
3443                <parameter>settings</parameter>
3444              </methodparam>
3445              <exceptionname>AuthenticationException</exceptionname>
3446            </methodsynopsis>
3447          </term>
3448          <listitem>
3449            <para>
3450            This method is called just after the object has been created with its argument
3451            taken from the <property>auth.init</property> setting in your <filename>base.config</filename> 
3452            file. This method is only called once for an instance of the object. The syntax and meaning of
3453            the parameter is driver-dependent and should be documented by the plug-in.
3454            It is irrelevant for the BASE core.
3455            </para>
3456          </listitem>
3457        </varlistentry>
3458       
3459        <varlistentry>
3460          <term>
3461            <methodsynopsis language="java">
3462              <modifier>public</modifier>
3463              <type>boolean</type>
3464              <methodname>supportsExtraInformation</methodname>
3465            </methodsynopsis>
3466          </term>
3467          <listitem>
3468            <para>
3469            This method should simply return <constant>TRUE</constant> or <constant>FALSE</constant> 
3470            depending on if the plug-in supports extra user information or not. The only required
3471            information about a user is a unique ID and the login. Extra information includes
3472            name, address, phone, email, etc.
3473            </para>
3474          </listitem>       
3475        </varlistentry>
3476
3477        <varlistentry>
3478          <term>
3479            <methodsynopsis language="java">
3480              <modifier>public</modifier>
3481              <type>AuthenticationInformation</type>
3482              <methodname>authenticate</methodname>
3483              <methodparam>
3484                <type>String</type>
3485                <parameter>login</parameter>
3486              </methodparam>
3487              <methodparam>
3488                <type>String</type>
3489                <parameter>password</parameter>
3490              </methodparam>
3491              <exceptionname>UnknownLoginException</exceptionname>
3492              <exceptionname>InvalidPasswordException</exceptionname>
3493              <exceptionname>AuthenticationException</exceptionname>
3494            </methodsynopsis>
3495          </term>
3496          <listitem>
3497            <para>
3498            Try to authenticate a login/password combination. The plug-in should return
3499            an <classname docapi="net.sf.basedb.core.authentication">AuthenticationInformation</classname> object if the
3500            authentication is successful or throw an exception if not.
3501           
3502            There are three exceptions to choose from:
3503           
3504            <itemizedlist>
3505            <listitem>
3506              <para>
3507              <exceptionname>UnknownLoginException</exceptionname>:
3508              This exception should be thrown if the login is not known to the
3509              external authentication system.
3510              </para>
3511            </listitem>
3512
3513            <listitem>
3514              <para>
3515              <exceptionname>InvalidPasswordException</exceptionname>:
3516              This exception should be thrown if the login is known but the
3517              password is invalid. In case it is considered a security issue
3518              to reveal that a login exists, the plugin may throw an
3519              <exceptionname>UnknowLoginException</exceptionname> instead.
3520              </para>
3521            </listitem>
3522           
3523            <listitem>
3524              <para>
3525              <exceptionname>AuthenticationException</exceptionname>:
3526              In case there is another problem, such as the authentication service
3527              being down. This exception triggers the use of cached passwords
3528              if caching has been enabled.
3529              </para>
3530            </listitem>
3531            </itemizedlist>
3532            </para>
3533          </listitem>       
3534        </varlistentry>
3535        </variablelist> 
3536      </sect3>
3537     
3538      <sect3 id="plugin_developer.other.authentication.settings">
3539        <title>Configuration settings</title>
3540     
3541        <para>
3542          The configuration settings for the authentication driver are located
3543          in the <filename>base.config</filename> file.
3544          Here is an overview of the settings. For more information read
3545          <xref linkend="appendix.base.config.authentication" />.
3546        </para>
3547       
3548        <variablelist>
3549        <varlistentry>
3550          <term><property>auth.driver</property></term>
3551          <listitem>
3552            <para>
3553            The class name of the authentication plug-in.
3554            </para>
3555          </listitem>
3556        </varlistentry>
3557
3558        <varlistentry>
3559          <term><property>auth.init</property></term>
3560          <listitem>
3561            <para>
3562            Initialisation parameters sent to the plug-in when calling the
3563            <methodname>Authenticator.init()</methodname> method.
3564            </para>
3565          </listitem>
3566        </varlistentry>
3567       
3568        <varlistentry>
3569          <term><property>auth.synchronize</property></term>
3570          <listitem>
3571            <para>
3572            If extra user information is synchronized at login time or not.
3573            This setting is ignored if the driver does not support extra information.
3574            </para>
3575          </listitem>
3576        </varlistentry>
3577       
3578        <varlistentry>
3579          <term><property>auth.cachepasswords</property></term>
3580          <listitem>
3581            <para>
3582              If passwords should be cached by BASE or not. If the passwords are
3583              cached a user may login to BASE even if the external authentication
3584              server is down.
3585            </para>
3586          </listitem>
3587        </varlistentry>
3588       
3589        <varlistentry>
3590          <term><property>auth.daystocache</property></term>
3591          <listitem>
3592            <para>
3593              How many days to cache the passwords if caching has been enabled.
3594              A value of 0 caches the passwords for ever.     
3595            </para>
3596          </listitem>
3597        </varlistentry>
3598        </variablelist>
3599      </sect3>
3600     
3601    </sect2>
3602   
3603    <sect2 id="plugin_developer.other.secondary">
3604      <title>Secondary file storage plugins</title>
3605     
3606      <sect3 id="plugin_developer.other.secondary.vsprimary">
3607        <title>Primary vs. secondary storage</title>
3608        <para>
3609          BASE has support for storing files in two locations, the primary storage and
3610          the secondary storage. The primary storage is always disk-based and must be
3611          accessible by the BASE server as a path on the file system. The path to the
3612          primary storage is configured by the <varname>userfiles</varname> setting in the
3613          <filename>base.config</filename> file. The primary storage is internal to
3614          the core. Client applications don't get access to read or manipulate the
3615          files directly from the file system.
3616        </para>
3617       
3618        <para>
3619          The secondary storage can be anything that can store files. It could, for
3620          example, be another directory, a remote FTP server, or a tape based archiving
3621          system. A file located in the secondary storage is not accessible by the
3622          core, client applications or plug-ins. The secondary storage can only be accessed
3623          by the secondary storage controller. The core (and client) applications uses
3624          flags on the file items to handle the interaction with the secondary storage.
3625        </para>
3626       
3627        <para>
3628          Each file has an <property>action</property> attribute which default's to
3629          <constant>File.Action.NOTHING</constant>. It can take two other values:
3630        </para>
3631       
3632        <orderedlist>
3633        <listitem>
3634          <para>
3635          <constant>File.Action.MOVE_TO_SECONDARY</constant>
3636          </para>
3637        </listitem>
3638        <listitem>
3639          <para>
3640          <constant>File.Action.MOVE_TO_PRIMARY</constant>
3641          </para>
3642        </listitem>
3643        </orderedlist>
3644       
3645        <para>
3646          All files with the action attribute set to <constant>MOVE_TO_SECONDARY</constant> 
3647          should be moved to the secondary storage by the controller, and all files
3648          with the action attribute set to <constant>MOVE_TO_PRIMARY</constant> should be
3649          brought back to primary storage.
3650        </para>
3651       
3652        <para>
3653          The moving of files between primary and secondary storage doesn't happen
3654          immediately. It is up to the server administrator to configure how often and
3655          at what times the controller should check for files that should be moved.
3656          This is configured by the <varname>secondary.storage.interval</varname> 
3657          and <varname>secondary.storage.time</varname> settings in the
3658          <filename>base.config</filename> file.       
3659        </para>
3660      </sect3>
3661     
3662      <sect3 id="plugin_developer.other.secondary.interface">
3663        <title>The SecondaryStorageController interface</title>
3664       
3665        <para>
3666          All you have to do to create a secondary storage controller is to
3667          create a class that implements the
3668          <interfacename docapi="net.sf.basedb.core">net.sf.basedb.core.SecondaryStorageController</interfacename>
3669          interface. In your <filename>base.config</filename> file you then specify the
3670          class name in the <varname>secondary.storage.driver</varname> setting and its
3671          initialisation parameters in the <varname>secondary.storage.init</varname> setting.
3672        </para>
3673
3674        <para>
3675          Your class must have a public no-argument constructor.
3676          The BASE application will create only one instance of the class for
3677          lifetime of the BASE server. Here are the methods that you must implement:
3678        </para>
3679       
3680        <variablelist>
3681        <varlistentry>
3682          <term>
3683          <methodsynopsis language="java">
3684            <modifier>public</modifier>
3685            <void />
3686            <methodname>init</methodname>
3687            <methodparam>
3688              <type>String</type>
3689              <parameter>settings</parameter>
3690            </methodparam>
3691          </methodsynopsis>
3692          </term>
3693          <listitem>
3694            <para>
3695            This method is called just after the object has been created with its argument
3696            taken from the <varname>secondary.storage.init</varname> setting in your
3697            <filename>base.config</filename> file. This method is only called once for
3698            an object.
3699            </para>
3700          </listitem>
3701        </varlistentry>
3702        <varlistentry>
3703          <term>
3704          <methodsynopsis language="java">
3705            <modifier>public</modifier>
3706            <void />
3707            <methodname>run</methodname>
3708          </methodsynopsis>
3709          </term>
3710          <listitem>
3711            <para>
3712            This method is called whenever the core thinks it is time to do some
3713            management of the secondary storage. How often the <methodname>run()</methodname>
3714            method is called is controlled by the <varname>secondary.storage.interval</varname>
3715            and <varname>secondary.storage.time</varname> settings in the
3716            <filename>base.config</filename> file.
3717            When this method is called the controller should:
3718            </para>
3719           
3720            <itemizedlist>
3721            <listitem>
3722              <para>
3723              Move all files which has <constant>action=MOVE_TO_SECONDARY</constant> to
3724              the secondary storage. When the file has been moved call
3725              <methodname>File.setLocation(Location.SECONDARY)</methodname> to tell the
3726              core that the file is now in the secondary storage. You should also call
3727              <methodname>File.setAction(File.Action.NOTHING)</methodname> to reset the
3728              action attribute.
3729              </para>
3730            </listitem>
3731           
3732            <listitem>
3733              <para>
3734              Restore all files which has <constant>action=MOVE_TO_PRIMARY</constant>.
3735              The core will set the location attribute automatically, but you should
3736              call <methodname>File.setAction(File.Action.NOTHING)</methodname> to reset
3737              the action attribute.
3738              </para>
3739            </listitem>
3740           
3741            <listitem>
3742              <para>
3743              Delete all files from the secondary storage that are not present
3744              in the database with <constant>location=Location.SECONDARY</constant>.
3745              This includes files which has been deleted and files that have been
3746              moved offline or re-uploaded.
3747              </para>
3748            </listitem>
3749           
3750            </itemizedlist>
3751           
3752            <para>
3753              As a final act the method should send a message to each user owning
3754              files that has been moved from one location to the other. The message
3755              should include a list of files that has been moved to the secondary
3756              storage and a list of files moved from the secondary storage and a
3757              list of files that has been deleted due to some of the reasons above.
3758            </para>
3759          </listitem>
3760        </varlistentry>       
3761       
3762        <varlistentry>
3763          <term>
3764          <methodsynopsis language="java">
3765            <modifier>public</modifier>
3766            <void />
3767            <methodname>close()</methodname>
3768          </methodsynopsis>
3769          </term>
3770          <listitem>
3771            <para>
3772            This method is called when the server is closing down. After this the object
3773            is never used again.
3774            </para>
3775          </listitem>
3776        </varlistentry>
3777        </variablelist>
3778      </sect3>
3779     
3780      <sect3 id="plugin_developer.other.secondary.settings">
3781        <title>Configuration settings</title>
3782       
3783        <para>
3784        The configuration settings for the secondary storage controller is located in the
3785        <filename>base.config</filename> file. Here is an overview of the settings.
3786        For more information read <xref linkend="appendix.base.config" />.
3787        </para>
3788       
3789        <variablelist>
3790        <varlistentry>
3791          <term><property>secondary.storage.driver</property></term>
3792          <listitem>
3793            <para>
3794            The class name of the secondary storage plug-in.
3795            </para>
3796          </listitem>
3797        </varlistentry>
3798        <varlistentry>
3799          <term><property>secondary.storage.init</property></term>
3800          <listitem>
3801            <para>
3802             Initialisation parameters sent to the plug-in by calling the
3803             <methodname>init()</methodname> method.
3804            </para>
3805          </listitem>
3806        </varlistentry>
3807        <varlistentry>
3808          <term><property>secondary.storage.interval</property></term>
3809          <listitem>
3810            <para>
3811             Interval in seconds between each execution of the secondary storage
3812             controller plug-in.
3813            </para>
3814          </listitem>
3815        </varlistentry>
3816        <varlistentry>
3817          <term><property>secondary.storage.time</property></term>
3818          <listitem>
3819            <para>
3820              Time points during the day when the secondary storage controller plugin
3821              should be executed.
3822            </para>
3823          </listitem>
3824        </varlistentry>
3825        </variablelist>
3826      </sect3>
3827    </sect2>
3828   
3829    <sect2 id="plugin_developer.other.unpacker">
3830      <title>File unpacker plug-ins</title>
3831      <para>
3832        The BASE web client has integrated support for unpacking of
3833        compressed files. See <xref linkend="file_system.handling.upload" />.
3834        Behind the scenes, this support is provided by plug-ins. The standard
3835        BASE distribution comes with support for ZIP files
3836        (<classname docapi="net.sf.basedb.plugins">net.sf.basedb.plugins.ZipFileUnpacker</classname>)
3837        and TAR files (<classname docapi="net.sf.basedb.plugins">net.sf.basedb.plugins.TarFileUnpacker</classname>).
3838      </para>
3839      <para>
3840        To add support for additional compressed formats you have to create a plug-in that
3841        implements the <interfacename docapi="net.sf.basedb.util.zip">net.sf.basedb.util.zip.FileUnpacker</interfacename>
3842        interface. The best way to do this is to extend the
3843        <classname docapi="net.sf.basedb.util.zip">net.sf.basedb.util.zip.AbstractFileUnpacker</classname> which
3844        implements all methods in the <interfacename docapi="net.sf.basedb.core.plugin">Plugin</interfacename>
3845        and <interfacename docapi="net.sf.basedb.core.plugin">InteractivePlugin</interfacename>
3846        interfaces except <methodname>Plugin.getAbout()</methodname>. This leaves
3847        you with the actual unpacking of the files as the only thing to implement.
3848      </para>
3849     
3850      <note>
3851        <title>No support for configurations</title>
3852        The integrated upload in the web interface only works with plug-ins that
3853        does not require a configuration to run.
3854      </note>
3855     
3856      <variablelist>
3857        <title>Methods in the <interfacename docapi="net.sf.basedb.util.zip">FileUnpacker</interfacename> interface</title>
3858        <varlistentry>
3859          <term>
3860            <methodsynopsis language="java">
3861              <modifier>public</modifier>
3862              <type>String</type>
3863              <methodname>getFormatName</methodname>
3864            </methodsynopsis>
3865          </term>
3866          <listitem>
3867            <para>
3868            Return a short string naming the file format. For example:
3869            <constant>ZIP files</constant> or <constant>TAR files</constant>.
3870            </para>
3871          </listitem>
3872        </varlistentry>
3873           
3874        <varlistentry>
3875          <term>
3876            <methodsynopsis language="java">
3877              <modifier>public</modifier>
3878              <type>Set&lt;String&gt;</type>
3879              <methodname>getExtensions</methodname>
3880            </methodsynopsis>
3881          </term>
3882          <listitem>
3883            <para>
3884            Return a set of strings with the file extensions that
3885            are most commonly used with the compressed file format.
3886            For example: <constant>[zip, jar]</constant>. Do not include
3887            the dot in the extensions. The web client and the
3888            <methodname>AbstractFlatFileUnpacker.isInContext()</methodname> method
3889            will use this information to automatically guess which plug-in to
3890            use for unpacking the files.
3891            </para>
3892          </listitem>
3893        </varlistentry>
3894       
3895        <varlistentry>
3896          <term>
3897            <methodsynopsis language="java">
3898              <modifier>public</modifier>
3899              <type>Set&lt;String&gt;</type>
3900              <methodname>getMimeTypes</methodname>
3901            </methodsynopsis>
3902          </term>
3903          <listitem>
3904            <para>
3905            Return a set of string with the MIME types that commonly used with
3906            the compressed file format. For example:
3907            <constant>[application/zip, application/java-archive]</constant>.
3908            This information is used by the
3909            <methodname>AbstractFlatFileUnpacker.isInContext()</methodname> 
3910            method to automatically guess which plug-in to use for unpacking
3911            the files.
3912            </para>
3913          </listitem>
3914        </varlistentry>
3915       
3916        <varlistentry>
3917          <term>
3918            <methodsynopsis language="java">
3919              <modifier>public</modifier>
3920              <type>int</type>
3921              <methodname>unpack</methodname>
3922              <methodparam>
3923                <type>DbControl</type>
3924                <parameter>dc</parameter>
3925              </methodparam>
3926              <methodparam>
3927                <type>Directory</type>
3928                <parameter>dir</parameter>
3929              </methodparam>
3930              <methodparam>
3931                <type>InputStream</type>
3932                <parameter>in</parameter>
3933              </methodparam>
3934              <methodparam>
3935                <type>boolean</type>
3936                <parameter>overwrite</parameter>
3937              </methodparam>
3938              <methodparam>
3939                <type>AbsoluteProgressReporter</type>
3940                <parameter>progress</parameter>
3941              </methodparam>
3942              <exceptionname>IOException</exceptionname>
3943              <exceptionname>BaseException</exceptionname>
3944            </methodsynopsis>
3945          </term>
3946          <listitem>
3947            <para>
3948            Unpack the files and store them in the BASE file system.
3949           
3950            <itemizedlist>
3951            <listitem>
3952              <para>
3953              Do not <methodname>close()</methodname> or
3954              <methodname>commit()</methodname> the
3955              <classname docapi="net.sf.basedb.core">DbControl</classname> passed to this method.
3956              This is done automatically by the
3957              <classname docapi="net.sf.basedb.util.zip">AbstractFileUnpacker</classname> or by the web client.
3958              </para>
3959            </listitem>
3960           
3961            <listitem>
3962              <para>
3963              The <varname>dir</varname> parameter is the root directory where
3964              the unpacked files should be placed. If the compressed file
3965              contains subdirectories the plug-in must create those subdirectories
3966              unless they already exists.
3967              </para>
3968            </listitem>
3969           
3970            <listitem>
3971              <para>
3972              If the <varname>overwrite</varname> parameter is
3973              <constant>FALSE</constant> no existing file should be overwritten
3974              unless the file is <systemitem>OFFLINE</systemitem>.
3975              </para>
3976            </listitem>
3977           
3978            <listitem>
3979              <para>
3980              The <varname>in</varname> parameter is the stream
3981              containing the compressed data. The stream may come
3982              directly from the web upload or from an existing
3983              file in the BASE file system.
3984              </para>
3985            </listitem>
3986           
3987            <listitem>
3988              <para>
3989              The <varname>progress</varname> parameter, if not
3990              <constant>null</constant>, should be used to report the
3991              progress back to the calling code. The plug-in should count
3992              the number of bytes read from the <varname>in</varname>
3993              stream. If it is not possible by other means the stream can
3994              be wrapped by a <classname docapi="net.sf.basedb.util">net.sf.basedb.util.InputStreamTracker</classname>
3995              object which has a <methodname>getNumRead()</methodname> method.
3996              </para>
3997            </listitem>
3998            </itemizedlist>
3999           
4000            </para>
4001          </listitem>
4002        </varlistentry>
4003      </variablelist>
4004     
4005      <para>
4006        When the compressed file is uncompressed during the file upload
4007        from the web interface, the call sequence to the plug-in is slightly
4008        altered from the standard call sequence described in
4009        <xref linkend="plugin_developer.api.callsequence.execute" />.
4010       
4011        <itemizedlist>
4012        <listitem>
4013          <para>
4014          After the plug-in instance has been created, the
4015          <methodname>Plugin.init()</methodname> method is called with <constant>null</constant>
4016          values for both the <varname>configuration</varname> and <varname>job</varname>
4017          parameters.
4018          </para>
4019        </listitem>
4020       
4021        <listitem>
4022          <para>
4023          Then, the <methodname>unpack()</methodname> method is called. The
4024          <methodname>Plugin.run()</methodname> method is never called in this case.
4025          </para>
4026        </listitem>
4027       
4028        </itemizedlist>
4029       
4030      </para>
4031     
4032    </sect2>
4033   
4034    <sect2 id="plugin_developer.other.packer">
4035      <title>File packer plug-ins</title>
4036   
4037      <para>
4038        BASE has support for compressing and downloading a set of selected files and/or
4039        directories. This functionality is provided by a plug-in, the
4040        <classname docapi="net.sf.basedb.plugins">PackedFileExporter</classname>. This plug-in doesn't do the actual
4041        packing itself. This is delegated to classes implementing the
4042        <interfacename docapi="net.sf.basedb.util.zip">net.sf.basedb.util.zip.FilePacker</interfacename> interface.
4043      </para>
4044     
4045      <para>
4046        BASE ships with a number of packing methods, including ZIP and TAR. To
4047        add support for other methods you have to provide an implementation
4048        of the <interfacename docapi="net.sf.basedb.util.zip">FilePacker</interfacename>
4049        interface. Then, create a new configuration for the <classname docapi="net.sf.basedb.plugins">PackedFileExporter</classname>
4050        and enter the name of your class in the configuration wizard.
4051      </para>
4052     
4053      <para>
4054        The <interfacename docapi="net.sf.basedb.util.zip">FilePacker</interfacename> interface is not a regular
4055        plug-in interface (ie. it is not a subinterface to
4056        <interfacename docapi="net.sf.basedb.core.plugin">Plugin</interfacename>). This means that you don't have to
4057        mess with configuration or job parameters. Another difference is that your
4058        class must be installed in Tomcat's classpath (ie. in one of the
4059        <filename>WEB-INF/classes</filename> or <filename>WEB-INF/lib</filename>
4060        folders).
4061      </para>
4062     
4063      <variablelist>
4064        <title>Methods in the <interfacename docapi="net.sf.basedb.util.zip">FilePacker</interfacename> interface</title>
4065        <varlistentry>
4066          <term>
4067            <methodsynopsis language="java">
4068              <modifier>public</modifier>
4069              <type>String</type>
4070              <methodname>getDescription</methodname>
4071            </methodsynopsis>
4072          </term>
4073          <listitem>
4074            <para>
4075            Return a short description the file format that is suitable for use
4076            in dropdown lists in client applications. For example:
4077            <constant>Zip-archive (.zip)</constant> or <constant>TAR-archive (.tar)</constant>.
4078            </para>
4079          </listitem>
4080        </varlistentry>
4081        <varlistentry>
4082          <term>
4083            <methodsynopsis language="java">
4084              <modifier>public</modifier>
4085              <type>String</type>
4086              <methodname>getFileExtension</methodname>
4087            </methodsynopsis>
4088          </term>
4089          <listitem>
4090            <para>
4091            Return the default file extension of the packed format. The returned
4092            value should not include the dot. For example:
4093            <constant>zip</constant> or <constant>tar</constant>.
4094            </para>
4095          </listitem>
4096        </varlistentry>
4097        <varlistentry>
4098          <term>
4099            <methodsynopsis language="java">
4100              <modifier>public</modifier>
4101              <type>String</type>
4102              <methodname>getMimeType</methodname>
4103            </methodsynopsis>
4104          </term>
4105          <listitem>
4106            <para>
4107            Return the standard MIME type of the packed file format.
4108            For example:
4109            <constant>application/zip</constant> or <constant>application/x-tar</constant>.
4110            </para>
4111          </listitem>
4112        </varlistentry>
4113        <varlistentry>
4114          <term>
4115            <methodsynopsis language="java">
4116              <modifier>public</modifier>
4117              <void />
4118              <methodname>setOutputStream</methodname>
4119              <methodparam>
4120                <type>OutputStream</type>
4121                <parameter>out</parameter>
4122              </methodparam>
4123              <exceptionname>IOException</exceptionname>
4124            </methodsynopsis>
4125          </term>
4126          <listitem>
4127            <para>
4128            Sets the outputstream that the packer should write the packed
4129            files to.
4130            </para>
4131          </listitem>
4132        </varlistentry>
4133        <varlistentry>
4134          <term>
4135            <methodsynopsis language="java">
4136              <modifier>public</modifier>
4137              <void />
4138              <methodname>pack</methodname>
4139              <methodparam>
4140                <type>String</type>
4141                <parameter>entryName</parameter>
4142              </methodparam>
4143              <methodparam>
4144                <type>InputStream</type>
4145                <parameter>in</parameter>
4146              </methodparam>
4147              <methodparam>
4148                <type>long</type>
4149                <parameter>size</parameter>
4150              </methodparam>
4151              <methodparam>
4152                <type>long</type>
4153                <parameter>lastModified</parameter>
4154              </methodparam>
4155              <exceptionname>IOException</exceptionname>
4156            </methodsynopsis>
4157          </term>
4158          <listitem>
4159            <para>
4160            Add another file or directory to the packed file. The
4161            <parameter>entryName</parameter> is the name of the new entry, including
4162            path information. The <parameter>in</parameter> is the stream to read
4163            the file data from. If <parameter>in</parameter> is <constant>null</constant>
4164            then the entry denotes a directory. The <parameter>size</parameter> parameter
4165            gives the size in bytes of the file (zero for empty files or directories).
4166            The <parameter>lastModified</parameter>
4167            is that time the file was last modified or 0 if not known.
4168            </para>
4169          </listitem>
4170        </varlistentry>
4171        <varlistentry>
4172          <term>
4173            <methodsynopsis language="java">
4174              <modifier>public</modifier>
4175              <void />
4176              <methodname>close</methodname>
4177              <exceptionname>IOException</exceptionname>
4178            </methodsynopsis>
4179          </term>
4180          <listitem>
4181            <para>
4182            Finish the packing. The packer should release any resources, flush
4183            all data and close all output streams, including the <varname>out</varname> stream
4184            set in the <methodname>setOutputStream</methodname> method.
4185            </para>
4186          </listitem>
4187        </varlistentry>
4188       
4189      </variablelist>
4190     
4191    </sect2>
4192   
4193    <sect2 id="plugin_developer.other.datafiles">
4194      <title>File validator and metadata reader plug-ins</title>
4195   
4196      <itemizedlist>
4197        <title>See also</title>
4198        <listitem><xref linkend="core_api.data_in_files" /></listitem>
4199        <listitem><xref linkend="data_api.platforms" /></listitem>
4200      </itemizedlist>
4201   
4202   
4203      <para>
4204        In those cases where files are used to store data instead
4205        of importing it to the database, BASE can use plug-ins to
4206        check that the supplied files are valid and also to extract
4207        metadata from the files. For example, the
4208        <classname docapi="net.sf.basedb.core.filehandler">net.sf.basedb.core.filehandler.CelFileHandler</classname>
4209        is used to check if a file is a valid Affymetrix CEL file and
4210        to extract data headers and the number of spots from it.
4211      </para>
4212     
4213      <para>
4214        The validator and metadata reader plug-ins are not regular plug-ins
4215        (ie. they don't have to implement the <interfacename docapi="net.sf.basedb.core.plugin">Plugin</interfacename>
4216        interface). This means that you don't have to mess with configuration or
4217        job parameters.
4218      </para>
4219     
4220      <para>
4221        Validator plug-ins must implement the
4222        <interfacename docapi="net.sf.basedb.core.filehandler">net.sf.basedb.core.filehandler.DataFileHandler</interfacename>
4223        and <interfacename docapi="net.sf.basedb.core.filehandler">net.sf.basedb.core.filehandler.DataValidator</interfacename>
4224        interfaces. Metadata reader plug-ins should implement the
4225        <interfacename docapi="net.sf.basedb.core.filehandler">net.sf.basedb.core.filehandler.DataFileHandler</interfacename>
4226        and <interfacename docapi="net.sf.basedb.core.filehandler">net.sf.basedb.core.filehandler.DataFileMetadataReader</interfacename>
4227        interfaces.
4228      </para>
4229
4230      <note>
4231        <para>
4232        Meta data extraction can only be done if the file has first been
4233        validated. We recommend that metadata reader plug-ins also takes the role as
4234        validator plug-ins. This will make BASE re-use the same object instance
4235        and the file doesn't have to be parsed twice.
4236        </para>
4237      </note>
4238
4239      <important>
4240        <title>
4241          Always extend the
4242          <classname docapi="net.sf.basedb.core.filehandler">net.sf.basedb.core.filehandler.AbstractDataFileHandler</classname>
4243          class
4244        </title>
4245        <para>
4246          We consider the mentioned interface to be part of the public API only
4247          from the caller side, not from the implementor side. Thus, we may
4248          add methods to those interfaces in the future without prior notice.
4249          The <classname docapi="net.sf.basedb.core.filehandler">AbstractDataFileHandler</classname> will provide default
4250          implementations of the new methods in order to not break existing
4251          plug-ins.
4252        </para>
4253      </important>
4254     
4255      <variablelist>
4256        <title>Methods in the <interfacename docapi="net.sf.basedb.core.filehandler">DataFileHandler</interfacename> interface</title>
4257        <varlistentry>
4258          <term>
4259            <methodsynopsis language="java">
4260              <modifier>public</modifier>
4261              <void/>
4262              <methodname>setFile</methodname>
4263              <methodparam>
4264                <type>FileSetMember</type>
4265                <parameter>member</parameter>
4266              </methodparam>
4267            </methodsynopsis>
4268          </term>
4269          <listitem>
4270            <para>
4271            Sets the file that is going to be validated or used for metadata
4272            extraction. If the same plug-in can be used for validating
4273            more than one type of file, this method will be called
4274            one time for each file that is present in the file set.
4275            </para>
4276          </listitem>
4277        </varlistentry>
4278       
4279        <varlistentry>
4280          <term>
4281            <methodsynopsis language="java">
4282              <modifier>public</modifier>
4283              <void/>
4284              <methodname>setItem</methodname>
4285              <methodparam>
4286                <type>FileStoreEnabled</type>
4287                <parameter>item</parameter>
4288              </methodparam>
4289            </methodsynopsis>
4290          </term>
4291          <listitem>
4292            <para>
4293            Sets the item that the files belong to. This method is only
4294            called once.
4295            </para>
4296          </listitem>
4297        </varlistentry>
4298      </variablelist>
4299     
4300      <variablelist>
4301        <title>Methods in the <interfacename docapi="net.sf.basedb.core.filehandler">DataFileValidator</interfacename> interface</title>
4302        <varlistentry>
4303          <term>
4304            <methodsynopsis language="java">
4305              <modifier>public</modifier>
4306              <void/>
4307              <methodname>validate</methodname>
4308              <methodparam>
4309                <type>DbControl</type>
4310                <parameter>dc</parameter>
4311              </methodparam>
4312              <exceptionname>InvalidDataException</exceptionname>
4313              <exceptionname>InvalidRelationException</exceptionname>
4314            </methodsynopsis>
4315          </term>
4316          <listitem>
4317            <para>
4318              Validate the file. The file is valid if this method returns
4319              sucessfully. If the file is not valid an
4320              <exceptionname>InvalidDataException</exceptionname> should be
4321              thrown. Note that BASE will still accept the file, but will indicate
4322              the failure with a flag and also keep the message of the exception in the
4323              database to remind the user of the failure.
4324            </para>
4325            <para>
4326              The <exceptionname>InvalidRelationException</exceptionname>
4327              should be used to indicate a partial success/partial failure,
4328              where the file as such is a valid file, but in relation to
4329              other files it is not. For example, we may assign a valid CEL
4330              file to a raw bioassay, but the chip type doesn't match
4331              the chip type of the CDF file of the related array design.
4332              This exception will also allow metadata to be extracted from
4333              the file.
4334            </para>
4335          </listitem>
4336        </varlistentry>
4337      </variablelist>
4338
4339      <variablelist>
4340        <title>Methods in the <interfacename docapi="net.sf.basedb.core.filehandler">DataFileMetadataReader</interfacename> interface</title>
4341        <varlistentry>
4342          <term>
4343            <methodsynopsis language="java">
4344              <modifier>public</modifier>
4345              <void/>
4346              <methodname>extractMetadata</methodname>
4347              <methodparam>
4348                <type>DbControl</type>
4349                <parameter>dc</parameter>
4350              </methodparam>
4351            </methodsynopsis>
4352          </term>
4353          <listitem>
4354            <para>
4355              Extract metadata from the file. It is up to the plug-in
4356              to decide what to extract and how to store it.
4357              The <classname docapi="net.sf.basedb.core.filehandler">CelFileHandler</classname> will, for
4358              example, extract headers and the number of spots from the file
4359              and store it with the raw bioassay.
4360            </para>
4361          </listitem>
4362        </varlistentry>
4363        <varlistentry>
4364          <term>
4365            <methodsynopsis language="java">
4366              <modifier>public</modifier>
4367              <void/>
4368              <methodname>resetMetadata</methodname>
4369              <methodparam>
4370                <type>DbControl</type>
4371                <parameter>dc</parameter>
4372              </methodparam>
4373            </methodsynopsis>
4374          </term>
4375          <listitem>
4376            <para>
4377              Remove all metadata that the plug-in usually can extract.
4378              This method is called if a file is unlinked from an item
4379              or if the validation fails. It is important that the
4380              plug-in cleans up everything so that data from a
4381              previous file doesn't remain in the database.
4382            </para>
4383          </listitem>
4384        </varlistentry>
4385      </variablelist>
4386
4387      <variablelist>
4388        <title>Methods in the <classname docapi="net.sf.basedb.core.filehandler">AbstractDataFileHandler</classname> class</title>
4389        <varlistentry>
4390          <term>
4391            <methodsynopsis language="java">
4392              <modifier>public</modifier>
4393              <type>FileStoreEnabled</type>
4394              <methodname>getItem</methodname>
4395            </methodsynopsis>
4396          </term>
4397          <listitem>
4398            <para>
4399              Get the item that was previously added to
4400              <methodname>setItem()</methodname>
4401            </para>
4402          </listitem>
4403        </varlistentry>
4404        <varlistentry>
4405          <term>
4406            <methodsynopsis language="java">
4407              <modifier>public</modifier>
4408              <type>FileSetMember</type>
4409              <methodname>getMember</methodname>
4410              <methodparam>
4411                <type>String</type>
4412                <parameter>dataFileTypeId</parameter>
4413              </methodparam>
4414            </methodsynopsis>
4415          </term>
4416          <listitem>
4417            <para>
4418              Get a file that was previously added to
4419              <methodname>setFile()</methodname>. The
4420              <parameter>dataFileTypeId</parameter> is
4421              the external ID of the <classname docapi="net.sf.basedb.core">DataFileType</classname>.
4422            </para>
4423          </listitem>
4424        </varlistentry>
4425      </variablelist>
4426
4427    </sect2>
4428   
4429  </sect1>
4430 
4431  <sect1 id="plugin_developer.signals">
4432    <title>Enable support for aborting a running a plug-in</title>
4433   
4434    <para>
4435      BASE includes a simple signalling system that can be used to send
4436      signals to plug-ins. The system was primarly developed to allow a user
4437      to kill a plug-in when it is executing. Therfore, the focus of this chapter
4438      will be how to implement a plug-in to make it possible to kill it
4439      during it's execution.
4440    </para>
4441   
4442    <para>
4443      Since we don't want to do this by brute force such as destroying the
4444      process or stopping thread the plug-in executes in, cooperation is needed
4445      by the plug-in. First, the plug-in must implement the
4446      <interfacename docapi="net.sf.basedb.core.signal">SignalTarget</interfacename>
4447      interface. From this, a <interfacename 
4448      docapi="net.sf.basedb.core.signal">SignalHandler</interfacename> can be created.
4449      A plug-in may choose to implement it's own signal handler or use an existing
4450      implementation. BASE, for example, provides the <classname docapi="net.sf.basedb.core.signal"
4451      >ThreadSignalHandler</classname> implementation that supports the <constant>ABORT</constant> signal.
4452      This is a simple implementation that just calls <code>Thread.interrupt()</code>
4453      on the plug-in worker thread. This may cause two different effects:
4454    </para>
4455   
4456    <itemizedlist>
4457    <listitem>
4458      <para>
4459      The <code>Thread.interrupted()</code> flag is set. The plug-in must check this
4460      at regular intervals and if the flag is set it must cleanup, rollback
4461      open transactions and exit as soon as possible.
4462      </para>
4463    </listitem>
4464   
4465    <listitem>
4466      <para>
4467      If the plug-in is waiting in a blocking call that is interruptable, for
4468      example <code>Thread.sleep()</code>, an <classname>InterruptedException</classname>
4469      is thrown. This should cause the same actions as if the flag was set to happen.
4470      </para>
4471     
4472      <warning>
4473        <title>Not all blocking calls are interruptable</title>
4474        <para>
4475        For example calling <code>InputStream.read()</code> may leave the
4476        plug-in waiting in a non-interruptable state. In this case there
4477        is nothing BASE can do to wake it up again.
4478        </para>
4479      </warning>
4480    </listitem>
4481    </itemizedlist>
4482   
4483    <para>
4484      Here is a general outline for a plug-in that uses the
4485      <classname>ThreadSignalHandler</classname>.
4486    </para>
4487    <programlisting language="java">
4488<![CDATA[
4489private ThreadSignalHandler signalHandler;
4490public SignalHandler getSignalHandler()
4491{
4492   signalHandler = new ThreadSignalHandler();
4493   return signalHandler;
4494}
4495
4496public void run(Request request, Response response, ProgressReporter progress)
4497{
4498   if (signalHandler != null) signalHandler.setWorkerThread(null);
4499   beginTransaction();
4500   boolean done = false;
4501   boolean interrupted = false;
4502   while (!done && !interrupted)
4503   {
4504      try
4505      {
4506         done = doSomeWork(); // NOTE! This must not take forever!
4507         interrupted = Thread.interrupted();
4508      }
4509      catch (InterruptedException ex)
4510      {
4511         // NOTE! Try-catch is only needed if thread calls
4512         // a blocking method that is interruptable
4513         interrupted = true;
4514      }
4515   }
4516   if (interrupted)
4517   {
4518      rollbackTransaction();
4519      response.setError("Aborted by user", null);
4520   }
4521   else
4522   {
4523      commitTransaction();
4524      response.setDone("Done");
4525   }
4526}
4527]]>
4528</programlisting>
4529   
4530    <para>
4531      Another signal handler implementation is the
4532      <classname docapi="net.sf.basedb.core.signal">ProgressReporterSignalHandler</classname>.
4533      See that javadoc for information about how to use it.
4534      For more information about the signalling system as a whole,
4535      see <xref linkend="core_api.signals" />.
4536    </para>
4537   
4538  </sect1>
4539 
4540  <sect1 id="plugin_developer.classload">
4541    <title>How BASE load plug-in classes</title>
4542   
4543    <para>
4544      We recommend that plug-in JAR files are installed outside the web server's
4545      classpath. If you are using Tomcat this means that you should not
4546      install the plug-in in the <filename>&lt;base-dir&gt;/www/WEB-INF/lib</filename>
4547      directory or any other directory where the web server keeps it's classes.
4548      The rest of the information in this section only applies to plug-ins that
4549      have been installed following this restriction.
4550    </para>
4551   
4552    <para>
4553      If the above recommendation has been followed BASE will use it's
4554      own classloader to load the plug-in classes. This have several
4555      benefits:
4556    </para>
4557   
4558    <itemizedlist>
4559    <listitem>
4560      <para>
4561      New plug-ins can be installed and existing plug-ins can be updated
4562      without restarting the web server. If the <property>plugins.autounload</property>
4563      setting in <filename>base.config</filename> has been enabled allyou have to
4564      do to update a plug-in is to replace the JAR file with a new version.
4565      BASE will automatically load the new classes the next time the plug-in
4566      is used. If the option isn't enabled, the server admin has to manually
4567      unload the old code from the web interface first.
4568      </para>
4569    </listitem>
4570   
4571    <listitem>
4572      <para>
4573      Plug-ins may use it's own 3-rd party libraries without interfering
4574      with other plug-ins. This may be important because a plug-in may depend on
4575      a certain version of a library while another plug-in may depend on
4576      a different version. Since BASE is using different class-loaders
4577      for different plug-ins this is not a problem.
4578      </para>
4579    </listitem>
4580     
4581    </itemizedlist>
4582   
4583    <para>
4584      The classloading scheme used by BASE also means plug-in
4585      developers must pay attention to a few things:
4586    </para>
4587   
4588    <itemizedlist>
4589    <listitem>
4590      <para>
4591      A plug-in can only access/use classes from it's own JAR file, BASE core
4592      classes, Java system classes and from JAR files listed in the plug-in's
4593      <filename>MANIFEST.MF</filename> file. See <xref linkend="plugin_developer.organize" />.
4594      </para>
4595    </listitem>
4596   
4597    <listitem>
4598      <para>
4599      A plug-in can also access other plug-ins, but only via the methods and
4600      interfaces defined in BASE. In the following example we assume that
4601      there are two plug-ins, <classname>ex.MyPlugin</classname> 
4602      and <classname>ex.MyOtherPlugin</classname>,
4603      located in two different JAR files. The code below is executing
4604      in the <classname>ex.MyPlugin</classname>:
4605      </para>
4606     
4607      <programlisting language="java">
4608// Prepare to load MyOtherPlugin
4609SessionControl sc = ...
4610DbControl dc = ...
4611PluginDefinition def = PluginDefinition.getByClassName(dc, "ex.MyOtherPlugin");
4612
4613// Ok
4614Plugin other = def.newInstance(Plugin.class, null, sc, null, null);
4615
4616// Not ok; fails with ClassCastException
4617MyOtherPlugin other = def.newInstance(MyOtherPlugin.class, null, sc, null, null);
4618</programlisting>
4619     
4620      <para>
4621      The reason that the second call fails is that BASE uses a different
4622      classloader to load the <classname>ex.MyOtherPlugin</classname> class. This
4623      class is not (in Java terms) the same as the <classname>ex.MyOtherPlugin</classname> 
4624      class loaded by the classloader that loaded the <classname>ex.MyPlugin</classname>
4625      class. If, on the other hand, both plug-ins are located in the same
4626      JAR file BASE uses the same classloader and the second call will succeed.
4627      </para>
4628     
4629      <para>
4630      The first call always succeeds because it uses the <interface>Plugin</interface>
4631      interface which is defined by BASE. This class is loaded by the web servers class
4632      loaded and is the same for all plug-ins.
4633      </para>
4634     
4635      <para>
4636      A third option is that the <classname>ex.MyPlugin</classname>
4637      lists the JAR file where <classname>ex.MyOtherPlugin</classname> is
4638      located in it's <filename>MANIFEST.MF</filename> file. Then, the following
4639      code can be used: <code>MyOtherPlugin other = new MyOtherPlugin();</code>
4640      </para>
4641     
4642    </listitem>
4643    </itemizedlist>
4644   
4645    <para>
4646      Tomcat includes a good document describing how classloading is implemented
4647      in Tomcat: <ulink url="http://tomcat.apache.org/tomcat-5.5-doc/class-loader-howto.html"
4648        >http://tomcat.apache.org/tomcat-5.5-doc/class-loader-howto.html</ulink>.
4649     
4650      BASE's classloading scheme isn't as complex as Tomcat's, but it
4651      very similar to how Tomcat loads different web applications.
4652      The figure on the linked document could be extended with another level
4653      with separate classloaders for each plug-in as child
4654      classloaders to the web application classloaders. 
4655    </para>
4656  </sect1>
4657 
4658  <sect1 id="plugin_developer.example">
4659    <title>Example plug-ins (with download)</title>
4660    <para>
4661      We have created some example plug-ins which demonstrates how to
4662      use the plug-in system and how to create an interactive plug-in that
4663      can ask a user for one or more parameters.
4664   
4665      You can <ulink url="http://baseplugins.thep.lu.se/wiki/net.sf.basedb.examples.plugins">download
4666      a tar file with the source and compiled plug-in code</ulink> from the BASE plug-ins website.
4667    </para>
4668   
4669  </sect1>
4670</chapter>
Note: See TracBrowser for help on using the repository browser.