source: trunk/doc/src/docbook/developer/plugin_developer.xml @ 5781

Last change on this file since 5781 was 5781, checked in by Nicklas Nordborg, 10 years ago

References #1590: Documentation cleanup

Moved "Internals of the Core API" to "The BASE API" section.

  • Property svn:eol-style set to native
  • Property svn:keywords set to Id
File size: 168.2 KB
Line 
1<?xml version="1.0" encoding="UTF-8"?>
2<!DOCTYPE chapter PUBLIC
3    "-//Dawid Weiss//DTD DocBook V3.1-Based Extension for XML and graphics inclusion//EN"
4    "../../../../lib/docbook/preprocess/dweiss-docbook-extensions.dtd">
5<!--
6  $Id: plugin_developer.xml 5781 2011-10-04 12:00:07Z nicklas $:
7 
8  Copyright (C) 2007 Johan Enell, Peter Johansson, Nicklas Nordborg, Martin Svensson
9 
10  This file is part of BASE - BioArray Software Environment.
11  Available at http://base.thep.lu.se/
12 
13  BASE is free software; you can redistribute it and/or
14  modify it under the terms of the GNU General Public License
15  as published by the Free Software Foundation; either version 3
16  of the License, or (at your option) any later version.
17 
18  BASE is distributed in the hope that it will be useful,
19  but WITHOUT ANY WARRANTY; without even the implied warranty of
20  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
21  GNU General Public License for more details.
22 
23  You should have received a copy of the GNU General Public License
24  along with BASE. If not, see <http://www.gnu.org/licenses/>.
25-->
26
27<chapter id="plugin_developer">
28  <title>Plug-in developer</title>
29  <sect1 id="plugin_developer.organize">
30    <title>How to organize your plug-in project</title>
31   
32    <sect2 id="plugin_developer.organize.ant">
33      <title>Using Ant</title>
34      <para>
35        Here is a simple example of how you might organize your project using ant
36        (<ulink url="http://ant.apache.org">http://ant.apache.org</ulink>) as the build
37        tool. This is just a recommendation that we have found to be working
38        well. You may choose to do it another way.
39      </para>
40
41      <sect3 id="plugin_developer.organize.ant.directories">
42        <title>Directory layout</title>
43        <para>
44       
45          Create a directory on your computer where you want
46          to store your plug-in project. This directory is the
47          <filename class="directory"><replaceable>pluginname</replaceable>/</filename>
48          directory in the listing below. You should also create some subdirectories:
49        </para>
50       
51        <literallayout>
52<filename class="directory"><replaceable>pluginname</replaceable>/</filename>
53<filename class="directory"><replaceable>pluginname</replaceable>/bin/</filename>
54<filename class="directory"><replaceable>pluginname</replaceable>/lib/</filename>
55<filename class="directory"><replaceable>pluginname</replaceable>/src/<replaceable>org/company</replaceable>/</filename>
56<filename class="directory"><replaceable>pluginname</replaceable>/META-INF/</filename>
57<filename class="directory"><replaceable>pluginname</replaceable>/META-INF/lib/</filename>
58        </literallayout>
59        <para>
60          The <filename class="directory">bin/</filename>
61          directory is empty to start with. It will contain the compiled code.
62          In the <filename class="directory">lib/</filename>
63          directory you should put <filename>base-core-3.x.x.jar</filename>
64          and other library files that is needed when compiling the source code,
65          but doesn't have to be distributed with your plug-in (since they are already
66          included in the BASE distribution). In the <filename class="directory">META-INF/lib/</filename>
67          directory you should put other library files that are needed both when compiling
68          and when distributing your plug-in.
69        </para>
70        <para>
71          The <filename class="directory">src/</filename>
72          directory contains your source code. In this directory you should create
73          subdirectories corresponding to the package name of your plug-in
74          class(es). See <ulink url="http://en.wikipedia.org/wiki/Java_package"
75          >http://en.wikipedia.org/wiki/Java_package</ulink> for information
76          about conventions for naming packages. The
77          <filename class="directory">META-INF/</filename> directory contains metadata
78          about the plug-in and are needed for easy installation. Typically you'll
79          always need <filename>MANIFEST.MF</filename> and <filename>extensions.xml</filename>
80          but, depending on what you are developing, other files are also needed.
81        </para>
82      </sect3>
83   
84      <sect3 id="plugin_developer.organize.ant.buildfile">
85        <title>The build file</title>
86        <para>
87          In the root of your directory, create the build file:
88          <filename>build.xml</filename>.
89          Here is an example that will compile your plug-in and put it in a JAR file.
90        </para>
91        <example id="plugin_developer.organize.build.file">
92          <title>A simple build file</title>
93          <programlisting language="xml">
94<![CDATA[
95<?xml version="1.0" encoding="UTF-8"?>
96<project
97   name="MyPlugin"
98   default="build.plugin"
99   basedir="."
100   >
101
102   <!-- variables used -->
103   <property name="plugin.name" value="MyPlugin" />
104   <property name="src" value="src" />
105   <property name="bin" value="bin" />
106
107   <!-- set up classpath for compiling -->
108   <path id="classpath">
109      <fileset dir="lib">
110         <include name="**/*.jar"/>
111      </fileset>
112      <fileset dir="META-INF/lib">
113         <include name="**/*.jar"/>
114      </fileset>
115   </path>
116
117   <!-- main target -->
118   <target
119      name="build.plugin" 
120      description="Compiles the plug-in and put in jar"
121      >
122      <javac
123         encoding="UTF-8"
124         srcdir="${src}"
125         destdir="${bin}"
126         classpathref="classpath">
127      </javac>
128      <jar
129         jarfile="${plugin.name}.jar"
130         manifest="META-INF/MANIFEST.MF"
131         >
132         
133         <!-- compiled source code -->
134         <fileset dir="${bin}" />
135         
136         <!-- metadata and extra JAR files -->
137         <fileset dir="." includes="META-INF/**" />
138      </jar>
139    </target>
140</project>
141]]>
142</programlisting>
143        </example>
144        <para>
145          If your plug-in depends JAR files not included with the standard
146          BASE installation, you must create a file called
147          <filename>MANIFEST.MF</filename> in the project <filename>META-INF</filename>
148          directory. List the other JAR files as in the following example.
149          If your plug-in does not depend on other JAR files, you may remove the
150          <sgmltag class="attribute">manifest</sgmltag> 
151          attribute of the <sgmltag class="starttag">jar</sgmltag> tag.
152<programlisting>Manifest-Version: 1.0
153Class-Path: lib/OtherJar.jar lib/ASecondJar.jar</programlisting>
154        See also <xref linkend="plugin_developer.classload" /> for more information
155        regarding class loading when a plug-in depends on a external JAR files.
156        </para>
157       
158        <para>
159          To make installation of your plug-in easier it is a good idea to include
160          the <filename>META-INF/extensions.xml</filename>. This file should
161          include information about your plug-in, such as the name, author, version,
162          and the main Java class for the plug-in. See
163          <xref linkend="plugin_developer.organize.autoinstallcompatible" />
164          for get more information about this feature.
165        </para>
166      </sect3>
167       
168      <sect3 id="plugin_developer.organize.ant.build">
169        <title>Building the plug-in</title>
170        <para>
171          Compile the plug-in simply by typing
172          <command>ant</command>
173          in the console window. If all went well the
174          <filename>MyPlugin.jar</filename>
175          will be created in the same directory.
176        </para>
177        <para>
178          To install the plug-in copy the JAR file to the server's plug-in directory.
179          For more information about the actual installation read
180          <xref linkend="plugins.installation" />.
181        </para>
182      </sect3>
183    </sect2>
184   
185    <sect2 id="plugin_developer.organize.autoinstallcompatible">
186      <title>Make the plug-in compatible with the auto-installation wizard</title>
187      <para>
188        BASE has support for automatically detecting new plug-ins with
189        the auto-installation wizard. The wizard makes it very easy for a
190        server administrator to install new plug-ins. See
191        <xref linkend="plugins.installation.wizard" />.
192      </para>
193     
194      <para>
195        The auto-install feature requires that a plug-in provides some information
196        about itself. The wizard looks in all JAR file for the file
197        <filename>META-INF/extensions.xml</filename>. This file contains
198        some information about the plug-in(s). Here is an example:
199      </para>
200 
201      <programlisting language="xml">
202<![CDATA[
203<?xml version="1.0" encoding="UTF-8" ?>
204<extensions xmlns="http://base.thep.lu.se/extensions.xsd">
205  <!-- information about the complete package (which may contain many plug-in definitions) -->
206  <about>
207    <name>My plug-in package</name>
208    <description>
209      This package contains some very useful plug-ins...
210    </description>
211    <version>1.3</version>
212    <min-base-version>3.0</min-base-version>
213    <copyright>You</copyright>
214    <url>http://www.company.org/with/more/info/about/plugins</url>
215    <email>some.email.address@company.org</email>
216  </about>
217 
218  <!-- Defines a single plug-in (can be repeated multiple times) -->
219  <plugin-definition id="PluginX">
220    <about>
221      <name>Plug-in X</name>
222      <description>
223        Calculates the X transform of....
224      </description>
225    </about>
226    <plugin-class>org.company.PluginClassX</plugin-class>
227  </plugin-definition>
228</extensions>
229]]>
230</programlisting>
231      <para>
232        The first two lines should be the same in all <filename>extensions.xml</filename> files.
233        The rest of the tags are described in following list.
234        <variablelist>
235          <varlistentry>
236            <term><sgmltag class="starttag">about</sgmltag></term>
237            <listitem>
238              <para>
239                The first <sgmltag class="starttag">about</sgmltag> tag is information
240                about the package as a whole. We recommend putting in as much information
241                as possible here. Supported sub-tags are:
242               
243                <sgmltag class="starttag">name</sgmltag>, <sgmltag class="starttag">description</sgmltag>,
244                <sgmltag class="starttag">version</sgmltag>, <sgmltag class="starttag">min-base-version</sgmltag>,
245                <sgmltag class="starttag">max-base-version</sgmltag>, <sgmltag class="starttag">contact</sgmltag>,
246                <sgmltag class="starttag">email</sgmltag>, <sgmltag class="starttag">url</sgmltag> and
247                <sgmltag class="starttag">copyright</sgmltag>
248              </para>
249              <para>
250                Each plug-in that is defined in this file may have it's <sgmltag class="starttag">about</sgmltag>
251                tag which override the global information. Typically, you'll only overide the name and
252                description tags.
253              </para>
254            </listitem>
255          </varlistentry>
256          <varlistentry>
257            <term><sgmltag class="starttag">plugin-definition</sgmltag></term>
258            <listitem>
259              <para>
260                This is the main element for defining a plug-in. It can override the global
261                <sgmltag class="starttag">about</sgmltag> information. We recommend that at
262                least a <sgmltag class="starttag">name</sgmltag> and <sgmltag class="starttag">description</sgmltag>
263                is provided.               
264              </para>
265            </listitem>
266          </varlistentry>
267          <varlistentry>
268            <term>
269              <sgmltag class="starttag">plugin-class</sgmltag>
270            </term>
271            <listitem>
272              <para>
273                The value is the fill class name of the plug-in's main
274                class.
275              </para>
276            </listitem>
277          </varlistentry>
278          <varlistentry>
279            <term>
280              <sgmltag class="starttag">min-base-version</sgmltag>
281              <sgmltag class="starttag">max-base-version</sgmltag>
282            </term>
283            <listitem>
284              <para>
285                Optional sub-tags in the <sgmltag class="starttag">about</sgmltag> section.
286                Values are two- or three-digits separated by a dot. For example,
287                3.0.0 and 3.1. If values are given the plug-in will only be installed
288                if the server's BASE version is within the bounds.
289              </para>
290            </listitem>
291          </varlistentry>
292        </variablelist>
293      </para>
294     
295      <sect3 id="plugin_developer.organize.autoinstallcompatible.configurations">
296        <title>Installing plug-in configurations</title>
297     
298        <para>
299          The installation wizard has support for installing plug-in configurations.
300          If some of your plug-ins has support for different configurations that you want
301          to ship as part of the package, use the "Plug-in configuration exporter" in BASE
302          and include the generated file in <filename>META-INF/plugin-configurations.xml</filename>
303          inside your JAR file.
304        </para>
305     
306      </sect3>
307    </sect2>
308   
309  </sect1>
310
311  <sect1 id="plugin_developer.api">
312    <title>The Plug-in API</title>
313    <para>
314    </para>
315   
316    <sect2 id="plugin_developer.api.interfaces">
317   
318      <title>The main plug-in interfaces</title>
319      <para>
320        The BASE API defines two interfaces and one
321        abstract class that are vital for implementing plug-ins:
322        <itemizedlist spacing="compact">
323          <listitem>
324            <simpara>
325              <interfacename docapi="net.sf.basedb.core.plugin">net.sf.basedb.core.plugin.Plugin</interfacename>
326            </simpara>
327          </listitem>
328          <listitem>
329            <simpara>
330              <interfacename docapi="net.sf.basedb.core.plugin">net.sf.basedb.core.plugin.InteractivePlugin</interfacename>
331            </simpara>
332          </listitem>
333          <listitem>
334            <simpara>
335              <classname docapi="net.sf.basedb.core.plugin">net.sf.basedb.core.plugin.AbstractPlugin</classname>
336            </simpara>
337          </listitem>
338        </itemizedlist>
339       
340        A plug-in must always implement the
341        <interfacename docapi="net.sf.basedb.core.plugin">Plugin</interfacename> interface.
342        The <interfacename docapi="net.sf.basedb.core.plugin">InteractivePlugin</interfacename>
343        interface is optional, and is only needed if you want user interaction.
344        The <classname docapi="net.sf.basedb.core.plugin">AbstractPlugin</classname> is a useful base class
345        that your plug-in can use as a superclass. It provides default implementations
346        for some of the interface methods and also has utility methods for
347        validating and storing job and configuration parameter values. Another
348        reason to use this class as a superclass is that it will shield your
349        plug-in from future changes to the Plug-in API. For example, if
350        we decide that a new method is needed in the <interfacename docapi="net.sf.basedb.core.plugin">Plugin</interfacename> 
351        interface we will also try to add a default implementation in
352        the <classname docapi="net.sf.basedb.core.plugin">AbstractPlugin</classname> class.
353      </para>
354 
355      <important>
356        <para>
357        A plug-in must also have public no-argument contructor. Otherwise, BASE will not
358        be able to create new instances of the plug-in class.
359        </para>
360      </important>
361     
362      <sect3 id="plugin_developer.api.interfaces.plugin">
363        <title>The net.sf.basedb.core.plugin.Plugin interface</title>
364        <para>
365          This interface defines the following methods and must be implemented
366          by all plug-ins.
367        </para>
368        <variablelist>
369          <varlistentry>
370            <term>
371              <methodsynopsis language="java">
372                <modifier>public</modifier>
373                <type>Plugin.MainType</type>
374                <methodname>getMainType</methodname>
375                <void />
376              </methodsynopsis>
377            </term>
378            <listitem>
379              <para>
380                Return information about the main type of plug-in. The
381                <classname docapi="net.sf.basedb.core.plugin">Plugin.MainType</classname>
382                is an enumeration with five possible values:
383                <itemizedlist>
384                  <listitem>
385                    <para>
386                      <constant>ANALYZE</constant>:
387                      An analysis plug-in
388                    </para>
389                  </listitem>
390                  <listitem>
391                    <para>
392                      <constant>EXPORT</constant>:
393                      A plug-in that exports data
394                    </para>
395                  </listitem>
396                  <listitem>
397                    <para>
398                      <constant>IMPORT</constant>:
399                      A plug-in that imports data
400                    </para>
401                  </listitem>
402                  <listitem>
403                    <para>
404                      <constant>INTENSITY</constant>:
405                      A plug-in that calculates the original spot intensities
406                      from raw data
407                    </para>
408                  </listitem>
409                  <listitem>
410                    <para>
411                      <constant>OTHER</constant>:
412                      Any other type of plug-in
413                    </para>
414                  </listitem>
415                </itemizedlist>
416                The returned value is stored in the database but is otherwise not used
417                by the core. Client applications (such as the web client) will probably
418                use this information to group the plug-ins, i.e., a button labeled
419                &gbExport;
420                will let you select among the export plug-ins.
421              </para>
422              <example id="net.sf.basedb.core.plugin.Plugin.getMainType">
423                <title>A typical implementation just return one of the values</title>
424<programlisting language="java">
425public Plugin.MainType getMainType()
426{
427   return Plugin.MainType.OTHER;
428}</programlisting>
429              </example>
430            </listitem>
431          </varlistentry>
432          <varlistentry>
433            <term>
434              <methodsynopsis language="java">
435                <modifier>public</modifier>
436                <type>boolean</type>
437                <methodname>supportsConfigurations</methodname>
438                <void />
439              </methodsynopsis>
440            </term>
441            <listitem>
442              <para>
443                If this method returns true, the plug-in can have different
444                configurations, (i.e.
445                <classname docapi="net.sf.basedb.core">PluginConfiguration</classname>).
446                Note that this method may return true even if the
447                <interfacename docapi="net.sf.basedb.core.plugin">InteractivePlugin</interfacename>
448                interface is not implemented. The
449                <classname docapi="net.sf.basedb.core.plugin">AbstractPlugin</classname>
450                returns true for this method, which is the old way before the
451                introduction of this method.
452              </para>
453            </listitem>
454          </varlistentry>
455          <varlistentry>
456            <term>
457              <methodsynopsis language="java">
458                <modifier>public</modifier>
459                <type>boolean</type>
460                <methodname>requiresConfiguration</methodname>
461                <void />
462              </methodsynopsis>
463            </term>
464            <listitem>
465              <para>
466                If this method returns true, the plug-in must have a configuration
467                to be able to run. For example, some of the core import plug-ins
468                must have information about the file format, to be able to import
469                any data.
470                The
471                <classname docapi="net.sf.basedb.core.plugin">AbstractPlugin</classname>
472                returns false for this method, which is the old way before the
473                introduction of this method.
474              </para>
475            </listitem>
476          </varlistentry>
477          <varlistentry>
478            <term>
479              <methodsynopsis language="java">
480                <modifier>public</modifier>
481                <type>Collection&lt;Permissions&gt;</type>
482                <methodname>getPermissions</methodname>
483                <void />
484              </methodsynopsis>
485            </term>
486            <listitem>
487              <para>
488                Return a collection of permissions that the plug-in needs
489                to be able to function as expected. This method may return null
490                or an empty collection. In this case the plug-in permission system
491                is not used and the plug-in always gets the same permissions as the
492                logged in user. If permissions are specified the plug-in should
493                list all permissions it requires. Permissions that are not listed
494                are denied.
495              </para>
496              <note>
497                <para>
498                The final assignment of permissions to a plug-in is always
499                at the hands of a server administrator. He/she may decide to
500                disable the plug-in permission system or revoke some of the
501                requested permissions. The permissions returned by this method is
502                only a recommendation that the server administrator may or may not
503                accept. See <xref linkend="plugins.permissions" />
504                for more information about plug-in permissions.
505                </para>
506              </note>
507            </listitem>
508          </varlistentry>
509          <varlistentry>
510            <term>
511              <methodsynopsis language="java">
512                <modifier>public</modifier>
513                <void />
514                <methodname>init</methodname>
515                <methodparam>
516                  <type>SessionControl</type>
517                  <parameter>sc</parameter>
518                </methodparam>
519                <methodparam>
520                  <type>ParameterValues</type>
521                  <parameter>configuration</parameter>
522                </methodparam>
523                <methodparam>
524                  <type>ParameterValues</type>
525                  <parameter>job</parameter>
526                </methodparam>
527                <exceptionname>BaseException</exceptionname>
528              </methodsynopsis>
529            </term>
530            <listitem>
531              <para>
532                Prepare the plug-in for execution or configuration. If the plug-in needs
533                to do some initialization this is the place to do it. A typical
534                implementation however only stores the passed parameters in instance
535                variables for later use. Since it is not possible to know what the user is going to
536                do at this stage, we recommend lazy initialisation of all other resources.
537              </para>
538              <para>
539                The parameters passed to this method has vital information that is
540                needed to execute the plug-in. The
541                <classname docapi="net.sf.basedb.core">SessionControl</classname>
542                is a central core object holding information about the logged in
543                user and is used to create
544                <classname docapi="net.sf.basedb.core">DbControl</classname>
545                objects which allows a plug-in to connect to the database to read, add or
546                update information. The two
547                <classname docapi="net.sf.basedb.core.plugin">ParameterValues</classname>
548                objects contain information about the configuration and job
549                parameters to the plug-in.
550                The <varname>configuration</varname> object holds all parameters stored
551                together with a <classname docapi="net.sf.basedb.core">PluginConfiguration</classname>
552                object in the database. If the plug-in is started without
553                a configuration this object is null.
554                The <varname>job</varname> object holds all parameters that are
555                stored together with a <classname docapi="net.sf.basedb.core">Job</classname> object in the
556                database. This object is null if the plug-in is started without a job.
557              </para>
558              <para>
559                The difference between a configuration parameter and a job parameter is
560                that a configuration is usually something an administrator sets up,
561                while a job is an actual execution of a plug-in. For example, a
562                configuration for an import plug-in holds the regular expressions needed
563                to parse a text file and find the headers, sections and data lines,
564                while the job holds the file to parse.
565              </para>
566              <para>
567                The
568                <classname docapi="net.sf.basedb.core.plugin">AbstractPlugin</classname>
569                contains an implementation of this method that saves the passed objects
570                in protected instance variables. If you override this method
571                we recommend that you also call <code>super.init()</code>.
572              </para>
573              <example id="net.sf.basedb.core.plugin.Plugin.init">
574                <title>The <classname docapi="net.sf.basedb.core.plugin">AbstractPlugin</classname> implementation of Plugin.init()</title>
575<programlisting language="java">
576protected SessionControl sc = null;
577protected ParameterValues configuration = null;
578protected ParameterValues job = null;
579/**
580   Store copies of the session control, plug-in and job configuration. These
581   are available to subclasses in the {@link #sc}, {@link #configuration}
582   and {@link #job} variables. If a subclass overrides this method it is
583   recommended that it also calls super.init(sc, configuration, job).
584*/
585public void init(SessionControl sc,
586   ParameterValues configuration, ParameterValues job)
587   throws BaseException
588{
589   this.sc = sc;
590   this.configuration = configuration;
591   this.job = job;
592}</programlisting>
593              </example>
594            </listitem>
595          </varlistentry>
596          <varlistentry>
597            <term>
598              <methodsynopsis language="java">
599                <modifier>public</modifier>
600                <void />
601                <methodname>run</methodname>
602                <methodparam>
603                  <type>Request</type>
604                  <parameter>request</parameter>
605                </methodparam>
606                <methodparam>
607                  <type>Response</type>
608                  <parameter>response</parameter>
609                </methodparam>
610                <methodparam>
611                  <type>ProgressReporter</type>
612                  <parameter>progress</parameter>
613                </methodparam>
614              </methodsynopsis>
615            </term>
616            <listitem>
617              <para>
618                Run the plug-in.
619              </para>
620              <para>
621                The <varname>request</varname>
622                parameter is of historical interest only. It has no useful information
623                and can be ignored.
624              </para>
625              <para>
626                The <varname>progress</varname> parameter
627                can be used by a plug-in to report its progress back to the core. The
628                core will usually send the progress information to the database, which
629                allows users to see exactly how the plug-in is progressing from the web
630                interface. This parameter can be null, but if it is not we recommend all
631                plug-ins to use it. However, it should be used sparingly, since each call
632                to set the progress may result in a database update. If the execution
633                involves several thousands of items it is a bad idea to update the
634                progress after processing each one of them. A good starting point is to
635                divide the work into 100 pieces each representing 1% of the work, i.e.,
636                if the plug-in should export 100 000 items it should report progress
637                after every 1000 items.
638              </para>
639              <para>
640                The <varname>response</varname>
641                parameter is used to tell the core if the plug-in was successful or
642                failed. Not setting a response is considered a failure by the core. From
643                the run method it is only allowed to use on of the
644                <methodname>Response.setDone()</methodname>,
645                <methodname>Response.setError()</methodname> or
646                <methodname>Response.setContinue()</methodname>
647                methods.
648              </para>
649             
650              <important>
651                It is also considered bad practice to let exceptions escape
652                out from this method. Always use <code>try...catch</code>
653                to catch exceptions and use <code>Response.setError()</code>
654                to report the error back to the core.
655              </important>
656             
657              <example id="net.sf.basedb.core.plugin.Plugin.run">
658                <title>
659                  Here is a skeleton that we recommend each plug-in to use in its
660                  implementation of the
661                  <methodname>run()</methodname>
662                  method
663                </title>
664<programlisting language="java">
665public void run(Request request, Response response, ProgressReporter progress)
666{
667   // Open a connection to the database
668   // sc is set by init() method
669   DbControl dc = sc.newDbControl();
670   try
671   {
672      // Insert code for plug-in here
673
674      // Commit the work
675      dc.commit();
676      response.setDone("Plug-in ended successfully");
677   }
678   catch (Throwable t)
679   {
680      // All exceptions must be catched and sent back
681      // using the response object
682      response.setError(t.getMessage(), Arrays.asList(t));
683   }
684   finally
685   {
686      // IMPORTANT!!! Make sure opened connections are closed
687      if (dc != null) dc.close();
688   }
689}</programlisting>
690              </example>
691            </listitem>
692          </varlistentry>
693          <varlistentry>
694            <term>
695              <methodsynopsis language="java">
696                <modifier>public</modifier>
697                <void />
698                <methodname>done</methodname>
699                <void />
700              </methodsynopsis>
701            </term>
702            <listitem>
703              <para>
704                Clean up all resources after executing the plug-in. This method must not
705                throw any exceptions.
706              </para>
707              <example id="net.sf.basedb.core.plugin.Plugin.done">
708                <title>
709                  The
710                  <classname docapi="net.sf.basedb.core.plugin">AbstractPlugin</classname>
711                  contains an implementation of the <methodname>done()</methodname>
712                  method simply sets the
713                  parameters passed to the
714                  <methodname>init()</methodname>
715                  method to null
716                </title>
717<programlisting language="java">
718/**
719   Clears the variables set by the init method. If a subclass
720   overrides this method it is recommended that it also calls super.done().
721*/
722public void done()
723{
724   configuration = null;
725   job = null;
726   sc = null;
727}</programlisting>
728              </example>
729            </listitem>
730          </varlistentry>
731        </variablelist>
732      </sect3>
733 
734      <sect3 id="plugin_developer.api.interfaces.interactive">
735        <title>The net.sf.basedb.core.plugin.InteractivePlugin interface</title>
736        <para>
737          If you want the plug-in to be able to interact with the user you must also implement
738          this interface. This is probably the case for most plug-ins. Among the core plug-ins
739          shipped with BASE the
740          <classname docapi="net.sf.basedb.plugins">SpotImageCreator</classname>
741          is one plug-in that does not interact with the user. Instead, the web client has
742          special JSP pages that handles all the interaction, creates a job for it and sets
743          the parameters. This, kind of hardcoded, approach can also be used for other
744          plug-ins, but then it usually requires modification of the client application
745          as well.
746        </para>
747        <para>
748          The
749          <interfacename docapi="net.sf.basedb.core.plugin">InteractivePlugin</interfacename>
750          has three main tasks:
751         
752          <orderedlist>
753          <listitem>
754            <para>
755            Tell a client application where the plug-in should be plugged
756            in.
757            </para>
758          </listitem>
759          <listitem>
760            <para>
761            Ask the users for configuration and job parameters.
762            </para>
763          </listitem>
764         
765          <listitem>
766            <para>
767            Validate parameter values entered by the user and store those in the
768            database.
769            </para>
770          </listitem>
771          </orderedlist>
772          This requires that the following methods are
773          implemented.
774        </para>
775       
776        <variablelist>
777          <varlistentry>
778            <term>
779              <methodsynopsis language="java">
780                <modifier>public</modifier>
781                <type>Set&lt;GuiContext&gt;</type>
782                <methodname>getGuiContexts</methodname>
783                <void />
784              </methodsynopsis>
785            </term>
786            <listitem>
787              <para>
788                Return information about where the plug-in should be plugged in. Each
789                place is identified by a
790                <classname docapi="net.sf.basedb.core.plugin">GuiContext</classname>
791                object, which is an
792                <classname docapi="net.sf.basedb.core">Item</classname>
793                and a
794                <classname docapi="net.sf.basedb.core">Type</classname>.
795                The item is one of the objects defined by the
796                <classname docapi="net.sf.basedb.core">Item</classname>
797                enumeration and the type is either
798                <constant>Type.LIST</constant>
799                or
800                <constant>Type.ITEM</constant>, which corresponde to the
801                list view and the single-item view in the web client.
802              </para>
803              <para>
804                For example, the
805                <varname>GuiContext</varname> =
806                (<constant>Item.REPORTER</constant>,
807                <constant>Type.LIST</constant>)
808                tells a client application that this plug-in can be plugged in whenever a
809                list of reporters is displayed. The
810                <varname>GuiContext</varname> =
811                (<constant>Item.REPORTER</constant>,
812                <constant>Type.ITEM</constant>)
813                tells a client application that this plug-in can be plugged in whenever
814                a single reporter is displayed. The first case may be appropriate for a
815                plug-in that imports or exports reporters. The second case may be used by
816                a plug-in that updates the reporter information from an external source
817                (well, it may make sense to use this in the list case as well).
818              </para>
819              <para>
820                The returned information is copied by the core at installation time to
821                make it easy to ask for all plug-ins for a certain
822                <classname docapi="net.sf.basedb.core.plugin">GuiContext</classname>.
823              </para>
824              <para>
825                A typical implementation creates a static unmodifiable
826                <classname>Set</classname>
827                which is returned by this method. It is important that the returned set
828                cannot be modified. It may be a security issue if a misbehaving
829                client application does that.
830              </para>
831              <example id="net.sf.basedb.core.plugin.InteractivePlugin.getGuiContexts">
832                <title>
833                  A typical implementation of
834                  <methodname>getGuiContexts</methodname>
835                </title>
836<programlisting language="java">
837// From the net.sf.basedb.plugins.RawDataFlatFileImporter plug-in
838private static final Set&lt;GuiContext&gt; guiContexts =
839   Collections.singleton(new GuiContext(Item.RAWBIOASSAY, GuiContext.Type.ITEM));
840
841public Set&lt;GuiContext&gt; getGuiContexts()
842{
843   return guiContexts;
844}</programlisting>
845              </example>
846            </listitem>
847          </varlistentry>
848          <varlistentry>
849            <term>
850              <methodsynopsis language="java">
851                <modifier>public</modifier>
852                <type>String</type>
853                <methodname>isInContext</methodname>
854                <methodparam>
855                  <type>GuiContext</type>
856                  <parameter>context</parameter>
857                </methodparam>
858                <methodparam>
859                  <type>Object</type>
860                  <parameter>item</parameter>
861                </methodparam>
862              </methodsynopsis>
863            </term>
864            <listitem>
865              <para>
866                This method is called to check if a particular item is usable for the
867                plug-in. This method is invoked to check if a plug-in can be used
868                in a given context. If invoked from a list context the <parameter>item</parameter>
869                parameter is <constant>null</constant>
870                The plug-in should return <constant>null</constant> if it
871                finds that it can be used. If the plug-in can't be used it
872                must decide if the reason should be a warning or an error condition.
873              </para>
874             
875              <para>
876                A warning is issued by returning a string with the warning
877                message. It should be used when the plug-in can't be used because
878                it is unrelated to the current task. For example, a plug-in for
879                importing Genepix data should return a warning when somebody wants
880                to import data to an Agilent raw bioassay.
881              </para>
882             
883              <para>
884                An error message is issued by throwing an exception. This
885                should be used when the plug-in is related to the current task
886                but still can't do what it is supposed to do. For example,
887                trying to import raw data if the logged in user doesn't have
888                write permission to the raw bioassay.
889              </para>
890             
891              <para>
892                As a rule of thumb, if there is a chance that another plug-in
893                might be able to perform the same task a warning should be used.
894                If it is guaranteed that no other plug-in can do it an error
895                message should be used.
896              </para>
897             
898              <para>
899                Here is a real example from the
900                <classname docapi="net.sf.basedb.plugins">RawDataFlatFileImporter</classname>
901                plug-in which imports raw data to a
902                <classname docapi="net.sf.basedb.core">RawBioAssay</classname>.
903               
904                Thus,
905                <varname>GuiContext</varname> =
906                (<constant>Item.RAWBIOASSAY</constant>,
907                <constant>Type.ITEM</constant>),
908               
909                but the plug-in can only import data if the logged in user has write permission,
910                there is no data already, and
911                if the raw bioassay has the same raw data type as the plug-in has been
912                configured for.
913              </para>
914              <example id="net.sf.basedb.core.plugin.InteractivePlugin.isInContext">
915                <title>
916                  A realistic implementation of the
917                  <methodname>isInContext()</methodname> method
918                </title>
919<programlisting language="java">
920/**
921   Returns null if the item is a {@link RawBioAssay} of the correct
922   {@link RawDataType} and doesn't already have spots.
923   @throws PermissionDeniedException If the raw bioasssay already has raw data
924   or if the logged in user doesn't have write permission
925*/
926public String isInContext(GuiContext context, Object item)
927{
928   String message = null;
929   if (item == null)
930   {
931      message = "The object is null";
932   }
933   else if (!(item instanceof RawBioAssay))
934   {
935      message = "The object is not a RawBioAssay: " + item;
936   }
937   else
938   {
939      RawBioAssay rba = (RawBioAssay)item;
940      String rawDataType = (String)configuration.getValue("rawDataType");
941      RawDataType rdt = rba.getRawDataType();
942      if (!rdt.getId().equals(rawDataType))
943      {
944         // Warning
945         message = "Unsupported raw data type: " + rba.getRawDataType().getName();
946      }
947      else if (!rdt.isStoredInDb())
948      {
949         // Warning
950         message = "Raw data for raw data type '" + rdt + "' is not stored in the database";
951      }
952      else if (rba.hasData())
953      {
954         // Error
955         throw new PermissionDeniedException("The raw bioassay already has data.");
956      }
957      else
958      {
959         // Error
960         rba.checkPermission(Permission.WRITE);
961      }
962   }
963   return message;   
964}</programlisting>
965              </example>
966            </listitem>
967          </varlistentry>
968          <varlistentry>
969            <term>
970              <methodsynopsis language="java">
971                <modifier>public</modifier>
972                <type>RequestInformation</type>
973                <methodname>getRequestInformation</methodname>
974                <methodparam>
975                  <type>GuiContext</type>
976                  <parameter>context</parameter>
977                </methodparam>
978                <methodparam>
979                  <type>String</type>
980                  <parameter>command</parameter>
981                </methodparam>
982                <exceptionname>BaseException</exceptionname>
983              </methodsynopsis>
984            </term>
985            <listitem>
986              <para>
987                Ask the plug-in for parameters that need to be entered by the user. The
988                <classname docapi="net.sf.basedb.core.plugin">GuiContext</classname>
989                parameter is one of the contexts returned by the
990                <methodname>getGuiContexts</methodname>
991                method. The command is a string telling the plug-in what command was
992                executed. There are two predefined commands but as you will see the
993                plug-in may define its own commands. The two predefined commands are
994                defined in the
995                <classname docapi="net.sf.basedb.core.plugin">net.sf.basedb.core.plugin.Request</classname>
996                class.
997                <variablelist>
998                  <varlistentry>
999                    <term>
1000                      <constant>Request.COMMAND_CONFIGURE_PLUGIN</constant>
1001                    </term>
1002                    <listitem>
1003                      <para>
1004                        Used when an administrator is initiating a configuration
1005                        of the plug-in.
1006                      </para>
1007                    </listitem>
1008                  </varlistentry>
1009                  <varlistentry>
1010                    <term>
1011                      <constant>Request.COMMAND_CONFIGURE_JOB</constant>
1012                    </term>
1013                    <listitem>
1014                      <para>
1015                        Used when a user has selected the plug-in for running a
1016                        job.
1017                      </para>
1018                    </listitem>
1019                  </varlistentry>
1020                </variablelist>
1021                Given this information the plug-in must return a
1022                <classname docapi="net.sf.basedb.core">RequestInformation</classname>
1023                object. This is simply a title, a description, and a list of parameters.
1024                Usually the title will end up as the input form title and the
1025                description as a help text for the entire form. Do not put information
1026                about the individual parameters in this description, since each
1027                parameter has a description of its own.
1028              </para>
1029              <example id="net.sf.basedb.core.plugin.InteractivePlugin.getRequestInformation">
1030                <title>
1031                  When running an import plug-in it needs to ask for the file to import
1032                  from and if existing items should be updated or not
1033                </title>
1034<programlisting language="java">
1035// The complete request information
1036private RequestInformation configure Job;
1037
1038// The parameter that asks for a file to import from
1039private PluginParameter&lt;File&gt; file Parameter;
1040
1041// The parameter that asks if existing items should be updated or not
1042private PluginParameter&lt;Boolean&gt; updateExistingParameter;
1043
1044public RequestInformation getRequestInformation(GuiContext context, String command)
1045   throws BaseException
1046{
1047   RequestInformation requestInformation = null;
1048   if (command.equals(Request.COMMAND_CONFIGURE_PLUGIN))
1049   {
1050      requestInformation = getConfigurePlugin();
1051   }
1052   else if (command.equals(Request.COMMAND_CONFIGURE_JOB))
1053   {
1054      requestInformation = getConfigureJob();
1055   }
1056   return requestInformation;
1057}
1058
1059/**
1060   Get (and build) the request information for starting a job.
1061*/
1062private RequestInformation getConfigureJob()
1063{
1064   if (configureJob == null)
1065   {
1066      // A file is required
1067      fileParameter = new PluginParameter&lt;File&gt;(
1068         "file",
1069         "File",
1070         "The file to import the data from",
1071         new FileParameterType(null, true, 1)
1072      );
1073     
1074      // The default value is 'false'
1075      updateExistingParameter = new PluginParameter&lt;Boolean&gt;(
1076         "updateExisting",
1077         "Update existing items",
1078         "If this option is selected, already existing items will be updated " +
1079         " with the information in the file. If this option is not selected " +
1080         " existing items are left untouched.",
1081         new BooleanParameterType(false, true)
1082      );
1083
1084      List&lt;PluginParameter&lt;?&gt;&gt; parameters =
1085         new ArrayList&lt;PluginParameter&lt;?&gt;&gt;(2);
1086      parameters.add(fileParameter);
1087      parameters.add(updateExistingParameter);
1088     
1089      configureJob = new RequestInformation
1090      (
1091         Request.COMMAND_CONFIGURE_JOB,
1092         "Select a file to import items from",
1093         "Description",
1094         parameters
1095      );
1096   }
1097   return configureJob;
1098}</programlisting>
1099              </example>
1100              <para>
1101                As you can see it takes a lot of code to put together a
1102                <classname docapi="net.sf.basedb.core">RequestInformation</classname>
1103                object. For each parameter you need one
1104                <classname docapi="net.sf.basedb.core">PluginParameter</classname>
1105                object and one
1106                <classname docapi="net.sf.basedb.core">ParameterType</classname>
1107                object. To make life a little easier, a
1108                <classname docapi="net.sf.basedb.core">ParameterType</classname>
1109                can be reused for more than one
1110                <classname docapi="net.sf.basedb.core">PluginParameter</classname>.
1111              </para>
1112             
1113<programlisting language="java">
1114StringParameterType stringPT = new StringParameterType(255, null, true);
1115PluginParameter one = new PluginParameter("one", "One", "First string", stringPT);
1116PluginParameter two = new PluginParameter("two", "Two", "Second string", stringPT);
1117// ... and so on
1118</programlisting>
1119              <para>
1120                The
1121                <classname docapi="net.sf.basedb.core">ParameterType</classname>
1122                is an abstract base class for several subclasses each implementing a
1123                specific type of parameter. The list of subclasses may grow in the
1124                future, but here are the most important ones currently implemented.
1125              </para>
1126              <note>
1127                <para>
1128                  Most parameter types include support for supplying a predefined list
1129                  of options to select from. In that case the list will be displayed
1130                  as a drop-down list for the user, otherwise a free input field is
1131                  used.
1132                </para>
1133              </note>
1134              <variablelist>
1135                <varlistentry>
1136                  <term>
1137                    <classname docapi="net.sf.basedb.core">StringParameterType</classname>
1138                  </term>
1139                  <listitem>
1140                    <para>
1141                      Asks for a string value. Includes an option for
1142                      specifying the maximum length of the string.
1143                    </para>
1144                  </listitem>
1145                </varlistentry>
1146                <varlistentry>
1147                  <term>
1148                    <classname docapi="net.sf.basedb.core">FloatParameterType</classname>,
1149                    <classname docapi="net.sf.basedb.core">DoubleParameterType</classname>,
1150                    <classname docapi="net.sf.basedb.core">IntegerParameterType</classname>,
1151                    <classname docapi="net.sf.basedb.core">LongParameterType</classname>
1152                  </term>
1153                  <listitem>
1154                    <para>
1155                      Asks for numerical values. Includes options for
1156                      specifying a range (min/max) of allowed values.
1157                    </para>
1158                  </listitem>
1159                </varlistentry>
1160                <varlistentry>
1161                  <term>
1162                    <classname docapi="net.sf.basedb.core">BooleanParameterType</classname>
1163                  </term>
1164                  <listitem>
1165                    <para>Asks for a boolean value.
1166                    </para>
1167                  </listitem>
1168                </varlistentry>
1169                <varlistentry>
1170                  <term>
1171                    <classname docapi="net.sf.basedb.core">DateParameterType</classname>
1172                  </term>
1173                  <listitem>
1174                    <para>Asks for a date.
1175                    </para>
1176                  </listitem>
1177                </varlistentry>
1178                <varlistentry>
1179                  <term>
1180                    <classname docapi="net.sf.basedb.core">FileParameterType</classname>
1181                  </term>
1182                  <listitem>
1183                    <para>Asks for a file item.
1184                    </para>
1185                  </listitem>
1186                </varlistentry>
1187                <varlistentry>
1188                  <term>
1189                    <classname docapi="net.sf.basedb.core">ItemParameterType</classname>
1190                  </term>
1191                  <listitem>
1192                    <para>
1193                      Asks for any other item. This parameter type requires
1194                      that a list of options is supplied, except when the item
1195                      type asked for matches the current <classname docapi="net.sf.basedb.core.plugin">GuiContext</classname>, in which
1196                      case the currently selected item is used as the
1197                      parameter value.
1198                    </para>
1199                  </listitem>
1200                </varlistentry>
1201                <varlistentry>
1202                  <term>
1203                    <classname docapi="net.sf.basedb.core">PathParameterType</classname>
1204                  </term>
1205                  <listitem>
1206                    <para>
1207                      Ask for a path to a file or directory. The path may be
1208                      non-existing and should be used when a plug-in needs an
1209                      output destination, i.e., the file to export to, or a
1210                      directory where the output files should be placed.
1211                    </para>
1212                  </listitem>
1213                </varlistentry>
1214              </variablelist>
1215              <para>
1216                You can also create a
1217                <classname docapi="net.sf.basedb.core">PluginParameter</classname>
1218                with a null name and
1219                <classname docapi="net.sf.basedb.core">ParameterType</classname>.
1220                In this case, the web client will not ask for input from the user, instead
1221                it is used as a section header, allowing you to group parameters into
1222                different sections which increase the readability of the input
1223                parameters page.
1224              </para>
1225<programlisting language="java">
1226PluginParameter firstSection = new PluginParameter(null, "First section", null, null);
1227PluginParameter secondSection = new PluginParameter(null, "Second section", null, null);
1228// ...
1229
1230parameters.add(firstSection);
1231parameters.add(firstParameterInFirstSection);
1232parameters.add(secondParameteInFirstSection);
1233
1234parameters.add(secondSection);
1235parameters.add(firstParameterInSecondSection);
1236parameters.add(secondParameteInSecondSection);
1237</programlisting>
1238            </listitem>
1239          </varlistentry>
1240          <varlistentry>
1241            <term>
1242              <methodsynopsis language="java">
1243                <modifier>public</modifier>
1244                <void />
1245                <methodname>configure</methodname>
1246                <methodparam>
1247                  <type>GuiContext</type>
1248                  <parameter>context</parameter>
1249                </methodparam>
1250                <methodparam>
1251                  <type>Request</type>
1252                  <parameter>request</parameter>
1253                </methodparam>
1254                <methodparam>
1255                  <type>Response</type>
1256                  <parameter>response</parameter>
1257                </methodparam>
1258              </methodsynopsis>
1259            </term>
1260            <listitem>
1261              <para>
1262                Sends parameter values entered by the user for processing by the plug-in.
1263                The plug-in must validate that the parameter values are
1264                correct and then store them in database.
1265              </para>
1266             
1267              <important>
1268                <para>
1269                No validation is done by the core, except converting the input to the
1270                correct object type, i.e. if the plug-in asked for a
1271                <classname>Float</classname>
1272                the input string is parsed and converted to a
1273                <classname>Float</classname>. If you have extended the
1274                <classname docapi="net.sf.basedb.core.plugin">AbstractPlugin</classname>
1275                class it is very easy to validate the parameters with the
1276                <methodname>AbstractPlugin.validateRequestParameters()</methodname>
1277                method. This method takes the same list of
1278                <classname docapi="net.sf.basedb.core">PluginParameter</classname>:s
1279                as used in the
1280                <classname docapi="net.sf.basedb.core">RequestInformation</classname>
1281                object and uses that information for validation. It returns null or a
1282                list of
1283                <exceptionname>Throwable</exceptionname>:s that can be
1284                given directly to the <code>response.setError()</code>
1285                methods.
1286                </para>
1287              </important>
1288              <para>
1289                When the parameters have been validated, they need to be stored
1290                in the database. Once again, it is very easy, if you use one of the
1291                <methodname>AbstractPlugin.storeValue()</methodname>
1292                or
1293                <methodname>AbstractPlugin.storeValues()</methodname>
1294                methods.
1295              </para>
1296              <para>
1297                The configure method works much like the <methodname>Plugin.run()</methodname> 
1298                method. It must return the result in the <classname docapi="net.sf.basedb.core.plugin">Response</classname> object,
1299                and should not throw any exceptions.
1300              </para>
1301             
1302              <example id="net.sf.basedb.core.plugin.InteractivePlugin.configure">
1303                <title>
1304                  Configuration implementation building on the examples above
1305                </title>
1306<programlisting language="java">
1307public void configure(GuiContext context, Request request, Response response)
1308{
1309   String command = request.getCommand();
1310   try
1311   {
1312      if (command.equals(Request.COMMAND_CONFIGURE_PLUGIN))
1313      {
1314         // TODO
1315      }
1316      else if (command.equals(Request.COMMAND_CONFIGURE_JOB))
1317      {
1318         // Validate user input
1319         List&lt;Throwable&gt; errors =
1320            validateRequestParameters(getConfigureJob().getParameters(), request);
1321         if (errors != null)
1322         {
1323            response.setError(errors.size() +
1324               " invalid parameter(s) were found in the request", errors);
1325            return;
1326         }
1327         
1328         // Store user input
1329         storeValue(job, request, fileParameter);
1330         storeValue(job, request, updateExistingParameter);
1331         
1332         // We are happy and done
1333         File file = (File)job.getValue("file");
1334         response.setSuggestedJobName("Import data from file " + file.getName());
1335         response.setDone("Job configuration complete", Job.ExecutionTime.SHORT);
1336      }
1337   }
1338   catch (Throwable ex)
1339   {
1340      response.setError(ex.getMessage(), Arrays.asList(ex));
1341   }
1342}</programlisting>
1343              </example>
1344             
1345              <para>
1346                Note that the call to
1347                <methodname>response.setDone()</methodname>
1348                has a second parameter
1349                <constant>Job.ExecutionTime.SHORT</constant>. It is an indication
1350                about how long time it will take to execute the
1351                plug-in. This is of interest for job queue managers which probably
1352                does not want to start too many long-running jobs at the same time
1353                blocking the entire system. Please try to use this parameter wisely and
1354                not use <constant>Job.ExecutionTime.SHORT</constant>
1355                out of old habit all the time.
1356              </para>
1357              <para>
1358                The <classname docapi="net.sf.basedb.core.plugin">Response</classname> class also has a
1359                <methodname>setContinue()</methodname>
1360                method which tells the core that the plug-in needs more parameters,
1361                i.e. the core will then call
1362                <methodname>getRequestInformation()</methodname>
1363                again with the new command, let the user enter values, and then call
1364                <methodname>configure()</methodname>
1365                with the new values. This process is repeated until the plug-in
1366                reports that it is done or an error occurs.
1367              </para>
1368              <tip>
1369                <para>
1370                You do not have to store all values the plug-in asked for in the
1371                first place. You may even choose to store different values than
1372                those that were entered. For example, you might ask for the mass
1373                and height of a person and then only store the body mass index,
1374                which is calculated from those values.
1375                </para>
1376              </tip>
1377              <para>
1378                An important note is that during this iteration it is the same instance
1379                of the plug-in that is used. However, no parameter values are stored in
1380                the database until the plugin sends a
1381                <methodname>response.setDone()</methodname>.
1382                After that, the plug-in instance is usually discarded, and a job is placed
1383                in the job queue. The execution of the plug-in happens in a new instance
1384                and maybe on a different server. This means that a plug-in can't store
1385                state from the configuration phase internally and expect it to be there
1386                in the execution phase. Everything the plug-in needs to do it's job must
1387                be stored as parameters in the database.
1388              </para>
1389              <para>
1390                The only exception to the above rule is if the plug-in answers with
1391                <methodname>Response.setExecuteImmediately()</methodname>
1392                or <methodname>Response.setDownloadImmediately()</methodname>.
1393                Doing so bypasses the entire job queue system and requests that the
1394                job is started immediately. This is a permission that has to
1395                be granted to each plug-in by the server administrator. If the
1396                plug-in has this permission, the same object instance that
1397                was used in the configuration phase is also used in the execution
1398                phase. This is the only case where a plug-in can retain internal
1399                state between the two phases.
1400              </para>
1401            </listitem>
1402          </varlistentry>
1403        </variablelist>
1404      </sect3>
1405   
1406    </sect2>
1407 
1408    <sect2 id="plugin_developer.api.callsequence">
1409      <title>How the BASE core interacts with the plug-in when...</title>
1410     
1411      <para>
1412        This section describes how the BASE core interacts with the
1413        plug-in in a number of use cases. We will outline the
1414        order the methods are invoked on the plug-in.
1415      </para>
1416     
1417      <sect3 id="plugin_developer.api.callsequence.install">
1418        <title>Installing a plug-in</title>
1419       
1420        <para>
1421          When a plug-in is installed the core is eager to find out
1422          information about the plug-in. To do this it calls the
1423          following methods in this order:
1424        </para>
1425       
1426        <orderedlist>
1427        <listitem>
1428          <para>
1429          A new instance of the plug-in class is created. The plug-in must
1430          have a public no-argument constructor.
1431          </para>
1432        </listitem>
1433       
1434        <listitem>
1435          <para>
1436          Calls are made to <methodname>Plugin.getMainType()</methodname>,
1437          <methodname>Plugin.supportsConfigurations()</methodname> and
1438          <methodname>Plugin.requiresConfiguration()</methodname> to find out information
1439          about the plug-in. This is the only time these methods are called.
1440          The information that is returned by them are copied and stored in
1441          the database for easy access.
1442          </para>
1443         
1444          <note>
1445            <para>
1446            The <methodname>Plugin.init()</methodname> method is
1447            never called during plug-in installation.
1448            </para>
1449          </note>
1450        </listitem>
1451       
1452        <listitem>
1453          <para>
1454          If the plug-in implements the <interfacename docapi="net.sf.basedb.core.plugin">InteractivePlugin</interfacename>
1455          interface the <methodname>InteractivePlugin.getGuiContexts()</methodname>
1456          method is called. This is the only time this method is called and the information it
1457          returns are copied and stored in the database.
1458          </para>
1459        </listitem>
1460       
1461        <listitem>
1462          <para>
1463          If the server admin decided to use the plug-in permission system, the
1464          <methodname>Plugin.getPermissions()</methodname> method is called.
1465          The returned information is copied and stored in the database.
1466          </para>
1467        </listitem>
1468       
1469        </orderedlist>
1470      </sect3>
1471   
1472      <sect3 id="plugin_developer.api.callsequence.configure">
1473        <title>Configuring a plug-in</title>
1474       
1475        <para>
1476          The plug-in must implement the <interfacename docapi="net.sf.basedb.core.plugin">InteractivePlugin</interfacename>
1477          interface and the <methodname>Plugin.supportsConfigurations()</methodname> method
1478          must return <constant>TRUE</constant>. The configuration is done with
1479          a wizard-like interface (see <xref linkend="plugins.configuration.wizard" />).
1480          The same plug-in instance is used throughout the entire configuration sequence.
1481        </para>
1482       
1483        <orderedlist>
1484        <listitem>
1485          <para>
1486          A new instance of the plug-in class is created. The plug-in must
1487          have a public no-argument constructor.
1488          </para>
1489        </listitem>
1490       
1491        <listitem>
1492          <para>
1493          The <methodname>Plugin.init()</methodname> method is called. The
1494          <varname>job</varname> parameter is null.
1495          </para>
1496        </listitem>
1497       
1498        <listitem id="plugin_developer.api.callsequence.configure.requestinformation">
1499          <para>
1500          The <methodname>InteractivePlugin.getRequestInformation()</methodname> 
1501          method is called. The <varname>context</varname> parameter is <constant>null</constant>
1502          and the <varname>command</varname> is the value of the string
1503          constant <constant>Request.COMMAND_CONFIGURE_PLUGIN</constant> 
1504          (<constant>_config_plugin</constant>).
1505          </para>
1506        </listitem>
1507       
1508        <listitem id="plugin_developer.api.callsequence.configure.form">
1509          <para>
1510          The web client process the returned information and displays a form for user
1511          input. The plug-in will have to wait some time while the user enters
1512          data.
1513          </para>
1514        </listitem>
1515       
1516        <listitem>
1517          <para>
1518          The <methodname>InteractivePlugin.configure()</methodname> method
1519          is called. The <varname>context</varname> parameter is still
1520          <constant>null</constant> and the <varname>request</varname>
1521          parameter contains the parameter values entered by the user.
1522          </para>
1523        </listitem>
1524       
1525        <listitem>
1526          <para>
1527          The plug-in must validate the values and decide whether they should be
1528          stored in the database or not. We recommend that you use the
1529          methods in the <classname docapi="net.sf.basedb.core.plugin">AbstractPlugin</classname> class for this.
1530          </para>
1531        </listitem>
1532       
1533        <listitem>
1534          <para>
1535          The plug-in can choose between three different respones:
1536         
1537          <itemizedlist>
1538          <listitem>
1539            <para>
1540            <methodname>Response.setDone()</methodname>: The configuration
1541            is complete. The core will write any configuation changes to the
1542            database, call the <methodname>Plugin.done()</methodname> method and
1543            then discard the plug-in instance.
1544            </para>
1545          </listitem>
1546         
1547          <listitem>
1548            <para>
1549            <methodname>Response.setError()</methodname>: There was one or more
1550            errors. The web client will display the error messages for the user and
1551            allow the user to enter new values. The process continues with
1552            step <xref linkend="plugin_developer.api.callsequence.configure.form" />.
1553            </para>
1554          </listitem>
1555         
1556          <listitem>
1557            <para>
1558            <methodname>Response.setContinue()</methodname>: The parameters are correct
1559            but the plug-in wants more parameters. The process continues with
1560            step <xref linkend="plugin_developer.api.callsequence.configure.requestinformation" />
1561            but the <varname>command</varname> has the value that was passed to the
1562            <methodname>setContinue()</methodname> method.
1563            </para>
1564          </listitem>
1565          </itemizedlist>
1566         
1567          </para>
1568        </listitem>
1569        </orderedlist>
1570
1571      </sect3>   
1572   
1573      <sect3 id="plugin_developer.api.callsequence.context">
1574        <title>Checking if a plug-in can be used in a given context</title>
1575       
1576        <para>
1577          If the plug-in is an <interfacename docapi="net.sf.basedb.core.plugin">InteractivePlugin</interfacename>
1578          it has specified in which contexts it can be used by the
1579          information returned from <methodname>InteractivePlugin.getGuiContexts()</methodname>
1580          method. The web client uses this information to decide whether,
1581          for example, a <guibutton>Run plugin</guibutton>
1582          button should be displayed on a page or not. However, this is not
1583          always enough to know whether the plug-in can be used or not.
1584          For example, a raw data importer plug-in cannot be used to
1585          import raw data if the raw bioassay already has data.
1586          So, when the user clicks the button, the web client will
1587          load all plug-ins that possibly can be used in the given context
1588          and let each one of them check whether they can be used or not.
1589        </para>
1590       
1591     
1592        <orderedlist>
1593        <listitem>
1594          <para>
1595          A new instance of the plug-in class is created. The plug-in must
1596          have a public no-argument constructor.
1597          </para>
1598        </listitem>
1599       
1600        <listitem>
1601          <para>
1602          The <methodname>Plugin.init()</methodname> method is called.
1603          The <varname>job</varname> parameter is <constant>null</constant>.
1604          The <varname>configuration</varname> parameter is <constant>null</constant>
1605          if the plug-in does not have any configuration parameters.
1606          </para>
1607        </listitem>
1608       
1609        <listitem>
1610          <para>
1611          The <methodname>InteractivePlugin.isInContext()</methodname>
1612          is called. If the context is a list context, the <varname>item</varname>
1613          parameter is null, otherwise the current item is passed. The plug-in
1614          should return <constant>null</constant> if it can be used under the
1615          current circumstances, or a message explaining why not.
1616          </para>
1617        </listitem>
1618       
1619        <listitem>
1620          <para>
1621            After this, <methodname>Plugin.done()</methodname> is called and
1622            the plug-in instance is discarded. If there are
1623            several configurations for a plug-in, this procedure is repeated
1624            for each configuration.
1625          </para>
1626        </listitem>
1627        </orderedlist>
1628      </sect3>
1629   
1630      <sect3 id="plugin_developer.api.callsequence.job">
1631        <title>Creating a new job</title>
1632       
1633        <para>
1634          If the web client found that the plug-in could be
1635          used in a given context and the user selected the plug-in,
1636          the job configuration sequence is started. It is a wizard-like interface
1637          identical to the configuration wizard. In fact, the same JSP pages,
1638          and calling sequence is used. See <xref linkend="plugin_developer.api.callsequence.configure" />.
1639          We do not repeat everything here. There are a few differences:
1640        </para>
1641       
1642        <itemizedlist>
1643        <listitem>
1644          <para>
1645          The <varname>job</varname> parameter is not null, but it does not contain
1646          any parameter values to start with. The plug-in should use this
1647          object to store job-related parameter values. The
1648          <varname>configuration</varname> parameter is <constant>null</constant> 
1649          if the plug-in is started without configuration. In any case,
1650          the configuration values are write-protected and cannot be modified.
1651          </para>
1652        </listitem>
1653       
1654        <listitem>
1655          <para>
1656          The first call to <methodname>InteractivePlugin.getRequestInformation()</methodname>
1657          is done with <constant>Request.COMMAND_CONFIGURE_JOB</constant> (<constant>_configjob</constant>)
1658          as the command. The <varname>context</varname> parameter reflects the
1659          current context.
1660          </para>
1661        </listitem>
1662       
1663        <listitem>
1664          <para>
1665          When calling <methodname>Response.setDone()</methodname> the plug-in
1666          should use the variant that takes an estimated execution time.
1667          If the plug-in has support for immediate execution or download
1668          (export plug-ins only), it can also respond with
1669          <methodname>Response.setExecuteImmediately()</methodname> or
1670          <methodname>Response.setDownloadImmediately()</methodname>.
1671          </para>
1672         
1673          <para>
1674          If the plug-in requested and was granted immediate execution or
1675          download the same plug-in instance is used to execute the plug-in.
1676          This may be done with the same or a new thread. Otherwise, a new
1677          job is added to the job queue, the parameter value are saved
1678          and the plug-in instance is discarded after calling the
1679          <methodname>Plugin.done()</methodname> method.
1680          </para>
1681        </listitem>
1682        </itemizedlist>
1683      </sect3>
1684   
1685      <sect3 id="plugin_developer.api.callsequence.execute">
1686        <title>Executing a job</title>
1687       
1688        <para>
1689          Normally, the creation of a job and the execution of it are
1690          two different events. The execution may as well be done on a
1691          different server. See <xref linkend="installation.jobagents" />.
1692          This means that the execution takes place in a different instance
1693          of the plug-in class than what was used for creating the job.
1694          The exception is if a plug-in supports immediate execution or download.
1695          In this case the same instance is used, and it is, of course,
1696          always executed on the web server.
1697        </para>
1698       
1699        <orderedlist>
1700        <listitem>
1701          <para>
1702          A new instance of the plug-in class is created. The plug-in must
1703          have a public no-argument constructor.
1704          </para>
1705        </listitem>
1706       
1707        <listitem>
1708          <para>
1709          The <methodname>Plugin.init()</methodname> method is called.
1710          The <varname>job</varname> parameter contains the job
1711          configuration parameters. The <varname>configuration</varname> parameter
1712          is <constant>null</constant> if the plug-in does not have any
1713          configuration parameters.
1714          </para>
1715        </listitem>
1716       
1717        <listitem>
1718          <para>
1719          The <methodname>Plugin.run()</methodname> method is called.
1720          It is finally time for the plug-in to do the work it has been
1721          designed for. This method should not throw any exceptions.
1722          Use the <methodname>Response.setDone()</methodname>
1723          method to report success, the <methodname>Response.setError()</methodname>
1724          method to report errors or the <methodname>Response.setContinue()</methodname>
1725          method to respond to a shutdown signal and tell the core to resume the
1726          job once the system is up and running again. The <methodname>Response.setContinue()</methodname>
1727          can also be used when the system is not shutting down. The job will then be
1728          put back into the job queue and executed again later.
1729          </para>
1730        </listitem>
1731       
1732        <listitem>
1733          <para>
1734          In all cases the <methodname>Plugin.done()</methodname>
1735          method is called and the plug-in instance is discarded.
1736          </para>
1737        </listitem>
1738       
1739        </orderedlist>
1740      </sect3>
1741   
1742    </sect2>
1743 
1744    <sect2 id="plugin_developer.api.jspparameters">
1745      <title>Using custom JSP pages for parameter input</title>
1746
1747        <para>
1748          This is an advanced option for plug-ins that require a different interface for
1749          specifying plug-in parameters than the default list showing one parameter at a
1750          time. This feature is used by setting the
1751          <methodname>RequestInformation.getJspPage()</methodname>
1752          property when constructing the request information object. If this property has
1753          a non-null value, the web client will send the browser to the specified JSP page
1754          instead of to the generic parameter input page.
1755        </para>
1756        <para>
1757          When setting the JSP page you can either specify an absolute path
1758          or only the filename of the JSP file. If only the filename is specified,
1759          the JSP file is expected to be located in a special location, generated
1760          from the package name of your plug-in. If the plug-in is located in the package
1761          <classname>org.company</classname> the JSP file must be located in
1762          <filename class="directory">&lt;base-dir&gt;/www/plugins/org/company/</filename>.
1763        </para>
1764       
1765        <para>
1766          An absolute path starts with '/' and may or may not include the
1767          root directory of the BASE installation. If, for example, BASE is intalled to
1768          <userinput>http://your.base.server.com/base</userinput>, the following absolute
1769          paths are equivalent  <filename>/base/path/to/file.jsp</filename>,
1770          <filename>/path/to/file.jsp</filename>.
1771        </para>
1772        <para>
1773          In both cases, please note that the browser still thinks that it is showing
1774          the regular parameter input page at the usual location:
1775          <filename class="directory">&lt;base-dir&gt;/www/common/plugin/index.jsp</filename>.
1776          All links in your JSP page should be relative to that directory.
1777        </para>
1778        <para>
1779          Even if you use your own JSP page we recommend that you use the built-in
1780          facility for passing the parameters back to the plug-in. For this to work you
1781          must:
1782        </para>
1783        <itemizedlist spacing="compact">
1784          <listitem>
1785            <simpara>
1786            Generate the list of <classname docapi="net.sf.basedb.core">PluginParameter</classname> 
1787            objects as usual.
1788            </simpara>
1789          </listitem>
1790          <listitem>
1791            <simpara>
1792              Name all your input fields in the JSP like:
1793              <parameter>
1794                parameter:<replaceable>name-of-parameter</replaceable>
1795              </parameter>
1796            </simpara>
1797<programlisting language="java:nogutter">
1798// Plug-in generate PluginParameter
1799StringParameterType stringPT = new StringParameterType(255, null, true);
1800PluginParameter one = new PluginParameter("one", "One", "First string", stringPT);
1801PluginParameter two = new PluginParameter("two", "Two", "Second string", stringPT);
1802
1803// JSP should name fields as:
1804First string: &lt;input type="text" name="parameter:one"&gt;&lt;br&gt;
1805Second string: &lt;input type="text" name="parameter:two"&gt;
1806</programlisting>
1807          </listitem>
1808          <listitem>
1809          <simpara>
1810            Send the form to
1811            <filename>index.jsp</filename>
1812            with the <varname>ID</varname>,
1813            <varname>cmd</varname> and <varname>requestId</varname> 
1814            parameters as shown below.
1815          </simpara>
1816<programlisting language="xml">
1817<![CDATA[
1818<form action="index.jsp" method="post">
1819<input type="hidden" name="ID" value="<%=ID%>">
1820<input type="hidden" name="requestId" value="<%=request.getParameter("requestId")%>">
1821<input type="hidden" name="cmd" value="SetParameters">
1822...
1823</form>
1824]]>
1825</programlisting>
1826          <simpara>
1827            The <varname>ID</varname> is the session ID for the logged
1828            in user and is required. The <varname>requestId</varname>
1829            is the ID for this particular plug-in/job configuration
1830            sequence. It is optional, but we recommend that you use it
1831            since it protects your plug-in from getting mixed up with
1832            other plug-in configuration wizards. The <varname>cmd=SetParameters</varname>
1833            tells BASE to send the parameters to the plug-in for validation
1834            and saving.
1835          </simpara>
1836
1837          <simpara>
1838            Values are sent as strings to BASE that converts them to the
1839            proper value type before they are passed on to your plug-in.
1840            However, there is one case that can't be
1841            accurately represented with custom JSP pages, namely 'null' values.
1842            A null value is sent by not sending any value at all. This is not
1843            possible with a fixed form. It is of course possible to add some custom
1844            JavaScript that adds and removes form elements as needed, but it is
1845            also possible to let the empty string represent null. Just include a
1846            hidden parameter like this if you want an empty value for the 'one'
1847            parameter converted to null:
1848          </simpara>
1849<programlisting language="xml">
1850&lt;input type="hidden" name="parameter:one:emptyIsNull" value="1"&gt;
1851</programlisting>
1852          </listitem>
1853          </itemizedlist>
1854         
1855          <para>
1856            If you want a &gbCancel; button to abort the configuration
1857            this should be linked to a page reload with with the url:
1858            <uri>index.jsp?ID=&lt;%=ID%&gt;&amp;cmd=CancelWizard</uri>. This
1859            allows BASE to clean up resources that has been put in global
1860            session variables.
1861          </para>
1862         
1863            <para>
1864              In your JSP page you will probably need to access some information like the
1865              <classname docapi="net.sf.basedb.core">SessionControl</classname>, <classname docapi="net.sf.basedb.core">Job</classname>
1866              and possible even the <classname docapi="net.sf.basedb.core">RequestInformation</classname>
1867              object created by your plug-in.
1868            </para>
1869<programlisting language="java">
1870// Get session control and its ID (required to post to index.jsp)
1871final SessionControl sc = Base.getExistingSessionControl(pageContext, true);
1872final String ID = sc.getId();
1873
1874// Get information about the current request to the plug-in
1875PluginConfigurationRequest pcRequest =
1876   (PluginConfigurationRequest)sc.getSessionSetting("plugin.configure.request");
1877PluginDefinition plugin =
1878   (PluginDefinition)sc.getSessionSetting("plugin.configure.plugin");
1879PluginConfiguration pluginConfig =
1880   (PluginConfiguration)sc.getSessionSetting("plugin.configure.config");
1881PluginDefinition job =
1882   (PluginDefinition)sc.getSessionSetting("plugin.configure.job");
1883RequestInformation ri = pcRequest.getRequestInformation();
1884</programlisting>
1885
1886    </sect2>
1887  </sect1>
1888
1889  <sect1 id="plugin_developer.import">
1890    <title>Import plug-ins</title>
1891
1892    <para>
1893      A plug-in becomes an import plugin simply by returning
1894      <constant>Plugin.MainType.IMPORT</constant>
1895      from the <methodname>Plugin.getMainType()</methodname> method.
1896    </para>
1897   
1898    <sect2 id="plugin_developer.import.autodetect">
1899      <title>Autodetect file formats</title>
1900      <para>
1901        BASE has built-in functionality for autodetecting file formats.
1902        Your plug-in can be part of that feature if it reads it data
1903        from a single file. It must also implement the
1904        <interfacename docapi="net.sf.basedb.core.plugin">AutoDetectingImporter</interfacename>
1905        interface.
1906      </para>
1907
1908      <sect3 id="plugin_developer.api.interfaces.autodetecting">
1909        <title>The net.sf.basedb.core.plugin.AutoDetectingImporter interface</title>
1910
1911        <variablelist>
1912        <varlistentry>
1913          <term>
1914            <methodsynopsis language="java">
1915              <modifier>public</modifier>
1916              <type>boolean</type>
1917              <methodname>isImportable</methodname>
1918              <methodparam>
1919                <type>InputStream</type>
1920                <parameter>in</parameter>
1921              </methodparam>
1922              <exceptionname>BaseException</exceptionname>
1923            </methodsynopsis>
1924          </term>
1925          <listitem>
1926            <para>
1927              Check the input stream if it seems to contain data that can be imported by
1928              the plugin. Usually it means scanning a few lines for some header
1929              matching a predefined string or regular expression.
1930            </para>
1931            <para>
1932              The <classname docapi="net.sf.basedb.plugins">AbstractFlatFileImporter</classname> 
1933              can be used for text-based files and implements this method
1934              by reading the headers from the input stream and checking if
1935              it stopped at an unknown type of line or not:
1936                <programlisting language="java">
1937public final boolean isImportable(InputStream in)
1938   throws BaseException
1939{
1940   FlatFileParser ffp = getInitializedFlatFileParser();
1941   ffp.setInputStream(in);
1942   try
1943   {
1944      ffp.nextSection();
1945      FlatFileParser.LineType result = ffp.parseHeaders();
1946      if (result == FlatFileParser.LineType.UNKNOWN)
1947      {
1948         return false;
1949      }
1950      else
1951      {
1952         return isImportable(ffp);
1953      }
1954   }
1955   catch (IOException ex)
1956   {
1957      throw new BaseException(ex);
1958   }
1959}
1960</programlisting>
1961           
1962              The <classname>AbstractFlatFileImporter</classname> also has functions
1963              for setting the character set and automatic unwrapping of compressed
1964              files. See the javadoc for more information.
1965            </para>
1966            <para>
1967              Note that the input stream doesn't have to be a text file (but you can't use the
1968              <classname>AbstractFlatFileImporter</classname> then).
1969              It can be any type of file, for example a binary or an XML file.
1970              In the case of an XML file you would need to validate the entire
1971              input stream in order to be a 100% sure that it is a valid
1972              xml file, but we recommend that you only check the first few XML tags,
1973              for example, the &lt;!DOCTYPE &gt; declaration and/or the root element
1974              tag.
1975            </para>
1976           
1977          <tip>
1978          <title>Try casting to ImportInputStream</title>
1979          <para>
1980            In many cases (but not all) the auto-detect functionality uses a
1981            <classname docapi="net.sf.basedb.core.plugin">ImportInputStream</classname>
1982            as the <varname>in</varname> parameter. This class contains some
1983            metadata about the file the input stream is originating from. The most
1984            useful feature is the possibility to get information about the
1985            character set used in the file. This makes it possible to open text
1986            files using the correct character set.
1987          </para>
1988          <programlisting language="java">
1989String charset = Config.getCharset(); // Default value
1990if (in instanceof ImportInputStream)
1991{
1992   ImportInputStream iim = (ImportInputStream)in;
1993   if (iim.getCharacterSet() != null) charset = iim.getCharacterSet();
1994}
1995Reader reader = new InputStreamReader(in, Charset.forName(charset)));
1996</programlisting>
1997        </tip>       
1998           
1999          </listitem>
2000        </varlistentry>
2001        <varlistentry>
2002          <term>
2003            <methodsynopsis language="java">
2004              <modifier>public</modifier>
2005              <void/>
2006              <methodname>doImport</methodname>
2007              <methodparam>
2008                <type>InputStream</type>
2009                <parameter>in</parameter>
2010              </methodparam>
2011              <methodparam>
2012                <type>ProgressReporter</type>
2013                <parameter>progress</parameter>
2014              </methodparam>
2015              <exceptionname>BaseException</exceptionname>
2016            </methodsynopsis>
2017          </term>
2018          <listitem>
2019            <para>
2020              Parse the input stream and import all data that is found.
2021              This method is of course only called if the
2022              <methodname>isImportable()</methodname> has returned true. Note
2023              however that the input stream is reopened at the start of the
2024              file. It may even be the case that the <methodname>isImportable()</methodname>
2025              method is called on one instance of the plugin and the
2026              <methodname>doImport()</methodname> method is called on another.
2027              Thus, the <methodname>doImport()</methodname> can't rely on any state set
2028              by the <methodname>isImportable()</methodname> method.
2029            </para>
2030          </listitem>
2031        </varlistentry>
2032        </variablelist>
2033       
2034      </sect3>
2035     
2036      <sect3 id="plugin_developer.import.autodetect.callsequence">
2037        <title>Call sequence during autodetection</title>
2038       
2039        <para>
2040          The call sequence for autodetection resembles the call sequence for
2041          checking if the plug-in can be used in a given context.
2042        </para>
2043
2044        <orderedlist>
2045        <listitem>
2046          <para>
2047          A new instance of the plug-in class is created. The plug-in must
2048          have a public no-argument constructor.
2049          </para>
2050        </listitem>
2051       
2052        <listitem>
2053          <para>
2054          The <methodname>Plugin.init()</methodname> method is called.
2055          The <varname>job</varname> parameter is <constant>null</constant>.
2056          The <varname>configuration</varname> parameter is <constant>null</constant>
2057          if the plug-in does not have any configuration parameters.
2058          </para>
2059        </listitem>
2060       
2061        <listitem>
2062          <para>
2063          If the plug-in is interactive the the <methodname>InteractivePlugin.isInContext()</methodname>
2064          is called. If the context is a list context, the <varname>item</varname>
2065          parameter is null, otherwise the current item is passed. The plug-in
2066          should return <constant>null</constant> if it can be used under the
2067          current circumstances, or a message explaining why not.
2068          </para>
2069        </listitem>
2070       
2071        <listitem>
2072          <para>
2073          If the plug-in can be used the <methodname>AutoDetectingImporter.isImportable()</methodname>
2074          method is called to check if the selected file is importable or not.
2075          </para>
2076        </listitem>
2077       
2078        <listitem>
2079          <para>
2080          After this, <methodname>Plugin.done()</methodname> is called and
2081          the plug-in instance is discarded. If there are
2082          several configurations for a plug-in, this procedure is repeated
2083          for each configuration. If the plug-in can be used without
2084          a configuration the procedure is also repeated without
2085          configuration parameters.
2086          </para>
2087        </listitem>
2088       
2089        <listitem>
2090          <para>
2091          If a single plug-in was found the user is taken to the regular
2092          job configuration wizard. A new plug-in instance is created for
2093          this. If more than one plug-in was found the user is presented
2094          with a list of the plug-ins. After selecting one of them the
2095          regular job configuration wizard is used with a new plug-in instance.
2096          </para>
2097        </listitem>
2098       
2099        </orderedlist>
2100       
2101      </sect3>
2102
2103    </sect2>
2104   
2105    <sect2 id="plugin_developer.import.abstractflatfileimporter">
2106      <title>The AbstractFlatFileImporter superclass</title>
2107      <para>
2108        The <classname docapi="net.sf.basedb.plugins">AbstractFlatFileImporter</classname> is a very useful abstract
2109        class to use as a superclass for your own import plug-ins. It can be used
2110        if your plug-in uses regular text files that can be parsed by an instance of the
2111        <classname docapi="net.sf.basedb.util">net.sf.basedb.util.FlatFileParser</classname> class. This class parses a file
2112        by checking each line against a few regular expressions. Depending on which regular
2113        expression matches the line, it is classified as a header line, a section line,
2114        a comment, a data line, a footer line or unknown. Header lines are inspected as a group,
2115        but data lines individually, meaning that it consumes very little memory since only
2116        a few lines at a time needs to be loaded.
2117      </para>
2118     
2119      <para>
2120        The <classname docapi="net.sf.basedb.plugins">AbstractFlatFileImporter</classname> defines
2121        <classname docapi="net.sf.basedb.core">PluginParameter</classname> objects
2122        for each of the regular expressions and other parameters used by the parser. It also
2123        implements the <methodname>Plugin.run()</methodname> method and does most of
2124        the ground work for instantiating a <methodname>FlatFileParser</methodname> and
2125        parsing the file. What you have to do in your plugin is to put together the
2126        <classname docapi="net.sf.basedb.core">RequestInformation</classname> objects
2127        for configuring the plugin and creating a job and implement the
2128        <methodname>InteractivePlugin.configure()</methodname> method for validating and
2129        storing the parameters. You should also implement or override some methods
2130        defined by <classname>AbstractFlatFileImporter</classname>.
2131      </para>
2132
2133      <para>
2134      Here is what you need to do:
2135      </para>
2136
2137      <itemizedlist> 
2138      <listitem>
2139        <para>
2140        Implement the <interfacename docapi="net.sf.basedb.core.plugin">InteractivePlugin</interfacename> methods.
2141        See <xref linkend="plugin_developer.api.interfaces.interactive" /> for more information. Note that the
2142        <classname docapi="net.sf.basedb.plugins">AbstractFlatFileImporter</classname>
2143        has defined many parameters for regular expressions used by the parser
2144        already. You should just pick them and put in your <classname docapi="net.sf.basedb.core">RequestInformation</classname>
2145        object.
2146        </para>
2147       
2148        <programlisting language="java">
2149// Parameter that maps the items name from a column
2150private PluginParameter&lt;String&gt; nameColumnMapping;
2151
2152// Parameter that maps the items description from a column
2153private PluginParameter&lt;String&gt; descriptionColumnMapping;
2154
2155private RequestInformation getConfigurePluginParameters(GuiContext context)
2156{
2157   if (configurePlugin == null)
2158   {
2159      // To store parameters for CONFIGURE_PLUGIN
2160      List&lt;PluginParameter&lt;?&gt;&gt; parameters =
2161         new ArrayList&lt;PluginParameter&lt;?&gt;&gt;();
2162
2163      // Parser regular expressions - from AbstractFlatFileParser
2164      parameters.add(parserSection);
2165      parameters.add(headerRegexpParameter);
2166      parameters.add(dataHeaderRegexpParameter);
2167      parameters.add(dataSplitterRegexpParameter);
2168      parameters.add(ignoreRegexpParameter);
2169      parameters.add(dataFooterRegexpParameter);
2170      parameters.add(minDataColumnsParameter);
2171      parameters.add(maxDataColumnsParameter);
2172
2173      // Column mappings
2174      nameColumnMapping = new PluginParameter&lt;String&gt;(
2175         "nameColumnMapping",
2176         "Name",
2177         "Mapping that picks the items name from the data columns",
2178         new StringParameterType(255, null, true)
2179      );
2180   
2181      descriptionColumnMapping = new PluginParameter&lt;String&gt;(
2182        "descriptionColumnMapping",
2183        "Description",
2184        "Mapping that picks the items description from the data columns",
2185        new StringParameterType(255, null, false)
2186      );
2187
2188      parameters.add(mappingSection);
2189      parameters.add(nameColumnMapping);
2190      parameters.add(descriptionColumnMapping);
2191     
2192      configurePlugin = new RequestInformation
2193      (
2194         Request.COMMAND_CONFIGURE_PLUGIN,
2195         "File parser settings",
2196         "",
2197         parameters
2198      );
2199
2200   }
2201   return configurePlugin;
2202}
2203</programlisting>
2204      </listitem>
2205     
2206      <listitem>
2207        <para>
2208        Implement/override some of the methods defined by
2209        <classname>AbstractFlatFileParser</classname>. The most important
2210        methods are listed below.
2211        </para>
2212      </listitem>
2213     
2214      </itemizedlist>
2215
2216      <variablelist>
2217      <varlistentry>
2218        <term>
2219          <methodsynopsis language="java">
2220            <modifier>protected</modifier>
2221            <type>FlatFileParser</type>
2222            <methodname>getInitializedFlatFileParser</methodname>
2223            <exceptionname>BaseException</exceptionname>
2224          </methodsynopsis>
2225        </term>
2226        <listitem>
2227          <para>
2228          The method is called to create a <classname docapi="net.sf.basedb.util.parser">FlatFileParser</classname>
2229          and set the regular expressions that should be used for parsing the file.
2230          The default implementation assumes that your plug-in has used the built-in
2231          <classname docapi="net.sf.basedb.core">PluginParameter</classname> objects and has stored the values
2232          at the configuration level. You should override this method if you need to
2233          initialise the parser in a different way. See for example the
2234          code for the <classname docapi="net.sf.basedb.plugins">PrintMapFlatFileImporter</classname> plug-in which
2235          has a fixed format and doesn't use configurations.
2236          </para>
2237          <programlisting language="java">
2238@Override
2239protected FlatFileParser getInitializedFlatFileParser()
2240   throws BaseException
2241{
2242   FlatFileParser ffp = new FlatFileParser();
2243   ffp.setSectionRegexp(Pattern.compile("\\[(.+)\\]"));
2244   ffp.setHeaderRegexp(Pattern.compile("(.+)=,(.*)"));
2245   ffp.setDataSplitterRegexp(Pattern.compile(","));
2246   ffp.setDataFooterRegexp(Pattern.compile(""));
2247   ffp.setMinDataColumns(12);
2248   return ffp;
2249}
2250</programlisting>
2251        </listitem>
2252      </varlistentry>
2253     
2254      <varlistentry>
2255        <term>
2256          <methodsynopsis language="java">
2257            <modifier>protected</modifier>
2258            <type>boolean</type>
2259            <methodname>isImportable</methodname>
2260            <methodparam>
2261              <type>FlatFileParser</type>
2262              <parameter>ffp</parameter>
2263            </methodparam>
2264            <exceptionname>IOException</exceptionname>
2265          </methodsynopsis>
2266        </term>
2267        <listitem>
2268          <para>
2269          This method is called from the <methodname>isImportable(InputStream)</methodname>
2270          method, AFTER <methodname>FlatFileParser.nextSection()</methodname> and
2271          <methodname>FlatFileParser.parseHeaders()</methodname> has been called
2272          a single time and if the <methodname>parseHeaders</methodname> method didn't
2273          stop on an unknown line. The default implementation of this method always returns
2274          TRUE, since obviously some data has been found. A subclass may override this method
2275          if it wants to do more checks, for example, make that a certain header is present
2276          with a certain value. It may also continue parsing the file. Here is a code example from
2277          the <classname docapi="net.sf.basedb.plugins">PrintMapFlatFileImporter</classname> which checks if a
2278          <constant>FormatName</constant> header is present and contains either
2279          <constant>TAM</constant> or <constant>MwBr</constant>.
2280          </para>
2281         
2282          <programlisting language="java">
2283/**
2284   Check that the file is a TAM or MwBr file.
2285   @return TRUE if a FormatName header is present and contains "TAM" or "MwBr", FALSE
2286      otherwise
2287*/
2288@Override
2289protected boolean isImportable(FlatFileParser ffp)
2290{
2291   String formatName = ffp.getHeader("FormatName");
2292   return formatName != null &amp;&amp; 
2293      (formatName.contains("TAM") || formatName.contains("MwBr"));
2294}
2295</programlisting>
2296        </listitem>
2297      </varlistentry>
2298     
2299      <varlistentry>
2300        <term>
2301          <methodsynopsis language="java">
2302            <modifier>protected</modifier>
2303            <void/>
2304            <methodname>begin</methodname>
2305            <methodparam>
2306              <type>FlatFileParser</type>
2307              <parameter>ffp</parameter>
2308            </methodparam>
2309            <exceptionname>BaseException</exceptionname>
2310          </methodsynopsis>
2311        </term>
2312        <listitem>
2313          <para>
2314          This method is called just before the parsing of the file
2315          begins. Override this method if you need to initialise some
2316          internal state. This is, for example, a good place to open
2317          a <classname docapi="net.sf.basedb.core">DbControl</classname> object, read parameters from the
2318          job and configuration and put them into more useful variables. The default
2319          implementation does nothing, but we recommend that
2320          <methodname>super.begin()</methodname> is always called.
2321          </para>
2322          <programlisting language="java">
2323// Snippets from the RawDataFlatFileImporter class
2324private DbControl dc;
2325private RawDataBatcher batcher;
2326private RawBioAssay rawBioAssay;
2327private Map&lt;String, String&gt; columnMappings;
2328private int numInserted;
2329
2330@Override
2331protected void begin()
2332   throws BaseException
2333{
2334   super.begin();
2335
2336   // Get DbControl
2337   dc = sc.newDbControl();
2338   rawBioAssay = (RawBioAssay)job.getValue(rawBioAssayParameter.getName());
2339
2340   // Reload raw bioassay using current DbControl
2341   rawBioAssay = RawBioAssay.getById(dc, rawBioAssay.getId());
2342   
2343   // Create a batcher for inserting spots
2344   batcher = rawBioAssay.getRawDataBatcher();
2345
2346   // For progress reporting
2347   numInserted = 0;
2348}         
2349</programlisting>
2350        </listitem>
2351      </varlistentry>
2352      <varlistentry>
2353        <term>
2354          <methodsynopsis language="java">
2355            <modifier>protected</modifier>
2356            <void/>
2357            <methodname>handleHeader</methodname>
2358            <methodparam>
2359              <type>FlatFileParser.Line</type>
2360              <parameter>line</parameter>
2361            </methodparam>
2362            <exceptionname>BaseException</exceptionname>
2363          </methodsynopsis>
2364        </term>
2365        <listitem>
2366          <para>
2367          This method is called once for every header line that is found in
2368          the file. The <varname>line</varname> parameter contains information
2369          about the header. The default implementation of this method does
2370          nothing.
2371          </para>
2372          <programlisting language="java">
2373@Override
2374protected void handleHeader(Line line)
2375   throws BaseException
2376{
2377   super.handleHeader(line);
2378   if (line.name() != null &amp;&amp; line.value() != null)
2379   {
2380      rawBioAssay.setHeader(line.name(), line.value());
2381   }
2382}
2383</programlisting>
2384        </listitem>
2385      </varlistentry>
2386      <varlistentry>
2387        <term>
2388          <methodsynopsis language="java">
2389            <modifier>protected</modifier>
2390            <void/>
2391            <methodname>handleSection</methodname>
2392            <methodparam>
2393              <type>FlatFileParser.Line</type>
2394              <parameter>line</parameter>
2395            </methodparam>
2396            <exceptionname>BaseException</exceptionname>
2397          </methodsynopsis>
2398        </term>
2399        <listitem>
2400          <para>
2401            This method is called once for each section that is found in the file.
2402            The <varname>line</varname> parameter contains information
2403            about the section. The default implementation of this method does
2404            nothing.
2405          </para>
2406        </listitem>
2407      </varlistentry>
2408     
2409      <varlistentry>
2410        <term>
2411          <methodsynopsis language="java">
2412            <modifier>protected abstract</modifier>
2413            <void/>
2414            <methodname>beginData</methodname>
2415            <exceptionname>BaseException</exceptionname>
2416          </methodsynopsis>
2417        </term>
2418        <listitem>
2419          <para>
2420          This method is called after the headers has been parsed, but before
2421          the first line of data. This is a good place to add code that
2422          depends on information in the headers, for example, put
2423          together column mappings.
2424          </para>
2425         
2426          <programlisting language="java">
2427private Mapper reporterMapper;
2428private Mapper blockMapper;
2429private Mapper columnMapper;
2430private Mapper rowMapper;
2431// ... more mappers
2432
2433@Override
2434protected void beginData()
2435{
2436   boolean cropStrings = ("crop".equals(job.getValue("stringTooLongError")));
2437
2438   // Mapper that always return null; used if no mapping expression has been entered
2439   Mapper nullMapper = new ConstantMapper((String)null);
2440   
2441   // Column mappers
2442   reporterMapper = getMapper(ffp, (String)configuration.getValue("reporterIdColumnMapping"),
2443      cropStrings ? ReporterData.MAX_EXTERNAL_ID_LENGTH : null, nullMapper);
2444   blockMapper = getMapper(ffp, (String)configuration.getValue("blockColumnMapping"),
2445      null, nullMapper);
2446   columnMapper = getMapper(ffp, (String)configuration.getValue("columnColumnMapping"),
2447      null, nullMapper);
2448   rowMapper = getMapper(ffp, (String)configuration.getValue("rowColumnMapping"),
2449      null, nullMapper);
2450   // ... more mappers: metaGrid coordinate, X-Y coordinate, extended properties
2451   // ...
2452}
2453</programlisting>
2454         
2455        </listitem>
2456      </varlistentry>
2457       
2458      <varlistentry>
2459        <term>
2460          <methodsynopsis language="java">
2461            <modifier>protected abstract</modifier>
2462            <void/>
2463            <methodname>handleData</methodname>
2464            <methodparam>
2465              <type>FlatFileParser.Data</type>
2466              <parameter>data</parameter>
2467            </methodparam>
2468            <exceptionname>BaseException</exceptionname>
2469          </methodsynopsis>
2470        </term>
2471        <listitem>
2472          <para>
2473          This method is abstract and must be implemented by all subclasses.
2474          It is called once for every data line in the the file.
2475          </para>
2476
2477          <programlisting language="java">
2478// Snippets from the RawDataFlatFileImporter class
2479@Override
2480protected void handleData(Data data)
2481   throws BaseException
2482{
2483   // Create new RawData object
2484   RawData raw = batcher.newRawData();
2485
2486   // External ID for the reporter
2487   String externalId = reporterMapper.getValue(data);
2488   
2489   // Block, row and column numbers
2490   raw.setBlock(blockMapper.getInt(data));
2491   raw.setColumn(columnMapper.getInt(data));
2492   raw.setRow(rowMapper.getInt(data));
2493   // ... more: metaGrid coordinate, X-Y coordinate, extended properties
2494   
2495   // Insert raw data to the database
2496   batcher.insert(raw, externalId);
2497   numInserted++;
2498}
2499</programlisting> 
2500         
2501        </listitem>
2502      </varlistentry>
2503     
2504      <varlistentry>
2505        <term>
2506          <methodsynopsis language="java">
2507            <modifier>protected</modifier>
2508            <void/>
2509            <methodname>end</methodname>
2510            <methodparam>
2511              <type>boolean</type>
2512              <parameter>success</parameter>
2513            </methodparam>
2514          </methodsynopsis>
2515        </term>
2516        <listitem>
2517          <para>
2518            Called when the parsing has ended, either because the end of
2519            file was reached or because an error has occurred. The subclass
2520            should close any open resources, ie. the <classname docapi="net.sf.basedb.core">DbControl</classname>
2521            object. The <varname>success</varname> parameter is <constant>true</constant>
2522            if the parsing was successful, <constant>false</constant> otherwise.
2523            The default implementation does nothing.
2524          </para>
2525         
2526          <programlisting language="java">
2527@Override
2528protected void end(boolean success)
2529   throws BaseException
2530{
2531   try
2532   {
2533      // Commit if the parsing was successful
2534      if (success)
2535      {
2536         batcher.close();
2537         dc.commit();
2538      }
2539   }
2540   catch (BaseException ex)
2541   {
2542      // Well, now we got an exception
2543      success = false;
2544      throw ex;
2545   }
2546   finally
2547   {
2548      // Always close... and call super.end()
2549      if (dc != null) dc.close();
2550      super.end(success);
2551   }
2552}     
2553</programlisting>         
2554        </listitem>
2555      </varlistentry>
2556     
2557      <varlistentry>
2558        <term>
2559          <methodsynopsis language="java">
2560            <modifier>protected</modifier>
2561            <type>String</type>
2562            <methodname>getSuccessMessage</methodname>
2563            <void/>
2564          </methodsynopsis>
2565        </term>
2566        <listitem>
2567          <para>
2568          This is the last method that is called, and it is only called if
2569          everything went suceessfully. This method allows a subclass to generate
2570          a short message that is sent back to the database as a final progress
2571          report. The default implementation returns null, which means that no
2572          message will be generated.
2573          </para>
2574          <programlisting language="java">
2575@Override
2576protected String getSuccessMessage()
2577{
2578   return numInserted + " spots inserted";
2579}
2580</programlisting>
2581        </listitem>
2582      </varlistentry>
2583      </variablelist>
2584
2585      <para>
2586        The <classname docapi="net.sf.basedb.plugins">AbstractFlatFileImporter</classname> has a lot of
2587        other methods that you may use and/or override in your own plug-in.
2588        Check the javadoc for more information.
2589      </para>
2590
2591      <sect3 id="plugin_developer.import.configurebyexample">
2592        <title>Configure by example</title>
2593       
2594        <para>
2595          The <interfacename docapi="net.sf.basedb.util.parser">ConfigureByExample</interfacename> 
2596          is a tagging interface that can be used by plug-ins using the
2597          <classname docapi="net.sf.basedb.util.parser">FlatFileParser</classname> class for parsing.
2598          The web client detects if a plug-in implements this interface and if the list of
2599          parameters includes a section parameter with the name <constant>parserSection</constant>
2600          a <guibutton>Test with file</guibutton> buttons is activated. This button will take the
2601          user to a form which allows the user to enter values for the parameters defined in
2602          the <classname>AbstractFlatFileImporter</classname> class. Parameters for column mappings
2603          must have the string "Mapping" in their names.
2604        </para>
2605      </sect3>
2606    </sect2>
2607   
2608  </sect1>
2609
2610  <sect1 id="plugin_developer.export">
2611    <title>Export plug-ins</title>
2612   
2613    <para>
2614      Export plug-ins are plug-ins that takes data from BASE, and
2615      prepares it for use with some external entity. Usually this
2616      means that data is taken from the database and put into a file
2617      with some well-defined file format.
2618      An export plug-in should return <constant>MainType.EXPORT</constant> from
2619      the <methodname>Plugin.getMainType()</methodname> method.
2620    </para>
2621   
2622    <sect2 id="plugin_developer.export.download">
2623      <title>Immediate download of exported data</title>
2624      <para>
2625        An export plug-in may want to give the user a choice
2626        between saving the exported data in the BASE file system
2627        or to download it immediately to the client computer. With the basic
2628        plug-in API the second option is not possible. The
2629        <interfacename docapi="net.sf.basedb.core.plugin">ImmediateDownloadExporter</interfacename> is an
2630        interface that extends the <interfacename docapi="net.sf.basedb.core.plugin">Plugin</interfacename>
2631        interface to provide this functionality. If your export
2632        plug-in wants to provide immediate download functionality it must
2633        implement the <interfacename docapi="net.sf.basedb.core.plugin">ImmediateDownloadExporter</interfacename>
2634        interface.
2635      </para>
2636     
2637      <sect3 id="plugin_developer.export.immediatedownloadexporter">
2638      <title>The ImmediateDownloadExporter interface</title>
2639      <variablelist>
2640      <varlistentry>
2641        <term>
2642          <methodsynopsis language="java">
2643            <modifier>public</modifier>
2644            <void/>
2645            <methodname>doExport</methodname>
2646            <methodparam>
2647              <type>ExportOutputStream</type>
2648              <parameter>out</parameter>
2649            </methodparam>
2650            <methodparam>
2651              <type>ProgressReporter</type>
2652              <parameter>progress</parameter>
2653            </methodparam>
2654          </methodsynopsis>
2655        </term>
2656        <listitem>
2657          <para>
2658          Perform the export. The plug-in should write the
2659          exported data to the <varname>out</varname> stream.
2660          If the <varname>progress</varname> parameter is not null,
2661          the progress should be reported at regular interval in the
2662          same manner as in the <methodname>Plugin.run()</methodname>
2663          method.
2664          </para>
2665        </listitem>
2666      </varlistentry>
2667      </variablelist>
2668     
2669      </sect3>
2670     
2671      <sect3 id="plugin_developer.export.exportoutputstream">
2672      <title>The ExportOutputStream class</title>
2673     
2674      <para>
2675        The <classname docapi="net.sf.basedb.core.plugin">ExportOutputStream</classname> is
2676        an extension to the
2677        <classname>java.io.OutputStream</classname>. Use the regular
2678        <methodname>write()</methodname> methods to write data to it.
2679        It also has some additional methods, which are used for setting
2680        metadata about the generated file. These methods are useful, for
2681        example, when generating HTTP response headers.
2682      </para>
2683     
2684      <note>
2685        <para>
2686        These methods must be called before starting to write data to
2687        the <varname>out</varname> stream.
2688        </para>
2689      </note>
2690     
2691      <variablelist>
2692      <varlistentry>
2693        <term>
2694          <methodsynopsis language="java">
2695            <modifier>public</modifier>
2696            <void/>
2697            <methodname>setContentLength</methodname>
2698            <methodparam>
2699              <type>long</type>
2700              <parameter>contentLength</parameter>
2701            </methodparam>
2702          </methodsynopsis>
2703        </term>
2704        <listitem>
2705          <para>
2706          Set the total size of the exported data. Don't call this method if the
2707          total size is not known.
2708          </para>
2709        </listitem>
2710      </varlistentry>
2711      <varlistentry>
2712        <term>
2713          <methodsynopsis language="java">
2714            <modifier>public</modifier>
2715            <void/>
2716            <methodname>setMimeType</methodname>
2717            <methodparam>
2718              <type>String</type>
2719              <parameter>mimeType</parameter>
2720            </methodparam>
2721          </methodsynopsis>
2722        </term>
2723        <listitem>
2724          <para>
2725          Set the MIME type of the file that is being generated.
2726          </para>
2727        </listitem>
2728      </varlistentry>
2729      <varlistentry>
2730        <term>
2731          <methodsynopsis language="java">
2732            <modifier>public</modifier>
2733            <void/>
2734            <methodname>setCharacterSet</methodname>
2735            <methodparam>
2736              <type>String</type>
2737              <parameter>charset</parameter>
2738            </methodparam>
2739          </methodsynopsis>
2740        </term>
2741        <listitem>
2742          <para>
2743          Sets the character set used in text files. For example,
2744          UTF-8 or ISO-8859-1.
2745          </para>
2746        </listitem>
2747      </varlistentry>
2748      <varlistentry>
2749        <term>
2750          <methodsynopsis language="java">
2751            <modifier>public</modifier>
2752            <void/>
2753            <methodname>setFilename</methodname>
2754            <methodparam>
2755              <type>String</type>
2756              <parameter>filename</parameter>
2757            </methodparam>
2758          </methodsynopsis>
2759        </term>
2760        <listitem>
2761          <para>
2762          Set a suggested name of the file that is being
2763          generated.
2764          </para>
2765        </listitem>
2766      </varlistentry>
2767      </variablelist>
2768     
2769      </sect3>
2770     
2771      <sect3 id="plugin_developer.export.callsequence">
2772        <title>Call sequence during immediate download</title>
2773
2774      <para>
2775        Supporting immediate download also means that the method call
2776        sequence is a bit altered from the standard sequence described
2777        in <xref linkend="plugin_developer.api.callsequence.execute" />.
2778      </para>
2779     
2780      <itemizedlist>
2781      <listitem>
2782        <para>
2783        The plug-in must call <methodname>Response.setDownloadImmediately()</methodname>
2784        instead of <methodname>Response.setDone()</methodname> in <methodname>Plugin.configure()</methodname>
2785        to end the job configuration wizard. This requests that the core starts
2786        an immediate download.
2787        </para>
2788       
2789        <note>
2790          <para>
2791          Even if an immediate download is requested by the plug-in this feature
2792          may have been disabled by the server administrator. If so, the plug-in
2793          can choose if the job should be added to job queue or if this is an
2794          error condition.
2795          </para>
2796        </note>
2797      </listitem>
2798     
2799      <listitem>
2800        <para>
2801        If immediate download is granted the web client will keep the
2802        same plug-in instance and call <methodname>ImmediateDownloadExporter.doExport()</methodname>.
2803        In this case, the <methodname>Plugin.run()</methodname> is never called.
2804        After the export, <methodname>Plugin.done()</methodname> is called as
2805        usual.
2806        </para>
2807       
2808      </listitem>
2809
2810      <listitem>
2811        <para>
2812        If immediate download is not granted and the job is added to the job queue
2813        the regular job execution sequence is used.
2814        </para>
2815      </listitem>
2816      </itemizedlist>
2817     
2818      </sect3>
2819
2820    </sect2>
2821   
2822    <sect2 id="plugin_developer.export.abstractexporter">
2823      <title>The AbstractExporterPlugin class</title>
2824   
2825      <para>
2826        This is an abstract superclass that will make it easier
2827        to implement export plug-ins that support immediate
2828        download. It defines <classname docapi="net.sf.basedb.core">PluginParameter</classname>
2829        objects for asking a user about a path where the exported
2830        data should be saved and if existing files should be overwritten or not.
2831        If the user leaves the path empty the immediate download functionality
2832        should be used. It also contains implementations of both the
2833        <methodname>Plugin.run()</methodname> method and the
2834        <methodname>ImmediateDownloadExporter.doExport()</methodname> method.
2835        Here is what you need to do in your own plug-in code (code examples are
2836        taken from the <classname docapi="net.sf.basedb.plugins">HelpExporter</classname>):
2837      </para>
2838     
2839      <itemizedlist>
2840      <listitem>
2841        <para>
2842          Your plug-in should extend the <classname docapi="net.sf.basedb.core.plugin">AbstractExporterPlugin</classname>
2843          class:
2844          <programlisting language="java">
2845public class HelpExporter
2846  extends AbstractExporterPlugin
2847  implements InteractivePlugin
2848</programlisting>
2849        </para>
2850      </listitem>
2851     
2852      <listitem>
2853        <para>
2854          You need to implement the
2855          <methodname>InteractivePlugin.getRequestInformation()</methodname>
2856          method. Use the <methodname>getSaveAsParameter()</methodname>
2857          and <methodname>getOverwriteParameter()</methodname> methods defined in the
2858          superclass to create plug-in parameters that asks for the file name to save
2859          to and if existing files can be overwritten or not.
2860          You should also check if the administrator has enabled the immediate execution
2861          functionality for your plug-in. If not, the only option is to
2862          export to a file in the BASE file system and the filename is a
2863          required parameter.
2864         
2865          <programlisting language="java">
2866// Selected parts of the getRequestConfiguration() method
2867...
2868List&lt;PluginParameter&lt;?&gt;&gt; parameters =
2869   new ArrayList&lt;PluginParameter&lt;?&gt;&gt;();
2870...
2871PluginDefinition pd = job.getPluginDefinition();
2872boolean requireFile = pd == null ?
2873   false : !pd.getAllowImmediateExecution();
2874
2875parameters.add(getSaveAsParameter(null, null, defaultPath, requireFile));
2876parameters.add(getOverwriteParameter(null, null));
2877
2878configureJob = new RequestInformation
2879(
2880   Request.COMMAND_CONFIGURE_JOB,
2881   "Help exporter options",
2882   "Set Client that owns the helptexts, " +
2883     "the file path where the export file should be saved",
2884   parameters
2885);
2886....
2887return configureJob;
2888</programlisting>
2889        </para>
2890      </listitem>
2891
2892      <listitem>
2893        <para>
2894        You must also implement the <methodname>configure()</methodname>
2895        method and check the parameters. If no filename has been given,
2896        you should check if immediate exection is allowed and set an
2897        error if it is not. If a filename is present, use the
2898        <methodname>pathCanBeUsed()</methodname> method to check if
2899        it is possible to save the data to a file with that name. If the
2900        file already exists it can be overwritten if the <varname>OVERWRITE</varname>
2901        is <constant>TRUE</constant> or if the file has been flagged for removal.
2902        Do not forget to store the parameters with the <methodname>storeValue()</methodname>
2903        method.
2904
2905        <programlisting language="java">
2906// Selected parts from the configure() method
2907if (request.getParameterValue(SAVE_AS) == null)
2908{
2909   if (!request.isAllowedImmediateExecution())
2910   {
2911      response.setError("Immediate download is not allowed. " +
2912         "Please specify a filename.", null);
2913      return;
2914   }
2915   Client client = (Client)request.getParameterValue("client");
2916   response.setDownloadImmediately("Export help texts for client application " +
2917     client.getName(), ExecutionTime.SHORTEST, true);
2918}
2919else
2920{
2921   if (!pathCanBeUsed((String)request.getParameterValue(SAVE_AS),
2922      (Boolean)request.getParameterValue(OVERWRITE)))
2923   {
2924      response.setError("File exists: " +
2925         (String)request.getParameterValue(SAVE_AS), null);
2926      return;
2927   }
2928   storeValue(job, request, ri.getParameter(SAVE_AS));
2929   storeValue(job, request, ri.getParameter(OVERWRITE));
2930   response.setDone("The job configuration is complete", ExecutionTime.SHORTEST);
2931}
2932</programlisting>
2933       
2934        </para>
2935      </listitem>
2936     
2937      <listitem>
2938        <para>
2939        Implement the <methodname>performExport()</methodname> method.
2940        This is defined as abstract in the <classname docapi="net.sf.basedb.core.plugin">AbstractExporterPlugin</classname>
2941        class. It has the same parameters as the <methodname>ImmediateDownloadExporter.doExport()</methodname>
2942        method and they have the same meaning. The only difference is that the
2943        <varname>out</varname> stream can be linked to a file in the BASE filesystem
2944        and not just to the HTTP response stream.
2945        </para>
2946      </listitem>
2947     
2948      <listitem>
2949        <para>
2950        Optionally, implement the <methodname>begin()</methodname>,
2951        <methodname>end()</methodname> and <methodname>getSuccessMessage()</methodname>
2952        methods. Theese methods do nothing by default.
2953        </para>
2954      </listitem>
2955      </itemizedlist>
2956     
2957      <para>
2958        The call sequence for plug-ins extending <classname docapi="net.sf.basedb.core.plugin">AbstractExporterPlugin</classname>
2959        is:
2960      </para>
2961     
2962      <orderedlist>
2963      <listitem>
2964        <para>
2965        Call <methodname>begin()</methodname>.
2966        </para>
2967      </listitem>
2968      <listitem>
2969        <para>
2970        Call <methodname>performExport()</methodname>.
2971        </para>
2972      </listitem>
2973      <listitem>
2974        <para>
2975        Call <methodname>end()</methodname>.
2976        </para>
2977      </listitem>
2978      <listitem>
2979        <para>
2980        Call <methodname>getSuccessMessage()</methodname> if running as a regular
2981        job. This method is never called when doing an immediate download since there
2982        is no place to show the message.
2983        </para>
2984      </listitem>
2985      </orderedlist>
2986     
2987    </sect2>
2988   
2989  </sect1>
2990 
2991  <sect1 id="plugin_developer.analyse">
2992    <title>Analysis plug-ins</title>
2993    <para>
2994      A plug-in becomes an analysis plug-in simply by returning
2995      <constant>Plugin.MainType.ANALYZE</constant> from the
2996      <methodname>Plugin.getMainType()</methodname> method. The information returned from
2997      <methodname>InteractivePlugin.getGuiContexts()</methodname>
2998      should include: [<constant>Item.BIOASSAYSET</constant>, <constant>Type.ITEM</constant>]
2999      or [<constant>Item.DERIVEDBIOASSAYSET</constant>, <constant>Type.ITEM</constant>]
3000      since web client doesn't look for analysis plug-ins in most other places. If
3001      the plug-in can work on a subset of the bioassays it may also include
3002      [<constant>Item.BIOASSAY</constant>, <constant>Type.LIST</constant>]
3003      among the contexts. This will make it possible for a user to select
3004      bioassays from the list and then invoke the plug-in. The following
3005      code examples are taken from an analysis plug-in that is used in an experiment
3006      context.
3007    </para>
3008   
3009<programlisting language="java">
3010private static final Set&lt;GuiContext&gt; guiContexts =
3011   Collections.singleton(new GuiContext(Item.BIOASSAYSET, GuiContext.Type.ITEM));
3012
3013public Set&lt;GuiContext&gt; getGuiContexts()
3014{
3015   return guiContexts;
3016}</programlisting>
3017
3018    <para>
3019    If the plug-in depends on a specific raw data type or on the number of
3020    channels, it should check that the current bioassayset is of the
3021    correct type in the <methodname>InteractivePlugin.isInContext()</methodname> 
3022    method. It is also a good idea to check if the current user has permission
3023    to use the current experiment. This permission is needed to create new bioassaysets or
3024    other data belonging to the experiment.
3025    </para>
3026 
3027    <programlisting language="java">
3028public boolean isInContext(GuiContext context, Object item)
3029{
3030   if (item == null)
3031   {
3032      message = "The object is null";
3033   }
3034   else if (!(item instanceof BioAssaySet))
3035   {
3036      message = "The object is not a BioAssaySet: " + item;
3037   }
3038   else
3039   {
3040      BioAssaySet bas = (BioAssaySet)item;
3041      int channels = bas.getRawDataType().getChannels();
3042      if (channels != 2)
3043      {
3044         message = "This plug-in requires 2-channel data, not " + channels + "-channel.";
3045      }
3046      else
3047      {
3048         Experiment e = bas.getExperiment();
3049         e.checkPermission(Permission.USE);
3050      }
3051   }
3052}
3053</programlisting>
3054
3055    <para>
3056    The plug-in should always include a parameter asking for the current
3057    bioassay set when the <methodname>InteractivePlugin.getRequestInformation()</methodname>
3058    is called with <literal>command = Request.COMMAND_CONFIGURE_JOB</literal>.
3059    </para> 
3060
3061    <programlisting language="java">
3062private static final RequestInformation configurePlugin;
3063private RequestInformation configureJob;
3064private PluginParameter&lt;BioAssaySet&gt; bioAssaySetParameter;
3065
3066public RequestInformation getRequestInformation(GuiContext context, String command)
3067   throws BaseException
3068{
3069   RequestInformation requestInformation = null;
3070   if (command.equals(Request.COMMAND_CONFIGURE_PLUGIN))
3071   {
3072      requestInformation = getConfigurePlugin(context);
3073   }
3074   else if (command.equals(Request.COMMAND_CONFIGURE_JOB))
3075   {
3076      requestInformation = getConfigureJob(context);
3077   }
3078   return requestInformation;
3079}
3080
3081private RequestInformation getConfigureJob(GuiContext context)
3082{
3083   if (configureJob == null)
3084   {
3085      bioAssaySetParameter; = new PluginParameter&lt;BioAssaySet&gt;(
3086         "bioAssaySet",
3087         "Bioassay set",
3088         "The bioassay set used as the source for this analysis plugin",
3089         new ItemParameterType&lt;BioAssaySet&gt;(BioAssaySet.class, null, true, 1, null)
3090      );
3091
3092      List&lt;PluginParameter&lt;?&gt;&gt; parameters = new ArrayList&lt;PluginParameter&lt;?&gt;&gt;();
3093      parameters.add(bioAssaySetParameter);
3094      // Add more plug-in-specific parameters here...
3095   
3096      configureJob = new RequestInformation(
3097         Request.COMMAND_CONFIGURE_JOB,
3098         "Configure job",
3099         "Set parameter for plug-in execution",
3100         parameters
3101      );
3102   }
3103   return configureJob;
3104}
3105</programlisting>
3106
3107    <para>
3108    Of course, the <methodname>InteractivePlugin.configure()</methodname> method needs
3109    to validate and store the bioassay set parameter as well:
3110    </para>
3111 
3112    <programlisting language="java">
3113public void configure(GuiContext context, Request request, Response response)
3114{
3115   String command = request.getCommand();
3116   try
3117   {
3118      if (command.equals(Request.COMMAND_CONFIGURE_PLUGIN))
3119      {
3120         // Validate and store configuration parameters
3121         response.setDone("Plugin configuration complete");
3122      }
3123      else if (command.equals(Request.COMMAND_CONFIGURE_JOB))
3124      {
3125         List&lt;Throwable&gt; errors =
3126            validateRequestParameters(configureJob.getParameters(), request);
3127         if (errors != null)
3128         {
3129            response.setError(errors.size() +
3130               " invalid parameter(s) were found in the request", errors);
3131            return;
3132         }
3133         storeValue(job, request, bioAssaySetParameter);
3134         // Store other plugin-specific parameters
3135     
3136         response.setDone("Job configuration complete", Job.ExecutionTime.SHORT);
3137      }
3138   }
3139   catch (Throwable ex)
3140   {
3141      // Never throw exception, always set response!
3142      response.setError(ex.getMessage(), Arrays.asList(ex));
3143   }
3144}
3145</programlisting>
3146
3147    <para>
3148    Now, the typical <methodname>Plugin.run()</methodname> method loads the specfied bioassay set
3149    and its spot data. It may do some filtering and recalculation of the spot
3150    intensity value(s). In most cases it will store the result as a child bioassay
3151    set with one bioassay for each bioassay in the parent bioassay set.
3152    Here is an example, which just copies the intensity values, while
3153    removing those with a negative value in either channel.
3154    </para>
3155 
3156    <programlisting language="java">
3157public void run(Request request, Response response, ProgressReporter progress)
3158{
3159   DbControl dc = sc.newDbControl();
3160   try
3161   {
3162      BioAssaySet source = (BioAssaySet)job.getParameter("bioAssaySet");
3163      // Reload with current DbControl
3164      source = BioAssaySet.getById(dc, source.getId());
3165      int channels = source.getRawDataType().getChannels();
3166     
3167      // Create transformation and new bioassay set
3168      Job j = Job.getById(dc, job.getId());
3169      Transformation t = source.newTransformation(j);
3170      t.setName("Copy spot intensities &gt;= 0");
3171      dc.saveItem(t);
3172
3173      BioAssaySet result = t.newProduct(null, "new", true);
3174      result.setName("After: Copying spot intensities");
3175      dc.saveItem(result);
3176
3177      // Get query for source data
3178      DynamicSpotQuery query = source.getSpotData();
3179     
3180      // Do not return spots with intensities &lt; 0
3181      for (int ch = 1; ch &lt;= channels; ++ch)
3182      {
3183         query.restrict(
3184            Restrictions.gteq(
3185               Dynamic.column(VirtualColumn.channel(ch)),
3186               Expressions.integer(0)
3187            )
3188         );
3189      }
3190     
3191      // Create batcher and copy data
3192      SpotBatcher batcher = result.getSpotBatcher();
3193      int spotsCopied = batcher.insert(query);
3194      batcher.close();
3195     
3196      // Commit and return
3197      dc.commit();
3198      response.setDone("Copied " + spotsCopied + " spots.");
3199   }
3200   catch (Throwable t)
3201   {
3202      response.setError(t.getMessage(), Arrays.asList(t));
3203   }
3204   finally
3205   {
3206      if (dc != null) dc.close();
3207   }
3208}
3209</programlisting>
3210
3211    <para>
3212    See <xref linkend="core_api.dynamic" />
3213    for more examples of using the analysis API.
3214    </para>
3215
3216    <sect2 id="plugin_developer.analyse.abstractanalysis">
3217      <title>The AbstractAnalysisPlugin class</title>
3218     
3219      <para>
3220        This class is an abstract base class. It is a useful
3221        class for most analysis plug-ins used in an experiment context
3222        to inherit from. Its main
3223        purpose is to define <classname docapi="net.sf.basedb.core">PluginParameter</classname>
3224        objects that are commonly used in analysis plug-ins. This includes:
3225      </para>
3226     
3227      <itemizedlist>
3228      <listitem>
3229        <para>
3230        The source bioassay set:
3231          <methodname>getSourceBioAssaySetParameter()</methodname>,
3232          <methodname>getCurrentBioAssaySet()</methodname>,
3233          <methodname>getSourceBioAssaySet()</methodname>
3234        </para>
3235      </listitem>
3236      <listitem>
3237        <para>
3238        The optional restriction of which bioassays to use.
3239        All bioassays in a bioassay set will be used if this
3240        parameter is empty. This is useful when the plugin only
3241        should run on a subset of bioassays in a bioassay set:
3242          <methodname>getSourceBioAssaysParameter()</methodname>,
3243          <methodname>getSourceBioAssays()</methodname>
3244        </para>
3245      </listitem>
3246      <listitem>
3247        <para>
3248        The name and description of the child bioassay set that
3249        is going to be created by the plug-in:
3250        <methodname>getChildNameParameter()</methodname>,
3251        <methodname>getChildDescriptionParameter()</methodname>
3252        </para>
3253      </listitem>
3254      <listitem>
3255        <para>
3256        The name and description of the transformation that
3257        represents the execution of the plug-in:
3258        <methodname>getTransformationNameParameter()</methodname>,
3259        <methodname>getTransformationName()</methodname>
3260        </para>
3261      </listitem>
3262      </itemizedlist>
3263     
3264    </sect2>
3265   
3266    <sect2 id="plugin_developer.analyse.filterplugin">
3267      <title>The AnalysisFilterPlugin interface</title>
3268     
3269      <para>
3270        The <interfacename docapi="net.sf.basedb.core.plugin">net.sf.basedb.core.plugin.AnalysisFilterPlugin</interfacename> 
3271        is a tagging interface, with no methods, that all analysis plug-ins that only filters
3272        data should implement. The benefit is that they will be linked from the
3273        <guibutton>Filter bioassay set</guibutton> button and not just
3274        the <guibutton>Run analysis</guibutton> button. They will also get
3275        a different icon in the experiment outline to make filtering
3276        transformations appear different from other transformations.
3277      </para>
3278     
3279      <para>
3280        The interface exists purely for making the user interaction better. There is
3281        no harm in not implementing it since the plug-in will always appear in
3282        from the <guibutton>Run analysis</guibutton> button. On the other hand,
3283        it doesn't cost anything to implement the interface since it doesn't
3284        have any methods.
3285      </para>
3286   
3287    </sect2>
3288
3289  </sect1>
3290 
3291  <sect1 id="plugin_developer.other">
3292    <title>Other plug-ins</title>
3293    <para></para>
3294   
3295    <sect2 id="plugin_developer.other.authentication">
3296      <title>Authentication plug-ins</title>
3297     
3298      <para>
3299        BASE provides a plug-in mechanism for authenticating users
3300        (validating the username and password) when they are logging in.
3301        This plug-in mechanism is not the same as the regular plug-in API.
3302        That is, you do not have worry about user interaction or implementing the
3303        <interfacename docapi="net.sf.basedb.core.plugin">Plugin</interfacename> interface.
3304      </para>
3305     
3306      <sect3 id="plugin_developer.other.authentication.internal_external">
3307        <title>Internal vs. external authentation</title>
3308     
3309        <para>
3310          BASE can authenticate users in two ways. Either it uses the internal
3311          authentication or the external authentication. With internal
3312          authentication BASE stores logins and passwords in its own database.
3313          With external authentication this is handled by some external
3314          application. Even with external authentication it is possible to
3315          let BASE cache the logins/passwords. This makes it possible to login to
3316          BASE if the external authentication server is down.
3317        </para>
3318       
3319        <note>
3320          <para>
3321          An external authentication server can only be used to grant or deny
3322          a user access to BASE. It cannot be used to give a user permissions,
3323          or put a user into groups or different roles inside BASE.
3324          </para>
3325        </note>
3326
3327        <para>
3328          The external authentication service is only used when a user logs in.
3329          Now, one or more of several things can happen:
3330         
3331          <itemizedlist>
3332          <listitem>
3333            <para>
3334              The ROOT user is logging on. Internal authentication is always
3335              used for the root user and the authenticator plug-in is never
3336              used.
3337            </para>
3338          </listitem>
3339
3340          <listitem>
3341            <para>
3342              The login is correct and the user is already known to BASE.
3343              If the plug-in supports extra information (name, email, phone, etc.)
3344              and the <property>auth.synchronize</property> setting
3345              is <constant>TRUE</constant> the extra information is copied to
3346              the BASE server.
3347            </para>
3348          </listitem>
3349         
3350          <listitem>
3351            <para>
3352              The login is correct, but the user is not known to BASE. This happens
3353              the first time a user logs in. BASE will create
3354              a new user account. If the driver supports extra information, it
3355              is copied to the BASE server (even if <property>auth.synchronize</property> 
3356              is not set). The new user account will get the default quota
3357              and be added to the all roles and groups which has been
3358              marked as <emphasis>default</emphasis>.
3359             
3360              <note>
3361              <para>
3362                Prior to BASE 2.4 it was hardcoded to add the new user to the
3363                <emphasis>Users</emphasis> role only.
3364              </para>
3365              </note>
3366            </para>
3367          </listitem>
3368
3369          <listitem>
3370            <para>
3371              If password caching is enabled, the password is copied to BASE.
3372              If an expiration timeout has been set, an expiration date
3373              will be calculated and set on the user account. The expiration
3374              date is only checked when the external authentication server is
3375              down.
3376            </para>
3377          </listitem>
3378
3379          <listitem>
3380            <para>
3381              The authentication server says that the login is invalid or
3382              the password is incorrect. The user will not be logged in.
3383              If a user account with the specified login already exists in
3384              BASE, it will be disabled.
3385            </para>
3386          </listitem>
3387         
3388          <listitem>
3389            <para>
3390              The authentication driver says that something else is wrong.
3391              If password caching is enabled, internal authentication will be
3392              used. Otherwise the user will not be logged in. An already
3393              existing account is not modified or disabled.
3394            </para>
3395          </listitem>
3396          </itemizedlist>
3397         
3398          <note>
3399            <para>
3400            The <guilabel>Encrypt password</guilabel> option that is
3401            available on the login page does not work with external
3402            authentication. The simple reason is that the password is
3403            encrypted with a one-way algorithm making it impossible to
3404            call <methodname>Authenticator.authenticate()</methodname>.
3405            </para>
3406          </note>
3407         
3408        </para>
3409      </sect3>
3410     
3411      <sect3 id="plugin_developer.other.authentication.authenticator">
3412        <title>The Authenticator interface</title>
3413       
3414        <para>
3415          To be able to use external authentication you must create a class
3416          that implements the
3417          <interfacename docapi="net.sf.basedb.core.authentication">net.sf.based.core.authentication.Authenticator</interfacename> 
3418          interface. Specify the name of the class in the <property>auth.driver</property> 
3419          setting in <filename>base.config</filename> and
3420          its initialisation parameters in the <property>auth.init</property> setting.
3421        </para>
3422       
3423        <para>
3424          Your class must have a public no-argument constructor. The BASE
3425          application will create only one instance of the class for lifetime
3426          of the BASE server. It must be thread-safe since it may be invoked by
3427          multiple threads at the same time. Here are the methods that you must
3428          implement
3429        </para>
3430       
3431        <variablelist>
3432        <varlistentry>
3433          <term>
3434            <methodsynopsis language="java">
3435              <modifier>public</modifier>
3436              <void/>
3437              <methodname>init</methodname>
3438              <methodparam>
3439                <type>String</type>
3440                <parameter>settings</parameter>
3441              </methodparam>
3442              <exceptionname>AuthenticationException</exceptionname>
3443            </methodsynopsis>
3444          </term>
3445          <listitem>
3446            <para>
3447            This method is called just after the object has been created with its argument
3448            taken from the <property>auth.init</property> setting in your <filename>base.config</filename> 
3449            file. This method is only called once for an instance of the object. The syntax and meaning of
3450            the parameter is driver-dependent and should be documented by the plug-in.
3451            It is irrelevant for the BASE core.
3452            </para>
3453          </listitem>
3454        </varlistentry>
3455       
3456        <varlistentry>
3457          <term>
3458            <methodsynopsis language="java">
3459              <modifier>public</modifier>
3460              <type>boolean</type>
3461              <methodname>supportsExtraInformation</methodname>
3462            </methodsynopsis>
3463          </term>
3464          <listitem>
3465            <para>
3466            This method should simply return <constant>TRUE</constant> or <constant>FALSE</constant> 
3467            depending on if the plug-in supports extra user information or not. The only required
3468            information about a user is a unique ID and the login. Extra information includes
3469            name, address, phone, email, etc.
3470            </para>
3471          </listitem>       
3472        </varlistentry>
3473
3474        <varlistentry>
3475          <term>
3476            <methodsynopsis language="java">
3477              <modifier>public</modifier>
3478              <type>AuthenticationInformation</type>
3479              <methodname>authenticate</methodname>
3480              <methodparam>
3481                <type>String</type>
3482                <parameter>login</parameter>
3483              </methodparam>
3484              <methodparam>
3485                <type>String</type>
3486                <parameter>password</parameter>
3487              </methodparam>
3488              <exceptionname>UnknownLoginException</exceptionname>
3489              <exceptionname>InvalidPasswordException</exceptionname>
3490              <exceptionname>AuthenticationException</exceptionname>
3491            </methodsynopsis>
3492          </term>
3493          <listitem>
3494            <para>
3495            Try to authenticate a login/password combination. The plug-in should return
3496            an <classname docapi="net.sf.basedb.core.authentication">AuthenticationInformation</classname> object if the
3497            authentication is successful or throw an exception if not.
3498           
3499            There are three exceptions to choose from:
3500           
3501            <itemizedlist>
3502            <listitem>
3503              <para>
3504              <exceptionname>UnknownLoginException</exceptionname>:
3505              This exception should be thrown if the login is not known to the
3506              external authentication system.
3507              </para>
3508            </listitem>
3509
3510            <listitem>
3511              <para>
3512              <exceptionname>InvalidPasswordException</exceptionname>:
3513              This exception should be thrown if the login is known but the
3514              password is invalid. In case it is considered a security issue
3515              to reveal that a login exists, the plugin may throw an
3516              <exceptionname>UnknowLoginException</exceptionname> instead.
3517              </para>
3518            </listitem>
3519           
3520            <listitem>
3521              <para>
3522              <exceptionname>AuthenticationException</exceptionname>:
3523              In case there is another problem, such as the authentication service
3524              being down. This exception triggers the use of cached passwords
3525              if caching has been enabled.
3526              </para>
3527            </listitem>
3528            </itemizedlist>
3529            </para>
3530          </listitem>       
3531        </varlistentry>
3532        </variablelist> 
3533      </sect3>
3534     
3535      <sect3 id="plugin_developer.other.authentication.settings">
3536        <title>Configuration settings</title>
3537     
3538        <para>
3539          The configuration settings for the authentication driver are located
3540          in the <filename>base.config</filename> file.
3541          Here is an overview of the settings. For more information read
3542          <xref linkend="appendix.base.config.authentication" />.
3543        </para>
3544       
3545        <variablelist>
3546        <varlistentry>
3547          <term><property>auth.driver</property></term>
3548          <listitem>
3549            <para>
3550            The class name of the authentication plug-in.
3551            </para>
3552          </listitem>
3553        </varlistentry>
3554
3555        <varlistentry>
3556          <term><property>auth.init</property></term>
3557          <listitem>
3558            <para>
3559            Initialisation parameters sent to the plug-in when calling the
3560            <methodname>Authenticator.init()</methodname> method.
3561            </para>
3562          </listitem>
3563        </varlistentry>
3564       
3565        <varlistentry>
3566          <term><property>auth.synchronize</property></term>
3567          <listitem>
3568            <para>
3569            If extra user information is synchronized at login time or not.
3570            This setting is ignored if the driver does not support extra information.
3571            </para>
3572          </listitem>
3573        </varlistentry>
3574       
3575        <varlistentry>
3576          <term><property>auth.cachepasswords</property></term>
3577          <listitem>
3578            <para>
3579              If passwords should be cached by BASE or not. If the passwords are
3580              cached a user may login to BASE even if the external authentication
3581              server is down.
3582            </para>
3583          </listitem>
3584        </varlistentry>
3585       
3586        <varlistentry>
3587          <term><property>auth.daystocache</property></term>
3588          <listitem>
3589            <para>
3590              How many days to cache the passwords if caching has been enabled.
3591              A value of 0 caches the passwords for ever.     
3592            </para>
3593          </listitem>
3594        </varlistentry>
3595        </variablelist>
3596      </sect3>
3597     
3598    </sect2>
3599   
3600    <sect2 id="plugin_developer.other.secondary">
3601      <title>Secondary file storage plugins</title>
3602     
3603      <sect3 id="plugin_developer.other.secondary.vsprimary">
3604        <title>Primary vs. secondary storage</title>
3605        <para>
3606          BASE has support for storing files in two locations, the primary storage and
3607          the secondary storage. The primary storage is always disk-based and must be
3608          accessible by the BASE server as a path on the file system. The path to the
3609          primary storage is configured by the <varname>userfiles</varname> setting in the
3610          <filename>base.config</filename> file. The primary storage is internal to
3611          the core. Client applications don't get access to read or manipulate the
3612          files directly from the file system.
3613        </para>
3614       
3615        <para>
3616          The secondary storage can be anything that can store files. It could, for
3617          example, be another directory, a remote FTP server, or a tape based archiving
3618          system. A file located in the secondary storage is not accessible by the
3619          core, client applications or plug-ins. The secondary storage can only be accessed
3620          by the secondary storage controller. The core (and client) applications uses
3621          flags on the file items to handle the interaction with the secondary storage.
3622        </para>
3623       
3624        <para>
3625          Each file has an <property>action</property> attribute which default's to
3626          <constant>File.Action.NOTHING</constant>. It can take two other values:
3627        </para>
3628       
3629        <orderedlist>
3630        <listitem>
3631          <para>
3632          <constant>File.Action.MOVE_TO_SECONDARY</constant>
3633          </para>
3634        </listitem>
3635        <listitem>
3636          <para>
3637          <constant>File.Action.MOVE_TO_PRIMARY</constant>
3638          </para>
3639        </listitem>
3640        </orderedlist>
3641       
3642        <para>
3643          All files with the action attribute set to <constant>MOVE_TO_SECONDARY</constant> 
3644          should be moved to the secondary storage by the controller, and all files
3645          with the action attribute set to <constant>MOVE_TO_PRIMARY</constant> should be
3646          brought back to primary storage.
3647        </para>
3648       
3649        <para>
3650          The moving of files between primary and secondary storage doesn't happen
3651          immediately. It is up to the server administrator to configure how often and
3652          at what times the controller should check for files that should be moved.
3653          This is configured by the <varname>secondary.storage.interval</varname> 
3654          and <varname>secondary.storage.time</varname> settings in the
3655          <filename>base.config</filename> file.       
3656        </para>
3657      </sect3>
3658     
3659      <sect3 id="plugin_developer.other.secondary.interface">
3660        <title>The SecondaryStorageController interface</title>
3661       
3662        <para>
3663          All you have to do to create a secondary storage controller is to
3664          create a class that implements the
3665          <interfacename docapi="net.sf.basedb.core">net.sf.basedb.core.SecondaryStorageController</interfacename>
3666          interface. In your <filename>base.config</filename> file you then specify the
3667          class name in the <varname>secondary.storage.driver</varname> setting and its
3668          initialisation parameters in the <varname>secondary.storage.init</varname> setting.
3669        </para>
3670
3671        <para>
3672          Your class must have a public no-argument constructor.
3673          The BASE application will create only one instance of the class for
3674          lifetime of the BASE server. Here are the methods that you must implement:
3675        </para>
3676       
3677        <variablelist>
3678        <varlistentry>
3679          <term>
3680          <methodsynopsis language="java">
3681            <modifier>public</modifier>
3682            <void />
3683            <methodname>init</methodname>
3684            <methodparam>
3685              <type>String</type>
3686              <parameter>settings</parameter>
3687            </methodparam>
3688          </methodsynopsis>
3689          </term>
3690          <listitem>
3691            <para>
3692            This method is called just after the object has been created with its argument
3693            taken from the <varname>secondary.storage.init</varname> setting in your
3694            <filename>base.config</filename> file. This method is only called once for
3695            an object.
3696            </para>
3697          </listitem>
3698        </varlistentry>
3699        <varlistentry>
3700          <term>
3701          <methodsynopsis language="java">
3702            <modifier>public</modifier>
3703            <void />
3704            <methodname>run</methodname>
3705          </methodsynopsis>
3706          </term>
3707          <listitem>
3708            <para>
3709            This method is called whenever the core thinks it is time to do some
3710            management of the secondary storage. How often the <methodname>run()</methodname>
3711            method is called is controlled by the <varname>secondary.storage.interval</varname>
3712            and <varname>secondary.storage.time</varname> settings in the
3713            <filename>base.config</filename> file.
3714            When this method is called the controller should:
3715            </para>
3716           
3717            <itemizedlist>
3718            <listitem>
3719              <para>
3720              Move all files which has <constant>action=MOVE_TO_SECONDARY</constant> to
3721              the secondary storage. When the file has been moved call
3722              <methodname>File.setLocation(Location.SECONDARY)</methodname> to tell the
3723              core that the file is now in the secondary storage. You should also call
3724              <methodname>File.setAction(File.Action.NOTHING)</methodname> to reset the
3725              action attribute.
3726              </para>
3727            </listitem>
3728           
3729            <listitem>
3730              <para>
3731              Restore all files which has <constant>action=MOVE_TO_PRIMARY</constant>.
3732              The core will set the location attribute automatically, but you should
3733              call <methodname>File.setAction(File.Action.NOTHING)</methodname> to reset
3734              the action attribute.
3735              </para>
3736            </listitem>
3737           
3738            <listitem>
3739              <para>
3740              Delete all files from the secondary storage that are not present
3741              in the database with <constant>location=Location.SECONDARY</constant>.
3742              This includes files which has been deleted and files that have been
3743              moved offline or re-uploaded.
3744              </para>
3745            </listitem>
3746           
3747            </itemizedlist>
3748           
3749            <para>
3750              As a final act the method should send a message to each user owning
3751              files that has been moved from one location to the other. The message
3752              should include a list of files that has been moved to the secondary
3753              storage and a list of files moved from the secondary storage and a
3754              list of files that has been deleted due to some of the reasons above.
3755            </para>
3756          </listitem>
3757        </varlistentry>       
3758       
3759        <varlistentry>
3760          <term>
3761          <methodsynopsis language="java">
3762            <modifier>public</modifier>
3763            <void />
3764            <methodname>close()</methodname>
3765          </methodsynopsis>
3766          </term>
3767          <listitem>
3768            <para>
3769            This method is called when the server is closing down. After this the object
3770            is never used again.
3771            </para>
3772          </listitem>
3773        </varlistentry>
3774        </variablelist>
3775      </sect3>
3776     
3777      <sect3 id="plugin_developer.other.secondary.settings">
3778        <title>Configuration settings</title>
3779       
3780        <para>
3781        The configuration settings for the secondary storage controller is located in the
3782        <filename>base.config</filename> file. Here is an overview of the settings.
3783        For more information read <xref linkend="appendix.base.config" />.
3784        </para>
3785       
3786        <variablelist>
3787        <varlistentry>
3788          <term><property>secondary.storage.driver</property></term>
3789          <listitem>
3790            <para>
3791            The class name of the secondary storage plug-in.
3792            </para>
3793          </listitem>
3794        </varlistentry>
3795        <varlistentry>
3796          <term><property>secondary.storage.init</property></term>
3797          <listitem>
3798            <para>
3799             Initialisation parameters sent to the plug-in by calling the
3800             <methodname>init()</methodname> method.
3801            </para>
3802          </listitem>
3803        </varlistentry>
3804        <varlistentry>
3805          <term><property>secondary.storage.interval</property></term>
3806          <listitem>
3807            <para>
3808             Interval in seconds between each execution of the secondary storage
3809             controller plug-in.
3810            </para>
3811          </listitem>
3812        </varlistentry>
3813        <varlistentry>
3814          <term><property>secondary.storage.time</property></term>
3815          <listitem>
3816            <para>
3817              Time points during the day when the secondary storage controller plugin
3818              should be executed.
3819            </para>
3820          </listitem>
3821        </varlistentry>
3822        </variablelist>
3823      </sect3>
3824    </sect2>
3825   
3826    <sect2 id="plugin_developer.other.unpacker">
3827      <title>File unpacker plug-ins</title>
3828      <para>
3829        The BASE web client has integrated support for unpacking of
3830        compressed files. See <xref linkend="file_system.handling.upload" />.
3831        Behind the scenes, this support is provided by plug-ins. The standard
3832        BASE distribution comes with support for ZIP files
3833        (<classname docapi="net.sf.basedb.plugins">net.sf.basedb.plugins.ZipFileUnpacker</classname>)
3834        and TAR files (<classname docapi="net.sf.basedb.plugins">net.sf.basedb.plugins.TarFileUnpacker</classname>).
3835      </para>
3836      <para>
3837        To add support for additional compressed formats you have to create a plug-in that
3838        implements the <interfacename docapi="net.sf.basedb.util.zip">net.sf.basedb.util.zip.FileUnpacker</interfacename>
3839        interface. The best way to do this is to extend the
3840        <classname docapi="net.sf.basedb.util.zip">net.sf.basedb.util.zip.AbstractFileUnpacker</classname> which
3841        implements all methods in the <interfacename docapi="net.sf.basedb.core.plugin">Plugin</interfacename>
3842        and <interfacename docapi="net.sf.basedb.core.plugin">InteractivePlugin</interfacename>
3843        interfaces except <methodname>Plugin.getAbout()</methodname>. This leaves
3844        you with the actual unpacking of the files as the only thing to implement.
3845      </para>
3846     
3847      <note>
3848        <title>No support for configurations</title>
3849        The integrated upload in the web interface only works with plug-ins that
3850        does not require a configuration to run.
3851      </note>
3852     
3853      <variablelist>
3854        <title>Methods in the <interfacename docapi="net.sf.basedb.util.zip">FileUnpacker</interfacename> interface</title>
3855        <varlistentry>
3856          <term>
3857            <methodsynopsis language="java">
3858              <modifier>public</modifier>
3859              <type>String</type>
3860              <methodname>getFormatName</methodname>
3861            </methodsynopsis>
3862          </term>
3863          <listitem>
3864            <para>
3865            Return a short string naming the file format. For example:
3866            <constant>ZIP files</constant> or <constant>TAR files</constant>.
3867            </para>
3868          </listitem>
3869        </varlistentry>
3870           
3871        <varlistentry>
3872          <term>
3873            <methodsynopsis language="java">
3874              <modifier>public</modifier>
3875              <type>Set&lt;String&gt;</type>
3876              <methodname>getExtensions</methodname>
3877            </methodsynopsis>
3878          </term>
3879          <listitem>
3880            <para>
3881            Return a set of strings with the file extensions that
3882            are most commonly used with the compressed file format.
3883            For example: <constant>[zip, jar]</constant>. Do not include
3884            the dot in the extensions. The web client and the
3885            <methodname>AbstractFlatFileUnpacker.isInContext()</methodname> method
3886            will use this information to automatically guess which plug-in to
3887            use for unpacking the files.
3888            </para>
3889          </listitem>
3890        </varlistentry>
3891       
3892        <varlistentry>
3893          <term>
3894            <methodsynopsis language="java">
3895              <modifier>public</modifier>
3896              <type>Set&lt;String&gt;</type>
3897              <methodname>getMimeTypes</methodname>
3898            </methodsynopsis>
3899          </term>
3900          <listitem>
3901            <para>
3902            Return a set of string with the MIME types that commonly used with
3903            the compressed file format. For example:
3904            <constant>[application/zip, application/java-archive]</constant>.
3905            This information is used by the
3906            <methodname>AbstractFlatFileUnpacker.isInContext()</methodname> 
3907            method to automatically guess which plug-in to use for unpacking
3908            the files.
3909            </para>
3910          </listitem>
3911        </varlistentry>
3912       
3913        <varlistentry>
3914          <term>
3915            <methodsynopsis language="java">
3916              <modifier>public</modifier>
3917              <type>int</type>
3918              <methodname>unpack</methodname>
3919              <methodparam>
3920                <type>DbControl</type>
3921                <parameter>dc</parameter>
3922              </methodparam>
3923              <methodparam>
3924                <type>Directory</type>
3925                <parameter>dir</parameter>
3926              </methodparam>
3927              <methodparam>
3928                <type>InputStream</type>
3929                <parameter>in</parameter>
3930              </methodparam>
3931              <methodparam>
3932                <type>boolean</type>
3933                <parameter>overwrite</parameter>
3934              </methodparam>
3935              <methodparam>
3936                <type>AbsoluteProgressReporter</type>
3937                <parameter>progress</parameter>
3938              </methodparam>
3939              <exceptionname>IOException</exceptionname>
3940              <exceptionname>BaseException</exceptionname>
3941            </methodsynopsis>
3942          </term>
3943          <listitem>
3944            <para>
3945            Unpack the files and store them in the BASE file system.
3946           
3947            <itemizedlist>
3948            <listitem>
3949              <para>
3950              Do not <methodname>close()</methodname> or
3951              <methodname>commit()</methodname> the
3952              <classname docapi="net.sf.basedb.core">DbControl</classname> passed to this method.
3953              This is done automatically by the
3954              <classname docapi="net.sf.basedb.util.zip">AbstractFileUnpacker</classname> or by the web client.
3955              </para>
3956            </listitem>
3957           
3958            <listitem>
3959              <para>
3960              The <varname>dir</varname> parameter is the root directory where
3961              the unpacked files should be placed. If the compressed file
3962              contains subdirectories the plug-in must create those subdirectories
3963              unless they already exists.
3964              </para>
3965            </listitem>
3966           
3967            <listitem>
3968              <para>
3969              If the <varname>overwrite</varname> parameter is
3970              <constant>FALSE</constant> no existing file should be overwritten
3971              unless the file is <systemitem>OFFLINE</systemitem>.
3972              </para>
3973            </listitem>
3974           
3975            <listitem>
3976              <para>
3977              The <varname>in</varname> parameter is the stream
3978              containing the compressed data. The stream may come
3979              directly from the web upload or from an existing
3980              file in the BASE file system.
3981              </para>
3982            </listitem>
3983           
3984            <listitem>
3985              <para>
3986              The <varname>progress</varname> parameter, if not
3987              <constant>null</constant>, should be used to report the
3988              progress back to the calling code. The plug-in should count
3989              the number of bytes read from the <varname>in</varname>
3990              stream. If it is not possible by other means the stream can
3991              be wrapped by a <classname docapi="net.sf.basedb.util">net.sf.basedb.util.InputStreamTracker</classname>
3992              object which has a <methodname>getNumRead()</methodname> method.
3993              </para>
3994            </listitem>
3995            </itemizedlist>
3996           
3997            </para>
3998          </listitem>
3999        </varlistentry>
4000      </variablelist>
4001     
4002      <para>
4003        When the compressed file is uncompressed during the file upload
4004        from the web interface, the call sequence to the plug-in is slightly
4005        altered from the standard call sequence described in
4006        <xref linkend="plugin_developer.api.callsequence.execute" />.
4007       
4008        <itemizedlist>
4009        <listitem>
4010          <para>
4011          After the plug-in instance has been created, the
4012          <methodname>Plugin.init()</methodname> method is called with <constant>null</constant>
4013          values for both the <varname>configuration</varname> and <varname>job</varname>
4014          parameters.
4015          </para>
4016        </listitem>
4017       
4018        <listitem>
4019          <para>
4020          Then, the <methodname>unpack()</methodname> method is called. The
4021          <methodname>Plugin.run()</methodname> method is never called in this case.
4022          </para>
4023        </listitem>
4024       
4025        </itemizedlist>
4026       
4027      </para>
4028     
4029    </sect2>
4030   
4031    <sect2 id="plugin_developer.other.packer">
4032      <title>File packer plug-ins</title>
4033   
4034      <para>
4035        BASE has support for compressing and downloading a set of selected files and/or
4036        directories. This functionality is provided by a plug-in, the
4037        <classname docapi="net.sf.basedb.plugins">PackedFileExporter</classname>. This plug-in doesn't do the actual
4038        packing itself. This is delegated to classes implementing the
4039        <interfacename docapi="net.sf.basedb.util.zip">net.sf.basedb.util.zip.FilePacker</interfacename> interface.
4040      </para>
4041     
4042      <para>
4043        BASE ships with a number of packing methods, including ZIP and TAR. To
4044        add support for other methods you have to provide an implementation
4045        of the <interfacename docapi="net.sf.basedb.util.zip">FilePacker</interfacename>
4046        interface. Then, create a new configuration for the <classname docapi="net.sf.basedb.plugins">PackedFileExporter</classname>
4047        and enter the name of your class in the configuration wizard.
4048      </para>
4049     
4050      <para>
4051        The <interfacename docapi="net.sf.basedb.util.zip">FilePacker</interfacename> interface is not a regular
4052        plug-in interface (ie. it is not a subinterface to
4053        <interfacename docapi="net.sf.basedb.core.plugin">Plugin</interfacename>). This means that you don't have to
4054        mess with configuration or job parameters. Another difference is that your
4055        class must be installed in Tomcat's classpath (ie. in one of the
4056        <filename>WEB-INF/classes</filename> or <filename>WEB-INF/lib</filename>
4057        folders).
4058      </para>
4059     
4060      <variablelist>
4061        <title>Methods in the <interfacename docapi="net.sf.basedb.util.zip">FilePacker</interfacename> interface</title>
4062        <varlistentry>
4063          <term>
4064            <methodsynopsis language="java">
4065              <modifier>public</modifier>
4066              <type>String</type>
4067              <methodname>getDescription</methodname>
4068            </methodsynopsis>
4069          </term>
4070          <listitem>
4071            <para>
4072            Return a short description the file format that is suitable for use
4073            in dropdown lists in client applications. For example:
4074            <constant>Zip-archive (.zip)</constant> or <constant>TAR-archive (.tar)</constant>.
4075            </para>
4076          </listitem>
4077        </varlistentry>
4078        <varlistentry>
4079          <term>
4080            <methodsynopsis language="java">
4081              <modifier>public</modifier>
4082              <type>String</type>
4083              <methodname>getFileExtension</methodname>
4084            </methodsynopsis>
4085          </term>
4086          <listitem>
4087            <para>
4088            Return the default file extension of the packed format. The returned
4089            value should not include the dot. For example:
4090            <constant>zip</constant> or <constant>tar</constant>.
4091            </para>
4092          </listitem>
4093        </varlistentry>
4094        <varlistentry>
4095          <term>
4096            <methodsynopsis language="java">
4097              <modifier>public</modifier>
4098              <type>String</type>
4099              <methodname>getMimeType</methodname>
4100            </methodsynopsis>
4101          </term>
4102          <listitem>
4103            <para>
4104            Return the standard MIME type of the packed file format.
4105            For example:
4106            <constant>application/zip</constant> or <constant>application/x-tar</constant>.
4107            </para>
4108          </listitem>
4109        </varlistentry>
4110        <varlistentry>
4111          <term>
4112            <methodsynopsis language="java">
4113              <modifier>public</modifier>
4114              <void />
4115              <methodname>setOutputStream</methodname>
4116              <methodparam>
4117                <type>OutputStream</type>
4118                <parameter>out</parameter>
4119              </methodparam>
4120              <exceptionname>IOException</exceptionname>
4121            </methodsynopsis>
4122          </term>
4123          <listitem>
4124            <para>
4125            Sets the outputstream that the packer should write the packed
4126            files to.
4127            </para>
4128          </listitem>
4129        </varlistentry>
4130        <varlistentry>
4131          <term>
4132            <methodsynopsis language="java">
4133              <modifier>public</modifier>
4134              <void />
4135              <methodname>pack</methodname>
4136              <methodparam>
4137                <type>String</type>
4138                <parameter>entryName</parameter>
4139              </methodparam>
4140              <methodparam>
4141                <type>InputStream</type>
4142                <parameter>in</parameter>
4143              </methodparam>
4144              <methodparam>
4145                <type>long</type>
4146                <parameter>size</parameter>
4147              </methodparam>
4148              <methodparam>
4149                <type>long</type>
4150                <parameter>lastModified</parameter>
4151              </methodparam>
4152              <exceptionname>IOException</exceptionname>
4153            </methodsynopsis>
4154          </term>
4155          <listitem>
4156            <para>
4157            Add another file or directory to the packed file. The
4158            <parameter>entryName</parameter> is the name of the new entry, including
4159            path information. The <parameter>in</parameter> is the stream to read
4160            the file data from. If <parameter>in</parameter> is <constant>null</constant>
4161            then the entry denotes a directory. The <parameter>size</parameter> parameter
4162            gives the size in bytes of the file (zero for empty files or directories).
4163            The <parameter>lastModified</parameter>
4164            is that time the file was last modified or 0 if not known.
4165            </para>
4166          </listitem>
4167        </varlistentry>
4168        <varlistentry>
4169          <term>
4170            <methodsynopsis language="java">
4171              <modifier>public</modifier>
4172              <void />
4173              <methodname>close</methodname>
4174              <exceptionname>IOException</exceptionname>
4175            </methodsynopsis>
4176          </term>
4177          <listitem>
4178            <para>
4179            Finish the packing. The packer should release any resources, flush
4180            all data and close all output streams, including the <varname>out</varname> stream
4181            set in the <methodname>setOutputStream</methodname> method.
4182            </para>
4183          </listitem>
4184        </varlistentry>
4185       
4186      </variablelist>
4187     
4188    </sect2>
4189   
4190    <sect2 id="plugin_developer.other.logging">
4191      <title>Logging plug-ins</title>
4192     
4193      <para>
4194        BASE provides a plug-in mechanism for logging changes that are made to items.
4195        This plug-in mechanism is not the same as the regular plug-in API. That is, you do not
4196        have worry about user interaction or implementing the <interfacename 
4197        docapi="net.sf.basedb.core.plugin">Plugin</interfacename> interface.
4198      </para>
4199     
4200      <para>
4201        The logging mechanism works on the data layer level and hooks into
4202        callbacks provided by Hibernate. <interfacename 
4203        docapi="net.sf.basedb.core.log">EntityLogger</interfacename>:s are used to
4204        extract relevant information from Hibernate and create log entries.
4205        While it is possible to have a generic logger it is usually better
4206        to have different implementations depending on the type of entity that
4207        was changed. For example, a change in a child item should, for usability
4208        reasons, be logged as a change in the parent item. Entity loggers
4209        are created by a <interfacename 
4210        docapi="net.sf.basedb.core.log">LogManagerFactory</interfacename>. All changes
4211        made in a single transaction are usually collected by a <interfacename 
4212        docapi="net.sf.basedb.core.log">LogManager</interfacename> which is also
4213        created by the factory.
4214      </para>
4215     
4216      <sect3 id="plugin_developer.other.logmanagerfactory">
4217        <title>The LogManagerFactory interface</title>
4218       
4219        <para>
4220          Which <interfacename 
4221          docapi="net.sf.basedb.core.log">LogManagerFactory</interfacename> to use
4222          is configured in <filename>base.config</filename> (See <xref 
4223          linkend="appendix.base.config.log"/>). A single factory instance is created
4224          when BASE starts and is used for the lifetime of the virtual machine. The
4225          factory implementation must of course be thread-safe. Here is a list of
4226          the methods the factory must implement:
4227        </para>
4228       
4229        <variablelist>
4230          <varlistentry>
4231            <term>
4232              <methodsynopsis language="java">
4233                <modifier>public</modifier>
4234                <type>LogManager</type>
4235                <methodname>getLogManager</methodname>
4236                <methodparam>
4237                  <type>LogControl</type>
4238                  <parameter>logControl</parameter>
4239                </methodparam>
4240              </methodsynopsis>
4241            </term>
4242            <listitem>
4243              <para>
4244              Creates a log manager for a single transaction. Since a transaction
4245              is not thread-safe the log manager implementation doesn't have to
4246              be either. The factory has the possibility to create new log managers
4247              for each transaction.
4248              </para>
4249            </listitem>
4250          </varlistentry>
4251         
4252          <varlistentry>
4253            <term>
4254              <methodsynopsis language="java">
4255                <modifier>public</modifier>
4256                <type>boolean</type>
4257                <methodname>isLoggable</methodname>
4258                <methodparam>
4259                  <type>Object</type>
4260                  <parameter>entity</parameter>
4261                </methodparam>
4262              </methodsynopsis>
4263            </term>
4264            <listitem>
4265              <para>
4266              Checks if changes to the given entity should be
4267              logged or not. For performance reasons, it usually makes sense to
4268              not log everything. For example, the database logger implementation
4269              only logs changes if the entity implements the <interfacename 
4270              docapi="net.sf.basedb.core.data">LoggableData</interfacename>
4271              interface. The return value of this method should be consistent
4272              with <methodname>getEntityLogger()</methodname>.
4273              </para>
4274            </listitem>
4275          </varlistentry>
4276
4277          <varlistentry>
4278            <term>
4279              <methodsynopsis language="java">
4280                <modifier>public</modifier>
4281                <type>EntityLogger</type>
4282                <methodname>getEntityLogger</methodname>
4283                <methodparam>
4284                  <type>LogManager</type>
4285                  <parameter>logManager</parameter>
4286                </methodparam>
4287                <methodparam>
4288                  <type>Object</type>
4289                  <parameter>entity</parameter>
4290                </methodparam>
4291              </methodsynopsis>
4292            </term>
4293            <listitem>
4294              <para>
4295              Create or get an entity logger that knows how to log
4296              changes to the given entity. If the entity should not be
4297              logged, <constant>null</constant> can be returned. This method
4298              is called for each modified item in the transaction.
4299              </para>
4300            </listitem>
4301          </varlistentry>
4302        </variablelist>
4303       
4304      </sect3>
4305     
4306      <sect3 id="plugin_developer.other.logmanager">
4307        <title>The LogManager interface</title>
4308       
4309        <para>
4310          A new log manager is created for each transaction. The log manager
4311          is responsible for collecting all changes made in the transaction
4312          and store those changes in the appropriate place. The interface doesn't define
4313          any methods for this collection, since each implementation may have
4314          very different needs.
4315        </para>
4316       
4317        <variablelist>
4318          <varlistentry>
4319            <term>
4320              <methodsynopsis language="java">
4321                <modifier>public</modifier>
4322                <type>LogControl</type>
4323                <methodname>getLogControl</methodname>
4324              </methodsynopsis>
4325            </term>
4326            <listitem>
4327              <para>
4328              Get the log control object that was supplied by the BASE core
4329              when the transaction was started. The log controller contains
4330              methods for accessing information about the transaction, such
4331              as the logged in user, executing plug-in, etc. It can also
4332              be used to execute queries against the database to get
4333              even more information.
4334              </para>
4335             
4336              <warning>
4337                <para>
4338                  Be careful about the queries that are executed by the log
4339                  controller. Since all logging code is executed at flush
4340                  time in callbacks from Hibernate we are not allowed to
4341                  use the regular session. Instead, all queries are sent
4342                  through the stateless session. The stateless session has no
4343                  caching functionality which means that Hibernate will use
4344                  extra queries to load associations. Our recommendation is
4345                  to avoid quires that return full entities, use scalar
4346                  queries instead to just load the values that are needed.
4347                </para>
4348              </warning>
4349            </listitem>
4350          </varlistentry>
4351          <varlistentry>
4352            <term>
4353              <methodsynopsis language="java">
4354                <modifier>public</modifier>
4355                <void/>
4356                <methodname>afterCommit</methodname>
4357              </methodsynopsis>
4358            </term>
4359            <term>
4360              <methodsynopsis language="java">
4361                <modifier>public</modifier>
4362                <void/>
4363                <methodname>afterRollback</methodname>
4364              </methodsynopsis>
4365            </term>
4366            <listitem>
4367              Called after a successful commit or after a rollback. Note
4368              that the connection to the database has been closed at this
4369              time and it is not possible to save any more information to
4370              it at this time.
4371            </listitem>
4372          </varlistentry>
4373        </variablelist>
4374      </sect3>
4375
4376      <sect3 id="plugin_developer.other.entitylogger">
4377        <title>The EntityLogger interface</title>
4378       
4379        <para>
4380          An entity logger is responsible for extracting the changes
4381          made to an entity and converting it to something that is useful
4382          as a log entry. In most cases, this is not very complicated, but
4383          in some cases, a change in one entity should actually be logged
4384          as a change in a different entity. For example, changes to
4385          annotations are handled by the <classname 
4386          docapi="net.sf.basedb.core.log.db">AnnotationLogger</classname> which
4387          which log it as a change on the parent item.
4388        </para>
4389       
4390        <variablelist>
4391          <varlistentry>
4392            <term>
4393              <methodsynopsis language="java">
4394                <modifier>public</modifier>
4395                <void/>
4396                <methodname>logChanges</methodname>
4397                <methodparam>
4398                  <type>LogManager</type>
4399                  <parameter>logManager</parameter>
4400                </methodparam>
4401                <methodparam>
4402                  <type>EntityDetails</type>
4403                  <parameter>details</parameter>
4404                </methodparam>
4405              </methodsynopsis>
4406            </term>
4407            <listitem>
4408              <para>
4409              This method is called whenever a change has been detected
4410              in an entity. The <varname>details</varname> variable contains
4411              information about the entity and, to a certain degree,
4412              what changes that has been made.
4413              </para>
4414            </listitem>
4415          </varlistentry>
4416        </variablelist>
4417      </sect3>
4418    </sect2>
4419  </sect1>
4420 
4421  <sect1 id="plugin_developer.signals">
4422    <title>Enable support for aborting a running a plug-in</title>
4423   
4424    <para>
4425      BASE includes a simple signalling system that can be used to send
4426      signals to plug-ins. The system was primarly developed to allow a user
4427      to kill a plug-in when it is executing. Therfore, the focus of this chapter
4428      will be how to implement a plug-in to make it possible to kill it
4429      during it's execution.
4430    </para>
4431   
4432    <para>
4433      Since we don't want to do this by brute force such as destroying the
4434      process or stopping thread the plug-in executes in, cooperation is needed
4435      by the plug-in. First, the plug-in must implement the
4436      <interfacename docapi="net.sf.basedb.core.signal">SignalTarget</interfacename>
4437      interface. From this, a <interfacename 
4438      docapi="net.sf.basedb.core.signal">SignalHandler</interfacename> can be created.
4439      A plug-in may choose to implement it's own signal handler or use an existing
4440      implementation. BASE, for example, provides the <classname docapi="net.sf.basedb.core.signal"
4441      >ThreadSignalHandler</classname> implementation that supports the <constant>ABORT</constant> signal.
4442      This is a simple implementation that just calls <code>Thread.interrupt()</code>
4443      on the plug-in worker thread. This may cause two different effects:
4444    </para>
4445   
4446    <itemizedlist>
4447    <listitem>
4448      <para>
4449      The <code>Thread.interrupted()</code> flag is set. The plug-in must check this
4450      at regular intervals and if the flag is set it must cleanup, rollback
4451      open transactions and exit as soon as possible.
4452      </para>
4453    </listitem>
4454   
4455    <listitem>
4456      <para>
4457      If the plug-in is waiting in a blocking call that is interruptable, for
4458      example <code>Thread.sleep()</code>, an <classname>InterruptedException</classname>
4459      is thrown. This should cause the same actions as if the flag was set to happen.
4460      </para>
4461     
4462      <warning>
4463        <title>Not all blocking calls are interruptable</title>
4464        <para>
4465        For example calling <code>InputStream.read()</code> may leave the
4466        plug-in waiting in a non-interruptable state. In this case there
4467        is nothing BASE can do to wake it up again.
4468        </para>
4469      </warning>
4470    </listitem>
4471    </itemizedlist>
4472   
4473    <para>
4474      Here is a general outline for a plug-in that uses the
4475      <classname>ThreadSignalHandler</classname>.
4476    </para>
4477    <programlisting language="java">
4478<![CDATA[
4479private ThreadSignalHandler signalHandler;
4480public SignalHandler getSignalHandler()
4481{
4482   signalHandler = new ThreadSignalHandler();
4483   return signalHandler;
4484}
4485
4486public void run(Request request, Response response, ProgressReporter progress)
4487{
4488   if (signalHandler != null) signalHandler.setWorkerThread(null);
4489   beginTransaction();
4490   boolean done = false;
4491   boolean interrupted = false;
4492   while (!done && !interrupted)
4493   {
4494      try
4495      {
4496         done = doSomeWork(); // NOTE! This must not take forever!
4497         interrupted = Thread.interrupted();
4498      }
4499      catch (InterruptedException ex)
4500      {
4501         // NOTE! Try-catch is only needed if thread calls
4502         // a blocking method that is interruptable
4503         interrupted = true;
4504      }
4505   }
4506   if (interrupted)
4507   {
4508      rollbackTransaction();
4509      response.setError("Aborted by user", null);
4510   }
4511   else
4512   {
4513      commitTransaction();
4514      response.setDone("Done");
4515   }
4516}
4517]]>
4518</programlisting>
4519   
4520    <para>
4521      Other signal handler implementations are
4522      <classname docapi="net.sf.basedb.core.signal">ProgressReporterSignalHandler</classname>
4523      and <classname docapi="net.sf.basedb.core.signal">EnhancedThreadSignalHandler</classname>.
4524      The latter handler also has support for the <constant>SHUTDOWN</constant> signal
4525      which is sent to plug-in when the system is shutting down. Clever plug-ins may use
4526      this to enable them to be restarted when the system is up and running again.
4527      See that javadoc for information about how to use it. For more information about the
4528      signalling system as a whole, see <xref linkend="core_api.signals" />.
4529    </para>
4530   
4531  </sect1>
4532 
4533  <sect1 id="plugin_developer.classload">
4534    <title>How BASE load plug-in classes</title>
4535   
4536    <para>
4537      All plug-ins should be installed in the location specified by the
4538      <varname>plugins.dir</varname> setting in <filename>base.config</filename>.
4539      While it is possible to also install them in a location that is on the
4540      classpath, for example <filename>&lt;base-dir&gt;/www/WEB-INF/lib</filename>,
4541      it is nothing that we recommend. The rest of the information in this section
4542      only applies to plug-ins that have been installed in the <varname>plugins.dir</varname>
4543      location.
4544    </para>
4545   
4546    <para>
4547      If the above recommendation has been followed BASE will use it's
4548      own classloader to load the plug-in classes. This have several
4549      benefits:
4550    </para>
4551   
4552    <itemizedlist>
4553    <listitem>
4554      <para>
4555      New plug-ins can be installed and existing plug-ins can be updated
4556      without restarting the web server. If the <property>plugins.autounload</property>
4557      setting in <filename>base.config</filename> has been enabled all you have to
4558      do to update a plug-in is to replace the JAR file with a new version.
4559      BASE will automatically load the new classes the next time the plug-in
4560      is used. If the option isn't enabled, the server admin has to manually
4561      unload the old code from the web interface first.
4562      </para>
4563    </listitem>
4564   
4565    <listitem>
4566      <para>
4567      Plug-ins may use it's own 3-rd party libraries without interfering
4568      with other plug-ins. This may be important because a plug-in may depend on
4569      a certain version of a library while another plug-in may depend on
4570      a different version. Since BASE is using different class-loaders
4571      for different plug-ins this is not a problem.
4572      </para>
4573    </listitem>
4574     
4575    </itemizedlist>
4576   
4577    <para>
4578      The classloading scheme used by BASE also means plug-in
4579      developers must pay attention to a few things:
4580    </para>
4581   
4582    <itemizedlist>
4583    <listitem>
4584      <para>
4585      A plug-in can only access/use classes from it's own JAR file, BASE core
4586      classes, Java system classes and from JAR files listed in the plug-in's
4587      <filename>MANIFEST.MF</filename> file. See <xref linkend="plugin_developer.organize" />.
4588      </para>
4589    </listitem>
4590   
4591    <listitem>
4592      <para>
4593      A plug-in can also access other plug-ins, but only via the methods and
4594      interfaces defined in BASE. In the following example we assume that
4595      there are two plug-ins, <classname>ex.MyPlugin</classname> 
4596      and <classname>ex.MyOtherPlugin</classname>,
4597      located in two different JAR files. The code below is executing
4598      in the <classname>ex.MyPlugin</classname>:
4599      </para>
4600     
4601      <programlisting language="java">
4602// Prepare to load MyOtherPlugin
4603SessionControl sc = ...
4604DbControl dc = ...
4605PluginDefinition def = PluginDefinition.getByClassName(dc, "ex.MyOtherPlugin");
4606
4607// Ok
4608Plugin other = def.newInstance();
4609
4610// Not ok; fails with ClassCastException
4611MyOtherPlugin other = (MyOtherPlugin)def.newInstance();
4612
4613// Ok; since now we are using the correct class loader
4614MyOtherPlugin other = def.newInstance(MyOtherPlugin.class);
4615</programlisting>
4616     
4617      <para>
4618      The first call succeeds because it uses the <interfacename 
4619      docapi="net.sf.basedb.core.plugin">Plugin</interfacename>
4620      interface which is defined by BASE. This class is loaded by the web servers
4621      class loader and is the same for all plug-ins.
4622      </para>
4623     
4624      <para>
4625      The second call fails because BASE uses a different classloader to load the
4626      <classname>ex.MyOtherPlugin</classname> class. This
4627      class is not (in Java terms) the same as the <classname>ex.MyOtherPlugin</classname> 
4628      class loaded by the classloader that loaded the <classname>ex.MyPlugin</classname>
4629      class. If, on the other hand, both plug-ins are located in the same
4630      JAR file BASE uses the same classloader and the second call will succeed.
4631      </para>
4632     
4633      <para>
4634      The third call succeeds because now that we specify the class as an argument,
4635      BASE uses that classloader instead.
4636      </para>
4637     
4638      <para>
4639      Another option is that the <classname>ex.MyPlugin</classname>
4640      lists the JAR file where <classname>ex.MyOtherPlugin</classname> is
4641      located in it's <filename>MANIFEST.MF</filename> file. Then, the following
4642      code can be used: <code>MyOtherPlugin other = new MyOtherPlugin();</code>
4643      </para>
4644     
4645    </listitem>
4646    </itemizedlist>
4647   
4648    <para>
4649      Tomcat includes a good document describing how classloading is implemented
4650      in Tomcat: <ulink url="http://tomcat.apache.org/tomcat-6.0-doc/class-loader-howto.html"
4651        >http://tomcat.apache.org/tomcat-5.5-doc/class-loader-howto.html</ulink>.
4652     
4653      BASE's classloading scheme isn't as complex as Tomcat's, but it
4654      very similar to how Tomcat loads different web applications.
4655      The figure on the linked document could be extended with another level
4656      with separate classloaders for each plug-in as child
4657      classloaders to the web application classloaders. 
4658    </para>
4659   
4660    <para>
4661      As of BASE 2.13 the default search order for classes has been changed. The
4662      default is now to first look in the plug-ins class path (eg. in the same
4663      JAR file and in files listed in the <filename>MANIFEST.MF</filename> file).
4664      Only if the class is not found the search is delegated to the parent class
4665      loader. This behaviour can be changed by setting <code>X-Delegate-First: true</code>
4666      in the <filename>MANIFEST.MF</filename> file. If this property is set the
4667      parent class loader is search first. This is the same as in BASE 2.12 and
4668      earlier.
4669    </para>
4670   
4671    <note>
4672      The benefit with the new search order is that plug-ins may use
4673      a specific version of any external package even if the same
4674      package is part of the BASE distribution. This was not possible before
4675      since the package in the BASE distribution was loaded first.
4676    </note>
4677  </sect1>
4678 
4679  <sect1 id="plugin_developer.example">
4680    <title>Example plug-ins (with download)</title>
4681    <para>
4682      We have created some example plug-ins which demonstrates how to
4683      use the plug-in system and how to create an interactive plug-in that
4684      can ask a user for one or more parameters.
4685   
4686      You can download
4687      a tar file with the source and compiled plug-in code from the BASE plug-ins website:
4688      <ulink url="http://baseplugins.thep.lu.se/wiki/net.sf.basedb.examples.extensions"
4689      >http://baseplugins.thep.lu.se/wiki/net.sf.basedb.examples.extensions</ulink>
4690    </para>
4691   
4692  </sect1>
4693</chapter>
Note: See TracBrowser for help on using the repository browser.