source: trunk/doc/src/docbook/admindoc/plugin_installation.xml @ 3753

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE chapter PUBLIC 
    "-//Dawid Weiss//DTD DocBook V3.1-Based Extension for XML and graphics inclusion//EN" 
    "../../../../lib/docbook/preprocess/dweiss-docbook-extensions.dtd">
<!--
  $Id: plugin_installation.xml 3753 2007-09-18 13:47:23Z martin $

  Copyright (C) 2007 Johan Enell, Jari Hakkinen, Peter Johansson, Nicklas Nordborg, Martin Svensson

  This file is part of BASE - BioArray Software Environment.
  Available at http://base.thep.lu.se/

  BASE is free software; you can redistribute it and/or
  modify it under the terms of the GNU General Public License
  as published by the Free Software Foundation; either version 2
  of the License, or (at your option) any later version.

  BASE is distributed in the hope that it will be useful,
  but WITHOUT ANY WARRANTY; without even the implied warranty of
  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
  GNU General Public License for more details.

  You should have received a copy of the GNU General Public License
  along with this program; if not, write to the Free Software
  Foundation, Inc., 59 Temple Place - Suite 330,
  Boston, MA  02111-1307, USA.
-->

<chapter id="plugins">
  <?dbhtml dir="plugin_installation"?>
  <title>Plug-ins</title>

  <para>
    BASE can get extended functionality by the use of plug-ins. In fact, most
    of the hard work, such as data import/export and analysis is done with plug-ins.
    BASE ships with a number of standard plug-ins, the core plug-ins, which gives
    basic import/export and analysis functionality. See <xref linkend="resources" />
    for more information about the core plug-ins and 3rd-party plug-ins.
  </para>


  <sect1 id="plugins.installation">
    <title>Installing plug-ins</title>
    <para>
      The first step is to install the plug-in on the web server. To make
      these instructions easier to read we assume the plug-in comes as
      a single JAR file and that it does not depend on any other JAR files.
    </para>
    
    <para>
      We recommend that you install the plug-in JAR file outside the web 
      server's Classpath. Do not put the plug-in in the 
      <filename>&lt;base-dir&gt;/www/WEB-INF/lib</filename>
      directory or any other directory where the web server keeps it's classes.
      This makes BASE use it's own class loader that support unloading of classes
      as well. This means that you can install new plug-ins and update
      existing ones without restarting the web server.
    </para>
    
    <para>
      We recommend that you create a separate directory for plug-ins and
      install all of them there. You may use a sub-directory for each plug-in
      if you like, for example:
      <filename>&lt;base-dir&gt;/plugins/&lt;name-of-plugin&gt;</filename>
    </para>
    
    <note>
      <title>Plug-ins that use other JAR files</title>
      <para>
        Some plug-ins may depend on other JAR files. Normally, 
        these JAR files should be put in the same directory as the
        plug-in JAR file. This may, however, vary from plug-in to plug-in
        so always check the plug-in documentation first.
      </para>
    </note>
    
    <para>
      When the plug-in is installed on the server you must register it with BASE. To register
      the plug-in with BASE go to
      <menuchoice>
        <guimenu>Administrate</guimenu>
        <guisubmenu>Plugins</guisubmenu>
        <guimenuitem>Definitions</guimenuitem>
      </menuchoice>
      and click on the &gbNew; button. The pop-up window that appears looks different
      depending on if
      <property>plugin.dir</property>
      in
      <filename>base.config</filename>
      has been defined or not, and if the defined directory contains any plugins. Is the directory
      not defined or there are no plugins in the defined directory, the pop-up window will
      have the look described in
      <xref linkend="plugins.install.properties" />
      otherwise it will look like described in
      <xref linkend="plugins.select.installtype" />
      below.
    </para>
    
    <sect2 id="plugins.select.installtype">
      <title>Select installation method</title>
      <para>
        This window appears, like described above, when automatic installation/registration
        of plugins are available.
        <figure id="plugins.figures.selectinstall">
          <title>Select installation type</title>
          <screenshot>
            <mediaobject>
              <imageobject>
                <imagedata fileref="figures/select_install.png" format="PNG"
                  scalefit="1" width="70%" />
              </imageobject>
            </mediaobject>
          </screenshot>
        </figure>
      </para>
      <helptext external_id="plugindefinition.installationtype"
        title="Select installation type">
        <variablelist>
          <varlistentry>
            <term><guilabel>Install</guilabel>
            </term>
            <listitem>
              <para>
                How the plugins should be registered to BASE.
                <variablelist>
                  <varlistentry>
                    <term><guilabel>Automatically</guilabel></term>
                    <listitem>
                      <para>
                        Lets you select one or many plugins from a list with
                        new external plugins found in the directory defined
                        in
                        <filename>base.config</filename>
                        . Next step is
                        <xref linkend="plugins.install.auto" />
                      </para>
                    </listitem>
                  </varlistentry>
                  <varlistentry>
                    <term><guilabel>Manually</guilabel></term>
                    <listitem>
                      <para>
                        Class name and jar-path have to be defined by the
                        administrator. Only one plugin can be registered at
                        a time in this way. Next step is
                        <xref linkend="plugins.install.properties" />
                      </para>
                    </listitem>
                  </varlistentry>
                </variablelist>
              </para>
            </listitem>
          </varlistentry>
          <varlistentry>
            <term><guilabel>Path</guilabel>(only <guilabel>Automatically</guilabel>)</term>
            <listitem>
              <para>
                Where BASE will look for jar-files with plugins.
              </para>
            </listitem>
          </varlistentry>
        </variablelist>
      </helptext>
    </sect2>

    <sect2 id="plugins.install.properties">
      <title>Plug-in properties</title>
    
    <figure id="plugins.figures.install">
      <title>Installing a plug-in</title>
      <screenshot>
        <mediaobject>
          <imageobject><imagedata 
            fileref="figures/install_plugin.png" format="PNG"
            /></imageobject>
        </mediaobject>
      </screenshot>
    </figure>
    
    <helptext external_id="plugindefinition.edit" 
      title="Plug-in properties">
      
      <variablelist>
      <varlistentry>
        <term><guilabel>Name</guilabel></term>
        <listitem>
          <para>
            The name of the plug-in. This name is set automatically by
            the plug-in and cannot be changed.
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><guilabel>Class</guilabel></term>
        <listitem>
          <para>
            The Java class name of the plug-in.
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><guilabel>Path</guilabel></term>
        <listitem>
          <para>
            The path to the JAR file on the web server. If left empty
            the plug-in must be on the web server's class path (not 
            recommended).
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><guilabel>Max memory</guilabel></term>
        <listitem>
          <para>
            The maximum amount of memory the plug-in may use. This setting
            only applies when the plug-in is executed with a job agent. If
            the internal job queue is used this setting has no effect and the
            plug-in may use as much memory as it likes. 
            <nohelp>See <xref linkend="plugins.jobagents" /> later 
            in this chapter.</nohelp>
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><guilabel>Trusted</guilabel></term>
        <listitem>
          <para>
            If the plug-in is trusted enough to be executed in an
            unprotected environment. This setting has currently no
            effect since BASE cannot run in a protected environment.
            When this becomes implemented in the future a 
            <guilabel>no</guilabel> value will apply security restrictions
            to plug-ins similar to those a web browser put on applets.
            For example, the plug-in is not allowed to access the file
            system, open ports, shut down the server, and a lot of
            other nasty things.
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><guilabel>Allow immediate execution</guilabel></term>
        <listitem>
          <para>
            If the plug-in is allowed to bypass the job queue
            and be executed immediately. 
            
            <itemizedlist>
              <listitem>
                <para>
                <guilabel>No</guilabel>: The plug-in must always use
                the job queue.
                </para>
              </listitem>
              <listitem>
                <para>
                <guilabel>Yes</guilabel>: The plug-in is allowed to bypass
                the job queue. This also means that the plug-in always 
                executes on the web server, job agents are not used. 
                This setting is mainly useful for export plug-ins that
                needs to support immediate download of the exported data.
                <nohelp>
                See <xref linkend="import_export_data.export.immediatedownload" />.
                </nohelp>
                </para>
                
                <note>
                  If a plug-in should be executed immediately or not
                  is always decided by the plug-in. BASE will never give
                  the users a choice.
                </note>
                
              </listitem>
              <listitem>
                <para>
                <guilabel>Auto</guilabel>: BASE will allow export plug-ins
                to execute immediately, and deny all other types of plug-ins.
                This alternative is only available when registering a new 
                plug-in.
                </para>
              </listitem>
            </itemizedlist>
            
          </para>
        </listitem>
      </varlistentry>
      </variablelist>
      
      <para>
        Click on &gbSave; to finish the registration
        or on &gbCancel; to abort.
      </para>
      
      <seeother>
        <other external_id="plugindefinition.edit.permissions">Plug-in permissions</other>
        <other external_id="plugindefinition.jobagents">Job agents</other>
      </seeother>   
      
    </helptext>
    </sect2>
    
    <sect2 id="plugins.install.auto">
      <title>Register plugins automatically</title>
      <para>
      
      </para>
    </sect2>
    
    <sect2 id="plugins.installation.base1">
      <title>BASE 1 plug-ins</title>
      <para>
        BASE 1 plug-ins are supported by BASE 2 through the use of
        the <emphasis>BASE 1 plug-in executer</emphasis> plug-in. This
        is itself a plug-in and BASE 1 plug-ins are added as configurations
        to this plug-in. See <xref linkend="plugins.configuration" />. To
        install a BASE 1 plug-in to BASE 2 follow these instructions:
      </para>
      
      <orderedlist>
        <listitem>
        <para>
          Install the BASE 1 plug-in executable and any
          other files needed by it on the BASE server. Check the documentation 
          for the plug-in for information about what is needed.
        </para>
        </listitem>
        
        <listitem>
        <para>
          Upload the <filename>*.base</filename> file for the BASE 1 plug-in
          to BASE 2. If you cannot find the file, you can let BASE 1 create one for you.
          In your BASE 1 installation go to 
          <menuchoice>
            <guimenu>Analyze data</guimenu>
            <guimenuitem>Plug-ins</guimenuitem>
          </menuchoice>
          and use the <guilabel>Export</guilabel> function. This will create a
          configuration file for your BASE 1 plug-in that you can upload to BASE 2.
        </para>
        </listitem>
        
        <listitem>
          <para>
            Create a new plug-in configuration in BASE 2 using, for example,
            the <guibutton>New configuration</guibutton> button
            in single-item view for the <emphasis>BASE 1 plug-in executer</emphasis>
            plug-in.
          </para>
        </listitem>
        
        <listitem>
          <para>
            Start the configuration wizard and select the <filename>*.base</filename>
            file describing the BASE 1 plug-in and enter the path and
            file name to the location of the executable.
          </para>
        </listitem>
        
        <listitem>
          <para>
            To check that the new plug-in works correctly, you need
            to have an experiment with some data. Go to the single-item
            view for a bioassayset and click on the <guibutton>Run analysis</guibutton>
            button. Select the <emphasis>Base1PluginExecuter</emphasis>
            plug-in. The list of configurations should include the newly
            installed plug-in. Select it and click on &gbNext;.
          </para>
        </listitem>
        
        <listitem>
          <para>
            This will enter regular plug-in execution wizard and you 
            will have to enter parameters needed by the plug-in.
          </para>
        </listitem>
        
      </orderedlist>
      

    </sect2>
    
  </sect1>

  <sect1 id="plugins.permissions">
    <title>Plug-in permissions</title>

    <helptext external_id="plugindefinition.edit.permissions" 
      title="Edit plug-in permissions">

    
      <para>
        When a plug-in is executed the default is to give it the same
        permissions as the user that started it. This can be seen as a 
        security risk if the plug-in is not trusted, or if someone manages
        to replace the plug-in code with their own code. A malicious plugin can, 
        for example, delete the entire database if invoked by the root user.
      </para>
      
      <para>
        To limit this problem it is possible to tune the permissions for
        a plug-in so that it only has permission to do things that it
        is supposed to do. For example, a plug-in that import reporters
        may only need permission to update and create new reporters and
        nothing else.
      </para>
    
      <nohelp>
      <para>
        To enable the permission system for a plug-in go the
        edit view of the plug-in and select the <guilabel>Permissions</guilabel>
        tab.
      </para>
      
      <figure id="plugins.figures.permissions">
        <title>Setting permissions on a plug-in</title>
        <screenshot>
          <mediaobject>
            <imageobject><imagedata 
              fileref="figures/plugin_permissions.png" format="PNG"
              /></imageobject>
          </mediaobject>
        </screenshot>
      </figure>
      </nohelp>

      <variablelist>
      <varlistentry>
        <term><guilabel>Use permissions</guilabel></term>
        <listitem>
          <para>
            Select if the plug-in should use the permission system
            or not. If <guilabel>no</guilabel> is selected, the rest
            of the form is disabled.
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><guilabel>Item types</guilabel></term>
        <listitem>
          <para>
            The list contains all item types found in BASE that
            can have permissions set on them. The list is more or less the
            same as the permission list for roles.
            <nohelp>
              See <xref linkend="user_administration.roles.edit.permissions" />.
            </nohelp>
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><guilabel>Always grant</guilabel></term>
        <listitem>
          <para>
            The selected permissions will always be granted
            to the plug-in no matter if the logged in user had the
            permission to begin with or not. This makes it possible to 
            develop a plugin that allows users to do things that they 
            are normally not allowed to do. The reporter importer is
            for example allowed to create and use reporter types.
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><guilabel>Always deny</guilabel></term>
        <listitem>
          <para>
            The selected permissions will always be denied
            to the plug-in no matter if the logged in user had the
            permission to begin with or not. The default is to always
            deny all permissions. Permissions that are not always
            denied and not always granted uses permissions from
            the logged in user.
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><guilabel>Requested by plug-in</guilabel></term>
        <listitem>
          <para>
            To make it easier for the server administrator
            to assign permissions, the plug-in developer can
            let the plug-in include a list of permissions that
            are needed. Plug-in developers are advised to
            only include the minimal set of permissions that are
            required for the plug-in to function. Click on the 
            <guibutton>Use requested permissions</guibutton>
            button to give the plug-in the requested permissions.
          </para>
        </listitem>
      </varlistentry>
      </variablelist>
    
      <seeother>
        <other external_id="plugindefinition.edit">Plug-in properties</other>
        <other external_id="plugindefinition.jobagents">Job agents</other>
        <other external_id="role.edit.permissions">Role permissions</other>
      </seeother>   
    </helptext>
    
    
  </sect1>


  <sect1 id="plugins.jobagents">
    <title>Job agents</title>
    <para>
      Job agents are used for executing plug-ins on external
      computers. Using job agents will free up resources on the BASE
      application server, and allow the server to concentrate on
      serving web pages. Job agents are optional and must be installed
      separately. See <xref linkend="installation_upgrade.jobagents"
      /> for more information about setting up job agents. This
      section assumes that at least one job agent is setup to serve
      your BASE installation.
    </para>
    
    <para>
      A job agent will not execute a plug-in unless the administrator
      has configured the job agent to do so. This can be done either
      from the plug-in pages or from the job agent pages. To register
      a plug-in with one or more job agents from the plug-in pages, go
      to the edit view of the plug-in and select the <guilabel>Job
      agents</guilabel> tab. To do the same from the job agent pages,
      go to the edit view of the job agent and select
      the <guilabel>Plugins</guilabel> tab. The registration dialogs
      are very similar but only the plug-in side of registration is
      described here, the job agent route is described in
      <xref linkend="installation_upgrade.jobagents" />.
    </para>
    
    <figure id="plugins.figures.jobagents">
      <title>Select job agents for a plug-in</title>
      <screenshot>
        <mediaobject>
          <imageobject><imagedata 
            fileref="figures/plugin_jobagents.png" format="PNG"
            /></imageobject>
        </mediaobject>
      </screenshot>
    </figure>
    
    <helptext external_id="plugindefinition.jobagents" 
      title="Job agents">
      
      <para>
        Use this tab to specify which job agents the plug-in is
        installed and allowed to be executed on.
      </para>
      
      <variablelist>
      <varlistentry>
        <term><guilabel>Run this plugin on</guilabel></term>
        <listitem>
          <para>
            A list with the job agents where the plug-in is installed and 
            allowed to be executed. Select a plug-in in this list to display 
            more configuration options for the plug-in.
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><guilabel>Add job agents</guilabel></term>
        <listitem>
          <para>
            Use this button to open a popup window for selecting
            job agents.
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><guilabel>Remove</guilabel></term>
        <listitem>
          <para>
            Remove the selected plug-in from the list.
          </para>
        </listitem>
      </varlistentry>
      </variablelist>
      
      <para>
        The following properties are only displayed when a
        job agent has been selected in the list. Each job agent may have it's
        own settings of these properties. If you leave the values unspecified 
        the job agent will use the default values specified on the
        <guilabel>Plugin</guilabel> tab.
      </para>
      
      <variablelist>
      <varlistentry>
        <term><guilabel>Jar path</guilabel></term>
        <listitem>
          <para>
            The path on the external server to the JAR file containing the 
            plug-in code. If not specified the same path as on the web server
            is used.
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><guilabel>Max memory</guilabel></term>
        <listitem>
          <para>
            The maximum amount of memory the plug-in is allowed to use.
            Add around 40MB for the Java runtime environment and BASE. 
            If not specified Java will choose it's default value which is 64MB.
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><guilabel>Trusted</guilabel></term>
        <listitem>
          <para>
            If the plug-in should be executed in a protected or 
            unprotected environment. Currently, BASE only supports
            running plug-ins in an unprotected environment.
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><guilabel>Priority boost</guilabel></term>
        <listitem>
          <para>
            Used to give a plug-in higher priority in the queue.
            Values between 0 and 10 are allowed. A higher value will 
            give the plug-in higher priority. The priority boost is useful 
            if we, for example, want to use one server mainly for importing 
            data. By giving all import plugins a priority boost they will 
            be executed before all other jobs, which will have to wait 
            until there are no more waiting imports.
          </para>
        </listitem>
      </varlistentry>
      </variablelist>
      
      <seeother>
        <other external_id="plugindefinition.edit">Plug-in properties</other>
        <other external_id="plugindefinition.edit.permissions">Plug-in permissions</other>
      </seeother>
    </helptext>
  </sect1>
  
  <sect1 id="plugins.configuration">
    <title>Plug-in configurations</title>

    <para>
      While some plug-ins work right out of the box, some
      may require configuration before they can be used. For example,
      most of the core import plug-ins need configurations in the
      form of regular expressions to be able to find headers and
      data in the data files and the BASE 1 plug-in executer uses
      configurations to store information about the BASE 1 plug-ins.
    </para>

    <para>
      Configurations are managed from a plug-in's single-item view page
      or from the
      <menuchoice>
        <guimenu>Administrate</guimenu>
        <guisubmenu>Plugins</guisubmenu>
        <guimenuitem>Configurations</guimenuitem>
      </menuchoice>
      page.
    </para>
    
    <para>
      Click on the &gbNew; button
      to create a new configuration.
    </para>
    
    <figure id="plugins.figures.configuration">
      <title>Create plug-in configuration</title>
      <screenshot>
        <mediaobject>
          <imageobject><imagedata 
            fileref="figures/plugin_configuration.png" format="PNG"
            /></imageobject>
        </mediaobject>
      </screenshot>
    </figure>
    
    <helptext external_id="pluginconfiguration.edit" 
      title="Edit plugin configuration">
      
      <variablelist>
      <varlistentry>
        <term><guilabel>Plugin</guilabel></term>
        <listitem>
          <para>
            The plug-in this configuration belongs to. This cannot
            be changed for existing configurations. Use the
            <guibutton>Select&hellip;</guibutton> button
            to open a popup window where you can select a plug-in.
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><guilabel>Name</guilabel></term>
        <listitem>
          <para>
            The name of the configuration.
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><guilabel>Description</guilabel></term>
        <listitem>
          <para>
            A description of the configuration (optional).
          </para>
        </listitem>
      </varlistentry>
      </variablelist>
      
      <note>
        You cannot create configurations for plug-ins that
        does not support being configured.
      </note>
    
      <para>
        Use the &gbSave; button to save the
        configuration or the <guibutton>Save and configure</guibutton>
        button to save and then start the configuration wizard.
      </para>
    
    </helptext>
    
    <sect2 id="plugins.configuration.wizard">
      <title>Configuring plug-in configurations</title>

      <helptext external_id="runplugin.configure"
        title="The plug-in configuration wizard">
      
      <para>
        Configuring a plug-in is done with a wizard-like 
        interface. Since the configuration parameters may vary
        from plug-in to plug-in BASE uses a generic interface
        to enter parameter values. In short, it works like this:
        
        <orderedlist>
          <listitem>
            <para>
              BASE asks the plug-in for information about 
              the parameters the plug-in needs. For example,
              if the value is a string or number or should be
              selected among a list of predefined values.
            </para>
          </listitem>
          
          <listitem>
            <para>
              BASE uses this information to create a generic form
              for entering the values. The form consists of three
              parts:
              
              <nohelp>
              <figure id="plugins.figures.wizard">
                <title>The plug-in configuration wizard</title>
                <screenshot>
                  <mediaobject>
                    <imageobject><imagedata 
                      fileref="figures/plugin_wizard.png" format="PNG"
                      /></imageobject>
                  </mediaobject>
                </screenshot>
              </figure>
              </nohelp>
              
              <itemizedlist>
              <listitem>
                <para>
                <emphasis>The top part</emphasis>: Displays
                the name of the selected plug-in and configuration.
                </para>
              </listitem>
              
              <listitem>
                <para>
                <emphasis>The left part</emphasis>: Displays
                a list of all parameters supported by the plug-in.
                Parameters with an <guilabel>X</guilabel> in front
                of their names already have a value. Parameters
                marked with a blue rectangle are required and
                must be given a value before it is possible to
                proceed.
                </para>
              </listitem>
              
              <listitem>
                <para>
                <emphasis>The right part</emphasis>: Click on a parameter
                in the list to display a form for entering values for
                that parameter. The form may be a simple free text field,
                a list of checkboxes or radiobuttons, or something else
                depending on the kind of values supported by that
                parameter.
                </para>
              </listitem>
              </itemizedlist> 
              
            </para>
          </listitem>

          <listitem>
            <para>
              When the user clicks &gbNext;
              the entered values are sent to the plug-in which 
              validate the correctness. The plug-in may return
              three different replies:
              
              <itemizedlist>
              <listitem>
                <para>
                <emphasis>ERROR</emphasis>:
                There is an error in the input. BASE will redisplay
                the same form with any additional error information that
                the plug-in sends back.
                </para>
              </listitem>
              <listitem>
                <para>
                <emphasis>DONE</emphasis>:
                All parameter values are okay and no more values
                are needed. BASE will save the values to the database
                and finish the configuration wizard.
                </para>
              </listitem>
              <listitem>
                <para>
                <emphasis>CONTINUE</emphasis>:
                All parameter values are okay, but the plug-in
                wants more parameters. The procedure is repeated from the
                first step.
                </para>
              </listitem>
              </itemizedlist>
            </para>
          </listitem>
          
        </orderedlist>
      </para>

      <note>
        <title>Do not go back</title>
        It is not possible to go backwards in the wizard.
        If you try it will most likely result in an
        unexpected error and the configuration must be restarted
        from the beginning.
      </note>
      
      <seeother>
        <other external_id="runplugin.configure.import">Common parameter for import plug-ins</other>
        <other external_id="runplugin.configure.export">Common parameter for export plug-ins</other>
        <other external_id="runplugin.configure.analysis">Common parameter for analysis plug-ins</other>
      </seeother>
      </helptext>
      
    </sect2>

    <sect2 id="plugins.configuration.importexport">
      <title>Importing and exporting plug-in configurations</title>
      <para>
        BASE ships with one importer and one exporter that allows
        you to import and export plug-in configurations. This makes
        it easy to copy configurations between servers. The BASE website
        also has a <ulink url="http://base.thep.lu.se/chrome/site/doc/admin/plugin_configuration/coreplugins.html"
        >page where you can download additional configurations</ulink>
        not included in the main distribution.
      </para>
      
      <para>
        Both the import and the export is started from the plug-in 
        configuration list view: 
        <menuchoice>
          <guimenu>Administrate</guimenu>
          <guisubmenu>Plugins</guisubmenu>
          <guimenuitem>Configurations</guimenuitem>
        </menuchoice>
      </para>
      
      <para>
        The importer supports auto detection. Simply upload and select the XML file with the
        configurations. No more parameters are needed. 
      </para>
      
      <para>
        If you don't want to import all
        configurations that exist in the XML-file, there is an option that lets you select
        each configuration individually. When the option to import all configurations is set
        to FALSE in the first step of job-configuration, the following step after pressing
        <guibutton>Next</guibutton>
        will be to select those configurations that should be imported, otherwise this step
        is skipped.
      </para>


      <para>
        To use the exporter you must first select the configurations
        that should be exported in the list. Then, enter a path and file name
        if you wish to leave the XML file on the BASE server or leave it
        empty to download it immediately.
      </para>
      
      <note>
        The import and export only supports simple values, such as strings,
        numbers, etc. It does not support configuration values that reference
        other items. If the plug-in has such values they must be fixed
        manually after the import.
      </note>
      
    </sect2>

    <sect2 id="plugins.configuration.testwithfile">
      <title>The <guilabel>Test with file</guilabel> function</title>
      
      <para>
        The <guilabel>Test with file</guilabel> function is a very useful
        function for specifying import file formats. It is supported by
        many of the import plug-ins that read data from a simple text 
        file. This includes the raw data importer, the reporter importer,
        plate reporter, etc.
      </para>

      <note>
        The <guilabel>Test with file</guilabel> function can only
        be used with simple (tab- or comma-separated) text files.
        It does not work with XML files or binary files. The text file
        may have headers in the beginning.
      </note>
      
      <para>
        As you can see in figure <xref linkend="plugins.figures.wizard"/>
        there is a <guibutton>Test with file</guibutton> button. This
        will appear in the file format setup step for all plug-ins that
        support the test with file function. For detailed technical 
        information about this see <xref linkend="plugin_developer.import" />
        in <xref linkend="plugin_developer" />. Clicking on the 
        <guibutton>Test with file</guibutton> button opens the following dialog:
      </para>
      
      <figure id="plugins.figures.testwithfile">
        <title>The test with file function</title>
        <screenshot>
          <mediaobject>
            <imageobject><imagedata 
              scalefit="1" width="100%"
              fileref="figures/test_with_file.png" format="PNG"
              /></imageobject>
          </mediaobject>
        </screenshot>
      </figure>
      
      <helptext external_id="runplugin.testwithfile" 
        title="Test with file">
        
      <para>
        The window consists of two parts, the upper part where the file 
        to parse and the parameters used to parse it are entered, and 
        the lower part that displays information about the parsing.
      </para>
        
      <variablelist>
      <varlistentry>
        <term><guilabel>File to test</guilabel></term>
        <listitem>
          <para>
            The path and file name of the file to use for testing.
            Use the &gbBrowse; button to 
            select a file from the BASE file system or upload
            a new file. Click on the <guibutton>Parse the file</guibutton>
            button to start parsing. The lower part will update itself with
            information about the parsed file. The file must follow a few simple rules:
            
            <itemizedlist>
            <listitem>
              <para>Data must be organised into columns, with one record per line.</para>
            </listitem>
            <listitem>
              <para>
                Each data column must be separated by some special character or 
                character sequence not occurring in the data, for example a tab or a comma. 
                Data in fixed-size columns cannot be parsed.
              </para>
            </listitem>
            <listitem>
              <para>
                Data may optionally be preceded by a data header, for example,
                the names of the columns.
              </para>
            </listitem>
            <listitem>
              <para>
                The data header may optionally be preceded by file headers. 
                A file header is something that can be split into a name-value pair.
              </para>
            </listitem>
            <listitem>
              <para>
                The file may contain comments, which are ignored by the parser.
              </para>
            </listitem>
            </itemizedlist>
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><guilabel>Lines to parse</guilabel></term>
        <listitem>
          <para>
            The number of lines to parse. The default is 100.
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><guilabel>Encoding</guilabel></term>
        <listitem>
          <para>
            The character encoding of the file. The default 
            is ISO-8859-1 (same as Latin-1). This list contains all encodings 
            supported by the underlying Java runtime and can be quite long.
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><guilabel>Header regexp</guilabel></term>
        <listitem>
          <para>
            A regular expression matching a header line. A header
            is a key-value pair with information about the
            data in the file. The regular expression must contain two capturing groups, 
            the first should capture the name and the second the value of the header. 
            For example, the file contains headers like:
<screen>
"Type=GenePix Results 3"
"DateTime=2006/05/16 13:17:59"
</screen>
            To match this we can use the following regular expression: 
            <userinput>"(.*)=(.*)"</userinput>.
          </para>
          
          <para>
            Use the &gbPredefined; button to select from a
            list of common regular expressions.
          </para>
          
        </listitem>
      </varlistentry>
      
      <varlistentry>
        <term><guilabel>Data splitter regexp</guilabel></term>
        <listitem>
          <para>
            A regular expression used to split a data line into columns. For
            example, <userinput>\t</userinput> to split on tabs. Use 
            &gbPredefined; button to select from a list of 
            common regular expressions.
          </para>
        </listitem>
      </varlistentry>
      
      <varlistentry>
        <term><guilabel>Ignore regexp</guilabel></term>
        <listitem>
          <para>
            A regular expression that matches all lines that should be ignored.
            For example, <userinput>\#.*</userinput> to ignore all lines starting with 
            a #. Use 
            &gbPredefined; button to select from a list of 
            common regular expressions.
          </para>
        </listitem>
      </varlistentry>
      
      <varlistentry>
        <term><guilabel>Data header regexp</guilabel></term>
        <listitem>
          <para>
            A regular expression that matches the line containing the data header.
            Usually the data header contains the column names separated with the
            same separator as the data. For example, the file contains a header
            like:
<screen>
"Block"{tab}"Column"{tab}"Row"{tab}"Name"{tab}"ID" ...and so on
</screen>
            To match this we can use the following regular expression: 
            <userinput>"Block"\t"Column"\t"Row"\t"Name"\t"ID".*</userinput>.
          </para>
          
          <para>
            The easiest way to set this regular is expression is to leave it empty
            to start with, click on the <guibutton>Parse the file</guibutton> button.
            Then, in the <guilabel>File data</guilabel> tab, use the dropdown lists 
            in the <guilabel>Use as</guilabel> column to select the line containing the
            data header. BASE will automatically generate a regular expression matching 
            the line.
          </para>
        </listitem>
      </varlistentry>

      <varlistentry>
        <term><guilabel>Date footer regexp</guilabel></term>
        <listitem>
          <para>
            A regular expression that matches the first line of non-data after
            all data lines. In most cases you can leave this empty.
          </para>
        </listitem>
      </varlistentry>

      <varlistentry>
        <term><guilabel>Min and max data columns</guilabel></term>
        <listitem>
          <para>
            If you specify values a data line is ignored if the number of
            columns does not fall within the range. If your data file does not have
            a data header with column names you can use these settings to find the
            start of data.
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><guilabel>Remove quotes</guilabel></term>
        <listitem>
          <para>
            If enabled, the parser will remove quotes around data entries.
          </para>
        </listitem>
      </varlistentry>
  
      <varlistentry>
        <term><guilabel>File data</guilabel></term>
        <listitem>
          <para>
            Press the <guilabel>Parse the file</guilabel> button
            to start parsing the file. This tab will be updated with the data
            from the file, organised as a table. For each line the following information
            is displayed:
            <itemizedlist>
            <listitem>
              <para>
              <emphasis>Line</emphasis>: The line number in the file
              </para>
            </listitem>

            <listitem>
              <para>
              <emphasis>Columns</emphasis>: The number of columns the line could be
              split into with the data splitter regular expression.
              </para>
            </listitem>

            <listitem>
              <para>
              <emphasis>Type</emphasis>: The type of line as detected by the parser.
              It should be one of the following: <emphasis>Unknown</emphasis>,
              <emphasis>Header</emphasis>, <emphasis>Data header</emphasis>,
              <emphasis>Data</emphasis> or <emphasis>Data footer</emphasis>.
              </para>
            </listitem>
            
            <listitem>
              <para>
              <emphasis>Use as</emphasis>: Use the dropdown lists to use a
              line as either the data header or data footer. BASE will automatically
              generate a regular expression.
              </para>
            </listitem>
            
            <listitem>
              <para>
              <emphasis>File data</emphasis>: The contents of the file after
              splitting and, optionally, removal of quotes.
              </para>
            </listitem>
            </itemizedlist>
            
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><guilabel>Column mappings</guilabel></term>
        <listitem>
          <para>
            This tab is only displayed if data has been found in the file and a
            data header is present. It allows you to easily select the mapping
            between columns in the file and the properties in the database.
          </para>
          <nohelp>
          <figure id="plugins.figures.testwithfilemappings">
            <title>Mapping columns from a file</title>
            <screenshot>
              <mediaobject>
                <imageobject><imagedata 
                  fileref="figures/test_with_file_mappings.png" format="PNG"
                  /></imageobject>
              </mediaobject>
            </screenshot>
          </figure>
          </nohelp>
          
          <itemizedlist>
          <listitem>
            <para>
              <emphasis>Mapping style</emphasis>: The type
              of mapping to use when you pick a column from the 
              <emphasis>File columns</emphasis> list boxes.
            </para>
          </listitem>
          
          <listitem>
            <para>
              <emphasis>Property</emphasis>: The database property.
            </para>
          </listitem>

          <listitem>
            <para>
              <emphasis>Mapping expression</emphasis>: An expression that
              maps the data in the file columns to the property in the 
              database. There are two types of mappings, simple and expressions.
              A simple mapping is a string template with placeholders for data
              from the file. An expression mapping starts with an equal sign and
              is evaluated dynamically for each line of data. The simple
              mapping has better performance and we recommend that you use
              it unless you have to recalculate any of the numerical values.
              Here are a few mapping examples:
            </para>
            
<informalexample>
<literallayout>\Name\
\1\
[\row\, \column\]
=2 * col('radius')
</literallayout>
</informalexample>
            
            <note>
              Column numbers are 0-based. We recommend that you use 
              column names at all times if they are present in the
              file.
            </note>
          </listitem>
          
          <listitem>
            <para>
              <emphasis>File columns</emphasis>: Lists of column found in the file.
              Select a value from this list to let BASE automatically
              generate a mapping that picks the selected column.
            </para>
          </listitem>
          </itemizedlist>
        </listitem>
      </varlistentry>
      </variablelist>     
      
      </helptext>
      
      
    </sect2>
  
  </sect1>
  

</chapter>