source: trunk/doc/src/docbook/developer/base_api.xml @ 5782

Last change on this file since 5782 was 5782, checked in by Nicklas Nordborg, 10 years ago

References #1590: Documentation cleanup

Restructured documentation to generate shorter filenames.

  • Property svn:eol-style set to native
  • Property svn:keywords set to Id
File size: 137.9 KB
Line 
1<?xml version="1.0" encoding="UTF-8"?>
2<!DOCTYPE chapter PUBLIC
3    "-//Dawid Weiss//DTD DocBook V3.1-Based Extension for XML and graphics inclusion//EN"
4    "../../../../lib/docbook/preprocess/dweiss-docbook-extensions.dtd">
5<!--
6  $Id: base_api.xml 5782 2011-10-04 13:43:16Z nicklas $
7
8  Copyright (C) 2007 Peter Johansson, Nicklas Nordborg, Martin Svensson
9
10  This file is part of BASE - BioArray Software Environment.
11  Available at http://base.thep.lu.se/
12
13  BASE is free software; you can redistribute it and/or
14  modify it under the terms of the GNU General Public License
15  as published by the Free Software Foundation; either version 3
16  of the License, or (at your option) any later version.
17
18  BASE is distributed in the hope that it will be useful,
19  but WITHOUT ANY WARRANTY; without even the implied warranty of
20  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
21  GNU General Public License for more details.
22
23  You should have received a copy of the GNU General Public License
24  along with BASE. If not, see <http://www.gnu.org/licenses/>.
25-->
26
27<chapter id="base_api">
28  <?dbhtml dir="api" filename="index.html" ?>
29  <title>The BASE API</title>
30
31  <sect1 id="base_api.public">
32    <?dbhtml filename="public_api.html" ?>
33    <title>The Public API of BASE</title>
34   
35    <para>
36      Not all public classes and methods in the <filename>base-*.jar</filename>
37      files and other JAR files shipped with BASE are considered as
38      <emphasis>Public API</emphasis>. This is important knowledge
39      since we will always try to maintain backwards compatibility
40      for classes that are part of the public API. For other
41      classes, changes may be introduced at any time without
42      notice or specific documentation. In other words:
43    </para>
44   
45    <note>
46      <title>Only use the public API when developing plug-ins and extensions</title>
47      <para>
48        This will maximize the chance that your code will continue
49        to work with the next BASE release. If you use the non-public API
50        you do so at your own risk.
51      </para>
52    </note>
53   
54    <para>
55      See the <ulink url="../../api/index.html"
56        >BASE API javadoc</ulink> for information about
57      what parts of the API that contributes to the public API.
58      Methods, classes and other elements that have been tagged as
59      <code>@deprecated</code> should be considered as part of the internal API
60      and may be removed in a subsequent release without warning.
61    </para>
62   
63    <para>
64      Keeping the backwards compatibility is an aim only. It may not
65      always be possible. See <xref linkend="appendix.incompatible" /> to
66      read more about changes that have been introduced by each release
67      that may affect existing code.
68    </para>
69
70    <sect2 id="base_api.compatibility">
71      <title>What is backwards compatibility?</title>
72     
73      <para>
74        There is a great article about this subject on <ulink 
75        url="http://wiki.eclipse.org/index.php/Evolving_Java-based_APIs"
76          >http://wiki.eclipse.org/index.php/Evolving_Java-based_APIs</ulink>.
77        This is what we will try to comply with. If you do not want to
78        read the entire article, here are some of the most important points:
79      </para>
80     
81     
82      <sect3 id="core_api.compatibility.binary">
83        <title>Binary compatibility</title>
84        <para>
85        <blockquote>
86          Pre-existing Client binaries must link and run with new releases of the
87          Component without recompiling.
88        </blockquote>
89       
90        For example:
91        <itemizedlist>
92        <listitem>
93          <para>
94            We cannot change the number or types of parameters to a method
95            or constructor.
96          </para>
97        </listitem>
98        <listitem>
99          <para>
100            We cannot add or change methods to interfaces that are intended
101            to be implemented by plug-in or client code.
102          </para>
103        </listitem>
104        </itemizedlist>
105        </para>       
106      </sect3>
107     
108      <sect3 id="base_api.compatibility.contract">
109        <title>Contract compatibility</title>
110        <para>
111          <blockquote>
112          API changes must not invalidate formerly legal Client code.
113          </blockquote>
114       
115          For example:
116          <itemizedlist>
117          <listitem>
118            <para>
119              We cannot change the implementation of a method to do
120              things differently than before. For example, allow <constant>null</constant>
121              as a return value when it was not allowed before.
122            </para>
123          </listitem>
124          </itemizedlist>
125       
126          <note>
127            <para>
128            Sometimes there is a very fine line between what is considered a
129            bug and what is considered a feature. For example, if the
130            actual implementation does not do what the javadoc says,
131            do we change the code or do we change the documentation?
132            This has to be considered from case to case and depends on
133            the age of the code and if we expect plug-ins and clients to be
134            affected by it or not.
135            </para>
136          </note>
137        </para>
138      </sect3>
139     
140      <sect3 id="base_api.compatibility.source">
141        <title>Source code compatibility</title>
142        <para>
143        This is not an important matter and is not always possible to
144        achieve. In most cases, the problems are easy to fix.
145        Example:
146       
147        <itemizedlist>
148        <listitem>
149          <para>
150          Adding a class may break a plug-in or client that import
151          classes with <constant>.*</constant> if the same class name
152          exists in another package.
153          </para>
154        </listitem>
155        </itemizedlist>
156        </para>
157      </sect3>
158    </sect2>
159  </sect1>
160
161  <sect1 id="base_api.data" chunked="1">
162    <?dbhtml filename="data_api.html" ?>
163    <title>The Data Layer API</title>
164
165    <para>
166      This section gives an overview of the entire data layer API.
167      The figure below show how different modules relate to each other.
168    </para>
169 
170    <figure id="data_api.figures.overview">
171      <title>Data layer overview</title>
172      <screenshot>
173        <mediaobject>
174          <imageobject>
175            <imagedata 
176              align="center"
177              scalefit="1" width="100%"
178              fileref="figures/uml/datalayer.overview.png" format="PNG" />
179          </imageobject>
180        </mediaobject>
181      </screenshot>
182    </figure>
183
184    <sect2 id="data_api.basic">
185      <title>Basic classes and interfaces</title>
186     
187      <para>
188        This document contains information about the basic classes and interfaces in this package.
189        They are important since all data-layer classes must inherit from one of the already
190        existing abstract base classes or implement one or more of the
191        existing interfaces. They contain code that is common to all classes,
192        for example implementations of the <methodname>equals()</methodname>
193        and <methodname>hashCode()</methodname> methods or how to link with the owner of an
194        item.
195      </para>
196     
197        <figure id="data_api.figures.basic">
198          <title>Basic classes and interfaces</title>
199          <screenshot>
200            <mediaobject>
201              <imageobject>
202                <imagedata 
203                  align="center"
204                  fileref="figures/uml/datalayer.basic.png" format="PNG" />
205              </imageobject>
206            </mediaobject>
207          </screenshot>
208        </figure>
209     
210      <sect3 id="data_api.basic.classes">
211        <title>Classes</title>
212       
213        <variablelist>
214        <varlistentry>
215          <term><classname docapi="net.sf.basedb.core.data">BasicData</classname></term>
216          <listitem>
217            <para>
218            The root class. It overrides the <methodname>equals()</methodname>,
219            <methodname>hashCode()</methodname> and <methodname>toString()</methodname> methods
220            from the <classname>Object</classname> class. It also defines the
221            <varname>id</varname> and <varname>version</varname> properties.
222            All data layer classes must inherit from this class or one of it's subclasses.
223            </para>
224          </listitem>
225        </varlistentry>
226       
227        <varlistentry>
228          <term><classname docapi="net.sf.basedb.core.data">OwnedData</classname></term>
229          <listitem>
230            <para>
231            Extends the <classname>BasicData</classname> class and adds
232            an <varname>owner</varname> property. The owner is a required link to a
233            <classname docapi="net.sf.basedb.core.data">UserData</classname> object, representing the user that
234            is the owner of the item.
235            </para>
236          </listitem>
237        </varlistentry>
238
239        <varlistentry>
240          <term><classname docapi="net.sf.basedb.core.data">SharedData</classname></term>
241          <listitem>
242            <para>
243            Extends the <classname>OwnedData</classname> class and adds
244            properties (<varname>itemKey</varname> and <varname>projectKey</varname>)
245            that holds access permission information for an item.
246            Access permissions are held in <classname docapi="net.sf.basedb.core.data">ItemKeyData</classname> and/or
247            <classname docapi="net.sf.basedb.core.data">ProjectKeyData</classname> objects. These objects only exists if
248            the item has been shared.
249            </para>
250          </listitem>
251        </varlistentry>
252
253        <varlistentry>
254          <term><classname docapi="net.sf.basedb.core.data">CommonData</classname></term>
255          <listitem>
256            <para>
257            This is a convenience class for items that extends the <classname>SharedData</classname>
258            class and implements the <interfacename docapi="net.sf.basedb.core.data">NameableData</interfacename> and
259            <interfacename docapi="net.sf.basedb.core.data">RemoveableData</interfacename> interfaces. This is one of
260            the most common situations.
261            </para>
262          </listitem>
263        </varlistentry>
264
265        <varlistentry>
266          <term><classname docapi="net.sf.basedb.core.data">AnnotatedData</classname></term>
267          <listitem>
268            <para>
269            This is a convenience class for items that can be annotated.
270            Annotations are held in <classname docapi="net.sf.basedb.core.data">AnnotationSetData</classname> objects.
271            The annotation set only exists if annotations has been created for the item.
272            </para>
273          </listitem>
274        </varlistentry>
275        </variablelist>
276       
277      </sect3>
278     
279      <sect3 id="data_api.basic.interfaces">
280        <title>Interfaces</title>
281       
282        <variablelist>
283        <varlistentry>
284          <term><classname docapi="net.sf.basedb.core.data">IdentifiableData</classname></term>
285          <listitem>
286            <para>
287            All items are identifiable, which means that they have a unique <varname>id</varname>.
288            The id is unique for all items of a specific type (ie. class). The id is number
289            that is automatically generated by the database and has no other meaning
290            outside of the application. The <varname>version</varname> property is used for
291            detecting and preventing concurrent modifications to an item.
292            </para>
293          </listitem>
294        </varlistentry>
295       
296        <varlistentry>
297          <term><classname docapi="net.sf.basedb.core.data">OwnableData</classname></term>
298          <listitem>
299            <para>
300            An ownable item is an item which has an owner. The owner is represented as a
301            required link to a <classname docapi="net.sf.basedb.core.data">UserData</classname> object.
302            </para>
303          </listitem>
304        </varlistentry>       
305
306        <varlistentry>
307          <term><classname docapi="net.sf.basedb.core.data">ShareableData</classname></term>
308          <listitem>
309            <para>
310            A shareable item is an item which can be shared to other users, groups or projects.
311            Access permissions are held in <classname docapi="net.sf.basedb.core.data">ItemKeyData</classname> and/or
312            <classname docapi="net.sf.basedb.core.data">ProjectKeyData</classname> objects.
313            </para>
314          </listitem>
315        </varlistentry>
316             
317        <varlistentry>
318          <term><classname docapi="net.sf.basedb.core.data">NameableData</classname></term>
319          <listitem>
320            <para>
321            A nameable item is an item that has a name (required) and a description
322            (optional). The name doesn't have to be unique, except in a few special
323            cases (for example, the name of a file).
324            </para>
325          </listitem>
326        </varlistentry>
327       
328        <varlistentry>
329          <term><classname docapi="net.sf.basedb.core.data">RemovableData</classname></term>
330          <listitem>
331            <para>
332            A removable item is an item that can be flagged as removed. This doesn't
333            remove the information about the item from the database, but can be used by
334            client applications to hide items that the user is not interested in.
335            A trashcan function can be used to either restore or permanently
336            remove items that has the flag set.
337            </para>
338          </listitem>
339        </varlistentry>
340               
341        <varlistentry>
342          <term><classname docapi="net.sf.basedb.core.data">SystemData</classname></term>
343          <listitem>
344            <para>
345            A system item is an item which has an additional id in the form of string. A system id
346            is required when we need to make sure that we can get a specific item without
347            knowing the numeric id. Example of such items are the root user and the everyone group.
348            A system id is generally constructed like:
349            <constant>net.sf.basedb.core.User.ROOT</constant>. The system id:s are defined in the
350            core layer by each item class.
351            </para>
352          </listitem>
353        </varlistentry>
354
355        <varlistentry>
356          <term><classname docapi="net.sf.basedb.core.data">DiskConsumableData</classname></term>
357          <listitem>
358            <para>
359            This interface is used by items which occupies a lot of disk space and
360            should be part of the quota system, for example files. The required
361            <classname docapi="net.sf.basedb.core.data">DiskUsageData</classname> contains information about the size,
362            location, owner etc. of the item.
363            </para>
364          </listitem>
365        </varlistentry>
366       
367        <varlistentry>
368          <term><classname docapi="net.sf.basedb.core.data">AnnotatableData</classname></term>
369          <listitem>
370            <para>
371            This interface is used by items which can be annotated. Annotations are name/value
372            pairs that are attached as extra information to an item. All annotations are
373            contained in an <classname docapi="net.sf.basedb.core.data">AnnotationSetData</classname> object.
374            </para>
375          </listitem>
376        </varlistentry>
377       
378        <varlistentry>
379          <term><classname docapi="net.sf.basedb.core.data">ExtendableData</classname></term>
380          <listitem>
381            <para>
382            This interface is used by items which can have extra administrator-defined
383            columns. The functionality is similar to annotations. It is not as flexible,
384            since it is a global configuration, but has better performance. BASE will
385            generate extra database columns to store the data in the tables for items that
386            can be extended.
387            </para>
388          </listitem>
389        </varlistentry>
390       
391        <varlistentry>
392          <term><classname docapi="net.sf.basedb.core.data">BatchableData</classname></term>
393          <listitem>
394            <para>
395            This interface is a tagging interface which is used by items that needs batch
396            functionality in the core.
397            </para>
398          </listitem>
399        </varlistentry>
400       
401        <varlistentry>
402          <term><interfacename docapi="net.sf.basedb.core.data">RegisteredData</interfacename></term>
403          <listitem>
404            <para>
405            This interface is used by items which registered the date they were
406            created in the database. The registration date is set at creation time
407            and can't be modified later. Since this didn't exist prior to BASE 2.10,
408            null values are allowed on all pre-existing items. Note! For backwards
409            compatibility reasons with existing code in
410            <classname docapi="net.sf.basedb.core.data">BioMaterialEventData</classname>
411            the method name is <methodname>getEntryDate()</methodname>.
412            </para>
413          </listitem>
414        </varlistentry>
415       
416        <varlistentry>
417          <term><interfacename docapi="net.sf.basedb.core.data">LoggableData</interfacename></term>
418          <listitem>
419            <para>
420            This is a tagging interface that indicates that the <classname 
421            docapi="net.sf.basedb.core.log.db">DbLogManagerFactory</classname> logging
422            implementation should log changes made to items that implements it.
423            </para>
424          </listitem>
425        </varlistentry>
426
427        <varlistentry>
428          <term><interfacename docapi="net.sf.basedb.core.data">FileStoreEnabledData</interfacename></term>
429          <listitem>
430            <para>
431            This interface is implemented by all items that can have files with related data
432            attached to them. The file types that can be used for a specific item are usually
433            determined by the main type, the subtype or platform.
434            </para>
435          </listitem>
436        </varlistentry>
437
438        <varlistentry>
439          <term><interfacename docapi="net.sf.basedb.core.data">SubtypableData</interfacename></term>
440          <listitem>
441            <para>
442            This interface should be implemented by all items that can be subtyped.
443            Unless otherwise noted the subtype is always an optional link to
444            a <classname docapi="net.sf.basedb.core.data">ItemSubtypeData</classname>.
445            item. In the simplest form, the subtype is a kind of an annotation, but
446            for items that also implements the <interfacename 
447            docapi="net.sf.basedb.core.data">FileStoreEnabledData</interfacename>
448            interface, the subtype can be used to specify the file types that
449            are applicable for each item.
450            </para>
451          </listitem>
452        </varlistentry>
453        </variablelist>
454
455      </sect3>
456    </sect2>
457   
458    <sect2 id="data_api.authentication">
459      <title>User authentication and access control</title>
460     
461      <para>
462         This section gives an overview of user authentication and
463         how groups, roles and projects are used for access control
464         to items.
465      </para>
466     
467        <figure id="data_api.figures.authentication">
468          <title>User authentication and access control</title>
469          <screenshot>
470            <mediaobject>
471              <imageobject>
472                <imagedata 
473                  align="center"
474                  scalefit="1" width="100%"
475                  fileref="figures/uml/datalayer.authentication.png" format="PNG" />
476              </imageobject>
477            </mediaobject>
478          </screenshot>
479        </figure>
480     
481      <sect3 id="data_api.authentication.users">
482        <title>Users and passwords</title>     
483     
484        <para>
485          The <classname docapi="net.sf.basedb.core.data">UserData</classname> class holds information about users.
486          We keep the passwords in a separate table and use proxies to avoid loading
487          password data each time a user is loaded to minimize security risks. It is
488          only if the password needs to be changed that the <classname docapi="net.sf.basedb.core.data">PasswordData</classname>
489          object is loaded. The one-to-one mapping between user and password is controlled
490          by the password class, but a cascade attribute on the user class makes sure
491          that the password is deleted when a user is deleted.
492        </para>
493      </sect3>
494
495      <sect3 id="data_api.authentication.groups">
496        <title>Groups, roles, projects and permission template</title>     
497     
498        <para>
499          The <classname docapi="net.sf.basedb.core.data">GroupData</classname>,
500          <classname docapi="net.sf.basedb.core.data">RoleData</classname> and
501          <classname docapi="net.sf.basedb.core.data">ProjectData</classname> classes holds
502          information about groups, roles
503          and projects respectively. A user may be a member of any number of groups,
504          roles and/or projects. New users are automatically added as members of all
505          groups and roles that has the <varname>default</varname> property set.
506        </para>
507         
508        <para>
509          The membership in a project comes with an attached
510          permission values. This is the highest permission the user has in the
511          project. No matter what permission an item has been shared with the
512          user will not get higher permission. Groups may be members of other groups and
513          also in projects. A <classname docapi="net.sf.basedb.core.data">PermissionTemplateData</classname>
514          is just a holder for permissions that users can use when sharing items. The
515          template is never part of the actual permission control mechanism.
516        </para>
517       
518        <para>
519          Group membership is always accounted for, but the core only allows
520          one project at a time to be use, this is the <emphasis>active project</emphasis>.
521          When a project is active new items that are created are automatically
522          shared according to the settings for the project. There are two cases.
523          If the project has a permission template, the new item is given the same
524          permissions as the template has. If the project doesn't have a permission
525          template, the new item is shared to the active project with the permission
526          given by the <varname>autoPermission</varname> property. Note that in the
527          first case the new item may or may not be shared to the active project
528          depending on if the template is shared to the project or not.
529        </para>
530       
531        <para>
532          Note that the permission template is only used (by the core) when creating
533          new items. The permissions held by the template are copied and when the new item
534          has been saved to the database there is no longer any reference back to
535          the template that was used to create it. This means that changes to the
536          template does not affect already existing items and that the template
537          can be deleted without problems.
538        </para>
539      </sect3>
540     
541      <sect3 id="data_api.authentication.keys">
542        <title>Keys</title>     
543     
544        <para>
545          The <classname docapi="net.sf.basedb.core.data">KeyData</classname> class and it's subclasses
546          <classname docapi="net.sf.basedb.core.data">ItemKeyData</classname>, <classname docapi="net.sf.basedb.core.data">ProjectKeyData</classname> and
547          <classname docapi="net.sf.basedb.core.data">RoleKeyData</classname>, are used to store information about access
548          permissions to items. To get permission to manipulate an item a user must have
549          access to a key giving that permission. There are three types of keys:
550        </para>
551       
552        <variablelist>
553        <varlistentry>
554          <term><classname docapi="net.sf.basedb.core.data">ItemKey</classname></term>
555          <listitem>
556            <para>
557            Is used to give a user or group access to a specific item. The item
558            must be a <interfacename docapi="net.sf.basedb.core.data">ShareableData</interfacename> item.
559            The permissions are usually set by the owner of the item. Once created an
560            item key cannot be changed. This allows the core to reuse a key if the
561            permissions match exactly, ie. for a given set of users/groups/permissions
562            there can be only one item key object.
563            </para>
564          </listitem>
565        </varlistentry>
566
567        <varlistentry>
568          <term><classname docapi="net.sf.basedb.core.data">ProjectKey</classname></term>
569          <listitem>
570            <para>
571            Is used to give members of a project access to a specific item. The item
572            must be a <interfacename docapi="net.sf.basedb.core.data">ShareableData</interfacename> item. Once created a
573            project key cannot be changed. This allows the core to reuse a key if the
574            permissions match exactly, ie. for a given set of projects/permissions
575            there can be only one project key object.
576            </para>
577          </listitem>
578        </varlistentry>
579
580        <varlistentry>
581          <term><classname docapi="net.sf.basedb.core.data">RoleKey</classname></term>
582          <listitem>
583            <para>
584            Is used to give a user access to all items of a specific type, ie.
585            <constant>READ</constant> all <constant>SAMPLES</constant>. The installation
586            will make sure that there already exists a role key for each type of item, and
587            it is not possible to add new or delete existing keys. Unlike the other two types
588            this key can be modified.
589            </para>
590           
591            <para>
592            A role key is also used to assign permissions to plug-ins. If a plug-in has
593            been specified to use permissions the default is to deny everything.
594            The mapping to the role key is used to grant permissions to the plugin.
595            The <varname>granted</varname> value gives the plugin access to all items
596            of the related item type regardless of if the user that is running the plug-in has the
597            permission or not. The <varname>denied</varname> values denies access to all
598            items of the related item type even if the logged in user has the permission.
599            Permissions that are not granted nor denied are checked against the
600            logged in users regular permissions. Permissions to items that are
601            not linked are always denied.
602            </para>
603          </listitem>
604        </varlistentry>
605        </variablelist>
606       
607      </sect3>
608
609      <sect3 id="data_api.authentication.permissions">
610        <title>Permissions</title>
611       
612        <para>
613          The <varname>permission</varname> property appearing in many classes is an
614          integer values describing the permission:
615        </para>
616       
617        <informaltable>
618        <tgroup cols="2">
619          <colspec colname="value" />
620          <colspec colname="permission" />
621          <thead>
622            <row>
623              <entry>Value</entry>
624              <entry>Permission</entry>
625            </row>
626          </thead>
627          <tbody>
628            <row>
629              <entry>1</entry>
630              <entry>Read</entry>
631            </row>
632            <row>
633              <entry>3</entry>
634              <entry>Use</entry>
635            </row>
636            <row>
637              <entry>7</entry>
638              <entry>Restricted write</entry>
639            </row>
640            <row>
641              <entry>15</entry>
642              <entry>Write</entry>
643            </row>
644            <row>
645              <entry>31</entry>
646              <entry>Delete</entry>
647            </row>
648            <row>
649              <entry>47 (=32+15)</entry>
650              <entry>Set owner</entry>
651            </row>
652            <row>
653              <entry>79 (=64+15)</entry>
654              <entry>Set permissions</entry>
655            </row>
656            <row>
657              <entry>128</entry>
658              <entry>Create</entry>
659            </row>
660            <row>
661              <entry>256</entry>
662              <entry>Denied</entry>
663            </row>
664          </tbody>
665        </tgroup>
666        </informaltable>
667       
668        <para>
669          The values are constructed so that
670          <constant>READ</constant> -&gt;
671          <constant>USE</constant> -&gt;
672          <constant>RESTRICTED_WRITE</constant> -&gt;
673          <constant>WRITE</constant> -&gt;
674          <constant>DELETE</constant>
675          are chained in the sense that a higher permission always implies the lower permissions
676          also. The <constant>SET_OWNER</constant> and <constant>SET_PERMISSION</constant>
677          both implies <constant>WRITE</constant> permission. The <constant>DENIED</constant>
678          permission is only valid for role keys, and if specified it overrides all
679          other permissions.               
680        </para>
681       
682        <para>
683          When combining permission for a single item the permission codes for the different
684          paths are OR-ed together. For example a user has a role key with <constant>READ</constant>
685          permission for <constant>SAMPLES</constant>, but also an item key with <constant>USE</constant>
686          permission for a specific sample. Of course, the resulting permission for that
687          sample is <constant>USE</constant>. For other samples the resulting permission is
688          <constant>READ</constant>.
689        </para>
690       
691        <para>
692          If the user is also a member of a project which has <constant>WRITE</constant>
693          permission for the same sample, the user will have <constant>WRITE</constant>
694          permission when working with that project.
695        </para>
696       
697        <para>
698          The <constant>RESTRICTED_WRITE</constant> permission is in most cases the same
699          as the <constant>WRITE</constant> permission. So far the <constant>RESTRICTED_WRITE</constant>
700          permission is only given to users to their own <classname docapi="net.sf.basedb.core.data">UserData</classname>
701          object so they can change their address and other contact information,
702          but not quota, expiration date and other administrative information.
703        </para>
704
705      </sect3>
706    </sect2>
707
708    <sect2 id="data_api.reporters">
709      <title>Reporters</title>
710      <para>
711         This section gives an overview of reporters in BASE.
712      </para>
713     
714        <figure id="data_api.figures.reporters">
715          <title>Reporters</title>
716          <screenshot>
717            <mediaobject>
718              <imageobject>
719                <imagedata 
720                  align="center"
721                  fileref="figures/uml/datalayer.reporters.png" format="PNG" />
722              </imageobject>
723            </mediaobject>
724          </screenshot>
725        </figure>
726     
727      <sect3 id="data_api.reporters.description">
728        <title>Reporters</title>
729        <para>
730          The <classname docapi="net.sf.basedb.core.data">ReporterData</classname> class holds information about reporters.
731          The <property>externalId</property> is a required property that must be unique
732          among all reporters. The external ID is the value BASE uses to match
733          reporters when importing data from files.
734        </para>
735       
736        <para>
737          The <classname>ReporterData</classname> is an <emphasis>extendable</emphasis>
738          class, which means that the server administrator can define additional
739          columns (=annotations) in the reporters table. These are accessed with
740          the <methodname>ReporterData.getExtended()</methodname> and
741          <methodname>ReporterData.setExtended()</methodname> methods.
742          See <xref linkend="appendix.extendedproperties" /> for more information about
743          this.
744        </para>
745       
746        <para>
747          The <classname>ReporterData</classname> is also a <emphasis>batchable</emphasis>
748          class which means that there is no corresponding class in the core
749          layer. Client applications and plug-ins should work directly with
750          the <classname>ReporterData</classname> class. To help manage the reporters
751          there is the <classname docapi="net.sf.basedb.core">Reporter</classname> and <classname docapi="net.sf.basedb.core">ReporterBatcher</classname>
752          classes. The main reason for this
753          is to increase the performance and lower the memory usage by bypassing
754          internal caching in the core and Hibernate. Performance is also
755          increased by the batchers which uses more efficient SQL against the
756          database than Hibernate.
757        </para>
758       
759        <para>
760          The
761          <property>lastUpdate</property>
762          property holds the data and time the reporter information was last updated. The
763          value is managed automatically by the
764          <classname>ReporterBatcher</classname>
765          class. That goes for
766          <property>lastSource</property>
767          property too, which holds information about where the last update comes from. By
768          default this is set to the name of the logged in user, but it can be changed by
769          calling
770          <methodname>ReporterBatcher.setUpdateSource(String source)</methodname>
771          before the batcher commits the updates to the database. The source-string
772          should have the format: <synopsis>[ITEM_TYPE]:[ITEM_NAME]</synopsis> where,in
773          the file-case, ITEM_TYPE is File and ITEM_NAME is the file's name.
774        </para>
775      </sect3>
776     
777      <sect3 id="data_api.reporters.lists">
778        <title>Reporter lists</title>
779       
780        <para>
781          Reporter lists can be used to group reporters that are somehow related
782          to each other. This could for example be a list of interesting reporters
783          found in the analysis of an experiment. Each reporter in the list may
784          optionally be assigned a score. The meaning of the score value is not
785          interpreted by BASE.
786        </para>
787       
788      </sect3>
789     
790     
791    </sect2>
792
793    <sect2 id="data_api.quota">
794      <title>Quota and disk usage</title>
795      <para>
796         This section gives an overview of quota system in BASE
797         and how the disk usage is kept track of.
798      </para>
799     
800        <figure id="data_api.figures.quota">
801          <title>Quota and disk usage</title>
802          <screenshot>
803            <mediaobject>
804              <imageobject>
805                <imagedata 
806                  align="center"
807                  fileref="figures/uml/datalayer.quota.png" format="PNG" />
808              </imageobject>
809            </mediaobject>
810          </screenshot>
811        </figure>
812     
813      <sect3 id="data_api.quota.description">
814        <title>Quota</title>
815       
816        <para>
817          The <classname docapi="net.sf.basedb.core.data">QuotaData</classname> holds information about a
818          single quota registration. The same quota may be used by many different users
819          and groups. This object encapsulates allowed
820          quota values for different types of quota types and locations.
821          BASE defines several quota types (file, raw data and experiment),
822          and locations (primary, secondary and offline).
823        </para>
824       
825        <para>
826          The <property>quotaValues</property> property is a map from
827          <classname docapi="net.sf.basedb.core.data">QuotaIndex</classname> to maximum byte values.
828          This map must contain at least one entry for the total
829          quota at the primary location.
830        </para>
831       
832      </sect3>
833     
834      <sect3 id="data_api.quota.diskusage">
835        <title>Disk usage</title>
836       
837        <para>
838          A <interfacename docapi="net.sf.basedb.core.data">DiskConsumableData</interfacename> (for example a file)
839          item is automatically linked to a <classname docapi="net.sf.basedb.core.data">DiskUsageData</classname>
840          item. This holds information about the number of bytes,
841          the location and quota type the item uses. It also holds information
842          about which user and group (optional) that should be charged for the disk usage.
843          The user is always the owner of the item.
844        </para>
845
846      </sect3>
847     
848    </sect2>
849
850    <sect2 id="data_api.clients">
851      <title>Client, session and settings</title>
852      <para>
853         This section gives an overview of hardware and software in BASE.
854      </para>
855     
856        <figure id="data_api.figures.clients">
857          <title>Client, sessions and settings</title>
858          <screenshot>
859            <mediaobject>
860              <imageobject>
861                <imagedata 
862                  align="center"
863                  scalefit="1" width="100%"
864                  fileref="figures/uml/datalayer.clients.png" format="PNG" />
865              </imageobject>
866            </mediaobject>
867          </screenshot>
868        </figure>
869
870      <sect3 id="data_api.clients.description">
871        <title>Clients</title>
872        <para>
873          The <classname docapi="net.sf.basedb.core.data">ClientData</classname> class holds information
874          about a client application. The <property>externalId</property>
875          property is a unique identifier for the application. To avoid ID clashes the ID
876          should be constructed in the same way as Java packages, for example
877          <constant>net.sf.basedb.clients.web</constant> is the ID for the
878          web client application.
879        </para>
880       
881        <para>
882          A client application doesn't have to be registered with BASE
883          to be able to use it. But we recommend it since:
884        </para>
885       
886        <itemizedlist>
887        <listitem>
888          <para>
889            The permission system allows an admin to specify exactly
890            which users that may use a specific application.
891          </para>
892        </listitem>
893       
894        <listitem>
895          <para>
896          The application can't store any context-sensitive or application-specific
897          settings unless it is registered.
898          </para>
899        </listitem>
900       
901        <listitem>
902          <para>
903          The application can store context-sensitive help in the BASE
904          database.
905          </para>
906        </listitem>
907        </itemizedlist>
908      </sect3>
909     
910      <sect3 id="data_api.clients.sessions">
911        <title>Sessions</title>
912       
913        <para>
914          A session represents the time between login and logout for a single
915          user. The <classname docapi="net.sf.basedb.core.data">SessionData</classname> object is entirely
916          managed by the BASE core, and should be considered read-only
917          for client applications.
918        </para>
919           
920      </sect3>
921     
922      <sect3 id="data_api.clients.settings">
923        <title>Settings</title>
924       
925        <para>
926          There are two types of settings: context-sensitive settings and regular
927          settings. The regular settings are simple key-value pairs of strings
928          and can be used for almost anything. There are four subtypes:
929        </para>
930       
931        <itemizedlist>
932        <listitem>
933          <para>
934          Global default settings: Settings that are used by all users
935          and client applications on the BASE server. These settings
936          are read-only except for administrators. BASE has not yet defined
937          any settings of this type.
938          </para>
939        </listitem>
940       
941        <listitem>
942          <para>
943          User default settings: Settings that are valid for a single user
944          for any client application. BASE has not yet defined
945          any settings of this type.
946          </para>
947        </listitem>
948       
949        <listitem>
950          <para>
951          Client default settings: Settings that are valid for all users using
952          a specific client application. Each client application is responsible
953          for defining it's own settings. Settings are read-only except
954          for administrators.
955          </para>
956        </listitem>
957       
958        <listitem>
959          <para>
960          User client settings: Settings that are valid for a single user using
961          a specific client application. Each client application is responsible
962          for defining it's own settings.
963          </para>
964        </listitem>
965       
966        </itemizedlist>
967       
968        <para>
969          The context-sensitive settings are designed to hold information
970          about the current status of options related to the listing of items
971          of a specific type. This includes:
972        </para>
973       
974        <itemizedlist>
975        <listitem>
976          <para>
977          Current filtering options (as 1 or more <classname docapi="net.sf.basedb.core.data">PropertyFilterData</classname>
978          objects).
979          </para>
980        </listitem>
981       
982        <listitem>
983          <para>
984          Which columns and direction to use for sorting.
985          </para>
986        </listitem>
987       
988        <listitem>
989          <para>
990          The number of items to display on each page, and which page that
991          is the current page.
992          </para>
993        </listitem>
994       
995        <listitem>
996          <para>
997          Simple key-value settings related to a given context.
998          </para>
999        </listitem>
1000        </itemizedlist>
1001       
1002        <para>
1003          Context-sensitive settings are only accessible if a client
1004          application has been registered. The settings may be
1005          named to make it possible to store several presets and to
1006          quickly switch between them. In any case, BASE maintains a
1007          current default setting with an empty name. An administrator
1008          may mark a named setting as public to allow other users to
1009          use it.
1010        </para>
1011       
1012      </sect3>
1013     
1014     
1015    </sect2>
1016
1017    <sect2 id="data_api.files">
1018      <title>Files and directories</title>
1019
1020      <para>
1021        This section covers the details of the BASE file
1022        system.
1023      </para>
1024
1025        <figure id="data_api.figures.files">
1026          <title>Files and directories</title>
1027          <screenshot>
1028            <mediaobject>
1029              <imageobject>
1030                <imagedata 
1031                  align="center"
1032                  fileref="figures/uml/datalayer.files.png" format="PNG" />
1033              </imageobject>
1034            </mediaobject>
1035          </screenshot>
1036        </figure>
1037     
1038        <para>
1039          The <classname docapi="net.sf.basedb.core.data">DirectoryData</classname> class holds
1040          information about directories. Directories are organised in the
1041          ususal way as as tree structure. All directories must have
1042          a parent directory, except the system-defined root directory.
1043        </para>
1044       
1045        <para>
1046          The <classname docapi="net.sf.basedb.core.data">FileData</classname> class holds information about
1047          a file. The actual file contents is stored on disk in the directory
1048          specified by the <varname>userfiles</varname> setting in
1049          <filename>base.config</filename>. The <varname>internalName</varname>
1050          property is the name of the file on disk, but this is never exposed to
1051          client applications. The filenames and directories
1052          on the disk doesn't correspond to the the filenames and directories in
1053          BASE.
1054        </para>
1055       
1056        <para>
1057          The <varname>url</varname> property is used for file items which are stored in
1058          an external location. In this case there is no local file data on the
1059          BASE server.
1060        </para>
1061       
1062        <para>
1063          The <varname>location</varname> property can take three values:
1064        </para>
1065       
1066        <itemizedlist>
1067        <listitem>
1068          <para>
1069          0 = The file is offline, ie. there is no file on the disk
1070          </para>
1071        </listitem>
1072        <listitem>
1073          <para>
1074          1 = The file is in primary storage, ie. it is located on the disk
1075          and can be used by BASE
1076          </para>
1077        </listitem>
1078        <listitem>
1079          <para>
1080          2 = The file is in secondary storage, ie. it has been moved to some
1081          other place and can't be used by BASE immediately.
1082          </para>
1083        </listitem>
1084        <listitem>
1085          <para>
1086          3 = The file is an external file whose location is referenced by the
1087          <varname>url</varname> property. If the file is protected by passwords
1088          or certificates the file item may reference a
1089          <classname docapi="net.sf.basedb.core.data">FileServerData</classname>
1090          object. Note that an external file in most cases can be used by client
1091          applications/plug-ins as if the file was stored locally on the BASE
1092          server.
1093          </para>
1094        </listitem>
1095        </itemizedlist>
1096       
1097        <para>
1098          The <varname>action</varname> property controls how a file is
1099          moved between primary and seconday storage. It can have the following
1100          values:
1101        </para>
1102       
1103        <itemizedlist>
1104        <listitem>
1105          <para>
1106          0 = Do nothing
1107          </para>
1108        </listitem>
1109        <listitem>
1110          <para>
1111          1 = If the file is in secondary storage, move it back to the primary storage
1112          </para>
1113        </listitem>
1114        <listitem>
1115          <para>
1116          2 = If the file is in primary storage, move it to the secondary storage
1117          </para>
1118        </listitem>
1119        </itemizedlist>
1120       
1121        <para>
1122          The actual moving between primary and secondary storage is done by an
1123          external program. See
1124          <xref linkend="appendix.base.config.secondary" /> and
1125          <xref linkend="plugin_developer.other.secondary" /> for more information.
1126        </para>
1127     
1128        <para>
1129          The <varname>md5</varname> property can be used to check for file
1130          corruption when it is moved between primary and secondary storage or
1131          when a user re-uploads a file that has been offline.
1132        </para>
1133       
1134        <para>
1135          BASE can store files in a compressed format. This is handled internally
1136          and is not visible to client applications. The <varname>compressed</varname>
1137          and <varname>compressedSize</varname> properties are used to store information
1138          about this. A file may always be compressed if the users says so, but
1139          BASE can also do this automatically if the file is uploaded
1140          to a directory with the <varname>autoCompress</varname> flag set
1141          or if the file has MIME type with the <varname>autoCompress</varname>
1142          flag set.
1143        </para>
1144       
1145        <para>
1146          The <classname docapi="net.sf.basedb.core.data">FileServerData</classname> class
1147          holds information about an external file server. If the <varname>connectionManagerFactory</varname>
1148          isn't set BASE automatically selects a factory based on the URL of the file. There is
1149          built-in support for HTTP and HTTPS, but it is possible to install extensions for
1150          support for other protocols. The <varname>host</varname> property can be set
1151          to override the host part of the URL from the file. See <xref 
1152          linkend="extensions_developer.connection_manager" /> for more
1153          information about connection managers.
1154        </para>
1155       
1156        <para>
1157          The <varname>username</varname> and <varname>password</varname> properties are used if
1158          the server requires the user to be logged in. BASE has built-in support for Basic and
1159          Digest authentication. The <varname>serverCertificate</varname> can be used with HTTPS
1160          servers that uses a non-trusted certificate to tell BASE to trust the server anyway.
1161          In most cases, this is only needed if the server uses a self-signed certificate, but could, for
1162          example, also be used if a trusted site has forgot to renew an expired certificate.
1163          The server certificate should be an X.509 certificate in either binary or text format.
1164          The <varname>clientCertificate</varname> and <varname>clientPassword</varname>
1165          properties are used for servers that require that users present a valid client
1166          certificate before they are allowed access. The client certificate is usually issued
1167          by the server operator and must be in PKCS #12 format.
1168        </para>
1169       
1170        <para>
1171          The <classname docapi="net.sf.basedb.core.data">FileTypeData</classname> class holds information about
1172          file types. It is used only to make it easier for users to organise
1173          their files.
1174        </para>
1175       
1176        <para>
1177          The <classname docapi="net.sf.basedb.core.data">MimeTypeData</classname> is used to register mime types and
1178          map them to file extensions. The information is only used to lookup values
1179          when needed. Given the filename we can set the <varname>File.mimeType</varname>
1180          and <varname>File.fileType</varname> properties. The MIME type is also
1181          used to decide if a file should be stored in a compressed format or not.
1182          The extension of a MIME type must be unique. Extensions should be registered
1183          without a dot, ie <emphasis>html</emphasis>, not <emphasis>.html</emphasis>
1184        </para> 
1185     
1186    </sect2>
1187   
1188    <sect2 id="data_api.platforms">
1189      <title>Experimental platforms and item subtypes</title>
1190
1191      <para>
1192         This section gives an overview of experimental platforms
1193         and how they are used to enable data storage in files
1194         instead of in the database. In some senses item subtypes
1195         are related to platforms so they are also included here.
1196      </para>
1197     
1198      <itemizedlist>
1199        <title>See also</title>
1200        <listitem><xref linkend="core_api.data_in_files" /></listitem>
1201        <listitem><xref linkend="appendix.rawdatatypes.platforms" /></listitem>
1202        <listitem><xref linkend="extensions_developer.fileset_validator" /></listitem>
1203      </itemizedlist>
1204         
1205        <figure id="data_api.figures.platforms">
1206          <title>Experimental platforms and item subtypes</title>
1207          <screenshot>
1208            <mediaobject>
1209              <imageobject>
1210                <imagedata 
1211                  align="center"
1212                  fileref="figures/uml/datalayer.platforms.png" format="PNG" />
1213              </imageobject>
1214            </mediaobject>
1215          </screenshot>
1216        </figure>
1217
1218      <sect3 id="data_api.platforms.platforms">
1219        <title>Platforms</title>
1220       
1221        <para>
1222          The <classname docapi="net.sf.basedb.core.data">PlatformData</classname> holds information about a
1223          platform. A platform can have one or more <classname docapi="net.sf.basedb.core.data">PlatformVariant</classname>:s.
1224          Both the platform and variant are identified by an external ID that
1225          is fixed and can't be changed. <emphasis>Affymetrix</emphasis>
1226          is an example of a platform.
1227          If the <varname>fileOnly</varname> flag is set data for the platform
1228          can only be stored in files and not imported into the database. If
1229          the flag is not set data can be imported into the database.
1230          In the latter case, the <varname>rawDataType</varname> property
1231          can be used to lock the platform
1232          to a specific raw data type. If the value is <constant>null</constant>
1233          the platform can use any raw data type.
1234        </para>
1235       
1236        <para>
1237          Each platform and it's variant can be connected to one or more
1238          <classname docapi="net.sf.basedb.core.data">DataFileTypeData</classname> items. This item
1239          describes the kind of files that are used to hold data for
1240          the platform and/or variant. The file types are re-usable between
1241          different platforms and variants. Note that a file type may be attached
1242          to either only a platform or to a platform with a variant. File
1243          types attached to platforms are inherited by the variants. The variants
1244          can only define additional file types, not remove or redefine file types
1245          that has been attached to the platform.
1246        </para>
1247        <para>
1248          The file type is also identified
1249          by a fixed, non-changable external ID. The <varname>itemType</varname>
1250          property tells us what type of item the file holds data for (ie.
1251          array design or raw bioassay). It also links to a <classname 
1252          docapi="net.sf.basedb.core.data">ItemSubtype</classname>
1253          which is the generic type of data in the file. This allows us to query
1254          the database for, as an example, files with the generic type
1255          <constant>FileType.RAW_DATA</constant>. If we are in an Affymetrix
1256          experiment we will get the CEL file, for another platform we will
1257          get another file.
1258        </para>
1259        <para>
1260          The <varname>required</varname> flag in <classname docapi="net.sf.basedb.core.data">PlatformFileTypeData</classname>
1261          is used to signal that the file is a required file. This is not
1262          enforced by the core. It is intended to be used by client applications
1263          for creating a better GUI and for validation of an experiment.
1264        </para>
1265        <para>
1266          The <varname>allowMultiple</varname> flag in <classname 
1267          docapi="net.sf.basedb.core.data">PlatformFileTypeData</classname>
1268          controls if it should be possible to store more than one file of
1269          the given type in file type. Again, this is not enforced by the core,
1270          but only a recommendation to client applications. The setting is
1271          also used for validation of an experiment.
1272        </para>
1273
1274      </sect3>
1275     
1276      <sect3 id="data_api.platforms.subtypes">
1277        <title>Item subtypes</title>
1278       
1279        <para>
1280          The <classname docapi="net.sf.basedb.core.data">ItemSubtypeData</classname> 
1281          class describes a subtype for a main <varname>itemType</varname>. In the simplest
1282          form the subtype is a kind of annotation that is used mainly for creating a
1283          better user experience. If the main item type is also implementing the
1284          <interfacename docapi="net.sf.basedb.core.data">FileStoreEnabledData</interfacename> 
1285          interface, it is possible to
1286          register associations to the file types that can be used together with a given
1287          item subtype. The <varname>required</varname> and <varname>allowMultiple</varname>
1288          have are used in the same way as in the <classname>PlatformFileTypeData</classname>
1289          class.
1290        </para>
1291       
1292        <para>
1293          A subtype can be related to other subtypes. This is used to "chain" together
1294          groups of item subtypes. For example, <constant>Hybridization</constant>
1295          is a subtype for <constant>PHYSICALBIOASSAY</constant>, which is related to
1296          the <constant>Labeled extract (EXTRACT)</constant> subtype which is related to
1297          the <constant>Label (TAG)</constant> subtype. In addition, there are also
1298          several protocol and hardware subetypes mixed into this. The relationship between
1299          subtypes makes it possible for client applications to filter out unrelated stuff,
1300          and to validate experiments.
1301        </para>
1302       
1303      </sect3>
1304     
1305      <sect3 id="data_api.platforms.files">
1306        <title>FileStoreEnabled items and data files</title>
1307       
1308        <para>
1309          An item must implement the <interfacename docapi="net.sf.basedb.core.data">FileStoreEnabledData</interfacename>
1310          interface to be able to store data in files instead of in the database.
1311          The interface creates a link to a <classname docapi="net.sf.basedb.core.data">FileSetData</classname> object,
1312          which can hold several <classname docapi="net.sf.basedb.core.data">FileSetMemberData</classname> items.
1313          Each member points to specific <classname docapi="net.sf.basedb.core.data">FileData</classname> item.
1314        </para>
1315       
1316      </sect3>
1317    </sect2>
1318
1319    <sect2 id="data_api.parameters">
1320      <title>Parameters</title>
1321     
1322      <para>
1323        This section gives an overview the generic parameter
1324        system in BASE that is used to store annotation values,
1325        plugin configuration values, job parameter values, etc.
1326      </para>
1327     
1328        <figure id="data_api.figures.parameters">
1329          <title>Parameters</title>
1330          <screenshot>
1331            <mediaobject>
1332              <imageobject>
1333                <imagedata 
1334                  align="center"
1335                  fileref="figures/uml/datalayer.parameters.png" format="PNG" />
1336              </imageobject>
1337            </mediaobject>
1338          </screenshot>
1339        </figure>
1340     
1341        <para>
1342          The parameter system is a generic system that can store almost
1343          any kind of simple values (string, numbers, dates, etc.) and
1344          also links to other items. It is, for example, used to store configuration
1345          parameters to plug-ins and jobs as well as annotation values to annotatable items.
1346          The <classname docapi="net.sf.basedb.core.data">ParameterValueData</classname> 
1347          class is an abstract base class that can hold multiple values (all must be of the
1348          same type). Unless only a specific type of values should be stored, this is
1349          the class that should be used when creating references for storing parameter
1350          values. It makes it possible for a single relation to use any kind of
1351          values or for a collection reference to mix multiple types of values.
1352          A typical use case maps a <classname>Map</classname> with the
1353          parameter name as the key:
1354        </para>
1355       
1356        <programlisting language="java">
1357private Map&lt;String, ParameterValueData&lt;?&gt;&gt; configurationValues;
1358/**
1359   Link parameter name with it's values.
1360   @hibernate.map table="`PluginConfigurationValues`" lazy="true" cascade="all"
1361   @hibernate.collection-key column="`pluginconfiguration_id`"
1362   @hibernate.collection-index column="`name`" type="string" length="255"
1363   @hibernate.collection-many-to-many column="`value_id`"
1364      class="net.sf.basedb.core.data.ParameterValueData"
1365*/
1366public Map&lt;String, ParameterValueData&lt;?&gt;&gt; getConfigurationValues()
1367{
1368   return configurationValues;
1369}
1370void setConfigurationValues(Map&lt;String, ParameterValueData&lt;?&gt;&gt; configurationValues)
1371{
1372   this.configurationValues = configurationValues;
1373}
1374</programlisting>
1375       
1376      <para>
1377      Now it is possible for the collection to store all types of values:
1378      </para>
1379     
1380      <programlisting language="java">
1381Map&lt;String, ParameterValueData&lt;?&gt;&gt; config = ...
1382config.put("names", new StringParameterValueData("A", "B", "C"));
1383config.put("sizes", new IntegerParameterValueData(10, 20, 30));
1384
1385// When you later load those values again you have to cast
1386// them to the correct class.
1387List&lt;String&gt; names = (List&lt;String&gt;)config.get("names").getValues();
1388List&lt;Integer&gt; sizes = (List&lt;Integer&gt;)config.get("sizes").getValues();
1389</programlisting>
1390
1391    </sect2>
1392
1393    <sect2 id="data_api.annotations">
1394      <title>Annotations</title>
1395     
1396      <para>
1397        This section gives an overview of how the BASE annotation
1398        system works.
1399      </para>
1400     
1401        <figure id="data_api.figures.annotations">
1402          <title>Annotations</title>
1403          <screenshot>
1404            <mediaobject>
1405              <imageobject>
1406                <imagedata 
1407                  align="center"
1408                  fileref="figures/uml/datalayer.annotations.png" format="PNG" />
1409              </imageobject>
1410            </mediaobject>
1411          </screenshot>
1412        </figure>
1413     
1414      <sect3 id="data_api.annotations.description">
1415        <title>Annotations</title>
1416       
1417        <para>
1418        An item must implement the <interfacename docapi="net.sf.basedb.core.data">AnnotatableData</interfacename>
1419        interface to be able to use the annotation system. This interface gives
1420        a link to a <classname docapi="net.sf.basedb.core.data">AnnotationSetData</classname> item. This class
1421        encapsulates all annotations for the item. There are two types of
1422        annotations:
1423        </para>
1424       
1425        <itemizedlist>
1426        <listitem>
1427          <para>
1428          <emphasis>Primary annotations</emphasis> are annotations that
1429          explicitely belong to the item. An annotation set can contain
1430          only one primary annotation of each annotation type. The primary
1431          annotation are linked with the <property>annotations</property>
1432          property. This property is a map with an
1433          <classname docapi="net.sf.basedb.core.data">AnnotationTypeData</classname>  as the key.
1434          </para>
1435        </listitem>
1436       
1437        <listitem>
1438          <para>
1439          <emphasis>Inherited annotations</emphasis> are annotations
1440          that belong to a parent item, but that we want to use on
1441          another item as well. Inherited annotations are saved as
1442          references to either a single annotation or to another
1443          annotation set. Thus, it is possible for an item to inherit
1444          multiple annotations of the same annotation type.
1445          </para>
1446        </listitem>
1447        </itemizedlist>
1448       
1449        <para>
1450          The <classname docapi="net.sf.basedb.core.data">AnnotationData</classname> class is also
1451          just a placeholder. It connects the annotation set and
1452          annotation type with a <classname docapi="net.sf.basedb.core.data">ParameterValueData</classname>
1453          object. This is the object that holds the actual annotation
1454          values.
1455        </para>
1456       
1457      </sect3>
1458     
1459      <sect3 id="data_api.annotations.types">
1460        <title>Annotation types</title>
1461       
1462        <para>
1463        Instances of the <classname docapi="net.sf.basedb.core.data">AnnotationTypeData</classname> class
1464        defines the various annotations. It must have a <property>valueType</property> 
1465        property which cannot be changed. The value of this property controls
1466        which <classname docapi="net.sf.basedb.core.data">ParameterValueData</classname> subclass is used to store
1467        the annotation values, ie. <classname docapi="net.sf.basedb.core.data">IntegerParameterValueData</classname>,
1468        <classname docapi="net.sf.basedb.core.data">StringParameterValueData</classname>, etc.
1469        The <property>multiplicity</property> property holds the maximum allowed
1470        number of values for an annotation, or 0 if an unlimited number is
1471        allowed.
1472        </para>
1473       
1474        <para>
1475        The <property>itemTypes</property> collection holds the codes for
1476        the types of items the annotation type can be used on. This is
1477        checked when new annotations are created but already existing
1478        annotations are not affected if the collection is modified.
1479        </para>
1480       
1481        <para>
1482        Annotation types with the <property>protocolParameter</property> flag set
1483        are treated a bit differently. They will not show up as annotations
1484        to items with a type found in the <property>itemTypes</property> collection.
1485        Instead, a protocol parameter should be attached to a protocol. Then, when an item
1486        is using that protocol it becomes possible to add annotation values for
1487        the annotation types specified as protocol parameters. It doesn't matter
1488        if the item's type is found in the <property>itemTypes</property> 
1489        collection or not.
1490        </para>
1491       
1492        <para>
1493        The <property>options</property> collection is used to store additional
1494        options required by some of the value types, for example a max string
1495        length for string annotations or the max and min allowed value for
1496        integer annotations.
1497        </para>
1498       
1499        <para>
1500        The <property>enumeration</property> property is a boolean flag
1501        indicating if the allowed values are predefined as an enumeration.
1502        In that case those values are found in the <property>enumerationValues</property>
1503        property. The actual subclass is determined by the <property>valueType</property>
1504        property.
1505        </para>
1506       
1507        <para>
1508        Most of the other properties are hints to client applications how
1509        to render the input field for the annotation.
1510        </para>
1511       
1512      </sect3>
1513     
1514      <sect3 id="data_api.annotations.units">
1515        <title>Units</title>
1516        <para>
1517        Numerical annotation values can have units. A unit is described by
1518        a <classname docapi="net.sf.basedb.core.data">UnitData</classname> object.
1519        Each unit belongs to a <classname docapi="net.sf.basedb.core.data">QuantityData</classname> 
1520        object which defines the class of units. For example, if the quantity is
1521        <emphasis>weight</emphasis>, we can have units, <emphasis>kg</emphasis>,
1522        <emphasis>mg</emphasis>, <emphasis>µg</emphasis>, etc. The <classname>UnitData</classname>
1523        contains a factor and offset that relates all units to a common reference
1524        defined by the <classname>QuantityData</classname> class. For example,
1525        <emphasis>1 meter</emphasis> is the reference unit for distance, and we
1526        have <code>1 meter * 0.001 = 1 millimeter</code>. In this case, the factor is
1527        <emphasis>0.001</emphasis> and the offset 0. Another example is the relationship between
1528        kelvin and Celsius, which is <code>1 kelvin + 273.15 = 1 °Celsius</code>.
1529        Here, the factor is 1 and the offset is <emphasis>+273.15</emphasis>.
1530        The <classname
1531        docapi="net.sf.basedb.core.data">UnitSymbolData</classname>
1532        is used to make it possible to assign alternative symbols to a single unit.
1533        This is needed to simplify input where it may be hard to know what to
1534        type to get <emphasis></emphasis> or <emphasis>°C</emphasis>. Instead,
1535        <emphasis>m2</emphasis> and <emphasis>C</emphasis> can be used as
1536        alternative symbols.
1537        </para>
1538       
1539        <para>
1540        The creator of an annotation type may select a
1541        <classname>QuantityData</classname>, which can't be changed later, and
1542        a default <classname>UnitData</classname>. When entering annotation values
1543        a user may select any unit for the selected quantity (unless annotation type
1544        owner has limited this by selecting <varname>usableUnits</varname>). Before
1545        the values are stored in the database, they are converted to the default
1546        unit. This makes it possible to compare and filter on annotation values
1547        using different units. For example, filtering with <emphasis>&gt;5mg</emphasis> 
1548        also finds items that are annotated with <emphasis>2g</emphasis>.
1549        </para>
1550       
1551        <para>
1552        The core should automatically update the stored annotation values if
1553        the default unit is changed for an annotation type, or if the reference
1554        factor for a unit is changed.
1555        </para>
1556      </sect3>
1557     
1558      <sect3 id="data_api.annotations.categories">
1559        <title>Categories</title>
1560       
1561        <para>
1562        The <classname docapi="net.sf.basedb.core.data">AnnotationTypeCategoryData</classname> class defines
1563        categories that are used to group annotation types that are related to
1564        each other. This information is mainly useful for client applications
1565        when displaying forms for annotating items, that wish to provide a
1566        clearer interface when there are many (say 50+) annotations type for
1567        an item. An annotation type can belong to more than one category.
1568        </para>
1569       
1570      </sect3>
1571     
1572    </sect2>
1573
1574    <sect2 id="data_api.protocols">
1575      <title>Protocols, hardware and software</title>
1576
1577      <para>
1578        This section gives an overview of how protocols that describe various
1579        processes, such as sampling, extraction and scanning, are used in BASE.
1580      </para>
1581     
1582        <figure id="data_api.figures.protocols">
1583          <title>Protocols, hardware and software</title>
1584          <screenshot>
1585            <mediaobject>
1586              <imageobject>
1587                <imagedata 
1588                  align="center"
1589                  fileref="figures/uml/datalayer.protocols.png" format="PNG" />
1590              </imageobject>
1591            </mediaobject>
1592          </screenshot>
1593        </figure>
1594     
1595      <sect3 id="data_api.protocols.description">
1596        <title>Protocols</title>
1597       
1598        <para>
1599        A protocol is something that defines a procedure or recipe for some
1600        kind of action, such as sampling, extraction and scanning. The subtype
1601        of the protocol is used to determine what the protocol is used for.
1602        In BASE we only store a short name and description. It is possible to
1603        attach a file that provides a longer description of the procedure.
1604        </para>
1605     
1606      </sect3>
1607     
1608      <sect3 id="data_api.protocols.parameters">
1609        <title>Parameters</title>
1610       
1611        <para>
1612        The procedure described by the protocol may have parameters
1613        that are set indepentently each time the protocol is used. It
1614        could for example be a temperature, a time or something else.
1615        The definition of parameters is done by creating annotation
1616        types and attaching them to the protocol. It is only possible
1617        to attach annotation types which has the <property>protocolParameter</property>
1618        property set to <constant>true</constant>. The same annotation type
1619        can be used for more than one protocol, but only do this if the
1620        parameters actually has the same meaning.
1621        </para>
1622     
1623      </sect3>
1624
1625      <sect3 id="data_api.wares.description">
1626        <title>Hardware and software</title>
1627        <para>
1628          BASE is pre-installed with a set of subtypes for hardware and software.
1629          They are typically used to filter the registered hardware and software
1630          depending on what a user is doing. For example, when adding raw data
1631          to BASE a user can select a scanner. The GUI will display the hardware
1632          that has been registered as <emphasis>scanner</emphasis> subtype.
1633          Other subtypes are <emphasis>hybridization station</emphasis>
1634          and <emphasis>print robot</emphasis>. An administrator may register more
1635          subtypes.
1636        </para>
1637      </sect3>
1638     
1639    </sect2>
1640
1641    <sect2 id="data_api.plugins">
1642      <title>Plug-ins, jobs and job agents</title>
1643     
1644      <para>
1645         This section gives an overview of plug-ins, jobs and job agents.
1646      </para>
1647     
1648      <itemizedlist>
1649        <title>See also</title>
1650        <listitem><xref linkend="plugins.installation" /></listitem>
1651        <listitem><xref linkend="installation.jobagents" /></listitem>
1652      </itemizedlist>
1653     
1654        <figure id="data_api.figures.plugins">
1655          <title>Plug-ins, jobs and job agents</title>
1656          <screenshot>
1657            <mediaobject>
1658              <imageobject>
1659                <imagedata 
1660                  align="center"
1661                  scalefit="1" width="100%"
1662                  fileref="figures/uml/datalayer.plugins.png" format="PNG" />
1663              </imageobject>
1664            </mediaobject>
1665          </screenshot>
1666        </figure>
1667
1668      <sect3 id="data_api.plugins.plugins">
1669        <title>Plug-ins</title>
1670       
1671        <para>
1672          The <classname docapi="net.sf.basedb.core.data">PluginDefinitionData</classname> holds information of the
1673          installed plugin classes. Much of the information is copied from the
1674          plug-in itself from the <classname docapi="net.sf.basedb.core.plugin">About</classname> object and by checking
1675          which interfaces it implements.
1676        </para>
1677       
1678        <para>
1679          There are five main types of plug-ins:
1680        </para>
1681       
1682        <itemizedlist>
1683        <listitem>
1684          <para>
1685          IMPORT (mainType = 1): A plug-in that imports data to BASE.
1686          </para>
1687        </listitem>
1688        <listitem>
1689          <para>
1690          EXPORT (mainType = 2): A plug-in that exports data from BASE.
1691          </para>
1692        </listitem>
1693        <listitem>
1694          <para>
1695          INTENSITY (mainType = 3): A plug-in that calculates intensity values
1696          from raw data.
1697          </para>
1698        </listitem>
1699        <listitem>
1700          <para>
1701          ANALYZE (mainType = 4): A plug-in that analyses data.
1702          </para>
1703        </listitem>
1704        <listitem>
1705          <para>
1706          OTHER (mainType = 5): Any other plug-in.
1707          </para>
1708        </listitem>
1709        </itemizedlist>
1710       
1711        <para>
1712          A plug-in may have different configurations. The flags <property>supportsConfigurations</property>
1713          and <property>requiresConfiguration</property> are used to specify if a plug-in
1714          must have or can't have any configurations. Configuration parameter values are
1715          versioned. Each time anyone updates a configuration the version number
1716          is increased and the parameter values are stored as a new entity.
1717          This is required because we want to be able to know exactly which
1718          parameters a job were using when it was executed. When a job is
1719          created we also store the parameter version number
1720          (<property>JobData.parameterVersion</property>). This means that even if
1721          someone changes the configuration later we will always know which
1722          parameters the job used.
1723        </para>
1724       
1725        <para>
1726          The <classname docapi="net.sf.basedb.core.data">PluginTypeData</classname> class is ued to group
1727          plug-ins that share some common functionality, by implementing
1728          additional (optional) interfaces. For example, the
1729          <interfacename docapi="net.sf.basedb.core.plugin">AutoDetectingImporter</interfacename> should be implemented
1730          by import plug-ins that supports automatic detection of file formats.
1731          Another example is the <interfacename docapi="net.sf.basedb.core.plugin">AnalysisFilterPlugin</interfacename>
1732          interface which should be implemented by all analysis plug-ins that
1733          only filters data.
1734        </para>
1735
1736      </sect3>
1737     
1738      <sect3 id="data_api.plugins.jobs">
1739        <title>Jobs</title>
1740       
1741        <para>
1742          A job represents a single invokation of a plug-in to do some work.
1743          The <classname docapi="net.sf.basedb.core.data">JobData</classname> class holds information about this.
1744          A job is usuallu executed by a plug-in, but doesn't have to be. The
1745          <property>status</property> property holds the current state of a job.
1746        </para>
1747       
1748        <itemizedlist>
1749        <listitem>
1750          <para>
1751            UNCONFIGURED (status = 0): The job is not yet ready to be executed.
1752          </para>
1753        </listitem>
1754        <listitem>
1755          <para>
1756            WAITING (status = 1): The job is waiting to be executed.
1757          </para>
1758        </listitem>
1759        <listitem>
1760          <para>
1761            PREPARING (status = 5): The job is about to be executed but hasn't started yet.
1762          </para>
1763        </listitem>
1764        <listitem>
1765          <para>
1766            EXECUTING (status = 2): The job is currently executing.
1767          </para>
1768        </listitem>
1769        <listitem>
1770          <para>
1771            ABORTING (status = 6): The job is executing but an ABORT signal has been sent
1772            requesting it to abort and finish.
1773          </para>
1774        </listitem>
1775        <listitem>
1776          <para>
1777            DONE (status = 3): The job finished successfully.
1778          </para>
1779        </listitem>
1780        <listitem>
1781          <para>
1782            ERROR (status = 4): The job finished with an error.
1783          </para>
1784        </listitem>
1785        </itemizedlist>
1786      </sect3>
1787
1788      <sect3 id="data_api.plugins.agents">
1789        <title>Job agents</title>
1790       
1791        <para>
1792          A job agent is a program running on the same or a different server that
1793          is regularly checking for jobs that are waiting to be executed. The
1794          <classname docapi="net.sf.basedb.core.data">JobAgentData</classname> holds information about a job agent
1795          and the <classname docapi="net.sf.basedb.core.data">JobAgentSettingsData</classname> links the agent
1796          with the plug-ins the agent is able to execute. The job agent will only
1797          execute jobs that are owned by users or projects that the job agent has
1798          been shared to with at least use permission. The <property>priorityBoost</property>
1799          property can be used to give specific plug-ins higher priority.
1800          Thus, for a job agent it is possible to:
1801        </para>
1802       
1803        <itemizedlist>
1804        <listitem>
1805          <para>
1806          Specify exactly which plug-ins it will execute. For example, it is possible
1807          to dedicate one agent to only run one plug-in.
1808          </para>
1809        </listitem>
1810        <listitem>
1811          <para>
1812          Give some plug-ins higher priority. For example a job agent that is mainly
1813          used for importing data should give higher priority to all import plug-ins.
1814          Other types of jobs will have to wait until there are no more data to be
1815          imported.
1816          </para>
1817        </listitem>
1818        <listitem>
1819          <para>
1820          Specify exactly which users/groups/projects that may use the agent. For
1821          example, it is possible to dedicate one agent to only run jobs for a certain
1822          project.
1823          </para>
1824        </listitem>
1825        </itemizedlist>
1826       
1827      </sect3>
1828
1829
1830    </sect2>
1831   
1832    <sect2 id="data_api.biomaterials">
1833      <title>Biomaterial LIMS</title>
1834     
1835        <figure id="data_api.figures.biomaterials">
1836          <title>Biomaterial LIMS</title>
1837          <screenshot>
1838            <mediaobject>
1839              <imageobject>
1840                <imagedata 
1841                  align="center"
1842                  fileref="figures/uml/datalayer.biomaterials.png" format="PNG" />
1843              </imageobject>
1844            </mediaobject>
1845          </screenshot>
1846        </figure>
1847     
1848      <sect3 id="data_api.biomaterials.description">
1849        <title>Biomaterials</title>
1850       
1851        <para>
1852          There are three main types of biomaterials: <classname docapi="net.sf.basedb.core.data">BioSourceData</classname>,
1853          <classname docapi="net.sf.basedb.core.data">SampleData</classname> and
1854          <classname docapi="net.sf.basedb.core.data">ExtractData</classname>.
1855          All types of are derived from the base class <classname docapi="net.sf.basedb.core.data">BioMaterialData</classname>.
1856          The reason for this is that they all share common functionality such as pooling
1857          and events. By using a common base class we do not have to create duplicate
1858          classes for keeping track of events and parents.
1859        </para>
1860       
1861        <para>
1862          The <classname docapi="net.sf.basedb.core.data">BioSourceData</classname> is the simplest of the biomaterials.
1863          It cannot have parents and can't participate in events. It's only used as a
1864          (non-required) parent for samples.
1865        </para>
1866       
1867        <para>
1868          The <classname docapi="net.sf.basedb.core.data">MeasuredBioMaterialData</classname> class is used as a base
1869          class for the other biomaterial types. It introduces quantity
1870          measurements and can store original and remaining quantities. They are
1871          both optional. If an original quantity has been specified the core
1872          automatically calculates the remaining quantity based on the events a
1873          biomaterial participates in.
1874        </para>
1875       
1876        <para>
1877          All measured biomaterial have at least one event associated with them,
1878          the <emphasis>creation event</emphasis>, which holds information about the creation of the
1879          biomaterial. A measured biomaterial can be created in three ways:
1880        </para>
1881       
1882        <itemizedlist>
1883        <listitem>
1884          <para>
1885          From a single item of the same type or the parent type. Biosource is the parent type of
1886          samples and sample is the parent type of extracts. The <property>parentType</property> 
1887          property must be set to the correct parent type and the <property>parent</property> property
1888          is set to point to the parent item. The parent information
1889          is also always duplicated in the <property>sources</property> collection of the <classname docapi="net.sf.basedb.core.data">BioMaterialEventData</classname>
1890          object representing the creation event. It is the responsibility of the
1891          core to make sure that everything is properly synchronized and that
1892          remaining quantities are calculated.
1893          </para>
1894        </listitem>
1895       
1896        <listitem>
1897          <para>
1898          From multiple items of the same type, i.e pooling.
1899          In this case the <property>parentType</property> property is set, but
1900          the <property>parent</property> property is null. All source
1901          biomaterials are contained in the <property>sources</property> collection.
1902          The core is still responsible for keeping everything synchronized and to
1903          update remaining quantities.
1904          </para>
1905        </listitem>
1906       
1907        <listitem>
1908          <para>
1909          As a standalone biomaterial without parents. The <property>parentType</property>
1910          property should be null, as should the <property>parent</property> property
1911          and the <property>sources</property> collection.
1912          </para>
1913        </listitem>
1914        </itemizedlist>
1915
1916      </sect3>
1917     
1918      <sect3 id="data_api.biomaterials.plates">
1919        <title>Bioplates and plate types</title>
1920       
1921        <para>
1922          Biomaterial (except biosource) may optionally be placed on <classname 
1923          docapi="net.sf.basedb.core.data">BioPlateData</classname>:s. A bioplate is something
1924          that collects multiple biomaterial as a unit. A bioplate typically has a
1925          <classname docapi="net.sf.basedb.core.data">PlateGeometryData</classname> that
1926          determines the number of locations on the plate (<classname docapi="net.sf.basedb.core.data">BioWellData</classname>).
1927          A single well can hold a single biomaterial at a time.
1928        </para>
1929       
1930        <para>
1931          The bioplate must be of a specific <classname docapi="net.sf.basedb.core.data">BioPlateTypeData</classname>.
1932          The type can be used to put limitations on how the plate can be used. For example,
1933          it can be limited to a single type of biomaterial. It is also possible to lock wells
1934          so that the biomaterial in them can't be changed. Supported lock modes are:
1935        </para>
1936       
1937        <itemizedlist>
1938        <listitem>
1939          <para>
1940          <emphasis>Unlocked</emphasis>: Wells are unlocked and the biomaterial may be changed
1941          any number of times.
1942          </para>
1943        </listitem>
1944        <listitem>
1945          <para>
1946          <emphasis>Locked-after-move</emphasis>: The well is locked after it has been used one
1947          time and the biomaterial that was put in it has been moved to another plate.
1948          </para>
1949        </listitem>
1950        <listitem>
1951          <para>
1952          <emphasis>Locked-after-add</emphasis>: The well is locked after biomaterial has been
1953          put into it. It is not possible to remove the biomaterial.
1954          </para>
1955        </listitem>
1956        <listitem>
1957          <para>
1958          <emphasis>Locked-after-create</emphasis>: The well is locked once it has been created.
1959          Biomaterial must be put into wells before the plate is saved to the database.
1960          </para>
1961        </listitem>
1962        </itemizedlist>
1963       
1964      </sect3>
1965 
1966      <sect3 id="data_api.biomaterials.events">
1967        <title>Biomaterial and plate events</title>
1968       
1969        <para>
1970          An event represents something that happened to one or more biomaterials, for example
1971          the creation of another biomaterial. The <classname docapi="net.sf.basedb.core.data">BioMaterialEventData</classname>
1972          holds information about entry and event dates, protocols used, the user who is
1973          responsible, etc. There are three types of events represented by the <property>eventType</property>
1974          property.
1975        </para>
1976       
1977        <orderedlist>
1978        <listitem>
1979          <para>
1980          <emphasis>Creation event</emphasis>: This event represents the creation of a (measured)
1981          biomaterial. The <property>sources</property> collection contains
1982          information about the biomaterials that were used to create the new
1983          biomaterial. All sources must be of the same type. There can only be one
1984          source of the parent type. These rules are maintained by the core.
1985          </para>
1986        </listitem>
1987       
1988        <listitem>
1989          <para>
1990          <emphasis>Bioassay event</emphasis>: This event represents the creation
1991          of a bioassay. This event type is needed because we want to keep track
1992          of quantities for extracts. This event has a <classname docapi="net.sf.basedb.core.data">PhysicalBioAssayData</classname> 
1993          as a product instead of a biomaterial. The sources collection can only contain
1994          extracts. If the bioassay can hold extracts in multiple positions the
1995          <property>position</property> property in <classname docapi="net.sf.basedb.core.data">BioMaterialEventSourceData</classname> 
1996          can be used to track which extract that was put in each position. It is allowed
1997          to put multiple extracts in the same position, but then the usually need
1998          to use different <classname docapi="net.sf.basedb.core.data">TagData</classname> 
1999          items. However, this is not enforced by the core.
2000          </para>
2001        </listitem>
2002
2003        <listitem>
2004          <para>
2005          <emphasis>Other event</emphasis>: This event represents some other important
2006          information about a single biomaterial that affected the remaining quantity.
2007          This event type doesn't have any sources.
2008          </para>
2009        </listitem>
2010        </orderedlist>
2011       
2012        <para>
2013          It is also possible to register events that applies to one or more
2014          bioplates using the <classname docapi="net.sf.basedb.core.data">BioPlateEventData</classname>
2015          class. The <classname docapi="net.sf.basedb.core.data">BioPlateEventParticipantData</classname>
2016          class holds information about each plate that is part of the event. The <property>role</property>
2017          property is a textual description of what happened to the plate. Eg. a move event, may have one
2018          <emphasis>source</emphasis> plate and one <emphasis>destination</emphasis> plate. It is
2019          recommended (but not required) that all biomaterial that are affected by the plate event
2020          are linked via a <code>BioMaterialEventData</code> to a <code>BioPlateEventParticipantData</code>.
2021          This will make it easier to keep track of the history of individual biomaterial items.
2022          Biomaterial events that are linked in this way are also automatically updated if the
2023          bioplate event is modified (eg. selecting a protocol, event date, etc.).
2024        </para>
2025       
2026      </sect3> 
2027    </sect2>
2028
2029    <sect2 id="data_api.plates">
2030      <title>Array LIMS - plates</title>
2031
2032        <figure id="data_api.figures.plates">
2033          <title>Array LIMS - plates</title>
2034          <screenshot>
2035            <mediaobject>
2036              <imageobject>
2037                <imagedata 
2038                  align="center"
2039                  scalefit="1" width="100%"
2040                  fileref="figures/uml/datalayer.plates.png" format="PNG" />
2041              </imageobject>
2042            </mediaobject>
2043          </screenshot>
2044        </figure>
2045
2046      <sect3 id="data_api.plates.description">
2047        <title>Plates</title>
2048       
2049        <para>
2050          The <classname docapi="net.sf.basedb.core.data">PlateData</classname> is the main class holding information
2051          about a single plate. The associated <classname docapi="net.sf.basedb.core.data">PlateGeometryData</classname>
2052          defines how many rows and columns there are on a plate. Since this
2053          information is used to create wells, and for various other checks it is
2054          not possible to change the number of rows or columns once a geometry has
2055          been created.
2056        </para>
2057         
2058        <para>
2059          All plates must have a <classname docapi="net.sf.basedb.core.data">PlateTypeData</classname> which defines
2060          the geometry and a set of event types (see below).
2061        </para>
2062       
2063        <para>
2064          If the destroyed flag of a plate is set it is not allowed to use the
2065          plate for a plate mapping or to create array designs. However, it
2066          is possible to change the flag to not destroyed.
2067        </para>
2068
2069        <para>
2070          The barcode is intended to be used as an external identifier of the plate.
2071          But, the core doesn't care about the value or if it is unique or not.
2072        </para>
2073      </sect3>
2074     
2075      <sect3 id="data_api.plates.events">
2076        <title>Plate events</title>
2077
2078        <para>
2079          The plate type defines a set of <classname docapi="net.sf.basedb.core.data">PlateEventTypeData</classname>
2080          objects, each one represening a particular event a plate of this type
2081          usually goes trough. For a plate of a certain type, it is possible to
2082          attach exactly one event of each event type. The event type defines an
2083          optional protocol type, which can be used by client applications to
2084          filter a list of protocols for the event. The core doesn't check that
2085          the selected protocol for an event is of the same protocol type as
2086          defined by the event type.
2087        </para>
2088
2089        <para>
2090          The ordinal value can be used as a hint to client applications in
2091          which order the events actually are performed in the lab. The core doesn't
2092          care about this value or if several event types have the same value.
2093        </para>
2094      </sect3>
2095
2096      <sect3 id="data_api.plates.mappings">
2097        <title>Plate mappings</title>
2098       
2099        <para>
2100          A plate can be created either from scratch, with the help of the information
2101          in a <classname docapi="net.sf.basedb.core.data">PlateMappingData</classname>, from a set of parent plates.
2102          In the first case it is possible to specify a reporter for each well on the
2103          plate. In the second case the mapping code creates all the wells and links
2104          them to the parent wells on the parent plates. Once the plate has been saved
2105          to the database, the wells cannot be modified (because they are used
2106          downstream for various validation, etc.)
2107        </para>
2108       
2109        <para>
2110          The details in a plate mapping are simply coordinates that for each
2111          destination plate, row and column define a source plate, row and column.
2112          It is possible for a single source well to be mapped to multiple destination
2113          wells, but for each destination well only a single source well can be
2114          used.
2115        </para>
2116       
2117      </sect3>
2118
2119    </sect2>
2120
2121    <sect2 id="data_api.arrays">
2122      <title>Array LIMS - arrays</title>
2123     
2124        <figure id="data_api.figures.arrays">
2125          <title>Array LIMS - arrays</title>
2126          <screenshot>
2127            <mediaobject>
2128              <imageobject>
2129                <imagedata 
2130                  align="center"
2131                  fileref="figures/uml/datalayer.arrays.png" format="PNG" />
2132              </imageobject>
2133            </mediaobject>
2134          </screenshot>
2135        </figure>
2136     
2137      <sect3 id="data_api.arrays.designs">
2138        <title>Array designs</title>
2139       
2140        <para>
2141          Array designs are stored in <classname docapi="net.sf.basedb.core.data">ArrayDesignData</classname> objects
2142          and can be created either as standalone designs or
2143          from plates. In the first case the features on an array design
2144          are described by a reporter map. A reporter map is a file
2145          that maps a coordinate (block, meta-grid, row, column),
2146          position or an external ID on an array design to a
2147          reporter. Which method to use is given by the
2148          <property>ArrayDesign.featureIdentificationMethod</property> property.
2149          The coordinate system on an array design is divided into blocks.
2150          Each block can be identified either by a <property>blockNumber</property>
2151          or by meta coordinates. This information is stored in
2152          <classname docapi="net.sf.basedb.core.data">ArrayDesignBlockData</classname> items. Each block
2153          contains several <classname docapi="net.sf.basedb.core.data">FeatureData</classname> items, each
2154          one identified by a row and column coordinate. Platforms that doesn't
2155          divide the array design into blocks or doesn't use the coordinate system at all
2156          must still create a single super-block that holds all features.
2157        </para>
2158       
2159        <para>
2160          Array designs that are created from plates use a print map file
2161          instead of a reporter map. A print map is similar to a plate mapping
2162          but maps features (instead of wells) to wells. The file should
2163          specifify which plate and well a feature is created from. Reporter
2164          information will automatically be copied by BASE from the well.
2165        </para>
2166       
2167        <para>
2168          It is also possible to skip the importing of features into the
2169          database and just keep the data in the orginal files instead.
2170          This is typically done for Affymetrix CDF files.
2171        </para>
2172       
2173      </sect3>
2174     
2175      <sect3 id="data_api.arrays.slides">
2176        <title>Array slides</title>
2177       
2178        <para>
2179          The <classname docapi="net.sf.basedb.core.data">ArraySlideData</classname> represents a single
2180          array. Arrays are usually printed several hundreds in a batch,
2181          represented by a <classname docapi="net.sf.basedb.core.data">ArrayBatchData</classname> item.
2182          The <property>batchIndex</property> is the ordinal number of the
2183          array in the batch. The <property>barcode</property> can be used
2184          as a means for external programs to identify the array. BASE doesn't
2185          care if a value is given or if they are unique or not. If the
2186          <property>destroyed</property> flag is set it prevents a slide from
2187          beeing used by a hybridization.
2188        </para>
2189
2190      </sect3>
2191    </sect2>
2192
2193    <sect2 id="data_api.bioassays">
2194      <title>Bioassays and raw data</title>
2195     
2196        <figure id="data_api.figures.rawdata">
2197          <title>Bioassays and raw data</title>
2198          <screenshot>
2199            <mediaobject>
2200              <imageobject>
2201                <imagedata 
2202                  align="center"
2203                  scalefit="1" width="100%"
2204                  fileref="figures/uml/datalayer.bioassays.png" format="PNG" />
2205              </imageobject>
2206            </mediaobject>
2207          </screenshot>
2208        </figure>
2209     
2210      <sect3 id="data_api.bioassays.physical">
2211        <title>Physical bioassays</title>
2212       
2213        <para>
2214        A <classname docapi="net.sf.basedb.core.data">PhysicalBioAssayData</classname>
2215        item connect the array slides from the Array LIMS part
2216        with extracts from the biomaterials part. The <property>creationEvent</property>
2217        is used to register which extracts that were used on the bioassay.
2218        The relation to slides is a one-to-one relation. A slide can only be used on
2219        a single physical bioassay and a bioassay can only use a single slide. The relation
2220        is optional from both sides.
2221        </para>
2222
2223        <para>
2224        Further processing of the bioassay is registered as a series
2225        of <classname docapi="net.sf.basedb.core.data">DerivedBioAssayData</classname>
2226        items. For microarray experiments the first step is typically a scanning
2227        of the hybridization. Information about the software/hardware and protocol
2228        used can be registered. Any data files generated by the process can be
2229        registered with the <classname docapi="net.sf.basedb.core.data">FileSetData</classname>
2230        item. If more than one processsing step is required child derived
2231        bioassays can be created that descrive each additional step. 
2232        </para>
2233       
2234        <para>
2235        If the root physical bioassay has multiple extracts in multiple positions, the
2236        <property>extract</property> property of a derived bioassay is used to link
2237        with the extract that the specific derived bioassay represents. If the
2238        link is null the derived bioassay represents all extracts on the
2239        physical bioassay.
2240        </para>
2241       
2242      </sect3>
2243     
2244      <sect3 id="data_api.bioassays.rawdata">
2245        <title>Raw data</title>
2246       
2247        <para>
2248        A <classname docapi="net.sf.basedb.core.data">RawBioAssayData</classname> object
2249        represents the raw data that is produced by analysing the data from the physical
2250        bioassay. You may register which software that was used, the
2251        protocol and any parameters (through the annotation system).
2252        </para>
2253
2254        <para>
2255        Files with the analysed data values can be attached to the
2256        associated <classname docapi="net.sf.basedb.core.data">FileSetData</classname> object.
2257        The platform and, optionally, the variant has information about the file types
2258        that can be used for that platform. If the platform file types support
2259        metadata extraction, headers, the number of spots, and other
2260        information may be automatically extracted from the raw data file(s).
2261        </para>
2262       
2263        <para>
2264        If the platform support it, raw data can also be imported into the database.
2265        This is handled by batchers and <classname docapi="net.sf.basedb.core.data">RawData</classname> objects.
2266        Which table to store the data in depends on the <property>rawDataType</property>
2267        property. The properties shown for the <classname>RawData</classname> class
2268        in the diagram are the mandatory properties. Each raw data type defines additional
2269        properties that are specific to that raw data type.
2270        </para>
2271       
2272      </sect3>
2273     
2274      <sect3 id="data_api.rawdata.spotimages">
2275        <title>Spot images</title>
2276       
2277        <para>
2278        Spot images can be created if you have the original image
2279        files. BASE can use 1-3 images as sources for the red, green
2280        and blue channel respectively. The creation of spotimages requires
2281        that x and y coordinates are given for each raw data spot. The scaling
2282        and offset values are used to convert the coordinates to pixel
2283        coordinates. With this information BASE is able to cut out a square
2284        from the source images that, theoretically, contains a specific spot and
2285        nothing else. The spot images are gamma-corrected independently and then
2286        put together into PNG images that are stored in a zip file.
2287        </para>
2288      </sect3>
2289     
2290    </sect2>
2291
2292    <sect2 id="data_api.experiments">
2293      <title>Experiments and analysis</title>
2294     
2295     
2296        <figure id="data_api.figures.experiments">
2297          <title>Experiments</title>
2298          <screenshot>
2299            <mediaobject>
2300              <imageobject>
2301                <imagedata 
2302                  align="center"
2303                  scalefit="1" width="75%"
2304                  fileref="figures/uml/datalayer.experiments.png" format="PNG" />
2305              </imageobject>
2306            </mediaobject>
2307          </screenshot>
2308        </figure>
2309     
2310      <sect3 id="data_api.experiments.description">
2311        <title>Experiments</title>
2312       
2313        <para>
2314          The <classname docapi="net.sf.basedb.core.data">ExperimentData</classname> 
2315          class is used to collect information about a single experiment. It
2316          links to any number of <classname docapi="net.sf.basedb.core.data">RawBioAssayData</classname>
2317          items, which must all be of the same <classname 
2318          docapi="net.sf.basedb.core">RawDataType</classname>.
2319        </para>
2320       
2321        <para>
2322          Annotation types that are needed in the analysis must connected to
2323          the experiment as experimental factors and the annotation values should
2324          be set on or inherited by each raw bioassay that is part of the
2325          experiment.
2326        </para>
2327       
2328        <para>
2329          The directory connected to the experiment is the default directory
2330          where plugins that generate files should store them.
2331        </para>
2332      </sect3>
2333           
2334      <sect3 id="data_api.experiments.bioassays">
2335        <title>Bioassay sets, bioassays and transformations</title>
2336       
2337        <para>
2338          Each line of analysis starts with the creation of a <emphasis>root</emphasis>
2339          <classname docapi="net.sf.basedb.core.data">BioAssaySetData</classname>,
2340          which holds the intensities calculated from the raw data.
2341          A bioassayset can hold one intensity for each channel. The number of
2342          channels is defined by the raw data type. For each raw bioassay used a
2343          <classname docapi="net.sf.basedb.core.data">BioAssayData</classname>
2344          is created.
2345        </para>
2346       
2347        <para>
2348          Information about the process that calculated the intensities are
2349          stored in a <classname docapi="net.sf.basedb.core.data">TransformationData</classname>
2350          object. The root transformation links with the raw bioassays that are used
2351          in this line of analysis and to a <classname 
2352          docapi="net.sf.basedb.core.data">JobData</classname> which has information
2353          about which plug-in and parameters that was used in the calculation.
2354        </para>
2355     
2356        <para>
2357          Once the root bioassayset has been created it is possible to
2358          again apply a transformation to it. This time the transformation
2359          links to a single source bioassayset instead of the raw bioassays.
2360          As before, it still links to a job with information about the plug-in and
2361          parameters that does the actual work. The transformation must make sure
2362          that new bioassays are created and linked to the bioassays in the
2363          source bioassayset. This above process may be repeated as many times
2364          as needed.
2365        </para>
2366       
2367        <para>
2368          Data to a bioassay set can only be added to it before it has been
2369          committed to the database. Once the transaction has been committed
2370          it is no longed possible to add more data or to modify existing
2371          data.
2372        </para>
2373     
2374      </sect3>
2375
2376      <sect3 id="data_api.experiments.virtualdb">
2377        <title>Virtual databases, datacubes, etc.</title>
2378       
2379        <para>
2380          The above processes requires a flexible storage solution for the data.
2381          Each experiment is related to a <classname docapi="net.sf.basedb.core.data">VirtualDb</classname>
2382          object. This object represents the set of tables that are needed to store
2383          data for the experiment. All tables are created in a special part of the
2384          BASE database that we call the <emphasis>dynamic database</emphasis>.
2385          In MySQL the dynamic database is a separate database, in Postgres it is
2386          a separate schema.
2387        </para>
2388       
2389        <para>
2390          A virual database is divided into data cubes. A data cube can be seen
2391          as a three-dimensional object where each point can hold data that in
2392          most cases can be interpreted as data for a single spot from an
2393          array. The coordinates to a point is given by <emphasis>layer</emphasis>,
2394          <emphasis>column</emphasis> and <emphasis>position</emphasis>. The
2395          layer and column coordinates are represented by the
2396          <classname docapi="net.sf.basedb.core.data">DataCubeLayerData</classname>
2397          and <classname docapi="net.sf.basedb.core.data">DataCubeColumnData</classname>
2398          objects. The position coordinate has no separate object associated with
2399          it.
2400        </para>
2401       
2402        <para>
2403          Data for a single bioassay set is always stored in a single layer. It
2404          is possible for more than one bioassay set to use the same layer. This
2405          usually happens for filtering transformations that doesn't modify the
2406          data.  The filtered bioassay set is then linked to a
2407          <classname docapi="net.sf.basedb.core.data">DataCubeFilterData</classname>
2408          object, which has information about which data points that
2409          passed the filter.
2410        </para>
2411       
2412        <para>
2413          All data for a bioassay is stored in a single column.
2414          Two bioassays in different bioassaysets (layers) can only have the same
2415          column if one is the parent of the other.
2416        </para>
2417       
2418        <para>
2419          The position coordinate is tied to a reporter.
2420        </para>
2421       
2422        <para>
2423          A child bioassay set may use the same data cube as it's parent
2424          bioassay set if all of the following conditions are true:
2425        </para>
2426       
2427        <itemizedlist>
2428        <listitem>
2429          <para>
2430          All positions are linked to the same reporter as the positions
2431          in the parent bioassay set.
2432          </para>
2433        </listitem>
2434       
2435        <listitem>
2436          <para>
2437          All data points are linked to the same (possible many) raw data
2438          spots as the corresponding data points in the parent bioassay set.
2439          </para>
2440        </listitem>
2441       
2442        <listitem>
2443          <para>
2444          The bioassays in the child bioassay set each have exactly one
2445          parent in the parent bioassay set. No parent bioassay may be the
2446          parent of more than one child bioassay.
2447          </para>
2448        </listitem>
2449        </itemizedlist>
2450       
2451        <para>
2452          If any of the above conditions are not true, a new data cube
2453          must be created for the child bioassay set.
2454        </para>
2455      </sect3>
2456     
2457      <sect3 id="data_api.dynamic.description">
2458        <title>The dynamic database</title>
2459
2460        <figure id="data_api.figures.dynamic">
2461          <title>The dynamic database</title>
2462          <screenshot>
2463            <mediaobject>
2464              <imageobject>
2465                <imagedata 
2466                  align="center"
2467                  fileref="figures/uml/datalayer.dynamic.png" format="PNG" />
2468              </imageobject>
2469            </mediaobject>
2470          </screenshot>
2471        </figure>
2472       
2473        <para>
2474          Each virtual database consists of several tables. The tables
2475          are dynamically created when needed. For each table shown in the diagram
2476          the # sign is replaced by the id of the virtual database object at run
2477          time.
2478        </para>
2479       
2480        <para>
2481          There are no classes in the data layer for these tables and they
2482          are not mapped with Hibernate. When we work with these tables we
2483          are always using batcher classes and queries that works with integer,
2484          floats and strings.
2485        </para>
2486       
2487        <bridgehead>The D#Spot table</bridgehead>
2488        <para>
2489          This is the main table which keeps the intensities for a single spot
2490          in the data cube. Extra values attached to the spot are kept in separate
2491          tables, one for each type of value (D#SpotInt, D#SpotFloat and D#SpotString).
2492        </para>
2493       
2494        <bridgehead>The D#Pos table</bridgehead>
2495        <para>
2496          This table stores the reporter id for each position in a cube.
2497          Extra values attached to the position are kept in separate tables,
2498          one for each type of value (D#PosInt, D#PosFloat and D#PosString).
2499        </para>
2500       
2501        <bridgehead>The D#Filter table</bridgehead>
2502        <para>
2503          This table stores the coordinates for the spots that remain after
2504          filtering. Note that each filter is related to a bioassayset which
2505          gives the cube and layer values. Each row in the filter table then
2506          adds the column and position allowing us to find the spots in the
2507          D#Spot table.
2508        </para>
2509       
2510        <bridgehead>The D#RawParents table</bridgehead>
2511        <para>
2512          This table holds mappings for a spot to the raw data it is calculated
2513          from. We don't need the layer coordinate since all layers in a cube
2514          must have the same mapping to raw data.
2515        </para>
2516       
2517      </sect3>     
2518
2519     
2520    </sect2>
2521   
2522    <sect2 id="data_api.misc">
2523      <title>Other classes</title>
2524     
2525        <figure id="data_api.figures.misc">
2526          <title>Other classes</title>
2527          <screenshot>
2528            <mediaobject>
2529              <imageobject>
2530                <imagedata 
2531                  align="center"
2532                  fileref="figures/uml/datalayer.misc.png" format="PNG" />
2533              </imageobject>
2534            </mediaobject>
2535          </screenshot>
2536        </figure>
2537     
2538    </sect2>
2539
2540  </sect1>
2541 
2542  <sect1 id="base_api.core" chunked="1">
2543    <?dbhtml filename="core_api.html" ?>
2544    <title>The Core API</title>
2545   
2546    <para>
2547      This section gives an overview of various parts of the core API.
2548    </para>
2549   
2550    <sect2 id="core_api.authentication">
2551      <title>Authentication and sessions</title>
2552      <para>
2553        This documentation is only available in the old format which may not be up-to-date with
2554        the current implementation.
2555        See <ulink url="http://base.thep.lu.se/chrome/site/doc/historical/development/overview/core/authentication.html"
2556          >http://base.thep.lu.se/chrome/site/doc/historical/development/overview/core/authentication.html</ulink>
2557      </para>
2558    </sect2>   
2559   
2560    <sect2 id="core_api.accesspermissions">
2561      <title>Access permissions</title>
2562      <para>
2563        This documentation is only available in the old format which may not be up-to-date with
2564        the current implementation.
2565        See <ulink url="http://base.thep.lu.se/chrome/site/doc/historical/development/overview/core/accesspermissions.html"
2566          >http://base.thep.lu.se/chrome/site/doc/historical/development/overview/core/accesspermissions.html</ulink>
2567      </para>
2568    </sect2>
2569   
2570    <sect2 id="core_api.datavalidation">
2571      <title>Data validation</title>
2572      <para>
2573        TODO
2574      </para>
2575    </sect2>   
2576    <sect2 id="core_api.transactions">
2577      <title>Transaction handling</title>
2578      <para>
2579        TODO
2580      </para>
2581    </sect2>   
2582    <sect2 id="core_api.crwd">
2583      <title>Create/read/write/delete operations</title>
2584      <para>
2585        This documentation is only available in the old format which may not be up-to-date with
2586        the current implementation.
2587        See <ulink url="http://base.thep.lu.se/chrome/site/doc/historical/development/overview/core/itemhandling.html"
2588          >http://base.thep.lu.se/chrome/site/doc/historical/development/overview/core/itemhandling.html</ulink>
2589      </para>
2590    </sect2>   
2591    <sect2 id="core_api.batch">
2592      <title>Batch operations</title>
2593      <para>
2594        This documentation is only available in the old format which may not be up-to-date with
2595        the current implementation.
2596        See <ulink url="http://base.thep.lu.se/chrome/site/doc/historical/development/overview/core/batchprocessing.html"
2597          >http://base.thep.lu.se/chrome/site/doc/historical/development/overview/core/batchprocessing.html</ulink>
2598      </para>
2599    </sect2>   
2600   
2601    <sect2 id="core_api.quota">
2602      <title>Quota</title>
2603      <para>
2604        TODO
2605      </para>
2606    </sect2>   
2607    <sect2 id="core_api.pluginexecution">
2608      <title>Plugin execution / job queue</title>
2609      <para>
2610        This documentation is only available in the old format which may not be up-to-date with
2611        the current implementation.
2612        See <ulink url="http://base.thep.lu.se/chrome/site/doc/historical/development/overview/core/plugins.html"
2613          >http://base.thep.lu.se/chrome/site/doc/historical/development/overview/core/plugins.html</ulink>
2614      </para>
2615    </sect2>   
2616   
2617    <sect2 id="core_api.data_in_files">
2618      <title>Using files to store data</title>
2619     
2620      <para>
2621        BASE 2.5 introduced the possibility to use files to store data instead
2622        of importing it into the database. Files can be attached
2623        to any item that implements the <interfacename docapi="net.sf.basedb.core">FileStoreEnabled</interfacename>
2624        interface. Currently this is <classname docapi="net.sf.basedb.core">RawBioAssay</classname>
2625        and <classname docapi="net.sf.basedb.core">ArrayDesign</classname>. The
2626        ability to store data in files is not a replacement for storing data in the
2627        database. It is possible (for some platforms/raw data types) to have data in
2628        files and in the database at the same time. We would have liked to enforce
2629        that (raw) data is always present in files, but this will not be backwards compatible
2630        with older installations, so there are three cases:
2631      </para>
2632     
2633      <itemizedlist>
2634      <listitem>
2635        <para>
2636        Data in files only
2637        </para>
2638      </listitem>
2639      <listitem>
2640        <para>
2641        Data in the database only
2642        </para>
2643      </listitem>
2644      <listitem>
2645        <para>
2646        Data in both files and in the database
2647        </para>
2648      </listitem>
2649      </itemizedlist>
2650     
2651      <para>
2652        Not all three cases are supported for all types of data. This is controlled
2653        by the <classname docapi="net.sf.basedb.core">Platform</classname> class, which may disallow
2654        that data is stored in the database. To check this call
2655        <methodname>Platform.isFileOnly()</methodname> and/or
2656        <methodname>Platform.getRawDataType()</methodname>. If the <methodname>isFileOnly()</methodname>
2657        method returns <constant>true</constant>, the platform can't store data in
2658        the database. If the value is <constant>false</constant> more information
2659        can be obtained by calling <methodname>getRawDataType()</methodname>,
2660        which may return:
2661      </para>
2662     
2663      <itemizedlist>
2664      <listitem>
2665        <para>
2666          <constant>null</constant>: The platform can store data with any
2667          raw data type in the database.
2668        </para>
2669      </listitem>
2670      <listitem>
2671        <para>
2672        A <classname docapi="net.sf.basedb.core">RawDataType</classname> that has <code>isStoredInDb() == true</code>:
2673        The platform can store data in the database but only data with the specified raw
2674        data type.
2675        </para>
2676      </listitem>
2677      <listitem>
2678        <para>
2679        A <classname docapi="net.sf.basedb.core">RawDataType</classname> that has <code>isStoredInDb() == false</code>:
2680        The platform can't store data in the database.
2681        </para>
2682      </listitem>
2683      </itemizedlist>
2684
2685      <para>
2686        One major change from earlier BASE versions is that the registration of raw data types
2687        has changed. The <filename>raw-data-types.xml</filename> file should
2688        only be used for raw data types that are stored in the database. The
2689        <sgmltag>storage</sgmltag> tag has been deprecated and BASE will refuse
2690        to start if it finds a raw data type definitions with <code>storage="file"</code>.
2691      </para>
2692     
2693      <para>
2694        For backwards compatibility reasons, each <classname docapi="net.sf.basedb.core">Platform</classname>
2695        that can only store data in files will create "virtual" raw data type
2696        objects internally. These raw data types all return <constant>false</constant>
2697        from the <methodname>RawDataType.isStoredInDb()</methodname>
2698        method. They also have a back-link to the platform/variant that
2699        created it: <methodname>RawDataType.getPlatform()</methodname>
2700        and <methodname>RawDataType.getVariant()</methodname>. These two methods
2701        will always return <constant>null</constant> when called on a raw data type
2702        that can be stored in the database.
2703      </para>
2704     
2705      <itemizedlist>
2706        <title>See also</title>
2707        <listitem><xref linkend="data_api.platforms" /></listitem>
2708        <listitem><xref linkend="plugin_developer.other.datafiles" /></listitem>
2709        <listitem><xref linkend="appendix.rawdatatypes.platforms" /></listitem>
2710        <listitem>
2711          <xref linkend="appendix.incompatible.2.5" /> in
2712          <xref linkend="appendix.incompatible" />
2713        </listitem>
2714      </itemizedlist>
2715     
2716      <sect3 id="core_api.data_in_files.diagram">
2717        <title>Diagram of classes and methods</title>
2718        <figure id="core_api.figures.data_in_files">
2719          <title>Store data in files</title>
2720          <screenshot>
2721            <mediaobject>
2722              <imageobject>
2723                <imagedata 
2724                  align="center"
2725                  scalefit="1" width="100%"
2726                  fileref="figures/uml/corelayer.datainfiles.png" format="PNG" />
2727              </imageobject>
2728            </mediaobject>
2729          </screenshot>
2730        </figure>
2731       
2732        <para>
2733          This is rather large set of classes and methods. The ultimate goal
2734          is to be able to create links between a <classname docapi="net.sf.basedb.core">RawBioAssay</classname>
2735          / <classname docapi="net.sf.basedb.core">ArrayDesign</classname> and <classname docapi="net.sf.basedb.core">File</classname>
2736          items and to provide some metadata about the files.
2737          The <classname docapi="net.sf.basedb.core">FileStoreUtil</classname> class is one of the most
2738          important ones. It is intended to make it easy for plug-in (and other)
2739          developers to access the files without having to mess with platform
2740          or file type objects. The API is best described
2741          by a set of use-case examples.
2742        </para>
2743       
2744      </sect3>
2745     
2746      <sect3 id="core_api.data_in_files.ask">
2747        <title>Use case: Asking the user for files for a given item</title>
2748
2749        <para>
2750          A client application must know what types of files it makes sense
2751          to ask the user for. In some cases, data may be split into more than
2752          one file so we need a generic way to select files.
2753        </para>
2754       
2755        <para>
2756          Given that we have a <interfacename docapi="net.sf.basedb.core">FileStoreEnabled</interfacename>
2757          item we want to find out which <classname docapi="net.sf.basedb.core">DataFileType</classname>
2758          items that can be used for that item. The
2759          <methodname>DataFileType.getQuery(FileStoreEnabled)</methodname>
2760          can be used for this. Internally, the method uses the result from
2761          <methodname>FileStoreEnabled.getPlatform()</methodname>
2762          and <methodname>FileStoreEnabled.getVariant()</methodname>
2763          methods to restrict the query to only return file types for
2764          a given platform and/or variant. If the item doesn't have
2765          a platform or variant the query will return all file types
2766          that are associated with the given item type. In any case, we get a list
2767          of <classname>DataFileType</classname> items, each one representing a
2768          specific file type that we should ask the user about. Examples:
2769        </para>
2770
2771        <orderedlist>
2772        <listitem>
2773          <para>
2774          The <constant>Affymetrix</constant> platform defines <constant>CEL</constant>
2775          as a raw data file and <constant>CDF</constant> as an array design (reporter map)
2776          file. If we have a <classname docapi="net.sf.basedb.core">RawBioAssay</classname> the query will only return
2777          the CEL file type and the client can ask the user for a CEL file.
2778          </para>
2779        </listitem>
2780        <listitem>
2781          <para>
2782          The <constant>Generic</constant> platform defines <constant>PRINT_MAP</constant>
2783          and <constant>REPORTER_MAP</constant> for array designs. If we have
2784          an <classname docapi="net.sf.basedb.core">ArrayDesign</classname> the query will return those two
2785          items.
2786          </para>
2787        </listitem>
2788        </orderedlist>
2789     
2790        <para>
2791          It might also be interesting to know the currently selected file
2792          for each file type and if the platform has set the <varname>required</varname>
2793          flag for a particular file type. Here is a simple code example
2794          that may be useful to start from:
2795        </para>
2796     
2797        <programlisting language="java">
2798DbControl dc = ...
2799FileStoreEnabled item = ...
2800Platform platform = item.getPlatform();
2801PlatformVariant variant = item.getVariant();
2802
2803// Get list of DataFileTypes used by the platform
2804ItemQuery&lt;DataFileType&gt; query =
2805   DataFileType.getQuery(item);
2806List&lt;DataFileType&gt; types = query.list(dc);
2807
2808// Always check hasFileSet() method first to avoid
2809// creating the file set if it doesn't exists
2810FileSet fileSet = item.hasFileSet() ?
2811   null : item.getFileSet();
2812   
2813for (DataFileType type : types)
2814{
2815   // Get the current file, if any
2816   FileSetMember member = fileSet == null || !fileSet.hasMember(type) ?
2817      null : fileSet.getMember(type);
2818   File current = member == null ?
2819      null : member.getFile();
2820   
2821   // Check if a file is required by the platform
2822   PlatformFileType pft = platform == null ?
2823      null : platform.getFileType(type, variant);
2824   boolean isRequired = pft == null ?
2825      false : pft.isRequired();
2826     
2827   // Now we can do something with this information to
2828   // let the user select a file ...
2829}
2830</programlisting>
2831     
2832        <note>
2833          <title>Also remember to catch PermissionDeniedException</title>
2834          <para>
2835            The above code may look complicated, but this is mostly because
2836            of all checks for <constant>null</constant> values. Remember
2837            that many things are optional and may return <constant>null</constant>.
2838            Another thing to look out for is
2839            <exceptionname>PermissionDeniedException</exceptionname>:s. The logged in
2840            user may not have access to all items. The above example doesn't include
2841            any code for this since it would have made it too complex.
2842          </para>
2843        </note>
2844      </sect3>
2845     
2846      <sect3 id="core_api.data_in_files.link">
2847        <title>Use case: Link, validate and extract metadata from the selected files</title>
2848        <para>
2849          When the user has selected the file(s) we must store the links
2850          to them in the database. This is done with a <classname docapi="net.sf.basedb.core">FileSet</classname>
2851          object. A file set can contain any number of files. The only limitation
2852          is that it can only contain one file for each file type.
2853          Call <methodname>FileSet.setMember()</methodname> to store
2854          a file in the file set. If a file already exists for the given file type
2855          it is replaced, otherwise a new entry is created. The following
2856          program example assumes that we have a map where <classname docapi="net.sf.basedb.core">File</classname>:s
2857          are related to <classname docapi="net.sf.basedb.core">DataFileType</classname>:s. When all files
2858          have been added we call <methodname>FileSet.validate()</methodname>
2859          to validate the files and extract metadata.
2860        </para>
2861       
2862        <programlisting language="java">
2863DbControl dc = ...
2864FileStoreEnabled item = ...
2865Map&lt;DataFileType, File&gt; files = ...
2866
2867// Store the selected files in the fileset
2868FileSet fileSet = item.getFileSet();
2869for (Map.Entry&lt;DataFileType, File&gt; entry : files)
2870{
2871   DataFileType type = entry.getKey();
2872   File file = entry.getValue();
2873   fileSet.setMember(type, file);
2874}
2875
2876// Validate the files and extract metadata
2877fileSet.validate(dc, true);
2878</programlisting>
2879
2880        <para>
2881          Validation and extraction of metadata is important since we want
2882          data in files to be equivalent to data in the database. The validation
2883          and metadata extraction is done by the core when the
2884          <methodname>FileSet.validate()</methodname> is called.
2885          The process is partly pluggable since each <classname docapi="net.sf.basedb.core">DataFileType</classname> 
2886          can name a class that should do the validation and/or metadata extraction.
2887        </para>
2888
2889        <note>
2890          <para>
2891          The <methodname>FileSet.validate()</methodname> only validates
2892          the files where the file types have specified plug-ins that can
2893          do the validation and metadata extraction. The method doesn't
2894          throw any exceptions. Instead, all validation errors
2895          are returned a list of <classname>Throwable</classname>:s. The
2896          validation result is also stored for each file and can be access
2897          with <methodname>FileSetMember.isValid()</methodname> and
2898          <methodname>FileSetMember.getErrorMessage()</methodname>.
2899          </para>
2900        </note>
2901
2902        <para>
2903          Here is the general outline of what is going on in the core:
2904        </para>
2905
2906        <orderedlist>
2907        <listitem>
2908          <para>
2909          The core checks the <classname docapi="net.sf.basedb.core">DataFileType</classname> of all
2910          members in the file set and creates <classname docapi="net.sf.basedb.core.filehandler">DataFileValidator</classname>
2911          and <classname docapi="net.sf.basedb.core.filehandler">DataFileMetadataReader</classname> objects. Only one instance
2912          of each class is created. If the file set contains members which has the
2913          same validator or metadata reader, they will all share the same instance.
2914          </para>
2915        </listitem>
2916       
2917        <listitem>
2918          <para>
2919          Each validator/metadata reader class is initialised with calls to
2920          <methodname>DataFileHandler.setItem()</methodname> and
2921          <methodname>DataFileHandler.setFile()</methodname>.
2922          </para>
2923        </listitem>
2924       
2925        <listitem>
2926          <para>
2927          Each validator is called. The result of the validation is saved for each
2928          file and can be retreieved by <methodname>FileSetMember.isValid()</methodname>
2929          and <methodname>FileSetMember.getErrorMessage()</methodname>.
2930          </para>
2931        </listitem>
2932       
2933        <listitem>
2934          <para>
2935          Each metadata reader is called, unless the metadata reader is the same class
2936          as the validator and the validation failed. If the metadata reader is a
2937          different class, it is called even if the validation failed.
2938          </para>
2939        </listitem>
2940        </orderedlist>
2941
2942        <note>
2943          <title>Only one instance of each validator class is created</title>
2944          <para>
2945          The validation/metadata extraction is not done until all files have been
2946          added to the fileset. If the same validator/meta data reader is
2947          used for more than one file, the same instance is reused. Ie.
2948          the <methodname>setFile()</methodname> is called one time
2949          for each file/file type pair. The <methodname>validate()</methodname>
2950          and <methodname>extractMetadata()</methodname> methods are only
2951          called once.
2952          </para>
2953        </note>
2954       
2955        <para>
2956          All validators and meta data extractors should extend
2957          the <classname docapi="net.sf.basedb.core.filehandler">AbstractDataFileHandler</classname> class. The reason
2958          is that we may want to add more methods to the <interfacename docapi="net.sf.basedb.core.filehandler">DataFileHandler</interfacename>
2959          interface in the future. The <classname docapi="net.sf.basedb.core.filehandler">AbstractDataFileHandler</classname> will
2960          be used to provide default implementations for backwards compatibility.
2961        </para>
2962       
2963      </sect3>
2964     
2965      <sect3 id="core_api.data_in_files.import">
2966        <title>Use case: Import data into the database</title>
2967       
2968        <para>
2969          This should be done by existing plug-ins in the same way as before.
2970          A slight modification is needed since it is good if the importers
2971          are made aware of already selected files in the <classname docapi="net.sf.basedb.core">FileSet</classname>
2972          to provide good default values. The <classname docapi="net.sf.basedb.core">FileStoreUtil</classname>
2973          class is very useful in cases like this:
2974        </para>
2975       
2976        <programlisting language="java">
2977RawBioAssay rba = ...
2978DbControl dc = ...
2979
2980// Get the current raw data file, if any
2981List&lt;File&gt; rawDataFiles =
2982   FileStoreUtil.getGenericDataFiles(dc, rba, FileType.RAW_DATA);
2983File defaultFile = rawDataFiles.size() > 0 ?
2984   rawDataFiles.get(0) : null;
2985   
2986// Create parameter asking for input file - use current as default
2987PluginParameter&lt;File&gt; fileParameter = new PluginParameter&lt;File&gt;(
2988   "file",
2989   "Raw data file",
2990   "The file that contains the raw data that you want to import",
2991   new FileParameterType(defaultFile, true, 1)
2992);
2993</programlisting>
2994
2995      <para>
2996        An import plug-in should also save the file that was used to the file set:
2997      </para>
2998     
2999      <programlisting language="java">
3000RawBioassay rba = ...
3001// The file the user selected to import from
3002File rawDataFile = (File)job.getValue("file");
3003
3004// Save the file to the fileset. The method will check which file
3005// type the platform uses as the raw data type. As a fallback the
3006// GENERIC_RAW_DATA type is used
3007FileStoreUtil.setGenericDataFile(dc, rba, FileType.RAW_DATA,
3008   DataFileType.GENERIC_RAW_DATA, rawDataFile);
3009</programlisting>
3010
3011      </sect3>
3012     
3013      <sect3 id="core_api.data_in_files.experiments">
3014        <title>Use case: Using raw data from files in an experiment</title>
3015       
3016        <para>
3017          Just as before, an experiment is still locked to a single
3018          <classname docapi="net.sf.basedb.core">RawDataType</classname>. This is a design issue that
3019          would break too many things if changed. If data is stored in files
3020          the experiment is also locked to a single <classname docapi="net.sf.basedb.core">Platform</classname>.
3021          This has been designed to have as little impact on existing
3022          plug-ins as possible. In most cases, the plug-ins will continue
3023          to work as before.
3024        </para>
3025       
3026        <para>
3027          A plug-in (using data from the database that needs to check if it can
3028          be used within an experiment can still do:
3029        </para>
3030       
3031        <programlisting language="java">
3032Experiment e = ...
3033RawDataType rdt = e.getRawDataType();
3034if (rdt.isStoredInDb())
3035{
3036   // Check number of channels, etc...
3037   // ... run plug-in code ...
3038}
3039</programlisting>
3040       
3041        <para>
3042          A newer plug-in which uses data from files should do:
3043        </para>
3044       
3045        <programlisting language="java">
3046Experiment e = ...
3047DbControl dc = ...
3048RawDataType rdt = e.getRawDataType();
3049if (!rdt.isStoredInDb())
3050{
3051   // Check that platform/variant is supported
3052   Platform p = rdt.getPlatform(dc);
3053   PlatformVariant v = rdt.getVariant(dc);
3054   // ...
3055
3056   // Get data files
3057   File aFile = FileStoreUtil.getDataFile(dc, ...);
3058   
3059   // ... run plug-in code ...
3060}
3061</programlisting>
3062       
3063      </sect3>
3064     
3065    </sect2>
3066   
3067    <sect2 id="core_api.signals">
3068      <title>Sending signals (to plug-ins)</title>
3069   
3070      <para>
3071        BASE has a simple system for sending signals between different parts of
3072        a system. This signalling system was initially developed to be able to
3073        kill plug-ins that a user for some reason wanted to abort. The signalling
3074        system as such is not limited to this and it can be used for other purposes
3075        as well. Signals can of course be handled internally in a single JVM but
3076        also sent externally to other JVM:s running on the same or a different
3077        computer. The transport mechanism for signals is decoupled from the actual
3078        handling of them. If you want to, you could implement a signal transporter
3079        that sends signal as emails and the target plug-in would never know.
3080      </para>
3081     
3082      <para>
3083        The remainder of this section will focus mainly on the sending and
3084        transportation of signals. For more information about handling signals
3085        on the receiving end, see <xref linkend="plugin_developer.signals" />.
3086      </para>
3087     
3088      <sect3 id="core_api.signals.diagram">
3089        <title>Diagram of classes and methods</title>
3090        <figure id="core_api.figures.signals">
3091          <title>The signalling system</title>
3092          <screenshot>
3093            <mediaobject>
3094              <imageobject>
3095                <imagedata 
3096                  align="center"
3097                  scalefit="1" width="100%"
3098                  fileref="figures/uml/corelayer.signals.png" format="PNG" />
3099              </imageobject>
3100            </mediaobject>
3101          </screenshot>
3102        </figure>
3103     
3104        <para>
3105          The signalling system is rather simple. An object that wish
3106          to receieve signals must implement the
3107          <interfacename docapi="net.sf.basedb.core.signal"
3108          >SignalTarget</interfacename>. It's only method
3109          is <methodname>getSignalHandler()</methodname>. A
3110          <interfacename docapi="net.sf.basedb.core.signal"
3111          >SignalHandler</interfacename> is an object that
3112          knows what to do when a signal is delivered to it. The target object
3113          may implement the <interfacename>SignalHandler</interfacename> itself
3114          or use one of the existing handlers.
3115        </para>
3116       
3117        <para>
3118          The difficult part here is to be aware that a signal is usually
3119          delivered by a separate thread. The target object must be aware
3120          of this and know how to handle multiple threads. As an example we
3121          can use the <classname docapi="net.sf.basedb.core.signal"
3122          >ThreadSignalHandler</classname> which simply
3123          calls <code>Thread.interrupt()</code> to deliver a signal. The target
3124          object that uses this signal handler it must know that it should check
3125          <code>Thread.interrupted()</code> at regular intervals from the main
3126          thread. If that method returns true, it means that the <constant>ABORT</constant>
3127          signal has been delivered and the main thread should clean up and exit as
3128          soon as possible.
3129        </para>
3130       
3131        <para>
3132          Even if a signal handler could be given directly to the party
3133          that may be interested in sending a signal to the target this
3134          is not recommended. This would only work when sending signals
3135          within the same virtual machine. The signalling system includes
3136          <interfacename docapi="net.sf.basedb.core.signal"
3137          >SignalTransporter</interfacename> and
3138          <interfacename docapi="net.sf.basedb.core.signal"
3139          >SignalReceiver</interfacename> objects that are used
3140          to decouple the sending of signals with the handling of signals. The
3141          implementation usually comes in pairs, for example
3142          <classname docapi="net.sf.basedb.core.signal"
3143          >SocketSignalTransporters</classname> and <classname 
3144          docapi="net.sf.basedb.core.signal">SocketSignalReceiver</classname>.
3145        </para>
3146       
3147        <para>
3148          Setting up the transport mechanism is usually a system responsibility.
3149          Only the system know what kind of transport that is appropriate for it's current
3150          setup. Ie. should signals be delievered by TCP/IP sockets, only internally, or
3151          should a delivery mechanism based on web services be implemented?
3152          If a system wants to receive signals it must create an appropriate
3153          <interfacename>SignalReceiver</interfacename> object. Within BASE the
3154          internal job queue set up it's own signalling system that can be used to
3155          send signals (eg. kill) running jobs. The job agents do the same but uses
3156          a different implementation. See <xref linkend="appendix.base.config.jobqueue" />
3157          for more information about how to configure the internal job queue's
3158          signal receiver. In both cases, there is only one signal receiver instance
3159          active in the system.
3160        </para>
3161       
3162        <para>
3163          Let's take the internal job queue as an example. Here is how it works:
3164        </para>
3165       
3166        <itemizedlist>
3167        <listitem>
3168          <para>
3169          When the internal job queue is started, it will also create a signal
3170          receiver instance according to the settings in <filename>base.config</filename>.
3171          The default is to create <classname docapi="net.sf.basedb.core.signal"
3172          >LocalSignalReceiver</classname>
3173          which can only be used inside the same JVM. If needed, this can
3174          be changed to a <classname docapi="net.sf.basedb.core.signal"
3175          >SocketSignalReceiver</classname> or any other
3176          user-provided implementation.
3177          </para>
3178        </listitem>
3179       
3180        <listitem>
3181          <para>
3182          When the job queue has found a plug-in to execute it will check if
3183          it also implements the <interfacename docapi="net.sf.basedb.core.signal"
3184          >SignalTarget</interfacename>
3185          interface. If it does, a signal handler is created and registered
3186          with the signal receiver. This is actually done by the BASE core
3187          by calling <methodname>PluginExecutionRequest.registerSignalReceiver()</methodname>
3188          which also makes sure that the the ID returned from the registration is
3189          stored in the database together with the job item representing the
3190          plug-in to execute.
3191          </para>
3192        </listitem>
3193       
3194        <listitem>
3195          <para>
3196          Now, when the web client see's a running job which has a non-empty
3197          signal transporter property, the <guilabel>Abort</guilabel>
3198          button is activated. If the user clicks this button the BASE core
3199          uses the information in the database to create
3200          <interfacename docapi="net.sf.basedb.core.signal"
3201          >SignalTransporter</interfacename> object. This
3202          is simply done by calling <code>Job.getSignalTransporter()</code>.
3203          The created signal transporter knows how to send a signal
3204          to the signal receiver it was first registered with. When the
3205          signal arrives at the receiver it will find the handler for it
3206          and call <code>SignalHandler.handleSignal()</code>. This will in it's turn
3207          trigger some action in the signal target which soon will abort what
3208          it is doing and exit.
3209          </para>
3210        </listitem>
3211        </itemizedlist>
3212       
3213       
3214      </sect3>
3215   
3216    </sect2>
3217   
3218  </sect1>
3219
3220  <sect1 id="core_api.query">
3221    <?dbhtml filename="query_api.html" ?>
3222    <title>The Query API</title>
3223    <para>
3224      This documentation is only available in the old format.
3225      See <ulink url="http://base.thep.lu.se/chrome/site/doc/historical/development/overview/query/index.html"
3226        >http://base.thep.lu.se/chrome/site/doc/historical/development/overview/query/index.html</ulink>
3227    </para>
3228   
3229  </sect1>
3230 
3231  <sect1 id="core_api.dynamic">
3232    <?dbhtml filename="dynamic_api.html" ?>
3233    <title>The Dynamic API</title>
3234    <para>
3235      This documentation is only available in the old format.
3236      See <ulink url="http://base.thep.lu.se/chrome/site/doc/historical/development/overview/dynamic/index.html"
3237        >http://base.thep.lu.se/chrome/site/doc/historical/development/overview/dynamic/index.html</ulink>
3238    </para>
3239  </sect1>
3240
3241  <sect1 id="core_api.extensions">
3242    <?dbhtml filename="extensions_api.html" ?>
3243    <title>The Extensions API</title>
3244   
3245    <sect2 id="extensions_api.core">
3246      <title>The core part</title>
3247   
3248      <para>
3249        The <emphasis>Extensions API</emphasis> is divided into two parts. A core
3250        part and a web client specific part. The core part can be found in the
3251        <package>net.sf.basedb.util.extensions</package> package and it's sub-packages,
3252        and consists of three sub-parts:
3253      </para>
3254     
3255      <itemizedlist>
3256      <listitem>
3257        <para>
3258        A set of interface definitions which forms the core of the Extensions API.
3259        The interfaces defines, for example, what an <interfacename 
3260        docapi="net.sf.basedb.util.extensions">Extension</interfacename> is and
3261        what an <interfacename 
3262        docapi="net.sf.basedb.util.extensions">ActionFactory</interfacename> should do.
3263        </para>
3264      </listitem>
3265     
3266      <listitem>
3267        <para>
3268        A <classname docapi="net.sf.basedb.util.extensions">Registry</classname> that is
3269        used to keep track of installed extensions. The registry also provides
3270        functionality for invoking and using the extensions.
3271        </para>
3272      </listitem>
3273     
3274      <listitem>
3275        <para>
3276        Utility classes that are useful when implementation a client application
3277        that can be extendable. The most useful example is the <classname
3278        docapi="net.sf.basedb.util.extensions.xml">XmlLoader</classname> which can
3279        read extension definitions from XML files and create the proper factories,
3280        etc.
3281        </para>
3282      </listitem>
3283      </itemizedlist>
3284     
3285      <figure id="extensions_api.figures.core">
3286        <title>The core part of the Extensions API</title>
3287        <screenshot>
3288          <mediaobject>
3289            <imageobject>
3290              <imagedata 
3291                align="center"
3292                fileref="figures/uml/corelayer.extensions_core.png" format="PNG" />
3293            </imageobject>
3294          </mediaobject>
3295        </screenshot>
3296      </figure>
3297     
3298      <para>
3299        The <classname docapi="net.sf.basedb.util.extensions">Registry</classname> 
3300        is one of the main classes in the extension system. All extension points and
3301        extensions must be registered before they can be used. Typically, you will
3302        first register extension points and then extensions, beacuse an extension
3303        can't be registered until the extension point it is extending has been
3304        registered.
3305      </para>
3306     
3307      <para>
3308        An <interfacename docapi="net.sf.basedb.util.extensions">ExtensionPoint</interfacename>
3309        is an ID and a definition of an <interfacename docapi="net.sf.basedb.util.extensions">Action</interfacename>
3310        class. The other options (name, description, renderer factory, etc.) are optional.
3311        An <interfacename docapi="net.sf.basedb.util.extensions">Extension</interfacename>
3312        that extends a specific extension point must provide an
3313        <interfacename docapi="net.sf.basedb.util.extensions">ActionFactory</interfacename>
3314        instance that can create actions of the type the extension point requires.
3315      </para>
3316     
3317      <example id="extensions_api.example.menu">
3318        <title>The menu extensions point</title>
3319        <para>
3320        The <code>net.sf.basedb.clients.web.menu.extensions</code> extension point
3321        requires <interfacename 
3322        docapi="net.sf.basedb.clients.web.extensions.menu">MenuItemAction</interfacename>
3323        objects. An extension for this extension point must provide a factory that
3324        can create <classname>MenuItemAction</classname>:s. BASE ships with default
3325        factory implementations, for example the <classname 
3326        docapi="net.sf.basedb.clients.web.extensions.menu">FixedMenuItemFactory</classname>
3327        class, but an extension may provide it's own factory implementation if it wants to.
3328        </para>
3329      </example>
3330     
3331      <para>
3332        Call the <methodname>Registry.useExtensions()</methodname> method
3333        to use extensions from one or several extension points. This method will
3334        find all extensions for the given extension points. If a filter is given,
3335        it checks if any of the extensions or extension points has been disabled.
3336        It will then call <methodname>ActionFactory.prepareContext()</methodname>
3337        for all remaining extensions. This gives the action factory a chance to
3338        also disable the extension, for example, if the logged in user doesn't
3339        have a required permission. The action factory may also set attributes
3340        on the context. The attributes can be anything that the extension point
3341        may make use of. Check the documentation for the specific extension point
3342        for information about which attributes it supports. If there are
3343        any renderer factories, their <methodname>RendererFactory.prepareContext()</methodname>
3344        is also called. They have the same possibility of setting attributes
3345        on the context, but can't disable an extension.
3346      </para>
3347     
3348      <para>
3349        After this, an <classname 
3350        docapi="net.sf.basedb.util.extensions">ExtensionsInvoker</classname>
3351        object is created and returned to the extension point. Note that
3352        the <methodname>ActionFactory.getActions()</methodname> has not been
3353        called yet, so we don't know if the extensions are actually
3354        going to generate any actions. The <methodname>ActionFactory.getActions()</methodname>
3355        is not called until we have got ourselves an
3356        <classname docapi="net.sf.basedb.util.extensions">ActionIterator</classname>
3357        from the <methodname>ExtensionsInvoker.iterate()</methodname> method and
3358        starts to iterate. The call to <methodname>ActionIterator.hasNext()</methodname>
3359        will propagate down to <methodname>ActionFactory.getActions()</methodname>
3360        and the generated actions are then available with the
3361        <methodname>ActionIterator.next()</methodname> method.
3362      </para>
3363     
3364      <para>
3365        The <methodname>ExtensionsInvoker.renderDefault()</methodname>
3366        and <methodname>ExtensionsInvoker.render()</methodname> are
3367        just convenience methods that will make it easer to render
3368        the actions. The first method will of course only work if the
3369        extension point is providing a renderer factory, that can
3370        create the default renderer.
3371      </para>
3372     
3373      <note>
3374        <title>Be aware of multi-threading issues</title>
3375        <para>
3376          When you are creating extensions you must be aware that
3377          multiple threads may access the same objects at the same time.
3378          In particular, any action factory or renderer factory has to be
3379          thread-safe, since only one exists for each extension.
3380          Action and renderer objects should be thread-safe if the
3381          factories re-use the same objects.
3382        </para>
3383      </note>
3384   
3385      <para>
3386        Any errors that happen during usage of an extension is handled by an
3387        <interfacename docapi="net.sf.basedb.util.extensions">ErrorHandler</interfacename>.
3388        The core provides two implementations. We usually don't want the
3389        errors to show up in the gui so the <classname 
3390        docapi="net.sf.basedb.util.extensions">LoggingErrorHandlerFactory</classname> 
3391        is the default implementation that only writes to the log file. The
3392        <classname 
3393        docapi="net.sf.basedb.util.extensions">RethrowErrorHandlerFactory</classname>
3394        error handler can be used to re-throw exceptions which usually means that
3395        they trickle up to the gui and are shown to the user. It is also
3396        possible for an extension point to provide its own implementation of
3397        an <interfacename docapi="net.sf.basedb.util.extensions">ErrorHandlerFactory</interfacename>.
3398      </para>
3399   
3400    </sect2>
3401   
3402    <sect2 id="extensions_api.web">
3403      <title>The web client part</title>
3404   
3405      <para>
3406        The web client specific parts of the Extensions API can be found
3407        in the <package>net.sf.basedb.client.web.extensions</package> package
3408        and it's subpackages. The top-level package contains classes used to
3409        administrate the extension system. Here is for example the
3410        <classname docapi="net.sf.basedb.client.web.extensions">ExtensionsControl</classname> 
3411        class which is the master controller for the web client extensions. It:
3412      </para>
3413     
3414      <itemizedlist>
3415      <listitem>
3416        <para>
3417        Keeps track of installed extensions and which JAR or XML file they are
3418        installed from.
3419        </para>
3420      </listitem>
3421     
3422      <listitem>
3423        <para>
3424        Can, manually or automatically, find and install new or
3425        updated extensions and uninstall deleted extensions.
3426        </para>
3427      </listitem>
3428     
3429      <listitem>
3430        <para>
3431        Adds permission control to the extension system, so that only an
3432        administrator is allowed to change settings, enable/disable extensions,
3433        etc.
3434        </para>
3435      </listitem>
3436      </itemizedlist>
3437     
3438      <para>
3439        In the top-level package there are also some abstract classes that may
3440        be useful to extend for developers creating their own extensions.
3441        For example, we recommend that all action factories extend the <classname 
3442        docapi="net.sf.basedb.client.web.extensions">AbstractJspActionFactory</classname>
3443        class.
3444      </para>
3445     
3446      <para>
3447        The sub-packages to <package>net.sf.basedb.client.web.extensions</package>
3448        are mostly specific to a single extension point or to a specific type of
3449        extension point. The <package>net.sf.basedb.client.web.extensions.menu</package>
3450        package, for example, contains classes that are/can be used for extensions
3451        adding menu items to the <menuchoice><guimenu>Extensions</guimenu></menuchoice>
3452        menu.
3453      </para>
3454     
3455      <figure id="extensions_api.figures.web">
3456        <title>The web client part of the Extensions API</title>
3457        <screenshot>
3458          <mediaobject>
3459            <imageobject>
3460              <imagedata 
3461                align="center"
3462                fileref="figures/uml/corelayer.extensions_web.png" format="PNG" />
3463            </imageobject>
3464          </mediaobject>
3465        </screenshot>
3466      </figure>
3467   
3468      <para>
3469        When the Tomcat web server is starting up, the <classname 
3470        docapi="net.sf.basedb.client.web.extensions">ExtensionsServlet</classname>
3471        is automatically loaded. This servlet has as two purposes:
3472      </para>
3473     
3474      <itemizedlist>
3475      <listitem>
3476        <para>
3477        Initialise the extensions system by calling
3478        <methodname>ExtensionsControl.init()</methodname>. This will result in
3479        an initial scan for installed extensions, which is equivalent to doing
3480        a manual scan with the force update setting to false. This means that
3481        the extension system is up an running as soon as the first user log's
3482        in to BASE.
3483        </para>
3484      </listitem>
3485     
3486      <listitem>
3487        <para>
3488        Act as a proxy for custom servlets defined by the extensions. URL:s
3489        ending with <code>.servlet</code> has been mapped to the
3490        <classname>ExtensionsServlet</classname>. When a request is made it
3491        will extract the name of the extension's JAR file from the
3492        URL, get the corresponding <classname 
3493        docapi="net.sf.basedb.client.web.extensions">ExtensionsFile</classname>
3494        and <classname docapi="net.sf.basedb.client.web.extensions">ServletWrapper</classname>
3495        and then invoke the custom servlet. More information can be found in
3496        <xref linkend="extensions_developer.servlets" />.
3497        </para>
3498      </listitem>
3499     
3500      </itemizedlist>
3501     
3502      <para>
3503        Using extensions only involves calling the
3504        <methodname>ExtensionsControl.createContext()</methodname> and
3505        <methodname>ExtensionsControl.useExtensions()</methodname> methods. This
3506        returns an <classname docapi="net.sf.basedb.util.extensions">ExtensionsInvoker</classname> 
3507        object as described in the previous section.
3508      </para>
3509     
3510      <para>
3511        To render the actions it is possible to either use the
3512        <methodname>ExtensionsInvoker.iterate()</methodname> method
3513        and generate HTML from the information in each action. Or
3514        (the better way) is to use a renderer together with the
3515        <classname docapi="net.sf.basedb.clients.web.taglib.extensions">Render</classname>
3516        taglib.
3517      </para>
3518     
3519      <para>
3520        To get information about the installed extensions, 
3521        change settings, enabled/disable extensions, performing a manual
3522        scan, etc. use the <methodname>ExtensionsControl.get()</methodname>
3523        method. This will create a permission-controlled object. All
3524        users has read permission, administrators has write permission.
3525      </para>
3526     
3527      <note>
3528        <para>
3529          The permission we check for is WRITE permission on the
3530          web client item. This means it is possible to give a user
3531          permissions to manage the extension system by assigning
3532          WRITE permission to the web client entry in the database.
3533          Do this from <menuchoice>
3534            <guimenu>Administrate</guimenu>
3535            <guimenuitem>Clients</guimenuitem>
3536          </menuchoice>.
3537        </para>
3538      </note>
3539   
3540      <para>
3541        The <classname docapi="net.sf.basedb.clients.web.extensions">XJspCompiler</classname>
3542        is mapped to handle the compilation <code>.xjsp</code> files
3543        which are regular JSP files with a different extension. This feature is
3544        experimental and requires installing an extra JAR into Tomcat's lib
3545        directory. See <xref linkend="plugins.installation.xjspcompiler" /> for
3546        more information.
3547      </para>
3548   
3549    </sect2>
3550   
3551  </sect1>
3552
3553  <sect1 id="base_api.other">
3554    <title>Other useful classes and methods</title>
3555    <para>
3556      TODO
3557    </para>
3558  </sect1>
3559 
3560</chapter>
Note: See TracBrowser for help on using the repository browser.