source: trunk/doc/src/docbook/developerdoc/api_overview.xml @ 4076

Last change on this file since 4076 was 4076, checked in by Martin Svensson, 14 years ago

Fixes #879 Add column holding the filename of last file used for updating reporter annotations. The documentation, reporter importer plugins and web client are now also updated to support this new column.

  • Property svn:eol-style set to native
  • Property svn:keywords set to Id
File size: 103.0 KB
Line 
1<?xml version="1.0" encoding="UTF-8"?>
2<!DOCTYPE chapter PUBLIC
3    "-//Dawid Weiss//DTD DocBook V3.1-Based Extension for XML and graphics inclusion//EN"
4    "../../../../lib/docbook/preprocess/dweiss-docbook-extensions.dtd">
5<!--
6  $Id: api_overview.xml 4076 2008-01-11 15:16:30Z martin $
7
8  Copyright (C) 2007 Peter Johansson, Nicklas Nordborg, Martin Svensson
9
10  This file is part of BASE - BioArray Software Environment.
11  Available at http://base.thep.lu.se/
12
13  BASE is free software; you can redistribute it and/or
14  modify it under the terms of the GNU General Public License
15  as published by the Free Software Foundation; either version 2
16  of the License, or (at your option) any later version.
17
18  BASE is distributed in the hope that it will be useful,
19  but WITHOUT ANY WARRANTY; without even the implied warranty of
20  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
21  GNU General Public License for more details.
22
23  You should have received a copy of the GNU General Public License
24  along with this program; if not, write to the Free Software
25  Foundation, Inc., 59 Temple Place - Suite 330,
26  Boston, MA  02111-1307, USA.
27-->
28
29<chapter id="api_overview">
30  <?dbhtml dir="api"?>
31  <title>API overview (how to use and code examples)</title>
32
33  <sect1 id="api_overview.public_api">
34    <title>The Public API of BASE</title>
35   
36    <para>
37      Not all public classes and methods in the <filename>BASE2Core.jar</filename>
38      and other JAR files shipped with BASE are considered as
39      <emphasis>Public API</emphasis>. This is important knowledge
40      since we will always try to maintain backwards compatibility
41      for classes that are part of the public API. For other
42      classes, changes may be introduced at any time without
43      notice or specific documentation. In other words:
44    </para>
45   
46    <note>
47      <title>Only use the public API when developing plug-ins</title>
48      <para>
49        This will maximize the chance that you plug-in will continue
50        to work with the next BASE release. If you use the non-public API
51        you do so at your own risk.
52      </para>
53    </note>
54   
55    <para>
56      See the <ulink url="http://base.thep.lu.se/chrome/site/doc/api/index.html"
57        >javadoc</ulink> for information about
58      what parts of the API that contributes to the public API.
59      Methods, classes and other elements that have been tagged as
60      <code>@deprecated</code> should be considered as part of the internal API
61      and may be removed in a subsequent release without warning.
62    </para>
63   
64    <para>
65      See <xref linkend="appendix.incompatible" /> to read more about
66      changes that have been introduced by each release.
67    </para>
68
69    <sect2 id="api_overview.compatibility">
70      <title>What is backwards compatibility?</title>
71     
72      <para>
73        There is a great article about this subject on <ulink 
74        url="http://wiki.eclipse.org/index.php/Evolving_Java-based_APIs"
75          >http://wiki.eclipse.org/index.php/Evolving_Java-based_APIs</ulink>.
76        This is what we will try to comply with. If you do not want to
77        read the entire article, here are some of the most important points:
78      </para>
79     
80     
81      <sect3 id="api_overview.compatibility.binary">
82        <title>Binary compatibility</title>
83        <para>
84        <blockquote>
85          Pre-existing Client binaries must link and run with new releases of the
86          Component without recompiling.
87        </blockquote>
88       
89        For example:
90        <itemizedlist>
91        <listitem>
92          <para>
93            We cannot change the number or types of parameters to a method
94            or constructor.
95          </para>
96        </listitem>
97        <listitem>
98          <para>
99            We cannot add or change methods to interfaces that are intended
100            to be implemented by plug-in or client code.
101          </para>
102        </listitem>
103        </itemizedlist>
104        </para>       
105      </sect3>
106     
107      <sect3 id="api_overview.compatibility.contract">
108        <title>Contract compatibility</title>
109        <para>
110          <blockquote>
111          API changes must not invalidate formerly legal Client code.
112          </blockquote>
113       
114          For example:
115          <itemizedlist>
116          <listitem>
117            <para>
118              We cannot change the implementation of a method to do
119              things differently than before. For example, allow <constant>null</constant>
120              as a return value when it was not allowed before.
121            </para>
122          </listitem>
123          </itemizedlist>
124       
125          <note>
126            <para>
127            Sometimes there is a very fine line between what is considered a
128            bug and what is considered a feature. For example, if the
129            actual implementation does not do what the javadoc says,
130            do we change the code or do we change the documentation?
131            This has to be considered from case to case and depends on
132            the age of the code and if we expect plug-ins and clients to be
133            affected by it or not.
134            </para>
135          </note>
136        </para>
137      </sect3>
138     
139      <sect3 id="api_overview.compatibility.source">
140        <title>Source code compatibility</title>
141        <para>
142        This is not an important matter and is not always possible to
143        achieve. In most cases, the problems are easy to fix.
144        Example:
145       
146        <itemizedlist>
147        <listitem>
148          <para>
149          Adding a class may break a plug-in or client that import
150          classes with <constant>.*</constant> if the same class name
151          exists in another package.
152          </para>
153        </listitem>
154        </itemizedlist>
155        </para>
156      </sect3>
157    </sect2>
158  </sect1>
159
160  <sect1 id="api_overview.data_api" chunked="1">
161    <title>The database schema and the Data Layer API</title>
162
163    <para>
164      This section gives an overview of the entire data layer API.
165      The figure below show how different modules relate to each other.
166    </para>
167   
168    <note>
169      All information has not yet been transfered from the old documentation.
170      The old documentation is available at
171      <ulink url="http://base.thep.lu.se/chrome/site/doc/development/overview/data/index.html"
172        >http://base.thep.lu.se/chrome/site/doc/development/overview/data/index.html</ulink>
173    </note>
174   
175    <figure id="data_api.figures.overview">
176      <title>Data layer overview</title>
177      <screenshot>
178        <mediaobject>
179          <imageobject>
180            <imagedata 
181              align="center"
182              scalefit="1" width="100%"
183              fileref="figures/uml/datalayer.overview.png" format="PNG" />
184          </imageobject>
185        </mediaobject>
186      </screenshot>
187    </figure>
188
189    <sect2 id="data_api.basic">
190      <title>Basic classes and interfaces</title>
191     
192      <para>
193        This document contains information about the basic classes and interfaces in this package.
194        They are important since all data-layer classes must inherit from one of the already
195        existing abstract base classes or implement one or more of the
196        existing interfaces. They contain code that is common to all classes,
197        for example implementations of the <methodname>equals()</methodname>
198        and <methodname>hashCode()</methodname> methods or how to link with the owner of an
199        item.
200      </para>
201     
202      <sect3 id="data_api.basic.uml">
203        <title>UML diagram</title>
204       
205        <figure id="data_api.figures.basic">
206          <title>Basic classes and interfaces</title>
207          <screenshot>
208            <mediaobject>
209              <imageobject>
210                <imagedata 
211                  align="center"
212                  fileref="figures/uml/datalayer.basic.png" format="PNG" />
213              </imageobject>
214            </mediaobject>
215          </screenshot>
216        </figure>
217      </sect3>
218     
219      <sect3 id="data_api.basic.classes">
220        <title>Classes</title>
221       
222        <variablelist>
223        <varlistentry>
224          <term><classname docapi="net.sf.basedb.core.data">BasicData</classname></term>
225          <listitem>
226            <para>
227            The root class. It overrides the <methodname>equals()</methodname>,
228            <methodname>hashCode()</methodname> and <methodname>toString()</methodname> methods
229            from the <classname>Object</classname> class. It also defines the
230            <varname>id</varname> and <varname>version</varname> properties.
231            All data layer classes must inherit from this class or one of it's subclasses.
232            </para>
233          </listitem>
234        </varlistentry>
235       
236        <varlistentry>
237          <term><classname docapi="net.sf.basedb.core.data">OwnedData</classname></term>
238          <listitem>
239            <para>
240            Extends the <classname>BasicData</classname> class and adds
241            an <varname>owner</varname> property. The owner is a required link to a
242            <classname docapi="net.sf.basedb.core.data">UserData</classname> object, representing the user that
243            is the owner of the item.
244            </para>
245          </listitem>
246        </varlistentry>
247
248        <varlistentry>
249          <term><classname docapi="net.sf.basedb.core.data">SharedData</classname></term>
250          <listitem>
251            <para>
252            Extends the <classname>OwnedData</classname> class and adds
253            properties (<varname>itemKey</varname> and <varname>projectKey</varname>)
254            that holds access permission information for an item.
255            Access permissions are held in <classname docapi="net.sf.basedb.core.data">ItemKeyData</classname> and/or
256            <classname docapi="net.sf.basedb.core.data">ProjectKeyData</classname> objects. These objects only exists if
257            the item has been shared.
258            </para>
259          </listitem>
260        </varlistentry>
261
262        <varlistentry>
263          <term><classname docapi="net.sf.basedb.core.data">CommonData</classname></term>
264          <listitem>
265            <para>
266            This is a convenience class for items that extends the <classname>SharedData</classname>
267            class and implements the <interfacename docapi="net.sf.basedb.core.data">NameableData</interfacename> and
268            <interfacename docapi="net.sf.basedb.core.data">RemoveableData</interfacename> interfaces. This is one of
269            the most common situations.
270            </para>
271          </listitem>
272        </varlistentry>
273
274        <varlistentry>
275          <term><classname docapi="net.sf.basedb.core.data">AnnotatedData</classname></term>
276          <listitem>
277            <para>
278            This is a convenience class for items that can be annotated.
279            Annotations are held in <classname docapi="net.sf.basedb.core.data">AnnotationSetData</classname> objects.
280            The annotation set only exists if annotations has been created for the item.
281            </para>
282          </listitem>
283        </varlistentry>
284        </variablelist>
285       
286      </sect3>
287     
288      <sect3 id="data_api.basic.interfaces">
289        <title>Interfaces</title>
290       
291        <variablelist>
292        <varlistentry>
293          <term><classname docapi="net.sf.basedb.core.data">IdentifiableData</classname></term>
294          <listitem>
295            <para>
296            All items are identifiable, which means that they have a unique <varname>id</varname>.
297            The id is unique for all items of a specific type (ie. class). The id is number
298            that is automatically generated by the database and has no other meaning
299            outside of the application. The <varname>version</varname> property is used for
300            detecting and preventing concurrent modifications to an item.
301            </para>
302          </listitem>
303        </varlistentry>
304       
305        <varlistentry>
306          <term><classname docapi="net.sf.basedb.core.data">OwnableData</classname></term>
307          <listitem>
308            <para>
309            An ownable item is an item which has an owner. The owner is represented as a
310            required link to a <classname docapi="net.sf.basedb.core.data">UserData</classname> object.
311            </para>
312          </listitem>
313        </varlistentry>       
314
315        <varlistentry>
316          <term><classname docapi="net.sf.basedb.core.data">ShareableData</classname></term>
317          <listitem>
318            <para>
319            A shareable item is an item which can be shared to other users, groups or projects.
320            Access permissions are held in <classname docapi="net.sf.basedb.core.data">ItemKeyData</classname> and/or
321            <classname docapi="net.sf.basedb.core.data">ProjectKeyData</classname> objects.
322            </para>
323          </listitem>
324        </varlistentry>
325             
326        <varlistentry>
327          <term><classname docapi="net.sf.basedb.core.data">NameableData</classname></term>
328          <listitem>
329            <para>
330            A nameable item is an item that has a name (required) and a description
331            (optional). The name doesn't have to be unique, except in a few special
332            cases (for example, the name of a file).
333            </para>
334          </listitem>
335        </varlistentry>
336       
337        <varlistentry>
338          <term><classname docapi="net.sf.basedb.core.data">RemovableData</classname></term>
339          <listitem>
340            <para>
341            A removable item is an item that can be flagged as removed. This doesn't
342            remove the information about the item from the database, but can be used by
343            client applications to hide items that the user is not interested in.
344            A trashcan function can be used to either restore or permanently
345            remove items that has the flag set.
346            </para>
347          </listitem>
348        </varlistentry>
349               
350        <varlistentry>
351          <term><classname docapi="net.sf.basedb.core.data">SystemData</classname></term>
352          <listitem>
353            <para>
354            A system item is an item which has an additional id in the form of string. A system id
355            is required when we need to make sure that we can get a specific item without
356            knowing the numeric id. Example of such items are the root user and the everyone group.
357            A system id is generally constructed like:
358            <constant>net.sf.basedb.core.User.ROOT</constant>. The system id:s are defined in the
359            core layer by each item class.
360            </para>
361          </listitem>
362        </varlistentry>
363
364        <varlistentry>
365          <term><classname docapi="net.sf.basedb.core.data">DiskConsumableData</classname></term>
366          <listitem>
367            <para>
368            This interface is used by items which occupies a lot of disk space and
369            should be part of the quota system, for example files. The required
370            <classname docapi="net.sf.basedb.core.data">DiskUsageData</classname> contains information about the size,
371            location, owner etc. of the item.
372            </para>
373          </listitem>
374        </varlistentry>
375       
376        <varlistentry>
377          <term><classname docapi="net.sf.basedb.core.data">AnnotatableData</classname></term>
378          <listitem>
379            <para>
380            This interface is used by items which can be annotated. Annotations are name/value
381            pairs that are attached as extra information to an item. All annotations are
382            contained in an <classname docapi="net.sf.basedb.core.data">AnnotationSetData</classname> object.
383            </para>
384          </listitem>
385        </varlistentry>
386       
387        <varlistentry>
388          <term><classname docapi="net.sf.basedb.core.data">ExtendableData</classname></term>
389          <listitem>
390            <para>
391            This interface is used by items which can have extra administrator-defined
392            columns. The functionality is similar to annotations. It is not as flexible,
393            since it is a global configuration, but has better performance. BASE will
394            generate extra database columns to store the data in the tables for items that
395            can be extended.
396            </para>
397          </listitem>
398        </varlistentry>
399       
400        <varlistentry>
401          <term><classname docapi="net.sf.basedb.core.data">BatchableData</classname></term>
402          <listitem>
403            <para>
404            This interface is a tagging interface which is used by items that needs batch
405            functionality in the core.
406            </para>
407          </listitem>
408        </varlistentry>
409        </variablelist>
410
411      </sect3>
412    </sect2>
413   
414    <sect2 id="data_api.authentication">
415      <title>User authentication and access control</title>
416     
417      <para>
418         This section gives an overview of user authentication and
419         how groups, roles and projects are used for access control
420         to items.
421      </para>
422     
423      <sect3 id="data_api.authentication.uml">
424        <title>UML diagram</title>
425       
426        <figure id="data_api.figures.authentication">
427          <title>User authentication and access control</title>
428          <screenshot>
429            <mediaobject>
430              <imageobject>
431                <imagedata 
432                  align="center"
433                  scalefit="1" width="100%"
434                  fileref="figures/uml/datalayer.authentication.png" format="PNG" />
435              </imageobject>
436            </mediaobject>
437          </screenshot>
438        </figure>
439      </sect3>
440     
441      <sect3 id="data_api.authentication.users">
442        <title>Users and passwords</title>     
443     
444        <para>
445          The <classname docapi="net.sf.basedb.core.data">UserData</classname> class holds information about users.
446          We keep the passwords in a separate table and use proxies to avoid loading
447          password data each time a user is loaded to minimize security risks. It is
448          only if the password needs to be changed that the <classname docapi="net.sf.basedb.core.data">PasswordData</classname>
449          object is loaded. The one-to-one mapping between user and password is controlled
450          by the password class, but a cascade attribute on the user class makes sure
451          that the password is deleted when a user is deleted.
452        </para>
453      </sect3>
454
455      <sect3 id="data_api.authentication.groups">
456        <title>Groups, roles and projects</title>     
457     
458        <para>
459          The <classname docapi="net.sf.basedb.core.data">GroupData</classname>, <classname docapi="net.sf.basedb.core.data">RoleData</classname> and
460          <classname docapi="net.sf.basedb.core.data">ProjectData</classname> classes holds information about groups, roles
461          and projects respectively. A user may be a member of any number of groups,
462          roles and/or projects. The membership in a project comes with an attached
463          permission values. This is the highest permission the user has in the
464          project. No matter what permission an item has been shared with the
465          user will not get higher permission. Groups may be members of other groups and
466          also in projects.
467        </para>
468       
469      </sect3>
470     
471      <sect3 id="data_api.authentication.keys">
472        <title>Keys</title>     
473     
474        <para>
475          The <classname docapi="net.sf.basedb.core.data">KeyData</classname> class and it's subclasses
476          <classname docapi="net.sf.basedb.core.data">ItemKeyData</classname>, <classname docapi="net.sf.basedb.core.data">ProjectKeyData</classname> and
477          <classname docapi="net.sf.basedb.core.data">RoleKeyData</classname>, are used to store information about access
478          permissions to items. To get permission to manipulate an item a user must have
479          access to a key giving that permission. There are three types of keys:
480        </para>
481       
482        <variablelist>
483        <varlistentry>
484          <term><classname docapi="net.sf.basedb.core.data">ItemKey</classname></term>
485          <listitem>
486            <para>
487            Is used to give a user or group access to a specific item. The item
488            must be a <interfacename docapi="net.sf.basedb.core.data">ShareableData</interfacename> item.
489            The permissions are usually set be the owner of the item. Once created an
490            item key cannot be changed. This allows the core to reuse a key if the
491            permissions match exactly, ie. for a given set of users/groups/permissions
492            there can be only one item key object.
493            </para>
494          </listitem>
495        </varlistentry>
496
497        <varlistentry>
498          <term><classname docapi="net.sf.basedb.core.data">ProjectKey</classname></term>
499          <listitem>
500            <para>
501            Is used to give members of a project access to a specific item. The item
502            must be a <interfacename docapi="net.sf.basedb.core.data">ShareableData</interfacename> item. Once created a
503            project key cannot be changed. This allows the core to reuse a key if the
504            permissions match exactly, ie. for a given set of projects/permissions
505            there can be only one project key object.
506            </para>
507          </listitem>
508        </varlistentry>
509
510        <varlistentry>
511          <term><classname docapi="net.sf.basedb.core.data">RoleKey</classname></term>
512          <listitem>
513            <para>
514            Is used to give a user access to all items of a specific type, ie.
515            <constant>READ</constant> all <constant>SAMPLES</constant>. The installation
516            will make sure that there already exists a role key for each type of item, and
517            it is not possible to add new or delete existing keys. Unlike the other two types
518            this key can be modified.
519            </para>
520           
521            <para>
522            A role key is also used to assign permissions to plug-ins. If a plug-in has
523            been specified to use permissions the default is to deny everything.
524            The mapping to the role key is used to grant permissions to the plugin.
525            The <varname>granted</varname> value gives the plugin access to all items
526            of the related item type regardless of if the user that is running the plug-in has the
527            permission or not. The <varname>denied</varname> values denies access to all
528            items of the related item type even if the logged in user has the permission.
529            Permissions that are not granted nor denied are checked against the
530            logged in users regular permissions. Permissions to items that are
531            not linked are always denied.
532            </para>
533          </listitem>
534        </varlistentry>
535        </variablelist>
536       
537      </sect3>
538
539      <sect3 id="data_api.authentication.permissions">
540        <title>Permissions</title>
541       
542        <para>
543          The <varname>permission</varname> property appearing in many classes is an
544          integer values describing the permission:
545        </para>
546       
547        <informaltable>
548        <tgroup cols="2">
549          <colspec colname="value" />
550          <colspec colname="permission" />
551          <thead>
552            <row>
553              <entry>Value</entry>
554              <entry>Permission</entry>
555            </row>
556          </thead>
557          <tbody>
558            <row>
559              <entry>1</entry>
560              <entry>Read</entry>
561            </row>
562            <row>
563              <entry>3</entry>
564              <entry>Use</entry>
565            </row>
566            <row>
567              <entry>7</entry>
568              <entry>Restricted write</entry>
569            </row>
570            <row>
571              <entry>15</entry>
572              <entry>Write</entry>
573            </row>
574            <row>
575              <entry>31</entry>
576              <entry>Delete</entry>
577            </row>
578            <row>
579              <entry>47 (=32+15)</entry>
580              <entry>Set owner</entry>
581            </row>
582            <row>
583              <entry>79 (=64+15)</entry>
584              <entry>Set permissions</entry>
585            </row>
586            <row>
587              <entry>128</entry>
588              <entry>Create</entry>
589            </row>
590            <row>
591              <entry>256</entry>
592              <entry>Denied</entry>
593            </row>
594          </tbody>
595        </tgroup>
596        </informaltable>
597       
598        <para>
599          The values are constructed so that
600          <constant>READ</constant> -&gt;
601          <constant>USE</constant> -&gt;
602          <constant>RESTRICTED_WRITE</constant> -&gt;
603          <constant>WRITE</constant> -&gt;
604          <constant>DELETE</constant>
605          are chained in the sense that a higher permission always implies the lower permissions
606          also. The <constant>SET_OWNER</constant> and <constant>SET_PERMISSION</constant>
607          both implies <constant>WRITE</constant> permission. The <constant>DENIED</constant>
608          permission is only valid for role keys, and if specified it overrides all
609          other permissions.               
610        </para>
611       
612        <para>
613          When combining permission for a single item the permission codes for the different
614          paths are OR-ed together. For example a user has a role key with <constant>READ</constant>
615          permission for <constant>SAMPLES</constant>, but also an item key with <constant>USE</constant>
616          permission for a specific sample. Of course, the resulting permission for that
617          sample is <constant>USE</constant>. For other samples the resulting permission is
618          <constant>READ</constant>.
619        </para>
620       
621        <para>
622          If the user is also a member of a project which has <constant>WRITE</constant>
623          permission for the same sample, the user will have <constant>WRITE</constant>
624          permission when working with that project.
625        </para>
626       
627        <para>
628          The <constant>RESTRICTED_WRITE</constant> permission is in most cases the same
629          as the <constant>WRITE</constant> permission. So far the <constant>RESTRICTED_WRITE</constant>
630          permission is only given to users to their own <classname docapi="net.sf.basedb.core.data">UserData</classname>
631          object so they can change their address and other contact information,
632          but not quota, expiration date and other administrative information.
633        </para>
634
635      </sect3>
636    </sect2>
637
638    <sect2 id="data_api.wares">
639      <title>Hardware and software</title>
640      <para>
641         This section gives an overview of hardware and software in BASE.
642      </para>
643     
644      <sect3 id="data_api.wares.uml">
645        <title>UML diagram</title>
646       
647        <figure id="data_api.figures.wares">
648          <title>Hardware and software</title>
649          <screenshot>
650            <mediaobject>
651              <imageobject>
652                <imagedata 
653                  align="center"
654                  fileref="figures/uml/datalayer.wares.png" format="PNG" />
655              </imageobject>
656            </mediaobject>
657          </screenshot>
658        </figure>
659      </sect3>
660     
661      <sect3 id="data_api.wares.description">
662        <title>Hardware and software</title>
663        <para>
664          BASE is pre-installed with a set of hardware and software types.
665          They are typically used to filter the registered hardware and software
666          depending on what a user is doing. For example, when adding raw data
667          to BASE a user can select a scanner. The GUI will display the hardware
668          that has been registered as <emphasis>scanner</emphasis> hardware types.
669          Other hardware types are <emphasis>hybridization station</emphasis>
670          and <emphasis>print robot</emphasis>. An administrator may register more
671          hardware and software types.
672        </para>
673      </sect3>
674    </sect2>
675   
676    <sect2 id="data_api.reporters">
677      <title>Reporters</title>
678      <para>
679         This section gives an overview of hardware and software in BASE.
680      </para>
681     
682      <sect3 id="data_api.reporters.uml">
683        <title>UML diagram</title>
684       
685        <figure id="data_api.figures.reporters">
686          <title>Reporters</title>
687          <screenshot>
688            <mediaobject>
689              <imageobject>
690                <imagedata 
691                  align="center"
692                  fileref="figures/uml/datalayer.reporters.png" format="PNG" />
693              </imageobject>
694            </mediaobject>
695          </screenshot>
696        </figure>
697      </sect3>
698     
699      <sect3 id="data_api.reporters.description">
700        <title>Reporters</title>
701        <para>
702          The <classname docapi="net.sf.basedb.core.data">ReporterData</classname> class holds information about reporters.
703          The <property>externalId</property> is a required property that must be unique
704          among all reporters. The external ID is the value BASE uses to match
705          reporters when importing data from files.
706        </para>
707       
708        <para>
709          The <classname>ReporterData</classname> is an <emphasis>extendable</emphasis>
710          class, which means that the server administrator can define additional
711          columns (=annotations) in the reporters table. These are accessed with
712          the <methodname>ReporterData.getExtended()</methodname> and
713          <methodname>ReporterData.setExtended()</methodname> methods.
714          See <xref linkend="appendix.extendedproperties" /> for more information about
715          this.
716        </para>
717       
718        <para>
719          The <classname>ReporterData</classname> is also a <emphasis>batchable</emphasis>
720          class which means that there is no corresponding class in the core
721          layer. Client applications and plug-ins should work directly with
722          the <classname>ReporterData</classname> class. To help manage the reporters
723          there is the <classname docapi="net.sf.basedb.core">Reporter</classname> and <classname docapi="net.sf.basedb.core">ReporterBatcher</classname>
724          classes. The main reason for this
725          is to increase the performance and lower the memory usage by bypassing
726          internal caching in the core and Hibernate. Performance is also
727          increased by the batchers which uses more efficient SQL against the
728          database than Hibernate.
729        </para>
730       
731        <para>
732          The
733          <property>lastUpdate</property>
734          property holds the data and time the reporter information was last updated. The
735          value is managed automatically by the
736          <classname>ReporterBatcher</classname>
737          class. That goes for
738          <property>lastSource</property>
739          property too, which holds information about where the last update comes from. By
740          default this is set to the name of the logged in user, but it can be changed by
741          calling
742          <methodname>ReporterBatcher.setUpdateSource(String source)</methodname>
743          before the batcher commits the updates to the database. The source-string
744          should have the format: <synopsis>[ITEM_TYPE]:[ITEM_NAME]</synopsis> where,in
745          the file-case, ITEM_TYPE is File and ITEM_NAME is the file's name.
746        </para>
747      </sect3>
748     
749      <sect3 id="data_api.reporters.lists">
750        <title>Reporter lists</title>
751       
752        <para>
753          Reporter lists can be used to group reporters that are somehow related
754          to each other. This could for example be a list of interesting reporters
755          found in the analysis of an experiment. Each reporter in the list may
756          optionally be assigned a score. The meaning of the score value is not
757          interpreted by BASE.
758        </para>
759       
760      </sect3>
761     
762     
763    </sect2>
764
765    <sect2 id="data_api.quota">
766      <title>Quota and disk usage</title>
767      <para>
768         This section gives an overview of quota system in BASE
769         and how the disk usage is kept track of.
770      </para>
771     
772      <sect3 id="data_api.quota.uml">
773        <title>UML diagram</title>
774       
775        <figure id="data_api.figures.quota">
776          <title>Quota and disk usage</title>
777          <screenshot>
778            <mediaobject>
779              <imageobject>
780                <imagedata 
781                  align="center"
782                  fileref="figures/uml/datalayer.quota.png" format="PNG" />
783              </imageobject>
784            </mediaobject>
785          </screenshot>
786        </figure>
787      </sect3>
788     
789      <sect3 id="data_api.quota.description">
790        <title>Quota</title>
791       
792        <para>
793          The <classname docapi="net.sf.basedb.core.data">QuotaData</classname> holds information about a
794          single quota registration. The same quota may be used by many different users
795          and groups. This object encapsulates allowed
796          quota values for different types of quota types and locations.
797          BASE defines several quota types (file, raw data and experiment),
798          and locations (primary, secondary and offline).
799        </para>
800       
801        <para>
802          The <property>quotaValues</property> property is a map from
803          <classname docapi="net.sf.basedb.core.data">QuotaIndex</classname> to maximum byte values.
804          This map must contain at least one entry for the total
805          quota at the primary location.
806        </para>
807       
808      </sect3>
809     
810      <sect3 id="data_api.quota.diskusage">
811        <title>Disk usage</title>
812       
813        <para>
814          A <interfacename docapi="net.sf.basedb.core.data">DiskConsumableData</interfacename> (for example a file)
815          item is automatically linked to a <classname docapi="net.sf.basedb.core.data">DiskUsageData</classname>
816          item. This holds information about the number of bytes,
817          the location and quota type the item uses. It also holds information
818          about which user and group (optional) that should be charged for the disk usage.
819          The user is always the owner of the item.
820        </para>
821
822      </sect3>
823     
824    </sect2>
825
826    <sect2 id="data_api.clients">
827      <title>Client, session and settings</title>
828      <para>
829         This section gives an overview of hardware and software in BASE.
830      </para>
831     
832      <sect3 id="data_api.clients.uml">
833        <title>UML diagram</title>
834       
835        <figure id="data_api.figures.clients">
836          <title>Client, sessions and settings</title>
837          <screenshot>
838            <mediaobject>
839              <imageobject>
840                <imagedata 
841                  align="center"
842                  scalefit="1" width="100%"
843                  fileref="figures/uml/datalayer.clients.png" format="PNG" />
844              </imageobject>
845            </mediaobject>
846          </screenshot>
847        </figure>
848      </sect3>
849     
850      <sect3 id="data_api.clients.description">
851        <title>Clients</title>
852        <para>
853          The <classname docapi="net.sf.basedb.core.data">ClientData</classname> class holds information
854          about a client application. The <property>externalId</property>
855          property is a unique identifier for the application. To avoid ID clashes the ID
856          should be constructed in the same way as Java packages, for example
857          <constant>net.sf.basedb.clients.web</constant> is the ID for the
858          web client application.
859        </para>
860       
861        <para>
862          A client application doesn't have to be registered with BASE
863          to be able to use it. But we recommend it since:
864        </para>
865       
866        <itemizedlist>
867        <listitem>
868          <para>
869            The permission system allows an admin to specify exactly
870            which users that may use a specific application.
871          </para>
872        </listitem>
873       
874        <listitem>
875          <para>
876          The application can't store any context-sensitive or application-specific
877          settings unless it is registered.
878          </para>
879        </listitem>
880       
881        <listitem>
882          <para>
883          The application can store context-sensitive help in the BASE
884          database.
885          </para>
886        </listitem>
887        </itemizedlist>
888      </sect3>
889     
890      <sect3 id="data_api.clients.sessions">
891        <title>Sessions</title>
892       
893        <para>
894          A session represents the time between login and logout for a single
895          user. The <classname docapi="net.sf.basedb.core.data">SessionData</classname> object is entirely
896          managed by the BASE core, and should be considered read-only
897          for client applications.
898        </para>
899           
900      </sect3>
901     
902      <sect3 id="data_api.clients.settings">
903        <title>Settings</title>
904       
905        <para>
906          There are two types of settings: context-sensitive settings and regular
907          settings. The regular settings are simple key-value pairs of strings
908          and can be used for almost anything. There are four subtypes:
909        </para>
910       
911        <itemizedlist>
912        <listitem>
913          <para>
914          Global default settings: Settings that are used by all users
915          and client applications on the BASE server. These settings
916          are read-only except for administrators. BASE has not yet defined
917          any settings of this type.
918          </para>
919        </listitem>
920       
921        <listitem>
922          <para>
923          User default settings: Settings that are valid for a single user
924          for any client application. BASE has not yet defined
925          any settings of this type.
926          </para>
927        </listitem>
928       
929        <listitem>
930          <para>
931          Client default settings: Settings that are valid for all users using
932          a specific client application. Each client application is responsible
933          for defining it's own settings. Settings are read-only except
934          for administrators.
935          </para>
936        </listitem>
937       
938        <listitem>
939          <para>
940          User client settings: Settings that are valid for a single user using
941          a specific client application. Each client application is responsible
942          for defining it's own settings.
943          </para>
944        </listitem>
945       
946        </itemizedlist>
947       
948        <para>
949          The context-sensitive settings are designed to hold information
950          about the current status of options related to the listing of items
951          of a specific type. This includes:
952        </para>
953       
954        <itemizedlist>
955        <listitem>
956          <para>
957          Current filtering options (as 1 or more <classname docapi="net.sf.basedb.core.data">PropertyFilterData</classname>
958          objects).
959          </para>
960        </listitem>
961       
962        <listitem>
963          <para>
964          Which columns and direction to use for sorting.
965          </para>
966        </listitem>
967       
968        <listitem>
969          <para>
970          The number of items to display on each page, and which page that
971          is the current page.
972          </para>
973        </listitem>
974       
975        <listitem>
976          <para>
977          Simple key-value settings related to a given context.
978          </para>
979        </listitem>
980        </itemizedlist>
981       
982        <para>
983          Context-sensitive settings are only accessible if a client
984          application has been registered. The settings may be
985          named to make it possible to store several presets and to
986          quickly switch between them. In any case, BASE maintains a
987          current default setting with an empty name. An administrator
988          may mark a named setting as public to allow other users to
989          use it.
990        </para>
991       
992      </sect3>
993     
994     
995    </sect2>
996
997    <sect2 id="data_api.files">
998      <title>Files and directories</title>
999
1000      <para>
1001        This section covers the details of the BASE file
1002        system.
1003      </para>
1004
1005      <sect3 id="data_api.files.uml">
1006      <title>UML diagram</title>
1007     
1008        <figure id="data_api.figures.files">
1009          <title>Files and directories</title>
1010          <screenshot>
1011            <mediaobject>
1012              <imageobject>
1013                <imagedata 
1014                  align="center"
1015                  fileref="figures/uml/datalayer.files.png" format="PNG" />
1016              </imageobject>
1017            </mediaobject>
1018          </screenshot>
1019        </figure>
1020      </sect3>
1021     
1022      <sect3 id="data_api.files.description">
1023        <title>Description</title>
1024       
1025        <para>
1026          The <classname docapi="net.sf.basedb.core.data">DirectoryData</classname> class holds
1027          information about directories. Directories are organised in the
1028          ususal way as as tree structure. All directories must have
1029          a parent directory, except the system-defined root directory.
1030        </para>
1031       
1032        <para>
1033          The <classname docapi="net.sf.basedb.core.data">FileData</classname> class holds information about
1034          a file. The actual file contents is stored on disk in the directory
1035          specified by the <varname>userfiles</varname> setting in
1036          <filename>base.config</filename>. The <varname>internalName</varname>
1037          property is the name of the file on disk, but this is never exposed to
1038          client applications. The filenames and directories
1039          on the disk doesn't correspond to the the filenames and directories in
1040          BASE.
1041        </para>
1042       
1043        <para>
1044          The <varname>location</varname> property can take three values:
1045        </para>
1046       
1047        <itemizedlist>
1048        <listitem>
1049          <para>
1050          0 = The file is offline, ie. there is no file on the disk
1051          </para>
1052        </listitem>
1053        <listitem>
1054          <para>
1055          1 = The file is in primary storage, ie. it is located on the disk
1056          and can be used by BASE
1057          </para>
1058        </listitem>
1059        <listitem>
1060          <para>
1061          2 = The file is in secondary storage, ie. it has been moved to some
1062          other place and can't be used by BASE immediately.
1063          </para>
1064        </listitem>
1065        </itemizedlist>
1066       
1067        <para>
1068          The <varname>action</varname> property controls how a file is
1069          moved between primary and seconday storage. It can have the following
1070          values:
1071        </para>
1072       
1073        <itemizedlist>
1074        <listitem>
1075          <para>
1076          0 = Do nothing
1077          </para>
1078        </listitem>
1079        <listitem>
1080          <para>
1081          1 = If the file is in secondary storage, move it back to the primary storage
1082          </para>
1083        </listitem>
1084        <listitem>
1085          <para>
1086          2 = If the file is in primary storage, move it to the secondary storage
1087          </para>
1088        </listitem>
1089        </itemizedlist>
1090       
1091        <para>
1092          The actual moving between primary and secondary storage is done by an
1093          external program. See
1094          <xref linkend="appendix.base.config.secondary" /> and
1095          <xref linkend="plugin_developer.other.secondary" /> for more information.
1096        </para>
1097     
1098        <para>
1099          The <varname>md5</varname> property can be used to check for file
1100          corruption when it is moved between primary and secondary storage or
1101          when a user re-uploads a file that has been offline.
1102        </para>
1103       
1104        <para>
1105          BASE can store files in a compressed format. This is handled internally
1106          and is not visible to client applications. The <varname>compressed</varname>
1107          and <varname>diskSize</varname> properties are used to store information
1108          about this. A file may always be compressed if the users says so, but
1109          BASE can also do this automatically if the file is uploaded
1110          to a directory with the <varname>autoCompress</varname> flag set
1111          or if the file has MIME type with the <varname>autoCompress</varname>
1112          flag set.
1113        </para>
1114       
1115        <para>
1116          The <classname docapi="net.sf.basedb.core.data">FileTypeData</classname> class holds information about
1117          file types. It is used only to make it easier for users to organise
1118          their files.
1119        </para>
1120       
1121        <para>
1122          The <classname docapi="net.sf.basedb.core.data">MimeTypeData</classname> is used to register mime types and
1123          map them to file extensions. The information is only used to lookup values
1124          when needed. Given the filename we can set the <varname>File.mimeType</varname>
1125          and <varname>File.fileType</varname> properties. The MIME type is also
1126          used to decide if a file should be stored in a compressed format or not.
1127          The extension of a MIME type must be unique. Extensions should be registered
1128          without a dot, ie <emphasis>html</emphasis>, not <emphasis>.html</emphasis>
1129        </para>
1130       
1131      </sect3>
1132     
1133     
1134    </sect2>
1135   
1136    <sect2 id="data_api.platforms">
1137      <title>Experimental platforms</title>
1138
1139      <para>
1140         This section gives an overview of experimental platforms
1141         and how they are used to enable data storage in files
1142         instead of in the database.
1143      </para>
1144     
1145      <itemizedlist>
1146        <title>See also</title>
1147        <listitem><xref linkend="core_api.data_in_files" /></listitem>
1148        <listitem><xref linkend="appendix.rawdatatypes.platforms" /></listitem>
1149        <listitem><xref linkend="plugin_developer.other.datafiles" /></listitem>
1150      </itemizedlist>
1151         
1152      <sect3 id="data_api.platforms.uml">
1153        <title>UML diagram</title>
1154       
1155        <figure id="data_api.figures.platforms">
1156          <title>Experimental platforms</title>
1157          <screenshot>
1158            <mediaobject>
1159              <imageobject>
1160                <imagedata 
1161                  align="center"
1162                  fileref="figures/uml/datalayer.platforms.png" format="PNG" />
1163              </imageobject>
1164            </mediaobject>
1165          </screenshot>
1166        </figure>
1167      </sect3>
1168     
1169      <sect3 id="data_api.platforms.platforms">
1170        <title>Platforms</title>
1171       
1172        <para>
1173          The <classname docapi="net.sf.basedb.core.data">PlatformData</classname> holds information about a
1174          platform. A platform can have one or more <classname docapi="net.sf.basedb.core.data">PlatformVariant</classname>:s.
1175          Both the platform and variant are identified by an external ID that
1176          is fixed and can't be changed. <emphasis>Affymetrix</emphasis>
1177          is an example of a platform.
1178          If the <varname>fileOnly</varname> flag is set data for the platform
1179          can only be stored in files and not imported into the database. If
1180          the flag is not set data can be imported into the database.
1181          In the latter case, the <varname>rawDataType</varname> property
1182          can be used to lock the platform
1183          to a specific raw data type. If the value is <constant>null</constant>
1184          the platform can use any raw data type.
1185        </para>
1186       
1187        <para>
1188          Each platform and it's variant can be connected to one or more
1189          <classname docapi="net.sf.basedb.core.data">DataFileTypeData</classname> items. This item
1190          describes the kind of files that are used to hold data for
1191          the platform and/or variant. The file types are re-usable between
1192          different platforms and variants. Note that a file type may be attached
1193          to either only a platform or to a platform with a variant. File
1194          types attached to platforms are inherited by the variants. The variants
1195          can only define additional file types, not remove or redefine file types
1196          that has been attached to the platform.
1197        </para>
1198        <para>
1199          The file type is also identified
1200          by a fixed, non-changable external ID. The <varname>itemType</varname>
1201          property tells us what type of item the file holds data for (ie.
1202          array design or raw bioassay). It also links to a <classname docapi="net.sf.basedb.core.data">FileType</classname>
1203          which is the generic type of data in the file. This allows us to query
1204          the database for, as an example, files with the generic type
1205          <constant>FileType.RAW_DATA</constant>. If we are in an Affymetrix
1206          experiment we will get the CEL file, for another platform we will
1207          get another file.
1208        </para>
1209        <para>
1210          The <varname>required</varname> flag in <classname docapi="net.sf.basedb.core.data">PlatformFileTypeData</classname>
1211          is used to signal that the file is a required file. This is not
1212          enforeced by the core. It is intended to be used by client applications
1213          for creating a better GUI and for validation of an experiment.
1214        </para>
1215
1216      </sect3>
1217     
1218      <sect3 id="data_api.platforms.files">
1219        <title>FileStoreEnabled items and data files</title>
1220       
1221        <para>
1222          An item must implement the <interfacename docapi="net.sf.basedb.core">FileStoreEnabledData</interfacename>
1223          interface to be able to store data in files instead of in the database.
1224          The interface creates a link to a <classname docapi="net.sf.basedb.core.data">FileSetData</classname> object,
1225          which can hold several <classname docapi="net.sf.basedb.core.data">FileSetMemberData</classname> items.
1226          Each member points to specific <classname docapi="net.sf.basedb.core.data">FileData</classname> item.
1227          A file set can only store one file of each <classname docapi="net.sf.basedb.core.data">DataFileTypeData</classname>.
1228        </para>
1229       
1230      </sect3>
1231    </sect2>
1232
1233    <sect2 id="data_api.parameters">
1234      <title>Parameters</title>
1235     
1236      <para>
1237        This section gives an overview the generic parameter
1238        system in BASE that is used to store annotation values,
1239        plugin configuration values, job parameter values, etc.
1240      </para>
1241     
1242      <sect3 id="data_api.parameters.uml">
1243        <title>UML diagram</title>
1244       
1245        <figure id="data_api.figures.parameters">
1246          <title>Parameters</title>
1247          <screenshot>
1248            <mediaobject>
1249              <imageobject>
1250                <imagedata 
1251                  align="center"
1252                  fileref="figures/uml/datalayer.parameters.png" format="PNG" />
1253              </imageobject>
1254            </mediaobject>
1255          </screenshot>
1256        </figure>
1257      </sect3>
1258     
1259      <sect3 id="data_api.parameters.description">
1260        <title>Parameters</title>
1261       
1262        <para>
1263          The parameter system is a generic system that can store almost
1264          any kind of simple values (string, numbers, dates, etc.) and
1265          also links to other items. The <classname docapi="net.sf.basedb.core.data">ParameterValueData</classname> 
1266          class is an abstract base class that can hold multiple values (all must be of the
1267          same type). Unless only a specific type of values should be stored, this is
1268          the class that should be used when creating references for storing parameter
1269          values. It makes it possible for a single relaltion to use any kind of
1270          values or for a collection reference to mix multiple types of values.
1271          A typical use case maps a <classname>Map</classname> with the
1272          parameter name as the key:
1273        </para>
1274       
1275        <programlisting language="java">
1276private Map&lt;String, ParameterValueData&lt;?&gt;&gt; configurationValues;
1277/**
1278   Link parameter name with it's values.
1279   @hibernate.map table="`PluginConfigurationValues`" lazy="true" cascade="all"
1280   @hibernate.collection-key column="`pluginconfiguration_id`"
1281   @hibernate.collection-index column="`name`" type="string" length="255"
1282   @hibernate.collection-many-to-many column="`value_id`"
1283      class="net.sf.basedb.core.data.ParameterValueData"
1284*/
1285public Map&lt;String, ParameterValueData&lt;?&gt;&gt; getConfigurationValues()
1286{
1287   return configurationValues;
1288}
1289void setConfigurationValues(Map&lt;String, ParameterValueData&lt;?&gt;&gt; configurationValues)
1290{
1291   this.configurationValues = configurationValues;
1292}
1293</programlisting>
1294       
1295      <para>
1296      Now it is possible for the collection to store all types of values:
1297      </para>
1298     
1299      <programlisting language="java">
1300Map&lt;String, ParameterValueData&lt;?&gt;&gt; config = ...
1301config.put("names", new StringParameterValueData("A", "B", "C"));
1302config.put("sizes", new IntegerParameterValueData(10, 20, 30));
1303
1304// When you later load those values again you have to cast
1305// them to the correct class.
1306List&lt;String&gt; names = (List&lt;String&gt;)config.get("names").getValues();
1307List&lt;Integer&gt; sizes = (List&lt;Integer&gt;)config.get("sizes").getValues();
1308</programlisting>
1309
1310      </sect3>
1311     
1312    </sect2>
1313
1314    <sect2 id="data_api.annotations">
1315      <title>Annotations</title>
1316     
1317      <para>
1318        This section gives an overview of how the BASE annotation
1319        system works.
1320      </para>
1321     
1322      <sect3 id="data_api.annotations.uml">
1323        <title>UML diagram</title>
1324       
1325        <figure id="data_api.figures.annotations">
1326          <title>Annotations</title>
1327          <screenshot>
1328            <mediaobject>
1329              <imageobject>
1330                <imagedata 
1331                  align="center"
1332                  fileref="figures/uml/datalayer.annotations.png" format="PNG" />
1333              </imageobject>
1334            </mediaobject>
1335          </screenshot>
1336        </figure>
1337      </sect3>
1338     
1339      <sect3 id="data_api.annotations.description">
1340        <title>Annotations</title>
1341       
1342        <para>
1343        An item must implement the <interfacename docapi="net.sf.basedb.core.data">AnnotatableData</interfacename>
1344        interface to be able to use the annotation system. This interface gives
1345        a link to a <classname docapi="net.sf.basedb.core.data">AnnotationSetData</classname> item. This class
1346        encapsulates all annotations for the item. There are two types of
1347        annotations:
1348        </para>
1349       
1350        <itemizedlist>
1351        <listitem>
1352          <para>
1353          <emphasis>Primary annotations</emphasis> are annotations that
1354          explicitely belong to the item. An annotation set can contain
1355          only one primary annotation of each annotation type. The primary
1356          annotation are linked with the <property>annotations</property>
1357          property. This property is a map with an
1358          <classname docapi="net.sf.basedb.core.data">AnnotationTypeData</classname>  as the key.
1359          </para>
1360        </listitem>
1361       
1362        <listitem>
1363          <para>
1364          <emphasis>Inherited annotations</emphasis> are annotations
1365          that belong to a parent item, but that we want to use on
1366          another item as well. Inherited annotations are saved as
1367          references to either a single annotation or to another
1368          annotation set. Thus, it is possible for an item to inherit
1369          multiple annotations of the same annotation type.
1370          </para>
1371        </listitem>
1372        </itemizedlist>
1373       
1374        <para>
1375          The <classname docapi="net.sf.basedb.core.data">AnnotationData</classname> class is also
1376          just a placeholder. It connects the annotation set and
1377          annotation type with a <classname docapi="net.sf.basedb.core.data">ParameterValueData</classname>
1378          object. This is the object that holds the actual annotation
1379          values.
1380        </para>
1381       
1382      </sect3>
1383     
1384      <sect3 id="data_api.annotations.types">
1385        <title>Annotation types</title>
1386       
1387        <para>
1388        Instances of the <classname docapi="net.sf.basedb.core.data">AnnotationTypeData</classname> class
1389        defines the various annotations. It must have a <property>valueType</property> 
1390        property which cannot be changed. The value of this property controls
1391        which <classname docapi="net.sf.basedb.core.data">ParameterValueData</classname> subclass is used to store
1392        the annotation values, ie. <classname docapi="net.sf.basedb.core.data">IntegerParameterValueData</classname>,
1393        <classname docapi="net.sf.basedb.core.data">StringParameterValueData</classname>, etc.
1394        The <property>multiplicity</property> property holds the maximum allowed
1395        number of values for an annotation, or 0 if an unlimited number is
1396        allowed.
1397        </para>
1398       
1399        <para>
1400        The <property>itemTypes</property> collection holds the codes for
1401        the types of items the annotation type can be used on. This is
1402        checked when new annotations are created but already existing
1403        annotations are not affected if the collection is modified.
1404        </para>
1405       
1406        <para>
1407        Annotation types with the <property>protocolParameter</property> flag set
1408        are treated a bit differently. They will not show up as annotations
1409        to items with a type found in the <property>itemTypes</property> collection.
1410        A protocol parameter should be attached to a protocol. Then, when an item
1411        is using that protocol it becomes possible to add annotation values for
1412        the annotation types specified as protocol parameters. It doesn't matter
1413        if the item's type is found in the <property>itemTypes</property> 
1414        collection or not.
1415        </para>
1416       
1417        <para>
1418        The <property>options</property> collection is used to store additional
1419        options required by some of the value types, for example a max string
1420        length for string annotations or the max and min allowed value for
1421        integer annotations.
1422        </para>
1423       
1424        <para>
1425        The <property>enumeration</property> property is a boolean flag
1426        indicating if the allowed values are predefined as an enumeration.
1427        In that case those values are found in the <property>enumerationValues</property>
1428        property. The actual subclass is determined by the <property>valueType</property>
1429        property.
1430        </para>
1431       
1432        <para>
1433        Most of the other properties are hints to client applications how
1434        to render the input field for the annotation.
1435        </para>
1436       
1437      </sect3>
1438     
1439      <sect3 id="data_api.annotations.categories">
1440        <title>Categories</title>
1441       
1442        <para>
1443        The <classname docapi="net.sf.basedb.core.data">AnnotationTypeCategoryData</classname> class defines
1444        categories that are used to group annotation types that are related to
1445        each other. This information is mainly useful for client applications
1446        when displaying forms for annotating items, that wish to provide a
1447        clearer interface when there are many (say 50+) annotations type for
1448        an item. An annotation type can belong to more than one category.
1449        </para>
1450       
1451      </sect3>
1452     
1453    </sect2>
1454
1455    <sect2 id="data_api.protocols">
1456      <title>Protocols</title>
1457
1458      <para>
1459        This section gives an overview of how protocols that describe various
1460        processes, such as sampling, extraction and scanning, are used in BASE.
1461      </para>
1462     
1463      <sect3 id="data_api.protocols.uml">
1464        <title>UML diagram</title>
1465       
1466        <figure id="data_api.figures.protocols">
1467          <title>Protocols</title>
1468          <screenshot>
1469            <mediaobject>
1470              <imageobject>
1471                <imagedata 
1472                  align="center"
1473                  fileref="figures/uml/datalayer.protocols.png" format="PNG" />
1474              </imageobject>
1475            </mediaobject>
1476          </screenshot>
1477        </figure>
1478      </sect3>
1479     
1480      <sect3 id="data_api.protocols.description">
1481        <title>Protocols</title>
1482       
1483        <para>
1484        A protocol is something that defines a procedure or recipe for some
1485        kind of action, such as sampling, extraction and scanning. In BASE we only
1486        store a short name and description. It is possible to attach a file
1487        that provides a longer description of the procedure.
1488        </para>
1489     
1490      </sect3>
1491     
1492      <sect3 id="data_api.protocols.parameters">
1493        <title>Parameters</title>
1494       
1495        <para>
1496        The procedure described by the protocol may have parameters
1497        that are set indepentently each time the protocol is used. It
1498        could for example be a temperature, a time or something else.
1499        The definition of parameters is done by creating annotation
1500        types and attaching them to the protocol. It is only possible
1501        to attach annotation types which has the <property>protocolParameter</property>
1502        property set to <constant>true</constant>. The same annotation type
1503        can be used for more than one protocol, but only do this if the
1504        parameters actually has the same meaning.
1505        </para>
1506     
1507      </sect3>
1508     
1509    </sect2>
1510
1511    <sect2 id="data_api.plugins">
1512      <title>Plug-ins, jobs and job agents</title>
1513     
1514      <para>
1515         This section gives an overview of plug-ins, jobs and job agents.
1516      </para>
1517     
1518      <itemizedlist>
1519        <title>See also</title>
1520        <listitem><xref linkend="plugins.installation" /></listitem>
1521        <listitem><xref linkend="installation_upgrade.jobagents" /></listitem>
1522      </itemizedlist>
1523     
1524      <sect3 id="data_api.plugins.uml">
1525        <title>UML diagram</title>
1526       
1527        <figure id="data_api.figures.plugins">
1528          <title>Plug-ins, jobs and job agents</title>
1529          <screenshot>
1530            <mediaobject>
1531              <imageobject>
1532                <imagedata 
1533                  align="center"
1534                  scalefit="1" width="100%"
1535                  fileref="figures/uml/datalayer.plugins.png" format="PNG" />
1536              </imageobject>
1537            </mediaobject>
1538          </screenshot>
1539        </figure>
1540      </sect3>
1541
1542      <sect3 id="data_api.plugins.plugins">
1543        <title>Plug-ins</title>
1544       
1545        <para>
1546          The <classname docapi="net.sf.basedb.core.data">PluginDefinitionData</classname> holds information of the
1547          installed plugin classes. Much of the information is copied from the
1548          plug-in itself from the <classname docapi="net.sf.basedb.core.plugin">About</classname> object and by checking
1549          which interfaces it implements.
1550        </para>
1551       
1552        <para>
1553          There are five main types of plug-ins:
1554        </para>
1555       
1556        <itemizedlist>
1557        <listitem>
1558          <para>
1559          IMPORT (mainType = 1): A plug-in that imports data to BASE.
1560          </para>
1561        </listitem>
1562        <listitem>
1563          <para>
1564          EXPORT (mainType = 2): A plug-in that exports data from BASE.
1565          </para>
1566        </listitem>
1567        <listitem>
1568          <para>
1569          INTENSITY (mainType = 3): A plug-in that calculates intensity values
1570          from raw data.
1571          </para>
1572        </listitem>
1573        <listitem>
1574          <para>
1575          ANALYZE (mainType = 4): A plug-in that analyses data.
1576          </para>
1577        </listitem>
1578        <listitem>
1579          <para>
1580          OTHER (mainType = 5): Any other plug-in.
1581          </para>
1582        </listitem>
1583        </itemizedlist>
1584       
1585        <para>
1586          A plug-in may have different configurations. The flags <property>supportsConfigurations</property>
1587          and <property>requiresConfiguration</property> are used to specify if a plug-in
1588          must have or can't have any configurations. Configuration parameter values are
1589          versioned. Each time anyone updates a configuration the version number
1590          is increased and the parameter values are stored as a new entity.
1591          This is required because we want to be able to know exactly which
1592          parameters a job were using when it was executed. When a job is
1593          created we also store the parameter version number
1594          (<property>JobData.parameterVersion</property>). This means that even if
1595          someone changes the configuration later we will always know which
1596          parameters the job used.
1597        </para>
1598       
1599        <para>
1600          The <classname docapi="net.sf.basedb.core.data">PluginTypeData</classname> class is ued to group
1601          plug-ins that share some common functionality, by implementing
1602          additional (optional) interfaces. For example, the
1603          <interfacename docapi="net.sf.basedb.core.plugin">AutoDetectingImporter</interfacename> should be implemented
1604          by import plug-ins that supports automatic detection of file formats.
1605          Another example is the <interfacename docapi="net.sf.basedb.core.plugin">AnalysisFilterPlugin</interfacename>
1606          interface which should be implemented by all analysis plug-ins that
1607          only filters data.
1608        </para>
1609
1610      </sect3>
1611     
1612      <sect3 id="data_api.plugins.jobs">
1613        <title>Jobs</title>
1614       
1615        <para>
1616          A job represents a single invokation of a plug-in to do some work.
1617          The <classname docapi="net.sf.basedb.core.data">JobData</classname> class holds information about this.
1618          A job is usuallu executed by a plug-in, but doesn't have to be. The
1619          <property>status</property> property holds the current state of a job.
1620        </para>
1621       
1622        <itemizedlist>
1623        <listitem>
1624          <para>
1625            UNCONFIGURED (status = 0): The job is not yet ready to be executed.
1626          </para>
1627        </listitem>
1628        <listitem>
1629          <para>
1630            WAITING (status = 1): The job is waiting to be executed.
1631          </para>
1632        </listitem>
1633        <listitem>
1634          <para>
1635            PREPARING (status = 5): The job is about to be executed but hasn't started yet.
1636          </para>
1637        </listitem>
1638        <listitem>
1639          <para>
1640            EXECUTING (status = 2): The job is currently executing.
1641          </para>
1642        </listitem>
1643        <listitem>
1644          <para>
1645            DONE (status = 3): The job finished successfully.
1646          </para>
1647        </listitem>
1648        <listitem>
1649          <para>
1650            ERROR (status = 4): The job finished with an error.
1651          </para>
1652        </listitem>
1653        </itemizedlist>
1654      </sect3>
1655
1656      <sect3 id="data_api.plugins.agents">
1657        <title>Job agents</title>
1658       
1659        <para>
1660          A job agent is a program running on the same or a different server that
1661          is regularly checking for jobs that are waiting to be executed. The
1662          <classname docapi="net.sf.basedb.core.data">JobAgentData</classname> holds information about a job agent
1663          and the <classname docapi="net.sf.basedb.core.data">JobAgentSettingsData</classname> links the agent
1664          with the plug-ins the agent is able to execute. The job agent will only
1665          execute jobs that are owner by users or projects that the job agent has
1666          been shared to with at least use permission. The <property>priorityBoost</property>
1667          property can be used to give specific plug-ins higher priority.
1668          Thus, for a job agent it is possible to:
1669        </para>
1670       
1671        <itemizedlist>
1672        <listitem>
1673          <para>
1674          Specify exactly which plug-ins it will execute. For example, it is possible
1675          to dedicate one agent to only run one plug-in.
1676          </para>
1677        </listitem>
1678        <listitem>
1679          <para>
1680          Give some plug-ins higher priority. For example a job agent that is mainly
1681          used for importing data should give higher priority to all import plug-ins.
1682          Other types of jobs will have to wait until there are no more data to be
1683          imported.
1684          </para>
1685        </listitem>
1686        <listitem>
1687          <para>
1688          Specify exactly which users/groups/projects that may use the agent. For
1689          example, it is possible to dedicate one agent to only run jobs for a certain
1690          project.
1691          </para>
1692        </listitem>
1693        </itemizedlist>
1694       
1695      </sect3>
1696
1697
1698    </sect2>
1699   
1700    <sect2 id="data_api.biomaterials">
1701      <title>Biomaterials</title>
1702     
1703      <sect3 id="data_api.biomaterials.uml">
1704        <title>UML diagram</title>
1705       
1706        <figure id="data_api.figures.biomaterials">
1707          <title>Biomaterials</title>
1708          <screenshot>
1709            <mediaobject>
1710              <imageobject>
1711                <imagedata 
1712                  align="center"
1713                  fileref="figures/uml/datalayer.biomaterials.png" format="PNG" />
1714              </imageobject>
1715            </mediaobject>
1716          </screenshot>
1717        </figure>
1718      </sect3>
1719     
1720      <sect3 id="data_api.biomaterials.description">
1721        <title>Biomaterials</title>
1722       
1723        <para>
1724          There are four types of biomaterials: <classname docapi="net.sf.basedb.core.data">BioSourceData</classname>,
1725          <classname docapi="net.sf.basedb.core.data">SampleData</classname>, <classname docapi="net.sf.basedb.core.data">ExtractData</classname> and
1726          <classname docapi="net.sf.basedb.core.data">LabeledExtractData</classname>.
1727          All four types of are derived from the base class <classname docapi="net.sf.basedb.core.data">BioMaterialData</classname>.
1728          The reason for this is that they all share common functionality such as pooling
1729          and events. By using a common base class we do not have to create duplicate
1730          classes for keeping track of events and parents.
1731        </para>
1732       
1733        <para>
1734          The <classname docapi="net.sf.basedb.core.data">BioSourceData</classname> is the simplest of the biomaterials.
1735          It cannot have parents and can't participate in events. It's only used as a
1736          (non-required) parent for samples.
1737        </para>
1738       
1739        <para>
1740          The <classname docapi="net.sf.basedb.core.data">MeasuredBioMaterialData</classname> class is used as a base
1741          class for the other three biomaterial types. It introduces quantity
1742          measurements and can store original and remaining quantities. They are
1743          both optional. If an original quantity has been specified the core
1744          automatically calculates the remaining quantity based on the events a
1745          biomaterial participates in.
1746        </para>
1747       
1748        <para>
1749          All measured biomaterial have at least one event associated with them,
1750          the creation event, which holds information about the creation of the
1751          biomaterial. A measured biomaterial can be created in three ways:
1752        </para>
1753       
1754        <itemizedlist>
1755        <listitem>
1756          <para>
1757          From a single item of the parent type. Biosource is the parent type of
1758          samples, sample is the parent type of extracts, and extract is the
1759          parent type of labeled extracts. In this case the
1760          <property>pooled</property> property is <constant>false</constant>
1761          and the parent is specified in the <property>parent</property> property.
1762          If the parent is not a <classname docapi="net.sf.basedb.core.data">BioSourceData</classname> this information
1763          is duplicated, with the addition of an optional used quantity value, in the
1764          <property>sources</property> collection of the <classname docapi="net.sf.basedb.core.data">BioMaterialEventData</classname>
1765          object representing the creation event. It is the responsibility of the
1766          core to make sure that everything is properly synchronized and that
1767          remaining quantities are calculated.
1768          </para>
1769        </listitem>
1770       
1771        <listitem>
1772          <para>
1773          From one or more items of the same type, i.e pooling.
1774          In this case the <property>pooled</property> property is <constant>true</constant> 
1775          and the <property>parent</property> property is null. All source
1776          biomaterials are contained in the <property>sources</property> collection.
1777          The core is still responsible for keeping everything synchronized and to
1778          update remaining quantities.
1779          </para>
1780        </listitem>
1781       
1782        <listitem>
1783          <para>
1784          As a standalone biomaterial without parents.
1785          </para>
1786        </listitem>
1787        </itemizedlist>
1788
1789      </sect3>
1790     
1791      <sect3 id="data_api.biomaterials.events">
1792        <title>Biomaterial events</title>
1793       
1794        <para>
1795          An event represents something that happened to one or more biomaterials, for example
1796          the creation of another biomaterial. The <classname docapi="net.sf.basedb.core.data">BioMaterialEventData</classname>
1797          holds information about entry and event dates, protocols used, the user who is
1798          responsible, etc. There are three types of events represented by the <property>eventType</property>
1799          property.
1800        </para>
1801       
1802        <orderedlist>
1803        <listitem>
1804          <para>
1805          <emphasis>Creation event</emphasis>: This event represents the creation of a (measured)
1806          biomaterial. The <property>sources</property> collection contains
1807          information about the biomaterials that were used to create the new
1808          biomaterial. If the biomaterial is a pooled biomaterial all sources must
1809          be of the same type. Otherwise there can only be one source of the parent
1810          type. These rules are maintained by the core.
1811          </para>
1812        </listitem>
1813       
1814        <listitem>
1815          <para>
1816          <emphasis>Hybridization event</emphasis>: This event represents the creation
1817          of a hybridization. This event type is needed because we want to keep track
1818          of quantities for labeled extracts. This event has a hybridization as a
1819          product instead of a biomaterial. The sources collection can only contain
1820          labeled extracts.
1821          </para>
1822        </listitem>
1823
1824        <listitem>
1825          <para>
1826          <emphasis>Other event</emphasis>: This event represents some other important
1827          information about a single biomaterial that affected the remaining quantity.
1828          This event type doesn't have any sources.
1829          </para>
1830        </listitem>
1831        </orderedlist>
1832      </sect3>
1833 
1834    </sect2>
1835
1836    <sect2 id="data_api.plates">
1837      <title>Array LIMS - plates</title>
1838
1839      <sect3 id="data_api.plates.uml">
1840        <title>UML diagram</title>
1841       
1842        <figure id="data_api.figures.plates">
1843          <title>Array LIMS - plates</title>
1844          <screenshot>
1845            <mediaobject>
1846              <imageobject>
1847                <imagedata 
1848                  align="center"
1849                  scalefit="1" width="100%"
1850                  fileref="figures/uml/datalayer.plates.png" format="PNG" />
1851              </imageobject>
1852            </mediaobject>
1853          </screenshot>
1854        </figure>
1855      </sect3>
1856
1857      <sect3 id="data_api.plates.description">
1858        <title>Plates</title>
1859       
1860        <para>
1861          The <classname docapi="net.sf.basedb.core.data">PlateData</classname> is the main class holding information
1862          about a single plate. The associated <classname docapi="net.sf.basedb.core.data">PlateGeometryData</classname>
1863          defines how many rows and columns there are on a plate. Since this
1864          information is used to create wells, and for various other checks it is
1865          not possible to change the number of rows or columns once a geometry has
1866          been created.
1867        </para>
1868         
1869        <para>
1870          All plates must have a <classname docapi="net.sf.basedb.core.data">PlateTypeData</classname> which defines
1871          the geometry and a set of event types (see below).
1872        </para>
1873       
1874        <para>
1875          If the destroyed flag of a plate is set it is not allowed to use the
1876          plate for a plate mapping or to create array designs. However, it
1877          is possible to change the flag to not destroyed.
1878        </para>
1879
1880        <para>
1881          The barcode is intended to be used as an external identifier of the plate.
1882          But, the core doesn't care about the value or if it is unique or not.
1883        </para>
1884      </sect3>
1885     
1886      <sect3 id="data_api.plates.events">
1887        <title>Plate events</title>
1888
1889        <para>
1890          The plate type defines a set of <classname docapi="net.sf.basedb.core.data">PlateEventTypeData</classname>
1891          objects, each one represening a particular event a plate of this type
1892          usually goes trough. For a plate of a certain type, it is possible to
1893          attach exactly one event of each event type. The event type defines an
1894          optional protocol type, which can be used by client applications to
1895          filter a list of protocols for the event. The core doesn't check that
1896          the selected protocol for an event is of the same protocol type as
1897          defined by the event type.
1898        </para>
1899
1900        <para>
1901          The ordinal value can be used as a hint to client applications in
1902          which order the events actually are performed in the lab. The core doesn't
1903          care about this value or if several event types have the same value.
1904        </para>
1905      </sect3>
1906
1907      <sect3 id="data_api.plates.mappings">
1908        <title>Plate mappings</title>
1909       
1910        <para>
1911          A plate can be created either from scratch, with the help of the information
1912          in a <classname docapi="net.sf.basedb.core.data">PlateMappingData</classname>, from a set of parent plates.
1913          In the first case it is possible to specify a reporter for each well on the
1914          plate. In the second case the mapping code creates all the wells and links
1915          them to the parent wells on the parent plates. Once the plate has been saved
1916          to the database, the wells cannot be modified (because they are used
1917          downstream for various validation, etc.)
1918        </para>
1919       
1920        <para>
1921          The details in a plate mapping are simply coordinates that for each
1922          destination plate, row and column define a source plate, row and column.
1923          It is possible for a single source well to be mapped to multiple destination
1924          wells, but for each destination well only a single source well can be
1925          used.
1926        </para>
1927       
1928      </sect3>
1929
1930    </sect2>
1931
1932    <sect2 id="data_api.arrays">
1933      <title>Array LIMS - arrays</title>
1934     
1935      <sect3 id="data_api.arrays.uml">
1936        <title>UML diagram</title>
1937       
1938        <figure id="data_api.figures.arrays">
1939          <title>Array LIMS - arrays</title>
1940          <screenshot>
1941            <mediaobject>
1942              <imageobject>
1943                <imagedata 
1944                  align="center"
1945                  fileref="figures/uml/datalayer.arrays.png" format="PNG" />
1946              </imageobject>
1947            </mediaobject>
1948          </screenshot>
1949        </figure>
1950      </sect3>
1951     
1952      <sect3 id="data_api.arrays.designs">
1953        <title>Array designs</title>
1954       
1955        <para>
1956          Array designs are stored in <classname docapi="net.sf.basedb.core.data">ArrayDesignData</classname> objects
1957          and can be created either as standalone designs or
1958          from plates. In the first case the features on an array design
1959          are described by a reporter map. A reporter map is a file
1960          that maps a coordinate on an array design to a reporter.
1961          The coordinate system on an array design is divided into blocks.
1962          Each block can be identified either by a <property>blockNumber</property>
1963          or by meta coordinates. This information is stored in
1964          <classname docapi="net.sf.basedb.core.data">ArrayDesignBlockData</classname> items. Each block
1965          contains several <classname docapi="net.sf.basedb.core.data">FeatureData</classname> items, each
1966          one identified by a row and column coordinate. Platforms that doesn't
1967          divide the array design into block should create a single super-block
1968          that holds all features.
1969        </para>
1970       
1971        <para>
1972          Array designs that are created from plates use a print map file
1973          instead of a reporter map. A print map is similar to a plate mapping
1974          but maps features (instead of wells) to wells. The file should
1975          specifify which plate and well a feature is created from. Reporter
1976          information will automatically be copied by BASE from the well.
1977        </para>
1978       
1979        <para>
1980          It is also possible to skip the importing of features into the
1981          database and just keep the data in the orginal files instead.
1982          This is typically done for Affymetrix CDF files.
1983        </para>
1984       
1985      </sect3>
1986     
1987      <sect3 id="data_api.arrays.slides">
1988        <title>Array slides</title>
1989       
1990        <para>
1991          The <classname docapi="net.sf.basedb.core.data">ArraySlideData</classname> represents a single
1992          array. Arrays are usually printed several hundreds in a batch,
1993          represented by a <classname docapi="net.sf.basedb.core.data">ArrayBatchData</classname> item.
1994          The <property>batchIndex</property> is the ordinal number of the
1995          array in the batch. The <property>barcode</property> can be used
1996          as a means for external programs to identify the array. BASE doesn't
1997          care if a value is given or if they are unique or not. If the
1998          <property>destroyed</property> flag is set it prevents a slide from
1999          beeing used by a hybridization.
2000        </para>
2001
2002      </sect3>
2003    </sect2>
2004
2005    <sect2 id="data_api.rawdata">
2006      <title>Hybridizations and raw data</title>
2007     
2008      <sect3 id="data_api.rawdata.uml">
2009        <title>UML diagram</title>
2010       
2011        <figure id="data_api.figures.rawdata">
2012          <title>Hybridizations and raw data</title>
2013          <screenshot>
2014            <mediaobject>
2015              <imageobject>
2016                <imagedata 
2017                  align="center"
2018                  scalefit="1" width="100%"
2019                  fileref="figures/uml/datalayer.rawdata.png" format="PNG" />
2020              </imageobject>
2021            </mediaobject>
2022          </screenshot>
2023        </figure>
2024      </sect3>
2025     
2026      <sect3 id="data_api.rawdata.hybridizations">
2027        <title>Hybridizations</title>
2028       
2029        <para>
2030        Hybridizations connects the slides from the Array LIMS part
2031        with labeled extracts from the biomaterials part. The <property>creationEvent</property>
2032        is used to register which labeled extracts that were used on the hybridization.
2033        The relation to slides is a one-to-one relation. A slide can only be used on
2034        a single hybridization and a hybridization can only use a single slide. The relation
2035        is optional from both sides.
2036        </para>
2037
2038        <para>
2039        The scanning of the hybridized slide is registered as separate scan events.
2040        One or more images can optionally be attached to each scan.
2041        The images are not used by BASE.
2042        </para>
2043       
2044      </sect3>
2045     
2046      <sect3 id="data_api.rawdata.description">
2047        <title>Raw data</title>
2048       
2049        <para>
2050        A <classname docapi="net.sf.basedb.core.data">RawBioAssayData</classname> object represents
2051        the raw data that is produced by analysing the image(s) from a
2052        single scan. You may register which software that was used, the
2053        protocol and any parameters (through the annotation system).
2054        </para>
2055
2056        <para>
2057        Files with the analysed data values can be attached to the
2058        associated <classname docapi="net.sf.basedb.core.data">FileSetData</classname> object. The platform
2059        and, optionally, the variant has information about the file types
2060        that can be used for that platform. If the platform file types support
2061        metadata extraction, headers, the number of spots, and other
2062        information may be automatically extracted from the raw data file(s).
2063        </para>
2064       
2065        <para>
2066        If the platform support it, raw data can also be imported into the database.
2067        This is handled by batchers and <classname docapi="net.sf.basedb.core.data">RawData</classname> objects.
2068        Which table to store the data in depends on the <property>rawDataType</property>
2069        property. The properties shown for the <classname>RawData</classname> class
2070        in the diagram are the mandatory properties. Each raw data type defines additional
2071        properties that are specific to that raw data type.
2072        </para>
2073       
2074      </sect3>
2075     
2076      <sect3 id="data_api.rawdata.spotimages">
2077        <title>Spot images</title>
2078       
2079        <para>
2080        Spot images can be created if you have the original image
2081        files. BASE can use 1-3 images as sources for the red, green
2082        and blue channel respectively. The creation of spotimages requires
2083        that x and y coordinates are given for each raw data spot. The scaling
2084        and offset values are used to convert the coordinates to pixel
2085        coordinates. With this information BASE is able to cut out a square
2086        from the source images that, theoretically, contains a specific spot and
2087        nothing else. The spot images are gamma-corrected independently and then
2088        put together into PNG images that are stored in a zip file.
2089        </para>
2090      </sect3>
2091     
2092    </sect2>
2093
2094    <sect2 id="data_api.experiments">
2095      <title>Experiments and analysis</title>
2096     
2097     
2098      <sect3 id="data_api.experiments.uml">
2099        <title>UML diagram</title>
2100       
2101        <figure id="data_api.figures.experiments">
2102          <title>Experiments</title>
2103          <screenshot>
2104            <mediaobject>
2105              <imageobject>
2106                <imagedata 
2107                  align="center"
2108                  scalefit="1" width="75%"
2109                  fileref="figures/uml/datalayer.experiments.png" format="PNG" />
2110              </imageobject>
2111            </mediaobject>
2112          </screenshot>
2113        </figure>
2114      </sect3>
2115     
2116      <sect3 id="data_api.experiments.description">
2117        <title>Experiments</title>
2118       
2119        <para>
2120          The <classname docapi="net.sf.basedb.core.data">ExperimentData</classname> 
2121          class is used to collect information about a single experiment. It
2122          links to any number of <classname docapi="net.sf.basedb.core.data">RawBioAssayData</classname>
2123          items, which must all be of the same <classname 
2124          docapi="net.sf.basedb.core">RawDataType</classname>.
2125        </para>
2126       
2127        <para>
2128          Annotation types that are needed in the analysis must connected to
2129          the experiment as experimental factors and the annotation values should
2130          be set on or inherited by each raw bioassay that is part of the
2131          experiment.
2132        </para>
2133       
2134        <para>
2135          The directory connected to the experiment is the default directory
2136          where plugins that generate files should store them.
2137        </para>
2138      </sect3>
2139           
2140      <sect3 id="data_api.experiments.bioassays">
2141        <title>Bioassay sets, bioassays and transformations</title>
2142       
2143        <para>
2144          Each line of analysis starts with the creation of a <emphasis>root</emphasis>
2145          <classname docapi="net.sf.basedb.core.data">BioAssaySetData</classname>,
2146          which holds the intensities calculated from the raw data.
2147          A bioassayset can hold one intensity for each channel. The number of
2148          channels is defined by the raw data type. For each raw bioassay used a
2149          <classname docapi="net.sf.basedb.core.data">BioAssayData</classname>
2150          is created.
2151        </para>
2152       
2153        <para>
2154          Information about the process that calculated the intensities are
2155          stored in a <classname docapi="net.sf.basedb.core.data">TransformationData</classname>
2156          object. The root transformation links with the raw bioassays that are used
2157          in this line of analysis and to a <classname 
2158          docapi="net.sf.basedb.core.data">JobData</classname> which has information
2159          about which plug-in and parameters that was used in the calculation.
2160        </para>
2161     
2162        <para>
2163          Once the root bioassayset has been created it is possible to
2164          again apply a transformation to it. This time the transformation
2165          links to a single source bioassayset instead of the raw bioassays.
2166          As before, it still links to a job with information about the plug-in and
2167          parameters that does the actual work. The transformation must make sure
2168          that new bioassays are created and linked to the bioassays in the
2169          source bioassayset. This above process may be repeated as many times
2170          as needed.
2171        </para>
2172       
2173        <para>
2174          Data to a bioassay set can only be added to it before it has been
2175          committed to the database. Once the transaction has been committed
2176          it is no longed possible to add more data or to modify existing
2177          data.
2178        </para>
2179     
2180      </sect3>
2181
2182      <sect3 id="data_api.experiments.virtualdb">
2183        <title>Virtual databases, datacubes, etc.</title>
2184       
2185        <para>
2186          The above processes requires a flexible storage solution for the data.
2187          Each experiment is related to a <classname docapi="net.sf.basedb.core.data">VirtualDb</classname>
2188          object. This object represents the set of tables that are needed to store
2189          data for the experiment. All tables are created in a special part of the
2190          BASE database that we call the <emphasis>dynamic database</emphasis>.
2191          In MySQL the dynamic database is a separate database, in Postgres it is
2192          a separate schema.
2193        </para>
2194       
2195        <para>
2196          A virual database is divided into data cubes. A data cube can be seen
2197          as a three-dimensional object where each point can hold data that in
2198          most cases can be interpreted as data for a single spot from an
2199          array. The coordinates to a point is given by <emphasis>layer</emphasis>,
2200          <emphasis>column</emphasis> and <emphasis>position</emphasis>. The
2201          layer and column coordinates are represented by the
2202          <classname docapi="net.sf.basedb.core.data">DataCubeLayerData</classname>
2203          and <classname docapi="net.sf.basedb.core.data">DataCubeColumnData</classname>
2204          objects. The position coordinate has no separate object associated with
2205          it.
2206        </para>
2207       
2208        <para>
2209          Data for a single bioassay set is always stored in a single layer. It
2210          is possible for more than one bioassay set to use the same layer. This
2211          usually happens for filtering transformations that doesn't modify the
2212          data.  The filtered bioassay set is then linked to a
2213          <classname docapi="net.sf.basedb.core.data">DataCubeFilterData</classname>
2214          object, which has information about which data points that
2215          passed the filter.
2216        </para>
2217       
2218        <para>
2219          All data for a bioassay is stored in a single column.
2220          Two bioassays in different bioassaysets (layers) can only have the same
2221          column if one is the parent of the other.
2222        </para>
2223       
2224        <para>
2225          The position coordinate is tied to a reporter.
2226        </para>
2227       
2228        <para>
2229          A child bioassay set may use the same data cube as it's parent
2230          bioassay set if all of the following conditions are true:
2231        </para>
2232       
2233        <itemizedlist>
2234        <listitem>
2235          <para>
2236          All positions are linked to the same reporter as the positions
2237          in the parent bioassay set.
2238          </para>
2239        </listitem>
2240       
2241        <listitem>
2242          <para>
2243          All data points are linked to the same (possible many) raw data
2244          spots as the corresponding data points in the parent bioassay set.
2245          </para>
2246        </listitem>
2247       
2248        <listitem>
2249          <para>
2250          The bioassays in the child bioassay set each have exactly one
2251          parent in the parent bioassay set. No parent bioassay may be the
2252          parent of more than one child bioassay.
2253          </para>
2254        </listitem>
2255        </itemizedlist>
2256       
2257        <para>
2258          If any of the above conditions are not true, a new data cube
2259          must be created for the child bioassay set.
2260        </para>
2261      </sect3>
2262     
2263      <sect3 id="data_api.dynamic.description">
2264        <title>The dynamic database</title>
2265
2266        <figure id="data_api.figures.dynamic">
2267          <title>The dynamic database</title>
2268          <screenshot>
2269            <mediaobject>
2270              <imageobject>
2271                <imagedata 
2272                  align="center"
2273                  fileref="figures/uml/datalayer.dynamic.png" format="PNG" />
2274              </imageobject>
2275            </mediaobject>
2276          </screenshot>
2277        </figure>
2278       
2279        <para>
2280          Each virtual database consists of several tables. The tables
2281          are dynamically created when needed. For each table shown in the diagram
2282          the # sign is replaced by the id of the virtual database object at run
2283          time.
2284        </para>
2285       
2286        <para>
2287          There are no classes in the data layer for these tables and they
2288          are not mapped with Hibernate. When we work with these tables we
2289          are always using batcher classes and queries that works with integer,
2290          floats and strings.
2291        </para>
2292       
2293        <bridgehead>The D#Spot table</bridgehead>
2294        <para>
2295          This is the main table which keeps the intensities for a single spot
2296          in the data cube. Extra values attached to the spot are kept in separate
2297          tables, one for each type of value (D#SpotInt, D#SpotFloat and D#SpotString).
2298        </para>
2299       
2300        <bridgehead>The D#Pos table</bridgehead>
2301        <para>
2302          This table stores the reporter id for each position in a cube.
2303          Extra values attached to the position are kept in separate tables,
2304          one for each type of value (D#PosInt, D#PosFloat and D#PosString).
2305        </para>
2306       
2307        <bridgehead>The D#Filter table</bridgehead>
2308        <para>
2309          This table stores the coordinates for the spots that remain after
2310          filtering. Note that each filter is related to a bioassayset which
2311          gives the cube and layer values. Each row in the filter table then
2312          adds the column and position allowing us to find the spots in the
2313          D#Spot table.
2314        </para>
2315       
2316        <bridgehead>The D#RawParents table</bridgehead>
2317        <para>
2318          This table holds mappings for a spot to the raw data it is calculated
2319          from. We don't need the layer coordinate since all layers in a cube
2320          must have the same mapping to raw data.
2321        </para>
2322       
2323      </sect3>     
2324
2325     
2326    </sect2>
2327   
2328    <sect2 id="data_api.misc">
2329      <title>Other classes</title>
2330     
2331      <sect3 id="data_api.misc.uml">
2332        <title>UML diagram</title>
2333       
2334        <figure id="data_api.figures.misc">
2335          <title>Other classes</title>
2336          <screenshot>
2337            <mediaobject>
2338              <imageobject>
2339                <imagedata 
2340                  align="center"
2341                  fileref="figures/uml/datalayer.misc.png" format="PNG" />
2342              </imageobject>
2343            </mediaobject>
2344          </screenshot>
2345        </figure>
2346      </sect3>
2347     
2348    </sect2>
2349
2350  </sect1>
2351 
2352  <sect1 id="api_overview.core_api" chunked="1">
2353    <title>The Core API</title>
2354   
2355    <para>
2356      This section gives an overview of various parts of the core API.
2357    </para>
2358   
2359    <sect2 id="core_api.data_in_files">
2360      <title>Using files to store data</title>
2361     
2362      <para>
2363        BASE 2.5 introduced the possibility to use files to store data instead
2364        of importing it into the database. Files can be attached
2365        to any item that implements the <interfacename docapi="net.sf.basedb.core">FileStoreEnabled</interfacename>
2366        interface. Currently this is <classname docapi="net.sf.basedb.core">RawBioAssay</classname>
2367        and <classname docapi="net.sf.basedb.core">ArrayDesign</classname>. The
2368        ability to store data in files is not a replacement for storing data in the
2369        database. It is possible (for some platforms/raw data types) to have data in
2370        files and in the database at the same time. We would have liked to enforce
2371        that (raw) data is always present in files, but this will not be backwards compatible
2372        with older installations, so there are three cases:
2373      </para>
2374     
2375      <itemizedlist>
2376      <listitem>
2377        <para>
2378        Data in files only
2379        </para>
2380      </listitem>
2381      <listitem>
2382        <para>
2383        Data in the database only
2384        </para>
2385      </listitem>
2386      <listitem>
2387        <para>
2388        Data in both files and in the database
2389        </para>
2390      </listitem>
2391      </itemizedlist>
2392     
2393      <para>
2394        Not all three cases are supported for all types of data. This is controlled
2395        by the <classname docapi="net.sf.basedb.core">Platform</classname> class, which may disallow
2396        that data is stored in the database. To check this call
2397        <methodname>Platform.isFileOnly()</methodname> and/or
2398        <methodname>Platform.getRawDataType()</methodname>. If the <methodname>isFileOnly()</methodname>
2399        method returns <constant>true</constant>, the platform can't store data in
2400        the database. If the value is <constant>false</constant> more information
2401        can be obtained by calling <methodname>getRawDataType()</methodname>,
2402        which may return:
2403      </para>
2404     
2405      <itemizedlist>
2406      <listitem>
2407        <para>
2408          <constant>null</constant>: The platform can store data with any
2409          raw data type in the database.
2410        </para>
2411      </listitem>
2412      <listitem>
2413        <para>
2414        A <classname docapi="net.sf.basedb.core">RawDataType</classname> that has <code>isStoredInDb() == true</code>:
2415        The platform can store data in the database but only data with the specified raw
2416        data type.
2417        </para>
2418      </listitem>
2419      <listitem>
2420        <para>
2421        A <classname docapi="net.sf.basedb.core">RawDataType</classname> that has <code>isStoredInDb() == false</code>:
2422        The platform can't store data in the database.
2423        </para>
2424      </listitem>
2425      </itemizedlist>
2426
2427      <para>
2428        One major change from earlier BASE versions is that the registration of raw data types
2429        has changed. The <filename>raw-data-types.xml</filename> file should
2430        only be used for raw data types that are stored in the database. The
2431        <sgmltag>storage</sgmltag> tag has been deprecated and BASE will refuse
2432        to start if it finds a raw data type definitions with <code>storage="file"</code>.
2433      </para>
2434     
2435      <para>
2436        For backwards compatibility reasons, each <classname docapi="net.sf.basedb.core">Platform</classname>
2437        that can only store data in files will create "virtual" raw data type
2438        objects internally. These raw data types all return <constant>false</constant>
2439        from the <methodname>RawDataType.isStoredInDb()</methodname>
2440        method. They also have a back-link to the platform/variant that
2441        created it: <methodname>RawDataType.getPlatform()</methodname>
2442        and <methodname>RawDataType.getVariant()</methodname>. These two methods
2443        will always return <constant>null</constant> when called on a raw data type
2444        that can be stored in the database.
2445      </para>
2446     
2447      <itemizedlist>
2448        <title>See also</title>
2449        <listitem><xref linkend="data_api.platforms" /></listitem>
2450        <listitem><xref linkend="plugin_developer.other.datafiles" /></listitem>
2451        <listitem><xref linkend="appendix.rawdatatypes.platforms" /></listitem>
2452        <listitem>
2453          <xref linkend="appendix.incompatible.2.5" /> in
2454          <xref linkend="appendix.incompatible" />
2455        </listitem>
2456      </itemizedlist>
2457     
2458      <sect3 id="core_api.data_in_files.diagram">
2459        <title>Diagram of classes and methods</title>
2460        <figure id="core_api.figures.data_in_files">
2461          <title>Store data in files</title>
2462          <screenshot>
2463            <mediaobject>
2464              <imageobject>
2465                <imagedata 
2466                  align="center"
2467                  scalefit="1" width="100%"
2468                  fileref="figures/uml/corelayer.datainfiles.png" format="PNG" />
2469              </imageobject>
2470            </mediaobject>
2471          </screenshot>
2472        </figure>
2473       
2474        <para>
2475          This is rather large set of classes and methods. The ultimate goal
2476          is to be able to create links between a <classname docapi="net.sf.basedb.core">RawBioAssay</classname>
2477          / <classname docapi="net.sf.basedb.core">ArrayDesign</classname> and <classname docapi="net.sf.basedb.core">File</classname>
2478          items and to provide some metadata about the files.
2479          The <classname docapi="net.sf.basedb.core">FileStoreUtil</classname> class is one of the most
2480          important ones. It is intended to make it easy for plug-in (and other)
2481          developers to access the files without having to mess with platform
2482          or file type objects. The API is best described
2483          by a set of use-case examples.
2484        </para>
2485       
2486      </sect3>
2487     
2488      <sect3 id="core_api.data_in_files.ask">
2489        <title>Use case: Asking the user for files for a given item</title>
2490
2491        <para>
2492          A client application must know what types of files it makes sense
2493          to ask the user for. In some cases, data may be split into more than
2494          one file so we need a generic way to select files.
2495        </para>
2496       
2497        <para>
2498          Given that we have a <interfacename docapi="net.sf.basedb.core">FileStoreEnabled</interfacename>
2499          item we want to find out which <classname docapi="net.sf.basedb.core">DataFileType</classname>
2500          items that can be used for that item. The
2501          <methodname>DataFileType.getQuery(FileStoreEnabled)</methodname>
2502          can be used for this. Internally, the method uses the result from
2503          <methodname>FileStoreEnabled.getPlatform()</methodname>
2504          and <methodname>FileStoreEnabled.getVariant()</methodname>
2505          methods to restrict the query to only return file types for
2506          a given platform and/or variant. If the item doesn't have
2507          a platform or variant the query will return all file types
2508          that are associated with the given item type. In any case, we get a list
2509          of <classname>DataFileType</classname> items, each one representing a
2510          specific file type that we should ask the user about. Examples:
2511        </para>
2512
2513        <orderedlist>
2514        <listitem>
2515          <para>
2516          The <constant>Affymetrix</constant> platform defines <constant>CEL</constant>
2517          as a raw data file and <constant>CDF</constant> as an array design (reporter map)
2518          file. If we have a <classname docapi="net.sf.basedb.core">RawBioAssay</classname> the query will only return
2519          the CEL file type and the client can ask the user for a CEL file.
2520          </para>
2521        </listitem>
2522        <listitem>
2523          <para>
2524          The <constant>Generic</constant> platform defines <constant>PRINT_MAP</constant>
2525          and <constant>REPORTER_MAP</constant> for array designs. If we have
2526          an <classname docapi="net.sf.basedb.core">ArrayDesign</classname> the query will return those two
2527          items.
2528          </para>
2529        </listitem>
2530        </orderedlist>
2531     
2532        <para>
2533          It might also be interesting to know the currently selected file
2534          for each file type and if the platform has set the <varname>required</varname>
2535          flag for a particular file type. Here is a simple code example
2536          that may be useful to start from:
2537        </para>
2538     
2539        <programlisting language="java">
2540DbControl dc = ...
2541FileStoreEnabled item = ...
2542Platform platform = item.getPlatform();
2543PlatformVariant variant = item.getVariant();
2544
2545// Get list of DataFileTypes used by the platform
2546ItemQuery&lt;DataFileType&gt; query =
2547   DataFileType.getQuery(item);
2548List&lt;DataFileType&gt; types = query.list(dc);
2549
2550// Always check hasFileSet() method first to avoid
2551// creating the file set if it doesn't exists
2552FileSet fileSet = item.hasFileSet() ?
2553   null : item.getFileSet();
2554   
2555for (DataFileType type : types)
2556{
2557   // Get the current file, if any
2558   FileSetMember member = fileSet == null || !fileSet.hasMember(type) ?
2559      null : fileSet.getMember(type);
2560   File current = member == null ?
2561      null : member.getFile();
2562   
2563   // Check if a file is required by the platform
2564   PlatformFileType pft = platform == null ?
2565      null : platform.getFileType(type, variant);
2566   boolean isRequired = pft == null ?
2567      false : pft.isRequired();
2568     
2569   // Now we can do something with this information to
2570   // let the user select a file ...
2571}
2572</programlisting>
2573     
2574        <note>
2575          <title>Also remember to catch PermissionDeniedException</title>
2576          <para>
2577            The above code may look complicated, but this is mostly because
2578            of all checks for <constant>null</constant> values. Remember
2579            that many things are optional and may return <constant>null</constant>.
2580            Another thing to look out for is
2581            <exceptionname>PermissionDeniedException</exceptionname>:s. The logged in
2582            user may not have access to all items. The above example doesn't include
2583            any code for this since it would have made it too complex.
2584          </para>
2585        </note>
2586      </sect3>
2587     
2588      <sect3 id="core_api.data_in_files.link">
2589        <title>Use case: Link, validate and extract metadata from the selected files</title>
2590        <para>
2591          When the user has selected the file(s) we must store the links
2592          to them in the database. This is done with a <classname docapi="net.sf.basedb.core">FileSet</classname>
2593          object. A file set can contain any number of files. The only limitation
2594          is that it can only contain one file for each file type.
2595          Call <methodname>FileSet.setMember()</methodname> to store
2596          a file in the file set. If a file already exists for the given file type
2597          it is replaced, otherwise a new entry is created. The following
2598          program example assumes that we have a map where <classname docapi="net.sf.basedb.core">File</classname>:s
2599          are related to <classname docapi="net.sf.basedb.core">DataFileType</classname>:s. When all files
2600          have been added we call <methodname>FileSet.validate()</methodname>
2601          to validate the files and extract metadata.
2602        </para>
2603       
2604        <programlisting language="java">
2605DbControl dc = ...
2606FileStoreEnabled item = ...
2607Map&lt;DataFileType, File&gt; files = ...
2608
2609// Store the selected files in the fileset
2610FileSet fileSet = item.getFileSet();
2611for (Map.Entry&lt;DataFileType, File&gt; entry : files)
2612{
2613   DataFileType type = entry.getKey();
2614   File file = entry.getValue();
2615   fileSet.setMember(type, file);
2616}
2617
2618// Validate the files and extract metadata
2619fileSet.validate(dc, true);
2620</programlisting>
2621
2622        <para>
2623          Validation and extraction of metadata is important since we want
2624          data in files to be equivalent to data in the database. The validation
2625          and metadata extraction is done by the core when the
2626          <methodname>FileSet.validate()</methodname> is called.
2627          The process is partly pluggable since each <classname docapi="net.sf.basedb.core">DataFileType</classname> 
2628          can name a class that should do the validation and/or metadata extraction.
2629        </para>
2630
2631        <note>
2632          <para>
2633          The <methodname>FileSet.validate()</methodname> only validates
2634          the files where the file types have specified plug-ins that can
2635          do the validation and metadata extraction. The method doesn't
2636          throw any exceptions. Instead, all validation errors
2637          are returned a list of <classname>Throwable</classname>:s. The
2638          validation result is also stored for each file and can be access
2639          with <methodname>FileSetMember.isValid()</methodname> and
2640          <methodname>FileSetMember.getErrorMessage()</methodname>.
2641          </para>
2642        </note>
2643
2644        <para>
2645          Here is the general outline of what is going on in the core:
2646        </para>
2647
2648        <orderedlist>
2649        <listitem>
2650          <para>
2651          The core checks the <classname docapi="net.sf.basedb.core">DataFileType</classname> of all
2652          members in the file set and creates <classname docapi="net.sf.basedb.core.filehandler">DataFileValidator</classname>
2653          and <classname docapi="net.sf.basedb.core.filehandler">DataFileMetadataReader</classname> objects. Only one instance
2654          of each class is created. If the file set contains members which has the
2655          same validator or metadata reader, they will all share the same instance.
2656          </para>
2657        </listitem>
2658       
2659        <listitem>
2660          <para>
2661          Each validator/metadata reader class is initialised with calls to
2662          <methodname>DataFileHandler.setItem()</methodname> and
2663          <methodname>DataFileHandler.setFile()</methodname>.
2664          </para>
2665        </listitem>
2666       
2667        <listitem>
2668          <para>
2669          Each validator is called. The result of the validation is saved for each
2670          file and can be retreieved by <methodname>FileSetMember.isValid()</methodname>
2671          and <methodname>FileSetMember.getErrorMessage()</methodname>.
2672          </para>
2673        </listitem>
2674       
2675        <listitem>
2676          <para>
2677          Each metadata reader is called, unless the metadata reader is the same class
2678          as the validator and the validation failed. If the metadata reader is a
2679          different class, it is called even if the validation failed.
2680          </para>
2681        </listitem>
2682        </orderedlist>
2683
2684        <note>
2685          <title>Only one instance of each validator class is created</title>
2686          <para>
2687          The validation/metadata extraction is not done until all files have been
2688          added to the fileset. If the same validator/meta data reader is
2689          used for more than one file, the same instance is reused. Ie.
2690          the <methodname>setFile()</methodname> is called one time
2691          for each file/file type pair. The <methodname>validate()</methodname>
2692          and <methodname>extractMetadata()</methodname> methods are only
2693          called once.
2694          </para>
2695        </note>
2696       
2697        <para>
2698          All validators and meta data extractors should extend
2699          the <classname docapi="net.sf.basedb.core.filehandler">AbstractDataFileHandler</classname> class. The reason
2700          is that we may want to add more methods to the <interfacename docapi="net.sf.basedb.core.filehandler">DataFileHandler</interfacename>
2701          interface in the future. The <classname docapi="net.sf.basedb.core.filehandler">AbstractDataFileHandler</classname> will
2702          be used to provide default implementations for backwards compatibility.
2703        </para>
2704       
2705      </sect3>
2706     
2707      <sect3 id="core_api.data_in_files.import">
2708        <title>Use case: Import data into the database</title>
2709       
2710        <para>
2711          This should be done by existing plug-ins in the same way as before.
2712          A slight modification is needed since it is good if the importers
2713          are made aware of already selected files in the <classname docapi="net.sf.basedb.core">FileSet</classname>
2714          to provide good default values. The <classname docapi="net.sf.basedb.core">FileStoreUtil</classname>
2715          class is very useful in cases like this:
2716        </para>
2717       
2718        <programlisting language="java">
2719RawBioAssay rba = ...
2720DbControl dc = ...
2721
2722// Get the current raw data file, if any
2723List&lt;File&gt; rawDataFiles =
2724   FileStoreUtil.getGenericDataFiles(dc, rba, FileType.RAW_DATA);
2725File defaultFile = rawDataFiles.size() > 0 ?
2726   rawDataFiles.get(0) : null;
2727   
2728// Create parameter asking for input file - use current as default
2729PluginParameter&lt;File&gt; fileParameter = new PluginParameter&lt;File&gt;(
2730   "file",
2731   "Raw data file",
2732   "The file that contains the raw data that you want to import",
2733   new FileParameterType(defaultFile, true, 1)
2734);
2735</programlisting>
2736
2737      <para>
2738        An import plug-in should also save the file that was used to the file set:
2739      </para>
2740     
2741      <programlisting language="java">
2742RawBioassay rba = ...
2743// The file the user selected to import from
2744File rawDataFile = (File)job.getValue("file");
2745
2746// Save the file to the fileset. The method will check which file
2747// type the platform uses as the raw data type. As a fallback the
2748// GENERIC_RAW_DATA type is used
2749FileStoreUtil.setGenericDataFile(dc, rba, FileType.RAW_DATA,
2750   DataFileType.GENERIC_RAW_DATA, rawDataFile);
2751</programlisting>
2752
2753      </sect3>
2754     
2755      <sect3 id="core_api.data_in_files.experiments">
2756        <title>Use case: Using raw data from files in an experiment</title>
2757       
2758        <para>
2759          Just as before, an experiment is still locked to a single
2760          <classname docapi="net.sf.basedb.core">RawDataType</classname>. This is a design issue that
2761          would break too many things if changed. If data is stored in files
2762          the experiment is also locked to a single <classname docapi="net.sf.basedb.core">Platform</classname>.
2763          This has been designed to have as little impact on existing
2764          plug-ins as possible. In most cases, the plug-ins will continue
2765          to work as before.
2766        </para>
2767       
2768        <para>
2769          A plug-in (using data from the database that needs to check if it can
2770          be used within an experiment can still do:
2771        </para>
2772       
2773        <programlisting language="java">
2774Experiment e = ...
2775RawDataType rdt = e.getRawDataType();
2776if (rdt.isStoredInDb())
2777{
2778   // Check number of channels, etc...
2779   // ... run plug-in code ...
2780}
2781</programlisting>
2782       
2783        <para>
2784          A newer plug-in which uses data from files should do:
2785        </para>
2786       
2787        <programlisting language="java">
2788Experiment e = ...
2789DbControl dc = ...
2790RawDataType rdt = e.getRawDataType();
2791if (!rdt.isStoredInDb())
2792{
2793   // Check that platform/variant is supported
2794   Platform p = rdt.getPlatform(dc);
2795   PlatformVariant v = rdt.getVariant(dc);
2796   // ...
2797
2798   // Get data files
2799   File aFile = FileStoreUtil.getDataFile(dc, ...);
2800   
2801   // ... run plug-in code ...
2802}
2803</programlisting>
2804       
2805      </sect3>
2806     
2807    </sect2>
2808  </sect1>
2809
2810  <sect1 id="api_overview.query_api">
2811    <title>The Query API</title>
2812    <para>
2813      This documentation is only available in the old format.
2814      See <ulink url="http://base.thep.lu.se/chrome/site/doc/development/overview/query/index.html"
2815        >http://base.thep.lu.se/chrome/site/doc/development/overview/query/index.html</ulink>
2816    </para>
2817   
2818  </sect1>
2819 
2820  <sect1 id="api_overview.dynamic_and_batch_api">
2821    <title>Analysis and the Dynamic and Batch API:s</title>
2822    <para>
2823      This documentation is only available in the old format.
2824      See <ulink url="http://base.thep.lu.se/chrome/site/doc/development/overview/dynamic/index.html"
2825        >http://base.thep.lu.se/chrome/site/doc/development/overview/dynamic/index.html</ulink>
2826    </para>
2827  </sect1>
2828
2829  <sect1 id="api_overview.other_api">
2830    <title>Other useful classes and methods</title>
2831    <para>
2832      TODO
2833    </para>
2834  </sect1>
2835 
2836</chapter>
Note: See TracBrowser for help on using the repository browser.