source: trunk/doc/src/docbook/overview/features.xml @ 5845

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE chapter PUBLIC 
    "-//Dawid Weiss//DTD DocBook V3.1-Based Extension for XML and graphics inclusion//EN" 
    "../../../../lib/docbook/preprocess/dweiss-docbook-extensions.dtd">

<!--
  $Id: features.xml 5845 2011-11-01 16:58:55Z jari $

  Copyright (C) 2008, 2011 Jari Häkkinen

  This file is part of BASE - BioArray Software Environment.
  Available at http://base.thep.lu.se/

  BASE is free software; you can redistribute it and/or
  modify it under the terms of the GNU General Public License
  as published by the Free Software Foundation; either version 3
  of the License, or (at your option) any later version.

  BASE is distributed in the hope that it will be useful,
  but WITHOUT ANY WARRANTY; without even the implied warranty of
  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
  GNU General Public License for more details.

  You should have received a copy of the GNU General Public License
  along with BASE. If not, see <http://www.gnu.org/licenses/>.
-->

<chapter id="features" chunked="0">
  <?dbhtml filename="features.html"?>
  <title>BASE features</title>

  <para>
    The BASE application features many components; MIAME compliance,
    multi-user, data sharing, data access management, array and
    biomaterial LIMS, multiple array platforms, RNAseq sequencing
    support, extensibility, configurable plug-ins, annotation
    customisation, streamlined access to analysis tools, integration
    of <ulink url='http://www.tm4.org/mev/'>MultiExperiment Viewer
    (MeV)</ulink>, web services API, and more. To support all
    components the underlying relational database has grown to become
    very large and complex, especially since BASE itself works with
    objects posing additional database tables to keep track of objects
    stored in a relational database. Thus, rather than trying to 
    describe every feature in detail here, we highlight some of the 
    more important features.
  </para>

  <sect1 id="features.webinterface">
    <title>Web interface</title>

    <para>
      The entire system is accessed through a web-interface over the
      Internet using a standard web browser, such as Firefox, Safari,
      Opera, or Internet Explorer. Access privileges to a particular
      BASE installation are managed by personal accounts through the
      web-interface. A local administrator creates new user accounts
      with specific roles and access privileges and has an overall
      managerial responsibility for an individual BASE
      installation. With exception for the administrator with global
      data access, individual users have sole access to and control
      their inputted data. Users have the possibility to share data
      they own (or have share credentials for) to other users of the
      same BASE installation.
    </para>

  </sect1>

  <sect1 id="features.datamangement">
    <title>Information and annotation management</title>

    <para>
      BASE features a biomaterial LIMS tracking biological material
      from its source to hybridisation/sequencing and ultimately to
      raw data and analysis. All events throughout sample handling are
      tracked and information on used and remaining quantities,
      physical sample locations, quality control information, and
      sample relations is stored in BASE. Racks or boxes holding 
      biomaterials can be created as BioPlates and plate events are
      easily performed for extraction or labelling events. Although 
      becoming less commonly used, the array production LIMS of 
      previous BASE versions is retained to support researchers 
      with spotting facilities, e.g., protein array production and 
      BAC array printing that may not be commercially available.
    </para>

    <para>
      Events in biomaterial and array LIMS are annotable with
      protocols and event dates, and most items can be annotated with
      customisable annotation types such as floats, integers, dates,
      and Boolean flags. Change history for biomaterial items is available 
      if configured and can be used to track modifications in the database. 
      Annotations are either free form or from a preset list of values, 
      and can be marked as required for MIAME compliance. The annotation 
      system is searchable and the user can select any annotations to be 
      an experimental factors in analysis whereby it becomes availabe to 
      analysis plugins and plot-tools.
    </para>

  </sect1>

  <sect1 id="features.sharingandprivacy">
    <title>Data sharing and privacy</title>

    <para>
      One of the important features of BASE is its capabilities as a
      local data repository. The repository functionality is amended
      with data grouping, sharing, and privacy policies. A BASE
      project is used to group items (biomaterial, raw data, and
      experiments) into a logical entity, and a BASE experiment is a
      collection of bioassays, e.g., array data, grouped logically together
      for further analysis. All items can co-exist in several projects
      and experiments without any unnecessary copying of information.
    </para>

    <para>
      Data privacy is guarded by the data owner and BASE allows the
      owner to set data access rules. To this end, each item in BASE
      is owned by a user enabling him to share data with
      colleagues. The grouping of data in projects allows the data
      owner to simply include other users in a project in order to
      share data. Each item can have different access levels even
      within a project, and project members can have different
      privileges. The data access rules are very flexible and can be
      overwhelming since access levels on almost any item can be
      individually set. However, using projects, the proper access
      levels can be set at a single point of interaction.
    </para>

  </sect1>

  <sect1 id="features.directorystructure">
    <title>File and directory structure</title>

    <para>
      BASE has an integrated file system to provide the possibility for
      researchers to collect all data files related to a project in
      one single storage location. Data files are uploaded using a web
      browser or an ftp client. The file storage is an integral part
      of a strategy to store all experiment relevant data in BASE,
      even data types not already supported in analysis. Collecting
      all data allows future reuse of the data as more data are
      produced, and new analysis tools becomes available.
    </para>

  </sect1>

  <sect1 id="features.plugininfrastructure">
    <title>Plugin and extension infrastructure</title>

<!--
Analysis, extensions, and plug-ins
-->
    <para>
      BASE features a hierarchically organised analysis interface that
      allows data filtering, normalisation, transformation, and other
      analyses. Parameters and settings are automatically stored for
      each step in the analysis. The selection of analysis tools
      depends on array type and available plug-ins where a wide range
      of tools are pre-installed with BASE, and optional plug-ins can
      be downloaded from the <ulink
      href='http://baseplugins.thep.lu.se'>BASE plug-in site
      </ulink>. BASE capitalise from other software tools, such as
      MEV, by integrating them into the user interface. Such
      integration provide streamlined access to analysis modules in
      external tools. BASE even features a rudimentary manual
      transform creator that enables researchers to add analysis steps
      within the hierarchical overview of analysis performed
      independently of BASE. The transform creator enables storage of
      result files and parameter information for archival, tracking,
      and sharing purposes.
    </para>

    <para>
      The analysis of genomics data is continuously evolving with new
      methods and techniques. To this end BASE provides extensions and
      plug-in programming interfaces (APIs) to enable straightforward
      additions of new analysis tools. The use of the APIs is well
      documented and there are numerous examples on how to create
      extensions. The MEV and ftp-server integrations all utilise the
      extension mechanism, and the automatically generated overview
      plots available in the experimental analysis view are also
      extensions. The plug-in API is used for all data imports and
      exports, and most analysis tools, providing new developers a lot
      of example code to examine when they create BASE plug-ins.
    </para>

  </sect1>

  <sect1 id="features.batchdata">
    <title>Batch upload and download of data</title>

    <para>
      File, annotation, and item upload can be done asynchronously as
      data are generated or information becomes available. To relieve
      researchers from the tedious task of entering data one by one a
      set of batch import were created; the information generated
      throughout the experimental work is uploaded to BASE in plain
      tab-separated files. These files are supplied to batch importer
      plug-ins that parse the files and create items and associations
      according to the information in the files. The same plug-ins can
      be used to batch update many items. Similarly, annotating items
      is done by creating tab-separated files with annotation
      information, uploading these to BASE, and loading the file
      content into the database using annotation importers. If needed,
      annotations are easily updated with the same mechanism.
    </para>

    <para>
      Files uploaded to BASE are stored in the directory structure
      within BASE and multiple files are easily transferred to BASE
      either packaged in compressed files with a single upload action,
      or by using an ftp client supporting transfer of file
      structures. Similarly, downloading multiple files is
      straightforward either using an ftp client or by a single click
      in the BASE web interface. Download of items is done through
      item listing views enabling users to filter and select what
      information should be downloaded.
    </para>

  </sect1>

  <sect1 id="features.supportedarrays">
    <title>Supported array platforms and raw data formats</title>

    <para>
      There are many types of microarrays, techniques, and brands
      available for researchers; one- or two-channel hybridizations,
      spotted cDNA/oligo arrays, Affymetrix (GeneChip), Illumina (SNP,
      DASL, WGEX, microRNA), aCGH, SNP, tiling arrays, and many
      more. Data are produced in different file formats that must be
      treated differently depending on type.
    </para>

    <para>
      Many platforms and experimental setups are supported in
      downstream analysis but some microarray techniques cannot
      currently be analysed within BASE simply because lack of support
      in available plug-ins. The problem is resolved by creating new,
      or extending available, plug-ins that add analysis capabilities
      of platforms and techniques not readily supported in
      analysis. Extending analysis capabilities to new technologies is
      only a matter of local needs and resources. We add support for
      platforms in use at the Lund University microarray facility and
      make our tools freely available to the community.
    </para>

    <para>
      For two channel array platforms it is straightforward to
      customize BASE for a specific array platform, the platform
      simply needs to be adapted to the (BASE) Generic platform. The
      adaptation is to create a raw data format definition and to
      configure raw data importers, or make use of already available
      raw data formats. However, it is not always possible to make an
      natural mapping of a platform to the Generic platform. Platforms
      such as Affymetrix and Illumina platforms cannot naturally be
      mapped on to the Generic two channel platform. For Affymetrix,
      BASE comes with a specific Affymetrix platform and Illumina can
      be supported by customizing BASE (go to the <ulink
      url="http://baseplugins.thep.lu.se/wiki/net.sf.basedb.illumina">
      Illumina package</ulink> web site for more information on adding
      Illumina support to BASE).
    </para>

    <para>
      How to adapt new array platforms to the Generic platform format
      or how to create a new platform type in BASE can be read
      elsewhere in this document. Here we list different array
      platforms used in BASE and also list raw data types supported
      by BASE. However, not all platforms nor raw data types listed
      below are available out-of-the box and a BASE administrator must
      customize his local BASE installation for their specific
      need. What comes pre-configured when BASE is installed is
      indicated in the lists below.
    </para>

    <sect2 id="features.supportedarrays.platforms">
      <title>Vendor specific and custom printing array
      platforms</title>

      <para>
        Not all array platforms listed below are available by
        default. The comments to specific platforms explain how to
        enable the use of the array platform in BASE. In some cases
        there is no confirmed usage of a platform but we believe it
        has been tested by anonymous users.
      </para>

      <variablelist>
        <varlistentry>
          <term>Affymetrix</term>
          <listitem>
            <para>
              The Affymetrix platform comes pre-configured with a
              new BASE installation. Affymetrix platform in this
              context are the Affymetrix expression arrays. So far
              there has been no reason for expanding the Array
              platform to other chip-types. In principle any
              Affymetrix chip type can be stored in BASE but current
              plug-ins will always assume that expression data is
              stored and analyzed. This can be resolved by adding
              variants of the Affymetrix platform but the Lund BASE
              team currently has no plans to create Affymetrix
              variants.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>Agilent</term>
          <listitem>
            <para>
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>Custom printing</term>
          <listitem>
            <para>
              The array layout options are endless and imagination is
              the only limitation ... almost. BASE can import many
              in-house array designs and platforms. The custom arrays
              usually fall back on one of the raw data types already
              available such as GenePix.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>Illumina</term>
          <listitem>
            <para>
              There are several variants of the Illumina platform. Using
              several variants allows BASE to adapt its handling of
              different Illumina chip types. Illumina platform support
              is not included in a standard BASE installation but there
              is
              a <ulink url="http://baseplugins.thep.lu.se/wiki/net.sf.basedb.illumina">
                Illumina package</ulink> available for seamless
              integration of the Illumina array platform to BASE.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>ImaGene</term>
          <listitem>
            <para>
              No successful use confirmed but ImaGene raw data is
              available in BASE.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>Unlisted</term>
          <listitem>
            <para>
              In principle any platform generating a matrix of data can
              be imported into BASE. Simply utilize the available raw
              data formats and data importers.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>

    </sect2>

    <sect2 id="features.supportedarrays.rawdatatypes">
      <title>Available raw data types</title>

      <para>
        Raw data comes in many different formats. These formats are
        usually defined by scanner software vendors and BASE must keep
        track of the different formats for analysis and plotting. BASE
        supports many formats out the box, but some formats need to
        be added manually by the BASE administrator (indicated in the
        list below).
      </para>

      <variablelist>

        <varlistentry>
          <term>Affymetrix</term>
          <listitem>
            <para>
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>AIDA</term>
          <listitem>
            <para>
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>Agilent</term>
          <listitem>
            <para>
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>BZScan</term>
          <listitem>
            <para>
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>ChipSkipper</term>
          <listitem>
            <para>
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>GenePix</term>
          <listitem>
            <para>
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>GeneTAC</term>
          <listitem>
            <para>
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>Illumina</term>
          <listitem>
            <para>
              The Illumina array platform usage is recommended to be
              based on the <emphasis>Illumina Bead Summary
              (IBS)</emphasis> raw data format below.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>Illumina Bead Summary (IBS)</term>
          <listitem>
            <para>
              Not available in BASE directly but it is added with
              the <ulink url="http://baseplugins.thep.lu.se/wiki/net.sf.basedb.illumina">
              Illumina plug-in</ulink> that adds Illumina array
              platform support to BASE.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>ImaGene</term>
          <listitem>
            <para>
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>QuantArray Biotin</term>
          <listitem>
            <para>
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>QuantArray Cy</term>
          <listitem>
          <para>
          </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>SpotFinder</term>
          <listitem>
            <para>
            </para>
          </listitem>
        </varlistentry>

      </variablelist>

    </sect2>

  </sect1>

  <sect1 id="features.repositoryandstandards">
    <title>Repository and standards</title>

    <para>
      The Microarray Gene Expression Data Society (MGED) develops and
      maintains standards for data acquisition, representation, and
      interchange such as the MIAME guidelines, the MAGE-TAB
      interchange format, and the MGED Ontology for microarray
      experiments. BASE does not enforce the use of the MGED standards
      but support storage of information required by MIAME. BASE has
      an experiment item overview functionality useful for validating
      information related to experiments. The validation level is user
      selectable of which the option regarding MIAME compliance is
      most relevant here. When users or server administrators create
      annotation types in BASE these annotation values can be marked
      as required by MIAME and optionally defined to be a list of
      pre-defined values from a controlled vocabulary. Validation will
      check for inconsistencies and report errors, and give the user
      an opportunity to fix issues immediately or later. After
      resolving the issues raised by the validation, data can be
      exported for submission to public repositories such as
      ArrayExpress, Gene Expression Omnibus (GEO), and CIBEX.
    </para>

  </sect1>

</chapter>