source: trunk/doc/src/docbook/user/rawbioassays.xml @ 7640

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE sect1 PUBLIC 
    "-//Dawid Weiss//DTD DocBook V3.1-Based Extension for XML and graphics inclusion//EN" 
    "../../../../lib/docbook/preprocess/dweiss-docbook-extensions.dtd">
<!--
  $Id: rawbioassays.xml 7640 2019-03-11 13:13:29Z nicklas $

  Copyright (C) 2007 Peter Johansson, Nicklas Nordborg, Martin Svensson

  This file is part of BASE - BioArray Software Environment.
  Available at http://base.thep.lu.se/

  BASE is free software; you can redistribute it and/or
  modify it under the terms of the GNU General Public License
  as published by the Free Software Foundation; either version 3
  of the License, or (at your option) any later version.

  BASE is distributed in the hope that it will be useful,
  but WITHOUT ANY WARRANTY; without even the implied warranty of
  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
  GNU General Public License for more details.

  You should have received a copy of the GNU General Public License
  along with BASE. If not, see <http://www.gnu.org/licenses/>.
-->

<sect1 id="experiments_analysis.rawbioassay">
  <?dbhtml filename="rawbioassays.html" ?>
  <title>Raw bioassays</title>
  <para>
    A <guilabel>Raw bioassay</guilabel> is the representation
    of the result of analyzing data from the physical bioassay
    down to the point where we have a file or a set of files
    containing measurements per feature (eg. spot, gene, etc.)
    for a single sample or extract. Further analysis is usually
    needed before we can say something about individual features
    or samples and how they relate to each other. This
    kind of analisys is done in <guilabel>Experiments</guilabel>.
    See <xref linkend="experiments_analysis.experiments" />.
  </para>
  
  <para>
    The term <guilabel>Raw bioassay</guilabel> is bit misleading since the
    real "raw data" is actually the images from a microarray scan or the
    output from a sequencer. For historical reasons we have chosen to keep
    the term raw bioassay since this represents the first possibility for
    a transition between file-base data and database-stored data. Typically, 
    all pre-rawbioassay analysis is done outside of BASE, and although
    we now have the possibility to track this in detail, it will
    probably remain so for some time in the future. See
    <xref linkend="experiments_analysis.derivedbioassays" />.
  
  </para>
  
  <sect2 id="experiments_analysis.rawbioassay.create">
    <title>Create raw bioassays</title>
    <para>
      Creating a new raw bioassay is a two- or three-step process:
    </para>
  
    <orderedlist>
      <listitem>
        <para>
          Create a new raw bioassay item with the &gbNew; button in the raw bioassays list view.
          It is also possible to create raw bioassays from the derived bioassays
          list- and single view- page.
        </para>
      </listitem>
      <listitem>
        <para>
        Upload the file(s) with the raw data and attach them to the
        raw bioassay.
        </para>
      </listitem>
      <listitem>
        <para>
          The used platform may require that data is imported to the database.
          See <xref linkend="import_data" />. If the platform is a
          file-only platform, this step can be skipped.
        </para>
      </listitem>
    </orderedlist>
  
    <note>
      <title>Supported file formats</title>
      BASE has built-in support for most file formats where the data comes
      in a tab-separated (or similar) form. Data for one raw bioassay
      must be in a single file. Support for other file formats
      may be added through plug-ins.
    </note>
  </sect2>
  
  <sect2 id="experiments_analysis.rawbioassay.properties">
    <title>Raw bioassay properties</title>
    
    <figure
      id="experiments_analysis.figures.rawbioassay.edit">
      <title>Raw bioassay properties</title>
      <screenshot>
        <mediaobject>
          <imageobject>
            <imagedata
              fileref="figures/rawbioassay_edit.png" format="PNG" />
          </imageobject>
        </mediaobject>
      </screenshot>
    </figure> 

    <helptext external_id="rawbioassay.edit" title="Edit raw bioassay">
    
    <variablelist>
      <varlistentry>
        <term><guilabel>Name</guilabel></term>
        <listitem>
          <para>
            The name of the raw bioassay.
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term>
          <guilabel>Platform</guilabel>
        </term>
        <listitem>
          <para>
            Select the platform / variant used for the
            raw bioassay. The selected options affects which
            files that can be selected on the <guilabel>Data files</guilabel>
            tab. If the platform supports importing data to the database
            you must also select a <guilabel>Raw data type</guilabel>.
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><guilabel>Raw data type</guilabel></term>
        <listitem>
          <para>
            The type of raw data. This option is disabled for file-only
            platforms and for platforms that are locked to a specific
            raw data type. This cannot be changed after raw data has been
            imported. <nohelp>See
            <xref linkend="experiments_analysis.rawdatatypes" />.</nohelp>
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><guilabel>Parent bioassay</guilabel></term>
        <listitem>
          <para>
            The derived bioassay that is the parent of this
            raw bioassay.
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><guilabel>Parent extract</guilabel></term>
        <listitem>
          <para>
            The extract which this raw bioassay has measured. This is normally selected
            among the extracts that are linked with the physical bioassay that this
            raw bioassay is coming from. Selecting the correct extract is important if the
            physical bioassay contains more than one extract, since otherwise it may affect
            how annotations are inherited and used in downstream analysis.
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><guilabel>Array design</guilabel></term>
        <listitem>
          <para>
            The array design used on the array slide (optional).
            If an array design is specified
            the import will verify that the raw data has
            the same reporter on the same position. This
            prevents mistakes but also speed up analysis
            since some optimizations can be used when assigning 
            positions in bioassay sets.
            The array design can be changed after raw data has been 
            imported, but this triggers a new validation. If the raw data
            is stored in the database, the features on the new array design must 
            match the the raw data. The verification can use three different methods:
          </para>
          
          <itemizedlist>
          <listitem>
            <para>
            Coordinates: Verify block, meta-grid, row and column coordinates.
            </para>
          </listitem>
          <listitem>
            <para>Position: Verify the position number.</para>
          </listitem>
          <listitem>
            <para>
            Feature ID: Verify the feature ID. This option can only be used
            if the raw bioassay is currently connected to an array design that
            has feature ID values already.
            </para>
          </listitem>
          </itemizedlist>
          <para>
            In all three cases it is also verified that the reporter of the raw
            data matches the reporter of the features.
          </para>

          <para>
            For Affymetrix data, the
            CEL file is validated against the CDF file of the new array design.
            If the validation fails, the array design is not changed.
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><guilabel>Software</guilabel></term>
        <listitem>
          <para>
            The software used to generate the raw data (optional).
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><guilabel>Protocol</guilabel></term>
        <listitem>
          <para>
            The protocol used when generating the raw data (optional). 
            Software parameters may be registered as part of
            the protocol.
          </para>
        </listitem>
      </varlistentry>
      <varlistentry>
        <term><guilabel>Description</guilabel></term>
        <listitem>
          <para>
            A description of the raw bioassay (optional).
          </para>
        </listitem>
      </varlistentry>
    </variablelist>
    
    <seeother>
      <other external_id="datafiles.edit">Data files</other>
      <other external_id="annotations.edit">Annotations</other>
      <other external_id="annotations.edit.inerited">Inherit annotations</other>
    </seeother>
    </helptext> 
    
    
    <para>
      The <guilabel>Data files</guilabel> tab allows BASE users to select
      files that contains data for the raw bioassay. 
      Read more about this in <xref linkend="platforms.selectfiles" />.
    </para>
  
    <para>
      The <guilabel>Annotations</guilabel> tab allows BASE users to use 
      annotation types to refine bioassay description. More about annotating items 
      can be read in <xref linkend="annotations.annotating" />
    </para>
        
    <para>
      This <guilabel>Inherited annotations</guilabel> tab contains a list of those annotations
      that are inherited from the bioassay's parents. Information about working with inherited 
      annotations can be found in <xref linkend="annotations.inheriting" />.
    </para>
    
  </sect2>
  
  <sect2 id="experiments_analysis.rawbioassay.rawdata">
    <title>Import raw data</title>
    <para>
      Depending on the platform, raw data may have to be imported after 
      you have created the raw bioassay item. This section doesn't apply
      to file-only platforms. The import is handled by plug-ins. To start
      the import click on the <guibutton>Import&hellip;</guibutton>
      button on the single-item view for the raw bioassay.
      If this button does not appear it may be because no file
      format has been specified for the raw data type used by the
      raw bioassay or that the logged in user does not have permission
      to use the import plug-in or file format.
      See <xref linkend="import_data" /> for more 
      information.
    </para>
    
    <note>
      <title>File-only platforms</title>
      File-only platforms, such as Affymetrix, is handled differently and data is not
      imported into the database.
    </note>
    
  </sect2>

  <sect2 id="experiments_analysis.rawdatatypes">
    <title>Raw data types</title>
    
    <para>
      A raw data type defines the types of measured values that can be stored
      for individual features in the database. Usually this includes some
      kind of foreground and background intensity values. The number and meaning
      of the values usually depends on the hardware and software used to analyze 
      the data from the experiment. Many tools provide mean and median values,
      standard deviations, quality control information, etc. Since there are so
      many existing tools with many different data file formats BASE uses a 
      separate database table for each raw data type to store data. The raw data
      tables have been optimized for the type of raw data they can hold and only
      has the columns that are needed to store the data. BASE ships with a large
      number of pre-defined raw data types. An administrator may also define
      additional raw data type. See <xref linkend="appendix.rawdatatypes" /> 
      for more information.
    </para>
    
    <sect3 id="experiments_analysis.fileonly">
      <title>File-only platforms</title>
      <para>
        In some cases it doesn't make sense to import any data into the
        database. The main reason is that performance will suffer as the
        number of entries in the database gets higher. A typical Genepix file 
        contains ~55K spots while an Affymetrix file may have millions. 
      </para>
      <para>
        The drawback of keeping the data in files is that none of the generic
        tools in BASE can read it. Special plug-ins must be developed for each
        type of data file that can be used to analyze and visualize the data.
        For the Affymetrix platform there are implementations of the RMAExpress
        and Plier normalizations available on the BASE plug-ins web site. 
        BASE also ships with built-in plug-ins for extracting metadata from 
        Affymetrix CEL and CDF files (ie. headers, number of spots, etc).
      </para>
      <para>
        Users of other file-only platforms should check the BASE plug-ins
        website for plug-ins related to their platform. If they can't
        find any we recommend that they try to find other users of the same 
        platform and try to cooperate in developing the required tools and
        plug-ins.
      </para>
    </sect3>
    
  </sect2>

</sect1>