source: trunk/doc/src/docbook/userdoc/experiments.xml @ 3651

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE sect1 PUBLIC 
    "-//Dawid Weiss//DTD DocBook V3.1-Based Extension for XML and graphics inclusion//EN" 
    "../../../../lib/docbook/preprocess/dweiss-docbook-extensions.dtd">
<!--
  $Id: experiments.xml 3651 2007-08-10 10:17:30Z jari $

  Copyright (C) Authors contributing to this file.

  This file is part of BASE - BioArray Software Environment.
  Available at http://base.thep.lu.se/

  BASE is free software; you can redistribute it and/or
  modify it under the terms of the GNU General Public License
  as published by the Free Software Foundation; either version 2
  of the License, or (at your option) any later version.

  BASE is distributed in the hope that it will be useful,
  but WITHOUT ANY WARRANTY; without even the implied warranty of
  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
  GNU General Public License for more details.

  You should have received a copy of the GNU General Public License
  along with this program; if not, write to the Free Software
  Foundation, Inc., 59 Temple Place - Suite 330,
  Boston, MA  02111-1307, USA.
-->

<sect1 id="experiments_analysis.experiments">
  <title>Experiments</title>
  <para>
    Experiments are the starting point for analysis. When you have
    uploaded and imported your raw data, collected and registered all 
    information and annotations about samples, hybridizations, and other
    items, it is time to collect everything in an experiment.
  </para>
  
  <para>
    To create a new experiment you can either mark one ore more
    raw biossays on the raw bioassays list view and use the 
    <guibutton>New experiment</guibutton> button. You can also
    create a new experiment from the experiments list view.
  </para>

  <sect2 id="experiments_analysis.experiment.properties">
    <title>Experiment properties</title>
    
    <figure id="experiments_analysis.figures.experiment_properties">
      <title>Create experiment</title>
      <screenshot>
        <mediaobject>
          <imageobject><imagedata 
            fileref="figures/experiment_properties.png" format="PNG"
            /></imageobject>
        </mediaobject>
      </screenshot>
    </figure>         
    
    
    <helptext external_id="experiment.edit" title="Edit experiment">
    
      <variablelist>
      <varlistentry>
        <term><guilabel>Name</guilabel></term>
        <listitem>
          <para>
            The name of the experiment.
          </para>
        </listitem>
      </varlistentry>
      
      <varlistentry>
        <term><guilabel>Raw data type</guilabel></term>
        <listitem>
          <para>
            The raw data type to use in the experiment. All raw bioassays
            must have raw data with this type.
          </para>
        </listitem>
      </varlistentry>

      <varlistentry>
        <term><guilabel>Directory</guilabel></term>
        <listitem>
          <para>
            A directory in the BASE file system where plug-ins 
            can save files that are generated during the analysis.
            This is optional and if not given the plug-ins must ask
            about a directory each time they need it. Use the 
            <guibutton>Select</guibutton> button to browse the file system
            or create a new directory.
          </para>
        </listitem>
      </varlistentry>
      
      <varlistentry>
        <term><guilabel>Raw bioassays</guilabel></term>
        <listitem>
          <para>
            The raw bioassays you want to analyze in this experiment. If
            you created the experiment from the raw bioassays list the
            selected raw bioassays are already filled in. Use the 
            <guibutton>Add raw bioassays</guibutton> button to add more
            raw bioassays or the &gbRemove; button
            to remove the selected raw bioassays from the list.
          </para>
        </listitem>
      </varlistentry>
      
      <varlistentry>
        <term><guilabel>Description</guilabel></term>
        <listitem>
          <para>
            A description of the experiment.
          </para>
        </listitem>
      </varlistentry>

      </variablelist>
      
      <para>
        Click on the &gbSave; button to save the
        changes or on &gbCancel; to abort.
      </para>
      
      <seeother>
        <other external_id="experiment.edit.publication">Publication</other>
        <other external_id="experiment.edit.factors">Experimental factors</other>
      </seeother>
    </helptext>
    
    <sect3 id="experiments_analysis.experiment.publication">
      <title>The publication tab</title>
      
      <helptext external_id="experiment.edit.publication" 
        title="Experiment publication">

        <para>
          On this tab you can enter information about a publication
          that is the result of the experiment. All of this information
          is optional.
        </para>
        
        <variablelist>
        <varlistentry>
          <term><guilabel>PubMedId</guilabel></term>
          <listitem>
            <para>
              The ID of the publication in the 
              <ulink url="http://www.ncbi.nlm.nih.gov/entrez/query/static/overview.html"
                >PubMed</ulink> database.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term><guilabel>Title</guilabel></term>
          <listitem>
            <para>
              The title of the publication.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term><guilabel>Publication date</guilabel></term>
          <listitem>
            <para>
              The date the article was published. Use the 
              <guibutton>Calendar</guibutton> button to select
              a date from a popup window.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term><guilabel>Abstract</guilabel></term>
          <listitem>
            <para>
              The article abstract.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term><guilabel>Experiment design</guilabel></term>
          <listitem>
            <para>
              An explanation of the experiment design.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term><guilabel>Experiment type</guilabel></term>
          <listitem>
            <para>
              A description of the experiment type.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term><guilabel>Affiliations</guilabel></term>
          <listitem>
            <para>
              Partners and other related organisations that
              have helped with the experiment.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term><guilabel>Authors</guilabel></term>
          <listitem>
            <para>
              The list of authors of the publication.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term><guilabel>Publication</guilabel></term>
          <listitem>
            <para>
              The body text of the publication.
            </para>
          </listitem>
        </varlistentry>
        </variablelist>
        
        <para>
          Click on the &gbSave; button to save the
          changes or on &gbCancel; to abort.
        </para>
      
        <seeother>
          <other external_id="experiment.edit">Experiment properties</other>
          <other external_id="experiment.edit.factors">Experimental factors</other>
        </seeother>
      </helptext>
    </sect3>
    
  </sect2>
  
  <sect2 id="experiments_analysis.experiment.factors">
    <title>Experimental factors</title>
    
    <helptext external_id="experiment.edit.factors" 
      title="Experimental factors">
    
      <para>
        The experimental factors of an experiment are the variables
        you are studying in the experiment. Typically the value of
        an experimental factor is varied between samples or group of samples.
        Different treatment methods is an example of an experimental factor.
      </para>
      
      <para>
        In the BASE world an experimental factor is the same as an annotation
        type. Since you probably have lots of annotations on your items that are not
        relevant for the experiment you must select the annotations types
        that should make up the experimental factors of the experiment.
      </para>
      
      <para>
        Use the <guibutton>Add annotation types</guibutton> button to
        select the annotation types that should be used as experimental 
        factors. The &gbRemove; button removes the 
        selected annotation types.
      </para>

      <para>
        Click on the &gbSave; button to save the
        changes or on &gbCancel; to abort.
      </para>
      
      <para>
        To be able to use the values of the experimental factors
        in the analysis of your data the values must be accessible
        from the raw bioassays. Since most of your annotations are
        probably made at the sample or biosource level the raw bioassays 
        must inherit those annotations. 
        <nohelp>Read <xref linkend="annotations.inheriting" /> 
        for more information about this.</nohelp>
      </para>
      
      <tip>
        <para>
        Use the <guilabel>Experiment overview</guilabel> function to 
        verify that all your raw bioassays has been annotated or 
        inherited values for all experimental factors. If not, you
        should do that before starting with the analysis.
        </para>
      </tip>
      
      <seeother>
        <other external_id="experiment.edit">Experiment properties</other>
        <other external_id="experiment.edit.publication">Publication</other>
        <other external_id="experiment.overview">Experiment overview</other>
        <other external_id="annotations.edit.inerited">Inherit annotations</other>
      </seeother>
      
    </helptext>
  </sect2>

  
  <sect2 id="experiments_analysis.experiment.overview">
    <title>Experiment overview</title>

    <helptext external_id="experiment.overview" 
      title="Experiment overview">
    
    <para>
      With the <guilabel>Experiment overview</guilabel> 
      function you can get an overview of all hybridizations,
      extracts, samples, annotations, etc. used in an 
      experiment. The overview includes a lot of checks
      that validates your experimental setup for missing
      or possibly incorrect information.
    </para>
    
    <nohelp>
    <para>
      You can access the overview of an experiment by navigating
      to the single-item view of the experiment you are interested in.
      Then, switch to the <guilabel>Overview</guilabel> tab that
      is present on that page. Here is an example of what is displayed:
    </para>
    
    <figure id="experiments_analysis.figures.experimentoverview">
      <title>The experiment overview</title>
      <screenshot>
        <mediaobject>
          <imageobject><imagedata 
            scalefit="1" width="100%"
            fileref="figures/experiment_overview.png" format="PNG"
            /></imageobject>
        </mediaobject>
      </screenshot>
    </figure>
    </nohelp>
    
    <para>
      The page is divided into three sections:
    </para>
    
    <itemizedlist>
      <listitem>
        <para>
        To the left is a tree displaying all items
        in the experiment and how they are linked to
        each other.
        </para>
      </listitem>
      
      <listitem>
        <para>
        The lower right shows a list of warnings and error
        messages that was found when validating the experiment.
        <nohelp>
        In the example you can see that we have failed to
        specify a value for the <guilabel>Temperature</guilabel>
        protocol parameter for one of the samples.
        </nohelp>
        </para>
      </listitem>
        
      <listitem>
        <para>
          The upper right shows information about the
          currently selected item in the tree. This part also
          contains more information errors or warnings for this
          item. It may also present you with one or more suggestions
          about how to fix the problem and with a link that
          takes you to the most probable location where you can fix
          the error or warning.
        </para>
        
        <note>
          <title>No links?</title>
          If you do not have permission to change things no links
          will be shown.
        </note>
        
      </listitem>
        
    </itemizedlist>
    
    <seeother>
      <other external_id="experiment.overview.validationoptions"
        >Validation options</other>
      <other external_id="experiment.overview.fixfailures"
        >How to fix validation failures</other>
    </seeother>
    
    </helptext>
    
    <sect3 id="experiments_analysis.validationoptions">
      <title>Validation options</title>
      <para>
        Click on the <guibutton>Validation options</guibutton>
        button in the toolbar to open the <guilabel>Validation
        options</guilabel> dialog.
      </para>
      
      <figure id="experiments_analysis.figures.validationoptions">
        <title>Validation options</title>
        <screenshot>
          <mediaobject>
            <imageobject><imagedata 
              fileref="figures/validation_options.png" format="PNG"
              /></imageobject>
          </mediaobject>
        </screenshot>
      </figure>         
      
      <helptext external_id="experiment.overview.validationoptions" 
        title="Validation options">
        <para>
          The validation procedure is highly 
          configurable and you can select what you want to
          ignore, or something should be displayed as an error
          or warning.
        </para>
        
        <variablelist>
          <varlistentry>
            <term>
            <guilabel>Presets</guilabel></term>
            <listitem>
              <para>
              The list contains predefined and
              user defined validation options.
              Use the <guibutton>Save as&hellip;</guibutton>
              button to save the current options as a user defined
              preset. The <guibutton>Remove&hellip;</guibutton>
              button is used to remove the currently selected
              preset. Predefined presets cannot be deleted.
              </para>
            </listitem>
          </varlistentry>
          
          <varlistentry>
            <term><guilabel>Project defaults</guilabel></term>
            <listitem>
              <para>
              The options in this section are used to check
              if your experiment uses the same values as set
              by the project default values of the currently active
              project<nohelp>
              (see <xref linkend="project_permission.projects" />)</nohelp>. 
              If no project is active or if the active project does not
              have default values these options are ignored.
              </para>
            </listitem>
            
          </varlistentry>
          
          <varlistentry>
            <term><guilabel>Missing items</guilabel></term>
            <listitem>
              <para>
              The options in this section are used to check if
              you have specified values for optional items. 
              For example, there is an option that warns you if 
              you hav not specified a protocol.
              </para>
            </listitem>
          </varlistentry>
          
          <varlistentry>
            <term><guilabel>Annotations</guilabel></term>
            <listitem>
              <para>
              The options in this section are used to check 
              problems related to annotations. The most
              important ones are listed here:
              </para>
              
              <itemizedlist>
              <listitem>
                <simpara>
                <emphasis>Missing MIAME annotation value</emphasis>:
                Checks that you have specified values
                for all annotations marked as 
                <guilabel>Required for MIAME</guilabel>.
                </simpara>
              </listitem>
              
              <listitem>
                <simpara>
                <emphasis>Missing factor value</emphasis>:
                Checks that you have specified values for
                all annotations used as experimental factors in
                the experiment.
                </simpara>
              </listitem>
              
              <listitem>
                <simpara>
                <emphasis>Missing parameter value</emphasis>:
                Checks that you have specified values
                for all protocol parameters.
                </simpara>
              </listitem>

              <listitem>
                <simpara>
                <emphasis>Annotation is protocol parameter</emphasis>:
                Checks if an item has been annotated with a 
                an annotation that is actually a protocol parameter.
                </simpara>
              </listitem>

              <listitem>
                <simpara>
                <emphasis>Annotation has invalid value</emphasis>:
                Checks if annotation values are correct with 
                respect to the rules given by the annotation type.
                This might include numeric values that are outside 
                the valid range, or values not in the list
                of allows values for an enumerated annotation type.
                </simpara>
              </listitem>

              <listitem>
                <simpara>
                <emphasis>Inheriting annotation from non-parent</emphasis>:
                Checks if inherited annotations really comes from a
                parent item. This might happen if you rearrange 
                parent-child relationship because you found that
                they were incorrectly linked.
                </simpara>
              </listitem>
              </itemizedlist>
              
            
            </listitem>
          </varlistentry>
          
          <varlistentry>
            <term><guilabel>Denied access</guilabel></term>
            <listitem>
              <para>
              The options in this section are used to
              check if you do not have access (read permission)
              to an item in the experiment hierarchy. If this 
              happens the validation cannot proceed in that branch.
              This might mask other validation problems.
              </para>
            </listitem>
          </varlistentry>
          
          <varlistentry>
            <term><guilabel>Other</guilabel></term>
            <listitem>
              <para>
              This section collects options that does not fit
              into any of the other sections. The most
              important options are:
              </para>
              
              <itemizedlist>
                <listitem>
                  <simpara>
                  <emphasis>Array deign mismatch</emphasis>:
                  Checks if the array design specified for
                  a raw bioassay is the same array design
                  specified for the hybridization.
                  </simpara>
                </listitem>
                
                <listitem>
                  <simpara>
                  <emphasis>Multiple array designs</emphasis>:
                  Checks if all raw bioassays use the
                  same array design or not.
                  </simpara>
                </listitem>
              
                <listitem>
                  <simpara>
                  <emphasis>Incorrect number of labled extracts</emphasis>:
                  Checks if the number of labeled extracts
                  match the number of channels for the experiment.
                  </simpara>
                </listitem>

                <listitem>
                  <simpara>
                  <emphasis>Non-unique name</emphasis>:
                  Checks if two items of the same type
                  have the same name. A unique name if important
                  when exporting data in Tab2Mage format.
                  </simpara>
                </listitem>

                <listitem>
                  <simpara>
                  <emphasis>Circular reference to pooled item</emphasis>:
                  If you have used pooling, checks that no 
                  circular references have been created.
                  </simpara>
                </listitem>
              
              </itemizedlist>
              
            </listitem>
          </varlistentry>
          
        </variablelist>
        
        <para>
          Click on the &gbSave; button
          to use the current settings. The display will
          automatically refresh itself.
        </para>
        
      </helptext>
      
      <helptext external_id="experiment.overview.validationoptions.savepreset" 
        title="Save preset" webonly="1">
        
        <para>
        Saves all validation options as a preset. 
        </para>
        
        <variablelist>
          <varlistentry>
            <term><guilabel>Name</guilabel></term>
            <listitem>
              <para>
                The name of the preset. The name must be unique
                and if a preset with the same name already exists
                you will be asked if you want to overwrite it or not.
              </para>
            </listitem>
          </varlistentry>
        </variablelist>
        
        <para>
          Click on the &gbSave; button
          to save the preset or &gbCancel;
          to abort.
        </para>
      </helptext>
      
    </sect3>
    
    <sect3 id="experiments_analysis.fixfailures">
      <title>Fixing validation failures</title>
      <helptext external_id="experiment.overview.fixfailures" 
        title="How to fix validation failures">
        
        <para>
        The experiment overview includes a function that allows
        you to quickly fix most of the problems found during the 
        validation. The easiest way to use the function is:
        </para>
        
        <orderedlist>
          <listitem>
            <simpara>
            Click on an error or warning in the list of failures in 
            the lower right
            pane. The tree in the left pane and the item overview in the
            top right pane will automatically be updated to show the
            exact location of the faulty item.
            </simpara>
          </listitem>
          <listitem>
            <simpara>
            The upper right pane should contain a list labeled
            <guilabel>Failure details</guilabel> with more information
            about each failure and also one or more suggestions for fixing
            the problem. For example, a failure due to a missing item
            should suggest that you add or select an item.
            </simpara>
          </listitem>
          
          <listitem>
            <simpara>
            The suggestions should also have links that takes
            you to an edit view where you can do the changes.
            </simpara>
          </listitem>
          
          <listitem>
            <simpara>
            After saving the changes you must click on the
            <guibutton>Revalidate</guibutton> button to update
            interface. If you want, you can fix more than one 
            failure before clicking on the button.
            </simpara>
          </listitem>
            
        </orderedlist>
      
      </helptext>
    </sect3>
    
  </sect2>
  
  <sect2 id="experiments_analysis.magexport">
    <title>Tab2Mage export</title>
    <para>
      <ulink url="http://tab2mage.sourceforge.net/"
      >Tab2Mage format</ulink> is a tab-delimited format veted by
      <ulink url="http://www.ebi.ac.uk/microarray/"
      >EBI's ArrayExpress</ulink>
      repository for submission microarray data.
      Tab2Mage format has been chosen by BASE to provide an
      easy way for data deposition to public repository when
      submitting a manuscript and publishing experimental
      data.
    </para>
      
    <para>
      BASE2 has been engineered to closely map the MIAME
      concepts and a number of BASE2 entities can be mapped
      directly to Tab2Mage elements. However, since MIAME is
      focused on microarray processing workflow, information
      about the biological sample is down to the user. To
      accomodate the annotation needs of users, BASE2 provides
      a mechanism that allows annotation customization to meet
      user specific requirements. BASE allows to create
      annotation type for quantitative annotation and
      qualitative annotation
    </para>     

    <para>
      BASE2 can export an experiment to Tab2Mage
      format thanks to a dedicated export plug-in.
      For the plug-in to work, it is important to understand that 
      information recorded in BASE should be
      formatted following a small number of rules. Failing to
      do so may impair the possibility of exporting to
      ArrayExpress.
    </para>
    
    <note>
      <para>
      The Tab2Mage export plug-in has not yet been included in the main
      distribution. Hopefully, it will appear in the next (2.4) release.
      </para>
    </note>

    <sect3 id="experiments_analysis.magexport.annotations">
      <title>Biomaterial annotations</title>
      <para>
        Tab2Mage specifications only allow <emphasis>BioSource</emphasis>
        items to be annotated with <emphasis>BioMaterialCharacteristics</emphasis>.
      </para>
      <warning>
        <para>
          All BASE2 Annotation Types used to annotate at
          the level of <emphasis>Sample</emphasis> and 
          <emphasis>Extract</emphasis> items will
          be lost during the export in Tab2Mage format in
          order to comply with the ArrayExpress Tab2Mage
          parser.
        </para>
      </warning>
      <note>
        <para>
          In the context of data exchange between BASE
          instances, the export function can be altered to
          allow attachement of annotations to items
          other than biosources, therefore avoiding loss of 
          information.
        </para>
      </note>
    </sect3>

    <sect3 id="experiments_analysis.magexport.units">
      <title>Annotation units</title>
      <para>
        To associate units to BASE2 annotation types and
        remain compatible with Tab2Mage specifications,
        users need to adhere to the following convention:
      </para>
      <para>
        <userinput>annotation_type_name(unit_name)</userinput> as in 
        <userinput>body mass(kg)</userinput> or <userinput>concentration(mg/ml)</userinput>
      </para>
      <warning>
        <para>
          Only one unit can be specified at any one time
          for any given annotation type. In order to
          enable Tab2Mage support, it might be necessary
          to declare several related Annotation Type in
          order to report similar kind of information but
          expressed in a different unit. Specifying Age
          for instance is a good example on how to deal
          with such cases: One should create several
          related annotation types e.g. <userinput>Age(week)</userinput>,
          <userinput>Age(year)</userinput> or <userinput>Age(month)</userinput> 
          as those variations maybe be necessary when reporting the age of a
          mouse or the age of a human volunteer.
        </para>
      </warning>
    </sect3>
    <sect3
      id="experiments_analysis.magexport.parameters">
      <title>Protocol parameters</title>
      <para>
        In order to ensure MIAME compliance, Tab2Mage
        specifications cater for reporting parameters
        attached to protocols and all parameters attached to
        a protocol should be declared in the protocol
        section of a Tab2Mage file.
      </para>
      
      <para>
        In BASE2 terms, Tab2Mage elements such as
        <emphasis>BioMaterialCharacteristics</emphasis>, 
        <emphasis>Parameter</emphasis> or
        <emphasis>FactorValues</emphasis> are all annotation
        types. But, it is necessary to flag those annotations types meant to
        be used as protocol parameters as such so that they
        can identified by the Tab2Mage exporter and handled
        appropriately.
      </para>

      <warning>
        <para>
          It is not possible to use the same
          annotation type <emphasis>Temperature</emphasis> for reporting a
          patient body temperature (which is a
          <emphasis>Biomaterial Characteristic</emphasis>) and hybridization
          temperature (which is a protocol parameter).
          Hence it will be necessary to declare 2 distinct
          annotation types:
        </para>
      </warning>
      
      <itemizedlist>
      <listitem>
        <para>
        Annotation type to be used as <emphasis>BioSource
          characteristics</emphasis>: <userinput>body temperature (degree_C)</userinput>
        </para>
      </listitem>
      <listitem>
        <para>
        Annotation type to be used as <emphasis>protocol parameter</emphasis>: 
        <userinput>hybridization temperature (degree_C)</userinput>
        </para>
      </listitem>
      </itemizedlist>
    </sect3>
          
    <sect3 id="experiments_analysis.magexport.factors">
      <title>Experimental factors</title>
      <para>
        It is a MIAME requirement to identify <emphasis>Experimental
        Variables</emphasis> when submitting data to ArrayExpress
        (provided the study is an intervention study).
        Therefore, BASE2 users willing to use the Tab2Mage
        export function will have to declare <emphasis>Experimental
        Factors</emphasis> using the the <guilabel>Experimental Factor</guilabel>
        tab available when editing experiments. See 
        <xref linkend="experiments_analysis.experiment.factors" /> for more information.
      </para>
      
      <para>
        Values for the experimental factors are take from annotations.
        The annotation must exist at the raw bioassay level, which 
        probably means that you have to inherit the annotation from
        some other item, for example, a biosource or a sample.
        It is also possible to use a protocol parameter as
        experimental factor. See <xref linkend="annotations" /> for
        more information about annotations.
      </para>
    </sect3>
  </sect2>
</sect1>