source: trunk/doc/src/docbook/userdoc/import_export_data.xml @ 3246

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE chapter PUBLIC 
  "-//Dawid Weiss//DTD DocBook V3.1-Based Extension for XML and graphics inclusion//EN" 
  "../../../../lib/docbook/preprocess/dweiss-docbook-extensions.dtd">
<!--
  $Id: import_export_data.xml 3246 2007-04-16 14:12:23Z martin $
  
  Copyright (C) Authors contributing to this file.
  
  This file is part of BASE - BioArray Software Environment.
  Available at http://base.thep.lu.se/
  
  BASE is free software; you can redistribute it and/or
  modify it under the terms of the GNU General Public License
  as published by the Free Software Foundation; either version 2
  of the License, or (at your option) any later version.
  
  BASE is distributed in the hope that it will be useful,
  but WITHOUT ANY WARRANTY; without even the implied warranty of
  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
  GNU General Public License for more details.
  
  You should have received a copy of the GNU General Public License
  along with this program; if not, write to the Free Software
  Foundation, Inc., 59 Temple Place - Suite 330,
  Boston, MA  02111-1307, USA.
-->

<chapter id="import_export_data">
  <?dbhtml dir="import_export"?>
  <title>Import and export of data</title>
  <para>
    The data need first of all to be imported into the database before it can be used by/in the
    environment. The opposite is if the data should be used outside the environment, it then
    have to be exported before it can be used. These two actions, importing and exporting, are
    done through the plugin interface.
  </para>
  <para>
    Different kinds of import and export plugins, depending on the data, are used to get
    data/information in and out from the BASE 2's database. Some of the plugins can take, some
    even requires, a configuration to be able to run. If a plugin has configuration support or
    not, can be found at
    <menuchoice>
      <guimenu>Administrate</guimenu>
      <guimenuitem>Plugins</guimenuitem>
      <guisubmenu>Definitions</guisubmenu>
    </menuchoice>
    . The installation of BASE2 includes a collection of plugins, so called core plugins, and
    these are, like all other plugins, supervised by the administrator or an user with
    similar(at least write) permission on plugins. A list with these core plugins can be found
    on
    <ulink
      url="http://base.thep.lu.se/chrome/site/doc/admin/plugin_configuration/coreplugins.html">
      base2's homepage
    </ulink>
    . On this page are also configuration examples that can be downloaded for some of the
    plugins.
  </para>
  <para>
    If the
    <guibutton>Import</guibutton>
    or
    <guibutton>Export</guibutton>
    button is available in a page's toolbar the current context and item type is supported by at
    least one import and export plugins respectively that the logged in user has permission to
    use.
  </para>
  <note>
    <title>Missing/unavailable button</title>
    <para>
      The logged in user does usually not have enough permission on the expected plugin if the
      import- or export button is unavailable/not visible. Contact the administrator or
      similar user that has permission to administrate the plugins.
    </para>
  </note>
  <para>
    Following subsections contains information how to proceed when importing or exporting data
    in BASE2.
  </para>
  <sect1 id="import_export_data.import">
    <title>Import</title>
    <para>
      Click on
      <guibutton>Import</guibutton>
      in the toolbar to import data into BASE2's database. This will open a pop-up window
      where the job, that shall perform the data import, can be configured. More about jobs
      and what a job is can be found under
      <xref linkend="jobs" />
      .
    </para>

    <sect2 id="import_export_data.import.plugin_fileformat">
      <title>Select plugin and file format</title>
      <helptext external_id="import.selectplugin"
        title="Select plugin and file format for data import">
        <para>
          The first thing to do when configuring a job is to set which import plugin and
          file format to use alternatively let the interface choose suitable plugin/file
          format based on the file that shall be imported, more about this in
          <xref linkend="import_export_data.import.plugin_fileformat.autodetect" />
          .
        </para>

        <sect3 id="import_export_data.import.plugin_fileformat.plugin">
          <title>Plugin</title>
          <para>
            Available import plugins that the logged in user has permission to use and
            that support the current context are listed in the pop-up window's
            drop-down-list with plugins. Select the appropriate import plugin for the
            job from this list. At the bottom of the frame stands a short description
            about what the selected plugin does. More information about the plugins can
            be found under the menu choice
            <menuchoice>
              <guimenu>Administrate</guimenu>
              <guimenuitem>Plugins</guimenuitem>
              <guisubmenu>Definitions</guisubmenu>
            </menuchoice>
          </para>
        </sect3>

        <sect3 id="import_export_data.import.plugin_fileformat.fileformat">
          <title>File format</title>
          <para>
            The drop down list with file format contains available file formats for the
            selected plugin. A file format configures a plugin how it should parse the
            file for importing it's data. Information about each file format can be
            found under the menu choice
            <menuchoice>
              <guimenu>Administrate</guimenu>
              <guimenuitem>Plugins</guimenuitem>
              <guisubmenu>Configurations</guisubmenu>
            </menuchoice>
            . Select the file format that should be used together with the plugin.
          </para>
          <para>
            Proceed to next window by clicking on
            <guibutton>Next</guibutton>
          </para>
        </sect3>

        <sect3 id="import_export_data.import.plugin_fileformat.autodetect">
          <title>Auto detect</title>
          <para>
            Auto detect lets the BASE2 plugin-interface to choose plugin and/or file
            format by validating the data file that shall be imported. This option is
            set by default in both plugin- and file format drop-down lists when there is
            at least one plugin in the list that supports auto detection.
          </para>
          <note>
            <title>Support of auto detect</title>
            <para>
              Not all plugins support auto detection. The ones that do are marked in
              the drop down list with a
              <guilabel>X</guilabel>
              .
            </para>
          </note>
          <para>
            Set either both plugins and file formats or only file format to auto detect
            to use this feature. Click then on
            <guibutton>Next</guibutton>
            to go to the next window where the file to import from is selected. Press
            <guibutton>Browse&hellip;</guibutton>
            in this window to browse after the import file in the BASE2's file system.
            If the file doesn't exist in the file system it first has to be uploaded
            before it's possible to use it, read more about this under
            <xref linkend="file_system" />
            .
          </para>
          <para>
            Proceed by clicking on
            <guibutton>Next</guibutton>
            .
          </para>
          <para>
            If there is a file format and a plugin compatible to the selected file, the
            next step will be to configure the job parameters. The file parameter is
            already set to the file that was selected earlier. Is there on the other
            hand no compatible file format or plugin that could be found there will be
            an error message that describes what went wrong.
          </para>
          <note>
            <title>More then one compatible plugin/file format</title>
            <para>
              The auto detect will pick the first plugin and file format it finds and
              it can therefore not be used if there are more then one plugin or file
              format that can import the selected file.
            </para>
          </note>
        </sect3>
      </helptext>
    </sect2>

    <sect2 id="import_export_data.import.jobparameters">
      <title>Job parameters</title>
      <para>
        This window contains of a top line with the names of the selected plugin and
        configuration, a list with parameters to the left, area for input fields to the
        right and buttons to proceed with at the bottom. Which the parameters in the list
        are depends on the selected plugin and it's configuration. Common parameter for all
        import plugin is however the
        <guilabel>File</guilabel>
        parameter (already set if auto detect is used) which holds the file to import the
        data from. A click on a parameter in the list will display an input field and a
        description to the right of the list. Parameters that have a
        <guilabel>X</guilabel>
        in front of their names has already a value set and those parameters, which names
        are marked with a rectangle, are required and need to be set before it's possible to
        continuing with the job configuration. There will pop up a dialog window if not all
        required parameters had been set.
      </para>
      <para>
        Continue the job configuration by clicking on
        <guibutton>Next</guibutton>
        . In this window should information about the job be filled in, like name and
        description. Where name is required and need to have valid string as a value. There
        is also an option if the user want to get a message or not after the job is done.
        The check-box should be ticked if the owner of the job wants to get a message with a
        summary of the job's execution.
      </para>
      <para>
        Clicking on
        <guibutton>Finish</guibutton>
        when everything is set will end the job configuration and place the job in the BASE
        2 server's job queue. A self-refreshing window appears with information about the
        job's status and execution time. How long time it takes before the job starts to run
        depends on which priority it and the other jobs in the queue have. The job doesn't
        dependent on the status window to be able to run and therefore can the window be
        closed without the execution is interrupted.
      </para>
      <tip>
        <title>View job status</title>
        <para>
          A job's status can be viewed at any time by opening it from the job list page,
          <menuchoice>
            <guimenuitem>View</guimenuitem>
            <guimenuitem>Jobs</guimenuitem>
          </menuchoice>
          .
        </para>
      </tip>
    </sect2>
  </sect1>

  <sect1 id="import_export_data.export">
    <title>Export</title>
    <para>
      Before data from BASE 2's database can be used outside the program, it first had to be
      exported to a file which then can be downloaded to a local path from the BASE file
      system.
    </para>
    <para>
      A data export is done by a job that runs an export plugin for the current context. An
      export job is started by clicking on
      <guibutton>Export&hellip;</guibutton>
      in the toolbar, which will open a pop-up window where the job can be configured.
    </para>
    <helptext external_id="runplugin.selectplugin" title="Select plugin">
      <para>
        The first thing in the configuration process is to choose which plugin the job
        should use and a plugin-configuration for those plugins that require it. More
        information about the plugins can be found in each plugin's documentation.
      </para>
      <variablelist>
        <title>Set plugin and configuration</title>
        <varlistentry>
          <term>
            <guilabel>Plugin</guilabel>
          </term>
          <listitem>
            <para>
              Select the plugin that shall do the export. Only plugins that are
              available for the logged in user and that supports the current item are
              in the drop down list.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <guilabel>Configuration</guilabel>
          </term>
          <listitem>
            <para>
              Select the configuration that should be used together with the plugin.
              Not all plugins supports configuration and this option is only visible
              if the selected plugin in
              <guilabel>Plugin</guilabel>
              has this support and visible configurations are those which can be used
              with the selected plugin above. More information about configuration
              support can be found for each plugin under.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
      <para>
        Click on
        <guibutton>Next</guibutton>
        to show the configuration of parameters for the job.
      </para>
    </helptext>
    <para>
      Common for all export plugins is the file parameter that decide where the export data
      should be saved. Some plugins support the option of download immediately which let the
      user to download the exported data to a local file system, by leaving the file parameter
      empty. For saving the exported data within the file system of BASE, it's recommended to
      use the
      <guibutton>Browse&hellip;</guibutton>
      button to get the right path and then complement it with the file's name.
    </para>
    <para>
      Click on
      <guibutton>Next</guibutton>
      to proceed to next configuration window.
    </para>
    <para>
      If the file parameter were left empty and immediately download was chosen the new window
      will have a
      <guibutton>Download</guibutton>
      button to specify where the data should be saved on the local computer.
    </para>
    <para>
      But if the file should be saved within the file system of BASE instead, there will be a
      window where the job should get a name and optionally a description. By then clicking on
      <guibutton>Finish</guibutton>
      the configuration process will end and the job will be put in the job queue. A
      self-refreshing window appears with information about the job's status and execution
      time. The job isn't dependent on the status window to run and it therefore be closed
      without interrupting the execution of the job.
    </para>
    <tip>
      <title>View job status</title>
      <para>
        A job's status can be viewed at any time by opening it from the job list page,
        <menuchoice>
          <guimenuitem>View</guimenuitem>
          <guimenuitem>Jobs</guimenuitem>
        </menuchoice>
        .
      </para>
    </tip>
    <sect2 id="import_export_data.export.table_export">
      <title>Table export</title>
      <para>
        The table exporter is a common export plugin for all list pages in BASE 2 and it
        makes it possible for the user to export the list to either a XML-file or a
        tab-separated text file. The table exporter supports all list pages and is started,
        like the other export plugins, by clicking on
        <guibutton>Export&hellip;</guibutton>
        in the toolbar.
      </para>
      <sect3 id="import_export_data.export.table_export.configuration">
        <title>Configuration</title>
        <para>
          Clicking on
          <guibutton>Export&hellip;</guibutton>
          will open a pop up window to configure the job in. If there are more plugins,
          that support the current context, then the table exporter the first thing will
          be to choose which plugin to use - in this case
          <guilabel>Table exporter</guilabel>
          . Confirm the choice and go to next window by clicking on
          <guibutton>Next</guibutton>
          .
        </para>
        <helptext external_id="simpleexport.options" title="Table-export options">
          <para>
            In this window are the options set, which the table exporter should use when
            exporting the table. See the list below for more information about each
            option.
            <variablelist>
              <title>Export options</title>
              <varlistentry>
                <term>
                  <guilabel>Format</guilabel>
                </term>
                <listitem>
                  <para>
                    The generated file can be either a tab-separated text file
                    or a XML file. In the text file will the columns be tab
                    separated and each row represent a row in the list. The
                    XML-file option will generate a tag for each item and these
                    will contain a child tag with property value for each
                    selected column.
                  </para>
                </listitem>
              </varlistentry>
              <varlistentry>
                <term>
                  <guilabel>Which items?</guilabel>
                </term>
                <listitem>
                  <para>
                    This option decide which items that should be included in
                    the exported data.
                    <itemizedlist>
                      <listitem>
                        <para>
                          Selected items(only available if items have been
                          selected on the list page). Only selected items
                          will be exported to the file.
                        </para>
                      </listitem>
                      <listitem>
                        <para>
                          Current page. Exports all items viewed on the
                          current list page.
                        </para>
                      </listitem>
                      <listitem>
                        <para>
                          All pages. Items on all pages will be exported
                          to the file.
                        </para>
                      </listitem>
                    </itemizedlist>
                  </para>
                </listitem>
              </varlistentry>
              <varlistentry>
                <term>
                  <guilabel>Which columns?</guilabel>
                </term>
                <listitem>
                  <para>
                    Names of those columns that should be included in the export
                    should be listed in the left list-box. A column name is
                    moved to the other list-box by first marking it and then
                    clicking on one of the buttons located between the
                    list-boxes.
                  </para>
                  <para>
                    The order in which the columns should be exported in can be
                    changed with the buttons to the left of the list. Simply
                    mark a name of a column and click on the buttons to move the
                    name either up or down in the list.
                  </para>
                  <tip>
                    <title>Using presets</title>
                    <para>
                      Use the drop down list of presets, located under the
                      option name, to easily get predefined or own presets of
                      column settings.
                    </para>
                  </tip>
                </listitem>
              </varlistentry>
              <varlistentry>
                <term>
                  <guilabel>Save as</guilabel>
                </term>
                <listitem>
                  <para>
                    The last option is where the file with the exported data
                    should be saved. Leave the text field empty if the file is
                    to be downloaded immediately or enter a path within base2's
                    file system to store the file on the server. Check
                    <guilabel>Overwrite existing file</guilabel>
                    if an already existing file with the same name should be
                    overwritten. Click on
                    <guibutton>Ok</guibutton>
                    to proceed when all options have been set for the export.
                  </para>
                </listitem>
              </varlistentry>
            </variablelist>
          </para>
        </helptext>
        <para>
          The rest of the configuration process is the same as for all the other export
          plugins.
        </para>
      </sect3>
    </sect2>
  </sect1>
</chapter>