source: trunk/doc/src/docbook/userdoc/import_export_data.xml @ 3314

Last change on this file since 3314 was 3314, checked in by Nicklas Nordborg, 14 years ago

Fixes #547: Read"Import and Export data"

  • Property svn:eol-style set to native
  • Property svn:keywords set to Id
File size: 23.7 KB
Line 
1<?xml version="1.0" encoding="UTF-8"?>
2<!DOCTYPE chapter PUBLIC
3  "-//Dawid Weiss//DTD DocBook V3.1-Based Extension for XML and graphics inclusion//EN"
4  "../../../../lib/docbook/preprocess/dweiss-docbook-extensions.dtd"
5[
6<!ENTITY runplugin.configure.common
7  "The top of the window displays the names of the selected plug-in and
8  configuration, a list with parameters to the left, an area for input fields to the
9  right and buttons to proceed with at the bottom.
10  Click on a parameter in the parameter list to show the form fields
11  for entering values for the parameter to the right. Parameters
12  with an <guilabel>X</guilabel> in front of their names already have a
13  value. Parameters marked with a blue rectangle are required and must
14  be given a value before it is possible to proceed."
15>
16]>
17<!--
18  $Id: import_export_data.xml 3314 2007-05-09 11:52:06Z nicklas $
19 
20  Copyright (C) Authors contributing to this file.
21 
22  This file is part of BASE - BioArray Software Environment.
23  Available at http://base.thep.lu.se/
24 
25  BASE is free software; you can redistribute it and/or
26  modify it under the terms of the GNU General Public License
27  as published by the Free Software Foundation; either version 2
28  of the License, or (at your option) any later version.
29 
30  BASE is distributed in the hope that it will be useful,
31  but WITHOUT ANY WARRANTY; without even the implied warranty of
32  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
33  GNU General Public License for more details.
34 
35  You should have received a copy of the GNU General Public License
36  along with this program; if not, write to the Free Software
37  Foundation, Inc., 59 Temple Place - Suite 330,
38  Boston, MA  02111-1307, USA.
39-->
40<chapter id="import_export_data">
41  <?dbhtml dir="import_export"?>
42  <title>Import and export of data</title>
43  <para>
44    In some places the only way to get data into BASE is to import it
45    from a file. This typically includes raw data, array design features,
46    reporters and other things, which would be inconvenient to enter
47    by hand due to the large number of data items.
48  </para>
49  <para>
50    On the other hand, if data should be used outside of BASE, you need
51    to export it. Exporting data is possible for almost all kind of data.
52    These two actions, importing and exporting, are not built into BASE but
53    are always handled by plug-ins.
54  </para>
55  <para>
56    Normally, a plug-in handles one type of items, but some plug-ins,
57    for example the table exporter plug-in, can work
58    with several types of items.
59    Some plug-ins requires a configuration, for example,
60    the import plug-ins need some information about how to find headers and
61    data lines in the files. BASE ships with a number of plug-ins, the core plug-ins.
62    A <ulink
63    url="http://base.thep.lu.se/chrome/site/doc/admin/plugin_configuration/coreplugins.html"
64      >list with the core plug-ins</ulink>
65    can be found on BASE's homepage. On this page are also configuration
66    examples that can be downloaded for some of the plugins.
67    Go to
68    <menuchoice>
69      <guimenu>Administrate</guimenu>
70      <guimenuitem>Plugins</guimenuitem>
71      <guisubmenu>Definitions</guisubmenu>
72    </menuchoice>
73    to check which plug-ins are installed on your BASE server.
74  </para>
75 
76  <para>
77    When a plug-in that supports import or export of a certain type of item
78    an <guibutton>Import</guibutton> or <guibutton>Export</guibutton>
79    button is displayed in the toolbar on either the list view or the single-item
80    view.
81  </para>
82  <note>
83    <title>Missing/unavailable button</title>
84    <para>
85      If the import and/or export button is missing from a page were you would expect
86      to find them this usually means that:
87    </para>
88    <itemizedlist>
89      <listitem>
90        <simpara>
91          The logged in user doesn't have permission to use the plug-in.
92        </simpara>
93      </listitem>
94      <listitem>
95        <simpara>
96          The plug-in requires a configuration, but no one has been
97          created or the logged in user doesn't have permission to
98          use any of the existing configurations.
99        </simpara>
100      </listitem>
101    </itemizedlist>
102    <para>
103      Contact the server administrator or a similar user that has permission to
104      administrate the plug-ins.
105    </para>
106  </note>
107
108  <sect1 id="import_export_data.import">
109    <title>Import</title>
110
111    <para>
112      Starting a data import is done by a wizard-like interface. There
113      are a number of step you have to go through:
114    </para>
115   
116    <orderedlist>
117      <listitem>
118        <simpara>
119          Select a plug-in and file format to use, or select the
120          auto detect option.
121        </simpara>
122      </listitem>
123      <listitem>
124        <simpara>
125          If you selected the auto detection funtion, you must select
126          a file to use.
127        </simpara>
128      </listitem>
129      <listitem>
130        <simpara>
131          Specify plug-in parameters.
132        </simpara>
133      </listitem>
134      <listitem>
135        <simpara>
136          Add the import job to the job queue.
137        </simpara>
138      </listitem>
139      <listitem>
140        <simpara>
141          Wait for the job to finish.
142        </simpara>
143      </listitem>
144    </orderedlist>
145
146    <sect2 id="import_export_data.import.plugin_fileformat">
147      <title>Select plug-in and file format</title>
148      <para>
149        Click on the <guibutton>Import</guibutton> button
150        in the toolbar to start the import wizard. The first step is to
151        select which plug-in and, if supported, which
152        file format to use. There is also an <guilabel>auto detect</guilabel>
153        option that lets you select a file and have BASE try to find a suitable
154        plug-in/file format to use.
155      </para>
156 
157      <figure id="import_export_data.figures.select_import_plugin">
158        <title>Select plug-in and file format</title>
159        <screenshot>
160          <mediaobject>
161            <imageobject><imagedata fileref="figures/select_import_plugin.png" format="PNG" /></imageobject>
162          </mediaobject>
163        </screenshot>
164      </figure>
165   
166
167      <helptext external_id="import.selectplugin"
168        title="Select plug-in and file format for data import">
169
170        <variablelist>
171          <varlistentry>
172            <term><guilabel>Plugin</guilabel></term>
173            <listitem>
174              <para>
175                A list of all plug-ins that are avilable in the
176                current context. The list only includes plug-ins that
177                the logged in user has permission to use. If you select
178                a plug-in a short description of about it is displayed
179                below the lists. More information about the plug-ins can
180                be found under the menu choice
181                <menuchoice>
182                  <guimenu>Administrate</guimenu>
183                  <guimenuitem>Plugins</guimenuitem>
184                  <guisubmenu>Definitions</guisubmenu>
185                </menuchoice>
186              </para>
187            </listitem>
188          </varlistentry>
189         
190          <varlistentry>
191            <term><guilabel>File format</guilabel></term>
192            <listitem>
193              <para>
194                A list of different file formats configurations
195                supported by the selected plug-in.
196                <menuchoice>
197                  <guimenu>Administrate</guimenu>
198                  <guimenuitem>Plugins</guimenuitem>
199                  <guisubmenu>Configurations</guisubmenu>
200                </menuchoice>.
201              </para>
202             
203              <note>
204                <title>File format vs. Configuration</title>
205                <simpara>
206                A file format is the same thing as a plug-in configuration.
207                It may be confusing that the interface sometimes use
208                <emphasis>file format</emphasis> and sometimes use
209                <emphasis>configuration</emphasis>, but for now, we'll have
210                to live with it.
211                </simpara>
212              </note>
213             
214            </listitem>
215          </varlistentry>
216       
217        </variablelist>
218
219        <para>
220          Proceed to the next step by clicking on the
221          <guibutton>Next</guibutton> button.
222        </para>
223
224        <seeother>
225          <other external_id="import.autodetect">The auto detect function</other>
226        </seeother>
227      </helptext>
228
229      <sect3 id="import_export_data.import.plugin_fileformat.autodetect">
230        <title>The auto detect function</title>
231       
232        <helptext
233          external_id="import.autodetect"
234          title="The auto detect function">
235       
236        <para>
237          The auto detect function lets you select a file and have
238          BASE try to find a suitable plug-in and file format. This option is
239          selected by default in both the plug-in and file format lists when there is
240          at least one plug-in that supports auto detection.
241        </para>
242        <note>
243          <title>Support of auto detect</title>
244          <para>
245            Not all plug-ins support auto detection. The ones that do are marked in
246            the list with <guilabel>×</guilabel>.
247          </para>
248        </note>
249       
250        <para>
251          Select the auto detect option either for both plug-ins and
252          file formats or only for file formats to use this feature.
253          Continue to the next step by clicking on the <guibutton>Next</guibutton>
254          button.
255        </para>
256       
257        <seeother>
258          <other external_id="import.selectplugin">Select plug-in and file format for data import</other>
259          <other external_id="import.autodetect.selectfile">Select file for auto detection</other>
260        </seeother>
261     
262        </helptext>
263       
264        <para>
265          You must now select a file to import from.
266        </para>
267       
268        <figure id="import_export_data.figures.select_autodetect_file">
269          <title>Select file for auto detection</title>
270          <screenshot>
271            <mediaobject>
272              <imageobject><imagedata fileref="figures/select_autodetect_file.png" format="PNG" /></imageobject>
273            </mediaobject>
274          </screenshot>
275        </figure>
276       
277        <helptext external_id="import.autodetect.selectfile" 
278          title="Select file for auto detection">
279 
280          <variablelist>
281            <varlistentry>
282              <term><guilabel>File</guilabel></term>
283              <listitem>
284                <para>
285                  Enter the path and filename for the
286                  file you want to use. Use the <guibutton>Browse&hellip;</guibutton>
287                  button to browse after the file in BASE's file system.
288                  If the file doesn't exist in the file system you have the option
289                  to upload it.
290                  <nohelp>Read more about this in <xref linkend="file_system" />.</nohelp>
291                </para>
292              </listitem>
293            </varlistentry>
294           
295            <varlistentry>
296              <term><guilabel>Recently used</guilabel></term>
297              <listitem>
298                <para>
299                  A list of files you have recently used
300                  for auto detection.
301                </para>
302              </listitem>
303            </varlistentry>
304          </variablelist>
305         
306          <para>
307            Click on the <guibutton>Next</guibutton> button
308            to start the auto detection.
309          </para>
310
311          <para>
312            If the auto detection finds a exactly one plug-in and file format
313            the next step is to configure any additional parameters needed
314            by the plug-in. This is the same step as if you had selected
315            the same plug-in and file format in the first step.
316            If no plug-in can be found an error message is displayed.
317          </para>
318
319          <note>
320            <title>More then one compatible plug-in/file format</title>
321            <para>
322              If more than one matching plug-in or file format is used
323              you will be taken back to the first step. This time
324              the lists will only include the matching plug-ins/file formats
325              and the auto detect option is not present.
326            </para>
327          </note>
328
329          <seeother>
330            <other external_id="import.selectplugin">Select plug-in and file format for data import</other>
331            <other external_id="import.autodetect">The auto detect function</other>
332          </seeother>
333         
334        </helptext>
335       
336      </sect3>
337
338    </sect2>
339
340    <sect2 id="import_export_data.import.pluginparameters">
341      <title>Specify plug-in parameters</title>
342      <para>
343        When you have selected a plug-in and file format or used
344        the auto detect function to find one, a form where you
345        you can enter additional parameters for the plug-in is displayed.
346      </para>
347     
348      <figure id="import_export_data.figures.confiure_plugin">
349        <title>Specify plug-in parameters</title>
350        <screenshot>
351          <mediaobject>
352            <imageobject><imagedata fileref="figures/plugin_parameters.png" format="PNG" /></imageobject>
353          </mediaobject>
354        </screenshot>
355      </figure>
356     
357      <helptext external_id="runplugin.configure.import" 
358        title="Specify plug-in parameters">
359      <para>
360        &runplugin.configure.common;
361      </para>
362     
363      <para>
364        The parameter list is very different from plug-in to plug-in.
365        Common parameters for import plug-ins are:
366      </para>
367     
368      <variablelist>
369        <varlistentry>
370          <term><guilabel>File</guilabel></term>
371          <listitem>
372            <para>
373            The file to import data from. A value is already set if
374            you used the auto detect function.
375            </para>
376          </listitem>
377        </varlistentry>
378       
379        <varlistentry>
380          <term><guilabel>Error handling</guilabel></term>
381          <listitem>
382            <para>
383              A section which contains different options how to
384              handle errors when parsing the file. Normally you can
385              select if the import should fail as a while or if
386              the line with the error should be skipped.
387            </para>
388          </listitem>
389        </varlistentry>
390      </variablelist>
391     
392      <para>
393        Continue to the next step by clicking the
394        <guibutton>Next</guibutton> button.
395      </para>
396      </helptext>
397
398    </sect2>
399   
400    <sect2 id="import_export_data.import.jobqueue">
401      <title>Add the import job to the job queue</title>
402
403      <para>
404        In this window should information about the job be filled in, like name and
405        description. Where name is required and need to have valid string as a value. There
406        is also an option if the user want to get a message or not after the job is done.
407        The check-box should be ticked if the owner of the job wants to get a message with a
408        summary of the job's execution.
409      </para>
410      <para>
411        Clicking on
412        <guibutton>Finish</guibutton>
413        when everything is set will end the job configuration and place the job in the job queue.
414        A self-refreshing window appears with information about the
415        job's status and execution time. How long time it takes before the job starts to run
416        depends on which priority it and the other jobs in the queue have. The job doesn't
417        depend on the status window to be able to run and the window can be
418        closed without interupting the execution.
419      </para>
420      <tip>
421        <title>View job status</title>
422        <para>
423          A job's status can be viewed at any time by opening it from the job list page,
424          <menuchoice>
425            <guimenuitem>View</guimenuitem>
426            <guimenuitem>Jobs</guimenuitem>
427          </menuchoice>.
428        </para>
429      </tip>
430    </sect2>
431  </sect1>
432
433  <sect1 id="import_export_data.export">
434    <title>Export</title>
435    <para>
436      Before data from BASE can be used outside the program, it first had to be
437      exported to a file which then can be downloaded to a local path from the BASE file
438      system.
439    </para>
440    <para>
441      A data export is done by a job that runs an export plug-in for the current context. An
442      export job is started by clicking on
443      <guibutton>Export&hellip;</guibutton>
444      in the toolbar, which will open a pop-up window allowing you to
445      select plug-in and specify parameters for it.
446    </para>
447   
448    <sect2 id="import_export_data.export.selectplugin">
449      <title>Select plug-in and configuration</title>
450
451      <para>
452        This dialog is very similar to the dialog for selecting an
453        import plug-in. See <xref linkend="import_export_data.figures.select_import_plugin" />
454        for a screenshot example.
455      </para>
456   
457      <helptext external_id="runplugin.selectplugin" title="Select plug-in">
458        <para>
459          The first thing in the configuration process is to choose which plug-in
460          to use and a configuration for those plug-ins that require it. More
461          information about the plug-ins can be found in each plug-in's documentation.
462        </para>
463        <note>
464          If there is only one plug-in and configuration available, this
465          step is skipped and you are taken directly to next step.
466        </note>
467       
468        <variablelist>
469          <varlistentry>
470            <term>
471              <guilabel>Plugin</guilabel>
472            </term>
473            <listitem>
474              <para>
475                Select the plug-in to use. Only plug-ins that supports the current
476                context and that the logged in user has permission to use are
477                available in the list.
478              </para>
479            </listitem>
480          </varlistentry>
481          <varlistentry>
482            <term>
483              <guilabel>Configuration</guilabel>
484            </term>
485            <listitem>
486              <para>
487                Select the configuration that should be used together with the plug-in.
488                Not all plug-ins supports configuration and this option is only visible
489                if the selected plug-in has this support.
490              </para>
491            </listitem>
492          </varlistentry>
493        </variablelist>
494        <para>
495          Click on
496          <guibutton>Next</guibutton>
497          to show the configuration of parameters for the job.
498        </para>
499       
500        <seeother>
501          <other external_id="runplugin.configure.export">Specify plug-in parameters</other>
502        </seeother>
503      </helptext>
504    </sect2>
505   
506    <sect2 id="import_export_data.export.pluginparameters">
507      <title>Specify plug-in parameters</title>
508   
509      <helptext external_id="runplugin.configure.export" 
510        title="Specify plug-in parameters">
511       
512      <para>
513        &runplugin.configure.common;
514      </para>
515     
516      <para>
517        The parameters list is very different from plug-in to plug-in.
518        Common parameters for export plug-ins are:
519      </para>
520     
521      <variablelist>
522        <varlistentry>
523          <term><guilabel>Save as</guilabel></term>
524          <listitem>
525            <para>
526              The path and filename in the BASE file system where
527              the exported data should be saved. Some plug-ins support immediate
528              download to the local file system if you leave the file parameter
529              empty. For saving the exported data within the BASE file system,
530              it's recommended to use the <guibutton>Browse&hellip;</guibutton>
531              button to get the right path and then complement it with the file's name.
532            </para>
533          </listitem>
534        </varlistentry>
535      </variablelist>
536      <para>
537        Click on
538        <guibutton>Next</guibutton>
539        to proceed to next configuration window.
540      </para>
541   
542    </helptext>
543   
544    <sect3 id="import_export_data.export.immediatedownload">
545      <title>Immediate download of the exported data</title>
546     
547      <para>
548        If the selected plug-in supports immediate download and the file parameter
549        were left empty a new windown with a <guibutton>Download</guibutton>
550        button is displayed. Click on this button to start the plug-in execution.
551        Do not close the window until a message saying that the export
552        was successful (or failed) is displayed. Your browser should open a dialog
553        asking you were to save the file on your local computer.
554      </para>
555     
556      <figure id="import_export_data.figures.download_immediately">
557        <title>Download immediately</title>
558        <screenshot>
559          <mediaobject>
560            <imageobject><imagedata fileref="figures/download_immediately.png" format="PNG" /></imageobject>
561          </mediaobject>
562        </screenshot>
563      </figure>
564     
565    </sect3>
566   
567    <sect3 id="import_export_data.export.savetofilesystem">
568      <title>Saving the exported data in the BASE file system</title>
569
570      <para>
571        If you choose to save the file within the BASE file system, there will be a
572        window where the job should get a name and optionally a description. By then clicking on
573        <guibutton>Finish</guibutton>
574        the configuration process will end and the job will be put in the job queue. A
575        self-refreshing window appears with information about the job's status and execution
576        time. The job isn't dependent on the status window to run and it therefore be closed
577        without interrupting the execution of the job.
578      </para>
579      <tip>
580        <title>View job status</title>
581        <para>
582          A job's status can be viewed at any time by opening it from the job list page,
583          <menuchoice>
584            <guimenuitem>View</guimenuitem>
585            <guimenuitem>Jobs</guimenuitem>
586          </menuchoice>.
587        </para>
588      </tip>
589    </sect3>
590  </sect2>
591 
592  <sect2 id="import_export_data.export.table_export">
593    <title>The table exporter plug-in</title>
594    <para>
595      The table exporter is a generic export plug-in that works with almost
596      all list views in BASE. It can export the lists as an XML-file or a
597      tab-separated text file. The table exporter is started,
598      like the other export plug-ins, by clicking on
599      <guibutton>Export&hellip;</guibutton> in the toolbar.
600    </para>
601   
602    <para>
603      Then select the <guilabel>Table exporter</guilabel> and click on
604      the <guibutton>Next</guibutton> button. The plug-in selection step is
605      only displayed if there is more than one export plug-in that can
606      be used in the current context. Usually, the table exporter is the
607      only plug-in and you will be take directly to the configuration
608      dialog.
609    </para>
610     
611    <para>
612      Unlike other plug-ins, the table exporter doesn't use the
613      generic parameter input dialog. It has a customized dialog
614      that should be easier to use.
615    </para>
616     
617    <figure id="import_export_data.figures.table_exporter">
618      <title>The table exporter configuration dialog</title>
619      <screenshot>
620        <mediaobject>
621          <imageobject><imagedata fileref="figures/table_exporter.png" format="PNG" /></imageobject>
622        </mediaobject>
623      </screenshot>
624    </figure>
625   
626     
627    <helptext external_id="simpleexport.options" title="Export options">
628      <variablelist>
629        <varlistentry>
630          <term>
631            <guilabel>Format</guilabel>
632          </term>
633          <listitem>
634            <para>
635              The generated file can be either a
636              <guilabel>tab-separated text file</guilabel>
637              or an <guilabel>XML</guilabel> file. The
638              XML-file option will generate a tag for each item and these
639              will contain a child tag with property value for each
640              selected column.
641            </para>
642          </listitem>
643        </varlistentry>
644        <varlistentry>
645          <term>
646            <guilabel>Which items?</guilabel>
647          </term>
648          <listitem>
649            <para>
650              This option decide which items that should be included in
651              the exported data.
652              <itemizedlist>
653                <listitem>
654                  <para>
655                    <guilabel>Selected items</guilabel>:
656                    Only selected items will be exported. This options
657                    is not available if no items have been
658                    selected on the list page.
659                  </para>
660                </listitem>
661                <listitem>
662                  <para>
663                    <guilabel>Current page</guilabel>: Exports all
664                    items viewed on the current list page.
665                  </para>
666                </listitem>
667                <listitem>
668                  <para>
669                    <guilabel>All pages</guilabel>: Items on all pages
670                    will be exported.
671                  </para>
672                </listitem>
673              </itemizedlist>
674            </para>
675          </listitem>
676        </varlistentry>
677        <varlistentry>
678          <term>
679            <guilabel>Which columns?</guilabel>
680          </term>
681          <listitem>
682            <para>
683              Names of those columns that should be included in the export
684              should be listed in the <guilabel>Exported columns</guilabel>
685              to the left. A column name is
686              moved to the other list-box by first marking it and then
687              clicking on one of the buttons located between the
688              list-boxes.
689            </para>
690            <para>
691              The order in which the columns should be exported in can be
692              changed with the buttons to the left of the list. Simply
693              mark a name of a column and click on the buttons to move the
694              name either up or down in the list.
695            </para>
696            <tip>
697              <title>Using presets</title>
698              <para>
699                Use the drop down list of presets, located under the
700                option name, to easily get predefined or own presets of
701                column settings.
702              </para>
703            </tip>
704          </listitem>
705        </varlistentry>
706        <varlistentry>
707          <term>
708            <guilabel>Save as</guilabel>
709          </term>
710          <listitem>
711            <para>
712              The path and filename where the exported data
713              should be saved. Leave the text field empty if the file is
714              to be downloaded immediately or enter a path within the BASE
715              file system to store the file on the server. Check
716              <guilabel>Overwrite existing file</guilabel>
717              if an already existing file with the same name should be
718              overwritten.
719            </para>
720          </listitem>
721        </varlistentry>
722      </variablelist>
723     
724      <para>
725        Click on <guibutton>Ok</guibutton>
726        to proceed when all options have been set for the export.
727      </para>
728    </helptext>
729
730    </sect2>
731  </sect1>
732</chapter>
Note: See TracBrowser for help on using the repository browser.