source: trunk/doc/src/docbook/userdoc/import_export_data.xml @ 3481

Last change on this file since 3481 was 3481, checked in by Martin Svensson, 15 years ago

Fixes #426 Option to automatically delete a (import/export) job once it has been succesfully completed

  • Property svn:eol-style set to native
  • Property svn:keywords set to Id
File size: 24.9 KB
Line 
1<?xml version="1.0" encoding="UTF-8"?>
2<!DOCTYPE chapter PUBLIC
3  "-//Dawid Weiss//DTD DocBook V3.1-Based Extension for XML and graphics inclusion//EN"
4  "../../../../lib/docbook/preprocess/dweiss-docbook-extensions.dtd"
5[
6<!ENTITY runplugin.configure.common
7  "The top of the window displays the names of the selected plug-in and
8  configuration, a list with parameters to the left, an area for input fields to the
9  right and buttons to proceed with at the bottom.
10  Click on a parameter in the parameter list to show the form fields
11  for entering values for the parameter to the right. Parameters
12  with an <guilabel>X</guilabel> in front of their names already have a
13  value. Parameters marked with a blue rectangle are required and must
14  be given a value before it is possible to proceed."
15>
16]>
17<!--
18  $Id: import_export_data.xml 3481 2007-06-13 11:41:36Z martin $
19 
20  Copyright (C) Authors contributing to this file.
21 
22  This file is part of BASE - BioArray Software Environment.
23  Available at http://base.thep.lu.se/
24 
25  BASE is free software; you can redistribute it and/or
26  modify it under the terms of the GNU General Public License
27  as published by the Free Software Foundation; either version 2
28  of the License, or (at your option) any later version.
29 
30  BASE is distributed in the hope that it will be useful,
31  but WITHOUT ANY WARRANTY; without even the implied warranty of
32  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
33  GNU General Public License for more details.
34 
35  You should have received a copy of the GNU General Public License
36  along with this program; if not, write to the Free Software
37  Foundation, Inc., 59 Temple Place - Suite 330,
38  Boston, MA  02111-1307, USA.
39-->
40<chapter id="import_export_data">
41  <?dbhtml dir="import_export"?>
42  <title>Import and export of data</title>
43  <para>
44    In some places the only way to get data into BASE is to import it
45    from a file. This typically includes raw data, array design features,
46    reporters and other things, which would be inconvenient to enter
47    by hand due to the large number of data items.
48  </para>
49  <para>
50    On the other hand, if data should be used outside of BASE, you need
51    to export it. Exporting data is possible for almost all kind of data.
52    These two actions, importing and exporting, are not built into BASE but
53    are always handled by plug-ins.
54  </para>
55  <para>
56    Normally, a plug-in handles one type of items, but some plug-ins,
57    for example the table exporter plug-in, can work
58    with several types of items.
59    Some plug-ins requires a configuration, for example,
60    the import plug-ins need some information about how to find headers and
61    data lines in the files. BASE ships with a number of plug-ins, the core plug-ins.
62    A <ulink
63    url="http://base.thep.lu.se/chrome/site/doc/admin/plugin_configuration/coreplugins.html"
64      >list with the core plug-ins</ulink>
65    can be found on BASE's homepage. On this page are also configuration
66    examples that can be downloaded for some of the plugins.
67    Go to
68    <menuchoice>
69      <guimenu>Administrate</guimenu>
70      <guimenuitem>Plugins</guimenuitem>
71      <guisubmenu>Definitions</guisubmenu>
72    </menuchoice>
73    to check which plug-ins are installed on your BASE server.
74  </para>
75 
76  <para>
77    When a plug-in that supports import or export of a certain type of item
78    an &gbImport; or &gbExport;
79    button is displayed in the toolbar on either the list view or the single-item
80    view.
81  </para>
82  <note>
83    <title>Missing/unavailable button</title>
84    <para>
85      If the import and/or export button is missing from a page were you would expect
86      to find them this usually means that:
87    </para>
88    <itemizedlist>
89      <listitem>
90        <simpara>
91          The logged in user doesn't have permission to use the plug-in.
92        </simpara>
93      </listitem>
94      <listitem>
95        <simpara>
96          The plug-in requires a configuration, but no one has been
97          created or the logged in user doesn't have permission to
98          use any of the existing configurations.
99        </simpara>
100      </listitem>
101    </itemizedlist>
102    <para>
103      Contact the server administrator or a similar user that has permission to
104      administrate the plug-ins.
105    </para>
106  </note>
107
108  <sect1 id="import_export_data.import">
109    <title>Import</title>
110
111    <para>
112      Starting a data import is done by a wizard-like interface. There
113      are a number of step you have to go through:
114    </para>
115   
116    <orderedlist>
117      <listitem>
118        <simpara>
119          Select a plug-in and file format to use, or select the
120          auto detect option.
121        </simpara>
122      </listitem>
123      <listitem>
124        <simpara>
125          If you selected the auto detection funtion, you must select
126          a file to use.
127        </simpara>
128      </listitem>
129      <listitem>
130        <simpara>
131          Specify plug-in parameters.
132        </simpara>
133      </listitem>
134      <listitem>
135        <simpara>
136          Add the import job to the job queue.
137        </simpara>
138      </listitem>
139      <listitem>
140        <simpara>
141          Wait for the job to finish.
142        </simpara>
143      </listitem>
144    </orderedlist>
145
146    <sect2 id="import_export_data.import.plugin_fileformat">
147      <title>Select plug-in and file format</title>
148      <para>
149        Click on the &gbImport; button
150        in the toolbar to start the import wizard. The first step is to
151        select which plug-in and, if supported, which
152        file format to use. There is also an <guilabel>auto detect</guilabel>
153        option that lets you select a file and have BASE try to find a suitable
154        plug-in/file format to use.
155      </para>
156 
157      <figure id="import_export_data.figures.select_import_plugin">
158        <title>Select plug-in and file format</title>
159        <screenshot>
160          <mediaobject>
161            <imageobject><imagedata fileref="figures/select_import_plugin.png" format="PNG" /></imageobject>
162          </mediaobject>
163        </screenshot>
164      </figure>
165   
166
167      <helptext external_id="import.selectplugin"
168        title="Select plug-in and file format for data import">
169
170        <variablelist>
171          <varlistentry>
172            <term><guilabel>Plugin</guilabel></term>
173            <listitem>
174              <para>
175                A list of all plug-ins that are avilable in the
176                current context. The list only includes plug-ins that
177                the logged in user has permission to use. If you select
178                a plug-in a short description of about it is displayed
179                below the lists. More information about the plug-ins can
180                be found under the menu choice
181                <menuchoice>
182                  <guimenu>Administrate</guimenu>
183                  <guimenuitem>Plugins</guimenuitem>
184                  <guisubmenu>Definitions</guisubmenu>
185                </menuchoice>
186              </para>
187            </listitem>
188          </varlistentry>
189         
190          <varlistentry>
191            <term><guilabel>File format</guilabel></term>
192            <listitem>
193              <para>
194                A list of different file formats configurations
195                supported by the selected plug-in.
196                <menuchoice>
197                  <guimenu>Administrate</guimenu>
198                  <guimenuitem>Plugins</guimenuitem>
199                  <guisubmenu>Configurations</guisubmenu>
200                </menuchoice>.
201              </para>
202             
203              <note>
204                <title>File format vs. Configuration</title>
205                <simpara>
206                A file format is the same thing as a plug-in configuration.
207                It may be confusing that the interface sometimes use
208                <emphasis>file format</emphasis> and sometimes use
209                <emphasis>configuration</emphasis>, but for now, we'll have
210                to live with it.
211                </simpara>
212              </note>
213             
214            </listitem>
215          </varlistentry>
216       
217        </variablelist>
218
219        <para>
220          Proceed to the next step by clicking on the
221          &gbNext; button.
222        </para>
223
224        <seeother>
225          <other external_id="import.autodetect">The auto detect function</other>
226        </seeother>
227      </helptext>
228
229      <sect3 id="import_export_data.import.plugin_fileformat.autodetect">
230        <title>The auto detect function</title>
231       
232        <helptext
233          external_id="import.autodetect"
234          title="The auto detect function">
235       
236        <para>
237          The auto detect function lets you select a file and have
238          BASE try to find a suitable plug-in and file format. This option is
239          selected by default in both the plug-in and file format lists when there is
240          at least one plug-in that supports auto detection.
241        </para>
242        <note>
243          <title>Support of auto detect</title>
244          <para>
245            Not all plug-ins support auto detection. The ones that do are marked in
246            the list with <guilabel>×</guilabel>.
247          </para>
248        </note>
249       
250        <para>
251          Select the auto detect option either for both plug-ins and
252          file formats or only for file formats to use this feature.
253          Continue to the next step by clicking on the &gbNext;
254          button.
255        </para>
256       
257        <seeother>
258          <other external_id="import.selectplugin">Select plug-in and file format for data import</other>
259          <other external_id="import.autodetect.selectfile">Select file for auto detection</other>
260        </seeother>
261     
262        </helptext>
263       
264        <para>
265          You must now select a file to import from.
266        </para>
267       
268        <figure id="import_export_data.figures.select_autodetect_file">
269          <title>Select file for auto detection</title>
270          <screenshot>
271            <mediaobject>
272              <imageobject><imagedata fileref="figures/select_autodetect_file.png" format="PNG" /></imageobject>
273            </mediaobject>
274          </screenshot>
275        </figure>
276       
277        <helptext external_id="import.autodetect.selectfile" 
278          title="Select file for auto detection">
279 
280          <variablelist>
281            <varlistentry>
282              <term><guilabel>File</guilabel></term>
283              <listitem>
284                <para>
285                  Enter the path and filename for the
286                  file you want to use. Use the <guibutton>Browse&hellip;</guibutton>
287                  button to browse after the file in BASE's file system.
288                  If the file doesn't exist in the file system you have the option
289                  to upload it.
290                  <nohelp>Read more about this in <xref linkend="file_system" />.</nohelp>
291                </para>
292              </listitem>
293            </varlistentry>
294           
295            <varlistentry>
296              <term><guilabel>Recently used</guilabel></term>
297              <listitem>
298                <para>
299                  A list of files you have recently used
300                  for auto detection.
301                </para>
302              </listitem>
303            </varlistentry>
304          </variablelist>
305         
306          <para>
307            Click on the &gbNext; button
308            to start the auto detection.
309          </para>
310
311          <para>
312            If the auto detection finds a exactly one plug-in and file format
313            the next step is to configure any additional parameters needed
314            by the plug-in. This is the same step as if you had selected
315            the same plug-in and file format in the first step.
316            If no plug-in can be found an error message is displayed.
317          </para>
318
319          <note>
320            <title>More then one compatible plug-in/file format</title>
321            <para>
322              If more than one matching plug-in or file format is used
323              you will be taken back to the first step. This time
324              the lists will only include the matching plug-ins/file formats
325              and the auto detect option is not present.
326            </para>
327          </note>
328
329          <seeother>
330            <other external_id="import.selectplugin">Select plug-in and file format for data import</other>
331            <other external_id="import.autodetect">The auto detect function</other>
332          </seeother>
333         
334        </helptext>
335       
336      </sect3>
337
338    </sect2>
339
340    <sect2 id="import_export_data.import.pluginparameters">
341      <title>Specify plug-in parameters</title>
342      <para>
343        When you have selected a plug-in and file format or used
344        the auto detect function to find one, a form where you
345        you can enter additional parameters for the plug-in is displayed.
346      </para>
347     
348      <figure id="import_export_data.figures.confiure_plugin">
349        <title>Specify plug-in parameters</title>
350        <screenshot>
351          <mediaobject>
352            <imageobject><imagedata fileref="figures/plugin_parameters.png" format="PNG" /></imageobject>
353          </mediaobject>
354        </screenshot>
355      </figure>
356     
357      <helptext external_id="runplugin.configure.import" 
358        title="Specify plug-in parameters">
359      <para>
360        &runplugin.configure.common;
361      </para>
362     
363      <para>
364        The parameter list is very different from plug-in to plug-in.
365        Common parameters for import plug-ins are:
366      </para>
367     
368      <variablelist>
369        <varlistentry>
370          <term><guilabel>File</guilabel></term>
371          <listitem>
372            <para>
373            The file to import data from. A value is already set if
374            you used the auto detect function.
375            </para>
376          </listitem>
377        </varlistentry>
378       
379        <varlistentry>
380          <term><guilabel>Error handling</guilabel></term>
381          <listitem>
382            <para>
383              A section which contains different options how to
384              handle errors when parsing the file. Normally you can
385              select if the import should fail as a while or if
386              the line with the error should be skipped.
387            </para>
388          </listitem>
389        </varlistentry>
390      </variablelist>
391     
392      <para>
393        Continue to the next step by clicking the
394        &gbNext; button.
395      </para>
396     
397      <seeother>
398        <other external_id="runplugin.configure">The plug-in configuration wizard</other>
399      </seeother>   
400      </helptext>
401
402    </sect2>
403   
404    <sect2 id="import_export_data.import.jobqueue">
405      <title>Add the import job to the job queue</title>
406
407      <para>
408        In this window should information about the job be filled in, like name and
409        description. Where name is required and need to have valid string as a value. There
410        are also two check boxes in this page.
411        <variablelist>
412          <varlistentry>
413            <term>
414              <guilabel>Send message</guilabel>
415            </term>
416            <listitem>
417              <para>
418                Tick this check box if the job should send you a message when it is
419                finished, otherwise untick it
420              </para>
421            </listitem>
422          </varlistentry>
423          <varlistentry>
424            <term>
425              <guilabel>Remove job</guilabel>
426            </term>
427            <listitem>
428              <para>
429                If this check box is ticked, the job will be marked as removed when
430                it is finished, on condition that it was finished successfully. This
431                is only available for import- and export- plugins.
432              </para>
433            </listitem>
434          </varlistentry>
435        </variablelist>
436      </para>
437      <para>
438        Clicking on
439        &gbFinish;
440        when everything is set will end the job configuration and place the job in the job queue.
441        A self-refreshing window appears with information about the
442        job's status and execution time. How long time it takes before the job starts to run
443        depends on which priority it and the other jobs in the queue have. The job doesn't
444        depend on the status window to be able to run and the window can be
445        closed without interupting the execution.
446      </para>
447      <tip>
448        <title>View job status</title>
449        <para>
450          A job's status can be viewed at any time by opening it from the job list page,
451          <menuchoice>
452            <guimenuitem>View</guimenuitem>
453            <guimenuitem>Jobs</guimenuitem>
454          </menuchoice>.
455        </para>
456      </tip>
457    </sect2>
458  </sect1>
459
460  <sect1 id="import_export_data.export">
461    <title>Export</title>
462    <para>
463      Before data from BASE can be used outside the program, it first had to be
464      exported to a file which then can be downloaded to a local path from the BASE file
465      system.
466    </para>
467    <para>
468      A data export is done by a job that runs an export plug-in for the current context. An
469      export job is started by clicking on
470      <guibutton>Export&hellip;</guibutton>
471      in the toolbar, which will open a pop-up window allowing you to
472      select plug-in and specify parameters for it.
473    </para>
474   
475    <sect2 id="import_export_data.export.selectplugin">
476      <title>Select plug-in and configuration</title>
477
478      <para>
479        This dialog is very similar to the dialog for selecting an
480        import plug-in. See <xref linkend="import_export_data.figures.select_import_plugin" />
481        for a screenshot example.
482      </para>
483   
484      <helptext external_id="runplugin.selectplugin" title="Select plug-in">
485        <para>
486          The first thing in the configuration process is to choose which plug-in
487          to use and a configuration for those plug-ins that require it. More
488          information about the plug-ins can be found in each plug-in's documentation.
489        </para>
490        <note>
491          If there is only one plug-in and configuration available, this
492          step is skipped and you are taken directly to next step.
493        </note>
494       
495        <variablelist>
496          <varlistentry>
497            <term>
498              <guilabel>Plugin</guilabel>
499            </term>
500            <listitem>
501              <para>
502                Select the plug-in to use. Only plug-ins that supports the current
503                context and that the logged in user has permission to use are
504                available in the list.
505              </para>
506            </listitem>
507          </varlistentry>
508          <varlistentry>
509            <term>
510              <guilabel>Configuration</guilabel>
511            </term>
512            <listitem>
513              <para>
514                Select the configuration that should be used together with the plug-in.
515                Not all plug-ins supports configuration and this option is only visible
516                if the selected plug-in has this support.
517              </para>
518            </listitem>
519          </varlistentry>
520        </variablelist>
521        <para>
522          Click on
523          &gbNext;
524          to show the configuration of parameters for the job.
525        </para>
526       
527        <seeother>
528          <other external_id="runplugin.configure.export">Specify plug-in parameters</other>
529        </seeother>
530      </helptext>
531    </sect2>
532   
533    <sect2 id="import_export_data.export.pluginparameters">
534      <title>Specify plug-in parameters</title>
535   
536      <helptext external_id="runplugin.configure.export" 
537        title="Specify plug-in parameters">
538       
539      <para>
540        &runplugin.configure.common;
541      </para>
542     
543      <para>
544        The parameters list is very different from plug-in to plug-in.
545        Common parameters for export plug-ins are:
546      </para>
547     
548      <variablelist>
549        <varlistentry>
550          <term><guilabel>Save as</guilabel></term>
551          <listitem>
552            <para>
553              The path and filename in the BASE file system where
554              the exported data should be saved. Some plug-ins support immediate
555              download to the local file system if you leave the file parameter
556              empty. For saving the exported data within the BASE file system,
557              it's recommended to use the <guibutton>Browse&hellip;</guibutton>
558              button to get the right path and then complement it with the file's name.
559            </para>
560          </listitem>
561        </varlistentry>
562      </variablelist>
563      <para>
564        Click on
565        &gbNext;
566        to proceed to next configuration window.
567      </para>
568      <seeother>
569        <other external_id="runplugin.configure">The plug-in configuration wizard</other>
570      </seeother>   
571   
572    </helptext>
573   
574    <sect3 id="import_export_data.export.immediatedownload">
575      <title>Immediate download of the exported data</title>
576     
577      <para>
578        If the selected plug-in supports immediate download and the file parameter
579        were left empty a new windown with a <guibutton>Download</guibutton>
580        button is displayed. Click on this button to start the plug-in execution.
581        Do not close the window until a message saying that the export
582        was successful (or failed) is displayed. Your browser should open a dialog
583        asking you were to save the file on your local computer.
584      </para>
585     
586      <figure id="import_export_data.figures.download_immediately">
587        <title>Download immediately</title>
588        <screenshot>
589          <mediaobject>
590            <imageobject><imagedata fileref="figures/download_immediately.png" format="PNG" /></imageobject>
591          </mediaobject>
592        </screenshot>
593      </figure>
594     
595    </sect3>
596   
597    <sect3 id="import_export_data.export.savetofilesystem">
598      <title>Saving the exported data in the BASE file system</title>
599
600      <para>
601        If you choose to save the file within the BASE file system, there will be a window
602        where the job should get a name and optionally a description. There are also two
603        check boxes in this window.
604        <variablelist>
605          <varlistentry>
606            <term>
607              <guilabel>Send message</guilabel>
608            </term>
609            <listitem>
610              <para>
611                Tick this check box if the job should send you a message when it is
612                finished, otherwise untick it
613              </para>
614            </listitem>
615          </varlistentry>
616          <varlistentry>
617            <term>
618              <guilabel>Remove job</guilabel>
619            </term>
620            <listitem>
621              <para>
622                If this is ticked, the job will be marked as removed when it is
623                finished, on condition that it was finished successfully. This is
624                only available for import- and export- plugins.
625              </para>
626            </listitem>
627          </varlistentry>
628        </variablelist>
629        By then clicking on &gbFinish; the configuration process will end and the job will
630        be put in the job queue. A self-refreshing window appears with information about the
631        job's status and execution time. The job isn't dependent on the status window to run
632        and it therefore be closed without interrupting the execution of the job.
633      </para>
634      <tip>
635        <title>View job status</title>
636        <para>
637          A job's status can be viewed at any time by opening it from the job list page,
638          <menuchoice>
639            <guimenuitem>View</guimenuitem>
640            <guimenuitem>Jobs</guimenuitem>
641          </menuchoice>.
642        </para>
643      </tip>
644    </sect3>
645  </sect2>
646 
647  <sect2 id="import_export_data.export.table_export">
648    <title>The table exporter plug-in</title>
649    <para>
650      The table exporter is a generic export plug-in that works with almost
651      all list views in BASE. It can export the lists as an XML-file or a
652      tab-separated text file. The table exporter is started,
653      like the other export plug-ins, by clicking on
654      <guibutton>Export&hellip;</guibutton> in the toolbar.
655    </para>
656   
657    <para>
658      Then select the <guilabel>Table exporter</guilabel> and click on
659      the &gbNext; button. The plug-in selection step is
660      only displayed if there is more than one export plug-in that can
661      be used in the current context. Usually, the table exporter is the
662      only plug-in and you will be take directly to the configuration
663      dialog.
664    </para>
665     
666    <para>
667      Unlike other plug-ins, the table exporter doesn't use the
668      generic parameter input dialog. It has a customized dialog
669      that should be easier to use.
670    </para>
671     
672    <figure id="import_export_data.figures.table_exporter">
673      <title>The table exporter configuration dialog</title>
674      <screenshot>
675        <mediaobject>
676          <imageobject><imagedata fileref="figures/table_exporter.png" format="PNG" /></imageobject>
677        </mediaobject>
678      </screenshot>
679    </figure>
680   
681     
682    <helptext external_id="simpleexport.options" title="Export options">
683      <variablelist>
684        <varlistentry>
685          <term>
686            <guilabel>Format</guilabel>
687          </term>
688          <listitem>
689            <para>
690              The generated file can be either a
691              <guilabel>tab-separated text file</guilabel>
692              or an <guilabel>XML</guilabel> file. The
693              XML-file option will generate a tag for each item and these
694              will contain a child tag with property value for each
695              selected column.
696            </para>
697          </listitem>
698        </varlistentry>
699        <varlistentry>
700          <term>
701            <guilabel>Which items?</guilabel>
702          </term>
703          <listitem>
704            <para>
705              This option decide which items that should be included in
706              the exported data.
707              <itemizedlist>
708                <listitem>
709                  <para>
710                    <guilabel>Selected items</guilabel>:
711                    Only selected items will be exported. This options
712                    is not available if no items have been
713                    selected on the list page.
714                  </para>
715                </listitem>
716                <listitem>
717                  <para>
718                    <guilabel>Current page</guilabel>: Exports all
719                    items viewed on the current list page.
720                  </para>
721                </listitem>
722                <listitem>
723                  <para>
724                    <guilabel>All pages</guilabel>: Items on all pages
725                    will be exported.
726                  </para>
727                </listitem>
728              </itemizedlist>
729            </para>
730          </listitem>
731        </varlistentry>
732        <varlistentry>
733          <term>
734            <guilabel>Which columns?</guilabel>
735          </term>
736          <listitem>
737            <para>
738              Names of those columns that should be included in the export
739              should be listed in the <guilabel>Exported columns</guilabel>
740              to the left. A column name is
741              moved to the other list-box by first marking it and then
742              clicking on one of the buttons located between the
743              list-boxes.
744            </para>
745            <para>
746              The order in which the columns should be exported in can be
747              changed with the buttons to the left of the list. Simply
748              mark a name of a column and click on the buttons to move the
749              name either up or down in the list.
750            </para>
751            <tip>
752              <title>Using presets</title>
753              <para>
754                Use the drop down list of presets, located under the
755                option name, to easily get predefined or own presets of
756                column settings.
757              </para>
758            </tip>
759          </listitem>
760        </varlistentry>
761        <varlistentry>
762          <term>
763            <guilabel>Save as</guilabel>
764          </term>
765          <listitem>
766            <para>
767              The path and filename where the exported data
768              should be saved. Leave the text field empty if the file is
769              to be downloaded immediately or enter a path within the BASE
770              file system to store the file on the server. Check
771              <guilabel>Overwrite existing file</guilabel>
772              if an already existing file with the same name should be
773              overwritten.
774            </para>
775          </listitem>
776        </varlistentry>
777      </variablelist>
778     
779      <para>
780        Click on &gbOk;
781        to proceed when all options have been set for the export.
782      </para>
783    </helptext>
784
785    </sect2>
786  </sect1>
787</chapter>
Note: See TracBrowser for help on using the repository browser.