source: trunk/uk/ac/ebi/Tab2MageImporter/README.txt @ 407

============================================================
General Information:
---------------------------

This plugin imports raw data files and an experiment meta-data in the tab2mage specification into base 2. The tab2mage specification is provided by the microarray informatics team at the EBI, Cambridge. The imported experiment can later be exported by the corresponding tab2mage export plugin. (link).

The entry point for this plugin in base is the detail view page of an experiment. Hence, the user needs to create an experiment first and execute the plugin from this page by clicking on the import tab. The displayed wizards asked for the zip file containing the meta- data i.e. tab2mage file and the raw data files. Furthermore, the user’s  email server name could be specified to received notices of success or failure of the uploading or can be omitted to receive no message.  

Installation
---------------

  - Place Tab2MageImporter.jar into the $BASE2_HOME/www/plugins/ directory on your BASE2 server. 
  - Install as new plugins uk.ac.ebi.nugo.plugins.Tab2MageImporter and uk.ac.ebi.nugo.plugins.AnnotationTypeCvImporter which are both contained in $BASE_HOME/www/plugins/Tab2MageImporter.jar.
  
  1. In the BASE web client, go to Administrate -> Plugins -> Definitions.
  2. Click New.
  3. In the dialog that opens, 
    for class enter " uk.ac.ebi.nugo.plugins.Tab2MageImporter".
     for path enter the path to the plugin jar file, i.e. <BASE_HOME>/www/plugins/ Tab2MageImporter.jar
  4. Click Save.
  5. Click New again
  6. In the dialog that opens, 
    for class enter " uk.ac.ebi.nugo.plugins.AnnotationTypeCvImporter".
    for path enter the path to the plugin jar file, i.e. <BASE_HOME>/www/plugins/ Tab2MageImporter.jar
  7. Click Save.
  The new plugins appears under the name "Tab2Mage Importer" and "AnnotationType and Cv Importer" respectively in the plugins list and are now ready for use. 
  8. Proceed to configure the "AnnotationType and Cv Importer" plugin. An example import file is provided here:       http://lev.thep.lu.se/trac/baseplugins/browser/trunk/uk/ac/ebi/AnnotationTypeCvImporter/sampleimportfile.txt
  
  For other users to use the plugin as detailed in the next section, please share them to the appropriate users by manually setting the permissions on the item.

  
  General Installation instruction is available at: http://base.thep.lu.se/chrome/site/doc/html/admindoc/plugin_installation/plugins.installation.html#plugins.installation
   
===========================================================

 Use
--------
 Warning:
  -- It is advisable not to execute the plugin as root but as a user with a specified quota.  The quota must be enough to store files and create base items.
  --  re- running the import plugin without having deleted the previously created directory from the zip file will make the   Importer fail.
  --  Note that in its current implementation, only one RawDataImporter plugin configuration can be used during the import process. This means that raw data files must be  consistent throughout. (this might be an issue when using different scanner versions).
  -- - Make sure that the user running Tab2Mage Importer plugin has enough permissions to do so. With the default Roles settings in BASE, User and PowerUser have not   enough permissions to run the plugin successfully.
  
  1. Platform supported  are: Affymetrix, Agilent, Genepix 
  2. Create all the array design used in the experiment as the BASE administrator. Array Design must then be shared to everyone or a suitable group that all the target users belong.
  3. Create tab2mage spreadsheet and zip it together with the raw data files  that make up the experiment. Note: no cdf file. Upload this file to your account on BASE (go to View -> Files -> Upload File). Image files may be supplied by the user and contained in the zip file.
  Note: To avoid the laborious process of checking and formatting your tab2mage spreadsheet. Please use the example files: tab2mage- agilent, tab2mage- genepix and tab2mage- affymetrix, provided with the distribution as the spreadsheet template. For instance, the protocol section of your spreadsheet must have the protocol type specified.If not the plugin will fail.
  4. To make sure you have all the required reporters and import configurations in place, import a single data file from the batch as a test first.  
  5. If the previous step was successful, create a new experiment  and specify the raw data type ( remember only affymetrix, genepix and agilent  platfroms are supported) . Ensure that  the experiment is of the same data type as the data in the files you want to import. Click the import tab from the plugin menu, select Tab2Mage Importer and click Next.
  6. Open the detail view of the newly created experiment and click on the import tab.
  7. In the dialog that opens, leave the settings for Plugin and File format on auto-detect and click Next.
  8 On the next screen, Select the batch file that contains the raw data files, the Tab2Mage text file and the image file if supplied  and  click Next.
  9. On the next screen, the ZipFile and Experiment parameters should already be configured now (as indicated by the black cross against them). Enter the name of your Tab2Mage file as a value for the "Tab2Mage FileName with Extension" parameter.  Specify  your email server if you want to receive notification on the success or failure of your experiment loading and click Next.
  10. On the next screen click Finish and wait for the job to complete. You will be shown the progress of the Tab2mage import in the dialog, with the page refreshing itself regularly.

  When the import has finished, you can check and review the uploaded experiment by navigating thorughbase2 web client, using the menu. If you have provided your email server name. You would receive an email on status of the import.

 ===========================================================

Dependencies:
--------------------
  
  - Base 2.2 and above

  -Reporter Importer Plugin - Required for loading Reporters

  Reporter Map Importer Plugin – Required for mapping features to Reporters when defining an Array Design
  
  Raw Data Importer Plugin – Required for loading raw data and import values associated to BASE rawBioassays. This is a critical requirement, for multi-channel experiments, before running the Tab2MageImporter. Users must ensure that raw data import plugin has   been properly configured and is able to import a raw datafile of the type of the submission.

  The definition for the above plug-ins is available by default in Base 2. The plugins should be configured by a Base 2 administrator and shared to everyone or a suitable group of target users.

  AnnotationTypeCvImporter Plugin – Required for loading annotation types. The plugin must have a proper configuration defined to import an annotation type file created by the Tab2MageImporter plugin when parsing the tab2mage file.

  Please ensure that you have used all the plugins above appropriately, before running Tab2Mage importer. Click AnnotationTypeCvImporter for further information on Annotation Type and Cv Importer plugin and for other plugins, please read base2 documentation.

  Also, do make sure that Array Design(s) corresponding to the one(s) referenced in the Array[accession] field of the tab2mage file have been properly created in BASE2. If not, the plugin will fail. For all non-Affymetrix Array Designs,this means that Reporters have been loaded and Feature to Reporter Map have been attached to Array Designs. Those tasks can be only be achieved if the above plugins have been properly set up and configured. 

optional
-------
  - AffyArrayDesignBatchImporter Plugin . – for batch uploading affymetrix array designs. Useful to upload all array designs used in an experiment in one go before running the Tab2Mage Importer.

  If you want to use messaging, mail.jar and activation. jar is required- these jar files should be downloaded and copied to your base installation class path /web-inf/lib folder. 

================================================================

AnnotationTypes and Cv Importer
----------------------------------------------
  --the annotation type and cv loader plugin; uk.ac.ebi. nugo.plugins.CvAnnotationTypeImporter must be defined and configured. Both the plugin and it's configured object must be shared to the user running the Tab2MageImporter.  Defining and configuring the plugin should be done preferably by the administrator. The administrator should share the configuration to the users as well. Otherwise suitable plugin configurations cannot be found and the import would fail. 
  --A naming comparism of the annotation type in the flat file to the annotation type in base2 is actualized. Only non-existing annotation types are created.

===============================================================

Limitations and Known Issues
----------------------------------
i. Currently, Pooling event represented in tab2mage format are not supported. It would be supported from next plugin release .

ii. Only one raw data importer plugin configuration can be used at any one time during an experiment import.
This means that all raw  data files in a tab2mage.zip archive should have the same data header. For Raw datafile from Genepix (or .gpr files) and Agilent, there is a certain level of polymorphism that may occur should scanner models used in a experiment run different versions of the software. Users will have to harmonize the files to one layout in order to upload the Experiment in BASE2 using the current plugin.

iii. Agilent support:

Some Agilent Feature Extraction raw data files can be loaded but not all. BASE2.4.0 plugin API differs from BASE2.3.0 plugin API and even if users can successfully import Agilent Raw data files in BASE2.4, the tab2mage import plugin has yet to be updated to reflect the changes in the BASE plugin API. All the infrastructure is being modified to take advantage of the API modification and Agilent support will be available shortly. We will post an updated version of the plugin when ready and make an announcement through the base mailing lists 

===============================================================

Email alert:
---------------
To receive an email alert on the success or failure of your experiment loading,

Windows users:
---------------------
  Your antivirus server and anti spam software may prevent you from sending email messages. This is because they must have been configured to block all application requiring connection to your email sever on port 25 (smtp).
  If this happens, you can simply ask your administrator to allow your machine to connect to the email server using port 25.

  The email alert should work on a linux / unix and may be mac machines.