Changeset 457


Ignore:
Timestamp:
Oct 31, 2007, 6:18:49 PM (14 years ago)
Author:
dominic
Message:

commit file for the latest version of the tab2mage importer

Location:
trunk/uk/ac/ebi/Tab2MageImporter
Files:
2 added
4 edited

Legend:

Unmodified
Added
Removed
  • trunk/uk/ac/ebi/Tab2MageImporter/ChangeLog.txt

    r417 r457  
    11,____Second release
    22
    3 
    4 -- pooling support added
    5 -- isisues relating to agilent files
    6 -- logging
    7 --
    8 
    9 
    10 This file will contain information about changes iin subsequent releases.
     3-- pooling support added (read the readme and pooling documentation for more details)
     4-- issues relating to agilent files, as stated in the 'known issues section' of the last release are now resolved
     5-- logging and  new validation reports
     6-- enhanced annotation handling
     7-- users with unlimited quota can now run tab2mage importer
     8-- the import can now be executed in a validation mode, to first validate the tab2mage spreadsheet before loading in to BASE. please see the instructions on the GUI when running the plugin for guidance.
     9-- transaction error while importing or attaching raw data to bioassay item fixed.
  • trunk/uk/ac/ebi/Tab2MageImporter/Importer FAQ.txt

    r408 r457  
    22======
    33Q:  Why do I get net.sf.basedb.core.BaseException: IMPORT FAILS: Invalid File format:
    4 files supplied not appropriate for the experiment platform: [ raw data type/plaffrom] when I re-run the plugin
     4files supplied not appropriate for the experiment platform: [ raw data type/platform] when I re-run the plugin
    55
    66A:  Whenever you run the plugin  an unzipped file directory may have been created from the zip file, with the same name  as the zip file. Whilst you re- run the plugin using the same zip file, an existing directory cannot be re-created. 
  • trunk/uk/ac/ebi/Tab2MageImporter/README.txt

    r440 r457  
    1 ============================================================
    2 General Information:
    3 ---------------------------
    4 
    5 This plugin imports raw data files and an experiment meta-data in the tab2mage specification into base 2. The tab2mage specification is provided by the microarray informatics team at the EBI, Cambridge. The imported experiment can later be exported by the corresponding tab2mage export plugin. (link).
    6 
    7 The entry point for this plugin in base is the detail view page of an experiment. Hence, the user needs to create an experiment first and execute the plugin from this page by clicking on the import tab. The displayed wizards asked for the zip file containing the meta- data i.e. tab2mage file and the raw data files. Furthermore, the user’s  email server name could be specified to received notices of success or failure of the uploading or can be omitted to receive no message. 
    8 
     1===========================================================
    92Installation
    103---------------
     
    3427  11. For other users to use the plugin as detailed in the next section, please share them to the appropriate users by manually setting the permissions on the item. (sharing it to group 'Everyone' should be sufficient in most cases).
    3528
    36  
     29  12. PERMISSION for users and/or the tab2mage importer plugin is important to run an import successfully. Please check the WARNING in the 'USE' section below
     30
    3731  General Installation instruction is available at: http://base.thep.lu.se/chrome/site/doc/html/admindoc/plugin_installation/plugins.installation.html#plugins.installation
    3832   
    3933===========================================================
     34 Use
     35=======
     36Warning:
     37----
     38  --  User with a specified quota must have the required capacity to store files and create base items. Note that user can have unlimited quota.
     39  --  Re- running the import plugin without having deleted the previously created directory from the zip file will make the Importer fail. Please delete from the trash can too.
     40  --  Note that in its current implementation, only one RawDataImporter plugin configuration can be used during the import process. This means that raw data files must be      consistent throughout. (this might be an issue when using different scanner versions).
     41  --  The importer can be executed in a validation mode, to first validate the tab2mage spreadsheet before loading in to BASE. Please see the instructions on the GUI   when running the plugin for guidance.
    4042
    41  Use
    42 --------
    43  Warning:
    44   -- Do not execute the plugin as root but as a user with a specified quota.  The quota must be enough to store files and create base items.
    45   --  re- running the import plugin without having deleted the previously created directory from the zip file will make the   Importer fail. please delete from the trash can too.
    46   --  Note that in its current implementation, only one RawDataImporter plugin configuration can be used during the import process. This means that raw data files must be  consistent throughout. (this might be an issue when using different scanner versions).
    47   -- - Make sure that the user running Tab2Mage Importer plugin has enough permissions to do so. With the default Roles settings in BASE, User and PowerUser have not   enough permissions to run the plugin successfully.
     43  Permission
     44  ----------------
     45  --  Make sure that the user running Tab2Mage Importer plugin has enough permission to do so. With the default Roles settings in BASE, User and PowerUser roles have     not enough permission to run the plugin successfully. There are two options available:
     46  i.  Edit the priviledge of the power user or user role in base.
     47  ii. Assign permissions to the Tab2Mage Importer Plugin. The plugin does not use permission by default, but the administrator can set permission for the plugin. Please see :http://base.thep.lu.se/chrome/site/doc/html/admindoc/plugin_installation/plugins.permissions.html for setting plugin permissions.
    4848 
    49   1. Platform supported  are: Affymetrix, Agilent, Genepix
    50   2. Create all the array design used in the experiment as the BASE administrator. Array Design must then be shared to everyone or a suitable group that all the target users belong.
     49  1. Platforms supported  are: Affymetrix, Agilent, Genepix
     50  2. Create all the array design used in the experiment as the BASE administrator. Array Design must then be shared to everyone or a suitable group that all the target users   belong.
    5151  3. Create tab2mage spreadsheet and zip it together with the raw data files  that make up the experiment. Note: no cdf file. Upload this file to your account on BASE (go to View -> Files -> Upload File). Image files may be supplied by the user and contained in the zip file.
    52   Note: To avoid the laborious process of checking and formatting your tab2mage spreadsheet. Please use the example files: tab2mage- agilent, tab2mage- genepix and tab2mage- affymetrix, provided with the distribution as the spreadsheet template. For instance, the protocol section of your spreadsheet must have the protocol type specified.If not the plugin will fail.
     52  Note: To avoid the laborious process of checking and formatting your tab2mage spreadsheet. Please use the example files: tab2mage- agilent, tab2mage- genepix and tab2mage- affymetrix, provided with the distribution as the spreadsheet template. For instance, the protocol section of your spreadsheet must have the protocol type specified.If not the plugin will fail. Running the import in validation mode will help identify all potential problems.
    5353  4. To make sure you have all the required reporters and import configurations in place, import a single data file from the batch as a test first. 
    54   5. If the previous step was successful, create a new experiment  and specify the raw data type ( remember only affymetrix, genepix and agilent  platfroms are supported) . Ensure that  the experiment is of the same data type as the data in the files you want to import. Click the import tab from the plugin menu, select Tab2Mage Importer and click Next.
     54  5. If the previous step was successful, create a new experiment  and specify the raw data type ( remember only affymetrix, genepix and agilent  platforms are supported) . Ensure that  the experiment is of the same data type as the data in the files you want to import. Click the import tab from the plugin menu, select Tab2Mage Importer and click Next.
    5555  6. Open the detail view of the newly created experiment and click on the import tab.
    56   7. In the dialog that opens, leave the settings for Plugin and File format on auto-detect and click Next.
     56  7. In the dialog that opens,  leave the settings for Plugin and File format on auto-detect and click Next.
    5757  8 On the next screen, Select the batch file that contains the raw data files, the Tab2Mage text file and the image file if supplied  and  click Next.
    58   9. On the next screen, the ZipFile and Experiment parameters should already be configured now (as indicated by the black cross against them). Enter the name of your Tab2Mage file as a value for the "Tab2Mage FileName with Extension" parameter.  Specify  your email server if you want to receive notification on the success or failure of your experiment loading and click Next.
    59   10. On the next screen click Finish and wait for the job to complete. You will be shown the progress of the Tab2mage import in the dialog, with the page refreshing itself regularly.
     58  9. On the next screen, the ZipFile and Experiment parameters should already be configured now (as indicated by the black cross against them). Enter the name of your Tab2Mage file as a value for the "Tab2Mage FileName with Extension" parameter.  Specify  your email server if you want to receive notification on the success or failure of your experiment loading.
     59  10. If you wish, you can change the default filename for the "Save Validation Report As" parameter. If the validation report already exists indicate whether it should overwritten with the "overwrite" parameter.
     60  11. To run the importer in validation mode only, set 'Run validation' parameter to true and click Next.
     61  12. On the next screen click Finish and wait for the job to complete. You will be shown the progress of the Tab2mage import in the dialog, with the page refreshing itself regularly.
    6062
    61   When the import has finished, you can check and review the uploaded experiment by navigating thorughbase2 web client, using the menu. If you have provided your email server name. You would receive an email on status of the import.
     63  When the import has finished, you can check and review the uploaded experiment by navigating through base2 web client, using the menu. If you have provided your email server name. You would receive an email message on the status of the import.
    6264
    6365 ===========================================================
    64 
    65 Dependencies:
    66 --------------------
    67  
    68   - Base 2.2 and above
    69 
    70   -Reporter Importer Plugin - Required for loading Reporters
    71 
    72   Reporter Map Importer Plugin – Required for mapping features to Reporters when defining an Array Design
    73  
    74   Raw Data Importer Plugin – Required for loading raw data and import values associated to BASE rawBioassays. This is a critical requirement, for multi-channel experiments, before running the Tab2MageImporter. Users must ensure that raw data import plugin has   been properly configured and is able to import a raw datafile of the type of the submission.
    75 
    76   The definition for the above plug-ins is available by default in Base 2. The plugins should be configured by a Base 2 administrator and shared to everyone or a suitable group of target users.
    77 
    78   AnnotationTypeCvImporter Plugin – Required for loading annotation types. The plugin must have a proper configuration defined to import an annotation type file created by the Tab2MageImporter plugin when parsing the tab2mage file.
    79 
    80   Please ensure that you have used all the plugins above appropriately, before running Tab2Mage importer. Click AnnotationTypeCvImporter for further information on Annotation Type and Cv Importer plugin and for other plugins, please read base2 documentation.
    81 
    82   Also, do make sure that Array Design(s) corresponding to the one(s) referenced in the Array[accession] field of the tab2mage file have been properly created in BASE2. If not, the plugin will fail. For all non-Affymetrix Array Designs,this means that Reporters have been loaded and Feature to Reporter Map have been attached to Array Designs. Those tasks can be only be achieved if the above plugins have been properly set up and configured.
    83 
    84 optional
    85 -------
    86   - AffyArrayDesignBatchImporter Plugin . – for batch uploading affymetrix array designs. Useful to upload all array designs used in an experiment in one go before running the Tab2Mage Importer.
    87 
    88   If you want to use messaging, mail.jar and activation. jar is required- these jar files should be downloaded and copied to your base installation class path /web-inf/lib folder.
    89 
    90 ================================================================
    91 
    9266AnnotationTypes and Cv Importer
    9367----------------------------------------------
     
    9670
    9771===============================================================
    98 
    9972Limitations and Known Issues
    10073----------------------------------
    101 i. Currently, Pooling event represented in tab2mage format are not supported. It would be supported from next plugin release .
    102 
    103 ii. Only one raw data importer plugin configuration can be used at any one time during an experiment import.
     74i. Only one raw data importer plugin configuration can be used at any one time during an experiment import.
    10475This means that all raw  data files in a tab2mage.zip archive should have the same data header. For Raw datafile from Genepix (or .gpr files) and Agilent, there is a certain level of polymorphism that may occur should scanner models used in a experiment run different versions of the software. Users will have to harmonize the files to one layout in order to upload the Experiment in BASE2 using the current plugin.
    10576
    106 iii. Agilent support:
    107 
    108 Some Agilent Feature Extraction raw data files can be loaded but not all. BASE2.4.0 plugin API differs from BASE2.3.0 plugin API and even if users can successfully import Agilent Raw data files in BASE2.4, the tab2mage import plugin has yet to be updated to reflect the changes in the BASE plugin API. All the infrastructure is being modified to take advantage of the API modification and Agilent support will be available shortly. We will post an updated version of the plugin when ready and make an announcement through the base mailing lists
    109 
    11077===============================================================
    111 
    11278Email alert:
    11379---------------
     
    12086
    12187  The email alert works on a linux / unix and may be mac machines.
     88==============================================================
  • trunk/uk/ac/ebi/Tab2MageImporter/simplied process.txt

    r434 r457  
    1111
    1212- go to Administrate -> Plugins -> Definitions
    13   Click New to create the Affymetrix Array Design Batch Importer Plugin. Follow  further instructions on : plugin website if required.
     13  Click New to create the Affymetrix Array Design Batch Importer Plugin. Follow  further instructions on the plugin website if required.
    1414
    1515- go to View - > File
     
    2929  click browse to select the uploaded xml file (annotationtypecvimporter.xml), click next and finish
    3030
    31   Follow further instructions on : plugin website if required
     31  Follow further instructions on the plugin website if required
    3232
    3333- go to Administrate -> Plugins -> Definitions
     
    4242
    4343
    44 PRESTO! following the above process should upload the experiment for you.
     44PRESTO! following the above process, the experiment should upload without problems.
    4545 
    4646 
Note: See TracChangeset for help on using the changeset viewer.