Changeset 2798

Timestamp:

Oct 24, 2006, 2:34:57 PM (16 years ago)

Author:

Jari Häkkinen

Message:

Updated information about affymetrix in BASE.

File:

• trunk/doc/affymetrix.txt (modified) (5 diffs)

trunk/doc/affymetrix.txt

@@ -23 +23 @@
 CEL file formats. There are EXP and RPT files, and also CHP files.
 
-
-
-Affymetrix/BASE issues:
-
-- The fact that BASE loads all raw result in a single table is a
-  problem for Affymetrix as Affymetrix chips soon have ~6 millions
-  probes.
-
-- Most of the plug-ins/tools freely available for Affymetrix expect
-  the Affymetrix file(s) as input, and since those are in an
-  Affymetrix binary format, it prevents from trying to hack these R
-  packages.
-
-- the fact that Affymetric has stuff like probeset, match/mismatch
-  oligos.
+CEL and CDF files are the only ones supported and needed by BASE.
 
 
@@ -79 +65 @@
 1) Loading Affymatrix probe values into the database is useless.
 
+   BASE stores CEL files as raw data, i.e. no data from the cel files
+   are pushed into the database.
+
+
 2) Plug-ins should be able to accept files as input.
+
+   The "RMAExpress plug-in for BASE" works with CEL and CDF files.
+
 
 3) BASE should be aware that raw data might not be stored in database
    tables but rather in files.
 
+   See 1)
+
+
 4) The dynamic part of BASE should also deal with data storage in
    files or database tables (or even a mix).
+
+   BASE stores files and directory hierarchies, and the RMAExpress
+   plug-ins pushes its result back into database tables.
+
 
 Clarification for item 3 and 4:
@@ -96 +96 @@
 Tentative list of descisions:
 
-A) Raw Affymetrix data will not be loaded into database
-   tables. [Target milestone: now]
-
-B) The Feature* classes/tables added in BASE 2 to allow for storage of
-   raw Affymetrix data are deprecated will be removed. [Target
-   milestone: BASE 2.0RC2]
-
-C) The CEL-files will be the raw Affymetrix data in BASE 2. This will
-   require some modifications to the core (rawdata related). [Target
-   milestone: BASE 2.0RC2]
-
 D) We have discussed to allow for four different types of imports,
    implemented in the following order:
@@ -114 +103 @@
       designs.
 
-   i) CEL-file import using a free C++ tool. The idea is (if possible)
-      to convert this to Java and ship it with BASE 2.0. [Target
-      milestone BASE 2.0RC2]
+      CDF files are stored as Reporter maps, i.e., array designs. CEL
+      files are associated with a CDF file when imported as raw
+      data. The RMAExpress plug-in requires a copy of the CDF when it
+      is running.
+
 
    ii) Create a CEL-file importer using the available R-packages. This
       will include normalization that provides provides minimum gcrma
       support from within base without other support based on
-      RMAExpress code [Target milestone BASE 2.0]
+      RMAExpress code.
 
-   iii) CHP-file import. For completeness but may be fairly
-      useless. No target release set since import of CHP data will
-      probably not be supported. [No target milestone]
+      Other normaliziation should be straightforward to incorporate
+      into the RMAExpress plug-in. If the plug-in is expanded, the
+      plug-in started should be extended to allow the user to choose
+      normalization scheme.
 
-   The current view on where the imported (non-raw) data should be
-   stored in the dynamic tables to allow other plug-ins to manipulate
-   the probe sets. Maybe we should in some cases store data in files?
-   [Target milestone for last question BASE 2.0+]?
-
-E) extended-properties.xml is data format dependent (as well as
-   migration dependent when migrating). The default is 2 channel
-   style. NuGO is to create a template for Affymetrix data. [Target
-   milestone BASE 2.0RC2]
-
-   This means that the extended-properties.xml file should include
-   Affymetrix related annotation (e.g. Ensembl gene id, Uniprot id,
-   refseq id...)
 
 F) Allow to create batch of slides in one go from any given Affymetrix
-   design using an import function [Target milestone BASE 2.0RC2]
+   design using an import function.
 
-G) Allow for upload and parsing of EXP and RPT files loading into
-   specific fields of relevant parameters (scaling
-   factor...). Philippe to document EXP and RPT format and important
-   tags. [Target milestone ?]
-
-
+   To be done.
 
 
 Other remarks and links:
-
-We have a Affymetrix data set available from
-/home/lev/jari/projects/base/base2/testdata/expt_111_cel.zip. The
-experiment is based on Affymetrix design MG_U74Av2. The archive
-contains cell files and a descriptive file formated according to the
-tab2mage specs (http://tab2mage.sourceforge.net/docs/spreadsheet.html)
 
 There is a resource for Affymetrix designs and annotations provided by
@@ -164 +132 @@
 http://www.bioconductor.org/packages/data/annotation/stable/src/contrib/html/
 
-RMAExpress is available at
-http://www.stat.berkeley.edu/~bolstad/RMAExpress/RMAExpress.html
-
-For item D.ii) One could use the annotation data package and use the
-Reporter Import plugin. This might requires some changes to the
-Reporter table to make it suitable fro Affymetrix annotation or just
-map to some of the fields.
-
-For item D.iii) Chip-to-chip normalization must be done, not possible
-with .CHP files. This will require a plug-in (with documentation). If
-R (BioConductor) is to be used we should contact Henrik Bengtsson,
-creator of a BASE 1.2 R dispatcher.
+For R (BioConductor) we should contact Henrik Bengtsson, creator of a
+BASE 1.2 R dispatcher.
 
 Future development: Connectivity to R/bioc and develop specific