Changeset 4083


Ignore:
Timestamp:
Jan 17, 2008, 11:08:21 AM (14 years ago)
Author:
Nicklas Nordborg
Message:

References #894: Support for array designs with other feature identification methods than coordinates

Everything should now be fixed. May need some more testing before closing this ticket.

Location:
trunk
Files:
24 edited

Legend:

Unmodified
Added
Removed
  • trunk/doc/src/docbook/appendix/incompatible.xml

    r4080 r4083  
    5252    <para>
    5353      Array design features can now be identified by three different methods:
    54       COORDINATES, POSITION and EXTERNAL_ID. The coordinates method was the
     54      COORDINATES, POSITION and FEATURE_ID. The coordinates method was the
    5555      only one supported earlier.
    5656    </para>
     
    6464      <classname docapi="net.sf.basedb.core">FeatureBatcher</classname> or
    6565      <classname docapi="net.sf.basedb.core">RawDataBatcher</classname> are
    66       probably affected by this. For example, includes raw data importers which
    67       doesn't know how to set the position or the feature ID when those identification
    68       methods are used.
     66      probably affected by this. For example, a raw data importer which
     67      doesn't know how to set the position or the feature ID can't import data to
     68      an array design that is using one of the new identification methods.
    6969      </para>
    7070    </listitem>
  • trunk/doc/src/docbook/userdoc/rawbioassays.xml

    r3940 r4083  
    138138            The array design can be changed after raw data has been
    139139            imported, but this triggers a new validation. If the raw data
    140             is stored in the database, the position, coordinates and reporter of all features
    141             on the new array design must exactly match the position,
    142             coordinates and reporter of the raw data. For Affymetrix data, the
     140            is stored in the database, the features on the new array design must
     141            match the the raw data. The verification can use three different methods:
     142          </para>
     143         
     144          <itemizedlist>
     145          <listitem>
     146            <para>
     147            Coordinates: Verify block, meta-grid, row and column coordinates.
     148            </para>
     149          </listitem>
     150          <listitem>
     151            <para>Position: Verify the position number.</para>
     152          </listitem>
     153          <listitem>
     154            <para>
     155            Feature ID: Verify the feature ID. This option can only be used
     156            if the raw bioassay is currently connected to an array design that
     157            has feature ID values already.
     158            </para>
     159          </listitem>
     160          </itemizedlist>
     161          <para>
     162            In all three cases it is also verified that the reporter of the raw
     163            data matches the reporter of the features.
     164          </para>
     165
     166          <para>
     167            For Affymetrix data, the
    143168            CEL file is validated against the CDF file of the new array design.
    144169            If the validation fails, the array design is not changed.
  • trunk/src/clients/migrate/net/sf/basedb/clients/migrate/RawBioAssayDataTransfer.java

    r3941 r4083  
    168168     */
    169169    cleanUpMemory();
    170     RawDataBatcher batcher = rawBioAssay.getRawDataBatcher();
     170    RawDataBatcher batcher = rawBioAssay.getRawDataBatcher(null);
    171171    batcher.setBatchSize(getBatchSize());
    172172    runRawBioAssay(rawBioAssay, batcher, hasArrayDesign, rawBioAssayId);
  • trunk/src/core/net/sf/basedb/core/FeatureBatcher.java

    r4080 r4083  
    5454  <li>COORDINATES: {@link FeatureData#setRow(int)} and {@link FeatureData#setColumn(int)}
    5555  <li>POSITION: {@link FeatureData#setPosition(int)}
    56   <li>EXTERNAL_ID: {@link FeatureData#setExternalId(String)}
     56  <li>FEATURE_ID: {@link FeatureData#setExternalId(String)}
    5757  </ul>
    5858 
    5959  For all methods, each feature that is inserted must have a unqiue value. For example,
    60   there can't be two features with the same external ID if the EXTERNAL_ID method is
     60  there can't be two features with the same external ID if the FEATURE_ID method is
    6161  used. Values for the methods that are not used can still be set and saved to the
    6262  database, but the uniqueness is not checked. The position number will be auto-generated
     
    7979      add features to
    8080    @throws BaseException If there is an error
    81     @see ArrayDesign#getFeatureBatcher()
     81    @see ArrayDesign#getFeatureBatcher(FeatureIdentificationMethod)
    8282  */
    8383  static FeatureBatcher getNew(DbControl dc, ArrayDesign arrayDesign)
  • trunk/src/core/net/sf/basedb/core/FeatureIdentificationMethod.java

    r4080 r4083  
    111111    Use the external feature ID.
    112112  */
    113   EXTERNAL_ID(3, true, "External ID")
     113  FEATURE_ID(3, true, "Feature ID")
    114114  {
    115115    /**
  • trunk/src/core/net/sf/basedb/core/RawBioAssay.java

    r4020 r4083  
    5858  <p>
    5959  Raw data is added by a {@link RawDataBatcher} object. You can get a
    60   batcher for this raw bioassay by calling the {@link #getRawDataBatcher()}
    61   method.
     60  batcher for this raw bioassay by calling the {@link
     61  #getRawDataBatcher(FeatureIdentificationMethod)} method.
    6262
    6363  @author Nicklas
     
    418418      if (getRawDataType().isStoredInDb() && getSpots() > 0)
    419419      {
    420         RawDataBatcher rdb = RawDataBatcher.getNew(getDbControl(), this);
     420        RawDataBatcher rdb = RawDataBatcher.getNew(getDbControl(), this, null);
    421421        rdb.deleteAll();
    422422      }
     
    731731    getData().setArrayDesign(arrayDesign == null ? null : arrayDesign.getData());
    732732  }
    733  
     733
     734  /**
     735    @deprecated Use {@link #updateArrayDesign(ArrayDesign, FeatureIdentificationMethod, ProgressReporter)}
     736    instead
     737  */
     738  public void updateArrayDesign(ArrayDesign arrayDesign, ProgressReporter progress)
     739    throws PermissionDeniedException, InvalidDataException
     740  {
     741    updateArrayDesign(arrayDesign, null, progress);
     742  }
     743
    734744  /**
    735745    Set the {@link ArrayDesign} this raw data uses. If data has already been
     
    739749 
    740750    @param arrayDesign The <code>ArrayDesign</code> item or null if not known
     751    @param fiMethod The method to use for identifying features on the
     752      new array design. Use null to use the method choosen by the new array design.
     753      The {@link FeatureIdentificationMethod#FEATURE_ID} will not work unless the
     754      raw bioassay is already connected to an array design which has features with
     755      external ID:s
    741756    @param progress An optional progress reporter
    742757    @throws PermissionDeniedException If the logged in user doesn't have
     
    745760      match the raw data
    746761    @throws BaseException If there is another error
    747     @since 2.4
    748   */
    749   public void updateArrayDesign(ArrayDesign arrayDesign, ProgressReporter progress)
     762    @since 2.6
     763  */
     764  public void updateArrayDesign(ArrayDesign arrayDesign, FeatureIdentificationMethod fiMethod, ProgressReporter progress)
    750765    throws PermissionDeniedException, InvalidDataException
    751766  {
     
    760775    if (getData().getSpots() > 0 && arrayDesign != null)
    761776    {
    762       validateFeatures(getDbControl(), arrayDesign, true, progress);
     777      validateFeatures(getDbControl(), arrayDesign, fiMethod, true, progress);
    763778    }
    764779    getData().setArrayDesign(arrayDesign == null ? null : arrayDesign.getData());
     
    956971    return query;
    957972  }
     973
     974  /**
     975    @deprecated Use {@link #getRawDataBatcher(FeatureIdentificationMethod)} instead
     976  */
     977  public RawDataBatcher getRawDataBatcher()
     978    throws PermissionDeniedException, BaseException
     979  {
     980    return getRawDataBatcher(null);
     981  }
    958982 
    959983  /**
    960984    Get a raw data batcher object which can be used to add raw data to
    961     the raw bioassay.
     985    the raw bioassay.
     986    @param fiMethod The method to use for identifying features, or null to
     987      use the method choosen by the array design. This parameter has no meaning
     988      if the raw bioassay is not connected to an array design or if the array
     989      design has no features.
     990    @return A <code>RawDataBatcher</code> object. If this method is called multiple
     991      times the fiMethod parameter is ignored and the same batcher as before
     992      is returned
    962993    @throws PermissionDeniedException If raw data has already been added
    963994      or the logged in user doesn't have write permission
    964995    @throws BaseException If there is another error
    965   */
    966   public RawDataBatcher getRawDataBatcher()
     996    @since 2.6
     997  */
     998  public RawDataBatcher getRawDataBatcher(FeatureIdentificationMethod fiMethod)
    967999    throws PermissionDeniedException, BaseException
    9681000  {
    9691001    checkPermission(Permission.WRITE);
    970     if (hasData())
     1002    if (getNumDbSpots() > 0)
    9711003    {
    9721004      throw new PermissionDeniedException("Raw data has already been added to "+this);
     
    9741006    if (rawDataBatcher == null)
    9751007    {
    976       rawDataBatcher = RawDataBatcher.getNew(getDbControl(), this);
     1008      rawDataBatcher = RawDataBatcher.getNew(getDbControl(), this, fiMethod);
    9771009    }
    9781010    return rawDataBatcher;
     
    10041036    if (getNumDbSpots() > 0)
    10051037    {
    1006       RawDataBatcher rdb = RawDataBatcher.getNew(getDbControl(), this);
     1038      RawDataBatcher rdb = RawDataBatcher.getNew(getDbControl(), this, null);
    10071039      rdb.deleteAll();
    10081040    }
     
    10731105    features/reporter on the array design.
    10741106    @param dc The DbControl to use for database access
    1075     @param design The array design to validate agains
     1107    @param design The array design to validate against
     1108    @param fiMethod The method to use for identifying features, or null
     1109      to use the method choosen by the array design
    10761110    @param update If the raw data should be linked to the new feature or not
    10771111    @param progress An optional progress reporter
    1078     @since 2.4
    1079   */
    1080   private void validateFeatures(DbControl dc, ArrayDesign design, boolean update, ProgressReporter progress)
     1112    @since 2.6
     1113  */
     1114  private void validateFeatures(DbControl dc, ArrayDesign design, FeatureIdentificationMethod fiMethod,
     1115    boolean update, ProgressReporter progress)
    10811116  {
    10821117    if (design.hasFeatures())
    10831118    {
     1119      if (fiMethod == null) fiMethod = design.getFeatureIdentificationMethod();
     1120      if (fiMethod == FeatureIdentificationMethod.NONE) fiMethod = FeatureIdentificationMethod.COORDINATES;
     1121     
    10841122      if (progress != null) progress.display(5, "Loading features from array design...");
    10851123     
     
    10891127      String entityName = getRawDataType().getEntityName();
    10901128     
    1091       // Preload features from array design
    1092       org.hibernate.Query query = HibernateUtil.getPredefinedQuery(dc.getStatelessSession(),
    1093         "COUNT_FEATURES_FOR_ARRAYDESIGN");
    1094       /*
    1095         SELECT count(*)
    1096         FROM FeatureData f
    1097         WHERE f.arrayDesignBlock.arrayDesign = :arrayDesign
    1098       */
    1099       query.setInteger("arrayDesign", design.getId());
    1100       int numFeatures = HibernateUtil.loadData(Long.class, query).intValue();
    1101       Map<Integer, FeatureData> features = new HashMap<Integer, FeatureData>(numFeatures);
    1102       query = HibernateUtil.getPredefinedQuery(dc.getStatelessSession(), "PRELOAD_FEATURES");
     1129      int numFeatures = design.getNumDbFeatures();
     1130      Map<Object, FeatureData> features = new HashMap<Object, FeatureData>(numFeatures);
     1131      org.hibernate.Query query = HibernateUtil.getPredefinedQuery(dc.getStatelessSession(), "PRELOAD_FEATURES");
    11031132      /*
    11041133        SELECT f
     
    11141143      {
    11151144        FeatureData f = si.next();
    1116         features.put(f.getPosition(), f);
     1145        Object featureId = fiMethod.getIdentifier(f);
     1146        if (features.put(featureId, f) != null)
     1147        {
     1148          throw new InvalidDataException("Can't identify features with " + fiMethod +
     1149            ". Found duplicate feature: " + featureId);
     1150        }
    11171151      }
    11181152      si.close();
     
    11211155      if (progress != null) progress.display(10, "Loading raw data from raw bioassay...");
    11221156      DataQuery<RawData> rawQuery = getRawData();
     1157      // NOTE!! Due to an issue in Hibernate we must join all three
     1158      // tables or we will get an AssertionFailure: possible non-threadsafe access to the session
     1159      // The query also works if don't join at all, but that will generate a lot
     1160      // of extra queries for each iteration
     1161      rawQuery.join(Hql.leftJoin(null, "reporter", "rep", null, true));
     1162      rawQuery.join(Hql.leftJoin(null, "feature", "feat", null, true));
     1163      rawQuery.join(Hql.leftJoin("feat", "arrayDesignBlock", "bl", null, true));
     1164
    11231165      DataResultIterator<RawData> result = rawQuery.iterate(dc);
    11241166      int numValidated = 0;
     
    11271169      {
    11281170        RawData raw = result.next();
    1129         int position = raw.getPosition();
    1130         FeatureData feature = features.get(position);
    1131         if (feature == null)
     1171
     1172        // Get the feature identifier for the current raw data
     1173        FeatureData oldFeature = raw.getFeature();
     1174        String externalFeatureId = oldFeature == null ? null : oldFeature.getExternalId();;
     1175        Object rawFeatureId = fiMethod.getIdentifier(raw, externalFeatureId);
     1176       
     1177        // Find the new feature with the same identifier
     1178        FeatureData newFeature = features.get(rawFeatureId);
     1179        if (newFeature == null)
    11321180        {
    11331181          throw new InvalidDataException("Array design '" + design.getName() +
    1134             "' is missing a feature at position " + raw.getPosition());
     1182            "' is missing a feature for " + fiMethod + "=" + rawFeatureId);
    11351183        }
    11361184       
    11371185        // Check that reporters are the same
    11381186        ReporterData rawReporter = raw.getReporter();
    1139         ReporterData featureReporter = feature.getReporter();
     1187        ReporterData featureReporter = newFeature.getReporter();
    11401188        if (rawReporter == null)
    11411189        {
    11421190          if (featureReporter != null)
    11431191          {
    1144             throw new InvalidDataException("Reporter mismatch at position " + position +
     1192            throw new InvalidDataException("Reporter mismatch at " +  fiMethod + "=" + rawFeatureId +
    11451193              "; raw reporter=none, feature reporter=" + featureReporter.getExternalId());
    11461194          }
     
    11501198          if (featureReporter == null)
    11511199          {
    1152             throw new InvalidDataException("Reporter mismatch at position " + position +
     1200            throw new InvalidDataException("Reporter mismatch at " + fiMethod + "=" + rawFeatureId +
    11531201              "; raw reporter=" + rawReporter.getExternalId() + ", feature reporter=none");
    11541202          }
    11551203          else if (rawReporter.getId() != featureReporter.getId())
    11561204          {
    1157             throw new InvalidDataException("Reporter mismatch at position " + position +
     1205            throw new InvalidDataException("Reporter mismatch at " + fiMethod + "=" + rawFeatureId +
    11581206              "; raw reporter=" + rawReporter.getExternalId() +
    11591207              ", feature reporter=" + featureReporter.getExternalId());
    11601208          }
    11611209        }
    1162          
    1163         // Check that coordinates are the same
    1164         FeatureCoordinate featureCoordinate = Feature.getFeatureCoordinate(feature);
    1165         FeatureCoordinate rawCoordinate = new FeatureCoordinate(raw.getBlock(),
    1166           raw.getMetaGridX(), raw.getMetaGridY(), raw.getRow(), raw.getColumn());
    1167        
    1168         if (!featureCoordinate.equals(rawCoordinate))
    1169         {
    1170           throw new InvalidDataException("Coordinate mismatch at position " + position +
    1171             "; raw=" + rawCoordinate +", feature=" + featureCoordinate);
    1172         }
    1173        
     1210   
    11741211        // All is ok, re-assign raw data to the new feature
    11751212        if (update)
    11761213        {
    1177           metaData.setPropertyValue(raw, "feature", feature, EntityMode.POJO);
     1214          metaData.setPropertyValue(raw, "feature", newFeature, EntityMode.POJO);
     1215          raw.setPosition(newFeature.getPosition());
    11781216          session.update(entityName, raw);
    11791217        }
  • trunk/src/core/net/sf/basedb/core/RawDataBatcher.java

    r4080 r4083  
    9292    @param rawBioAssay The raw bioassay this batcher should
    9393      add raw data to
     94    @param fiMethod The method to use for identifying features, use null
     95      to use the method specified by the array design
    9496    @throws BaseException If there is an error
    95     @see RawBioAssay#getRawDataBatcher()
    96   */
    97   static RawDataBatcher getNew(DbControl dc, RawBioAssay rawBioAssay)
    98     throws BaseException
    99   {
    100     RawDataBatcher rdb = new RawDataBatcher(dc, rawBioAssay);
     97    @see RawBioAssay#getRawDataBatcher(FeatureIdentificationMethod)
     98  */
     99  static RawDataBatcher getNew(DbControl dc, RawBioAssay rawBioAssay, FeatureIdentificationMethod fiMethod)
     100    throws BaseException
     101  {
     102    RawDataBatcher rdb = new RawDataBatcher(dc, rawBioAssay, fiMethod);
    101103    rdb.initPermissions(0, 0);
    102104    return rdb;
     
    164166  private boolean useNullIfReporterNotFound = true;
    165167 
    166   RawDataBatcher(DbControl dc, RawBioAssay rawBioAssay)
     168  RawDataBatcher(DbControl dc, RawBioAssay rawBioAssay, FeatureIdentificationMethod fim)
    167169    throws BaseException
    168170  {
     
    178180    }
    179181
    180     this.fiMethod = arrayDesign != null && arrayDesign.getHasFeatures() ?
    181         FeatureIdentificationMethod.fromValue(arrayDesign.getFeatureIdentificationMethod()) :
    182         FeatureIdentificationMethod.NONE;
    183 
    184     if (fiMethod != FeatureIdentificationMethod.NONE)
     182    // Use the feature identification method set by the array design if none was given
     183    this.fiMethod = fim;
     184    if (arrayDesign != null && arrayDesign.getHasFeatures())
     185    {
     186      if (fiMethod == null)
     187      {
     188        fiMethod = FeatureIdentificationMethod.fromValue(arrayDesign.getFeatureIdentificationMethod());
     189      }
     190      // As a last resort fallback to the coordinates method
     191      if (fiMethod == null || fiMethod == FeatureIdentificationMethod.NONE)
     192      {
     193        fiMethod = FeatureIdentificationMethod.COORDINATES;
     194      }
     195    }
     196    else
     197    {
     198      fiMethod = FeatureIdentificationMethod.NONE;
     199    }
     200   
     201    // Preload features
     202    if (fiMethod != FeatureIdentificationMethod.NONE && rawBioAssay.getNumDbSpots() == 0)
    185203    {
    186204      org.hibernate.Query query = HibernateUtil.getPredefinedQuery(dc.getStatelessSession(), "COUNT_FEATURES_FOR_ARRAYDESIGN");
     
    208226        FeatureData feature = si.next();
    209227        Object featureId = fiMethod.getIdentifier(feature);
    210         preloaded.put(featureId, feature);
     228        if (preloaded.put(featureId, feature) != null)
     229        {
     230          throw new InvalidDataException("Can't identify features with " + fiMethod +
     231            ". Found duplicate feature: " + featureId);
     232        }
    211233      }
    212234      si.close();
     
    309331    {
    310332      granted |= Permission.grant(Permission.WRITE, Permission.DELETE, Permission.CREATE);
     333      if (rawBioAssay.getNumDbSpots() > 0)
     334      {
     335        denied |= Permission.deny(Permission.CREATE);
     336      }
    311337    }
    312338    super.initPermissions(granted, denied);
  • trunk/src/core/net/sf/basedb/core/data/ArrayDesignData.java

    r4080 r4083  
    159159  /**
    160160    The method to use for identiying features.
    161     1 = COORDINATES, 2 = POSITION, 3 = EXTERNAL_ID
     161    1 = COORDINATES, 2 = POSITION, 3 = FEATURE_ID
    162162    @hibernate.property column="`identification_method`" type="int" not-null="true"
    163163    @since 2.6
  • trunk/src/plugins/core/net/sf/basedb/plugins/IlluminaRawDataImporter.java

    r4080 r4083  
    3838
    3939
     40import net.sf.basedb.core.ArrayDesign;
    4041import net.sf.basedb.core.BaseException;
    4142import net.sf.basedb.core.BasicItem;
     
    109110   
    110111  <li>
    111     If the array design uses the {@link FeatureIdentificationMethod#EXTERNAL_ID}
     112    If the array design uses the {@link FeatureIdentificationMethod#FEATURE_ID}
    112113    method for identifying features. The plug-in assumes that the feature ID is the
    113114    same as the reporter ID.
     
    151152    ));
    152153 
     154  private static final PluginParameter<String> featureIdentificationParameter =
     155    new PluginParameter<String>(
     156      "featureIdentification",
     157      "Identify features by",
     158      "Choose which method to use for identifying features. If no value " +
     159      "is selected the identification method used on the array design is used: \n\n" +
     160      "* COORDINATES: Use auto-generated coordinates (block = 1, column = 1, row = line number in file)\n" +
     161      "* POSITION: Use auto-generated position = line number in file\n" +
     162      "* FEATURE_ID: Use TargetID (requires that each Target ID only appears once in the file)\n\n" +
     163      "NOTE! This parameter has no meaning unless an array design is selected",
     164      new StringParameterType(255, null, false, 1, 0, 0,
     165        Arrays.asList(new String[] {"COORDINATE", "POSITION", "FEATURE_ID" } ))
     166    );
     167
     168 
    153169  private static final PluginParameter<String> invalidColumnsErrorParameter = new PluginParameter<String>(
    154170      "invalidColumnsError",
     
    277293        // Associations
    278294        storeValue(job, request, ri.getParameter("experiment"));
     295        storeValue(job, request, ri.getParameter("arrayDesign"));
     296        storeValue(job, request, featureIdentificationParameter);
    279297        storeValue(job, request, ri.getParameter("scan"));
    280298        storeValue(job, request, ri.getParameter("protocol"));
     
    306324  private DbControl dc;
    307325  private Experiment experiment;
     326  private ArrayDesign design;
    308327  private Scan scan;
    309328  private Software software;
    310329  private Protocol protocol;
     330  private FeatureIdentificationMethod fiMethod;
    311331  private List<RawBioAssay> rawBioAssays;
    312332  private List<BatchAndMapHolder> holders;
     
    369389    this.headerLines = new LinkedList<Line>();
    370390    this.experiment = (Experiment)job.getValue("experiment");
     391    this.design = (ArrayDesign)job.getValue("arrayDesign");
    371392    this.scan = (Scan)job.getValue("scan");
    372393    this.protocol = (Protocol)job.getValue("protocol");
    373394    this.software = (Software)job.getValue("software");
    374395   
     396    // Feature identification
     397    try
     398    {
     399      String fiTemp = (String)job.getValue("featureIdentification");
     400      fiMethod = FeatureIdentificationMethod.valueOf(fiTemp);
     401    }
     402    catch (Exception ex)
     403    {
     404      fiMethod = null;
     405    }
    375406    // Setup error handling
    376407    this.nullIfException = "null".equals(getErrorOption("numberFormatError"));
     
    429460      // Need to reload raw bioassay with current DbControl
    430461      rba = RawBioAssay.getById(dc, rba.getId());
    431       RawDataBatcher batcher = rba.getRawDataBatcher();
     462      RawDataBatcher batcher = rba.getRawDataBatcher(fiMethod);
    432463      batcher.useNullIfReporterNotFound(nullIfMissingReporter);
    433464      holders.add(new BatchAndMapHolder(batcher, illumina, ffp));
     
    566597            RawBioAssay rba = RawBioAssay.getNew(dc, generic, illumina);
    567598            rba.setName(arrayName);
     599            if (design != null) rba.setArrayDesign(design);
    568600            if (scan != null) rba.setScan(scan);
    569601            if (protocol != null) rba.setProtocol(protocol);
     
    618650        parameters.add(getCharsetParameter(null, null, null));
    619651 
    620         // parameters for scan, protocol and software
     652        // parameters for scan, protocol, software and array design
    621653        dc = sc.newDbControl();
    622654        List<Scan> scans = getItems(dc, Scan.getQuery());
     
    633665            )
    634666          );
     667        List<ArrayDesign> designs = getItems(dc, ArrayDesign.getQuery(),
     668          Restrictions.gt(
     669              Hql.property("numDbFeatures"),
     670              Expressions.integer(0)
     671            )
     672          );
    635673       
    636674        boolean hasAssociations =
    637675          context.getItem() == Item.EXPERIMENT || scans.size() > 0 ||
    638           protocols.size() > 0 || software.size() > 0;
     676          protocols.size() > 0 || software.size() > 0 || designs.size() > 0;
    639677       
    640678        if (hasAssociations)
     
    650688                new ItemParameterType<Experiment>(Experiment.class, null, true, 1, null)
    651689              ));
     690          }
     691          if (!designs.isEmpty())
     692          {
     693            parameters.add(new PluginParameter<ArrayDesign>(
     694              "arrayDesign",
     695              "Array design",
     696              "The imported raw bioassays will be linked to the selected array design.",
     697              new ItemParameterType<ArrayDesign>(ArrayDesign.class, null, false, 1, designs)
     698            ));
     699            parameters.add(featureIdentificationParameter);
    652700          }
    653701          if (!scans.isEmpty())
  • trunk/src/plugins/core/net/sf/basedb/plugins/RawDataFlatFileImporter.java

    r4080 r4083  
    2626
    2727import net.sf.basedb.core.AnnotationType;
     28import net.sf.basedb.core.ArrayDesign;
    2829import net.sf.basedb.core.BaseException;
    2930import net.sf.basedb.core.DataFileType;
    3031import net.sf.basedb.core.DbControl;
     32import net.sf.basedb.core.FeatureIdentificationMethod;
    3133import net.sf.basedb.core.File;
    3234import net.sf.basedb.core.FileParameterType;
     
    208210      "Mapping that picks the spot's feature ID from the data columns. " +
    209211      "This column is only used when the raw data is connected to an array design " +
    210       "which uses the EXTERNAL_ID method for identifying features. The value is not saved " +
     212      "which uses the FEATURE_ID method for identifying features. The value is not saved " +
    211213      "to the database." +
    212214      "For example: \\Feature ID\\",
     
    530532    }
    531533   
    532     batcher = rawBioAssay.getRawDataBatcher();
     534    FeatureIdentificationMethod fiMethod = null;
     535    if (job.getValue("featureIdentification") != null)
     536    {
     537      try
     538      {
     539        fiMethod = FeatureIdentificationMethod.valueOf((String)job.getValue("featureIdentification"));
     540      }
     541      catch (Exception ex)
     542      {}
     543    }
     544   
     545    batcher = rawBioAssay.getRawDataBatcher(fiMethod);
    533546    numInserted = 0;
    534547    this.ffp = ffp;
     
    678691    {
    679692      allColumnMappings = new ArrayList<PluginParameter<String>>();
     693      allColumnMappings.add(reporterIdColumnMapping);
     694      allColumnMappings.add(featureIdColumnMapping);
    680695      allColumnMappings.add(positionColumnMapping);
    681696      allColumnMappings.add(blockColumnMapping);
     
    686701      allColumnMappings.add(xColumnMapping);
    687702      allColumnMappings.add(yColumnMapping);
    688       allColumnMappings.add(reporterIdColumnMapping);
    689       allColumnMappings.add(featureIdColumnMapping);
    690703      for(RawDataProperty rdp : rdt.getProperties())
    691704      {
     
    726739      dc = sc.newDbControl();
    727740      boolean configIsAnnotated = false;
     741      ArrayDesign design = null;
    728742      try
    729743      {
     
    731745        rawDataFiles = FileStoreUtil.getGenericDataFiles(dc, rba, FileType.RAW_DATA);
    732746        configIsAnnotated = getCurrentConfiguration(dc).isAnnotated();
     747        design = rba.getArrayDesign();
    733748      }
    734749      catch (Throwable t)
     
    748763          null : rawDataFiles.get(0), true, 1)
    749764      );
    750       parameters.add(fileParameter);     
     765      parameters.add(fileParameter);
     766     
     767      if (design != null && design.getNumDbFeatures() > 0)
     768      {
     769        // Parameter for asking which feature identification method to use
     770        // Only used if the connected array design has features
     771       
     772        FeatureIdentificationMethod fiMethod = design.getFeatureIdentificationMethod();
     773        List<String> methods = new ArrayList<String>();
     774        String description = "Choose which method to use for identifying features: \n";
     775        if (configuration.getValue("rowColumnMapping") != null && configuration.getValue("columnColumnMapping") != null)
     776        {
     777          methods.add("COORDINATES");
     778          description += "COORDINATES: Use block, meta-grid, row and column coordinates\n";
     779        }
     780        if (configuration.getValue("positionColumnMapping") != null)
     781        {
     782          methods.add("POSITION");
     783          description += "POSITION: Use a position number only\n";
     784        }
     785        if (configuration.getValue("featureIdColumnMapping") != null)
     786        {
     787          methods.add("FEATURE_ID");
     788          description += "FEATURE_ID: Use the feature ID value (string)\n";
     789        }
     790       
     791        if (methods.size() > 0)
     792        {
     793          PluginParameter<String> featureIdentificationParameter =
     794            new PluginParameter<String>(
     795              "featureIdentification",
     796              "Identify features by",
     797              description,
     798              new StringParameterType(255, methods.contains(fiMethod.name()) ? fiMethod.name() : null,
     799                true, 1, 0, 0, methods)
     800            );
     801          parameters.add(featureIdentificationParameter);
     802        }
     803      }
     804     
    751805      parameters.add(getCharsetParameter(null, null, (String)configuration.getValue(CHARSET)));
    752806      parameters.add(getDecimalSeparatorParameter(null, null, (String)configuration.getValue(DECIMAL_SEPARATOR)));
  • trunk/src/plugins/core/net/sf/basedb/plugins/ReporterMapFlatFileImporter.java

    r4080 r4083  
    118118      "COORDINATES: Use block, meta-grid, row and column coordinates\n" +
    119119      "POSITION: Use a position number only\n" +
    120       "EXTERNAL_ID: Use the feature ID value (string)\n" +
     120      "FEATURE_ID: Use the feature ID value (string)\n" +
    121121      "In all cases, the identifier value must be unique.",
    122122      new StringParameterType(255, "COORDINATES", true, 1, 0, 0,
    123         Arrays.asList(new String[] {"COORDINATES", "POSITION", "EXTERNAL_ID"}))
     123        Arrays.asList(new String[] {"COORDINATES", "POSITION", "FEATURE_ID"}))
    124124    );
    125125 
     
    136136    "Feature ID",
    137137    "Mapping that picks the feature's ID from the data columns. This column is " +
    138     "only used when the array design uses the EXTERNAL_ID method for " +
     138    "only used when the array design uses the FEATURE_ID method for " +
    139139    "identifying features. In the other cases, the value is just stored as it is." +
    140140    "For example: \\Feature ID\\",
     
    407407          if (positionMapping == null) mapErrors.add(new InvalidUseOfNullException("Position"));
    408408        }
    409         else if (fiMethod == FeatureIdentificationMethod.EXTERNAL_ID)
     409        else if (fiMethod == FeatureIdentificationMethod.FEATURE_ID)
    410410        {
    411411          String positionMapping = (String)request.getParameterValue("featureIdColumnMapping");
     
    421421          response.setError("Missing column mapping for feature identification method: "
    422422            + fiMethod, mapErrors);
     423          return;
    423424        }
    424425       
     
    571572   
    572573    FeatureData feature = batcher.newFeature(adb, reporter);
     574    feature.setExternalId(featureId);
    573575    Integer position = positionMapper.getInt(data);
    574576    Integer row = rowMapper.getInt(data);
  • trunk/src/test/TestAnalyzePluginUtil.java

    r3820 r4083  
    7272        rba.setName("TestAnalyzePluginUtil"+i);
    7373        rba.setDescription("RawBioAssay created by TestAnalyzePluginUtil. Number of spots should be "+spots);
    74         RawDataBatcher rawDataBatcher = rba.getRawDataBatcher();
     74        RawDataBatcher rawDataBatcher = rba.getRawDataBatcher(null);
    7575        for (int j = 0; parser.hasMoreData() && j < spots; j++)
    7676        {
  • trunk/src/test/TestArrayDesign.java

    r4080 r4083  
    7575    int id3 = test_create(Platform.AFFYMETRIX, true);
    7676    int id4 = test_create(Platform.AFFYMETRIX, false);
     77    int id6 = test_create(Platform.GENERIC, false);
    7778    test_load(id);
    7879    test_list(-1);
     
    8283    test_import_from_file(id2, "data/test.reportermap.import.txt",
    8384      "\"Block\"\\t\"Column\"\\t\"Row\"\\t\"ID\"\\t\"Name\".*", 0, 2, 1, 3);
    84     test_add_features(id, 10);
     85    test_add_features(id, FeatureIdentificationMethod.COORDINATES, 10);
     86    test_add_features(id6, FeatureIdentificationMethod.FEATURE_ID, 10);
    8587
    8688    write_feature_header();
     
    115117    test_delete(id4);
    116118    test_delete(id5);
     119    test_delete(id6);
    117120   
    118121    TestPlate.test_delete_all();
     
    310313  }
    311314
    312   static void test_add_features(int arrayDesignId, int numFeatures)
     315  static void test_add_features(int arrayDesignId, FeatureIdentificationMethod fiMethod, int numFeatures)
    313316  {
    314317    if (arrayDesignId == 0) return;
     
    329332      }
    330333     
    331       FeatureBatcher batcher = ad.getFeatureBatcher(FeatureIdentificationMethod.COORDINATES);
     334      FeatureBatcher batcher = ad.getFeatureBatcher(fiMethod);
    332335      int block = 1;
    333336      int row = 1;
    334337      int column = 1;
     338      int position = 1;
    335339      ArrayDesignBlock adBlock = ad.addArrayDesignBlock(new BlockInfo(block, (int)Math.ceil(block/4F), ((block-1)%4)+1));
    336340      time = System.currentTimeMillis();
     
    345349        data.setRow(row);
    346350        data.setColumn(column);
     351        data.setPosition(position);
     352        data.setExternalId(reporter.getExternalId() + "." + position);
    347353
    348354        batcher.insert(data);
    349355        row++;
     356        position++;
    350357        if (row == 11)
    351358        {
  • trunk/src/test/TestBioAsssaySetExporter.java

    r3820 r4083  
    171171      ReporterData r2 = Reporter.getById(dc, r2Id);
    172172      ReporterData r3 = Reporter.getById(dc, r3Id);
    173       RawDataBatcher rdb = rba.getRawDataBatcher();
     173      RawDataBatcher rdb = rba.getRawDataBatcher(null);
    174174      for (int i=0; i < 4; ++i)
    175175      {
  • trunk/src/test/TestRawBioAssay.java

    r3820 r4083  
    315315      RawDataType rdt = rba.getRawDataType();
    316316      List<RawDataProperty> extraProperties = rdt.getProperties();
    317       RawDataBatcher rdb = rba.getRawDataBatcher();
     317      RawDataBatcher rdb = rba.getRawDataBatcher(null);
    318318      int block = 1;
    319319      int row = 1;
     
    459459      dc = TestUtil.getDbControl();
    460460      RawBioAssay rba = RawBioAssay.getById(dc, rawBioAssayId);
    461       RawDataBatcher rawDataBatcher = rba.getRawDataBatcher();
     461      RawDataBatcher rawDataBatcher = rba.getRawDataBatcher(null);
    462462      ReporterBatcher reporterBatcher = ReporterBatcher.getNew(dc);
    463463   
  • trunk/src/test/net/sf/basedb/test/merge/MergeTest.java

    r4080 r4083  
    238238    TestUtil.write("--Creating data for raw bioassay: " + rba.getName() + "\n");
    239239    dc.reattachItem(rba);
    240     RawDataBatcher batcher = rba.getRawDataBatcher();
     240    RawDataBatcher batcher = rba.getRawDataBatcher(null);
    241241    for (int block = 1; block <= 10; block++)
    242242    {
  • trunk/www/lims/arraydesigns/features/index.jsp

    r3775 r4083  
    4646<%@ taglib prefix="tbl" uri="/WEB-INF/table.tld" %>
    4747<%!
    48   private static final ItemContext defaultContext = Base.createDefaultContext("position", "position,block,row,column,reporter.name,reporter.externalId");
     48  private static final ItemContext defaultContext = Base.createDefaultContext("position",
     49    "position,externalId,block,row,column,reporter.name,reporter.externalId");
    4950  private static final Item itemType = Item.FEATURE;
    5051%>
  • trunk/www/lims/arraydesigns/features/list_features.jsp

    r3679 r4083  
    3636  import="net.sf.basedb.core.ArrayDesignBlock"
    3737  import="net.sf.basedb.core.Feature"
     38  import="net.sf.basedb.core.FeatureIdentificationMethod"
    3839  import="net.sf.basedb.core.ReporterType"
    3940  import="net.sf.basedb.core.Reporter"
     
    8889  final ArrayDesign design = ArrayDesign.getById(dc, arrayDesignId);
    8990  final boolean isAffy = design.isAffyChip();
     91  final FeatureIdentificationMethod fiMethod = design.getFeatureIdentificationMethod();
    9092  Formatter<Date> dateFormatter = FormatterFactory.getDateTimeFormatter(sc);
    9193 
     
    112114  int numListed = 0;
    113115  %>
    114   <base:page title="<%=title%>">
     116<base:page title="<%=title%>">
    115117  <base:head scripts="table.js,tabcontrol.js" styles="table.css,headertabcontrol.css,path.css">
    116118    <script language="JavaScript">
     
    210212        filterable="true"
    211213        exportable="true"
    212         show="always"
     214        show="<%=fiMethod == FeatureIdentificationMethod.EXTERNAL_ID ? "auto" : "always"%>"
     215      />
     216      <tbl:columndef
     217        id="externalId"
     218        property="externalId"
     219        datatype="string"
     220        title="Feature ID"
     221        sortable="true"
     222        filterable="true"
     223        exportable="true"
     224        show="<%=fiMethod == FeatureIdentificationMethod.EXTERNAL_ID ? "always" : "auto"%>"
    213225      />
    214226      <tbl:columndef
     
    540552                <tbl:cell column="position"><div class="link" onclick="itemOnClick(event, <%=itemId%>)"
    541553                  title="<%=tooltip%>"><%=item.getPosition()%></div></tbl:cell>
     554                <tbl:cell column="externalId"><div class="link" onclick="itemOnClick(event, <%=itemId%>)"
     555                  title="<%=tooltip%>"><%=HTML.encodeTags(item.getExternalId())%></div></tbl:cell>
    542556                <tbl:cell column="block"><%=block.getBlockNumber()%></tbl:cell>
    543557                <tbl:cell column="row"><%=item.getRow()%></tbl:cell>
  • trunk/www/lims/arraydesigns/features/view_feature.jsp

    r3679 r4083  
    7979
    8080  final ArrayDesign design = ArrayDesign.getById(dc, arrayDesignId);
     81 
    8182  final boolean isAffy = design.isAffyChip();
    8283  FeatureData feature = design.getFeatureById(itemId);
     
    107108      </tr>
    108109      <tr>
    109         <td class="prompt">Affy chip</td>
    110         <td><%=isAffy ? " yes" : "no"%></td>
     110        <td class="prompt">Platform / Variant</td>
     111        <td><base:propertyvalue item="<%=design%>" property="platform" /> / <base:propertyvalue item="<%=design%>" property="variant" /></td>
     112      </tr>
     113      <tr>
     114        <td class="prompt">Feature identification method</td>
     115        <td><%=design.getFeatureIdentificationMethod()%></td>
    111116      </tr>
    112117      </table>
     
    114119      <h4>Feature / block</h4>
    115120      <table class="form" cellspacing=0>
     121      <tr>
     122        <td class="prompt">Feature ID</td>
     123        <td colspan="7"><%=HTML.encodeTags(feature.getExternalId())%></td>
     124      </tr>
    116125      <tr>
    117126        <td class="prompt">Position</td>
  • trunk/www/lims/arraydesigns/list_designs.jsp

    r4003 r4083  
    4848  import="net.sf.basedb.core.PermissionDeniedException"
    4949  import="net.sf.basedb.core.PluginDefinition"
     50  import="net.sf.basedb.core.FeatureIdentificationMethod"
    5051  import="net.sf.basedb.core.query.Hql"
    5152  import="net.sf.basedb.core.query.Restrictions"
     
    7273  private static final Item itemType = Item.ARRAYDESIGN;
    7374  private static final GuiContext guiContext = new GuiContext(itemType, GuiContext.Type.LIST);
     75  private static final Enumeration<String, String> fiMethods = new Enumeration<String, String>();
     76  static
     77  {
     78    for (FeatureIdentificationMethod fim : FeatureIdentificationMethod.values())
     79    {
     80      fiMethods.add(Integer.toString(fim.getValue()), fim.toString());
     81    }
     82    fiMethods.lock();
     83  }
    7484%>
    7585<%
     
    305315        filterable="true"
    306316        exportable="true"
     317      /> 
     318      <tbl:columndef
     319        id="featureIdentificationMethod"
     320        property="featureIdentificationMethod"
     321        datatype="int"
     322        title="Feature ID meth."
     323        tooltip="Feature identification method"
     324        sortable="true"
     325        filterable="true"
     326        exportable="true"
     327        enumeration="<%=fiMethods%>"
    307328      /> 
    308329      <tbl:columndef
     
    547568                <tbl:cell column="numDbFeatures"><%=item.getNumDbFeatures()%></tbl:cell>
    548569                <tbl:cell column="numFileFeatures"><%=item.getNumFileFeatures()%></tbl:cell>
     570                <tbl:cell column="featureIdentificationMethod"><%=item.getFeatureIdentificationMethod()%></tbl:cell>
    549571                <tbl:cell column="batches">
    550572                  <%
  • trunk/www/lims/arraydesigns/view_design.jsp

    r4003 r4083  
    303303        <td class="prompt">Features</td>
    304304        <td><%=design.hasFeatures() ? "yes (db: " + design.getNumDbFeatures() + "; file: " + design.getNumFileFeatures() + ")" : "no"%></td>
     305      </tr>
     306      <tr>
     307        <td class="prompt">Feature identification method</td>
     308        <td><%=design.getFeatureIdentificationMethod()%></td>
    305309      </tr>
    306310      <tr>
  • trunk/www/views/rawbioassays/edit_rawbioassay.jsp

    r4029 r4083  
    509509    {
    510510      parentsChanged = true;
     511      <%
     512      if (rawBioAssay != null && rawBioAssay.getNumDbSpots() > 0)
     513      {
     514        %>
     515        var frm = document.forms['rawbioassay'];
     516        var selectedId = frm.arraydesign_id[frm.arraydesign_id.selectedIndex].value;
     517        var showFiMethod = (selectedId > 0 && selectedId != <%=currentArrayDesign == null ? 0 : currentArrayDesign.getId()%>);
     518        if (showFiMethod)
     519        {
     520          Main.show('fiMethod');
     521        }
     522        else
     523        {
     524          Main.hide('fiMethod');
     525        }
     526        <%
     527      }
     528      %>
    511529    }
    512530    function selectArrayDesignOnClick()
     
    535553      list.selectedIndex = 1;
    536554      parentsChanged = true;
     555      arrayDesignOnChange();
    537556    }
    538557   
     
    653672        </td>
    654673      </tr>
    655 
     674      <tr id="fiMethod" style="display: none;">
     675        <td></td>
     676        <td>
     677        <select name="fiMethod">
     678        <option value="">-auto- (array design decides)
     679        <option value="COORDINATES">Coordinates
     680        <option value="POSITION">Position
     681        <option value="FEATURE_ID">Feature ID (*)
     682        </select>
     683        <br>
     684        Select a method for feature identification. <br>
     685        (*) The Feature ID method only works if the raw data is currently
     686        connected to an array design which has Feature ID values.
     687        </td>
     688      </tr>
    656689      <tr>
    657690        <td class="prompt">Protocol</td>
  • trunk/www/views/rawbioassays/index.jsp

    r3909 r4083  
     1<%@page import="net.sf.basedb.core.FeatureIdentificationMethod"%>
    12<%-- $Id$
    23  ------------------------------------------------------------------
     
    9091    private final int arrayDesignId;
    9192    private final SessionControl sc;
    92    
    93     private UpdateArrayDesign(Job job, RawBioAssay rba, ArrayDesign design)
     93    private final FeatureIdentificationMethod fiMethod;
     94   
     95    private UpdateArrayDesign(Job job, RawBioAssay rba, ArrayDesign design, FeatureIdentificationMethod fiMethod)
    9496    {
    9597      this.jobId = job.getId();
     
    9799      this.arrayDesignId = design.getId();
    98100      this.sc = job.getSessionControl();
     101      this.fiMethod = fiMethod;
    99102    }
    100103   
     
    113116        try
    114117        {
    115           rba.updateArrayDesign(design, progress);
     118          rba.updateArrayDesign(design, fiMethod, progress);
    116119          progress.display(99, "Committing changes to database...");
    117120          dc.commit();
     
    124127        catch (Throwable t)
    125128        {
     129          if (dc != null) dc.close();
    126130          dc = sc.newDbControl();
    127131          j = Job.getById(dc, jobId);
     
    284288    Job job = null;
    285289    ArrayDesign ad = null;
     290    FeatureIdentificationMethod fiMethod = null;
    286291    int arrayDesignId = Values.getInt(request.getParameter("arraydesign_id"), -1);
    287292    if (arrayDesignId >= 0) // < 0 = denied or unchanged
     
    290295      if (rba.hasData())
    291296      {
     297        try
     298        {
     299          fiMethod = FeatureIdentificationMethod.valueOf(request.getParameter("fiMethod"));
     300        }
     301        catch (Exception ex)
     302        {}
    292303        if (rba.getRawDataType().isStoredInDb() && ad != null && ad.hasFeatures())
    293304        {
     
    301312        else
    302313        {
    303           rba.updateArrayDesign(ad, null);
     314          rba.updateArrayDesign(ad, fiMethod, null);
    304315        }
    305316      }
     
    323334    {
    324335      redirect = "../jobs/index.jsp?ID="+ID+"&cmd=ViewItem&item_id="+job.getId();
    325       Thread updateThread = new Thread(new UpdateArrayDesign(job, rba, ad));
     336      Thread updateThread = new Thread(new UpdateArrayDesign(job, rba, ad, fiMethod));
    326337      updateThread.setPriority(Thread.currentThread().getPriority() - 1);
    327338      updateThread.start();
  • trunk/www/views/rawbioassays/rawdata/list_rawdata.jsp

    r3679 r4083  
    9595
    9696  final DataQuery<RawData> query = rawBioAssay.getRawData();
     97  query.join(Hql.leftJoin(null, "reporter", "r", null, true));
     98  query.join(Hql.leftJoin(null, "feature", "f", null, true));
     99  query.join(Hql.leftJoin("f", "arrayDesignBlock", "b", null, true));
    97100  cc.configureQuery(query, true);
    98101  if (!"row".equals(cc.getSortProperty())) query.order(Orders.asc(Hql.property("row")));
     
    397400        datatype="int"
    398401        title="[Feat] Position"
     402        sortable="true"
     403        filterable="true"
     404        exportable="true"
     405      />
     406      <tbl:columndef
     407        id="feature.externalId"
     408        property="feature.externalId"
     409        datatype="string"
     410        title="[Feat] Feature ID"
    399411        sortable="true"
    400412        filterable="true"
     
    704716                  %>
    705717                  <tbl:cell column="feature.position"><i>- none -</i></tbl:cell>
     718                  <tbl:cell column="feature.externalId"><i>- none -</i></tbl:cell>
    706719                  <tbl:cell column="feature.row"><i>- none -</i></tbl:cell>
    707720                  <tbl:cell column="feature.column"><i>- none -</i></tbl:cell>
     
    721734                  %>
    722735                  <tbl:cell column="feature.position"><%=feature.getPosition()%></tbl:cell>
     736                  <tbl:cell column="feature.externalId"><%=HTML.encodeTags(feature.getExternalId())%></tbl:cell>
    723737                  <tbl:cell column="feature.row"><%=feature.getRow()%></tbl:cell>
    724738                  <tbl:cell column="feature.column"><%=feature.getColumn()%></tbl:cell>
Note: See TracChangeset for help on using the changeset viewer.