Opened 15 years ago

Closed 14 years ago

Last modified 14 years ago

#411 closed enhancement (fixed)

Extending compressed file support and storage

Reported by: Jari Häkkinen Owned by: Nicklas Nordborg
Priority: major Milestone: BASE 2.5
Component: core Version:
Keywords: Cc:

Description (last modified by Nicklas Nordborg)

It would be nice if BASE core support would include storage of compressed files. Affy files CDFs and CELs shrinks with a factor 5 to 10 when compressed. BASE can store compressed files already but does not know how to handle them. Currently clients and plug-ins must know how to handle files. It is usually assumed that the files are uncompressed.

The core should have support for uncompressing files when clients request them and support client queries like isCompressedFile().

Change History (13)

comment:1 Changed 15 years ago by Jari Häkkinen

Type: defectenhancement

comment:2 Changed 15 years ago by Jari Häkkinen

Milestone: BASE 2.x+

comment:3 Changed 14 years ago by Jari Häkkinen

Milestone: BASE 2.x+BASE 2.4

comment:4 Changed 14 years ago by Jari Häkkinen

Milestone: BASE 2.4BASE 2.3

Milestone BASE 2.4 deleted

comment:5 Changed 14 years ago by Jari Häkkinen

Milestone: BASE 2.4BASE 2.5

comment:6 Changed 14 years ago by Nicklas Nordborg

Status: newassigned

comment:7 Changed 14 years ago by Nicklas Nordborg

(In [3719]) References #411: Most of the basic functionality is now in place and has been documented On major issue is that the Affymetrix Fusion SDK will not work with compressed files since it expects direct access to the java.io.File object. I will try to solve this by modifying the Affymetric Fusion SDK to work with InputStream:s instead. This would be better than to decompress CEL and CDF files everytime they are needed. In most cases we only scan the headers anyway, so it would be good if we can avoid copying the entire 40+ MB file(s) each time.

comment:8 Changed 14 years ago by Nicklas Nordborg

(In [3737]) Added modified AffxFusion??.jar that can read from InputStream?? for some file formats.

comment:9 Changed 14 years ago by Nicklas Nordborg

(In [3750]) References #411. Compiled AffxFusion?.jar with target="1.5" to make it possible to compile BASE with Java 1.5

comment:10 Changed 14 years ago by Nicklas Nordborg

(In [3751]) References #411. Migration should now automatically compress files based on MIME type.

comment:11 Changed 14 years ago by Nicklas Nordborg

Description: modified (diff)

comment:12 Changed 14 years ago by Nicklas Nordborg

Resolution: fixed
Status: assignedclosed

(In [3762]) References #554, #746 and fixes #411. File and directories diagram transfered to new MagicDraw? and docbook. Added documentation about compression support.

comment:13 Changed 14 years ago by Nicklas Nordborg

(In [3889]) References #411: Extending compressed file support and storage

AbstractFlatFileImporter? should use File.getSize() instead of File.getBytes() for progress reporting.

Note: See TracTickets for help on using tickets.