Restructure directory layout

[Gregory Vincic]

As the project is growing our directory layout feels cluttered. Restructuring the layout(see attached file) should simplify our test, build and distribution processes. See the suggested initial structure.

[Gregory Vincic]

Each build file should contain at least the following targets

• clean

Remove everything that targets compile and/or dist generate

• compile

• dist

Generate a package e.g. .jar or .war files

• doc

Generate javadoc

The global build.xml file should have the above targets linking to each corresponding target building a complete package. There should not be any cross references between the sub build.xml files e.g. servlet/build.xml should not depend on core/build.xml. It should however depend on the package that core/build.xml -> dist target generates.

[Gregory Vincic]

Directory layout

[Fredrik Levander]

How to compile the io class trunk/org/proteios/io/SpectrumFileImpl.java which depends on plugins. Should io have its own build.xml? Preferably the mzData functions should be moved to io and these get into the core jar. Then the plugin classes can depend on that, but the io classes don't need the plugins. Are there more dependencies like this that need to be restructured?

[Gregory Vincic]

Yes, I would put the io package inside of core and let your plugin depend on core.jar. I added to the structure a /plugin directory with it's own build file. See the updated layout file.

[Gregory Vincic]

To each build.xml file we should also add the target

• test

Runs all tests

Replying to gregory:

Each build file should contain at least the following targets

• clean

Remove everything that targets compile and/or dist generate

• compile

• dist

Generate a package e.g. .jar or .war files

• doc

Generate javadoc

The global build.xml file should have the above targets linking to each corresponding target building a complete package. There should not be any cross references between the sub build.xml files e.g. servlet/build.xml should not depend on core/build.xml. It should however depend on the package that core/build.xml -> dist target generates.

[Gregory Vincic]

(In [1735]) Refs #219. Initial revision of the new directory structure. Nothing has been moved!

[Gregory Vincic]

(In [1736]) Refs #219. Added initial build file for api/waf. Still nothing has been moved!

[Gregory Vincic]

(In [1737]) Refs #219. Implemented test target in api/waf/build.xml

[Gregory Vincic]

(In [1738]) Refs #219. Initial revision of api/core/build.xml. Still nothing is moved!

[Gregory Vincic]

(In [1744]) Refs #219. Major restructure is done. There are still files that need to be moved, I'll do this shortly. Then compiling an installable proteios will work much like before though the ant command has changed. A complete proteios is build and copied to the /dist directory when running

ant clean compile hibernate dist

Each target has been made independent because dependencies would make it impossible to combine ant targets with the fact that eclipse compiles code on the fly. If you configure eclipse to compile files as the build.xml files specify, then you only have to run ant dist in the root directory to get a working package.

Please be patient as I move the remaining files into their right places.

[Gregory Vincic]

(In [1745]) Refs #219. Adding servlet/www

[Gregory Vincic]

(In [1746]) Refs #219. Moving api/core related files

[Gregory Vincic]

(In [1747]) Refs #219. Moving waf related files

[Gregory Vincic]

(In [1748]) Refs #219. Adding libraries to api/external

[Gregory Vincic]

(In [1749]) Refs #219. Adding additional libraries

[Gregory Vincic]

(In [1750]) Refs #219. Removing servlet/www/WEB-INF

[Gregory Vincic]

(In [1751]) Refs #219. Removing old /lib dir. These libs have been added to api/external

[Gregory Vincic]

(In [1753]) Refs #219. Removing most from /src directory. First commit failed due to new plugin changes

[Gregory Vincic]

(In [1754]) Refs #219. Removing old www directory

[Gregory Vincic]

(In [1755]) Refs #219. Removing libraries in test directory

[Gregory Vincic]

(In [1756]) Refs #219. Adding last changes to core and client

[Gregory Vincic]

(In [1757]) Refs #219.

[Gregory Vincic]

(In [1758]) Refs #219. Removing start scripts for ftp server. They have been moved

[Gregory Vincic]

(In [1759]) Refs #219. Adding doclet libs

[Gregory Vincic]

(In [1760]) Refs #219. Moved ftp.properties.in to client/ftpd/conf

[Gregory Vincic]

(In [1761]) Refs #219. Moved log4j and proteios config files to client/servlet/conf

[Gregory Vincic]

(In [1762]) Refs #219. Removed old source directories. Leaving only configuration files. These will be moved shortly.

[Gregory Vincic]

(In [1763]) Refs #219. Removed dependency between waf compilation and core api.

[Gregory Vincic]

(In [1764]) Refs #219. Moving org.proteios.plugins into plugin/src directory.

[Gregory Vincic]

(In [1767]) Refs #219. Removed src/ from root directory.

[Gregory Vincic]

(In [1769]) Refs #219. Removed old test/lib directory.

[Gregory Vincic]

(In [1770]) Refs #219. ant package now does the 'ant clean compile hibernate dist' walz. Also fixed so that 'ant release' works as before.

[Gregory Vincic]

(In [1771]) Refs #219. Moved javadoc generation from root build.xml into core/build.xml.

[Gregory Vincic]

(In [1772]) Refs #219. Created a separate build.xml for core/test directory. This should be done for the other test directories also.

[Gregory Vincic]

(In [1773]) Refs #219. Added junit-4.3.1.jar to api/external/test. Moved wav test cases to api/waf/test. Current version of ant does not support junit4 so make sure to extend TestCase? in each Test.

[Gregory Vincic]

(In [1774]) Refs #219. Moved org.proteios.gui tests into client/servlet/test

[Gregory Vincic]

(In [1776]) Refs #219. Moved action related tests into client/servlet/test.

[Gregory Vincic]

(In [1784]) Refs #219. Fixed dupblicate copy of config files from servlet/conf into root dist.

[Gregory Vincic]

(In [1785]) Refs #219. Added batch running of tests in core/test/build.xml

[Gregory Vincic]

(In [1786]) Refs #219. Added dependency between client/servlet/test building and api/core/test. Because the PropertiesFile? is used in most of our tests it's part of the core/test suite.

[Gregory Vincic]

(In [1787]) Refs #219. Changed classpath in core/test/build.xml

[Gregory Vincic]

Tests still haven't been moved.

[Gregory Vincic]

Each test directory should have it's own build.xml file.

[Gregory Vincic]

(In [1793]) Refs #219. Moving selenium tests to client/servlet/test/src

[Gregory Vincic]

(In [1794]) Refs #219. Removing empty test packages.

[olle]

Ticket accepted.

[olle]

(In [1808]) Refs #219. Adding new directories to api/core/test/src/ and plugin/test/src/ in preparation for moving remaining test classes.

[olle]

(In [1809]) Refs #219. Files and directories under test/src/org/proteios/core/ and test/src/org/proteios/plugins/ removed, as the contents have been moved to new locations.

[olle]

(In [1810]) Refs #219. Class/file PropertiesFile?.java in client/servlet/test/ renamed PropertiesFile2.java, as a file named PropertiesFile?.java in the same package exists in api/core/test/. The alternative solution of using the same file was rejected, in order to decrease dependencies between client/servlet/test/ and api/core/test/. Class/file action/Login.java in client/servlet/test/ updated to use PropertiesFile2 instead of PropertiesFile?.

[olle]

(In [1811]) Refs #219. Update of plugin/test/:

1. Added ant build file build.xml.

2. Class/file plugins/TestRobotFileImportTest.java in

api/core/test/ moved to plugin/test/.

3. Added copy of PropertiesFile?.java from api/core/test/ to

plugin/test/ with new name PropertiesFile3.java. The alternative solution of using the same file was rejected, in order to decrease dependencies between plugin/test/ and api/core/test/. Class/file plugins/TestRobotFileImportTest.java in plugin/test updated to use PropertiesFile3 instead of PropertiesFile?.

[olle]

(In [1812]) Refs #219. Added copy of plugins/mzdata/ApplicationTestSetup.java from api/core/test/ to plugins/ in plugin/test/. The new class was changed from package org.proteios.plugins.mzdata to org.proteios.plugins. The alternative solution of using the same file was rejected, in order to decrease dependencies between plugin/test/ and api/core/test/. Class/file plugins/TestRobotFileImportTest.java in plugin/test updated to use the new ApplicationTestSetup?.

[olle]

(In [1825]) Refs #219. Added class/file PropertiesFile?.java to api/core/test, equal in functionality to PropertiesFile2.java in client/servlet/test/ and PropertiesFile3.java in plugin/test/.

[olle]

(In [1826]) Refs #219. Test properties template file src/test.properties.in added to api/core/test/ and /plugin/test/. The template file now contains a property test.data.url with the url to the directory with test data.

[olle]

(In [1827]) Refs #219. Ant build file build.xml in api/core/test/ updated:

1. Directory api/core/conf/ included in class path in order to allow

access to configuration file proteios.config.

2. Properties file src/test.properties copied to build directory

before running tests.

[olle]

(In [1828]) Refs #219. Ant build file build.xml in plugin/test/ updated:

1. Directory api/core/conf/ included in class path in order to allow

access to configuration file proteios.config.

2. Properties file src/test.properties copied to build directory

before running tests.

[olle]

(In [1829]) Refs #219. Class/file plugins/TestRobotFileImportTest.java in plugin/test/ updated to obtain test data from directory given by url set in property test.data.url. This url is used for testing methods taking an input stream as argument. If the url points to a directory on the local system, this directory is also used to construct file paths for testing methods taking a file path as argument.

[olle]

(In [1834]) Refs #219. Ant build file build.xml in plugin/test/ updated in class path:

1. Directory "../conf" added, in order for XML Schema Definition file

plugin/conf/spot.xsd to be accessible.

2. Directory "../../api/core/test/build" excluded, as plugin/test now

uses its own PropertiesFile? class, PropertiesFile3.java in plugin/test/src/org/proteios/. Keeping "../../api/core/test/build" early in the class was problematic, as a property file test.properties in this directory may override plugin/test/build/test.properties.

[olle]

(In [1835]) Refs #219. Test properties template files test.properties.in in api/core/test/ and plugin/test/ updated to only recommend use of "​file://" + absolute path for url to resource on local system. Added test properties template file test.properties.in to client/servlet/test/, identical to the ones for api/core/test/ and plugin/test/.

[olle]

Correction of subversion commit message: In the last sentence in the text for changeset ​[1834], "class" should be exchanged for "class path". The sentence should read: Keeping "../../api/core/test/build" early in the class path was problematic, as a property file test.properties in this directory may override plugin/test/build/test.properties.

[olle]

(In [1837]) Refs #219. Core test classes/files in api/core/test/ updated by creation of ItemFactory? object in method setUp().

[olle]

(In [1845]) Refs #219. Ant build file build.xml in api/core/test/ updated for target "compile". Properties file src/test.properties is now copied to build directory after compiling the source code, in addition to before running tests via ant. This allows tests to be run without using ant target "test", e.g. for running single tests.

[olle]

(In [1846]) Refs #219. Ant build file build.xml in api/core/ updated by moving target "hibernate", so targets "clean", "compile", and "hibernate" come in order. If ant is used from an IDE where the default target order is used, this will work as a reminder to rebuild the hibernate mapping after a successful compilation.

[olle]

(In [1847]) Refs #219. Class/file plugins/TestRobotFileImportTest.java in plugin/test/ updated in method setUp() to only support use of "​file://" + absolute path for url to resource on local system.

[olle]

(In [1848]) Refs #219. Classes/files plugins/mzdata/*.java in api/core/test/ related to test of mzData import and export updated to obtain test data from directory given by url set in property test.data.url. This url is used for testing methods taking an input stream as argument. If the url points to a directory on the local system, this directory is also used to construct file paths for testing methods taking a file path as argument.

[olle]

(In [1849]) Refs #219. Class/file plugins/mzdata/TestMzDataExportPlugin.java in api/core/test/ updated in method setUp() by rearrangement of some method calls and addition of comments, in order to increase source code similarity between this and other classes related to test of mzData import and export.

[olle]

(In [1850]) Refs #219. Class/file plugins/mzdata/TestXMLValidator.java in api/core/test/ updated to obtain test data from directory given by url set in property test.data.url. The url is expected to point to a directory on the local system, as it will be used to construct file paths for testing methods taking a file path as argument.

[olle]

(In [1851]) Refs #219. Class/file plugins/mzdata/io/XMLValidator.java in api/core/ updated in public static method Schema compileSchema(String xsdFilePath) by allowing the path for the XSD file to be outside the installation directory. As in the original version, first is tested if a url can be found relative to the installation directory using XMLValidator.class.getResource(xsdFilePath). If this does not succeed, an attempt is made to obtain a uri based on the file path being an absolute path on the local system.

[olle]

(In [1852]) Refs #219. Class/file plugins/mzdata/io/XMLValidatorErrorHandler.java updated to fix a bug in public static method boolean validate_w_error_output(String xsdFilePath, StreamSource? xmlStreamSource) in class/file plugins/mzdata/io/XMLValidator.java. The metod uses an instance of XMLValidatorErrorHandler as error handler, but failed to return "false" for an invalid XML stream, as XMLValidatorErrorHandler did not propagate the SAXExceptions thrown. Now non-warning SAXExceptions are propagated, in addition to error information being printed. This change of functionality of the class was made as it is currently only used by method validate_w_error_output(String xsdFilePath, StreamSource? xmlStreamSource) in XMLValidator, and this method did not work correctly with the previous version.

[olle]

(In [1872]) Refs #219. Files under test/src/org/proteios/io/ moved to new location api/core/test/src/org/proteios/io/.

[olle]

(In [1873]) Refs #219. Changes to two files in plugin/ erroneously committed in changeSet ​[1872] removed, and the files are reverted to previous versions.

[olle]

(In [1874]) Refs #219. Classes/files io/TestPeakListFileImpl.java and io/TestRobotFileImpl.java in api/core/test/ updated to obtain test data from directory given by url set in property test.data.url. The url is expected to point to a directory on the local system, as it will be used to construct file paths for testing methods taking a file path as argument.

[olle]

(In [1880]) Refs #219. Removal of obsolete files from old test directory.

1. Ant build file test/build.xml and test properties template file

test/src/test.properties.in removed as test files have been moved to new locations in api/core/test/, api/waf/test/, client/ftpd/test/, client/manager/test/, client/servlet/test/, and plugin/test/.

2. Test data files in directory test/testData/ removed, as they have

been made available for download via anonymous ftp (current link ​ftp://bioinfo.thep.lu.se/proteios/testData/).

[olle]

(In [1881]) Refs #219. As test files have been moved from directory test/ and its subdirectories to new locations, the empty directories have been removed. The removed directories are test/src/org/proteios/io/, test/src/org/proteios/, test/src/org/, test/src/, test/testData/, and test/.

[olle]

(In [1885]) Refs #219. Directories with test programs using external test data files (configuration files like test.properties excluded) updated with a README.txt text file with information on what data files are needed for the tests.

[olle]

(In [1888]) Refs #219. Class/file io/TestRobotFileImpl.java and text file io/README.txt in api/core/test/ updated by changing name of test data file KAISA20040506a.xml to spot_handling_workstation_log.xml, in order to have the file name describe the contents of the file more clearly.

[olle]

(In [1899]) Refs #219. Class/file io/TestXMLReadUtil.java in api/core/test/ updated to obtain test data from directory given by url set in property test.data.url.

Test information text file io/README.txt in api/core/test/ updated with note on intended use of TestXMLReadUtil.java.

[Fredrik Levander]

(In [1921]) Refs #219. Moved mzData import and export helper classes to org.proteios.io.mzdata. Moved XML help classes to org.proteios.io. Finally the mzData plugins were moved to plugin.

[Fredrik Levander]

(In [1922]) Refs #219. Moved test files for mzData.

[Fredrik Levander]

(In [1929]) Refs #219. Moved xsd files to separate folder.

[olle]

Traceability note: Next ticket related to JUnit tests and restructuring is Ticket #280 (Fix problems with JUnit tests).

[olle]

Ticket reopened to remove library file(s).

[olle]

(In [2346]) Refs #219. Library file api/external/ojdbc14.jar for Oracle JDBC driver has been removed, as it is not used. It was moved to current location in changeset ​[1748].

[olle]

Ticket closed as the selected library file has been removed.

[olle]

(In [2350]) Refs #219. Refs #1. Sun Java Streaming XML Parser library file sjsxp.jar in api/external/ was corrupt, and has therefore been replaced by a new copy obtained from web site ​http://www.java2s.com/Code/Jar/wsit/Downloadsjsxpjar.htm.

1. Library file api/external/sjsxp.jar for Sun Java Streaming XML

Parser has been replaced by a new copy obtained from web site ​http://www.java2s.com/Code/Jar/wsit/Downloadsjsxpjar.htm, as the original jar file was corrupt.

2. Added license for sjsxp.jar library.

[olle]

Severity set after closing of ticket; if set when the ticket was created, the severity would probably have been set to 16, due to the many design choices that had to be made.