| 1 | == Requirements == |
|---|
| 2 | |
|---|
| 3 | 1. BASE 3.0 or later. |
|---|
| 4 | |
|---|
| 5 | == Introduction == |
|---|
| 6 | |
|---|
| 7 | This package is an extension package to BASE that adds support |
|---|
| 8 | for external files located on a Hadoop Distributed File System |
|---|
| 9 | (HDFS). |
|---|
| 10 | |
|---|
| 11 | * The homepage for this extension: |
|---|
| 12 | http://baseplugins.thep.lu.se/wiki/net.sf.basedb.hdfs |
|---|
| 13 | |
|---|
| 14 | == Installation == |
|---|
| 15 | |
|---|
| 16 | 1. Download the hdfs-*.tar.gz file. |
|---|
| 17 | 2. Unpack the downloaded file to a directory of your choice. |
|---|
| 18 | 3. Copy the 'hdfs.jar' file and the 'lib' directory to your BASE plug-ins |
|---|
| 19 | directory. Look in your 'base.config' file if you don't know where this is. |
|---|
| 20 | 4. Run the 'Extensions->Manual scan' command and install the new extension. |
|---|
| 21 | 5. Done. |
|---|
| 22 | |
|---|
| 23 | |
|---|
| 24 | == Using == |
|---|
| 25 | |
|---|
| 26 | Create an external file in BASE using an URL on the form: |
|---|
| 27 | |
|---|
| 28 | hdfs://namenode.server:port/path/on/hdfs/filesystem |
|---|
| 29 | |
|---|
| 30 | where 'namenode.server' is the ip-address or name of the |
|---|
| 31 | Namenode for the HDFS cluster. The port is optional. If not |
|---|
| 32 | specified 9000 is used as the default port. |
|---|
| 33 | |
|---|
| 34 | It is also possible to create a "File server" item in BASE |
|---|
| 35 | to force the use of HDFS. Since HDFS does not yet support |
|---|
| 36 | autentication the rest of the options for the file server |
|---|
| 37 | item are not used. |
|---|
| 38 | |
|---|
| 39 | |
|---|
| 40 | == Compiling == |
|---|
| 41 | |
|---|
| 42 | To compile this package you also need: |
|---|
| 43 | |
|---|
| 44 | 1. Ant 1.6 |
|---|
| 45 | 2. Java 1.6 |
|---|
| 46 | |
|---|
| 47 | Follow these instructions: |
|---|
| 48 | |
|---|
| 49 | 1. Download the source code from the subversion repository. See |
|---|
| 50 | http://baseplugins.thep.lu.se/wiki/net.sf.basedb.hdfs for instructions. |
|---|
| 51 | |
|---|
| 52 | 2. Type `ant download-lib` to automatically download the BASE core JAR files that |
|---|
| 53 | are neeed for compilation. You can also do this manually by copying the |
|---|
| 54 | base-core-3.x.jar file from the BASE installation directory |
|---|
| 55 | (<basedir>/www/WEB-INF/lib/) to the ./lib/compile directory. |
|---|
| 56 | |
|---|
| 57 | 3. Type 'ant' to compile the code and generate the 'hdfs.jar' file |
|---|
| 58 | in the project directory. |
|---|
| 59 | |
|---|
| 60 | 4. Use 'ant package' to create a downloadable tar.gz package. |
|---|
| 61 | |
|---|
| 62 | Tip: If you need different values for any of the properties defined in |
|---|
| 63 | the 'build.xml' file, create a file named 'build.properties' and set |
|---|
| 64 | the values there. |
|---|
| 65 | |
|---|
