source: extensions/net.sf.basedb.hdfs/trunk/README @ 1332

Requirements

1. BASE 3.0 or later.

Introduction

This package is an extension package to BASE that adds support for external files located on a Hadoop Distributed File System (HDFS).

• The homepage for this extension: ​http://baseplugins.thep.lu.se/wiki/net.sf.basedb.hdfs

Installation

1. Download the hdfs-*.tar.gz file.
2. Unpack the downloaded file to a directory of your choice.
3. Copy the 'hdfs.jar' file and the 'lib' directory to your BASE plug-ins directory. Look in your 'base.config' file if you don't know where this is.
4. Run the 'Extensions->Manual scan' command and install the new extension.
5. Done.

Using

Create an external file in BASE using an URL on the form:

hdfs://namenode.server:port/path/on/hdfs/filesystem

where 'namenode.server' is the ip-address or name of the Namenode for the HDFS cluster. The port is optional. If not specified 9000 is used as the default port.

It is also possible to create a "File server" item in BASE to force the use of HDFS. Since HDFS does not yet support autentication the rest of the options for the file server item are not used.

Compiling

To compile this package you also need:

1. Ant 1.6
2. Java 1.6

Follow these instructions:

1. Download the source code from the subversion repository. See ​http://baseplugins.thep.lu.se/wiki/net.sf.basedb.hdfs for instructions.

2. Type ant download-lib to automatically download the BASE core JAR files that are neeed for compilation. You can also do this manually by copying the base-core-3.x.jar file from the BASE installation directory (<basedir>/www/WEB-INF/lib/) to the ./lib/compile directory.

3. Type 'ant' to compile the code and generate the 'hdfs.jar' file in the project directory.

4. Use 'ant package' to create a downloadable tar.gz package.

Tip: If you need different values for any of the properties defined in the 'build.xml' file, create a file named 'build.properties' and set the values there.