source: extensions/net.sf.basedb.hdfs/trunk/README @ 1358

Last change on this file since 1358 was 1358, checked in by Nicklas Nordborg, 12 years ago

References #304: Implement HDFS external file support

Re-packaging of extra JAR files since BASE now supports including them with the main JAR file (http://base.thep.lu.se/ticket/1594).

  • Property svn:mime-type set to text/x-trac-wiki
File size: 2.0 KB

Requirements

  1. BASE 3.0 or later.

Introduction

This package is an extension package to BASE that adds support for external files located on a Hadoop Distributed File System (HDFS).

Installation

  1. Download the hdfs-*.tar.gz file.
  2. Unpack the downloaded file to a directory of your choice.
  3. Copy the 'hdfs.jar' file to your BASE plug-ins directory. Look in your 'base.config' file if you don't know where this is.
  4. Go to 'Administrate->Plug-ins & Extensions->Overview' page.
  5. Run the installation wizard and select to install 'hdfs.jar'.
  6. Done.

Using

Create an external file in BASE using an URL on the form:

hdfs://namenode.server:port/path/on/hdfs/filesystem

where 'namenode.server' is the ip-address or name of the Namenode for the HDFS cluster. The port is optional. If not specified 9000 is used as the default port.

It is also possible to create a "File server" item in BASE to force the use of HDFS. Since HDFS does not yet support autentication the rest of the options for the file server item are not used.

Compiling

To compile this package you also need:

  1. Ant 1.6
  2. Java 1.6

Follow these instructions:

  1. Download the source code from the subversion repository. See http://baseplugins.thep.lu.se/wiki/net.sf.basedb.hdfs for instructions.

  1. Type ant download-lib to automatically download the BASE core JAR files that are neeed for compilation. You can also do this manually by copying the base-core-3.x.jar file from the BASE installation directory (<basedir>/www/WEB-INF/lib/) to the ./lib/compile directory.
  1. Type 'ant' to compile the code and generate the 'hdfs.jar' file in the project directory.
  1. Use 'ant package' to create a downloadable tar.gz package.

Tip: If you need different values for any of the properties defined in the 'build.xml' file, create a file named 'build.properties' and set the values there.

Note: See TracBrowser for help on using the repository browser.