Package it.unimi.di.big.mg4j.io

Class HadoopFileSystemIOFactory

java.lang.Object

it.unimi.di.big.mg4j.io.HadoopFileSystemIOFactory

All Implemented Interfaces:

IOFactory

public class HadoopFileSystemIOFactory
extends Object
implements IOFactory

An I/O factory using a Hadoop file system.

The factory provides some constructors (HadoopFileSystemIOFactory() and HadoopFileSystemIOFactory(String)) that are suitable for use with an ObjectParser. For instance, the object specification it.unimi.di.mg4j.io.HadoopFileSystemIOFactory(hdfs://127.0.0.1:9000/) could be used with Scan or IndexBuilder to make them use a local HDFS file system.

Note that if you use the constructors based on the default configuration you must set up your configuration files suitably. If you use an unconfigured Hadoop file system, you will get a Hadoop LocalFileSystem.

Field Summary

Fields inherited from interface it.unimi.di.big.mg4j.io.IOFactory

FILESYSTEM_FACTORY

Constructor Summary

Constructors

Constructor	Description
HadoopFileSystemIOFactory()	Creates a factory using a file system with a default configuration.
HadoopFileSystemIOFactory(String uri)	Creates a factory using a file system specified by a given URI with a default configuration
HadoopFileSystemIOFactory(URI uri)	Creates a factory using a file system specified by a given URI with a default configuration.
HadoopFileSystemIOFactory(FileSystem fileSystem)	Creates a factory using a given Hadoop file system.

Method Summary

Modifier and Type	Method	Description
void	createNewFile(String name)
boolean	delete(String name)
boolean	exists(String name)
FSDataInputStream	getInputStream(String name)
FSDataOutputStream	getOutputStream(String name)
ReadableByteChannel	getReadableByteChannel(String name)
WritableByteChannel	getWritableByteChannel(String name)
long	length(String name)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HadoopFileSystemIOFactory

public HadoopFileSystemIOFactory()
                          throws IOException

Creates a factory using a file system with a default configuration.

Throws:

IOException

HadoopFileSystemIOFactory

public HadoopFileSystemIOFactory(URI uri)
                          throws IOException

Creates a factory using a file system specified by a given URI with a default configuration.

Parameters:

uri - a URI that will be passed to FileSystem.get(URI, Configuration).

Throws:

IOException

HadoopFileSystemIOFactory

public HadoopFileSystemIOFactory(String uri)
                          throws IOException

Creates a factory using a file system specified by a given URI with a default configuration

This constructor is essentially identical to HadoopFileSystemIOFactory(URI), but it can be used with an ObjectParser.

Parameters:

uri - a URI, specified as a string, that will be passed to FileSystem.get(URI, Configuration).

Throws:

IOException

HadoopFileSystemIOFactory

public HadoopFileSystemIOFactory(FileSystem fileSystem)

Creates a factory using a given Hadoop file system.

Parameters:

fileSystem - a Hadoop file system.

Method Detail

getInputStream

public FSDataInputStream getInputStream(String name)
                                 throws IOException

Specified by:

getInputStream in interface IOFactory

Throws:

IOException

getOutputStream

public FSDataOutputStream getOutputStream(String name)
                                   throws IOException

Specified by:

getOutputStream in interface IOFactory

Throws:

IOException

getWritableByteChannel

public WritableByteChannel getWritableByteChannel(String name)
                                           throws IOException

Specified by:

getWritableByteChannel in interface IOFactory

Throws:

IOException

getReadableByteChannel

public ReadableByteChannel getReadableByteChannel(String name)
                                           throws IOException

Specified by:

getReadableByteChannel in interface IOFactory

Throws:

IOException

exists

public boolean exists(String name)
               throws IOException

Specified by:

exists in interface IOFactory

Throws:

IOException

delete

public boolean delete(String name)
               throws IOException

Specified by:

delete in interface IOFactory

Throws:

IOException

createNewFile

public void createNewFile(String name)
                   throws IOException

Specified by:

createNewFile in interface IOFactory

Throws:

IOException

length

public long length(String name)
            throws IOException

Specified by:

length in interface IOFactory

Throws:

IOException