org.apache.hadoop.io
Class SequenceFile.Sorter

java.lang.Object
  extended by org.apache.hadoop.io.SequenceFile.Sorter
Enclosing class:
SequenceFile

public static class SequenceFile.Sorter
extends Object

Sorts key/value pairs in a sequence-format file.

For best performance, applications should make sure that the Writable.readFields(DataInput) implementation of their keys is very efficient. In particular, it should avoid allocating memory.


Constructor Summary
SequenceFile.Sorter(FileSystem fs, Class keyClass, Class valClass, Configuration conf)
          Sort and merge files containing the named classes.
SequenceFile.Sorter(FileSystem fs, WritableComparator comparator, Class valClass, Configuration conf)
          Sort and merge using an arbitrary WritableComparator.
 
Method Summary
 int getFactor()
          Get the number of streams to merge at once.
 int getMemory()
          Get the total amount of buffer memory, in bytes.
 void merge(Path[] inFiles, Path outFile)
          Merge the provided files.
 void setFactor(int factor)
          Set the number of streams to merge at once.
 void setMemory(int memory)
          Set the total amount of buffer memory, in bytes.
 void sort(Path[] inFiles, Path outFile, boolean deleteInput)
          Perform a file sort from a set of input files into an output file.
 void sort(Path inFile, Path outFile)
          The backwards compatible interface to sort.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

SequenceFile.Sorter

public SequenceFile.Sorter(FileSystem fs,
                           Class keyClass,
                           Class valClass,
                           Configuration conf)
Sort and merge files containing the named classes.


SequenceFile.Sorter

public SequenceFile.Sorter(FileSystem fs,
                           WritableComparator comparator,
                           Class valClass,
                           Configuration conf)
Sort and merge using an arbitrary WritableComparator.

Method Detail

setFactor

public void setFactor(int factor)
Set the number of streams to merge at once.


getFactor

public int getFactor()
Get the number of streams to merge at once.


setMemory

public void setMemory(int memory)
Set the total amount of buffer memory, in bytes.


getMemory

public int getMemory()
Get the total amount of buffer memory, in bytes.


sort

public void sort(Path[] inFiles,
                 Path outFile,
                 boolean deleteInput)
          throws IOException
Perform a file sort from a set of input files into an output file.

Parameters:
inFiles - the files to be sorted
outFile - the sorted output file
deleteInput - should the input files be deleted as they are read?
Throws:
IOException

sort

public void sort(Path inFile,
                 Path outFile)
          throws IOException
The backwards compatible interface to sort.

Parameters:
inFile - the input file to sort
outFile - the sorted output file
Throws:
IOException

merge

public void merge(Path[] inFiles,
                  Path outFile)
           throws IOException
Merge the provided files.

Throws:
IOException


Copyright © 2006 The Apache Software Foundation