|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectorg.apache.hadoop.util.MRAsyncDiskService
public class MRAsyncDiskService
This class is a container of multiple thread pools, each for a volume, so that we can schedule async disk operations easily. Examples of async disk operations are deletion of files. We can move the files to a "toBeDeleted" folder before asychronously deleting it, to make sure the caller can run it faster. Users should not write files into the "toBeDeleted" folder, otherwise the files can be gone any time we restart the MRAsyncDiskService. This class also contains all operations that will be performed by the thread pools.
Field Summary | |
---|---|
static org.apache.commons.logging.Log |
LOG
|
static String |
TOBEDELETED
|
Constructor Summary | |
---|---|
MRAsyncDiskService(FileSystem localFileSystem,
String[] nonCanonicalVols)
Create a AsyncDiskServices with a set of volumes (specified by their root directories). |
|
MRAsyncDiskService(JobConf conf)
Initialize MRAsyncDiskService based on conf. |
Method Summary | |
---|---|
boolean |
awaitTermination(long milliseconds)
Wait for the termination of the thread pools. |
void |
cleanupAllVolumes()
Move all files/directories inside volume into TOBEDELETED, and then delete them. |
boolean |
moveAndDeleteAbsolutePath(String absolutePathName)
Move the path name to a temporary location and then delete it. |
boolean |
moveAndDeleteFromEachVolume(String pathName)
Move the path name on each volume to a temporary location and then delete them. |
boolean |
moveAndDeleteRelativePath(String volume,
String pathName)
Move the path name on one volume to a temporary location and then delete them. |
void |
shutdown()
Gracefully start the shut down of all ThreadPools. |
List<Runnable> |
shutdownNow()
Shut down all ThreadPools immediately. |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
public static final org.apache.commons.logging.Log LOG
public static final String TOBEDELETED
Constructor Detail |
---|
public MRAsyncDiskService(FileSystem localFileSystem, String[] nonCanonicalVols) throws IOException
localFileSystem
- The localFileSystem used for deletions.nonCanonicalVols
- The roots of the file system volumes, which may
be absolte paths, or paths relative to the ${user.dir} system property
("cwd").
IOException
public MRAsyncDiskService(JobConf conf) throws IOException
conf
- local file system and local dirs will be read from conf
IOException
Method Detail |
---|
public void shutdown()
public List<Runnable> shutdownNow()
public boolean awaitTermination(long milliseconds) throws InterruptedException
milliseconds
- The number of milliseconds to wait
InterruptedException
public boolean moveAndDeleteRelativePath(String volume, String pathName) throws IOException
volume
- The disk volumepathName
- The path name relative to volume root.
IOException
- If the move failedpublic boolean moveAndDeleteFromEachVolume(String pathName) throws IOException
pathName
- The path name relative to each volume root
IOException
- If any of the move failedpublic void cleanupAllVolumes() throws IOException
IOException
public boolean moveAndDeleteAbsolutePath(String absolutePathName) throws IOException
absolutePathName
- The path name from root "/"
IOException
- If the move failed
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |