org.apache.hadoop.mapred
Class TaskInProgress

java.lang.Object
  extended by org.apache.hadoop.mapred.TaskInProgress

public class TaskInProgress
extends Object

TaskInProgress maintains all the info needed for a Task in the lifetime of its owning Job. A given Task might be speculatively executed or reexecuted, so we need a level of indirection above the running-id itself.
A given TaskInProgress contains multiple taskids, 0 or more of which might be executing at any one time. (That's what allows speculative execution.) A taskid is now *never* recycled. A TIP allocates enough taskids to account for all the speculation and failures it will ever have to handle. Once those are up, the TIP is dead. **************************************************************


Field Summary
static org.apache.commons.logging.Log LOG
           
 
Constructor Summary
TaskInProgress(JobID jobid, String jobFile, int numMaps, int partition, JobTracker jobtracker, JobConf conf, JobInProgress job, int numSlotsRequired)
          Constructor for ReduceTask
TaskInProgress(JobID jobid, String jobFile, JobSplit.TaskSplitMetaInfo split, JobTracker jobtracker, JobConf conf, JobInProgress job, int partition, int numSlotsRequired)
          Constructor for MapTask
 
Method Summary
 void addDiagnosticInfo(TaskAttemptID taskId, String diagInfo)
          Save diagnostic information for a given task.
 Task addRunningTask(TaskAttemptID taskid, String taskTracker)
           
 Task addRunningTask(TaskAttemptID taskid, String taskTracker, boolean taskCleanup)
          Adds a previously running task to this tip.
 void completed(TaskAttemptID taskid)
          Indicate that one of the taskids in this TaskInProgress has successfully completed!
 void doCommit(TaskAttemptID taskid)
          Commit this task attempt for the tip.
 TaskReport generateSingleReport()
          Creates a "status report" for this task.
 Counters getCounters()
          Get the task's counters
 List<String> getDiagnosticInfo(TaskAttemptID taskId)
          Get the diagnostic messages for a given task within this tip.
 long getExecFinishTime()
          Return the exec finish time
 long getExecStartTime()
          Return the exec start time
 int getIdWithinJob()
          Get the id of this map or reduce task.
 JobInProgress getJob()
          Return the parent job
 long getMapInputSize()
           
 int getNumberOfFailedMachines()
          Get the number of machines where this task has failed.
 double getProgress()
          Get the overall progress (from 0 to 1.0) for this TIP
 String[] getSplitLocations()
          Get the split locations
 String getSplitNodes()
          Gets the Node list of input split locations sorted in rack order.
 long getStartTime()
          Return the start time
 int getSuccessEventNumber()
          Get the event number that was raised for this tip
 TaskStatus getTaskStatus(TaskAttemptID taskid)
          Get the status of the specified task
 TaskStatus[] getTaskStatuses()
          Get the Status of the tasks managed by this TIP
 Task getTaskToRun(String taskTracker)
          Return a Task that can be sent to a TaskTracker for execution.
 TaskID getTIPId()
          Return an ID for this task, not its component taskid-threads
 boolean hasFailedOnMachine(String trackerHost)
          Has this task already failed on this machine?
 boolean hasRunOnMachine(String trackerHost, String trackerName)
          Was this task ever scheduled to run on this machine?
 int idWithinJob()
          Return the index of the tip within the job, so "task_200707121733_1313_0002_m_012345" would return 12345;
 void incompleteSubTask(TaskAttemptID taskid, JobStatus jobStatus)
          Indicate that one of the taskids in this TaskInProgress has failed.
 boolean isCommitPending(TaskAttemptID taskId)
           
 boolean isComplete()
          Is this tip complete?
 boolean isComplete(TaskAttemptID taskid)
          Is the given taskid the one that took this tip to completion?
 boolean isFailed()
          Is the tip a failure?
 boolean isFirstAttempt(TaskAttemptID taskId)
          Is the Task associated with taskid is the first attempt of the tip?
 boolean isJobCleanupTask()
           
 boolean isJobSetupTask()
           
 boolean isMapTask()
          Whether this is a map task
 boolean isOnlyCommitPending()
           
 boolean isRunning()
          Is this tip currently running any tasks?
 void kill()
          The TIP's been ordered kill()ed.
 int numKilledTasks()
          Number of times the TaskInProgress has been killed by the framework.
 int numTaskFailures()
          Number of times the TaskInProgress has failed.
 void setExecFinishTime(long finishTime)
          Set the exec finish time
 void setExecStartTime(long startTime)
          Set the exec start time
 void setJobCleanupTask()
           
 void setJobSetupTask()
           
 void setSuccessEventNumber(int eventNumber)
          Set the event number that was raised for this tip
 boolean shouldClose(TaskAttemptID taskid)
          Returns whether a component task-thread should be closed because the containing JobInProgress has completed or the task is killed by the user
 boolean shouldCommit(TaskAttemptID taskid)
          Returns whether the task attempt should be committed or not
 boolean wasKilled()
          Was the task killed?
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

LOG

public static final org.apache.commons.logging.Log LOG
Constructor Detail

TaskInProgress

public TaskInProgress(JobID jobid,
                      String jobFile,
                      JobSplit.TaskSplitMetaInfo split,
                      JobTracker jobtracker,
                      JobConf conf,
                      JobInProgress job,
                      int partition,
                      int numSlotsRequired)
Constructor for MapTask


TaskInProgress

public TaskInProgress(JobID jobid,
                      String jobFile,
                      int numMaps,
                      int partition,
                      JobTracker jobtracker,
                      JobConf conf,
                      JobInProgress job,
                      int numSlotsRequired)
Constructor for ReduceTask

Method Detail

idWithinJob

public int idWithinJob()
Return the index of the tip within the job, so "task_200707121733_1313_0002_m_012345" would return 12345;

Returns:
int the tip index

isJobCleanupTask

public boolean isJobCleanupTask()

setJobCleanupTask

public void setJobCleanupTask()

isJobSetupTask

public boolean isJobSetupTask()

setJobSetupTask

public void setJobSetupTask()

isOnlyCommitPending

public boolean isOnlyCommitPending()

isCommitPending

public boolean isCommitPending(TaskAttemptID taskId)

getStartTime

public long getStartTime()
Return the start time


getExecStartTime

public long getExecStartTime()
Return the exec start time


setExecStartTime

public void setExecStartTime(long startTime)
Set the exec start time


getExecFinishTime

public long getExecFinishTime()
Return the exec finish time


setExecFinishTime

public void setExecFinishTime(long finishTime)
Set the exec finish time


getJob

public JobInProgress getJob()
Return the parent job


getTIPId

public TaskID getTIPId()
Return an ID for this task, not its component taskid-threads


isMapTask

public boolean isMapTask()
Whether this is a map task


isFirstAttempt

public boolean isFirstAttempt(TaskAttemptID taskId)
Is the Task associated with taskid is the first attempt of the tip?

Parameters:
taskId -
Returns:
Returns true if the Task is the first attempt of the tip

isRunning

public boolean isRunning()
Is this tip currently running any tasks?

Returns:
true if any tasks are running

isComplete

public boolean isComplete()
Is this tip complete?

Returns:
true if the tip is complete, else false

isComplete

public boolean isComplete(TaskAttemptID taskid)
Is the given taskid the one that took this tip to completion?

Parameters:
taskid - taskid of attempt to check for completion
Returns:
true if taskid is complete, else false

isFailed

public boolean isFailed()
Is the tip a failure?

Returns:
true if tip has failed, else false

numTaskFailures

public int numTaskFailures()
Number of times the TaskInProgress has failed.


numKilledTasks

public int numKilledTasks()
Number of times the TaskInProgress has been killed by the framework.


getProgress

public double getProgress()
Get the overall progress (from 0 to 1.0) for this TIP


getCounters

public Counters getCounters()
Get the task's counters


shouldClose

public boolean shouldClose(TaskAttemptID taskid)
Returns whether a component task-thread should be closed because the containing JobInProgress has completed or the task is killed by the user


doCommit

public void doCommit(TaskAttemptID taskid)
Commit this task attempt for the tip.

Parameters:
taskid -

shouldCommit

public boolean shouldCommit(TaskAttemptID taskid)
Returns whether the task attempt should be committed or not


generateSingleReport

public TaskReport generateSingleReport()
Creates a "status report" for this task. Includes the task ID and overall status, plus reports for all the component task-threads that have ever been started.


getDiagnosticInfo

public List<String> getDiagnosticInfo(TaskAttemptID taskId)
Get the diagnostic messages for a given task within this tip.

Parameters:
taskId - the id of the required task
Returns:
the list of diagnostics for that task

addDiagnosticInfo

public void addDiagnosticInfo(TaskAttemptID taskId,
                              String diagInfo)
Save diagnostic information for a given task.

Parameters:
taskId - id of the task
diagInfo - diagnostic information for the task

incompleteSubTask

public void incompleteSubTask(TaskAttemptID taskid,
                              JobStatus jobStatus)
Indicate that one of the taskids in this TaskInProgress has failed.


completed

public void completed(TaskAttemptID taskid)
Indicate that one of the taskids in this TaskInProgress has successfully completed!


getSplitLocations

public String[] getSplitLocations()
Get the split locations


getTaskStatuses

public TaskStatus[] getTaskStatuses()
Get the Status of the tasks managed by this TIP


getTaskStatus

public TaskStatus getTaskStatus(TaskAttemptID taskid)
Get the status of the specified task

Parameters:
taskid -

kill

public void kill()
The TIP's been ordered kill()ed.


wasKilled

public boolean wasKilled()
Was the task killed?

Returns:
true if the task killed

getTaskToRun

public Task getTaskToRun(String taskTracker)
                  throws IOException
Return a Task that can be sent to a TaskTracker for execution.

Throws:
IOException

addRunningTask

public Task addRunningTask(TaskAttemptID taskid,
                           String taskTracker)

addRunningTask

public Task addRunningTask(TaskAttemptID taskid,
                           String taskTracker,
                           boolean taskCleanup)
Adds a previously running task to this tip. This is used in case of jobtracker restarts.


hasFailedOnMachine

public boolean hasFailedOnMachine(String trackerHost)
Has this task already failed on this machine?

Parameters:
trackerHost - The task tracker hostname
Returns:
Has it failed?

hasRunOnMachine

public boolean hasRunOnMachine(String trackerHost,
                               String trackerName)
Was this task ever scheduled to run on this machine?

Parameters:
trackerHost - The task tracker hostname
trackerName - The tracker name
Returns:
Was task scheduled on the tracker?

getNumberOfFailedMachines

public int getNumberOfFailedMachines()
Get the number of machines where this task has failed.

Returns:
the size of the failed machine set

getIdWithinJob

public int getIdWithinJob()
Get the id of this map or reduce task.

Returns:
The index of this tip in the maps/reduces lists.

setSuccessEventNumber

public void setSuccessEventNumber(int eventNumber)
Set the event number that was raised for this tip


getSuccessEventNumber

public int getSuccessEventNumber()
Get the event number that was raised for this tip


getSplitNodes

public String getSplitNodes()
Gets the Node list of input split locations sorted in rack order.


getMapInputSize

public long getMapInputSize()


Copyright © 2009 The Apache Software Foundation