The tracejob command

• Overview
• Syntax
• Example with output
• Additional commands to view job status

All user jobs are run by torque.

The tracejob command allows you to see detailed information about your job. It extracts job status and job events from accounting records, mom log files, server log files, and scheduler log files. Using it can help identify where, how, and why a job failed.

The format of the tracejob:

tracejob jobid

Detailed syntax:

tracejob [-a|s|l|m|v] [-w size] [-p path] [-n days] [-f filter_type]
   -p : path to PBS_SERVER_HOME
   -w : number of columns of your terminal
   -n : number of days in the past to look for job(s) [default 1]
   -f : filter out types of log entries, multiple -f's can be specified
        error, system, admin, job, job_usage, security, sched, debug,
        debug2, or absolute numberic equiv
   -z : toggle filtering excessive messages
   -c : what message count is considered excessive
   -a : don't use accounting log files
   -s : don't use server log files
   -l : don't use scheduler log files
   -m : don't use mom log files
   -v : verbose mode - show more error messages

Job: 2216.argo-new.cc.uic.edu

01/19/2006 16:41:49  S    enqueuing into batch, state 1 hop 1
01/19/2006 16:41:49  S    Job Queued at request of jsmithargo-new.cc.uic.edu, owner =
                          jsmith@argo-new.cc.uic.edu, job name = test, queue = batch
01/19/2006 16:41:49  A    queue=batch
01/19/2006 16:41:53  S    Job Modified at request of Scheduler@argo-new.cc.uic.edu
01/19/2006 16:41:53  S    Job Run at request of Scheduler@argo-new.cc.uic.edu
01/19/2006 16:41:54  L    Job Run
01/19/2006 16:41:54  A    user=jsmith group=student jobname=test queue=batch ctime=1137710509
                          qtime=1137710509 etime=1137710509 start=1137710514 exec_host=argo4-4/0
                          Resource_List.nodect=64 Resource_List.nodes=1
                          Resource_List.walltime=336:00:00

qstat