Visit the new ACCC website! (beta)
ACCC Home Page Academic Computing and Communications Center  
Accounts / Passwords Email Labs / Classrooms Telecom Network Security Software Computing and Network Services Education / Teaching Getting Help
 

Gaussian 03 on Argo

     
 
     
What is Gaussian
 

Gaussan provides electronic structure modeling. For more detailed informationsee the Gaussian web site.

 
     
To use Gaussian 03, you need:
 
  • An account on the argo system
  • Your login shell must be the C-shell
  • A .cshrc file in your home directory
  • A Default.Route file in your home directory
  • A .tsnet.config file in your home directory
  • The subg03 script
 
     
C Login Shell
 

    Your login shell must be C. By default, all newly-created argo accounts use the bash shell (/bin/bash). To find out what shell you currently use:

    echo $SHELL

    If the answer to the above command is /bin/bash, then you are using bash. If it is /bin/csh, then you're using C.

    Client infomation including the name of your default shell is not kept locally on the argo system; instead it is kept in the ldap system and argo make calls to ldap to get the information. If you want your shell changed, send email to systems@uic.edu with the Subject line Argo - change my default shell to C.

 
     
.cshrc file
 

    A .cshrc file (the leading period is required as are the lowercase letters) in your home directory is required. If it doesn't exist, create it.

       cshrc    Wrong:  the leading period is missing
      .Cshrc   Wrong:  the letter c following the required leading period is a capital letter
      .cshrc    Correct

    The file must include the following lines (type it EXACTLY as they appear below):

    setenv g03root "/usr/common"
    setenv GAUSS_SCRDIR "/tmp"
    source $g03root/g03/bsd/g03.login
    if (! ($?LD_LIBRARY_PATH)) then
    setenv LD_LIBRARY_PATH "/usr/common/g03"
    else
    setenv LD_LIBRARY_PATH $LD_LIBRARY_PATH":$g03root/g03"
    endif

    Warning: avoid using Windows/DOS utilities to cut and paste the above into your .cshrc file; they may add DOS-type carriage return/line feeds that are not the same as those in LINUX, thereby causing an error when you login.

    The permissions for the file should be 755:

      text

    To change the permissions, enter:

      cd ; chmod 755 .cshrc
 
     
Default.Route file
 

    It is recommended though not required that you have a copy of the Gaussian 03 system file in your home directory. Changes made to the copy in your home directory ($HOME/Default.Route) will take precedence over corresponding entries in the system copy ($g03root/g03/Default.Route).

    To get a copy which must be placed in your home directory:

      cd
      cp -p /usr/common/g03/Default.Route $HOME

    The system defaults are:

    -S- UIC
    -#- MaxDisk=2GB
    -M- 64MB

    The permissions for the file should be 755:

    cd ; chmod 755 Default.Route

 
     
.tsnet.config file
 

    Having a .tsnet.config in your home directory is is required. It is your copy of the Linda global system
    configuration file and it will allow you to customize the Linda/g03 environment.

    To get a copy which must be placed in your home directory:

    cp -p /usr/common/g03/.tsnet.config.model $HOME/.tsnet.config

    The permissions for the file should be 644:

      cd; chmod 644 .tsnet.config

    Common mistakes regrding the name of the file:

      cp -p /usr/common/g03/.tsnet.config.model $HOME
        Wrong: the generic config file is named .tsnet.config.model; however, your copy must be named .tsnet.config
        and not .tsnet.config.model

      cp -p /usr/common/g03/.tsnet.config.model $HOME/tsnet.config
        Wrong: the name of your copy must have a leading period ( .tsnet.config and not  tsnet.config)

      cp -p /usr/common/g03/.tsnet.config.model $HOME/.Tsnet.config
        Wrong: the characters following the leading period must all be lower case (the T should be t)

    Common mistakes regarding changing the content of the file:

      "I changed the content of the file, removing some of the nodes, and now my Gaussian jobs don't run."

      One of the parameters in the file, Tsnet.Appl.nodelist, (see arrow) identifies to Gaussian/Linda all nodes that are eligible to run your program:

        text

      However, the torque system which is responsible for assigning nodes to your job may select one of the nodes you removed. But, by removing the named node from the file, you've told Gaussian it no longer exists. Torque assigns a node that Gaussian doesn't know about. As a result, your job will not run. Unless you know the ramifications of your changes, it's best not to edit the .tsnet.config file.
 
     
Subg03 script
 

    The subg03 is used to execute your Gaussian job.

    To get a copy of the generic script:

    cp -p /usr/common/g03/subg03 $HOME

    There are no more than two lines that you should change; they are at the beginning and follow the line # Users should change the next two lines.:

      text

    Of the two lines, only the set input is should be changed for each new run.

    set input = line in subg03

      text

      The only requirement is the filename MUST have as its last three characters .in. If the name does not end with .in, use the Linux mv command to rename the file. For example, your input file is named mydata:

      mv mydata mydata.in

      The input filename, mydata.in, is placed after the equal sign in the set input line of the subg03 script:

      set input = mydata.in

      The location of your input file is defined in the third line:

      set WORKDIR = "$HOME"

 
     
Requesting multiple nodes and multiple processors per node
 

For most jobs running on argo, all you have to do to request multiple nodes and multiple processors is to include the -l nodes=X and the :ppn=Y specifications on the qsub command. However, there is a additional requirement for Gaussian jobs: you must include both the %NProcLinda= and %NProcShared= keywords in your input file:

    %NProcLinda=
    %NProcShared=
You don't have to put values after the equal signs; you just have to have the keywords, including the leading percent characters (%) and the trailing equal signs, in the file. The two keywords are directives to Linda, the component within Gaussian that does the parallelization. Linda doesn't know anything about torque and torque doesn't know anything about Linda. Torque is the manager that assigns resources to all jobs running on argo. The -l nodes=X and the :ppn=Y specifications request torque assign multiple nodes and multiple processors per node to your job. There is a command in the subg03 script that takes the assignments (nodes=X and ppn=Y) and sets the corresponding Linda options %NProcLinda= and %NProcShared= to them:
    %NProcLinda=X
    %NProcShared=Y
However, the command looks for the two keywords in your input file. If they are not there then, regardless of how many nodes and processors torque has assigned to your job, Linda will assume that you want only one node and one processor per node.
 
     
Work files - Handling and Location
 

There are several work files that are used in the course of a g03 computation:

text

The files all have the same name; it is the file extension that identifies the type. By default, these files are given a name that appends the job process ID to the characters Gau-. For example, the following all have the name Gau-13089:

    -rw-r--r-- 1 jsmith student 21037056 Feb 14 11:48 Gau-13089.rwf
    -rw-r--r-- 1 jsmith student  5107056 Feb 14 11:48 Gau-13089.chk
    -rw-r--r-- 1 jsmith student 0        Feb 14 11:47 Gau-13089.int
    -rw-r--r-- 1 jsmith student 0        Feb 14 11:47 Gau-13089.d2e
    -rw-r--r-- 1 jsmith student 524288   Feb 14 11:48 Gau-13089.scr

If your job completes successfully, then the subg03 script will delete the files. If you want to retain the files, then comment out the following lines in your subg03 script file:

     tail -l $job.out
     set exits = `tail -1 $job.out | cut -d' ' -f 2,3`
     if ( "$exits" == "Normal termination" ) then
        echo "Normal termination - erasing: "%JOB%.rwf  %JOB%.chk  %JOB%.int  %JOB%.2de  JOB%.src
        rm -f %JOB%.rwf; rm -f %JOB%.chk; rm -f %JOB%.int; rm -f %JOB%.2de; rm -f %JOB%.src
     endif
One of the reasons to keep the files is that they may be used with GaussView. However, the files will use excessive disk space. Retain only those files that are necessary.

You may override the default location and place all of the files in some other directory via the environmental variable GAUSS_SCRDIR. For example, to tell Gaussian to locate them in your scratch directory (/scratch/yournetid), enter the following command at the LINUX prompt:

setenv GAUSS_SCRDIR /scratch/$USER

As a result, all subsequent runs during the current login session will place the files in your scratch directory. To make this change permanent, enter the command in your .cshrc file:

    text

If, instead, you wish to relocate one or more of the individual files, this is done by a statement in your input file. For example, if you want to place the rwf file in your scratch directory but leave the remaining files in your home, then you would do two things. First, you would undo any global change made via the setenv statement. Two, you would include the following in your input file:

%rwf=/scratch/jsmith

As a result, the Read_Write file is written to scratch while the other files are written to your home directory. Notice, that you must hardcode your netid into the statement; using the environmental variable $USER will not work.

If you want to relocate a file as well as use a different name, include the new name after the path:

%rwf=/scratch/jsmith/my_example

The system will append the appropriate extension; in this example, rwf.

Argo is a 32-bit machine. No file may be no larger than 2GB. If any of your scratch files will exceed 2GB, then you must "break up" the file into multiple pieces with each piece no larger than 2GB. What follows is the generic syntax to do this:

rwf=loc1,size,loc2,size2,...

The format also applies to the Integral file and/or the Derivative file. Obviously, for those files, the rwf would be changed to the appropriate extension. What follows is an example of how to partition the rwf file into two separate pieces in the scratch filesystem:

rwf=/scratch/jsmith/piece1,245mw,/scratch/jsmith/piece2,245mw,-1

Several things need to be addressed:

  • Change the example netid, jsmith, to your netid.
  • You may select some other name for the component pieces; the names piece1 and piece2 were selected for illustration purposes.
  • Leave the figure 245mw as is; it translates to 1.9GB.
  • Information is not written equally to the partitions. The first 1.9GB of I/O is done to the first partition. When that piece is filled, then I/O occurs on the second partition.

You are limited to a maximum of 8 partitions (approximately 16GB).

One of the advantages of using the scratch system is that it has a substantial amount of space. However, there is a performance penalty for using scratch, which is an NFS-mounted filesystem. If your rwf file is less than 5GB, you may use the /tmp filesystem on the supervisor node running your Gaussian job. To do that (the rwf file will again be used for example purposes):

rwf=/tmp/piece1,245mw,/tmp/piece2,245mw,-1

This will partition the file into two 1.9 GB pieces on the supervisor node. The supervisor node is the first one listed when you display your job status using the qstat -an statement.

      qstat -an

      520.argo.cc.uic  user4  scali_ex job4.pbs   8512  4 -- -- -- R 19:50
      argo1-3/1+argo1-4/1+argo1-2/1+argo1-1/1
place both partitions (piece1 and piece2) in the /tmp filesystem on argo1-3 (the supervisor node)
 
     
Starting your Gaussian job
 

The commands to execute a Gaussian job are in the file subg03. To run the job, enter:

    text

DO NOT do the following:

    text
In the correct format of the qsub, there are three fields that require an operand:
    text

    • the X following nodes=
    • the Y following ppn=
    • the queue following -q
Each of the fields are explained in other documents. For more information about the X and Y values, click here. For more information about queues, click here.
 
     
Warnings and Problems
 
  • If you see the following message in your output file:

Warning: no access to tty (Bad file descriptor).
Thus no job control in this shell.

Disregard it; it has to do with a bug in shell scripts and has no bearing on your output.

  • I submitted my subg03 script for execution and received the following message:

Permission denied..

It means that you don't have user execute permssion on the file:

    text
To fix it, execute the following command:

chmod u+x subg03

  • If you see:

ntsnet: unable to schedule the minimum 3 workers.

the likely cause is an incompatability among the number given on %NProcLinda and the number of nodes specified in either .tsnet.config file or on the qsub command. For example:

.tsnet.config: Tsnet.Appl.nodelist: argo3-4.cc.uic.edu argo3-3.cc.uic.edu argo3-2.cc.uic.edu
qsub: qsub -l nodes=4 -q staff subg03
input file: %NProcLinda=4

the problem is that the number of nodes in the %NProcLinda variable is set to four but only three compute nodes (3-4, 3-3, and 3-2) are listed in the .tsnet.config. As stated before, it is best not to edit the .tsnet.config file.

  • If you see something like:

    node argo.cc.uic.edu(1): port xxxxx: keepalive failed:
    node argox-x.cc.uic.edu(2), port xxxx did not respond, aborting
    Linda Error: node argo.cc.uic.edu(1): keepalive failure
    ntsnet: unable to stop all processes
    ntsnet: process on node argox-x.cc.uic.edu running

    the problem is one of the nodes on which you were running your job has crashed.

  • If you see something like:

    Linda // error: [0,17516]: send_recv.c 304:
    tcpread failed for 64 reading from 64:
    no route to host.
    Signal #15 received

    the problem is that the job has lost communication with one of the nodes on which you were running your job.

  • If your job starts executing but produces no output and then unexpectantly ends, then one explanation, though there may be other causes, is that you failed to properly path one or more of your file specifications in your input file:
    text
    The specification of the two files in the above input file does not include the path to the file; only the name has been included. That is incorrect. The specificatin MUST both the path to and the name of the file. Examples of properly path files:
      %Chk=/home/homes50/jsmith/myrun.chk
      %Chk=/scratch/jsmith/myrun.chk

      In the first example, the path is /home/homes50/jsmith/ and the name of the file is myrun.chk. In the second example, the path is /scratch/jsmith/ and the name of the file is myrun.chk.
    You may not use environmental variables in the path specification. For example, using $HOME and $USER will not work:
      Wrong: %Chk=$HOME/myrun.chk
      Wrong: %Chk=/scratch/$USER/myrun.chk
  •  
         
    Additional Help
     

    The Gaussian web site

     

     


    2011-10-1  ACCC Systems Group
    UIC Home Page Search UIC Pages Contact UIC