Disk Space on Borg

	Academic Computing and Communications Center


2001-2002, Number 1 Contents	What's New at the ACCC	Managing Your ACCC Email (Quotas)	Figure 1: Quota, Download, Delete Tool for icarus and tigger	Figure 2: Quota, Download, Delete Tool for mailserv
Statistical Tools of the Trade	Disk Space on borg	Repeated Infections with a Particular Virus	About the A3C Connection

Disk Space on Borg

The Head Crash

UNIX Everyone

Question: Hi, my name is Hillary Cypert, hcyper1. I have an account on the borg system. I am running some heavy duty programs and I have exceeded my disk quota. [See figure 4 below.] Borg says that I should remove 25000 Kb in eight hours. I CAN'T! I need the files I have now and I am probably going to need twice or three times as much in the future. Please stop borg from removing my files and assign me more disk space as soon as possible.

Answer: Hi.

When you login, did you notice the third line of the quota message? [See figure 4 below.]

Figure 4. Quota for Student Accounts on borg

Borg executes the quota command for you when you login. This is Hillary Cypert's. The third line says that Hillary is over her soft quota for her home directory. (Her quota is 50 Mb and she's using 75 Mb.) The second line says that Hillary has a 500 Mb -- one-half gigabyte -- quota in /scratch.

Even though Hillary isn't using her scratch space, she has one file in /scratch. Everyone has at least one file in /scratch -- their scratch "home" directory, /scratch/netid/
For Hillary, that's /scratch/hcyper1/

Disk quotas for hcyper1 (uid 12345):
       Filesystem   usage  quota  limit   timeleft files quota limit
       /scratch         0 500000  550000         -     1     0     0
       /homes/home5 74999  50000   75000 7.8 hours   191     0     0

It is true that you have only 50 Mb of disk space in your home directory. But you also have a directory in the scratch area, /scratch/hcyper1. That directory has a quota of 500 Mb [that's half of a gigabyte!], ten times the quota in your home directory.

I took a look in your home directory and one file is the cause of your problem:

 -rw-------   1 hcyper1 student 75236352 Feb 22 03:59 1013796123.21405.out

[That's 75 Mb in the one output file.]

Here's what you should do. Before you run your job, in this case using bsub, move your data file and your job file from your home directory to your scratch directory. Then, cd to /scratch/hcyper1 and execute the bsub job from scratch.

Not only will you have a soft quota of 500 Mb instead of 50 Mb, you will also see performance gains. The scratch filesystem is stripped unlike the home filesystem. A stripped filesystem is one that spans several disks. In the case of scratch, there are fourteen disks comprising it. That means you have much less competition for access to disk heads, because disk writes span several disks. On the other hand, the home filesystem is on a single disk. That means you compete with every other logged-in user for access to the disk.

(Think of borg's filesystem IO like cars travelling on toll roads. Each chunk of data is in a separate car. The road to the scratch filesystem has fourteen lanes, each with its own toll booth. The road to your home directory has only one lane and one booth. Which do you think is faster?)

To ease the urgency of the situation, I've doubled your home directory to 100 Mb. However, I urge you to use scratch for the reasons I've outlined above.

Michael Homa
Systems Group

[Michael is the keeper of our compute server, borg. bsub is part of the LSF, Load Sharing Facility, the set of commands that you use on borg to submit and manage jobs. For more information, see "LSF on the Borg System," http://www.accc.uic.edu/hardware/borg/lsf/ ]

The A3C Connection, 2001-2002, Number 1

Previous: Statistical Tools of the Trade

Next: Repeated Infections with a Particular Virus

2002-5-10 connect@uic.edu