post.essc.psu.edu

earth is a vector processor machine, which means that its performance on some common scalar operations is less than stellar. This seems to be most noticeable when trying to use certain commands such as scp to transfer files, or gzip to compress files. These commands perform best on scalar machines, and their performance suffers when run on the Cray. Furthermore performing such tasks where substantial CPU time will be consumed on the Cray a is not a particularly good use of the machine.

To address this issue post.essc.psu.edu a dual-processor Sun Ultra 80 with 1GB of memory that is dedicated to performing postprocessing scalar operations on earth's /svarch filesystem, and the IBM SP's /sparch filesystem, which are both readable and writable from post. You can use this machine for post-processing tasks that would take too long to run on earth. We recently had a perfect example that demonstrated the differences between the two architectures:

A user was running several instances of a code on earth that sub-setted a large (180+ MB) NetCDF file, using the NetCDF libraries. Due to what we believe is partly due to inefficiency in the Cray implementation of the NetCDF libraries, the code not only took an extremely long time to execute, but created a massive load on earth's disks, slowing the entire machine down.
When run on post, the exact same code took a tiny fraction of the time it took to run on earth, and multiple instances caused no problems whatsoever on either machine.

post is set up exactly like the other sun machines, so you will still be able to access your home directory and storage directories, in addition to your directories in /svarch and /sparch.

Here is how the Cray and SP filesystems are cross-mounted for access to data.

	`earth`	SP	`post.essc.psu.edu`	other ECF Suns
`earth:/svarch`	local	read-only	read-write	read-only
`SP:/sparch`	read-only	local	read-write	read-only

Basically, the two filesystems are cross-mounted read-only everywhere on the ECF network to provide fast access to data, but post has read-write access to do postprocessing work.

Here are some of the tasks for which post is ideally suited:

subsetting or other postprocessing of large architecture-independent (such as NetCDF) files
compressing or uncompressing files
securely transferring files via scp
processing large files using standard cpu-intensive unix commands
processing files using commands and programs that aren't available on earth

Please refrain from using post for any of the following:

Using post solely to transfer large amounts of data to /svarch or /sparch (transferring from these filesystems to a directory on the suns is fine). This is because writing to these filesystems can cause a heavy load on earth and the SP, since it's done over the network.
Heavy write activity is often an unavoidable aspect of postprocessing, which is a perfectly acceptable use of post. We simply want to make sure it's not used as a means of simply transferring files to and from the suns, which is better accomplished via FTP.

Anything not related to the postprocessing of data from /svarch or /sparch.

Since post is a shared machine, please respect our usual recommendations for etiquette and shared resources so that as many people can benefit from this machine as possible. Thank you.

Back to the index