USP FAQ

Questions

1.0 What is the easiest way to reciprocate a dataset?

2.0 I ran [davc -N data1 -Odata1 -w 1000 -amp22.0], the job bombed and my data was erased. What gives??? ;-<

3.0 Where can I find a reference for all the line and trace header mnemonics in USP?

4.0 Why does ufh tend to run very slow on very large datasets?

5.0 I created a USP dataset on 8mm tape using dd bs=32768 on a Solaris system. I tried to read it on a Linux system and got an I/O error, what gives?

6.0 How can I manage the creation of USP printout files during execution?

7.0 I ran rmprint but many printout files were not deleted. Instead I get a message that says something like

 "file  UTOP.4543 appears to be data ; bypassing".printout files

What is causing this and how can I correct for this problem?

Answers

1.0 What is the easiest way to reciprocate a dataset?

Use program scale with an exponent of -1 as in:

scale -exp -1 

2.0 I ran [davc -N data1 -O data1 -w 1000 -amp22.0], the job bombed and my data was erased. What gives??? ;-<
In USP datasets attached at -N are opened for read access while datasets attached at -O are opened for write access. In UNIX you can do both simultaneous. Unfortunately the open for write access put the file pointer at the start of the file which is equivalent to erasing your file. This is a pitfall in USP which most users hit ONCE. Sorry.

3.0 Where can I find a reference for all the line and trace header mnemonics in USP?
Type in:

hdrdesc | xcat &

and you will be provided with a complete listing of the USP line and trace header mnemonics. You can also view this information in the man page for program scan . You can also see the control file used by our code during compilation at: /home/usp/include/save.h.

4.0 Why does ufh tend to run very slow on very large datasets?
If your dataset is over say....10GBytes you will find that writing a full blown program will far outperform ufh in throughput. When ufh was invented a 2Gbyte dataset was all that could be handled. The author was driven out of the company prior to filesystems that could handle > 2Gbyte files. It it probably time to revisit the code specifically for large file performance. Of course now we are merged and have no one with the time to do such a thing................unless you want to give it a try. Martin Smith [the author] has kept the product development over the years and has a version that I am sure he would sell us that is probably a better performer.

This all said, over the years since 1999, [it is now 2005], ufh performance on large datasets is still pretty good for the most part. The thing that will really slow it down is non-vector access of the time series in the trace. Watch out for things like:

Tr.Series[i] = variable

If you just have to do time series element assignment on a very large dataset, you probably want to look for another way of accomplishing this than utilizing ufh.

5.0 I created a USP dataset on 8mm tape using dd bs=32768 on a Solaris system. I tried to read it on a Linux system and got an I/O error, what gives?
Look in the /var/log/messages file on the Linux system and see if you see "incorrect block size" error in there. If so, the problem can be solved by running
mt -f /dev/st0 setblk 0

before attempting the dd if=/dev/st0 bs=32768 .

6.0 How can I manage the creation of USP printout files during execution?
Several tools are now available for manipulating the USP print files. The environment variables USP_PRINT_DIRECTORY, USP_PRINT_PREFIX, and USP_PRINT_SUFFIX manipulate the location and naming of the print files. The variable USP_PRINT_BYPASS may be used to completely suppress their creation.

For a more detailed discussion, including examples, on the handling of USP print files, visit the page on Printout Management.

7.0 I ran rmprint but many printout files were not deleted. What is causing this and how can I correct for this problem?
rmprint utilizes a magic cookie file out of:

/apps/global/etc/usp/

named magic, or on Linux systems, magic.little-endian to know if the file it "thinks" is a printout file is actually a printout file and not some other type of file that the user has accidentally named with the same "look and feel" as a printout file [i.e. NAME.pid].

USP has another routine [file] that utilizes the same magic cookie file. You can issue the command:

usp file *

in any directory and see what USP thinks the various files are. This is actually quite useful when you have lots of different file types in the same directory. Here is a listing from my current Linux working directory:

EDIT.17422:         ASCII text
FFTXY.17251:        ASCII text
FFTXY.17323:        ASCII text
migagc_1201_to_1300.usp_TI90: USP data: Format 3, NumRec=1, NumTrc=999,NumSmp=4801
migagc_1201_to_1300.usp_TI90_taper:  USP data: Format 3, NumRec=1,NumTrc=999, NumSmp=4801
migagc_1201_to_1300.usp_TI90_taperKXKY:  USP data: Format 3, NumRec=1,NumTrc=1024, NumSmp=9720

As you see, the printout files are of type "ASCII text". If I issue an rmprint command the above printout files are toasted.

There are many ways an odd character can find its way into a printout file and cause the magic cookie derived file association to become something other than ASCII text. When this happens, rmprint will not remove that file, as it no longer is "certain" that it is a printout file. It has been many years since rmprint "accidentally" removed someones 3 Tbyte data file due to an unfortunate choice of naming convention :-)

Run the usp file command in the directory where you are having trouble removing printout files and see if they turn up as ASCII text, or more likely "data".

Most likely something binary is being listed out of the incoming historical line header that is echoed to the printout file. Once you see what is causing the problem, you should be able to get your rmprint working again.

Also be aware that once a USP dataset has some non-ASCII characters in one of ascii fields of the line header or the HLH, that dataset will continue to create printout files which appear as data throughout the rest of the processing in most cases until action is taken to correct those fields. Normally, this can easily be done in the processing flow with a utop step.

Joe Wade wrote a script that corrects all the ASCII header entries to actually be ASCII so that no further carnage is wrecked upon your flow. Of course determining the cause of the non-ASCII characters in the first place is more desireable.

Traditionally, these problems arise when a new program is written which creates a USP dataset from scratch. An unknowing programmer zeroes out the space for the line header and then fails to set all of the ascii fields with valid values. Those zeroed fields then result in non-printable characters in the common USP print routines that print the line header values and the HLH.

Another common occurance is running back and forth between DDS and USP. The new program could be a DDS routine that is doing something equally as dumb.

If your dataset originated in ProMax and you utilized the Promax-USP module to get your data into USP form you may sometimes run into trouble. Not all the promax steps that end up in the hlh have been vetted for non-ASCII content.