NAME
tfdnoise - noise suppression, sub spectral balancing using
sample wise median thresholding withing frequency subbands
in time - frequency space.
Keywords: glitch, spurious, spike, air blast, prop noise,
TF filtering, time, frequency, subband.
SYNOPSIS
tfdnoise [ -Nntap ] [ -Ootap ] [ -sstart ] [ -eend ] [
-nsnstr ] [ -nenetr ] [ -rsnrst ] [ -renred ] [
-swinsample_window ] [ -twintrace_window ] [
-tmultthreshold_mult ] [ -hwHdrWrd ] [ -hwtIstWrd ] [ -bias-
bias ] [ -thresholdthreshold ] [ -minsbminsb ] [ -maxsbmaxsb
] [ -roll ] [ -V ] [ -? or -h or -help ]
DESCRIPTION
tfdnoise reads in USP formatted data one record at a time.
Each record is transformed into time - frequency space using
the classic stft [Short Time Fourier Transform] algorithm
used in the USP routine stft . The transform is separated
into amplitude and phase components for each frequency sub-
band. If auto-thresholding is requested [ no -threshold[]
entry on the command line] then the median spectral ampli-
tude within each requested frequency subband is calculated
for the record. The median of these medians [multiplied by
any -tmult[] entry] becomes the threshold for that record.
Each sample of each requested subband is compared against
this threshold. If the sample amplitude exceeds the thres-
hold then the median spectral amplitude of the adjacent
-twin[] samples within that subband is computed and
installed at this location. The effect is to replace
outlier spectral amplitudes as defined with the -twin[] in
use. If the threshold is too small the data will be heavily
balanced [i.e. a lot of amplitudes will be replaced with
local median values]. If too large, glitches and spikes may
not be adequately damped. In practice, the autothreshold
tends to be a bit small. The command line entry -tmult[]
enables the user to boost the autothreshold still allowing
an adaptive threshold but tuned to affect only the events
[glitches, noise etc] of interest. The inverse stft is
then applied and the balanced data written out. This routine
consumes a lot of RAM and requires quite a few CPU cycles to
operate. If your noise is isolated to a small set of sub-
bands or a subset of times you can decrease run times by
using the -minsb[],-maxsb[], -s[], -e[] or -hwt[] entries.
The threshold determination and threshold violation search
will be constrained to only those subbands and times
requested. These parameters only affect the amount of data
transformed and balanced. The full range of data input will
also be output from this routine.. If you are using large
-twin[] values you may see edge effects at the onset and
offset of your records. To reduce this effect try the -roll
parameter will will cause the median search window to roll
into and off of the records. The trade off is that glitches
near the edges of records may not be adequately suppressed.
tfdnoise gets both its data and its parameters from command
line arguments. These arguments specify the input, output,
filter start and end times, the start and end traces, start
and end records, the number of samples in the decomposition
window, the number of traces over which to compute a spatial
median spectral amplitude, an auto threshold multiplier, a
global threshold, a user defined record trace header
mnemonic and verbose printout, if desired.
Command line arguments
-N ntap [default: stdin]
Enter the input data set name or file immediately after
typing -N unless the input is from a pipe in which case
the -N entry must be omitted. This input file should
include the complete path name if the file resides in a
different directory. Example -N/b/vsp/dummy tells the
program to look for file 'dummy' in directory '/b/vsp'.
-O otap [default: stdout]
Enter the output data set name or file immediately
after typing -O. This output file is not required when
piping the output to another process. The output data
set also requires the full path name (see above).
-s start [default: 0]
Enter the filter start time in the units of the data.
Data above this time will be passed unfiltered. This
entry defines the upper limit of the data to be
transformed and hence has a large affect on run time.
If you can get away with starting the process some tem-
poral [or depth] distance into your data you can save
quite a few CPU cycles.
-e end [default: end of trace]
Enter the filter end time in the units of the data.
Data below this time will be passed unfiltered. This
entry defines the lower limit of the data to be
transformed and hence has a large affect on run time.
If you can get away with ending the process some dis-
tance up from the end of your record you can save quite
a few CPU cycles.
-ns nstr [default: 1]
Enter the first sequential trace to filter within a
record. Traces prior to this will be passed unfil-
tered. This entry defines the first trace to be
transformed and hence has a large affect on run time.
If you can start the process some number of traces into
your record you can save quite a few CPU cycles.
-ne netr [default: last trace of record]
Enter the last sequential trace to filter within a
record. Traces after this will be passed unfiltered.
This entry defines the last trace to be transformed and
hence has a large affect on run time. If you can end
the process some number of traces from the end of your
record you can save quite a few CPU cycles.
-rs nrst [default: 1]
Enter start sequential record to filter. Records prior
to this will be passed unfiltered.
-re nred [default: last record]
Enter end sequential record to filter. Records past
this will be passed unfiltered.
-swin sample_window [default: 32]
Enter the number of samples in the stft sampling win-
dow. If this is not a power of 2, the next higher
power of two will be used. The number of frequency sub-
bands in the TF transform will be the total number of
samples in the sampling window divided by 2 plus 1 [N/2
+ 1]. Under no circumstances will this entry be allowed
below 32 as values less than that number result in
quite a bit of energy loss in the full circle
transform. This is due to the Gaussian taper used in
the sampling window. It takes at least 32 samples to
give a stable inverse.
-minsb min_sub_band [default: 1]
Enter the minimum time frequency subband to be pro-
cessed. Any subband less than this value will not be
part of the threshold determination [if used] nor will
it be searched for threshold violations. This parame-
ter is usefull to decrease run times in certain cases.
If you are unfamiliar with the time frequency character
of the noise you are suppressing you are better off to
let this value default. To get an idea of what is rea-
sonable you could use the USP routine stft to decompose
into time-frequency a selected set of input records.
Observation of the nature of your noise might help you
decide if this parameter could speed things up for you.
-maxsb max_sub_band [default: N/2 + 1]
Enter the maximum time frequency subband to be pro-
cessed. Any subband greater than this value will not
be part of the threshold determination [if used] nor
will it be searched for threshold violations. This
parameter is usefull to decrease run times in certain
cases. If you are unfamiliar with the time frequency
character of the noise you are suppressing you are
better off to let this value default. To get an idea
of what is reasonable you could use the USP routine
stft to decompose into time-frequency a selected set of
input records. Observation of the nature of your noise
might help you decide if this parameter could speed
things up for you.
-hwt start time mnemonic [default: not used]
Enter the trace header mnemonic used to contain a hor-
izon based start time for the program. This will
define [along with -bias below] the start time for
the threshold balance operation for the program. Data
above this time will be passed unaffected by the pro-
cess. Unlike the -ist[] and -iend[] entries above,
this entry does not affect how much data actually gets
transformed. All data between -ist[], -iend, -ns[]
and -ne[] will actually be transformed. Only data
within the limits defined by this entry will be exam-
ined and balanced.
-bias start time bias [default: 0.0]
Enter any start time bias desired. This time will be
added to the time extracted from -hwt[] above to
form the actual time at which the process will start
for a given trace. This allows one to vary start time
along a horizon +/- a constant time which comes in
handy if you want to avoid a particularly powerful
event [such as direct arrivals] in the threshold calcu-
lation.
-twin trace_window [default: 5]
Enter the number of adjacent traces over which to com-
pute the median spectral amplitude when the spectral
amplitude threshold is surpassed at a given sample.
The larger you make this number, generally the more
balancing is done in the TF spectra. The result in the
TX domain is more even amplitudes. If you are trying
to de-glitch a dataset where the nature of the glitches
are that they occur on single traces here and there but
perhaps over many samples within a trace then the
default of 5 is great. If you have many adjacent
traces involved in the noise you are trying to suppress
you will need to make this large enough to see not just
all the noise traces but a reasonable number of regular
traces as well so that the noise trace samples do not
occupy the median. A few simple tests on some sample
data are usually sufficient to determine this parame-
ter. In general shorter trace apertures are better. If
you find that you need such a large value of -twin[]
that the edges of your records start to look odd you
can augment this argument with the -roll flag which
will cause the median search window spatial length to
roll into and out of your record. That is it will
start at 3 traces and progress up to -twin[] as you
roll into the data. It will also shrink from -twin[]
to 3 as you roll off the record.
-roll [default: not used]
Enter on the command line to activate a rolling median
calculation window [see -twin discussion above]. If
not present the median calculation window will remain
static throughout the record at the value set by
-twin[] above.
-tmult auto threshold multiplier [default: 1.0]
Enter the desired auto threshold multiplier. If you
find that too much of your data is being affected by
this routine you can cause less filter affect by rais-
ing the value of the threshold calculated. If a global
threshold can be found that works for your data it can
be installed on the command line using -threshold[]
below. If a record dependant auto threshold is desired
but less filtered that the default simply experiment
with larger values of -tmult[] until the desired output
filter level is reached. On the few datasets passed
through the routine to date -tmult entries between 2.5
and 8 have been found useful.
-hw auto threshold key mnemonic [default: RecNum]
Enter the trace header mnemonic which when change is
detected will key the routine to recalculate the auto
threshold. If you are making use of the USP routine
partition to split up large records this parameter will
prevent multiple thresholds per record from being cal-
culated.
-threshold global threshold [default: 0.0]
If a non zero -threshold[] entry is detected all auto
threshold activity is turned off. At that point all
-tmult[], -hw[] entries are ignored and this single
threshold is used for the entire dataset.
-V Enter the command line argument '-V' to get additional
printout.
-? or -h or -help
Enter the command line argument '-?' or -h to get
online help. The program terminates after the help
screen is printed.
DISCUSSION
It is recommended that you let the program do auto thres-
holding as it can adapt in a record to record sense to
changes in your data. It is difficult, if not impossible,
to do this using a global threshold. In practice a -tmult[]
entry of near 6.0 has proven to be useful though this may
vary depending on the noise characteristics of your data.
If you wish to try a global threshold first extract a selec-
tion of traces from your dataset that exhibit the noise
characteristics you wish to filter out anduUsing the USP
routine stft [ with the -TF option flagged] generate TF
transforms of these traces. Plot the TF amplitude spectra
[first half of the output trace...last half is phase] and
determine the background threshold that would isolate your
noise event. Install this on the tfdnoise command line
using the -threshold[] entry above.
You may notice that the noise is isolated within a select
set of subbands. If so, you may make use of -minsb[] and
-maxsb[] to decrease the run time of this routine. The run
time is also affected by the choice of processing window [
-s[] -e[] -ns[] -ne[] ] as data outside these limits need
not be transformed.
POTENTIAL PITFALLS
Since the auto threshold is a median over all traces and all
time it is a good idea to have at least a t**n type ampli-
tude scalar applied to the input data to account for spheri-
cal divergence. Also since the median replacement algo-
rithm works across samples it is a good idea to run this on
nmo corrected data, or perhaps on offset gathers so that
adjacent samples might actually be appropriate candidates
for spectral amplitude replacement. Remember that the
thresholding is done within frequency subbands so that only
part of the spectrum is usually affected event in the pres-
ence of some pretty strong glitch energy. For more informa-
tion, and some examples, see:
http://eptgweb.amoco.com/~usp/TechTransfer/Reports/USPNotes/tfdnoise/tfdnoise.html
or call the USP shop.
BUGS
unknown
SEE ALSO
tfskill(1), partition(1), skill(1), glitches(1), stft(1)
AUTHOR
[Paul G.A. Garossino: 321-3615] September, 2000
COPYRIGHT
copyright 2001, Amoco Production Company
All Rights Reserved
an affiliate of BP America Inc.
Man(1) output converted with
man2html