pyrocko.squirrel.client.fdsn

Squirrel client to access FDSN web services for seismic waveforms and metadata.

Functions

codes_to_selection(codes_list, tmin, tmax)	Convert codes patterns and a time span to an FDSN-style selection.
combine_selections(selection)	Merge adjacent NSLC-plus-time-span entries into contiguous ones.
diff(fn_a, fn_b)	Check if two files differ in size or content.
make_task(*args)	Create progress indicator task (see pyrocko.progress), logging to this module's logger.
move_or_keep(fn_temp, fn)	Replace a file with a freshly downloaded temporary one, unless their contents are identical.
orders_sort_key(order)	Get sort key for a WaveformOrder, for sorting by codes, then start time.
orders_to_selection(orders[, pad])	Convert waveform orders to an FDSN dataselect-style selection.

Classes

ErrorAggregate(**kwargs)	Summary of several ErrorEntry occurrences sharing the same kind/details, see ErrorLog.iter_aggregates().
ErrorEntry(**kwargs)	Single waveform download error occurrence.
ErrorLog(**kwargs)	Log of waveform download errors encountered by FDSNSource.
FDSNSource(site[, query_args, codes])	Squirrel data-source to transparently get data from FDSN web services.

Exceptions

Aborted

Raised internally to abort an in-progress waveform download, e.g. when unexpectedly much data is being received.

make_task(*args)

Create progress indicator task (see pyrocko.progress), logging to this module’s logger.

diff(fn_a, fn_b)

Check if two files differ in size or content.

move_or_keep(fn_temp, fn)

Replace a file with a freshly downloaded temporary one, unless their contents are identical.

Parameters:

• fn_temp (str) – Path of newly downloaded temporary file.

• fn (str) – Path of target file to be replaced.

Returns:

'updated', 'new' or 'upstream unchanged', describing the action taken.

Return type:

str

combine_selections(selection)

Merge adjacent NSLC-plus-time-span entries into contiguous ones.

Parameters:

selection (iterable of tuple) – Entries of the form (net, sta, loc, cha, tmin, tmax), sorted by codes then start time, where adjacent entries sharing the same codes and touching time spans (end time of one equal to start time of the next) are merged.

Return type:

list of tuple

orders_sort_key(order)

Get sort key for a WaveformOrder, for sorting by codes, then start time.

orders_to_selection(orders, pad=1.0)

Convert waveform orders to an FDSN dataselect-style selection.

Parameters:

• orders (iterable of WaveformOrder objects) – Orders to be converted.

• pad (float) – Number of samples of padding to add at each end of the requested time spans.

Returns:

Combined (see combine_selections()), padded selection entries of the form (net, sta, loc, cha, tmin, tmax).

Return type:

list of tuple

codes_to_selection(codes_list, tmin, tmax)

Convert codes patterns and a time span to an FDSN-style selection.

Parameters:

• codes_list (list of CodesNSLCE objects, or None) – Codes patterns to be converted, or None.

• tmin (get_time_float()) – Start time to be used for all entries.

• tmax (get_time_float()) – End time to be used for all entries.

Returns:

Selection entries of the form (net, sta, loc, cha, tmin, tmax), or None if codes_list is None.

Return type:

list of tuple or None

class ErrorEntry(**kwargs)

Bases: Object

Single waveform download error occurrence.

♦ time

pyrocko.util.get_time_float (pyrocko.guts.Timestamp)

Time at which the error occurred.

♦ order

pyrocko.squirrel.model.WaveformOrder

Waveform order which caused the error.

♦ kind

str

Error category, e.g. 'empty result', 'invalid waveform', 'http error', 'aborted' or 'partial result'.

♦ details

str, optional

Additional free-form details about the error.

class ErrorAggregate(**kwargs)

Bases: Object

Summary of several ErrorEntry occurrences sharing the same kind/details, see ErrorLog.iter_aggregates().

♦ site

str

FDSN site url or alias name the errors occurred at.

♦ kind

str

Common error category of the aggregated entries, see ErrorEntry.kind.

♦ details

str

Common details of the aggregated entries, see ErrorEntry.details.

♦ entries

list of ErrorEntry objects, default: []

Aggregated error entries.

♦ codes

list of pyrocko.model.codes.CodesNSLCE objects, default: []

Channel codes affected by the aggregated errors.

♦ time_spans

list of tuple of 2 pyrocko.util.get_time_float (pyrocko.guts.Timestamp) objects objects, default: []

Distinct (tmin, tmax) time spans affected by the aggregated errors.

class ErrorLog(**kwargs)

Bases: Object

Log of waveform download errors encountered by FDSNSource.

♦ site

str

FDSN site url or alias name the errors occurred at.

♦ entries

list of ErrorEntry objects, default: []

Logged error entries, in chronological order.

♦ checkpoints

list of int objects, default: []

Entry-count marks recorded by append_checkpoint(), used to delimit “recent” entries for summarize_recent().

append_checkpoint()

Mark current end of log, for later use by summarize_recent().

append(time, order, kind, details='')

Append an error entry to the log.

Parameters:

• time (get_time_float()) – Time at which the error occurred.

• order (WaveformOrder) – Waveform order which caused the error.

• kind (str) – Error category, see ErrorEntry.kind.

• details (str) – Additional free-form details about the error.

iter_aggregates()

Iterate over ErrorAggregate summaries, grouping logged entries by (kind, details).

Yields:

ErrorAggregate objects.

summarize_recent()

Get one-line summary of errors logged since the last checkpoint (see append_checkpoint()), or of all entries if no checkpoint was set.

Return type:

str

exception Aborted

Bases: SquirrelError

Raised internally to abort an in-progress waveform download, e.g. when unexpectedly much data is being received.

class FDSNSource(site, query_args=None, codes=None, **kwargs)

Bases: Source

Squirrel data-source to transparently get data from FDSN web services.

Attaching an FDSNSource object to a Squirrel allows the latter to download station and waveform data from an FDSN web service should the data not already happen to be available locally.

♦ site

str

FDSN site url or alias name (see pyrocko.client.fdsn).

♦ query_args

dict of str objects, optional

Common query arguments, which are appended to all queries.

♦ codes

list of pyrocko.model.codes.CodesNSLCE objects, optional

List of codes patterns to query via POST parameters.

♦ expires

float (pyrocko.guts.Duration), optional

Expiration time [s]. Information older than this will be refreshed. This only applies to station-metadata. Waveforms do not expire. If set to None neither type of data expires.

♦ anxious

float (pyrocko.guts.Duration), default: 600.0

Anxiety period [s]. Missing waveforms will not be treated as permanently missing for orders spanning into times later than (current time - anxious). Default: 600.

♦ cache_path

str (pyrocko.has_paths.Path), optional

Directory path where any downloaded waveforms and station meta-data are to be kept. By default the Squirrel environment’s cache directory is used.

♦ shared_waveforms

bool, default: False

If True, waveforms are shared with other FDSN sources in the same Squirrel environment. If False, they are kept separate.

♦ storage_path

str (pyrocko.has_paths.Path), optional

If set, manage waveforms in the given directory rather than in the Squirrel cache.

♦ storage_scheme

str (pyrocko.squirrel.storage.StorageSchemeChoice), default: 'default'

Set layout of waveform storage. Available: 'default', 'rug-store-10', 'rug-store-100', 'rug-store-1000', 'sds'. Default: 'default'.

♦ user_credentials

tuple of 2 str objects, optional

User and password for FDSN servers requiring password authentication

♦ auth_token

str, optional

Authentication token to be presented to the FDSN server.

♦ auth_token_path

str (pyrocko.has_paths.Path), optional

Path to file containing the authentication token to be presented to the FDSN server.

♦ hotfix_module_path

str (pyrocko.has_paths.Path), optional

Path to Python module to locally patch metadata errors.

get_cache_path()

Get path to this source’s on-disk cache directory.

Return type:

str

describe()

Get short identifying description of the source.

Return type:

str

make_hash()

Compute hash identifying this source’s configuration.

Used to distinguish between differently-configured FDSNSource instances pointing at the same site when choosing on-disk cache locations, so e.g. anonymous and authenticated access to the same site are cached separately.

Return type:

str

get_hash()

Get hash identifying this source’s configuration (see make_hash()).

Return type:

str

get_auth_token()

Get authentication token, reading it from auth_token_path if auth_token is not set directly.

Return type:

str

Raises:

FileLoadError – If auth_token_path cannot be read.

setup(squirrel, check=True, upgrade=False)

Set up cache directories and add already-downloaded local content (waveforms, channel and response meta-data) to the squirrel.

See setup() for the parameters. If upgrade is True, any waveform archive found using the old (pre -v2) on-disk storage layout is migrated in place; otherwise it is added as-is and a warning is logged.

update_channel_inventory(squirrel, constraint=None)

Query and cache channel/station meta-data for a given constraint, unless already cached and not yet expired (see expires).

Parameters:

• squirrel (Squirrel) – Squirrel instance the source is attached to.

• constraint (Constraint) – Selection of times or areas to be brought up to date. If not given, an unconstrained (global) query is made.

update_waveform_promises(squirrel, constraint)

Add waveform promises for channels with cached meta-data but missing (or gappy) local waveform coverage, for a given constraint.

Parameters:

• squirrel (Squirrel) – Squirrel instance the source is attached to.

• constraint (Constraint) – Selection of times or areas to promise waveforms for.

remove_waveform_promises(squirrel, from_database='selection')

Remove waveform promises from live selection or global database.

Parameters:

from_database – Remove from live selection 'selection' or global database 'global'.

save_waveforms(trs)

Save downloaded waveforms to the local waveform archive.

Parameters:

trs (list of Trace objects) – Waveforms to be saved.

Returns:

Paths of the created/updated files.

Return type:

list of str

download_waveforms(orders, success, error_permanent, error_temporary, aborted)

Execute a batch of waveform download orders.

Parameters:

• orders (list of WaveformOrder objects) – Waveform orders to be downloaded.

• success (callable) – Callback success(order, traces), called for each successfully (or partially) downloaded order.

• error_permanent (callable) – Callback error_permanent(order), called for orders which failed in a way not expected to be resolved by retrying later.

• error_temporary (callable) – Callback error_temporary(order), called for near-real-time orders (see is_near_real_time()) which failed but may succeed if retried later.

• aborted (callable) – Callback aborted(), returning True if downloading should be aborted before processing further orders.

Downloaded waveforms are not saved automatically; success is expected to do so, e.g. via save_waveforms(). Download errors are logged to an internal ErrorLog, aggregated summaries of which are logged as warnings.

update_response_inventory(squirrel, constraint)

Query and cache instrument responses for channels matching a given constraint’s codes, unless already cached and not yet expired (see expires).

Parameters:

• squirrel (Squirrel) – Squirrel instance the source is attached to.

• constraint (Constraint) – Selection of codes to update responses for. Only codes is used; responses are queried for the channels’ entire available time span.