Back End Requirements¶

The “backend” handles all of the connections to dynamo objects and databases as well as connections for performing listings of remote or local filesystems.

A complete list and description of the modules and functions that should be accessible through this module is the following.

DatasetFilter
check_site
deletion_requests
filelist_to_blocklist
get_listers
inventory
registry
siteinfo

DatasetFilter ¶

class dynamo_consistency.backend.DatasetFilter(datasets)[source]¶

Parameters:	datasets (set) – A set of datasets that are protected by dynamo.

protected(filename)[source]¶

Parameters:	filename (str) – Name of a file that is checked for filtering
Returns:	True if the file is protected
Return type:	bool

check_site ¶

dynamo_consistency.backend.check_site(site)[source]¶

Parameters:	site (str) – Site to check status of
Returns:	True, if site is ready to run on
Return type:	bool

deletion_requests ¶

dynamo_consistency.backend.deletion_requests(site)[source]¶

Parameters:	site (str) – The site that we want the deletion requests for
Returns:	A set of datasets that have pending deletion requests
Return type:	set

filelist_to_blocklist ¶

dynamo_consistency.backend.filelist_to_blocklist(site, infile, outfile)[source]¶

Converts a file of list of files to a summary of blocks and owning groups.

Parameters:	site (str) – Name of the site to get group names for. infile (str) – Location of the file that contains the list of files outfile (str) – Location of the file to output blocks summary

get_listers ¶

dynamo_consistency.backend.get_listers(site)[source]¶

Parameters:	site (str) – The name of the site that we want listers for
Returns:	A tuple containing a constructor (or function) and object creation parameters (or None) for passing to `dynamo_consistency.create.create_dirinfo()`.
Return type:	tuple

inventory ¶

A module that access contents of sites and other information from Dynamo’s internal inventory.

dynamo_consistency.backend.inventory.protected_datasets(site)¶

Parameters:	site (str) – Site to query
Returns:	The datasets that are protected by Dynamo
Return type:	set

dynamo_consistency.backend.inventory.list_files(site)¶

Parameters:	site (str) – Site to query
Returns:	List of files at the site. Each element of the list is a tuple with (file name, size in bytes, datetime object)
Return type:	list

registry ¶

Module that handles transfers and deletions

dynamo_consistency.backend.registry.delete(site, files)¶

Requests deletion of files from a site

Parameters:	site (str) – The site to remove files from files (list) – List of LFNs of files (and directories) to remove

dynamo_consistency.backend.registry.transfer(site, files)¶

Requests transfer of files to a site

Parameters:	site (str) – The site to transfer files to files (list) – List of LFNs of files to transfer

siteinfo ¶

Site information from inventory

dynamo_consistency.backend.siteinfo.site_list()¶

Returns:	List of sites known by Dynamo
Return type:	list

dynamo_consistency.backend.siteinfo.ready_sites()¶

Returns:	Sites that are ready to run on
Return type:	set