Back End Requirements

The “backend” handles all of the connections to dynamo objects and databases as well as connections for performing listings of remote or local filesystems.

A complete list and description of the modules and functions that should be accessible through this module is the following.

DatasetFilter

class dynamo_consistency.backend.DatasetFilter(datasets)[source]
Parameters:datasets (set) – A set of datasets that are protected by dynamo.
protected(filename)[source]
Parameters:filename (str) – Name of a file that is checked for filtering
Returns:True if the file is protected
Return type:bool

check_site

dynamo_consistency.backend.check_site(site)[source]
Parameters:site (str) – Site to check status of
Returns:True, if site is ready to run on
Return type:bool

deletion_requests

dynamo_consistency.backend.deletion_requests(site)[source]
Parameters:site (str) – The site that we want the deletion requests for
Returns:A set of datasets that have pending deletion requests
Return type:set

filelist_to_blocklist

dynamo_consistency.backend.filelist_to_blocklist(site, infile, outfile)[source]

Converts a file of list of files to a summary of blocks and owning groups.

Parameters:
  • site (str) – Name of the site to get group names for.
  • infile (str) – Location of the file that contains the list of files
  • outfile (str) – Location of the file to output blocks summary

get_listers

dynamo_consistency.backend.get_listers(site)[source]
Parameters:site (str) – The name of the site that we want listers for
Returns:A tuple containing a constructor (or function) and object creation parameters (or None) for passing to dynamo_consistency.create.create_dirinfo().

Optionally, the first parameter of the tuple can be a fully constructed dynamo_consistency.datatypes.DirectoryInfo. In that case, the second returned value is ignored by dynamo_consistency.remotelister.listing().
Return type:tuple

inventory

A module that access contents of sites and other information from Dynamo’s internal inventory.

dynamo_consistency.backend.inventory.protected_datasets(site)
Parameters:site (str) – Site to query
Returns:The datasets that are protected by Dynamo
Return type:set
dynamo_consistency.backend.inventory.list_files(site)
Parameters:site (str) – Site to query
Returns:List of files at the site. Each element of the list is a tuple with (file name, size in bytes, datetime object)
Return type:list

registry

Module that handles transfers and deletions

dynamo_consistency.backend.registry.delete(site, files)

Requests deletion of files from a site

Parameters:
  • site (str) – The site to remove files from
  • files (list) – List of LFNs of files (and directories) to remove
dynamo_consistency.backend.registry.transfer(site, files)

Requests transfer of files to a site

Parameters:
  • site (str) – The site to transfer files to
  • files (list) – List of LFNs of files to transfer

siteinfo

Site information from inventory

dynamo_consistency.backend.siteinfo.site_list()
Returns:List of sites known by Dynamo
Return type:list
dynamo_consistency.backend.siteinfo.ready_sites()
Returns:Sites that are ready to run on
Return type:set