Advanced Buckets¶
- class qarnot.advanced_bucket.AbstractFiltering[source]¶
Bases:
object
Abstract base class for resources filtering, allowing to select only a subset of a resources bucket as task resources.
- abstract to_json()[source]¶
Get a dict ready to be json packed.
- Raises
NotImplementedError – this is an abstract method, it should be overridden in child classes
- abstract classmethod from_json(json)[source]¶
Static method called to create the class obj from a json object.
- Parameters
json (Dict) – the json elements of the class
- Raises
NotImplementedError – this is an abstaract method, override it in it’s child classes.
- abstract sanitize_filter_paths(show_warnings: bool)[source]¶
Sanitize the filters bucket path by removing extra separators
- Raises
NotImplementedError – this is an abstract method, it should be overridden in child classes
- class qarnot.advanced_bucket.BucketPrefixFiltering(prefix: str)[source]¶
Bases:
qarnot.advanced_bucket.AbstractFiltering
Allows to filter a resources bucket by name prefix. Only bucket files starting with the given prefix will be used as task resources.
- classmethod from_json(json: Dict[str, str])[source]¶
Static method called to create the class obj from a json object.
- Parameters
json (dict) – the json elements of the class
- class qarnot.advanced_bucket.Filtering[source]¶
Bases:
object
Groups the various object filters on an advanced resources bucket.
- append(filtering: qarnot.advanced_bucket.AbstractFiltering)[source]¶
Add a new filtering object
- Parameters
filtering (AbstractFiltering) – a filtering object of the bucket
- classmethod from_json(json) qarnot.advanced_bucket.Filtering [source]¶
Create the class sub objects of a Filtering from a json.
- Parameters
json (dict) – the json elements of the class
- class qarnot.advanced_bucket.AbstractResourcesTransformation[source]¶
Bases:
object
Abstract base class for resources transformation, allowing to transform bucket objects before they are presented to the task as resources.
- abstract classmethod from_json(json)[source]¶
Static method called to create the class obj from a json object.
- Parameters
json (dict) – the json elements of the class
- abstract sanitize_transformation_paths(show_warnings: bool)[source]¶
Sanitize the bucket path by removing extra separators
- Raises
NotImplementedError – this is an abstract method, it should be overridden in child classes
- class qarnot.advanced_bucket.PrefixResourcesTransformation(prefix: str)[source]¶
Bases:
qarnot.advanced_bucket.AbstractResourcesTransformation
Allows to remove a prefix from bucket files before they are presented to the task. During execution, the task will see files with paths stripped of the given prefix in its working directory.
- __init__(prefix: str)[source]¶
The PrefixResourcesTransformation constructor
- Parameters
prefix (str) – the prefix path of the resource bucket to be removed.
- class qarnot.advanced_bucket.ResourcesTransformation[source]¶
Bases:
object
Groups the various object transformation on an advanced resources bucket.
- append(resource: qarnot.advanced_bucket.AbstractResourcesTransformation)[source]¶
Add a new resource transformation object
- Parameters
filtering (AbstractResourcesTransformation) – a filtering object of the bucket
- classmethod from_json(json) qarnot.advanced_bucket.ResourcesTransformation [source]¶
Create the class sub objects of a ResourcesTransformation from a json.
- Parameters
json (dict) – the json elements of the class