Python API datasets module

The zoautil_py.datasets module contains the utilities needed to interact with and manipulate z/OS data sets. This module has classes and functions.


Classes

This module contains a single Dataset class with its own methods and non object-oriented functions.



class zoautil_py.datasets.Dataset(name, record_format=None, record_length=None, block_size=None, organization=None, volume=None, last_referenced=None, used_space=None, total_space=None)


Description

  • This class represents the z/OS dataset.

  • Bases: object

  • Defined in: zoautil_py.datasets

Parameters

  • name (str) - Name of the dataset.

  • organization (str) - Dataset organization of the dataset. None if unknown.

  • record_format (str) - Record format of the dataset. None if unknown.

  • volume (str) - Volume the dataset resides on. None if unknown.

  • block_size (int) - Block size of the dataset. None if unknown.

  • record_length (int) - Record length of the dataset. None if unknown.

  • total_space (int) - Estimated total space of the dataset. None if unknown.

  • used_space (int) - Estimated used space of the dataset. None if unknown.

  • last_referenced (datetime) - Date when the dataset was last referenced. None if unknown.

Other methods



Functions

The datasets module provides functions for interacting with z/OS Dataset objects.



zoautil_py.datasets.blockinfile(dataset: str, state: bool = True, *args, **kwargs)


Description

  • ZOAU dmod function to be used by zos_blockinfile Ansible module.

Returns

  • ZOAUResponse - Response object from dsed call.

Return type

  • str

Parameters

  • dataset (str) – The target dataset or z/OS UNIX file to modify to (example: “IBMUSER.TEST.MOD”).

  • state (boolean) – True (default): Insert or replace block. False: Remove block.

Other Parameters

  • block (str) – The line(s) to insert inside the marker lines separated by ‘\\n’. (Example: “line 1\\nline 2\\nline 3”)

  • marker (str) – The marker line template in this format <marker_begin>\\n<marker_end>\\n< {mark} marker>

    • The template should be three sections separated by ‘\\n’. (Example: “OPEN\\nCLOSE\\n# {mark} IBM BLOCK”).

    • “{mark}” should be included in the < {mark} marker> (default = ”# {mark} MANAGED BLOCK”) section and will be replaced with <marker_begin> (default = ”BEGIN”) and <marker_end> (default = ”END”).

    • The two marker lines will be surrounding the lines that are going to be inserted.

    • Marker lines can only be used once. If marker lines already exist in the target dataset or HFS file, they will be removed with the surrounded lines before new block get inserted.

  • insert_after (str) – Insert block after matching regex pattern. The special value “EOF” inserts the block at the end of the target dataset or HFS file.

  • insert_before (str) – Insert block before matching regex pattern. The special value “BOF” inserts the block at the beginning of the target dataset or HFS file.

  • encoding (str) – Encoding of the dataset.

  • lock (bool) – Obtain exclusive lock for the dataset.

  • force (bool) – Force open. Open dataset member in DISP=SHR mode. Default is DISP=OLD mode when False.

  • as_json (bool) – Display output in JSON format.

Notes

  • Use debug = True parameter for debug information in JSON format.


zoautil_py.datasets.compare(source, target, *args, **kwargs)


Description

  • Compare two datasets, output the ISRSUPC output.

Returns

  • str (ISRSUPC output) – The output from ISRSUPC.

  • None – Datasets are the same.

Parameters

  • source (str) – First dataset to compare.

  • target (str) – Second dataset to compare.

Other Parameters

  • lines (str, int) - Display n lines of context around a displayed line.

  • columns (str, int) - Specify column comparison specifiers to pass to SUPERCE.

  • ignore_case (boolean) - Ignore case in comparison.



zoautil_py.datasets.copy(source, target, *args, **kwargs)


Description

  • Copy a z/OS source (dataset, HFS file) to a z/OS target.

Returns

  • RC from z/OS.

Return type

  • int

Raises

  • ZOAUException – See exception text for ZOAU error.

Parameters

  • source (str) – Absolute path of z/OS dataset or zFS file to copy from.

  • target (str) – Absolute path of z/OS dataset or zFS file to copy to.

  • alias (bool) - If the source data set has aliases, they will be recreated in the target data set.

  • executable (bool) - This should be set if the source data set is an executable.

  • force (bool) – Forces the copy. IMPORTANT: Use of this option can lead to permanent loss of the original target information.

    • NOTE: If a dataset member has aliases and is not a program object, copying that member to a dataset that is in use results in the aliases not being preserved in the target dataset. When this scenario occurs, an exception is raised, along with an error message and a non-zero return code.

Notes

  • Users can call _copy for a raw function that returns ZOAUResponse.

  • Generation data sets (GDS) are supported with the following considerations:

    • The generation data group (GDG) is not locked when referencing a GDS.

    • Relative names have their scope in the function call. This means that two consecutive copy calls that target a new generation, i.e. "GDG(+1)", result in the creation of two consecutive new generations.

  • Generation data groups (GDG) are supported as source, and can be copied to a new GDG as target. The source GDG attributes and all its active generations are copied to the target. The source GDG base will not be locked, so prefer a job to ensure the GDG order is protected from concurrent operations.



zoautil_py.datasets.create(name, dataset_type, *args, **kwargs)


Description

  • Create a z/OS dataset.

Returns

  • Dataset() object.

Return type

  • types.Dataset object

Raises

  • ZOAUException – See exception text for ZOAU error.

  • DatasetVerificationError – Failed to verify the creation of the dataset after ZOAU call.

Parameters

  • name (str) – Name of the dataset.

  • dataset_type (str) – Type of dataset (also known as dsorg).

    • Options: KSDS, ESDS, RRDS, LDS, SEQ, LARGE, PDS, PDSE.

Other Parameters

  • primary_space (str) – Space to allocate for the dataset.

    • Defaults to 5M.

    • Examples: 1M, 2MB, 1G, 2GB.

  • secondary_space (str) – Secondary (extent) space to allocate for the dataset. Defaults to 1/10 of primary space.

    • Examples: 1M, 2MB, 1G, 2GB.

  • block_size (int) – Block size of dataset.

    • Default varies on record format: FBA=32718, FB=32720, VBA=32743, VB/U=32760.
  • record_format (str) – Record format of dataset.

    • Options: FB (default), F, FBA, FBS, U, VB, VBA, VBS.

  • storage_class_name (str) – The storage class for an SMS-managed dataset.

    • Required for SMS-managed datasets that do not match an SMS-rule.

    • Not valid for datasets that are not SMS-managed.

    • Note that all non-linear VSAM datasets are SMS-managed.

  • data_class_name (str) – Data class name for dataset.

  • management_class_name (str) – The management class for an SMS-managed dataset.

    • Optional for SMS-managed datasets that do not match an SMS-rule.

    • Not valid for datasets that are not SMS-managed.

    • Note that all non-linear VSAM datasets are SMS-managed.

  • record_length (int) – Logical record length, expressed in bytes.

    • Defaults vary on format. F/FB/FBS=80, FBA=133, VB/VBA/VBS=137, U=0.

    • For variable datasets, the length must include the 4-byte prefix area.

  • key_length (int) – Mutally inclusive with key_offset. Required for KSDS datasets.

  • key_offset (int) – Mutally inclusive with key_length. Required for KSDS datasets.

  • volumes (str) – Comma separated list of volume serials. Offline volumes are not considered.

  • directory_blocks (int) – Directory blocks for PDS-type datasets. Default is 5.

Notes

  • Users can call _create for a raw create function that returns a ZOAUResponse object.


zoautil_py.datasets.delete(dataset, *args, **kwargs)


Description

  • Delete a z/OS dataset.

Returns

  • Return code from z/OS.

Return type

  • int

Parameters

  • dataset (str) – Dataset to delete.


zoautil_py.datasets.delete_members(pattern, *args, **kwargs)


Description

  • Delete members contained in a dataset.

Returns

  • Return code from z/OS action.

Return type

  • int

Parameters

  • pattern (str) – Dataset pattern to match.

Other Parameters

  • force (bool) – Force the deletion of a member that is in use.

Notes

  • Users can call _delete_members function to receive a ZOAU response.


zoautil_py.datasets.dunzip(file, high_level_qualifier, *args, **kwargs)


Description

  • Unzips a .dzp file.

Returns

  • Return code from z/OS.

Return type

  • int

Parameters

  • file (str) – HFS path or data set containing backup.

  • high_level_qualifier (str) – Single segment HLQ to unpack to.

Notes

  • Users can call _dunzip for a ZOAUResponse.


zoautil_py.datasets.dzip(file, target, *args, **kwargs)


Description

  • Zip datasets into an HFS file.

Returns

  • Return code from z/OS.

Return type

  • int

Parameters

  • file (str) – Fully qualified HFS file path or data set.

  • target (str) – Dataset name or dataset pattern.

Notes

  • Users can call _dzip for a ZOAUResponse.


zoautil_py.datasets.exists(dataset, *args, **kwargs)


Description

  • Check whether or not a dataset exists.

Returns

  • True or False.

Return type

  • bool

Raises

  • ZOAUException – See message for details.

Parameters

  • dataset (str) – The dataset to check for.


zoautil_py.datasets.find_member(member, concatentation, *args, **kwargs)


Description

  • Find dataset that contains member within a concatenation. Returns the first dataset that contains member.

Returns

  • The dataset name containing the member.

Return type

  • str

Parameters

  • member (str) – The member to find.

  • concatenation (str) – The concatenation of datasets (e.g “IBMUSER.A:IBMUSER.B”).

Examples

  • datasets.find_member(“PGM”, “:”.join(“IBMUSER.A”, “IBMUSER.B”))

Notes

  • User can call _find_member to receive a ZOAUResponse object.


zoautil_py.datasets.find_replace(dataset, find, replace, *args, **kwargs)


Description

  • Replace text within a dataset.

Returns

  • Return code from z/OS.

Return type

  • int

Parameters

  • dataset (str) – Dataset to act on.

  • find (str) – String to find.

  • replace (str) – String to replace find with.

Notes

  • Users can call _find_replace to receive ZOAUResponse object.


zoautil_py.datasets.get_hlq(**kwargs)


Description

  • Get the active TSO high-level qualifier (HLQ).

Returns

  • The active TSO high-level qualifier.

Return type

  • str


zoautil_py.datasets.lineinfile(dataset, line, state = True, regex = None, insert_after = None, insert_before = None, **kwargs)


Description

  • ZOAU dsed function to be used by zos_lineinfile Ansible module.

Returns

  • Return code from z/OS.

Return type

  • int

Parameters

  • dataset (str) – The target dataset or HFS file to modify to (e.g “IBMUSER.TEST.MOD”).

  • state (bool) – True (default): Insert or replace block. False: Remove block.

Other Parameters

  • line (str) – The line to insert/replace.

  • regex (str) – The regular expression to look for in every line of the dataset or HFS file.

    • For state = True, the pattern to replace if found. Only the last line found will be replaced.

    • For state = False, the pattern of the line(s) to remove.

    • If the regular expression is not matched, the line will be added to the dataset or HFS file in keeping with insert_after or insert_before settings.

  • insert_after (str) – Insert line after matching regex pattern.

    • The special value “EOF” will insert the line at the end of the target dataset or HFS file.

    • If regex is provided, insert_after is only honored if no match for regex is found.

    • insert_before will be ignored if provided.

  • insert_before (str) – Insert line before matching regex pattern.

    • The special value “BOF” will insert the line at the beginning of the target dataset or HFS file.

    • If regex is provided, insert_before is only honored if no match for regex is found.

    • insert_before will be ignored if insert_after is provided.

  • first_match (bool) – If set, insert_after and insert_before will work with the first line that matches the given regular expression.

  • encoding (str) – Encoding of the dataset.

  • lock (bool) – Obtain exclusive lock for the dataset.

  • force (bool) – Force open. Open dataset member in DISP=SHR mode. Default is DISP=OLD mode when False.

Notes

  • Use debug = True parameter for debug information in Json format.


zoautil_py.datasets.list_datasets(pattern, *args, **kwargs)


Description

  • Returns a listing of the datasets matching the supplied pattern.

Returns

  • List of Dataset objects, empty list if none found.

Return type

  • list

Raises

  • ZOAUException – See message for details.

Parameters

  • pattern (str) – Pattern to match, such as "IBMUSER.*".

  • volume (str) - Filter dataset by volume name.

Notes

  • Users can call _dls_call for a raw listing function that returns a ZOAUResponse object.


zoautil_py.datasets.list_datasets_by_volume(volume, **kwargs)


Description

  • List the contents of a given volume.

Returns

  • List of Dataset objects with only these attributes set: name, organization, record_format, record_length, volume

Return type

  • list

Parameters

  • volume (str) – Volume to list.


zoautil_py.datasets.list_dataset_names(pattern, migrated=False, **kwargs)


Description

  • Returns a listing of names matching the supplied pattern.

Returns

  • List of dataset names as strings, empty list if none found.

Return type

  • list

Raises

  • ZOAUException – See message for details.

Parameters

  • pattern (str) – Pattern to match, such as "IBMUSER.*".

  • migrated (bool) - Enables fetching information from migrated datasets, by default False.



zoautil_py.datasets.list_members(pattern, *args, **kwargs)


Description

  • Get a list of members from a dataset.

Returns

  • Members contained within dataset.

Return type

  • list[str]

Parameters

  • alias (bool) - Display dataset member aliases. Enabled by default.

  • pattern (str) – The dataset pattern to search.



zoautil_py.datasets.list_vsam_datasets(pattern, migrated=False, **kwargs)


Description

  • Returns a listing of Datasets matching the supplied pattern.

Returns

  • List of Dataset objects found. These objects will have only the name attribute set

Return type

  • list

Raises

  • ZOAUException – See message for details.

Parameters

  • pattern (str) – Pattern to match, such as "IBMUSER.*".

  • migrated (bool) - Display migrated datasets.



zoautil_py.datasets.move(source, target, *args, **kwargs)


Description

  • Move (rename) a dataset.

Returns

  • True if successful.

Return type

  • bool

  • ZOAUException – See message for details.

Parameters

  • source (str) – The source dataset.

  • target (str) – The target dataset.



zoautil_py.datasets.move_member(dataset, source, target, *args, **kwargs)


Description

  • Move (rename) a member.

Returns

  • Return code from z/OS.

Return type

  • int

Parameters

  • dataset (str) – The dataset that contains the member.

  • source (str) – The current member name.

  • target (str) – The desired member name.

Notes

  • Users can call _move_member to receive a ZOAUResponse object.


zoautil_py.datasets.read(dataset, *args, **kwargs)


Description

  • Get the string contents of a dataset.

Returns

  • Contents of the dataset.

Return type

  • str

Parameters

  • dataset (str) – The dataset to read.

Other Parameters

  • tail (int) – Read the last tail lines from the dataset.

  • from_line (int) – Returns lines from the given line.

Notes

  • Users can call _read for a raw function that returns ZOAUResponse. Files to be read must be text files, and in particular, files must not include embedded null values.


zoautil_py.datasets.read_as_bytes(dataset, *args, **kwargs)


Description

  • Get the contents of a dataset.

Returns

  • Contents of the dataset.

Return type

  • Bytes

Parameters

  • dataset (str) – The dataset to read.

  • tail (int) – Read the last tail lines from the dataset.

  • from_line (int) – Returns lines from the given line.

  • suppress_newlines (boolean) – Do not print a newline character between each line in the dataset output. Useful if the dataset has binary data.



zoautil_py.datasets.read_head(dataset, *args, **kwargs)


Description

  • A function to display the head content of a non-VSAM dataset. Gets the head content of a dataset.

Returns

  • Contents of the dataset.

Return type

  • str

Parameters

  • dataset (str) – The dataset to read.

Other Parameters

  • nlines (int) – Read the first nlines lines from the dataset.

Notes

  • Users can call _read_head for a raw function that returns ZOAUResponse.


zoautil_py.datasets.search(dataset, value, *args, **kwargs)


Description

  • Search a dataset using ISRSUPC.

Returns

  • str (ISRUPSC output) – The ISRUPSC output from the search command.

  • None - Content was not found in dataset.

Parameters

  • dataset (str) – The dataset to search.

  • value (str) – The string to search for in dataset.

Other Parameters

  • count_lines (bool) - Print only a count of matched lines in the dataset.

  • display_lines (bool) - Display the line number for each match.

  • lines (int) - Number of lines to be shown before and after each match.

  • ignore_case (bool) - Ignore case for search.



zoautil_py.datasets.tmp_name(high_level_qualifier = None, *args, **kwargs)


Description

  • Creates a temporary dataset name.

Returns

  • Temporary dataset name.

Return type

  • str

Parameters

  • high_level_qualifier (str) – The high-level qualifier of the temporary dataset name. Maximum of 17 characters long. Example: "HLQ.NAME".


zoautil_py.datasets.write(dataset, content, append = False, *args, **kwargs)


Description

  • Write content to a z/OS data set.

Returns

  • RC from z/OS.

Return type

  • int

Parameters

  • dataset (str) – Dataset to write content to.

  • content (str) – String to write to the dataset.

  • append (bool) – Append-mode will add the content to the end of the dataset.

Other Parameters

  • force (bool) – Force open. Open dataset member in DISP=SHR mode. Default is DISP=OLD mode when False.

Notes

  • Users can call _write for a raw function that returns ZOAUResponse.