Migrating to ZOAU v1.4 from v1.3.x

Changes to the gdg module

The gdg module includes changes to object parameters and how to address a given GenerationDataGroupView generation.

GenerationDataGroupView.generations

Generation data group (GDG) generations is now a Python property. Every time the property is addressed a new dataset fetch operation is performed. This ensures the generation information is up to date.

The property generations now returns a custom RelativeList object. It is inherited from a regular list and the index resolves as the GDG relative naming scheme.

Each generation in the list is a Dataset object. It can be accessed and used in the same way as any other Dataset instance.

Addressing:

Due to the nature of z/OS GDG stepping, generations cannot be addressed with a relative name. All operations either require the full resolved name (ZOAU.GDG1.G0003V00) or the object oriented version (gdg_1.generations[-2]).

Examples:

# Create GDG
gdg1 = gdgs.create(name="ZOAU.GDG1", limit=10)

# Generate the next generations, step by 1.
gdg1.generate(1, dataset_type="SEQ")
gdg1.generate(1, dataset_type="SEQ")
gdg1.generate(1, dataset_type="SEQ")

# Print the generations in a GDG
print(gdg1.generations)

Result:

[zoautil_py.datasets.Dataset object with name:ZOAU.GDG1.G0001V00,
 zoautil_py.datasets.Dataset object with name:ZOAU.GDG1.G0002V00,
 zoautil_py.datasets.Dataset object with name:ZOAU.GDG1.G0003V00]

 # Reference by relative index
print(gdg1.generations[0]).name
print(gdg1.generations[-1].name)
print(gdg1.generations[-2].name)

Result

# 0 - Latest generation.
ZOAU.GDG1.G0003V00
# -1 - Second most recent generation.
ZOAU.GDG1.G0002V00
# -2 - third most recent generation.
ZOAU.GDG1.G0001V00

GenerationDataGRoupView.generate()

# Create GDG
gdg1 = gdgs.create(name="ZOAU.XXXX.ABC", limit=10)
# Generate the next GDG, step by 1.
gdg_gen  = gdg1.generate(1, dataset_type="SEQ")

Changes to the datasets module

The datasets module includes changes to arguments, attributes, functions, and parameters. Migration-related changes are described as follows. For a full description of the datasets module, see Python API datasets module.

Changes to dataset listing operations

In ZOAU v1.4, the function datasets.list_members() separates member names and alias names into individual elements of the returned list. In previous releases, each member name was followed by all of its aliases in a single string.

Changes to dataset read and write operations

Read and write operations for non-VSAM datasets are reworked to use the zoau_io python module. These functions now directly use the file APIs from the z/OS Language Environment.

This new architecture grants more control over data set operations. It also relaxes constraints on maximum write string length and multi-line string writes.

Read as bytes

The datasets.read_as_bytes() API to read the contents of a dataset as raw bytes has changed. This requires a migration action.

• Returns a list of bytes.
• Each entry of the list corresponds to a dataset record.
• Byte content includes non-printable characters and null bytes.

Both static and object-oriented approaches support the following arguments:

• records, int = 0: Number of records to read.
• offset, int = 0: Starting record
• tail: bool = false: Starts reading from the end if true.

The static function requires dataset_name to be provided.

Static function:

datasets.read_as_bytes(dataset_name: str,
                      records: int = 0,
                      offset: int = 0,
                      tail: bool = False) -> list[bytes]

Object method:

Dataset.read_as_bytes(number_of_records: int = 0,
                      offset: int = 0,
                      tail: bool = False) -> list[bytes]

Read text content

The operations to read text content from a given dataset into a python text sequence type (string) have been rewritten. The new implementation is based on the read_as_bytes() method. This requires a migration action.

Key changes:

• The character encoding codepage can be provided through the encoding parameter. Content read from the dataset will be decoded with the provided codepage and converted into a UTF-8 encoded Python text sequence.

• The operations support multiple serialization methods, set by the serialize argument. For more information, see the serialization section later in this guide.

• Null (0x0) bytes are converted into blank spaces encoded in the provided codepage.

Both static and object-oriented approaches support the following arguments:

• lines, int = 0: Amount of lines to read.
• offset, int = 0: Index of starting record.
• tail: bool = false: Starts reading from the end if true.
• serialize, str = "record": Serialization method used to build the string.
• next_record_char, str = "\n": Character inserted to delimit records. (Only with serialization by record.)

Static vs. Object

• The static function requires dataset_name to be provided.
• The object method supports the argument member_name.

Static function:

datasets.read(dataset_name: str,
     encoding: str = "cp1047",
     lines: int = 0,
     offset: int = 0,
     tail: bool = False,
     serialize: str = "record",
     next_record_char: str = "\n" ) -> str:

Object method:

Dataset.read(member_name: str = "",
             encoding: Union[str, None] = None,
             lines: int = 0,
             offset: int = 0,
             tail: bool = False,
             serialize: str = "record",
             next_record_char: str = "\n" ) -> str:

Examples:

Reading content encoded in EBCDIC, serialized by record. Default behavior.

# Static
content_string = datasets.read(dataset_name="ZOAU.DST.A1")

# Object Oriented

content_string = dataset_1.read()

Reading content encoded in UTF-8, serialized by byte. UTF-8 support requires byte serialization.

# Static
content_string = datasets.read(dataset_name="ZOAU.DST.A1",
                                encoding="utf_8", serialize="byte")

# Object Oriented
content_string = dataset_1.read(encoding="utf_8", serialize="byte")

Reading content encoded in EBCDIC, serialized by record, from a PDSE member within a GDG generation.

# Static
content_string = datasets.read(dataset_name="ZOAU.GDG1.G0001V00(MEM1)")

# Object Oriented
# from GenerationDataGroupView object gdg_1

content_string = gdg_1.generations[-2].read(member_name="MEM1")

Write text content

The datasets.write() function has been completely reimplemented and requires migration action.

Key changes:

• The character encoding codepage can be provided through the encoding argument. Content read from the dataset will be encoded with it and will be written as raw bytes.
• Implicit creation through writing is no longer possible.
• The operations now support multiple serialization methods, set by the serialize argument. For more information see the serialization section later in this guide.
• For datasets with fixed record length, a fill character can be set. Defaults to the encoding's whitespace char.
• When serialized by record an additional record length validation is performed to protect the dataset's content before writing. It can be disabled by the validate_record_length argument.

Both static and object-oriented approaches support the following arguments:

• content, str: Content to write.
• append, bool = False: Starting record
• encoding: str = "cp1047": Content's target encoding
• serialize: str = "record": Serialization method
• next_record_char = str = "\n": Character delimiting records. (Exclusive to serialize by record)
• max_record_length: int = 0: Overrides max record length auto discovery.
• fill_char: str = " ": Character to fill record with. (Exclusive to fixed record datasets)
• validate_record_length:bool = True: If false, disables text record length validation. (Exclusive to serialize by record)

Static function:

datasets.write(dataset_name: str,
            content: str,
            append: bool = False,
            encoding: str = "cp1047",
            serialize: str = "record",
            next_record_char: str = "\n",
            max_record_length: int = 0,
            fill_char: str = " ",
            validate_record_length: bool = True) -> None:

Object method:

Dataset.write(content: str,
            member_name: str = "",
            append: bool = False,
            encoding: Union[str, None] = None,
            serialize: Union[str, None] = None,
            next_record_char: Union[str, None] = None,
            max_record_length: Union[int, None] = None,
            fill_char: str = " ",
            validate_record_length: bool = True,
    ) -> None:

Static vs. Object

• The static function requires dataset_name to be provided.
• The object method supports the argument member_name.

Examples:

Writing content encoded in EBCDIC, serialized by record. Default behavior.

# Static
#given a str object named runtime_content

datasets.write(dataset_name="ZOAU.DST.A1", content=runtime_content)

# Object Oriented

dataset_1.write(content=runtime_content)

Writing content encoded in UTF-8, serialized by byte. UTF-8 support requires byte serialization.

# Static
datasets.write(dataset_name="ZOAU.DST.A1", content=runtime_content,
                encoding="utf_8", serialize="byte")

# Object Oriented
dataset_1.write(content=runtime_content,
                encoding="utf_8", serialize="byte")

Write content encoded in EBCDIC, serialized by record, to a PDSE member within a GDG generation.

# Static
datasets.write(dataset_name="ZOAU.GDG1.G0001V00(MEM1)", content=runtime_content)

# Object Oriented
# Using GenerationDataGroupView object gdg_1

content_string = gdg_1.generations[-2].write(member_name="MEM1", content=runtime_content)

Serialization

Two serialization methods are supported with ZOAU 1.4.0, Record and Byte. Both describe an approach to translating the content of a dataset into a python text, as well as the reverse process.

Record:

The content is matched 1:1 record to line.

When read into a string, a next record character is inserted between records. This character can be set with the next_record_char argument. Defaults to the newline character provided by the given text encoding (typically \n).

When written to a dataset the next record character (next_record_char) is used as a delimiter for each record. The character is not written to the dataset.

Writing with additional newlines within the dataset content generates additional records.

Byte:

The content is sliced byte-wise to fill the record up to the maximum record length of the dataset. This is required by multi-byte encoding character pages like UTF-8. Record steps are ignored and any newlines in the content remain unchanged.

Note that content serialized by the Byte method may not be readable by other programs gwithout first being deserialized.