DDM Support - Distributed Data Management Overview

Resolving The Problem

DDM allows application programs or users to access data files that reside on remote systems, and also allows remote systems to access data files on the local IBM OS/400 or IBM i5/OS system. Any system that supports the DDM architecture as a source system can access data (if authorized to do so) on any other system to which it is attached. The attached system must support DDM as a target system. However, the source and target systems must support compatible subsets and levels of the DDM architecture.

When DDM is in use, neither the application program nor the program user needs to know if the file exists locally or on a remote system. DDM handles remote file processing in essentially the same way as local file processing is handled on the local system, and the application program normally does not receive any indication of where the requested file is located. (However, in error conditions, messages are returned to the user that indicate, when necessary, that a remote system was accessed.) Informational messages about the use of target system files are included in the source system's job log.

System Compatibility

DDM can be used to communicate between systems that are architecturally different. For example, although the architectures of the OS/400 or i5/OS system and IBM System/36 are different, these systems can use DDM to access files in each other's database. To successfully communicate with each other, each system must have an implementation of DDM that is compatible with Level 2.0 or below of the IBM DDM architecture. Also, each type of system may use all or only part of the IBM DDM architecture or may have extensions to the architecture.

If you are communicating with any non-OS/400 systems, you must consider the level of DDM support provided by those systems for such things as unique security considerations.

Note: The OS/400 and i5/OS system supports only advanced peer-to-peer networking (APPN) to allow DDM access to systems not directly linked to the local system.

DDM Functions

Types of DDM Functions That Can Be Done on a Target System

Operations that can be done on files at target systems:

1.	Allocating, opening, or closing one or more files
2.	Reading, writing, changing, or deleting records in a file

File and nonfile operations, normally specified in CL programs or commands, can be done on files at the target systems:

1.	Copying the contents of a file.
2.	Performing operations on physical or logical file members (such as adding, clearing, or removing members), but only if the target is an OS/400 or i5/OS system or IBM® System/38™.
3.	Accessing remote files for nondata purposes, such as: - Displaying information about one or more files, using commands such as: Display File Description (DSPFD) Display File Field Description (DSPFFD) These commands can display the file attributes of the DDM file on the source system or the file or field attributes of the remote file on the target system. - Controlling the locking of files on the target system. Allocate Object (ALCOBJ) Deallocate Object (DLCOBJ) commands. - Deleting, renaming, creating, and changing files. Delete File (DLTF) Rename Object (RNMOBJ) Create Physical File (CRTPF) Create Source Physical File (CRTSRCPF) Create Logical File (CRTLF) Change Physical File (CHGPF) Change Logical File (CHGLF) Change Source Physical File (CHGSRCPF)
4.	Accessing remote systems for nondata purposes: - Sending a CL command to the target system (an OS/400 or i5/OS system and a System/38 only) so it can be run on the target system. Submit Remote Command (SBMRMTCMD) The SBMRMTCMD command is the method used to move, save, or restore files on a target system. For example, a Move Object (MOVOBJ) command might be sent to move a database file on the target system.

Basic DDM Concepts

The following are the primary parts used in DDM:

o	Source DDM
o	Target DDM
o	DDM file

Because remote file processing is much like local file processing, these topics should provide sufficient conceptual information for most users of DDM.

Source DDM (SDDM): Support on the source (or local) OS/400 or i5/OS system that is started within a source job to do DDM functions. Translates requests for remote file access from source system application programs into DDM requests that are routed to the target system for processing. Establishes and manages a DDM conversation with the target system that has the desired remote file.

Target DDM (TDDM): Job that is started on the target (or remote) system as a result of an incoming DDM request and ends when the associated DDM conversation ends. Translates DDM requests for remote file access into data management requests on the target system. Handles the return of the information that is to be sent to the source system.

DDM file: System object with type *FILE that exists on the source system to identify a remote file. Combines the characteristics of a device file and a database file.

As a device file: For an SNA connection, refers to a remote location name, local location name, device name, mode, and a remote network ID to identify a remote system. For a TCP/IP connection, refers to the TCP/IP address of the remote system.

As a database file: Appears to the application program as a database file and serves as the access device between a source system program and a remote file.

Source DDM (SDDM)

When an application program first attempts to access a remote file: A search for the requested DDM file is done on the source system.

Note: No library name, the library list for the job is searched for the specified file. When the file is found, the system accesses the file, determines that it is a DDM file, and starts the SDDM.

When the SDDM is started: Checks to see if a DDM conversation is already active between the source job starting the SDDM and the target system identified by the remote location and mode values in the DDM file. If a conversation exists that can be used, it is used. Otherwise, a program start request is issued to the appropriate target system to start a TDDM (a target job) on the target system to establish a DDM conversation between the SDDM and TDDM.

Notes:

1.	Parameters that were automatically created from information in the DDM file about the remote file are passed when the remote system sends a program start request.
2.	A DDM connection using TCP/IP does not use "sessions" or program start requests. A port pair is opened between the two systems in the same manner as most other TCP/IP applications. The target system must have the DDM server running in order to become the "server" for the connection, while the source system acts as the "client".

After the TDDM is started: SDDM can forward each program request to the target job for processing.

Example: Input/output (I/O) operations are to be done on a remote file, the program opens the file, and then issues the desired operation requests; the SDDM forwards the open request and the TDDM opens the remote file. Then the SDDM forwards each file operation request to the TDDM, and both of them handle the interchange of data between the application program and the remote file. When a DDM function is being processed, the requesting program waits for the function to be completed and the results to be received, the same as it does for local file operations.

Target DDM (TDDM)

The TDDM is started when the remote system sends a program start request: Started as a batch job on the target system. After the TDDM is started and a DDM conversation is established, the TDDM waits for a request (such as a file open or read operation, or a nonfile-related operation) to be sent by the SDDM.

When the TDDM receives a request to access an object: TDDM searches for the requested object.

When the requested object is found: TDDM passes the first operation requested to database or folder management on the target system, which performs the operation on the object. When the operation is completed, database or folder management services returns the results of the operation to the TDDM, which passes it to the SDDM. The SDDM passes the results and any accompanying data (such as records requested on a read operation) to the application program.

Notes:

1.	These actions are repeated for each subsequent I/O operation request received, until the object is closed.
2.	If an operation does not complete successfully, the SDDM returns an error message to the program, providing information about the error.
3.	The TDDM and the target job remain active until the DDM conversation is ended by the source system job that started it.

DDM File

A DDM file is a file on the source system that contains the information needed to access a data file on a target system.

Note: It is not a data file that can be accessed by a program for database operations. Instead, when a source system program specifies the name of a DDM file, the file information is used by DDM to locate the remote file whose data is to be accessed.

OS/400 or i5/OS DDM file information is based on locations: The remote location, which is where the remote file is located, is specified for an SNA connection using the following:

o	Remote location name (RMTLOCNAME)
o	Remote network ID (RMTNETID)
o	Local location name (LCLLOCNAME)
o	Device name (DEV)
o	Mode (MODE)

For a TCP/IP connection, the TCP/IP address or TCP/IP host domain name of the target system is used, and the above parameters are ignored.

Note: In the System/38 environment, only the device name and mode can be specified. In this case, the remote location is taken from the device description.

DDM File Description

Each OS/400 or i5/OS DDM file is created by the Create DDM File (CRTDDMF) command or the Work with DDM Files (WRKDDMF) command. The DDM file is stored as a file object in a library, the same as any other file or object.

Each DDM file contains the following information:

Create DDM File (CRTDDMF)                                                                                       Type choices, press Enter.                                                                                                                 DDM file . . . . . . . . . . . .                 Name                   Library  . . . . . . . . . . .     *CURLIB     Name, *CURLIB       Remote file:                                                           File . . . . . . . . . . . . .                 Name, *NONSTD           Library  . . . . . . . . . .                 Name, *LIBL, *CURLIB   Nonstandard file 'name'  . . .                                                                                                            Remote location:    Name or address  . . . . . . .                 Name, *RDB    Type . . . . . . . . . . . . .                 *SNA, *IP  Relational Database  . . . . . .                 Text 'description' . . . . . . .   *BLANK                                                        Additional Parameters                             Device:                                                                       APPC device description  . . .   *LOC          Name, *LOC                 Local location . . . . . . . . .   *LOC          Name, *LOC, *NETATR         Mode . . . . . . . . . . . . . .   *NETATR       Name, *NETATR               Remote network identifier  . . .   *LOC          Name, *LOC, *NETATR, *NONE  Port number  . . . . . . . . . .   *DRDA         *DRDA, 1-65535 Access Method:                                                                 Remote file attribute  . . . .   *RMTFILE      *RMTFILE, *COMBINED...       Local access method  . . . . .                 *BOTH, *RANDOM, *SEQUENTIAL Share open data path . . . . . .   *NO           *NO, *YES                   Protected conversation . . . . .   *NO           *NO, *YES                   Record format level check  . . .   *RMTFILE      *RMTFILE, *NO               Authority  . . . . . . . . . . .   *LIBCRTAUT    Name, *LIBCRTAUT, *ALL...   Replace file . . . . . . . . . .   *YES          *YES, *NO

DDM File Value and Description of Values:

DDM file name: Name of the DDM file on the source system that is used to identify a specific remote file.

Remote file name The name by which the file is known on the target system. (For a target System/36, this is the file label of the remote file.)

*NONSTD - For target systems that allow naming conventions other than those used by the OS/400 or i5/OS system, System/38 simple file name and System/36, or when specifying a file member name for System/38 and OS/400 or i5/OS system. Up to 255 characters for the name of the remote file being accessed. The name must be in the form required by the target system. The name must always be enclosed in apostrophes, and it may contain lowercase letters, blanks, periods, or other special characters.

Remote location name: The remote location where the remote file exists. Provides the data link to the target system (remote location) via APPN/APPC, over which a DDM conversation is established when this DDM file is accessed. If TCP/IP is used, then this can be the TCP/IP address or TCP/IP host domain name of the target system.

Note: In the System/38 environment, only the device name and mode can be specified. In this case, the remote location is taken from the device description.

Type: Specifies whether this DDM file is using SNA or TCP/IP.

Device: The device on the source system used to communicate with the remote location (used only for APPC)

Note: (APPC only) Should be specified only if there is more than one APPC device associated with the same remote location and local location and you wish to select a specific device.

Local location name: The name the target system knows the source system by (used only for APPC).

Note: The source system can consist of more than one local location.

Mode: The mode to be used to communicate between the local location and remote location (used only for APPC).

Remote network ID: Further qualifies the remote location name (used only for APPC).

Note: Two locations with the same remote location name but different remote network IDs are viewed as two separate locations.

Port Number: Specifies the TCP/IP port that is used at the remote location to communicate with the system on which the remote file is located. If *DRDA is used, then port 446 is used. This is the port on which the OS/400 or i5/OS system DDM TCP/IP server listens. You may also specify a specific port number between 1 and 65535 (used only for TCP/IP).

For additional information on other parameters, refer to the DDM Reference manual or the online help text.

OS/400 or i5/OS System as the Source System

When an application program or user in a source system job first refers to a DDM file, several actions occur as part of processing the request on the source system. All of these actions, as well as those required on the target system, must complete successfully before any operations (file or nonfile) requested by the source program can be done.

When the DDM file is referred to:

Request to open a file: Its information is used simultaneously to create an open data path (ODP) on the source system and to start the SDDM support, which runs within the same job as the source program. The SDDM also uses the information to:

o	Convert the source system request into a DDM request.
o	Communicate with the appropriate target
o	Establish a DDM conversation to be used for the source job.

The communications portion of DDM: Establishes a communications path with the target system using the Advanced Program-to-Program Communications (APPC) support or Transmission Control Protocol / Internet Protocol (TCP/IP). The target system is identified via the remote location information (remote location name, device, local location name, mode, remote network ID or TCP/IP address or host domain name) specified in the DDM file, and the target file is identified by the remote file name. Other parts of the remote location information are stored by the SDDM.

o	Transaction program name
o	User ID
o	Activation group number

Note: The above parameters refer to an APPC connection, not a TCP/IP connection.

Using the remote location information, the TDDM is started on the target system, and a DDM conversation is established when the remote system receives the program start request (for SNA) or when the target DDM server successfully establishes an open port pair to be used for DDM over TCP/IP. The conversation is established the first time the remote file is accessed, but only if a conversation using the same remote location values for that target system does not already exist for the source job (for SNA). TCP/IP will allow separate port pairs to be opened independent of each other.

After the DDM conversation is established: The SDDM (which can be used by multiple programs and multiple DDM files in the same source job) sends the DDM architecture command to the TDDM, for file-related requests. This command describes the file operation to be done and contains the name of the remote file to be accessed.

Note: For nonfile-related requests, such as Submit Remote Command, the remote file name is not sent to the TDDM; the remote file name is ignored.

The SDDM converts each program request for a file open or input/output operation (received via the DDM file and ODP) into an equivalent DDM command request and then sends it to the target system.

After each request is handled by the target job: The DDM response from the target system is returned via APPC, converted by the SDDM into the appropriate form, and passed back to the user. The response may include data (if data was requested) or an indication of status (for other types of file access). The source program waits until the function completes and the results are received.

After the first DDM file that was opened in the job is closed: The DDM conversation that it used is normally kept active. This allows the same program or another program in the job to use the same conversation when opening another DDM file, or doing other DDM-related operations. This saves the time and resources required to establish a new conversation every time a new DDM file that uses the same remote location information is used in that job.

When a DDM file is closed: The DDM conversation remains active, but nothing happens in the conversation until the SDDM processes the next DDM-related request from a program. The conversation can be dropped. This can occur if the DDMCNV job attribute's default value of *KEEP is changed to *DROP using the Change Job (CHGJOB) command, or if the Reclaim DDM Conversations (RCLDDMCNV) command or Reclaim Resources (RCLRSC) command is used while the job is active.

OS/400 or i5/OS System as the Target System

The OS/400 or i5/OS target DDM (or TDDM) is actually a job that runs a DDM-related target system program; it is started when the source system sends a program start request (a SDDM). For source OS/400 or i5/OS systems, the program start request is started on the source system using information contained in the IBM-supplied intersystem communications function (ICF) file for DDM.

For SNA connections, the attributes of the target job are determined by the values specified on the Add Communications Entry (ADDCMNE) command, which is used on the target system to add a communications entry to the subsystem description used for the job. This command identifies the device description, the job description (including the library list for the target job), and the default user profile to be used by the subsystem.

Note: For a Client Access connection, the routing entry in the QIWS subsystem for DDM (CMPVAL ('DDM')), along with the device description the personal computer is connected to, is used to obtain the attributes of the target job.

After it is started, the TDDM does the following:

For database files:

o	Handles communications with the source system via a DDM conversation established over an APPC data link, a Client Access data link, or a TCP/IP link.
o	Converts the access requests from the source system into the equivalent OS/400 or i5/OS functions and runs them on the target system. Once the target object is located, the target system-created ODP and target database management services are used to access the object for whatever operation is requested. The TDDM can, for example, pass requests that open the object and then do requested I/O operations to the objects.
o	Includes OS/400, i5/OS, or System/38 extensions to the DDM Level 2.0 architecture for requests received from the source system (if the source is an OS/400 or i5/OS system or a System/38), which allow most OS/400 or i5/OS functions that operate on local systems to also work on remote OS/400 or i5/OS systems. For example, it might receive a SBMRMTCMD command from the source system (an OS/400 or i5/OS system or a System/38) to do a nonfile-related operation, such as using the CL command Replace Library List (RPLLIBL) to replace the library list within the current target job.
o	Converts target OS/400 or i5/OS responses to the equivalent DDM responses and sends them back to the source system. When the source system is an OS/400 or i5/OS system or System/38, the actual OS/400, i5/OS, or System/38 messages are sent back to the source system.

For folder management services objects: Converts the DDM stream and directory access requests into the equivalent OS/400 or i5/OS folder management services functions and then runs them on the target system. The following commands are supported:

o	Change Current Directory (CHGCD)
o	Change File Attributes (CHGFAT)
o	Close Directory (CLSDRC)
o	Close Document (CLOSE)
o	Copy File (CPYFIL)
o	Create Directory (CRTDRC)
o	Create Stream File (CRTSTRF)
o	Delete Directory (DELDRC)
o	Delete File (DELFIL)
o	Force Buffer (FRCBFF)
o	Get Data Stream (GETSTR)
o	Get Directory Entry (GETDRCEN)
o	List File Attributes (LSTFAT)
o	Load Stream File (LODSTRF)
o	Lock Data Stream (LCKSTR)
o	Open Directory (OPNDRC)
o	Open Document (OPEN)
o	Put Data Stream (PUTSTR)
o	Query Current Directory (QRYCD)
o	Query Space Available (QRYSPC)
o	Rename Directory (RNMDRC)
o	Rename File (RNMFIL)
o	Unload Stream File (ULDSTRF)
o	Unlock Data Stream (UNLSTR)

The TDDM runs as a separate batch job, just as any other user APPC target application or Client Access target application. A new TDDM, using additional target system resources, is started for each distinct source system program start request received by the target system. There is one target job for each DDM conversation. Each TDDM can handle access requests for multiple files in the DDM conversation.

Note: The subsystem, user profiles, and system resources to be used by the TDDM are defined the same as they are for other types of jobs.

Security Requirements

You can prevent intentional and unintentional access to the data resources of a system by the DDM user. Access to data in the DDM environment can be limited--or prevented altogether--by a network attribute DDM Request Access. This attribute allows the system (as a target system) to prevent all remote access; or it allows the system to control file access by using standard authority to files and, further, by using an optional user exit program to restrict the types of operations allowed on the files for particular users.

Network attribute value: DDM Request Access

Possible values include the following:

*REJECT: The system does not allow any DDM requests from remote systems. However, this system may use DDM to access files on remote systems.

*OBJAUT: All requests are allowed and controlled by the object authorization on the system. The name of a user-written validation program that is called each time a DDM request is made from a remote system. This program indicates to DDM whether the request should proceed or be ended. System security still applies. See the Distributed Data Management manual for more information about this program.

To provide adequate security, you may need to set up additional user profiles on the target system, one for each source system user who can have access to one or more target system files. Or, a default user profile should be provided for multiple source system users. The default user profile is determined by the communications entry used in the subsystem in which the target jobs are run.

For further security issues with DDM, refer to Chapter 4, Security in the Distributed Data Management manual.

Submit Remote CL Commands

Another use of the DDM file is to submit control language (CL) commands to the target system to run on that system. In this case, the remote file normally associated with the DDM file is ignored.

Submit Remote Command

The Submit Remote Command (SBMRMTCMD) command submits a command using DDM to run on the target system. The remote location information in the DDM file is used to determine the communications line to be used, and thus, indirectly identifies the target system that is to receive the submitted command.

Use the SBMRMTCMD command to send commands to any of the following target systems:

o	IBM System i products
o	System/38
o	System/36

Any system that supports the Submit System Command (SBMSYSCMD) DDM command

The primary purpose of this command is to allow a user or program using the source system to perform file management operations and file authorization activities on files located on a target system. The user must have the proper authority for the target system objects that the command is to operate on. The following actions are examples of what can be performed on remote files using the SBMRMTCMD command:

o	Create or delete device files
o	Grant or revoke object authority to remote files
o	Verify files or other objects
o	Save or restore files or other objects

Although the command can be used to do many things with files or objects, some are not as useful as others.

Example: Could use this command to display the file descriptions or field attributes of remote files, or to dump files or other objects, but the output remains at the target system. Another way to display remote file descriptions and field attributes at the source system is to use the Display File Description (DSPFD) and Display File Field Description (DSPFFD) commands. Specify the SYSTEM(*RMT) parameter and the names of the DDM files associated with the remote files. This returns the information you desire directly to the local system.

A secondary purpose of this command is to allow a user to perform nonfile operations (such as creating a message queue) or to submit user-written commands to run on the target system. The CMD parameter allows you to specify a character string of up to 2000 characters that represents a command to be run on the target system.

Problem Analysis on the Remote System

Some functions that involve a target system may take a relatively long period of time to complete. In these situations, the target system may not appear to be functioning when it is actually waiting for a reply. Any messages created on the target system (such as file full) are sent to the system operator's message queue on the target system. (All DDM-related messages are logged in the target system's job log.) In most cases, a message similar to the one sent to the target system operator is also sent to the source system (with a different message number), but only after the target system operator has replied to the message.

If no job log is found on the target system, the Submit Remote Command (SBMRMTCMD) command can be used to send a Change Job Description (CHGJOBD) command to the target system to change the message logging level.

Another consideration is when end-of-file delay is being used between two OS/400 or i5/OS systems. When this function is being used, canceling the job on the source system does not cancel the job on the target system. Or, if the source system job is canceled while the target job is performing some function, the target job is not canceled.

In some situations, it may be necessary for a user on either the source or target system to call the other location or use
pass-through to determine the status of the job on that end and to reply to any messages waiting for a response.

Reclaim DDM Conversations RCLDDMCNV

The Reclaim DDM Conversations (RCLDDMCNV) command is used to reclaim all DDM source system conversations that are not currently being used by a source job. The conversations are reclaimed even if the value of the job's DDMCNV attribute is *KEEP, or if the command is entered within an activation group. The command allows the user to reclaim unused DDM conversations without closing all open files or doing any of the other functions performed by the Reclaim Resources (RCLRSC) command.

The RCLDDMCNV command applies only to the DDM conversations for the job on the source system in which the command is entered. For each DDM conversation used by the source job, there is an associated job on the target system; the target job ends automatically when the associated DDM conversation ends.

Although this command applies to all DDM conversations used by a job, using it does not mean that all of them will be reclaimed. A conversation is reclaimed only if it is not being actively used.

Increasing the DDM Job Priority

The target DDM job priority is controlled by the job class specified by the associated subsystem description routing entry. The following routing entry is normally the one used for all target (program start request) jobs:

ADDRTGE ... PGM(*RTGDTA) ... CMPVAL(PGMEVOKE 29)

The subsystems QBASE and QCMN, which are shipped with the OS/400 or i5/OS system, have this routing entry.

To have target DDM jobs in a subsystem run at a different priority than other APPC target jobs in the same subsystem, insert the following routing entry with the appropriate sequence number:

ADDRTGE SBSD(xxx) SEQNBR(nnn) CMPVAL(QCNTEDDM 37) PGM(*RTGDTA) CLS(uuu)

The class uuu is used to determine target job priority.