Synchronous IBM Spectrum Symphony Python client and service tutorial

This tutorial guides you through the process of building, packaging, deploying, and running the hello grid sample client and service. It also walks you through the sample client application code. You learn the minimum amount of code that you need to create a client.

1. Package the sample service.

   To deploy the service, you first need to package it.

   1. Change to the directory where the service is located:

      cd $SOAM_HOME/version/samples/Python/SampleApp/Service

   2. Create and deploy the service package:

      ./makepackage.sh

2. Check the list of deployed services with the soamdeploy command:

   soamdeploy view SampleServicePython

   You should be able to see the package you just deployed.

3. If you are using a Python 3.8.0 or 3.9.0, or 3.10.0 environment, set the PYTHONPATH environment variable on the service side by updating the application profile:
   • On Windows, use a semicolon (;) as the separator, but do not include a semicolon at the end of the PYTHONPATH value as that impacts the execution of soamapiversion.py. For example:

     Updating the application profile for IBM Spectrum Symphony Developer Edition

     <Servicedescription="Python Sample Service"name="SampleServicePython"packageName="SampleServicePython">
             <osTypes>
                 <osType name="all"startCmd="python ${SOAM_DEPLOY_DIR}/MyServiceContainer.py"workDir="${SOAM_HOME}/work">    
                  <env name="PYTHONPATH">${SOAM_DEPLOY_DIR};${SOAM_HOME}/${VERSION_NUM}/${EGO_MACHINE_TYPE}/lib64/pythonapi_3.8.0;${SOAM_HOME}/${VERSION_NUM}/${EGO_MACHINE_TYPE}/lib64</env>
                  <env name="PATH">c:/python3.8.0</env>
             </osType>
     </Service>

     Updating the application profile for IBM Spectrum Symphony

     <Servicedescription="Python Sample Service"name="SampleServicePython"packageName="SampleServicePython">
             <osTypes>
                 <osType name="all"startCmd="python ${SOAM_DEPLOY_DIR}/MyServiceContainer.py"workDir="${SOAM_HOME}/work">    
                  <env name="PYTHONPATH">${SOAM_DEPLOY_DIR};${SOAM_HOME}/${VERSION_NUM}/${EGO_MACHINE_TYPE}/lib64/pythonapi_3.8.0;${SOAM_HOME}/${VERSION_NUM}/${EGO_MACHINE_TYPE}/lib64;${SOAM_HOME}/../ego_version/lib</env>
                  <env name="PATH">c:/python3.8.0</env>
             </osType>
     </Service>

   • On Linux®, use a colon (:) as the separator, but do not include a colon at the end of the PYTHONPATH value as that impacts the execution of soamapiversion.py. For example:

     Updating the application profile for IBM Spectrum Symphony Developer Edition or IBM Spectrum Symphony

     <Service description="Python Sample Service" name="SampleServicePython" packageName="SampleServicePython">
             <osTypes>
                 <osType name="all" startCmd="python ${SOAM_DEPLOY_DIR}/MyServiceContainer.py" workDir="${SOAM_HOME}/work">
     		<env name="PYTHONPATH">${SOAM_DEPLOY_DIR}:${LD_LIBRARY_PATH}:${SOAM_HOME}/${VERSION_NUM}/${EGO_MACHINE_TYPE}/lib64/pythonapi_3.8.0:${SOAM_HOME}/${VERSION_NUM}/${EGO_MACHINE_TYPE}/lib64</env>
                  <env name="PATH">/usr/local/python38</env>
                 </osType>
             </osTypes>
     </Service>

4. Navigate up one directory and register the application with the soamreg command.
   Note: If your Python binary is not in the default location (/usr/local/bin), in the PATH environment variable in the application profile SampleAppPython.xml, provide the directory that contains the Python binary (for example, <env name="PATH">/usr/bin</env>).

   To register the application profile, run:

   cd ../; soamreg SampleAppPython.xml

   The application is registered and enabled.

5. If you are using a Python 3.8.0 or 3.9.0, or 3.10.0 environment, set the PYTHONPATH environment variable on the client side:
   • On Windows, use a semicolon (;) as the separator, but do not include a semicolon at the end of the PYTHONPATH value as that impacts the execution of soamapiversion.py. For example:

     Setting the environment variable for IBM Spectrum Symphony Developer Edition to run as the client

     set PYTHONPATH=%SOAM_HOME%\soam_version\w2k3_x64-vc7-psdk\lib64\pythonapi_3.8.0;%SOAM_HOME%\soam_version\w2k3_x64-vc7-psdk\lib64;%SOAM_HOME%\soam_version\w2k3_x64-vc7-psdk\ego_lib

     Setting the environment variable for IBM Spectrum Symphony to run as the client

     set PYTHONPATH=%SOAM_HOME%\soam_version\w2k3_x64-vc7-psdk\lib64\pythonapi_3.8.0;%SOAM_HOME%\soam_version\w2k3_x64-vc7-psdk\lib64;%SOAM_HOME%\..\ego_version\lib

     Setting the environment variable for the IBM Spectrum Symphony client to run workload

     set PYTHONPATH=%SOAM_HOME%\lib64\pythonapi_3.8.0;%SOAM_HOME%\lib64

   • On Linux, use a colon (:) as the separator, but do not include a colon at the end of the PYTHONPATH value as that impacts the execution of soamapiversion.py. For example:

     Setting the environment variable for IBM Spectrum Symphony Developer Edition to run as the client

     export PYTHONPATH=$SOAM_HOME/soam_version/linux-x86_64/lib64/pythonapi_3.8.0:$SOAM_HOME/soam_version/linux-x86_64/lib64:$SOAM_HOME/soam_version/linux-x86_64/ego_lib64

     Setting the environment variable for IBM Spectrum Symphony to run as the client

     export PYTHONPATH=$SOAM_HOME/soam_version/linux-x86_64/lib64/pythonapi_3.8.0:$SOAM_HOME/soam_version/linux-x86_64/lib64:${SOAM_HOME}/../ego_version/linux-x86_64/lib

     Setting the environment variable for the IBM Spectrum Symphony client to run workload

     export PYTHONPATH=$SOAM_HOME/lib64/pythonapi_3.8.0:$SOAM_HOME/lib64

6. Run the sample client and service.

   To run the service, you run the client application. The service a client application uses is specified in the application profile.

   1. Change to the Python sample SampleApp directory:

      cd $SOAM_HOME/version/samples/Python/SampleApp/

   2. Source the environment:
      • (csh) source cshrc.sampleapp
      • (bash) . profile.sampleapp
   3. Change to SyncClient directory and run the application:

      cd SyncClient/
      python SyncClient.py

      You should see output on the command line as work is submitted to the system. The client starts and the system starts the corresponding service. The client displays messages indicating that it is running.

7. Review and understand the samples.

   You review the sample client application code to learn how you can create a synchronous client application.

   1. Locate the code samples:

      Files	Location of code sample
      Client	$SOAM_HOME/version/samples/Python/SampleApp/SyncClient
      Message object	$SOAM_HOME/version/samples/Python/SampleApp/Common
      Service code	$SOAM_HOME/version/samples/Python/SampleApp/Service
      Application profile	The service required to compute the input data along with additional application parameters are defined in the application profile: $SOAM_HOME/version/samples/Python/SampleApp/SampleAppPython.xml

   2. Understand what the sample does.

      The client application sample opens a session and sends ten input messages, and retrieves the results. The client application is a synchronous client that sends input and blocks the output until all the results are returned.

      The service takes input data sent by client applications, returns the input data you have sent and replies Hello Client!!.

   3. Review the sample code:

      Input and output: implement the message object

      Your client application needs to handle data that it sends as input, and output data that it receives from the service.

      

      Tip: Client applications and services share the same message class.

      In MyMessage.py:

      • We implement the MyMessage class
      • We define serialization methods for input and output messages
      • We implement methods to handle the data. For data types that are supported by IBM Spectrum Symphony Developer Edition, see the appropriate API reference.
        Note: If you already have an application with a message object that is serialized, you can pass a binary blob through the SoamDataBlock or the DefaultBinaryMessage class. For examples of using the SoamDataBlock class, refer to SoamDataBlock binary container.

      For this example, we have defined the same class for input and output messages. However, you can define separate classes for input and output messages.

      import soamapi
      
      class MyMessage(soamapi.Message):
          def __init__(self, i = None, is_sync = None, str = None):
              self.__int = i
              self.__bool = is_sync
              self.__string = str
              
          def get_int(self):
              return self.__int
      
          def set_int(self, i):
              self.__int = i
              
          def get_bool(self):
              return self.__bool
      
          def set_bool(self, is_sync):
              self.__bool = is_sync
              
          def get_string(self):
              return self.__string
      
          def set_string(self, str):
              self.__string = str
              
          def on_serialize(self, stream):
              stream.write_int(self.__int)
              stream.write_bool(self.__bool)
              stream.write_string(self.__string)
      
          def on_deserialize(self, stream):
              self.__int = stream.read_int()
              self.__bool = stream.read_bool()
              self.__string = stream.read_string()
      

      Initialize the client

      In SyncClient.py, when you initialize, you initialize the IBM Spectrum Symphony client infrastructure. You initialize once per client.

      Important: Initialization is required. Otherwise, API calls fail.

      soamapi.initialize()
      

      Connect to an application

      To send data to be calculated in the form of input messages, you connect to an application.

      You specify an application name, a user name, and password. The application name must match the one defined in the application profile.

      For IBM Spectrum Symphony Developer Edition, there is no security checking and login credentials are ignored so you can specify any user name and password. Security checking is done however, when your client application submits workload to the actual grid.

      The default security callback encapsulates the callback for the user name and password.

      Tip: When you connect, a connection object is returned.

      application_name = "SampleAppPython"
      sec_cb = soamapi.DefaultSecurityCallback("User1", "User1pw")
      connection = soamapi.connect(application_name, sec_cb)
      prt_msg = "Connected to application:" + application_name + " Connection ID:" + connection.get_id()
      print (prt_msg)
      

      Create a session to group tasks

      A session is a way of logically grouping tasks that are sent to a service for execution. The tasks are sent and received synchronously.

      When creating a session, you need to specify the session attributes by using the SessionCreationAttributes object. In this sample, we create a SessionCreationAttributes object called session_attr and set two parameters in the object.

      The first parameter is the session type. The session type is optional. If you leave this parameter blank " " or do not set a session type, system default values are used for session attributes. If you specify a session type in the client application, you must also configure the session type in the application profile. The session type name in your application profile and session type you specify in the client must match. If you use an incorrect session type in the client and the specified session type cannot be found in the application profile, an exception is thrown to the client.

      The second parameter is the session name. This is optional. The session name can be any descriptive name you want to assign to your session. It is for information purposes, such as in the command-line interface.

      We pass the attributes object to the create_session() method, which returns a pointer to the session:

      session_attr = soamapi.SessionCreationAttributes()
      session_attr.set_session_type("RecoverableAllHistoricalData")
      session_attr.set_session_name("MySessionName")
      session = connection.create_session(session_attr)
      prt_msg = "Created session:" + session.get_id()
      print (prt_msg)
      

      Send input data to be processed

      In this step, we create ten input messages to be processed by the service. When a message is sent, a task input handle is returned. This task input handle contains the ID for the task that was created for this input message.

      tasks_to_Submit = 10
      for count in range(0, tasks_to_Submit):
          hello = 'Hello Grid !!'
          task_data = MyMessage.MyMessage(count, True, hello)
          
          task_attr = soamapi.TaskSubmissionAttributes()
          task_attr.set_task_input(task_data)
          task_input_handle = session.send_task_input(task_attr)
          prt_msg = "Sent task:" + task_input_handle.get_id()
          print(prt_msg)
          task_input_handle = None
      

      Retrieve output

      Pass the number of tasks to the fetch_task_output() method to retrieve the output messages that were produced by the service. This method blocks until the output for all tasks is retrieved. The return value is an enumeration that contains the completed task results. Iterate through the task results and extract the messages using the populate_task_output() method. Display the task ID and the results from the output message:

      task_output_handle_list = session.fetch_task_output(tasks_to_Submit)
      for task_output_handle in task_output_handle_list:
          if task_output_handle.is_successful():
              out_msg = MyMessage.MyMessage()
              task_output_handle.populate_task_output(out_msg)
              
              # Display content of reply
              prt_msg = "Task Succeeded [" + task_output_handle.get_id() + "] \n"
              prt_msg += "Integer Value : " + str(out_msg.get_int()) + "\n"
              prt_msg += out_msg.get_string()
              print (prt_msg)
          else:
              ex = task_output_handle.get_exception()
              prt_msg = "Task Not Succeeded : " + str(ex)
              print (prt_msg)
              if ex.get_embedded_exception():
                  appEx = ex.get_embedded_exception()
                  print (appEx)
              else:
                  print ("No application exception")
          task_output_handle = None

      Uninitialize

      Always uninitialize the client API at the end of all API calls. If you do not call uninitialize, the client API is in an undefined state, resources used by the client are held indefinitely, and there is no guarantee your client will be stable.

      Important: Once you uninitialize, all objects become invalid. For example, you can no longer create a session or send an input message.

      soamapi.uninitialize()
      print ("All done!!!")
      

      Define a service container

      For a service to be managed by IBM Spectrum Symphony, it needs to be in a container object. This is the service container.

      In MyServiceContainer.py, MyServiceContainer inherits from the base class ServiceContainer:

      class MyServiceContainer(soamapi.ServiceContainer):
      

      Process the input

      IBM Spectrum Symphony calls on_invoke() on the service container once per task. Once you inherit from the ServiceContainer class, implement handlers so that the service can function properly. This is where the calculation is performed.

      The task context contains all information and functionality that is available to the service during an on_invoke() call in relation to the task that is being processed.

      To gain access to the data set for the client, you must present an instance of the message object to the populate_task_input() method on the task context. During this call, the data sent from the client is used to populate the message object.

      Since we are using the same service for synchronous and asynchronous clients, the if statement is used to indicate that the message was sent from a synchronous client. When the string in the output message is completely assembled, pass it to the set_task_output() method, which sets the task output message that is sent to the client.

      def on_invoke(self, task_context):
              # Get the task input
              inMsg = MyMessage.MyMessage()
              task_context.populate_task_input(inMsg)
      
              # We simply echo the data back to the client
              out_msg = MyMessage.MyMessage()
              out_msg.set_int(inMsg.get_int())
              out_msg.set_bool(inMsg.get_bool())
      
              str = "you sent : " + inMsg.get_string() + "\nwe replied : Hello Client !!\n>>> "
              if (inMsg.get_bool()):
                  str += "Synchronously.\n"
              else:
                  str += "Asynchronously.\n"
      
              out_msg.set_string(str)
              
              #Set our output message
              task_context.set_task_output(out_msg)
      

      Run the container

      The service is implemented within an executable. As a minimum, we need to create an instance of the service container and run it:

      myContainer = MyServiceContainer()
      myContainer.run()