IC SunsetThe developerWorks Connections platform will be sunset on December 31, 2019. On January 1, 2020, this community and its apps will no longer be available. More details available on our FAQ.
  • 2 replies
  • Latest Post - ‏2012-12-08T13:20:36Z by BruceGlassford
1245 Posts

Pinned topic How to define variables which can be globally accessed

‏2012-12-07T09:41:22Z |
I need to create a map whose values would change at run time to be accessible to all streams.
As per my understanding
expression<rstring> $id : 100;
is used to define constants.

please tell me how can i define a collection globally?

  • hnasgaard
    200 Posts

    Re: How to define variables which can be globally accessed

    The statement
    expression<rstring> $id : 100;
    is how you declare a parameter to a composite operator. The : 100 establishes a default value if no value for that parameter is specified at the operator invocation site. That has nothing to do with collections as such.

    There is no way in the SPL language to define a variable at "global scope" that is available across operators. This is because operators can run on disparate machines, and would need a "shared memory" infrastructure to support the sharing of data between those machines. You could potentially write your own shared memory facility and access if from operators using native functions.
  • BruceGlassford
    71 Posts

    Re: How to define variables which can be globally accessed

    There's no global variable access for the reasons already given - it's across machines. One way to handle this is to have a stream containing all the updates that are read by each of the operators (you'll probably need to use Custom to submit something onto that stream, but most operators could consume it in their logic clause.

    When you run across a situation like this, it may be a good idea to revisit how you're setting up the logic of the application - if you have to keep a set of state variables around, are you really using streaming processing? Maybe the state values need to be appended to the streams early on in the program and passed on as needed? What happens to a stream in process if the map changes while it's being processed through the job? Should it use the values from when it started the process, or the new values? and if so, why?

    The usual solution in these cases is to have a control stream going into operators that require that data. They then maintain their own copies of the data they need and update state variables in the operator. That way if you have a payroll system, for example, that gives across-the-board pay raises for employees (maybe my boss is listening - if so, it's a good idea, boss!), and the rate is input periodically (we'll ignore how, but it's in stream<float32 percentage> Raises for the following example), and you have a Custom that applies raises:

    stream <rstring name, float32 newsalary> GiveRaises = Custom(EmployeeData; Raises) {
    state: float32 raiserate = 0.0; // No raises today!
    onTuple Raises:
    raiserate = percentage;
    onTuple EmployeeData:
    submit({name = EmployeeData.name, newsalary = (raiserate + 1.0) * EmployeeData.salary}, GiveRaises);