Generators

A generator is a set of methods and objects that can be used for managing asynchronous processes in a synchronous manner.

Asynchronous processes occur in GatewayScript when you use callback functions. Coordinating the operation of callback functions can be difficult in complex programs. You can call asynchronous functions synchronously by using generators, which GatewayScript supports. You can also use third-party libraries that use the generators to synchronize asynchronous function calls.

The generators use statements that start with the yield keyword and the next() method to sequence processing. The yield keyword suspends processing and returns the value of the expression that contains the yield keyword to the previously called next() method. The function* declaration defines a generator function, which returns a generator object. The next() method either begins processing from the start of the generator or resumes processing from the previous statement that contains a yield keyword that was encountered in the generator. Statements with the yield keyword suspend processing of the generator and return the value of the yield expression to the previously called next() method. The return value of the next() method is either the yielded expression value or is undefined if the generator completes. The done property indicates whether the generator function completed processing.

In the following example, generator functions manage the sequencing of expressions by suspending them with statements that contain yield keywords and by resuming them with next() methods.

 1    function* foo() {
 2       console.info(yield "foo 1");
 3       console.info(yield "foo 2");
 4    }
 5    var test = foo();
 6    console.info(test.next());
 7    console.info(test.next("baz 1"));
 8    console.info(test.next("baz 2"));

Line	Explanation
1	The `foo()` generator function is defined with two calls to write text in the system log.
2	When resumed, the `yield` expression `"baz 1"` is written in the system log.
3	When resumed, the `yield` expression `"baz 2"` is written in the system log.
5	A variable is set to a generator object of the `foo()`generator function.
6	The `next()` method is called on the `test` generator. The string `"foo 1"` is written when the suspended `yield` expression is returned from the call to the `foo()` generator. The value of `done` is false because the `foo()` function is not complete. The value of the `next()` function is a JSON object, where the `value` property is the yielded value.
7	The `next()` method is called on the `test` generator to resume the suspended `yield` expression. The string `"baz 1"` is written in the system log by the generator because it is provided as an argument of the `test.next()`statement. The string `"foo 2"` is written due to the resumed expression in the call to the `foo()` generator. The value of `done` is false.
8	The `next()` method is called on the `test` generator to resume another suspended `yield` expression. The generator runs to completion without encountering another `yield` statement. Thus the `next()`method returns an `undefined` value and the `done` property is true. The string `"baz 2"` is written in the system log because it is part of the `test.next()` method call. The `next()` method returns an `undefined` value and a `done` property of true as the generator runs to completion without encountering another `yield` expression.

This function writes the following logs in the following order.

{ value: 'foo 1', done: false }
baz 1
{ value: 'foo 2', done: false }
baz 2
{ value: undefined, done: true }

By suspending and resuming code, you can model asynchronous code that uses callback functions.

The yield* operator statement delegates control to another generator that contains yield statements for asynchronous function calls. When the delegated iterator completes, the first generator continues at the point of the statement that contains

yield
*

 1  function* foo() {
 2  console.info(yield "foo 1");
 3  yield* bar();
 4  }
 5  function* bar() {
 6    console.info(yield "bar 1");
 7  }
 8  var test = foo();
 9  console.info(test.next());
10  console.info(test.next("baz 1"));
11  console.info(test.next("baz 2"));

Line	Explanation
1	The `foo*()` generator function is defined.
2	When called, the yield expression `"baz 1"` is written in the system log.
3	Control is delegated to the `*bar()` function.
5	The `bar*()` generator function is defined.
6	When called, the yield expression `"baz 2"` is written in the system log.
8	A variable is set to a generator object of the `foo()` generator function.
9	The `next()` method is called on the `test` generator. The result of the `test.next()` call writes `{ value: "foo 1", done: false }` in the system log.
10	The `next()` method is called on the `test` generator to resume the suspended `yield` expression. The text `{ value: "bar 1", done: false }` is written in the system log.
11	The `next()` method is called on the `test` generator to resume the suspended `yield` expression. The text `{ value: undefined, done: true }` is written in the system log.

The output sequence is as follows.

{ value: "foo 1", done: false }
baz 1
{ value: "bar 1", done: false }
baz 2
{ value: undefined, done: true }

A more complicated example shows the interaction between called functions and the next() generator method and yield statement.

 1   var util = require('util');
 2   function sync(gen) {
 3      var it = gen();
 4      function next(err, result) {
 5         var n = err ? it.throw(err) : it.next(result);
 6         if (!n.done) {
 7             if (util.safeTypeOf(n.value) == 'function') {
 8                 try {
 9                     n.value(function(err, result) {
10                         next(err, result);
11                     });
12                 } catch(e) {
13                     next(e);
14                 }
15             } else { next(null, n.value);
16             }
17         }
18      }
19      next();
20   }
21   function xpathSync(expr, xmldom) {
22      return function(callback) {
23         trans.xpath(expr, xmldom, callback);
24         }
25      }
26   var xmlString = 
27          '<?xml version="1.0"?>' +
28          '<library>' +
29          '<book id="1">' +
30             '<title>DataPower Admin</title>' +
31             '<author>Hermann</author>' +
32             '<media>Hardcover</media>' +
33          '</book>' +
34          '<book id="2">' +
35             '<title>DataPower Development</title>' +
36             '<author>Tim</author>' +
37             '<media>Hardcover</media>' +
38          '</book>' +
39          '<book id="3">' +
40             '<title>DataPower Capacity</title>' +
41             '<author>Hermann</author>' +
42             '<media>eBook</media>' +
43          '</book>' +
44          '</library>';
45   var domTree = XML.parse(xmlString);
46   var trans = require('transform');
47   sync(function*() {
48      var nl1 = yield xpathSync("//book[author/text()='Hermann']", domTree);
49      var nl2 = yield xpathSync("//book[media/text()='Hardcover']", nl1);
50      session.output.write(XML.stringify(nl2));
51   });

Line	Explanation
2	The `sync()` function controls processing. The `gen` parameter is the generator function that is defined in line 47.
3	Set the variable `it` to an instantiation of the generator object by calling the generator function.
4	Define `next()` as a mechanism to control the sequencing. This `next()` function is different from the `it.next()` generator method call on line 5.
5	The variable `n` is set to the result of the `it.throw()` statement or the call to the `it.next()` method. If the error object `err` is not null, the `throw()` method is called. Otherwise, the `it.next()` statement is called. The result in either case is a JSON object. The JSON object has a `value` property and a Boolean `done` property. The `value` is either a scalar data type or a function. A generator either catches an exception that is thrown by the `it.throw()` method or the generator stops if the exception is not uncaught. The `it.throw()` method enables the generator to resume with an exception. Whether caught or not by the generator, the variable `n` is an object with the `value` and `done` properties for the subsequent yield statement or the completion of the generator.
6	Test `n` to see whether the `done` property of the previous `next()` call is true.
7	Test whether the value that is returned by the generator is a function data type. The generator can return a scalar data type or a function. If a function is returned, that function is called. If the return is not a function data type, the scalar value is used. For example, a string is a scalar data type.
9-10	When the yielded value `n` is a function, call this function by providing a callback function for the asynchronous call. When the callback function is called, it recursively calls the `next()` function, by providing the result and error to the callback function, to sequence processing. The yielded function is called in the `try/catch` block to handle any exceptions from that function.
12-14	An error is caught and the `next()` function in the `sync()` function is recursively called. The error provides information the `next()` function throws to the generator.
15	The `next()` function is recursively called by providing the scalar result from the previous yield expression.
19	The `next()` function is called to begin execution.
21-24	`xpathSync()` wraps the asynchronous `transform.xpath()` function.
26-44	XML string
45	The `domTree` variable contains the XML parsing of the sample XML string.
47	The `sync()` function that is defined on line 2 is started with an anonymous generator function as its parameter.
48-50	The body of the generator function contains code that is run by the `sync()` function by using the `next()` function. Variables are set to the suspended result of a call to the `xpathSync()` function. Variables `nl1` and `nl2` are set to the result of their yield expressions. The yield expression resumes the generator in the `sync()` method, which gets the wanted function as its value. That function is run and the result is provided in a subsequent `it.next()` statement. The `it.next()` statement resumes the generator and returns the value of the function call. The values that are returned are XML nodelists, which results from the `transform.xpath()` calls. The value of the `n12` variable is written in the system log.