Add operators to enhance monitoring

Two new operators are needed to make your application easier to monitor and debug. The Throttle operator copies tuples from input to output at a specified rate rather than as fast as possible. The DirectoryScan operator periodically scans a given directory; for each new file that satisfies optional criteria, it sends out a tuple that contains the file’s full path.

Instead of using the palette’s filter field to quickly pick up the operators you want, let’s browse the full palette to achieve the same result.

  1. In the graphical editor’s palette, expand spl (under Toolkits), and then spl.adapter.
  2. Drag DirectoryScan into the main composite. The editor names the operator DirectoryScan_4.
  3. Scroll down in the palette and expand spl.utility.
  4. Scroll down further, find Throttle.
  5. Drag and drop it onto the stream Observations, exactly as you did with the LocationType schema previously. (Make sure the stream is highlighted by green handles before you let go.)
    The operator will be called Throttle_5. The editor automatically connects the Observations stream to its input port and creates a new stream, with the same schema as Observations, from its output port to the input of Filtered. There is no need to adjust the schema of this new stream: The Throttle operator merely controls the rate at which tuples flow, without changing their contents.
    To straighten out the graph, click Layout and Fit to Content.
  6. Rename the new stream to Throttled. Rename the operator to the name of the stream by blanking out its alias. (That’s in the General tab of the Properties view; review Lab 1 if you forgot how to get there.)
  7. Drag a stream from the output of DirectoryScan_4 to the input of Observations.
  8. Click Layout > Fit to Content.
    Your graph should look like this at this point:9
    Tip for input ports: The FileSource operator can have an input port, but it is optional. In the original graph you did not use it, so there is no yellow box on the left. But while dragging a stream from another operator’s output port, the optional input port is indicated by a lightly outlined, unfilled box, and you can connect the stream to it like any other port.