Run-Time Graph Control API

Parameter Inference

If an integer scalar value appears in the formal arguments of a kernel function, then that parameter becomes a run-time parameter. In the following example, the argument select is a run-time parameter.

#ifndef FUNCTION_KERNELS_H
#define FUNCTION_KERNELS_H
 
  void simple_param(input_window_cint16 * in, output_window_cint16 * out, int select);

#endif

#ifndef FUNCTION_KERNELS_H
#define FUNCTION_KERNELS_H

  void filter_with_array_param(input_window_cint16 * in, output_window_cint16 * out, const int32 (&coefficients)[32]);

#endif

Implicit ports are inferred for each parameter in the function argument, including the array parameters. The following table describes the type of port inferred for each function argument.

From the table, you can see that when the AI Engine cannot make externally visible changes to the function parameter, an input port is inferred. When the formal parameter is passed by value, a copy is made, so changes to that copy are not externally visible. When a parameter is passed with a const qualifier, the parameter cannot be written, so these are also treated as input ports.

When the AI Engine kernel is passed a parameter reference and it is able to modify it, an inout port is inferred and can be used to pass parameters between AI Engine kernels or to allow reading back of results from the control processor.

Parameter Hookup

Both input and inout run-time parameter ports can be connected to corresponding hierarchical ports in their enclosing graph. This is the mechanism that parameters are exposed for run-time modification. In the following graph, an instance is created of the previously defined simple_param kernel. This kernel has two input ports and one output port. The first argument to appear in the argument list, in[0], is an input window. The second argument is an output window. The third argument is a run-time parameter (it is not a window or stream type) and is inferred as an input parameter, in[1], because it is passed by value.

In the following graph definition, a simple_param kernel is instantiated and windows are connected to in[0] and out[0] (the input and output windows of the kernel). The run-time parameter is connected to the graph input port, select_value.

class parameterGraph : public graph {
private:
  kernel first;
  
public:
  input_port select_value;
  input_port in;
  output_port out;
  parameterGraph() {
    first = kernel::create(simple_param);

    connect< window <32> >(in, first.in[0]);
    connect< window <32> >(first.out[0], out);
    connect<parameter>(select_value, first.in[1]);
  }
};

An array parameter can be hooked up in the same way. The compiler automatically allocates space for the array data so that it is accessible from the processor where this kernel gets mapped.

class arrayParameterGraph : public graph {
private:
  kernel first;
  
public:
  input_port coeffs;
  input_port in;
  output_port out;
  arrayParameterGraph() {
    first = kernel::create(filter_with_array_param);

    connect< window <32> >(in, first.in[0]);
    connect< window <32> >(first.out[0], out);
    connect<parameter>(coeffs, first.in[1]);
  }
};

Input Parameter Synchronization

The default behavior for input run-time parameters ports is triggering behavior. This means that the parameter plays a part in the rules that determine when a kernel could execute. In this graph example, the kernel only executes when three conditions are met:

• A valid window of 32 bytes of input data is available
• An empty window of 32 bytes is available for the output data
• A write to the input parameter takes place

In triggering mode, a single write to the input parameter allows the kernel to execute once, setting the input parameter value on every individual kernel call.

There is an alternative mode to allow input kernels parameters to be set asynchronously. To specify that parameters update asynchronously, use the async modifier when connecting a port.

connect<parameter>(param_port, async(first.in[1]));

When a kernel port is designated as asynchronous, it no longer plays a role in the firing rules for the kernel. When the parameter is written once, the value is observed in subsequent firings. At any time, the PS can write a new value for the run-time parameter. That value is observed on the next and any subsequent kernel firing.

Inout Parameter Synchronization

The default behavior for inout run-time parameters ports is asynchronous behavior. This means that the parameter can be read back by the controlling processor or another kernel, but the producer kernel execution is not affected. For synchronous behavior from the inout parameter where the kernel blocks until the parameter value is read out on each invocation of the kernel, you can use a sync modifier when connecting the inout port to the enclosing graph as follows.

connect<parameter>(sync(first.out[1]), param_port);

In default compilation mode, the main application is compiled as a separate control thread which needs to be executed on the PS in parallel with the graph executing on the AI Engine array. The main application can use update and read APIs to access run-time parameters declared within the graphs at any level. This section describes these APIs using examples.

Synchronous Update/Read

The following code shows the main application of the simple_param graph described in Specifying Run-Time Data Parameters.

#include "param.h"
parameterGraph mygraph;

int main(void) {
  mygraph.init();
  mygraph.run(2);
  
  mygraph.update(mygraph.select_value, 23);
  mygraph.update(mygraph.select_value, 45);

  mygraph.end();
  return 0;
}

In this example, the graph mygraph is initialized first and then run for two iterations. It has a triggered input parameter port select_value that must be updated with a new value for each invocation of the receiving kernel. The first argument of the update API identifies the port to be updated and the second argument provides the value. Several other forms of update APIs are supported based on the direction of the port, its data type, and whether it is a scalar or array parameter, see Adaptive Data Flow Graph Specification Reference.

If the program is compiled with a fixed number of test iterations, then for triggered parameters the number of update API calls in the main program must match the number of test iterations, otherwise the simulation could be waiting for additional updates. For asynchronous parameters, the updates are done asynchronously with the graph execution and the kernel uses the old value if the update was not made.

If additionally, the previous graph was compiled with a synchronous inout parameter, then the update and read calls must be interleaved as shown in the following example.

#include "param.h"
parameterGraph mygraph;

int main(void) {
  int result0, result1;
  mygraph.init();
  mygraph.run(2);

  mygraph.update(mygraph.select_value, 23);
  mygraph.read(mygraph.result_out, result0);
  mygraph.update(mygraph.select_value, 45);
  mygraph.read(mygraph.result_out, result1);

  mygraph.end();
  return 0;
}

In this example, it is assumed that the graph produces a scalar result every iteration through the inout port result_out. The read API is used to read out the value of this port synchronously after each iteration. The first argument of the read API is the graph inout port to be read back and the second argument is the location where the value will be stored (passed by reference).

The synchronous protocol ensures that the read operation will wait for the value to be produced by the graph before sampling it and the graph will wait for the value to be read before proceeding to the next iteration. This is why it is important to interleave the update and read operations.

Asynchronous Update/Read

When an input parameter is specified with asynchronous protocol, the kernel execution waits for the first update to happen for parameter initialization. However, an arbitrary number of kernel invocations can take place before the next update. This is usually the intent of the asynchronous update during application deployment. However, for debugging, wait API can be used to finish a predetermined set of iterations before the next update as shown in the following example.

#include "param.h"
asyncGraph mygraph;

int main(void) {
  int result0, result1;
  mygraph.init();

  mygraph.update(mygraph.select_value, 23);
  mygraph.run(5);
  mygraph.wait();
  mygraph.update(mygraph.select_value, 45);
  mygraph.run(15);
  mygraph.end();
  return 0;
}

In the previous example, after the initial update, five iterations are run to completion followed by another update, then followed by another set of 15 iterations. If the graph has asynchronous inout ports, that data can also be read back immediately after the wait (or end).

Another template for asynchronous updates is to use timeouts in wait API as shown in the following example.

#include "param.h"
asyncGraph mygraph;

int main(void) {
  int result0, result1;
  mygraph.init();
  mygraph.run();
  mygraph.update(mygraph.select_value, 23);
  mygraph.wait(10000);
  mygraph.update(mygraph.select_value, 45);
  mygraph.resume();
  mygraph.end(15000);
  return 0;
}

In this example, the graph is set up to run forever. However, after the run API is called, it still blocks for the first update to happen for parameter initialization. Then, it runs for 10,000 cycles (approximately) before allowing the control thread to make another update. The new update takes effect at the next kernel invocation boundary. Then the graph is allowed to run for another 15,000 cycles before terminating.

The previous run-time parameter examples highlight the ability to do run-time parameter updates from the control processor. It is also possible to propagate parameter updates between AI Engines. If an inout port on a kernel is connected to an input port on another kernel, then a chain of updates can be triggered through multiple AI Engines. Consider the two kernels defined in the following code. The producer has an input port that reads a scalar integer and an inout port that can read and write to an array of 32 integers. The consumer has an input port that can read an array of coefficients, and an output port that can write a window of data.

#ifndef FUNCTION_KERNELS_H
#define FUNCTION_KERNELS_H
 
  void producer(const int32 &, int32 (&)[32] );
  void consumer(const int32 (&)[32], output_window_cint16 *);

#endif

As shown in the following graph, the PS updates the scalar input of the producer kernel. When the producer kernel is run, it automatically triggers execution of the consumer kernel (when a buffer is available for the output data).

#include <adf.h>
#include "kernels.h"

using namespace adf;

class chainedGraph : public graph {
private:
  kernel first;
  kernel second;
  
public:
  input_port select_value; 
  output_port out;

  chainedGraph() {
      first =  kernel::create(producer);
      second = kernel::create(consumer);

      connect< window <32> >(second.out[0], out);
      connect<parameter>(select_value, first.in[0]);
      connect<parameter>(first.inout[0], second.in[0]);
  }
};

If the intention is to make a one-time update of values that are used in continuous processing of streams, the consumer parameter input port can use the async modifier to ensure it runs continuously (when a parameter is provided).