Specialized Graph Constructs

Static File-scoped Tables

Kernel functions can use private, read-only data structures that are accessed as file-scoped variables. The compiler allocates a limited amount of static heap space for such data. As an example, consider the following header file (user_parameter.h):

#ifndef USER_PARAMETER_H
#define USER_PARAMETER_H

#include <adf.h>

static int32 lutarray[8] = {1,2,3,4,5,6,0,0} ; 

#endif

This header file can be included in the kernel source file and the look-up table can be accessed inside a kernel function directly. The static modifier ensures that the array definition is local to this file. The AI Engine compiler then allocates this array in static heap space for the processor where this kernel is used.

#include "user_parameter.h"
void simple_lut(input_window_cint16 * in, output_window_cint16 * out){
  v4cint32 tmp; 
  v4cacc48 acc;
  v32cint16 coeffs; 

  upd_w(coeffs, 0, lutarray);
  window_readincr(in, tmp);
  acc = mul4(tmp 0, 0x3210, 1, coeffs, 0, 0x0000, 1);
  acc = mac4(acc, tmp, 2, 0x3210, 1, coeffs, 2, 0x0000, 1) ;
  acc = mac4(acc, tmp, 4, 0x3210, 1, coeffs, 4, 0x0000, 1) ;
  window_writeincr(out, srs(acc) ) ;
  }

Global Graph-scoped Tables

While the previous example only includes an eight entry look-up table accessed as a global variable, many other algorithms require much larger look-up tables. Because AI Engine local memory is at a premium, it is much more efficient for the AI Engine compiler to manage the look-up table explicitly for specific kernels than to leave a large amount of stack or heap space on every processor. Such tables should not be declared static in the kernel header file.

#ifndef USER_PARAMETER_H
#define USER_PARAMETER_H

#include <adf.h>

int32 lutarray[8] = {1,2,3,4,5,6,0,0} ; 

#endif

The kernel source continues to include the header file and use the table as before. But, now you must declare this table as extern in the graph class header and use the parameter::array(…) function to create a parameter object explicitly in the graph. You also need to attach this parameter object to the kernel as shown in the following code:

#include <adf.h>
extern int32 lutarray[8];
class simple_lut_graph : public graph {
public:
  kernel k;
  parameter p;

  simple_lut_graph() {
    k = kernel::create(simple);
    p = parameter::array(lutarray);
    connect<>(p,k);
    ...
  }
}

Including this explicit specification of the look-up table in the graph description ensures that the compiler is aware of the requirement to reserve a suitably sized piece of memory for the look-up table when it allocates memory for kernel input and output buffers.

Shared Graph-scoped Tables

Sometimes, the same table definition is used in multiple kernels. Because the AI Engine architecture is a distributed address-space architecture, each processor binary image that executes such a kernel needs to have that table defined in its own local memory. To get the correct graph linkage spread across multiple processors, you must declare the tables as extern within the kernel source file as well as the graph class definition file. Then, the actual table definition needs to be specified in a separate header file that is attached as a property to the kernel as shown below.

#include <adf.h>
extern int32 lutarray[8];
class simple_lut_graph : public adf::graph {
public:
  kernel k;
  parameter p;

  simple_lut_graph() {
    k = kernel::create(simple);
    p = parameter::array(lutarray);
    connect<>(p,k);

    std::vector<std::string> myheaders;
    myheaders.push_back("./user_parameter.h")
    headers(k) = myheaders;
    ...
  }
}

This ensures that the header file that defines the table is included in the final binary link wherever this kernel is used without causing re-definition errors.

The AI Engine compiler automatically allocates the desired number of buffers for each memory connection. There are several different cases.

• Lookup tables are always allocated as single buffers because they are expected to be read-only and private to a kernel. No locks are needed to synchronize lookup table accesses because they are expected to be accessed in an exclusive manner.
• Window connections are usually assigned double buffers if the producer and consumer kernels are mapped to different processors or if the producer or the consumer is a DMA. This enables the two agents to operate in a pipelined manner using ping-pong synchronization with two locks. The AI Engine compiler automatically generates this synchronization in the respective processor main functions.
• If the producer and consumer kernels are mapped to the same processor, then the window connection is given only one buffer and no lock synchronization is needed because the kernels are executed sequentially.
• Run-time parameter connections are always assigned double buffers along with a selector word to choose the next buffer to be accessed.

Sometimes, with window connections, it is desirable to use only single buffer synchronization instead of double buffer. This is useful when the local data memory is at a premium and the performance penalty of using a single buffer for data transfer is not critical. This can be achieved using single_buffer(port<T>&) constraint.

single_buffer(first.in[0]);

The AI Engine compiler supports C++ kernel classes. The following example shows how to set filter coefficients and the number of samples of a FIR filter class through a constructor. The C++ kernel class allows internal states for each kernel instance to be encapsulated within the corresponding class object. In the following code, you can see an example of this where the filter coefficients (coeffs) are specified through the constructor. This resolves the problem of using file scope variable, global variable, or static function scope variable to store the internal states of a C function kernel. When multiple instances of such a kernel are mapped to the same core, the internal state variables are shared across multiple instances and cause conflicts.

//fir.h
#pragma once
#include "adf.h"
#define NUM_COEFFS 12

class FIR
{
private:
    int32 coeffs[NUM_COEFFS];
    int32 tapDelayLine[NUM_COEFFS];
    uint32 numSamples;

public:
    FIR(const int32(&coefficients)[NUM_COEFFS], uint32 samples);
    void filter(input_window_int32* in, output_window_int32* out);
    static void registerKernelClass()
    {
        REGISTER_FUNCTION(FIR::filter);
    }
};

You are required to write the static void registerKernelClass() method in the header file. Inside the registerKernelClass() method, you need to call the REGISTER_FUNCTION macro. This macro is used to register the class run method to be executed on the AI Engine core to perform the kernel functionality. In the preceding example FIR::filter is registered using this macro. The kernel class constructor and run method should be implemented inside a separate source file. The implementation of a run method of a kernel class is the same as writing a kernel function described in previous chapters.

//fir.cpp
//implementation in this example is not optimized and is for illustration purpose
#include "fir.h"

FIR::FIR(const int32(&coefficients)[NUM_COEFFS], uint32 samples)
{
    for (int i = 0; i < NUM_COEFFS; i++)
        coeffs[i] = coefficients[i];

    for (int i = 0; i < NUM_COEFFS; i++)
        tapDelayLine[i] = 0;

    numSamples = samples;
}

void FIR::filter(input_window_int32* in, output_window_int32* out)
{
    for (int i = 0; i < numSamples; i++)
    {
        for (int j = NUM_COEFFS-1; j > 0; j--)
            tapDelayLine[j] = tapDelayLine[j - 1];

        tapDelayLine[0] = window_readincr(in);

        int32 y = 0;
        for (int j = 0; j < NUM_COEFFS; j++)
        {
            y += coeffs[j] * tapDelayLine[j];
        }

        window_writeincr(out, y);
    }
}

//graph.h
#pragma once
#include "adf.h"
#include "fir.h"
using namespace adf;

class mygraph : public graph
{
public:
    input_port in1, in2;
    output_port out1, out2;
    kernel k1, k2;

    mygraph()
    {
        //see lab8.3 for narrow filter coefficients
        k1 = kernel::create_object<FIR>(std::vector<int>({ 180, 89, -80, -391, -720, -834, -478, 505, 2063, 3896, 5535, 6504 }), 8);
        runtime<ratio>(k1) = 0.1;
        source(k1) = "src/fir.cpp";

        //see lab8.3 for wide filter coefficients
        k2 = kernel::create_object<FIR>(std::vector<int>({ -21, -249, 319, -78, -511, 977, -610, -844, 2574, -2754, -1066, 18539 }), 8);
        runtime<ratio>(k2) = 0.1;
        source(k2) = "src/fir.cpp";

        connect<window<32>>(in1, k1.in[0]);
        connect<window<32>>(in2, k2.in[0]);

        connect<window<32>>(k1.out[0], out1);
        connect<window<32>>(k2.out[0], out2);
    }
};

For a kernel class with a non-default constructor, you can specify the constructor parameter values in the arguments of kernel::create_object, when creating a representation of a kernel instance. In the previous example, two FIR filter kernels (k1 and k2) are created using kernel::create_object<FIR>. k1 has filter coefficients { 180, 89, -80, -391, -720, -834, -478, 505, 2063, 3896, 5535, 6504 } and k2 has filter coefficients { -21, -249, 319, -78, -511, 977, -610, -844, 2574, -2754, -1066, 18539 }. Both of them consume eight samples for each invocation.

The following code shows the AI Engine compiler generated program. The two FIR kernel objects are instantiated with the proper constructor parameters.

//Work/aie/x_y/src/x_y.cc
...
FIR i4({180, 89, -80, -391, -720, -834, -478, 505, 2063, 3896, 5535, 6504}, 8);
FIR i5({-21, -249, 319, -78, -511, 977, -610, -844, 2574, -2754, -1066, 18539}, 8);

int main(void) {
    ...
    // Kernel call : i4:filter
    i4.filter(get_input_window_int32(window_buf0_buf0d),get_output_window_int32(window_buf2_buf2d));
    ...
    // Kernel call : i5:filter
    i5.filter(get_input_window_int32(window_buf1_buf1d),get_output_window_int32(window_buf3_buf3d));
    ...
}

A kernel class can have a member variable occupying a significant amount of memory space that might not fit into program memory. The location of the kernel class member variable can be controlled. The AI Engine compiler supports array reference member variables that allow the compiler to allocate or constrain the memory space while passing the reference to the object.

//fir.h
#pragma once
#include "adf.h"
#define NUM_COEFFS 12

class FIR
{
private:
    int32 (&coeffs)[NUM_COEFFS];
    int32 tapDelayLine[NUM_COEFFS];
    uint32 numSamples;

public:
    FIR(int32(&coefficients)[NUM_COEFFS], uint32 samples);
    void filter(input_window_int32* in, output_window_int32* out);
    static void registerKernelClass()
    {
        REGISTER_FUNCTION(FIR::filter);
        REGISTER_PARAMETER(coeffs);
    }
};

//fir.cpp
#include "fir.h"
FIR::FIR(int32(&coefficients)[NUM_COEFFS], uint32 samples)
    : coeffs(coefficients)
{
    for (int i = 0; i < NUM_COEFFS; i++)
        tapDelayLine[i] = 0;

    numSamples = samples;
}

void FIR::filter(input_window_int32* in, output_window_int32* out)
{
...
}

The previous example shows a slightly modified version of the FIR kernel class. Here, member variable coeffs is a int32 (&)[NUM_COEFFS] data type. The constructor initializer coeffs(coefficients) initializes coeffs to the reference to an array allocated externally to the class object. To let the AI Engine compiler know that the coeffs member variable is intended to be allocated by the compiler, you must use REGISTER_PARAMETER to register an array reference member variable inside the registerKernelClass

The use of kernel::create_object to create a representation of a FIR kernel instance and to specify the initial value of the constructor parameters is the same as in the previous example. See the following code.

//graph.h
...
class mygraph : public graph
{
...
    mygraph()
    {
        k1 = kernel::create_object<FIR>(std::vector<int>({ 180, 89, -80, -391, -720, -834, -478, 505, 2063, 3896, 5535, 6504 }), 8);
        ...
        k2 = kernel::create_object<FIR>(std::vector<int>({ -21, -249, 319, -78, -511, 977, -610, -844, 2574, -2754, -1066, 18539 }), 8);
        ...
    }
};

The following code shows the corresponding AI Engine compiler generated program. The memory spaces for int32 i4_coeffs[12] and int32 i5_coeffs[15] are outside the kernel object instances and are passed into the FIR objects by reference.

//Work/aie/x_y/src/x_y.cc
int32 i4_coeffs[12] = {180, 89, -80, -391, -720, -834, -478, 505, 2063, 3896, 5535, 6504};
FIR i4(i4_coeffs, 8);
int32 i5_coeffs[12] = {-21, -249, 319, -78, -511, 977, -610, -844, 2574, -2754, -1066, 18539};
FIR i5(i5_coeffs, 8);

int main(void) {
    ...
    // Kernel call : i4:filter
    i4.filter(get_input_window_int32(window_buf0_buf0d),get_output_window_int32(window_buf2_buf2d));
    ...
    // Kernel call : i5:filter
    i5.filter(get_input_window_int32(window_buf1_buf1d),get_output_window_int32(window_buf3_buf3d));
    ...
}

Because the memory space for an array reference member variable is allocated by the AI Engine compiler, the location constraint can be applied to constrain the memory location of these arrays, as shown in the following example code. The REGISTER_PARAMETER macro allows kernel::create_object to create a parameter handle for an array reference member variable, like k1.param[0] and k2.param[0], and the location<parameter> constraint can be applied.

//graph.h
...
class mygraph : public graph
{
...
    mygraph()
    {
        k1 = kernel::create_object<FIR>(std::vector<int>({ 180, 89, -80, -391, -720, -834, -478, 505, 2063, 3896, 5535, 6504 }), 8);
        ...
        k2 = kernel::create_object<FIR>(std::vector<int>({ -21, -249, 319, -78, -511, 977, -610, -844, 2574, -2754, -1066, 18539 }), 8);
        ...

        location<parameter>(k1.param[0]) = address(…);
        location<parameter>(k2.param[0]) = bank(…);
    }
};

The C++ kernel class header files and the C++ kernel function template (see C++ Template Support) should not contain single-core specific intrinsic APIs and pragmas. This is the same programming guideline as writing regular C function kernels. This is because these header files are included in the graph header file and can be cross-compiled as part of the PS program. The Arm® cross-compiler cannot understand single-core intrinsic APIs or pragmas. Single-core specific programming content must be kept inside the source files.

A template is a powerful tool in C++. By passing the data type as a parameter, you eliminate the need to rewrite code to support different data types. Templates are expanded at compile time, like macros. The difference is that the compiler performs type checking before template expansion. The source code contains template functions and class definitions, but the compiled code can contain multiple copies of same function or class. Type parameters, non-type parameters, default arguments, scalar parameters, and template parameters can be passed to a template, where the compiler instantiates the function or class accordingly.

• Support for general C++ template features.
• Supported data types (T) and connection types between kernels:
  • Data type (T): int8, uint8, int16, uint16, cint16, int32, uint32, cint32, int64, uint64, float, cfloat
    IMPORTANT: acc48 and cacc48 data types are not supported in template stream connections.
  • Function parameter type: input_window<T>, output_window<T>, input_stream<T>, output_stream<T>
• The compiler does not support pre-compiled headers for template kernels.

Function Templates

Function template source code defines a generic function that can be used for different data types. Example function template:

// add.h
template<typename ELEMENT_TYPE, int FACTOR, size_t NUM_SAMPLES> void add(input_window<ELEMENT_TYPE>* in, 
 output_window<ELEMENT_TYPE>* out);

// add.cpp
template<typename ELEMENT_TYPE, int FACTOR, size_t NUM_SAMPLES> void add(input_window<ELEMENT_TYPE>* in, 
 output_window<ELEMENT_TYPE>* out)
{
    for (int i=0; i<NUM_SAMPLES; i++)
    {
        ELEMENT_TYPE value = window_readincr(in);
        value += FACTOR;
        window_writeincr(out, value);
    }
}

// graph.h
mygraph()
{
    k[0] = kernel::create(add<int32, 6, 8>);
    k[1] = kernel::create(add<int16, 3, 8>);
    for (int i=0; i<NUM_KERNELS; i++)
    {
        runtime<ratio>(k[i]) = 0.3;
        source(k[i]) = "src/add.cpp";
    }

    connect<window<32>>(in[0], k[0].in[0]);
    connect<window<32>>(k[0].out[0], out[0]);

    connect<window<16>>(in[1], k[1].in[0]);
    connect<window<16>>(k[1].out[0], out[1]);
}

where:

• add.h defines a template add() function.
• add.cpp defines the code for the template add() function.
• graph.h uses the template add() function within mygraph class.

Class Templates

Like function templates, class templates are useful when a class defines an object that is independent of a specific data type. Example class template:

// fir.h
...
template<size_t NUM_COEFFS, typename ELEMENT_TYPE> class FIR
{
private:
    ELEMENT_TYPE (&coeffs)[NUM_COEFFS];
    ELEMENT_TYPE tapDelayLine[NUM_COEFFS];
    uint32 numSamples;

public:
    FIR(ELEMENT_TYPE(&coefficients)[NUM_COEFFS], uint32 samples);

    void filter(input_window<ELEMENT_TYPE>* in, output_window<ELEMENT_TYPE>* out);

    //user needs to write this function to register necessary info
    static void registerKernelClass()
    {
        REGISTER_FUNCTION(FIR::filter);
        REGISTER_PARAMETER(coeffs);
    }
};

// fir.cpp
...
template<size_t NUM_COEFFS, typename ELEMENT_TYPE> FIR<NUM_COEFFS, ELEMENT_TYPE>::FIR(ELEMENT_TYPE(&coefficients)[NUM_COEFFS], uint32 samples):coeffs(coefficients)
{
    ...
}

template<size_t NUM_COEFFS, typename ELEMENT_TYPE> void FIR<NUM_COEFFS, ELEMENT_TYPE>::filter(input_window<ELEMENT_TYPE>* in, output_window<ELEMENT_TYPE>* out)
{
    ...
}

// graph.h
...
mygraph()
{
    k1 = kernel::create_object<FIR<12, int32>>(std::vector<int>({ 180, 89, -80, -391, -720, -834, -478, 505, 2063, 3896, 5535, 6504 }), 8);
    runtime<ratio>(k1) = 0.1;
    source(k1) = "src/fir.cpp";
    headers(k1) = { "src/fir.h" };

    k2 = kernel::create_object<FIR<15, int32>>(std::vector<int>({ -21, -249, 319, -78, -511, 977, -610, -844, 2574, -2754, -1066, 18539, 0, 0, 0 }), 8);
    runtime<ratio>(k2) = 0.1;
    source(k2) = "src/fir.cpp";
    headers(k2) = { "src/fir.h" };
    ...
}

where:

• fir.h defines a class template where class FIR is declared.
• fir.cpp contains class FIR implementation and the class FIR member function filter implementation.
• graph.h demonstrates the template class FIR instantiation within the mygraph class.