AI Engine/Programmable Logic Integration

The following operations read data from the given input stream and advance the stream for the kernels mapped to the programmable logic.

int32 readincr(input_stream_int32 *w);
uint32 readincr(input_stream_uint32 *w);
cint16 readincr(input_stream_cint16 *w);
float readincr(input_stream_float *w);

int64 readincr(input_stream_int64 *w);
uint64 readincr(input_stream_uint64 *w);
cint32 readincr(input_stream_cint32 *w);
cfloat readincr(input_stream_cfloat *w);

The following operations write data to the given output stream and advance the stream for the kernels mapped to the programmable logic.

void writeincr(output_stream_int32 *w, int32 v);
void writeincr(output_stream_uint32 *w, uint32 v);
void writeincr(output_stream_cint16 *w, cint16 v);
void writeincr(output_stream_float *w, float v);

void writeincr(output_stream_int64 *w, int64 v);
void writeincr(output_stream_uint64 *w, uint64 v);
void writeincr(output_stream_cint32 *w, cint32 v);
void writeincr(output_stream_cfloat *w, cfloat v);

The AI Engine compiler supports a subset of high-level synthesis (HLS) interfaces.

The supported input and output data types of HLS functions are:

• hls::stream<ap_axis<N,0,0,0>> and hls::stream<ap_axiu<N,0,0,0>>, where N can be 32, 64, or 128
• hls::stream<ap_int<N>> and hls::stream<ap_uint<N>>, where N can be 32 or 64
• hls::stream<T>, where T can be int, unsigned int, long long, unsigned long long, or float
• hls::stream<std::complex<T>>, where T can be int, short or float

The following scalar run-time parameter (RTP) data types are also supported:

• ap_int<N> and ap_uint<N>, where N can be 8, 16, 32, or 64
• short, unsigned short, int, unsigned int, long long, unsigned long long, and float
• std::complex<T>, where T can be int, short, or float

The following array RTP data types are supported:

• Arrays of ap_int<N> and ap_uint<N>, where N can be 8, 16, 32, or 64
• Arrays of short, unsigned short, int, unsigned int, long long, unsigned long long, or float
• Arrays of std::complex<T>, where T can be int, short, or float

When ap_int<N> and ap_uint<N> are used as RTPs, the PS program can use adf::graph::update and adf::graph::read with compatible data types to access RTPs. For example, use adf::graph::update(input_port& rtpPort, int32 value) to update the ap_int<32> run-time parameter. For more information on the use of RTPs, see Run-Time Graph Control API.

The only supported HLS function return type is void.

graph::update for an HLS scalar RTP greater than 32 bits should be called before graph::run.

To support the HLS math library inside the kernel, an additional linker option is needed for the AI Engine compiler:

--Xpslinker="-lhlsmc++-GCC46
-lIp_floating_point_v7_0_bitacc_cmodel 
-lIp_xfft_v9_1_bitacc_cmodel 
-lIp_fir_compiler_v7_2_bitacc_cmodel 
-lIp_dds_compiler_v6_0_bitacc_cmodel 
-L$(XILINX_HLS)/lnx64/tools/fpo_v7_0 
-L$(XILINX_HLS)/lnx64/lib/csim 
-L$(XILINX_HLS)/lnx64/tools/dds_v6_0 
-L$(XILINX_HLS)/lnx64/tools/fir_v7_0 
-L$(XILINX_HLS)/lnx64/tools/fft_v9_1 
-Wl,-rpath,$(XILINX_HLS)/lnx64/lib/csim 
-Wl,-rpath,$(XILINX_HLS)/lnx64/tools/fpo_v7_0 
-Wl,-rpath,$(XILINX_HLS)/lnx64/tools/fft_v9_1 
-Wl,-rpath,$(XILINX_HLS)/lnx64/tools/fir_v7_0 
-Wl,-rpath,$(XILINX_HLS)/lnx64/tools/dds_v6_0"

HLS functions can be directly used in data-flow graphs. The only extra work is to write a separate header file to declare function signatures and to specify hls::stream directions for HLS functions to be used in data-flow graphs. In this special header file, the function signature is declared the same as in the HLS function definition, except that the input or output direction of a hls::stream data type is specially qualified using adf::dir::in<T> and adf::dir::out<T>, respectively.

#include "hls_stream.h"
#include "ap_int.h"
extern "C" void hls_mul(hls::stream<ap_axis<32,0,0,0>>& in, hls::stream<ap_axis<32,0,0,0>>& out, ap_int<32> inputParam, unsigned int loop_count)
{
  for (unsigned int n=0; n<loop_count; n++)
  {
    for (int i=0; i<8; i++)
    {
      ap_axis<32,0,0,0> value = in.read();
      value.data *= inputParam;
      out.write(value);
    }
  }
}

#ifndef __HLS_KERNELS_H__
#define __HLS_KERNELS_H__
#include "hls_stream.h"
#include "ap_int.h"
#include "adf.h"
void hls_mul(adf::dir::in<hls::stream<ap_axis<32,0,0,0>>&> in, adf::dir::out<hls::stream<ap_axis<32,0,0,0>>&> out, ap_int<32> inputParam, unsigned int loop_count);
#endif

#include "hls_kernels.h" //declare hls_mul as shown above
#include "aie_kernels.h" //declare producer and consumer aie kernel functions
#include "adf.h"
using namespace adf;

class mygraph : public graph
{
public:
  kernel prd, mul, cns;
  mygraph()
  {
    prd = kernel::create(producer);
    cns = kernel::create(consumer);

    mul = kernel::create(hls_mul); //create a hls kernel from the declaration in hls_kernels.h
    fabric<pl>(mul); //hls kernel runs on programmable logic
    source(mul) = "hls_mul.cpp" //specify source file containing hls function definition

    adf::connect<window<32>, stream>(prd.out[0], mul.in[0]);
    adf::connect<stream, window<32>>(mul.out[0], cns.in[0]);
  }
};

hls::stream<ap_axis<N,0,0,0>> and hls::stream<ap_axiu<N,0,0,0>> represent an AXI4-Stream with API to specify the tkeep per byte. However, the stream implementation inside the AI Engine and adf stream API do not support a per-byte tkeep, instead per-word tkeep is supported. As a result, ap_axis<N,0,0,0> and ap_axiu<N,0,0,0> are assumed to be fully packed within each 32-bit word boundary when used inside data flow graphs. The following tables specify the allowed connections between hls::stream and adf stream and window data types in various scenarios where users want to pack and unpack data using ap_axis and ap_axiu.

To conform to stream interface protocol and data flow graph control semantics, PL kernels (whether written in HDL or HLS) are required to adhere to one or more of the following interface protocols.

• Block level: AXI4-Lite interface with ap_ctrl_hs protocol.

  For HLS kernels, the corresponding INTERFACE pragma is:

  #pragma HLS interface s_axilite port=return bundle=control
  #pragma HLS interface ap_ctrl_hs port=return

  IMPORTANT: When a PL kernel inside a graph is under the control of the ADF API in each adf::graph::run, the PL kernel is started once, regardless of the number of iterations set for adf::graph::run(). However, in each graph adf::graph::run(), an AI Engine kernel will run the number of iterations set for adf::graph::run(). In the graph adf::graph::wait(), both the AI Engine kernel and PL kernel inside the graphs are waiting to be completed. Make sure that the AI Engine kernel and PL kernel inside the graph can start and complete in synchronous mode.
• hls::stream port: AXI4-Stream interface.

  For HLS kernels, the corresponding INTERFACE pragma is:

  #pragma HLS interface axis port={hls::stream port name}

• Synchronous run-time parameter (RTP): AXI4-Lite interface with the ap_hs protocol.

  For HLS kernels, the corresponding INTERFACE pragmas are:

  #pragma HLS interface s_axilite port={RTP port name} bundle=control
  #pragma HLS interface ap_hs port={RTP port name}

• Asynchronous RTP: AXI4-Lite interface with the ap_none protocol.

  For HLS kernels, the corresponding INTERFACE pragmas are:

  #pragma HLS interface s_axilite port={RTP port name} bundle=control
  #pragma HLS interface ap_none port={RTP port name}

  IMPORTANT: An asynchronous run-time parameter for a PL kernel should be initialized using adf::graph::update before adf::graph::run.
• Asynchronous array run-time parameter: AXI4-Lite interface with the ap_memory protocol.

  For HLS kernels, the corresponding INTERFACE pragmas are:

  #pragma HLS interface s_axilite port={RTP port name} bundle=control
  #pragma HLS interface ap_memory port={RTP port name}

• Array or pointer port: Memory-mapped AXI4 interface.

  For HLS kernels, the corresponding INTERFACE pragmas are:

  #pragma HLS interface m_axi port=mem offset=slave
  #pragma HLS interface s_axilite port=mem bundle=control

Using the previous hls_mul function as an example, and assuming inputParam and loop_count are asynchronous run-time parameters, the required HLS pragmas to conform with the data flow semantics are:

#include "hls_stream.h"
#include "ap_int.h"
extern "C" void hls_mul(hls::stream<ap_axis<32,0,0,0>>& in, hls::stream<ap_axis<32,0,0,0>>& out, ap_int<32> inputParam, unsigned int loop_count)
{
#pragma HLS interface axis port=in
#pragma HLS interface axis port=out
#pragma HLS interface s_axilite port=inputParam bundle=control
#pragma HLS interface ap_none port=inputParam
#pragma HLS interface s_axilite port=loop_count bundle=control
#pragma HLS interface ap_none port=loop_count
#pragma HLS interface s_axilite port=return bundle=control
#pragma HLS interface ap_ctrl_hs port=return
  for (unsigned int n=0; n<loop_count; n++)
  {
    for (int i=0; i<8; i++)
    {
      ap_axis<32,0,0,0> value = in.read();
      value.data *= inputParam;
      out.write(value);
    }
  }
}

Here, loop_count can be used to control the loop count when adf::graph::run is called. Before each adf::graph::run, call adf::graph::update to update the loop_count and the samples by PL kernel will match the samples by AI Engine kernel with adf::graph::run(<ITERATION_NUM>).

Free-running Programmable Logic Kernels without AXI4-Lite Interface

By default, all PL kernels inside the graph are free-running (that is, run forever after reset). The free-running kernels are kernels without any AXI4-Lite interface and control protocols.

To support PL kernels with an AXI4-Lite interface and control protocols, add --pl-axi-lite=true option (default is false) to the AI Engine compiler (aiecompiler).

HLS Programmable Logic Kernels AXI4-Lite Interface

The run-time parameter is supported using the AXI4-Lite interface. To use the AXI4-Lite interface, add the --pl-axi-lite=true option to the AI Engine compiler.

The AI Engine clock is set to run at minimum of 1 GHz or higher based on the device speed grade. The default width of a stream channel is 32 bits. Because this frequency is higher than the PL clock frequency, it is always necessary to perform a clock domain crossing to the PL region, for example, to either one-half or a quarter of the AI Engine clock frequency.

For C++ HLS PL kernels, choose a reasonable target frequency depending on the complexity of the algorithm implemented. adf::pl_frequency constraints can be used to constrain each PL kernel in a graph. For example, adf::pl_frequency(<PL_KERNEL>) = freq; construct can be used in the AI Engine compiler and the --hls.clock option can be used in the Vitis compiler when compiling HLS C/C++ into Xilinx object (XO) files.

The AI Engine runs at 1 GHz (or more, depending on the device) and can write at most two streams with a 32-bit data width per cycle. In contrast, an IP implemented in the PL can run at 500 MHz (depending on the device), while consuming a larger bit width. Rate matching is concerned with balancing the throughput from the producer to the consumer, and is used to ensure that neither of the processes creates a bottleneck with respect to the total performance. The following equation shows the rate matching for each channel:

The following table shows a PL rate matching example for a 32-bit channel written to each cycle by the AI Engine at 1 GHz. As shown, the PL IP has to consume two times the data at half the frequency or four times the data at one quarter of the frequency.

Because the need to match frequency and adjust data-path width is well understood by the Vitis compiler (v++), the tool automatically extracts the port width from the PL kernel, the frequency from the clock specification, and introduces an upsizer/downsizer to temporarily store the data exchanged between the AI Engine and the PL regions to manage the rate match.

To avoid deadlocks, it is important to ensure that if multiple channels are read or written between the AI Engine and the PL, the data rate per cycle is concurrently achieved on all channels. For example, if one channel requires 32 bits, and the second 64 bits, the AI Engine code must ensure that both channels are written adequately to avoid back pressure or starvation on the channel. Additionally, to avoid deadlock, writing/reading from the AI Engine and reading/writing in the PL code must follow the same chronological order.

The number of interfaces used in the graph function definition for the PL defines the number of AXI4-Stream interfaces. Each argument results in the creation of a separate stream.

Multiple PL kernels can be defined inside the graph, with stream connections between the PL kernels. But only PL kernel streams with the same width can be connected for the AI Engine simulator.

However, the Vitis compiler can automatically insert a data-width converter and clock-domain crossing IP between the connections of PL kernels outside the graph to resolve any mismatch in bit width or frequency. Taking these facts into consideration, you must carefully choose which PL kernels are inside or outside the graph.