Programming C/C++ Kernels

Arbitrary precision integer data types are defined by ap_int or ap_uint for signed and unsigned integer respectively inside the header file ap_int.h. To use arbitrary precision integer data type:

• Add header file ap_int.h to the source code.
• Change the bit types to ap_int<N> or ap_uint<N>, where N is a bit-size from 1 to 1024.

#include “ap_int.h” 
ap_int<9> var1 // 9 bit signed integer
ap_uint<10> var2 // 10 bit unsigned integer

Some existing applications use floating point data types as they are written for other hardware architectures. However, fixed-point data types are a useful replacement for floating point types which require many clock cycles to complete. Carefully evaluate trade-offs in power, cost, productivity, and precision when choosing to implement floating-point vs. fixed-point arithmetic for your application and accelerators.

As discussed in Deep Learning with INT8 Optimization on Xilinx Devices (WP486), using fixed-point arithmetic instead of floating point for applications like machine learning can increase power efficiency, and lower the total power required. Unless the entire range of the floating-point type is required, the same accuracy can often be implemented with a fixed-point type resulting in the same accuracy with smaller and faster hardware. The paper Reduce Power and Cost by Converting from Floating Point to Fixed Point (WP491) provides some examples of this conversion.

Fixed-point data types model the data as an integer and fraction bits. The fixed-point data type requires the ap_fixed header, and supports both a signed and unsigned form as follows:

• Header file: ap_fixed.h
• Signed fixed point: ap_fixed<W,I,Q,O,N>
• Unsigned fixed point: ap_ufixed<W,I,Q,O,N>
  • W = Total width < 1024 bits
  • I = Integer bit width. The value of I must be less than or equal to the width (W). The number of bits to represent the fractional part is W minus I. Only a constant integer expression can be used to specify the integer width.
  • Q = Quantization mode. Only predefined enumerated values can be used to specify Q. The accepted values are:
    • AP_RND: Rounding to plus infinity.
    • AP_RND_ZERO: Rounding to zero.
    • AP_RND_MIN_INF: Rounding to minus infinity.
    • AP_RND_INF: Rounding to infinity.
    • AP_RND_CONV: Convergent rounding.
    • AP_TRN: Truncation. This is the default value when Q is not specified.
    • AP_TRN_ZERO: Truncation to zero.
  • O = Overflow mode. Only predefined enumerated values can be used to specify O. The accepted values are:
    • AP_SAT: Saturation.
    • AP_SAT_ZERO: Saturation to zero.
    • AP_SAT_SYM: Symmetrical saturation.
    • AP_WRAP: Wrap-around. This is the default value when O is not specified.
    • AP_WRAP_SM: Sign magnitude wrap-around.
  • N = The number of saturation bits in the overflow WRAP modes. Only a constant integer expression can be used as the parameter value. The default value is zero.
  TIP: The ap_fixed and ap_ufixed data types permit shorthand definition, with only W and I being required, and other parameters assigned default values. However, to define Q or N, you must also specify the parameters before those, even if you just specify the default values.

In the example code below, the ap_fixed type is used to define a signed 18-bit variable with 6 bits representing the integer value above the binary point, and by implication, 12 bits representing the fractional value below the binary point. The quantization mode is set to round to plus infinity (AP_RND). Because the overflow mode and saturation bits are not specified, the defaults AP_WRAP and 0 are used.

#include <ap_fixed.h>
...
  ap_fixed<18,6,AP_RND> my_type;
...

When performing calculations where the variables have different numbers of bits (W), or different precision (I), the binary point is automatically aligned. See the "C++ Arbitrary Precision Fixed-Point Types" in the Vivado Design Suite User Guide: High-Level Synthesis (UG902) for more information on using fixed-point data types.

The main data processed by the kernel computation, often in a large volume, should be transferred through the global memory banks on the FPGA board. The host machine transfers a large chunk of data to one or more global memory bank(s). The kernel accesses the data from those global memory banks, preferably in burst. After the kernel finishes the computation, the resulting data is transferred back to the host machine through the global memory banks.

When writing the kernel interface description, pragmas are used to denote the interfaces coming to and from the global memory banks.

vadd: for(int i = 0; i < len; i++) {
  c[i] = a[i] + b[i];
}

vadd: for(int i = 0; i < len; i++) {
  #pragma HLS PIPELINE
  c[i] = a[i] + b[i];
}

In the example above, it is assumed that every iteration of the loop takes three cycles: read, add, and write. Without pipelining, each successive iteration of the loop starts in every third cycle. With pipelining the loop can start subsequent iterations of the loop in fewer than three cycles, such as in every second cycle, or in every cycle.

The number of cycles it takes to start the next iteration of a loop is called the initiation interval (II) of the pipelined loop. So II = 2 means each successive iteration of the loop starts every two cycles. An II = 1 is the ideal case, where each iteration of the loop starts in the very next cycle. When you use the pragma HLS PIPELINE the compiler always tries to achieve II = 1 performance.

The following figure illustrates the difference in execution between pipelined and non-pipelined loops. In this figure, (A) shows the default sequential operation where there are three clock cycles between each input read (II = 3), and it requires eight clock cycles before the last output write is performed.

Figure: Loop Pipelining

If there are data dependencies inside a loop it might not be possible to achieve II = 1, and a larger initiation interval might be the result. Loop dependencies are discussed in Loop Dependencies.

vadd: for(int i = 0; i < 20; i++) {
  #pragma HLS UNROLL
  c[i] = a[i] + b[i];
}

In the preceding example, you can see pragma HLS UNROLL has been inserted into the body of the loop to instruct the compiler to unroll the loop completely. All 20 iterations of the loop are executed in parallel if that is permitted by any data dependency.

Completely unrolling a loop can consume significant device resources, while partially unrolling the loop provides some performance improvement without causing a significant impact on hardware resources.

array_sum:for(int i=0;i<4;i++){
  #pragma HLS UNROLL factor=2
  sum += arr[i];
}

array_sum_unrolled:for(int i=0;i<2;i+=2){
  // Manual unroll by a factor 2
  sum += arr[i];
  sum += arr[i+1];
}

Minim_Loop: while (a != b) { 
  if (a > b) 
    a -= b; 
  else 
    b -= a;
}

This loop cannot be pipelined. The next iteration of the loop cannot begin until the previous iteration ends.

Dealing with various types of dependencies with the xocc compiler is an extensive topic requiring a detailed understanding of the high-level synthesis procedures underlying the compiler. Refer to the Vivado Design Suite User Guide: High-Level Synthesis (UG902) for more information on "Dependencies with Vivado HLS."

Coding with nested loops is a common practice. Understanding how loops are pipelined in a nested loop structure is key to achieving the desired performance.

If the pragma HLS PIPELINE is applied to a loop nested inside another loop, the xocc compiler attempts to flatten the loops to create a single loop, and apply the PIPELINE pragma to the constructed loop. The loop flattening helps in improving the performance of the kernel.

ROW_LOOP: for(int i=0; i< MAX_HEIGHT; i++) {
  COL_LOOP: For(int j=0; j< MAX_WIDTH; j++) {
    #pragma HLS PIPELINE
    // Main computation per pixel
  }
}

The above example shows a nested loop structure with two loops that performs some computation on incoming pixel data. In most cases, you want to process a pixel in every cycle, hence PIPELINE is applied to the nested loop body structure. The compiler is able to flatten the nested loop structure in the example because it is a perfect nested loop.

The nested loop in the preceding example contains no logic between the two loop declarations. No logic is placed between the ROW_LOOP and COL_LOOP; all of the processing logic is inside the COL_LOOP. Also, both the loops have a fixed number of iterations. These two criteria help the xocc compiler flatten the loops and apply the PIPELINE constraint.

void adder(unsigned int *in, unsigned int *out, int inc, int size) {

  unsigned int in_internal[MAX_SIZE];
  unsigned int out_internal[MAX_SIZE];
  mem_rd: for (int i = 0 ; i < size ; i++){
    #pragma HLS PIPELINE
    // Reading from the input vector "in" and saving to internal variable
    in_internal[i] = in[i];
  }
  compute: for (int i=0; i<size; i++) {
  #pragma HLS PIPELINE
    out_internal[i] = in_internal[i] + inc;
  } 

  mem_wr: for(int i=0; i<size; i++) {
  #pragma HLS PIPELINE
    out[i] = out_internal[i];
  }
}

In the previous example, three sequential loops are shown: mem_rd, compute, and mem_wr.

• The mem_rd loop reads input vector data from the memory interface and stores it in internal storage.
• The main compute loop reads from the internal storage and performs an increment operation and saves the result to another internal storage.
• The mem_wr loop writes the data back to memory from the internal storage.

This code example is using two separate loops for reading and writing from/to the memory input/output interfaces to infer burst read/write.

By default, these loops are executed sequentially without any overlap. First, the mem_rd loop finishes reading all the input data before the compute loop starts its operation. Similarly, the compute loop finishes processing the data before the mem_wr loop starts to write the data. However, the execution of these loops can be overlapped, allowing the compute (or mem_wr) loop to start as soon as there is enough data available to feed its operation, before the mem_rd (or compute) loop has finished processing its data.

The loop execution can be overlapped using dataflow optimization as described in Dataflow Optimization.

In the dataflow coding example you should notice the following:

1. The HLS DATAFLOW pragma is applied to instruct the compiler to enable dataflow optimization. This is not a data mover, which deals with interfacing between the PS and PL, but how the data flows through the accelerator.
2. The stream class is used as a data transferring channel between each of the functions in the dataflow region.
   TIP: The stream class infers a first-in first-out (FIFO) memory circuit in the programmable logic. This memory circuit, which acts as a queue in software programming, provides data-level synchronization between the functions and achieves better performance. For additional details on the hls::stream class, see the Vivado Design Suite User Guide: High-Level Synthesis (UG902).

void compute_kernel(ap_int<256> *inx, ap_int<256> *outx, DTYPE alpha) {
  hls::stream<unsigned int>inFifo;
  #pragma HLS STREAM variable=inFifo depth=32
  hls::stream<unsigned int>outFifo; 
  #pragma HLS STREAM variable=outFifo depth=32

  #pragma HLS DATAFLOW
  read_data(inx, inFifo);
  // Do computation with the acquired data
  compute(inFifo, outFifo, alpha);
  write_data(outx, outFifo);
  return;
}

The following behaviors can prevent the xocc compiler from performing DATAFLOW optimizations:

1. Single producer-consumer violations.
2. Bypassing tasks.
3. Feedback between tasks.
4. Conditional execution of tasks.
5. Loops with multiple exit conditions or conditions defined within the loop.

If any of the above conditions occur inside the dataflow region, you might need to re-architect the code to successfully achieve dataflow optimization.