Optimizing the Hardware Function

Hardware functions are synthesized in the PL by the VivadoHLS tool compiler. This compiler automatically translates C/C++ code into an FPGA hardware implementation, and as with all compilers, does so using compiler defaults.

In addition to the compiler defaults, the HLS tool provides a number of optimizations that are applied to the C/C++ code through the use of pragmas in the code. This chapter explains the optimizations that can be applied and a recommended methodology for applying them.

There are two flows for optimizing the hardware functions:

Top-down flow

In this flow, program decomposition into hardware functions proceeds top-down within the SDSoC environment, letting the system cross-compiler create pipelines of functions that automatically operate in dataflow mode. The microarchitecture for each hardware function is optimized using the HLS tool.

Bottom-up flow

In this flow, the hardware functions are optimized in isolation from the system using the HLS tool compiler provided in the Vivado Design Suite. The hardware functions are analyzed, optimizations directives can be applied to create an implementation other than the default, and the resulting optimized hardware functions are then incorporated into the SDSoC environment.

The bottom-up flow is often used in organizations where the software and hardware are optimized by different teams and can be used by software programmers who wish to take advantage of existing hardware implementations from within their organization or from partners. Both flows are supported, and the same optimization methodology is used in either case. Both workflows result in the same high-performance system. Xilinx sees the choice as a workflow decision made by individual teams and organizations and provides no recommendation on which flow to use.

The optimization methodology for hardware functions is shown in the following figure:

Figure: Hardware Function Optimization Methodology

This figure details all the steps in the methodology and the subsequent sections in this chapter explain the optimizations in detail.

• Step 1: See Optimizing Metrics, and review the topics in this chapter prior to attempting to optimize.
• Step 2: See Pipelining for Performance
• Step 3: See Optimizing Structures for Performance
• Step 4: See Reducing Latency. This step is used to minimize or specifically control the latency through the design and is required only for applications where this is of concern.
• Step 5: See Reducing Area. This topic explains how to reduce the resources required for hardware implementation and is typically applied only when larger hardware functions fail to implement in the available resources. The FPGA has a fixed number of resources, and there is typically no benefit in creating a smaller implementation if the performance goals have been met.

Before you perform any hardware function optimization, it is important to understand the performance achieved with the existing code and compiler defaults, and appreciate how performance is measured. Select the functions to implement hardware and build the project.

After you build a project, a report is available in the Hardware Reports section of the IDE. The report is also available from <project name>/<build_config>/_sds/vhls/<hw_function>/solution/syn/report/<hw_function>.rpt. This report details the performance estimates and usage estimates.

The key factors in the performance estimates are ordered by the timing, interval (which includes loop initiation interval), and latency.

• The timing summary shows the target and estimated clock period. If the estimated clock period is greater than the target, the hardware will not function at this clock period. Reduce the clock period by using the Project Settings > Data Motion Network Clock Frequency option. Alternatively, because this is only an estimate at this point in the flow, it might be possible to proceed through the remainder of the flow if the estimate only exceeds the target by 20%. Further optimizations are applied when the bitstream is generated, and it might still be possible to satisfy the timing requirements. However, this is an indication that the hardware function is not guaranteed to meet timing.
• The function initiation interval (II) is the number of clock cycles before the function can accept new inputs and is generally the most critical performance metric in any system. In an ideal hardware function, the hardware processes data at the rate of one sample per clock cycle. If the largest data set passed into the hardware is size <N> (for example: my_array[<N>]), the most optimal II is <N> + 1. This means the hardware function processes <N> data samples in <N> clock cycles and can accept new data one clock cycle after all <N> samples are processed. It is possible to create a hardware function with an II <N>; however, this requires greater resources in the programmable logic (PL) with typically little benefit. Often, this hardware function is ideal because it consumes and produces data at a rate faster than the rest of the system.
• The loop initiation interval is the number of clock cycles before the next iteration of a loop starts to process data. This metric becomes important as you delve deeper into the analysis to locate and remove performance bottlenecks.
• The latency is the number of clock cycles required for the function to compute all output values. This is simply the lag from when data is applied until when it is ready. For most applications this is of little concern, especially when the latency of the hardware function vastly exceeds that of the software or system functions, such as DMA; however, it is a performance metric that you should review and confirm is not an issue for your application.
• The loop iteration latency is the number of clock cycles it takes to complete one iteration of a loop, and the loop latency is the number of cycles to execute all iterations of the loop. See Optimizing Metrics.

The Area Estimates section of the report details how many resources are required in the PL to implement the hardware function and how many are available. The key metric here is the Utilization (%). The utilization (%) should not exceed 100% for any of the resources. A figure greater than 100% means there are not enough resources to implement the hardware function, and a larger FPGA device might be required. As with the timing, at this point in the flow, this is an estimate. If the numbers are only slightly over 100%, it might be possible for the hardware to be optimized during bitstream creation.

You should already have an understanding of the required performance of your system and what metrics are required from the hardware functions; however, even if you are unfamiliar with hardware concepts such as clock cycles, you are now aware that the highest performing hardware functions have an II = <N> + 1, where <N> is the largest data set processed by the function. With an understanding of the current design performance and a set of baseline performance metrics, you can now proceed to apply optimization directives to the hardware functions.

The following table shows the first directive for you to consider adding to your design.

A common issue when hardware functions are first compiled is report files showing the latency and interval as a question mark “?” rather than as numerical values. If the design has loops with variable loop bounds, the compiler cannot determine the latency or II and uses the “?” to indicate this condition. Variable loop bounds are where the loop iteration limit cannot be resolved at compile time, as when the loop iteration limit is an input argument to the hardware function, such as variable height, width, or depth parameters.

To resolve this condition, use the hardware function report to locate the lowest level loop, which fails to report a numerical value, and use the LOOP_TRIPCOUNT directive to apply an estimated tripcount. The tripcount is the minimum, average, and/or maximum number of expected iterations. This allows values for latency and interval to be reported and allows implementations with different optimizations to be compared.

Because the LOOP_TRIPCOUNT value is only used for reporting and has no impact on the resulting hardware implementation, any value can be used. However, an accurate expected value results in more useful reports.

The next stage in creating a high-performance design is to pipeline the functions, loops, and operations. Pipelining results in the greatest level of concurrency and a very high level of performance. The following table shows the directives you can use for pipelining.

At this stage of the optimization process, you want to create as much concurrent operation as possible. You can apply the PIPELINE directive to functions and loops. You can use the DATAFLOW directive at the level that contains the functions and loops to make them work in parallel. Although rarely required, the RESOURCE directive can be used to squeeze out the highest levels of performance.

A recommended strategy is to work from the bottom up and be aware of the following:

• Some functions and loops contain sub-functions. If the sub-function is not pipelined, the function above it might show limited improvement when it is pipelined. The non-pipelined sub-function will be the limiting factor.
• Some functions and loops contain sub-loops. When you use the PIPELINE directive, the directive automatically unrolls all loops in the hierarchy below. This can create a great deal of logic. It might make more sense to pipeline the loops in the hierarchy below.
• For cases where it does make sense to pipeline the upper hierarchy and unroll any loops lower in the hierarchy, loops with variable bounds cannot be unrolled, and any loops and functions in the hierarchy above these loops cannot be pipelined. To address this issue, pipeline these loops with variable bounds, and use the DATAFLOW optimization to ensure the pipelined loops operate concurrently to maximize the performance of the tasks that contains the loops. Alternatively, rewrite the loop to remove the variable bound. Apply a maximum upper bound with a conditional break.

The basic strategy at this point in the optimization process is to pipeline the tasks (functions and loops) as much as possible. For detailed information on which functions and loops to pipeline, see Hardware Function Pipeline Strategies.

Although not commonly used, you can also apply pipelining at the operator level. For example, wire routing in the FPGA can introduce large and unanticipated delays that make it difficult for the design to be implemented at the required clock frequency. In this case, you can use the RESOURCE directive to pipeline specific operations such as multipliers, adders, and block RAM to add additional pipeline register stages at the logic level and allow the hardware function to process data at the highest possible performance level without the need for recursion.

C code can contain descriptions that prevent a function or loop from being pipelined with the required performance. This is often implied by the structure of the C code or the default logic structures used to implement the PL. In some cases, this might require a code modification, but in most cases these issues can be addressed using additional optimization directives.

The following example shows a case where an optimization directive is used to improve the structure of the implementation and the performance of pipelining. In this initial example, the PIPELINE directive is added to a loop to improve the performance of the loop. This example code shows a loop being used inside a function.

#include "bottleneck.h"
dout_t bottleneck(...) {
  ...
  SUM_LOOP: for(i=3;i<N;i=i+4) {
#pragma HLS PIPELINE
    sum += mem[i] + mem[i-1] + mem[i-2] + mem[i-3];
  }
  ...
}

When the code above is compiled into hardware, the following message appears as output:

INFO: [SCHED 61] Pipelining loop 'SUM_LOOP'.
WARNING: [SCHED 69] Unable to schedule 'load' operation ('mem_load_2', bottleneck.c:62) on array 'mem' due to limited memory ports.
INFO: [SCHED 61] Pipelining result: Target II: 1, Final II: 2, Depth: 3.
I

The issue in this example is that arrays are implemented using the efficient block RAM resources in the PL. This results in a small, cost-efficient, fast design. The disadvantage of block RAM is that, like other memories, such as DDR or SRAM, they have a limited number of data ports, typically a maximum of two.

In the code above, four data values from mem are required to compute the value of sum. Because mem is an array and implemented in a block RAM that only has two data ports, only two values can be read (or written) in each clock cycle. With this configuration, it is impossible to compute the value of sum in one clock cycle and thus consume or produce data with an II of 1 (process one data sample per clock).

The memory port limitation issue can be solved by using the ARRAY_PARTITION directive on the mem array. This directive partitions arrays into smaller arrays, improving the data structure by providing more data ports and allowing a higher performance pipeline.

With the additional directive shown below, array mem is partitioned into two dual-port memories so that all four reads can occur in one clock cycle. There are multiple options to partitioning an array. In this case, cyclic partitioning with a factor of two ensures the first partition contains elements 0, 2, 4, and so forth, from the original array and the second partition contains elements 1, 3, 5, and so forth. Because the partitioning ensures there are now two dual-port block RAMs (with a total of four data ports), this allows elements 0, 1, 2, and 3 to be read in a single clock cycle.

#include "bottleneck.h"
dout_t bottleneck(...) {
#pragma HLS ARRAY_PARTITION variable=mem cyclic factor=2 dim=1
  ...
  SUM_LOOP: for(i=3;i<N;i=i+4) {
#pragma HLS PIPELINE
    sum += mem[i] + mem[i-1] + mem[i-2] + mem[i-3];
  }
  ...
}

Other such issues might be encountered when trying to pipeline loops and functions. The following table lists the directives that are likely to address these issues by helping to reduce bottlenecks in data structures.

In addition to the ARRAY_PARTITION directive, the configuration for array partitioning can be used to automatically partition arrays.

The DEPENDENCE directive might be required to remove implied dependencies when pipelining loops. Such dependencies are reported by message SCHED-68.

@W [SCHED-68] Target II not met due to carried dependence(s)

The INLINE directive removes function boundaries. This can be used to bring logic or loops up one level of hierarchy. It might be more efficient to pipeline the logic in a function by including it in the function above it, and merging loops into the function above them where the DATAFLOW optimization can be used to execute all the loops concurrently without the overhead of the intermediate sub-function call. This might lead to a higher performing design.

The UNROLL directive might be required for cases where a loop cannot be pipelined with the required II. If a loop can only be pipelined with II = 4, it will constrain the other loops and functions in the system to be limited to II = 4. In some cases, it might be worth unrolling or partially unrolling the loop to create more logic, and remove a potential bottleneck. If the loop can only achieve II = 4, unrolling the loop by a factor of 4 creates logic that can process four iterations of the loop in parallel and achieve II = 1.

The configuration commands are used to change the optimization default settings and are only available from within the Vivado HLS tool when using a bottom-up flow. For more details, see the Vivado Design Suite User Guide: High-Level Synthesis (UG902). If optimization directives cannot be used to improve the initiation interval, it might require changes to the code. See examples of such in the same guide.

When the compiler finishes minimizing the initiation interval (II), it automatically seeks to minimize the latency. The optimization directives listed in the following table can help specify a particular latency or inform the compiler to achieve a latency lower than the one produced, namely, instruct the compiler to satisfy the latency directive even if it results in a higher II. This could result in a lower performance design.

Latency directive are generally not required because most applications have a required throughput but no required latency. When hardware functions are integrated with a processor, the latency of the processor is generally the limiting factor in the system.

If the loops and functions are not pipelined, the throughput is limited by the latency because the task does not start reading the next set of inputs until the current task has completed.

The loop optimization directives can be used to flatten a loop hierarchy or merge consecutive loops together. The benefit to the latency is due to the fact that it typically costs a clock cycle in the control logic to enter and leave the logic created by a loop. The fewer the number of transitions between loops, the lesser number of clock cycles a design takes to complete.

In hardware, the number of resources required to implement a logic function is referred to as the design area. Design area also refers to how much area the resource used on the fixed-size PL fabric. The area is important when the hardware is too large to be implemented in the target device, and when the hardware function consumes a very high percentage (> 90%) of the available area. This can result in difficulties when trying to wire the hardware logic together because the wires themselves require resources.

After meeting the required performance target or initiation interval (II), the next step might be to reduce the area while maintaining the same performance. This step can be optimal because there is no advantage to reducing the area if the hardware function is operating at the required performance, and no other hardware functions are to be implemented in the remaining space in the PL.

The most common area optimization is the optimization of dataflow memory channels to reduce the number of block RAM resources required to implement the hardware function. Each device has a limited number of block RAM resources.

If you used the DATAFLOW optimization, and the compiler cannot determine whether the tasks in the design are streaming data, it implements the memory channels between dataflow tasks using ping-pong buffers. These require two block RAMs each of size <N>, where <N> is the number of samples to be transferred between the tasks (typically the size of the array passed between tasks). If the design is pipelined and the data is streaming from one task to the next with values produced and consumed in a sequential manner, you can greatly reduce the area by using the STREAM directive to specify that the arrays are to be implemented in a streaming manner that uses a simple FIFO for which you can specify the depth. FIFOs with a small depth are implemented using registers and the PL fabric has many registers.

For most applications, the depth can be specified as 1, which results in the memory channel being implemented as a simple register. If the algorithm implements data compression or extrapolation, where some tasks consume more data than they produce or produce more data than they consume, some arrays must be specified with a higher depth:

• For tasks which produce and consume data at the same rate, specify the array between them to stream with a depth of 1.
• For tasks which reduce the data rate by a factor of X-to-1, specify arrays at the input of the task to stream with a depth of X. All arrays prior to this in the function should also have a depth of X to ensure the hardware function does not stall because the FIFOs are full.
• For tasks which increase the data rate by a factor of 1-to-Y, specify arrays at the output of the task to stream with a depth of Y. All arrays after this in the function should also have a depth of Y to ensure the hardware function does not stall because the FIFOs are full.

The following table lists the other directives and configurations to consider when attempting to minimize the resources used to implement the design.

The ALLOCATION and RESOURCE directives are used to limit the number of operations and to select which cores (hardware resources) are used to implement the operations. For example, you could limit the function or loop to using only one multiplier, and specify it to be implemented using a pipelined multiplier.

If the ARRAY_PARITION directive is used to improve the initiation interval you might want to consider using the ARRAY_RESHAPE directive instead. The ARRAY_RESHAPE optimization performs a similar task to array partitioning, however, the reshape optimization recombines the elements created by partitioning into a single block RAM with wider data ports. This might prevent an increase in the number of block RAM resources required.

If the C code contains a series of loops with similar indexing, merging the loops with the LOOP_MERGE directive might allow some optimizations to occur. Finally, in cases where a section of code in a pipeline region is only required to operate at an initiation interval lower than the rest of the region, the OCCURENCE directive is used to indicate that this logic can be optimized to execute at a lower rate.

Before performing any optimizations, it is recommended to create a new build configuration within the project. Using different build configurations allows one set of results to be compared against a different set of results. In addition to the standard Debug and Release configurations, custom configurations with more useful names (for example, Opt_ver1 and UnOpt_ver) might be created in the window using the Project Settings > Manage Build Configurations for the Project toolbar button.

Different build configurations allow you to compare not only the results, but also the log files and even output RTL files used to implement the FPGA (the RTL files are only recommended for users very familiar with hardware design).

The basic optimization strategy for a high-performance design is:

• Create an initial or baseline design.
• Pipeline the loops and functions. Apply the DATAFLOW optimization to execute loops and functions concurrently.
• Address any issues that limit pipelining, such as array bottlenecks and loop dependencies (with ARRAY_PARTITION and DEPENDENCE directives).
• Specify a specific latency or reduce the size of the dataflow memory channels and use the ALLOCATION and RESOURCES directives to further reduce area.

In summary, the goal is to always meet performance first, before reducing area. If the strategy is to create a design with the fewest resources, omit the steps to improving performance, although the baseline results might be very close to the smallest possible design.

Throughout the optimization process, it is highly recommended to review the console output (or log file) after compilation. When the compiler cannot reach the specified performance goals of an optimization, it automatically relaxes the goals (except the clock frequency) and creates a design with the goals that can be satisfied. It is important to review the output from the compilation log files and reports to understand what optimizations have been performed.

For specific details on applying optimizations, refer to Vivado Design Suite User Guide: High-Level Synthesis (UG902).

This section documents several fundamental optimization techniques to enhance hardware function performance using the Vivado HLS tool. These techniques include: function inlining, loop and function pipelining, loop unrolling, increasing local memory bandwidth, and streaming data flow between loops and functions.

After defining what function is needed for acceleration, there are a few key items to ensure compilation is valid. The Vivado HLS tool data types (ap_int, ap_uint, ap_fixed, etc.) cannot be part of the function parameter list that the software part of the application calls. These data types are unique to the HLS tool and have no bearing outside of the intended tool and associated compiler.

For example, if the following function was written in the HLS tool, the parameter list needs to be adjusted, and the function body has to handle moving the data from the HLS tool to a more generic data type, as shown below:

void foo(ap_int *a, ap_int *b, ap_int *c) { /* Function body */ }

This needs to be modified if using local variables:

void foo(int *a, int *b, int *c) {
	ap_int *local_a = a;
	ap_int *local_b = b;
	ap_int *local_c = c;
	// Remaining function body
}