A survey of 1,133 engineers on HLS vs. manual RTL time savings

http://www.deepchip.com/items/0488-04.html

From: Shawn McCloud <shawn_mccloud=user domain=mentor hot calm>
Subject: A survey of 1,133 engineers on HLS vs. manual RTL time savings

Hi John,

For the third year in a row, Mentor's Catapult group ran its annual blind 
worldwide survey on High Level Synthesis (HLS), following the ones in 2010 
and 2009.  This year a total of 1,133 engineers responded.

We asked for input on three new items:

     1. HLS time savings versus manual RTL implementation
     2. Length of delays due to late functional changes and bug fixes
     3. Preferred abstraction level for different design types

For the percentages below, the numbers are based on all ~1100 respondents, 
unless otherwise noted.

HLS TIME VERSUS MANUAL RTL IMPLEMENTATION

     Question #1: "If your organization currently uses HLS tools, what 
     overall time savings have you realized in getting to RTL (including 
     verification) versus manual approaches?"

          HLS longer than manual            : ## 3%
          HLS zero savings                  : # 2%
          <20%                              : ##### 9%
          20-40%                            : ####### 14%
          40-60%                            : ### 6% 
          60-80%                            : ## 3%
          >80%                              : # 2%
          N/A no experience with HLS        : ################ 32%
          Don't know                        : ############### 29%

When we look closely at this data, we see that an impressive 39% (452) of
the 1,133 respondents said that their organization had HLS experience.

     Breakdown of HLS experience:

      Organization had HLS experience       : #################### 39%
      Organization had no HLS experience    : ################ 32%
      Don't know                            : ############### 29%

Below is the time savings feedback from the 39% of respondents (452
engineers) whose organization had High Level Synthesis experience, and who 
knew the relative implementation times between deploying an HLS versus a 
manual RTL implementation.  All time savings numbers reflect the time to 
implement RTL through verification.

 HLS versus manual RTL time savings

 HLS had time savings    : ############################################ 88%
 HLS zero time savings   : ## 4%
 HLS longer than manual  : #### 8%

A whopping 88% realized time savings from deploying HLS in the design flow.
This is especially noteworthy given that many of these organizations are 
likely to still be in the early stages of setting up and deploying HLS.

For those respondents that specified the amount of time savings to reach 
verified RTL, the average was an impressive 35% using HLS versus manual RTL.

     HLS Implementation Time Savings vs. Manual RTL

          <20%            : ############## 27%
          20-40%          : ##################### 41%
          40-60%          : ######## 16%
          60-80%          : ##### 10%
          >80%            : ### 6%

LENGTH OF DELAYS DUE TO LATE FUNCTIONAL CHANGES AND BUG FIXES WITH 
EXISTING RTL METHODOLOGY

     Question #2: "If your organization experienced a project/tapeout 
     delay due to late functional changes or bug fixes, what was the
     total length of the delay?"
     
          Haven't had delays        : #### 8%
          < 2 weeks                 : ###  7%
          2 weeks to 1 month        : ########## 19%
          1-3 months                : ############## 27%
          3-6 months                : ####### 13%
          >6 months                 : ## 4%
          Don't know                : ########### 21%

For those that had experienced project delays associated with late 
functional changes or bug fixes, the average delay was 2.2 months.  This is 
highly relevant to an HLS discussion because the time savings for using HLS 
go up considerably for late design changes.  Late functional changes and 
fixes can be easily made and verified at the C level, then automatically 
implemented as RTL using HLS.  A good HLS tool can dramatically reduce the 
time from the change to verified RTL.

PREFERRED LEVEL OF HLS ABSTRACTION FOR DESIGN IMPLEMENTATION

We also asked users to cite their preferred level of abstraction for HLS, 
both for implementing algorithms, and for implementing control logic. 
Below is what they told us.

     Question #3: "What is the most beneficial level of abstraction
     for HLS for implementing ALGORITHMS?"

          Untimed                : ################ 32%
          Partially Timed        : ############### 30%
          Cycle Accurate         : ############# 26%
          Don't Know             : ###### 12%

When implementing algorithms, the higher the level of abstraction the more 
favorable the sentiment, with untimed HLS input languages taking the top 
spot at 32%, followed by partially timed at 30%, for a total of 62%  
favoring untimed or partially timed input language.  Only 26% picked cycle 
accurate as the preferred abstraction level.

The responses were reversed for control logic.

     Question #4: "What is the most beneficial level of abstraction
     for HLS for implementing CONTROL LOGIC?"

          Cycle Accurate         : ####################### 46%
          Partially Timed        : ############### 29%
          Untimed                : ###### 12%
          Don't Know             : ###### 12%

The most beneficial level of abstraction for implementing control logic was 
considered to be the cycle accurate level (46%), followed by partially 
timed (29%), and untimed (12%).

The survey indicates the preferred abstraction level varies by type of 
design (control centric versus algorithm) and even then there is not a 
complete consensus for a particular level.  Mentor recognizes that 
different design teams have varying preferences as to which abstraction 
level they deploy.  Thus we fully support all these levels with our C++ and 
SystemC support for our Catapult C synthesis tool.

For example, in ESNUG 486 #2, Tim Koeppe of Nokia Siemens Network chose
untimed C++ for his control logic implementation.  Further, Tim is one
of the people that sees a 80%+ time savings for HLS:

   "I estimate that from start to finish, it's about 5 to 10 times faster
    to use Catapult than to do hand coding for a given application...

   "The time savings is not just about automatically converting the 
    C-code to RTL versus writing it manually.  The other powerful thing is 
    Catapult's verification flow.  Catapult offers an environment for 
    quick verification of the generated code - it automatically verifies 
    the RTL without me having to write an RTL-testbench."

What Tim and the recent HLS survey results both point to is that the 
fundamental benefit of HLS is in the overall reduction of the time to 
verified RTL.  An HLS tool must be assessed not only in terms of quality of 
results, but also and most importantly in terms of time-to-results, 
including verification of the RTL.  Time-to-results encompasses a number of 
attributes of an HLS solution including maturity of product, flexibility in 
input language, ease of analysis, user control and links to the full tool 
chain.  This is what you'll want to measure and evaluate when choosing your 
HLS tool.

    - Shawn McCloud
      Mentor Graphics                            Wilsonville, OR