My First Example with OS-VVM CoveragePkg

A Guest Blog from Alex Grove of FirstEDA

Alex Grove, Alex Grove, Applications Specialist at FirstEDA
Like(3)  Comments  (1)

Here in Europe, I recently had the opportunity to work with Jim Lewis, OS-VVM Chief Architect and IEEE 1076 Working Group Chair, on the first Advanced VHDL Testbenches & Verification training course. This training, held in Bracknell, UK, was attended by engineers from several major European system companies who design and verify programmable devices (FPGAs). VHDL is by far the dominate language used by Europe’s system companies for the design and verification of FPGAs, however it is unclear to many how to enhance their verification with VHDL. What I have found is that experienced FPGA design engineers (including myself) are not utilising the VHDL language for verification.

 

Verification with VHDL

Jim Lewis introduces VHDL’s verification capabilities, including new VHDL 2008 features and the Open Source VHDL Verification Methodology (OSVVM). OSVVM provides a methodology for testbench development and verification packages that provide functional coverage and random value generation.

To demonstrate the use of the OSVVM functional coverage package ‘CoveragePkg’, I took a familiar Finite-state Machine (FSM) the IEEE 1149.1 TAP (Test Access Port) controller. This is a clever little state machine that I am rather fond of, having spent a lot of time working with it as part of my degree dissertation on Built-in Self Test (BIST).

So let’s take a look at our IEEE 1149.1 TAP FSM.

The TAP FSM transitions are controlled by the Test Mode Select (TMS) signal.  The TAP controls the JTAG system performing serial shift and parallel load of the data registers (DR) and instruction registers (IR).

 

 

blog_img_050614_rv01_625

Figure 1: TAP State Machine

Code Coverage vs Functional Coverage

To explain what functional coverage is and the value in terms of verification, let’s first take a look at code coverage.

Code coverage is a technique that uses instrumentation to record the execution of the code within the simulator.  Two well-known techniques are statement and branch coverage.  Statement coverage tells you which statements have been executed in simulation i.e. “hit”.  Branch coverage provides information about the branches taken of control structures such as ‘if-then-else’ statements.

So let’s run my example test scenario for the TAP FSM and take a look at the code coverage reports. 

blog_img_050614_c_448

Figure 2: Sample Code Coverage Reports

In Figure 2, statements that have not been “hit” are highlighted in red. From the report we can see that our test scenario does not test (cover) all the state transitions. This is what code coverage provides; it tells you what has not been tested, and is a very valuable metric for any verification process.  

However, we would be wrong to say we have tested (covered) all the “green” state transitions.  Why?  Code coverage records the execution of the code not the function and so is sensitive to delta cycles.  Let’s work through an example to illustrate what can happen.

Looking at the statement coverage report, you might think that we have exercised the two transitions of the SHIFT_DR state (i.e. SHIFT_DR to SHIFT_DR, and SHIFT_DR to EXIT1_DR).   For the test scenario the FSM is only ever in SHIFT_DR for one clock cycle.  So what happened?  Let’s take a look at the waveform (Figure 3) with the delta cycles enabled.

blog_img_050614_rv02_625

Figure 3: Waveform showing delta cycles

 

The TAP FSM consists of two VHDL processes; combinatorial next state logic and a sequential state register.   For this simple example I added a concurrent signal assignment (TMS_SIG) to TMS so as to introduce a delta cycle.  What has happened is the combinatorial next state process was run twice, once with TMS_SIG equal to ‘0’ and a second time with TMS_SIG equal to ‘1’.  This was due to the introduced delta cycle and the sensitivity list of the next state process.

The execution in the simulator is shown in Table 1.

blog_img_050614_e_390

Table 1: The execution of signal evaluation and update in the simulator

So the TAP FSM next state logic settled on EXIT1_DR.  The functional state transition that took place was SHIFT_DR to EXIT1_DR.  However, in the simulation delta cycles 2 and 3 we executed each branch of the if-then-else statement for the SHIFT_DR next state logic.

This is the difference between functional coverage and code coverage. Functional coverage collects the actual state transitions (the function) so as to build up a complete picture of how the FSM has been functionally exercised. This is achieved with a coverage model.  

 

Adding Functional Coverage with OS-VVM

Creating a coverage model with the OS-VVM package ‘CoveragePkg’ turned out to be very simple.  Let’s walk through the steps to create a coverage model for the TAP FSM. First you need the OS-VVM packages, these are included in the Aldec simulator and are available for download on the OS-VVM website.

Next we need to create a coverage object for the FSM, which is a protected type from ‘CoveragePkg’ and used to collect and report the functional coverage.

 blog_img_050614_rv03_801

 

So to observe the TAP FSM state register in the testbench we use a VHDL 2008 external name and alias this to the TAP_STATE.  

The TAP FSM uses an enumerated type which is defined in the package ‘TapTypes’.

 blog_img_050614_rv04_801

Next we use a process to define the valid state transitions and collect data - this is the coverage model for our TAP FSM.   

We can give our coverage object a unique name which is then used for reporting.

 blog_img_050614_h_507

To define the state transitions we use cross coverage which defines relationships between multiple objects, in this case the current and previous state of our TAP FSM. To do this we need to capture the state transitions in tabular form with ‘current’ and ‘next state’.  You could extract the FSM from the RTL as this is a relatively straightforward task, assuming you have a defined FSM coding style.  However as a verification model it would be considered good practice to use the original specification.

 blog_img_050614_i_714

VHDL is a very rich language and for the enumerated type ‘TapStateType’ we can use the predefined type attributes.  The coverage model is based on integer bins so we use the T’POS attribute to define an integer value for each state.

Once we have defined all the valid state transitions we mark the rest of the possible transitions as illegal.

 blog_img_050614_j_rv_775

Next we need to define our collector.  This will sample on the negative edge of the TCK clock and will keep track of the previous state.  The call of the protected type procedure ICover (method) accumulates the coverage.   We call ICover with an integer vector that represents the state transition.

 blog_img_050614_k_776

The last step is to generate a coverage report at the end of simulation.

 blog_img_050614_l_765

The reports shows a 78.125% coverage for the test scenario and as expected the SHIFT_DR -> SHIFT_DR transition is not covered. The report also confirms that the state PAUSE_DR was not covered.

 blog_img_050614_m_626_01

The test scenario that I am running is using another OS-VVM package ‘RandomPkg’ to stimulate the TAP FSM with a pseudo-random sequence for TMS.   The test scenario was run with a target of >75% coverage to demonstrate the coverage holes.  If we change this to 100% and rerun we will now hit 100% coverage in 119 cycles.

 blog_img_050614_n_774

Summary

We looked at the difference between code coverage and functional coverage and how to add functional coverage with VHDL OS-VVM ‘CoveragePkg’ to a FSM. Often customers raise concerns about the amount of work required to create a complete coverage model for a design. This can be significant and is one of the reasons why ASIC teams often have dedicated verification engineers.

With programmable devices, a complete coverage model is generally not as critical (unless the application is safety critical).  Using OS-VVM ‘CoveragePkg’ you can incrementally introduce functional coverage to your test environment to analyse your current test scenarios in terms of functional coverage.  This has the immediate benefit of providing additional coverage metrics for your current verification.  These metrics can be used to identify areas for improvement.   For example, they could be used to “grade” tests so that test scenarios with the greatest coverage are run first in a regression.

 

For more on OSVVM, visit www.osvvm.org.

Alex Grove has over 20 years’ experience in the EDA industry.  Alex has worked for Synopsys, ARM’s EDA business unit, Synplicity, Mentor Graphics and is currently employed as an Applications Specialist at FirstEDA.

Alex has extensive experience in the design and verification of ASICs and FPGAs, as well as a broad knowledge of the EDA industry.  After graduating from Aston University, with an honours degree in Electronic Engineering & Computer Science, Alex joined Synopsys Northern Europe to work on synthesis and test.  

During his time at Synplicity, Alex supported some of the most current complex FPGA designs and was a technical specialist for Synplicity’s ASIC synthesis and ASIC prototyping products.

At FirstEDA, the specialist European EDA distributor, Alex is a solutions expert on the Aldec line with a particular focus on the hardware-assisted verification and RTL simulation products.

Comments

Hi! Thank you for this nice example of using the CoveragePkg in FSM functional verification. I am just curious why in the last image you have in an illegal value if this transition (CAPTURE_IR -> SHIFT_IR) is defined in the state machine diagram.

Thanks!
Jorge Lucio T. over 7 years
Ask Us a Question
x
Ask Us a Question
x
Captcha ImageReload Captcha
Incorrect data entered.
Thank you! Your question has been submitted. Please allow 1-3 business days for someone to respond to your question.
Internal error occurred. Your question was not submitted. Please contact us using Feedback form.
We use cookies to ensure we give you the best user experience and to provide you with content we believe will be of relevance to you. If you continue to use our site, you consent to our use of cookies. A detailed overview on the use of cookies and other website information is located in our Privacy Policy.