Metric Driven Verification

Metric Driven Verification is a methodology based on metrics collections. It is used to improve the predictability, productivity, and quality of the verification effort.

In a nutshell, the methodology is based on four steps executed continuously until results fulfill assumed criteria:

• Planning - the creation and formalization of a document that contains specifications and details the designer’s intent. The document allows project managers or leaders to track goals assigned to each milestone and prioritize features included in the plan. We call this the Verification Plan.
• Constructing verifications environment - at this stage, engineers try to reuse existing verification IPs and Constraint Random based SystemVerilog UVM or OSVMM environment to create verification scenarios that cover the Verification Plan.
• Test scenarios execution - During this stage regression tests are executed for each revision of the project. Metrics like code or functional coverage are captured using ACDB technology. Results are mapped into the Verification Plan.
• Measure and analyze the process - this provides a clear picture of actual project status and cross references it against key milestones – and is when Riviera-PRO’s coverage feature is extremely useful.

Code Coverage

Functional Coverage - Assertions

Assertions and cover directives specify and validate the expected behavior of a design. They are written directly in the source code to observe signal values over a period of time.

Assertions declare that something must always hold (assertion failure implies a bug in the design).

Cover directives declare that something should occur (cover success confirms coverage).

The PSL cover directives and SystemVerilog cover statements are forms of Assertion Coverage. The specification of properties, and their use in assertions and functional coverage, is essential when designing a system and defining its verification algorithms. Aldec Simulators support three popular languages serving this purpose:

• SystemVerilog Assertions (SVA)

  SVA is a part of SystemVerilog and blends well with Verilog code, both at the module-binding level and in the behavioral code. Verification modules written in SVA can also be bound to VHDL components in the mixed-language simulator environment. Assertions Bundle is shipped by default in the high-end configurations of Aldec’s simulators. Once the bundle is available, all the related graphical tools become available as well: all assertions defined in the design can be displayed in a dedicated Assertion Viewer that allows for the changing of the properties of individual assertions, setting assertion breakpoints, and adding assertions to other debugging tools such as Waveform Viewer or Advanced Dataflow, both of which include powerful debugging features.

• Property Specification Language (PSL)

  PSL is the most thorough, yet easy to learn, property language. To help it coexist with other languages, it comes in flavors that blend nicely with VHDL, Verilog, SystemVerilog or SystemC code. The Simple Subset of PSL (the section of the language guaranteed to simulate) was even incorporated in the most recent version of the VHDL standard (IEEE Std 1076™-2008). The feature of PSL most interesting for current VHDL or Verilog users is its ability to be placed in the separate verification units managed by a Verification Engineer as well as directly into the HDL code managed by a Design Engineer.

• OpenVera Assertions (OVA)

  OVA blends better with Verilog code than with VHDL code, although it can be used with both. OpenVera was donated to the Accellera group working on SystemVerilog, leading to many similarities between OpenVera Assertions and SystemVerilog Assertions.

Functional Coverage - SystemVerilog Covergroups

Covergroup Coverage is a form of Functional Coverage that calculates SystemVerilog coverage model statistics. It is a user-defined metric that measures the percentage of design specification that has been examined by running the simulation session. Covergroup Coverage verifies interesting and relevant design features (listed within the verification plan) have been observed using the generated stimuli. In contrast with Assertion Coverage, which is used to verify design features over time, Covergroup Coverage is focused on covering relevant values accepted during the entire simulation. It is used for high-level verification of complex designs.

Functional Coverage - OSVVM

The Open Source VHDL Verification Methodology (OSVVM) library provides subprograms that facilitate implementation of the OSVVM Functional Coverage in VHDL designs. The CoveragePkg package included in this library offers a similar functional coverage verification as the SystemVerilog hardware description language counterpart.

Functional Coverage - FSM

A Finite State Machine consists of a set of states, input and output events, and logical conditions. A calculation based on the conditional logic, input events, and the current state returns a new set of output events and the next state. FSM Coverage allows the user to identify unvisited states and unevaluated transitions. The following FSM components and coverage statistics can be collected:

• State - an internal condition that determines the machine outputs.
• Transition - a change of the current machine state from one state to another.
• Sequence - a set of two or more transitions.

Coverage Database- Unified Coverage Interoperability Standard

Aldec Coverage DataBase (ACDB) is a unified storage format for several types of coverage data. ACDB is Aldec's implementation of Accellera UCIS (Unified Coverage Interoperability Standard) requirements

In Aldec’s simulators, all kinds of coverage statistics are collected and can be saved into the ACDB.

The coverage results saved to the ACDB can be ranked by using the acdb rank command.

Multiple ACDB files can be merged into one. This feature allows for the examination of statistics generated during different simulation sessions (plus report generation is available for merged ACDB files).

Ranking Test Results

The coverage results saved to an ACDB database can be ranked. This function classifies the results based on the contribution of individual tests to the total coverage score. The command allows comparing a single coverage result against all others to determine its usability. Cost-sensitive parameters, such as simulation time and CPU time, may also be taken into account to identify test runs that yield the greatest coverage value in the shortest amount of time. The acdb rank command operates on a union of contribution test results to produce the most comprehensive overall results. It is possible to create the report either in a plain text or in an HTML format

Verification Plan

Instead of focusing on detailed coverage results, the verification process aims at meeting overall requirements and achieving a higher level of validation.

Requirements can be used to link together various design features and coverage metrics providing an advantage over the basic verification methodology that focuses on inspecting detailed coverage results.