When is robustness verification for DO-254 projects complete?

Janusz Kitel, DO-254 Program Manager
Like(6)  Comments  (0)

Understandably, hardware designed for an aircraft, or indeed any safety critical application, must be robust. I also believe that all engineers wish to verify their designs as thoroughly as possible, anyway. However, there are limiting factors; most notably the high complexity of most designs. Since we are unable to discover and verify the design against all abnormal conditions, the main question is: when is robustness verification truly complete?


Random nature of robustness

Test scenarios for robustness verification always contain many input stability issues, such as erroneous values, lack or loss of value, unexpected timing and unpredicted toggling. Certainly, there is a significant random factor but it should not lead us to oversimplify this part of verification by applying less or more advanced randomization methods only. The robustness of any design, and especially for projects where human lives are potentially at risk, cannot be achieved by inspecting the results of randomly generated scenarios. It must be part of the original design.



The “Design Assurance Guidance for Airborne Electronic Hardware” document does not explicitly address robustness testing. However, two supplements - “FAA Order 8110.105A” and the “EASA Certification Memorandum” - clarify that to demonstrate robustness, the applicant should also define the requirements-based testing. In other words, it is expected that abnormal operating conditions be captured and documented as derived requirements.


How many “robustness requirements” do we need?

Having extra requirements for robustness verification does not solve the problem stated above, so the question really is: how many “robustness requirements” will be enough?


To capture the requirements, the system specification must be analyzed. We need to look deeper into the non-functional, quality, and performance requirements. The second important source of “robustness requirements” is the system design where any unpredicted behavior that might result in functional degradation should be considered and documented as derived requirements.


From robustness test cases to higher quality of requirements

Test cases for robustness verification will not derive from the “robustness requirements” alone. They also need to come from the regular functional requirements; and it is this combination that can reveal inadequacies in the functional specification. This is one of the situations where I really appreciate the RTCA/DO-254 guidance and the defined processes, because the product cannot be safe without having the highest quality of requirements.


Robustness Hardware Verification

DO-254 requires that the hardware requirements are verified in real hardware and should be done in the operating environment (i.e. LRU, CCA). However, it may be very difficult or even impossible to verify some robustness test cases in the operating environment.


Fortunately, we may streamline this task by using the standalone test equipment like DO-254/CTS. This equipment allows us to verify all functional requirements (including robustness) in the target device and at real speed. All can be done without struggling with test procedures by reusing test benches prepared for the simulation. In addition, DO-254/CTS provides the features to perform the verification with normal and abnormal power supply and timing conditions.

Janusz Kitel is a DO-254 Program Manager at Aldec. He is responsible for FPGA level in-target testing technology and requirements lifecycle management for DO-254 and other safety-critical industry standards. Janusz has 7 years of experience in requirements engineering and over 12 years of experience in product quality assurance. Janusz received his M.S. in Electronics and Telecommunication from Silesian University of Technology and increased his knowledge around software engineering from complementary studies at AGH University of Science and Technology (Poland). His practical engineering experience includes the areas of functional verification, DO-254 compliance and software development and he has held a wide range of engineering positions that include Application Engineer, Software Developer and Project Manager.


Ask Us a Question
Ask Us a Question
Captcha ImageReload Captcha
Incorrect data entered.
Thank you! Your question has been submitted. Please allow 1-3 business days for someone to respond to your question.
Internal error occurred. Your question was not submitted. Please contact us using Feedback form.
We use cookies to ensure we give you the best user experience and to provide you with content we believe will be of relevance to you. If you continue to use our site, you consent to our use of cookies. A detailed overview on the use of cookies and other website information is located in our Privacy Policy.