FDA Raises Bar on Medical Device Software Testing
Blog Medical Devices Today did a post last week on the FDA’s adoption of a relatively new software testing technology in the CDRH’s software forensics lab.
In the past year or so, the forensics lab, housed within the Office of Science and Engineering Laboratories, has used high-power instruments and techniques to analyze “source code” in roughly a half dozen medical devices whose software was thought to be responsible for causing serious adverse events.
The analysis, which employ tools routinely used in the automotive and aeronautical industries, exposed several software design errors linked to the adverse events and at least one latent error that had the potential to cause an injury, according to Al Taylor, the Director of OSEL’s Division of Electrical and Software Engineering.
The story hints strongly about the FDA’s keen interest in seeing medical device vendors adopt this technology as part of their verification test in product development. Why is this a big deal? In 1996 10% of medical devices recalls were caused by software-related issues. In June of 2006, software errors in medical devices made up 21% of recalls.
The problem with testing software is that even very basic applications can have so many variables and permutations of outcomes that they become virtually impossible to test with conventional means. This is one of the main reasons that the FDA has focused their regulations on ensuring that vendors follow accepted design processes that maximize resulting quality, rather than focusing on verification and validation testing. The Quality System regulation is the manifestation of this approach.
But about a decade ago, another potential approach emerged. In 1996, a European space rocket, the Ariane 5, exploded just after launch because of a computer bug. Scientists made quick use of a sophisticated programming tool called “static analysis” to figure out what went wrong.
That experience validated the strategy as a useful tool for automating the search for hard-to-find coding vulnerabilities in complex software. It has since been embraced by the aeronautical and automotive sectors to help minimize errors and accidents. Now FDA is trying to bring medical device manufacturers into the fold.
A number of recent recalls (St Jude Medical Merlin PCS/Identity pacemakers, Baxter Colleague infusion pump, Difibtech’s AED) were all software related. Right now the CDRH software forensics lab is used to test vendor’s proposed fixes for recalled devices. In addition to verifying the vendor’s fix the static analysis also uncovers previously undiscovered software bugs.
According to the Deputy Director of the Division of Electrical and Software Engineering, Brian Fitzgerald, “No manufacturer wants to be in a position where FDA [is] finding bugs in his code,” he said. “And so we treat these very, very privately.” Indeed.
The story goes on to emphasize the value static analysis would have on premarket analysis. The story suggests that if medical device vendors don’t get with the program and adopt static analysis and/or other more rigorous software testing methodologies, the FDA might have to start doing premarket static analysis themselves – would very probably adversely affect premarket approval time frames.
This past summer I attended the workshop, Improving Patient Safety Through Medical Device Interoperability and High Confidence Software, in Boston. Much of this conference dealt with the challenges and solutions available to better test medical device software. Static analysis was discussed in a number of sessions. The bottom line from the conference is that the medical device industry has not kept up with other high reliability industries, especially aviation. (You can read posts from the conference here, here, here, here and here.)
I fully expect static analysis and other similar techniques to play an increasing role in medical device product development – mainly because it seems the FDA has similar expectations. Certainly the big vendors will adopt these techniques to:
- Differentiate their products (claims of improved safety),
- Avoid the considerable costs of preventable product recalls, and
- Better influence time-to-market schedules by removing the need for the FDA to do their own static analysis prior to premarket approval.
These advanced testing methods will also be promoted and offered by contract engineering firms. Firms like HighRely, that work in both aerospace and medical devices, can offer static analysis and other high confidence software testing methods. This will enable smaller medical device firms to benefit from these testing methods without investing in the overhead to do it in house.
Pictured right is a graphic report from a static analysis of software from the Centre for Software Engineering in Dublin, Ireland.