"The Gray Sheet" has an article called CDRH Software Forensics Lab: Applying Rocket Science To Device Analysis. Can it really be true that CDRH is doing static code analysis for detecting software defects in recall investigations?
Static code analysis has been around for a long time. I remember using Lint back in the old days. Nowadays all commonly used computer languages (like C and C++) have fairly advanced analysis tools. For Microsoft .NET languages there are tools like FxCop and ReShaper. These tools are great for spotting trouble areas and maintaining code conventions during the development process.
I just have a hard time imagining a process using these tools that would successfully detect real defects in complex medical device software. Especially software that's controlling hardware devices, doing communications tasks, and recording sensor data. Also, what about all of the assembly language code for embedded processors and DSPs?
Anyway, from the article:
A recent review in the forensics lab found 180 "questionable constructs" in 100,000 lines of code, but only two turned out to be real design issues, Taylor said.
He also pointed to two other cases where static analysis of the software did not find any bugs, thus clearing software as a root cause candidate in the recall investigations.
These statements beg the following questions:
- Were the two "real design issues" related at all to the device failures?
- Just because no static analysis "bugs" were found, why does that exonerate the software from being the cause of the failure?
I'm not impressed that static analysis has "been embraced by the aeronautical and automotive sectors". IMHO this approach just seems like it would create a lot of work for very little return. The manufacturer that produced the device should be responsible for tracking down and fixing the software (and/or hardware) defects.
I'll stop now.
UPDATE (25-Oct-07):
Check out Tim's post on this subject: FDA Raises Bar on Medical Device Software Testing.
[…] interesting one. The difference between CAPA and FRACAS is similar to the argument I made regarding Software Forensics — these techniques should be used to ensure quality before the product is released. […]
[…] discussed software forensics tools before. Yes, bad software has hurt and killed people, and there’s no excuse for it. I still […]
[…] Static Analysis to Find Bugs in the Real World More Software Forensics and Why Analogies Suck Medical Device Software Forensics Pascal’s 3 part static analysis series that starts here: Guest Article: Static Analysis in […]
While static code analysis is perhaps not the most efficient way of tracking down bugs causing device failure, it is a good tool to have in the developer’s toolbox.
Finding 180 parts in 100’000 lines of code warranting closer inspection and actually finding 2 defects (of unknown severity/consequence) is fairly good. Manually inspecting 100’000 lines of code takes days if not weeks. Using a static analysis tool manages to reduce the amount up for inspection down to something that can be reviewed in less than a day.
While this is not a safe guard against any device failure (verification & validation anyone?) – it is a good tool for improving overall software quality. Spending some time on grinding the analysis tools might even make it more efficient (and streamline it for ensuring a consistent style and quality of code within ones own guidelines).
A be all, end all – nope, but should be kept as one of several options for improving overall quality.