Archive for the ‘FDA’ Category

Validation of Off-The-Shelf Software Development Tools

Saturday, November 12th, 2011

A reader asked me about OTS software tool validation. He says:

It seems to me that the editor and any other tool used to create the software is exactly that, a productivity tool. The end result (compiled binary installed on a validated PC configuration) is still going to go through verification and validation, therefore, it seems validating any of the items used to actually create the binary is unnecessary.

Any thoughts or guidance to help me understand this process?

This is a great question and the source of a lot of confusion.

The bottom line is that all third party tools (and libraries) used to construct or test FDA regulated software need to be validated.

You may think validating a compiler is unnecessary, but the FDA says otherwise -- section 6.3 of the FDA Guidance on General Principles of Software Validation discussion includes "off-the-shelf software development tools, such as software compilers, linkers, editors, and operating systems."

The form of the required documentation is detailed in the Off-The-Shelf Software Use in Medical Devices (PDF) guidance document.  Section 2.1 has the questions that the OTS software BASIC DOCUMENTATION needs to answer:

  1. What is it?
  2. What are the Computer System Specifications for the OTS Software?
  3. How will you assure appropriate actions are taken by the End User?
  4. What does the OTS Software do?
  5. How do you know it works?
  6. How will you keep track of (control) the OTS Software?
For most products (again, OTS tools and libraries, including open source products) this documentation is not as onerous as you might think.  #5 is where you apply the intended use validation to the specific product. I have done this for many products: Visual Studio,  Subversion/TortoiseSVN, NUnit, Log4Net, etc.  You also need to validate custom developed testing tools and fixtures.
Like it or not, this is the reality of developing FDA regulated software.
UPDATE (2/16/2012): See OTS/SOUP Software Validation Strategies.

Open Source Medical Device Connectivity

Sunday, October 2nd, 2011

In The Case for Open Source Healthcare IT John Zaleski uses the VistA open source software as a model for improving the medical device data gathering in order to produce a "more robust end product".  On the whole, I could not agree more. Achieving this in such a diverse and fragmented community would be a real challenge, but it may indeed be a worthwhile path to pursue.

There was one item I'd like to comment on:

The challenge is, of course, regarding regulatory management of open source frameworks. To a large degree open source software is anathema to the FDA regulatory process–and it relates to control and management of access.

OSS detested or loathed by the FDA? I don't think so. The open source framework itself would not be subject to regulatory control.  The FDA does not really care where any specific software component comes from.  Also, security and  access management are certainly important, but I'm not sure of their applicability at the device connectivity level.

The FDA cares about intended use, efficacy, and safety of medical devices. All FDA regulated software is subject to the same design controls (§820.30) -- design, risk analysis, verification, validation, etc. I.e. any company that included these open source software components would ultimately be responsible for proving these processes are followed.

Shahid N Shah's OSCon 2011 Talk: The implications of open source technologies in safety critical medical device platforms does a good job of detailing these points (Will the FDA accept open source in safety-critical system? Yes!) as well as presenting an OSS connectivity software architecture.

Third Annual Medical Device Connectivity Conference

Sunday, August 14th, 2011

This year’s Medical Device Connectivity Conference is less than a month away. It's being held Sept. 8-9, 2010 in Boston.


In addition to the post-conference workshops, there is also a special preconference event: an open house at the Medical Device Plug and Play Interoperability program’s lab. September 7, from 4pm to 6pm attendees can tour the lab, interact with various demonstrations and chat with program staff.

Tim has put together another great conference.

Discomfort with Computerized Medical Devices

Monday, April 11th, 2011

Here are some thoughts regarding the article: I feel a little uncomfortable about computerized medical devices, and here's why.

  • Just about all medical devices are computerized these days. Most will not harm or kill you if their software fails (Class I & II), but that's no excuse for writing crappy code.
  • As pointed out, the mission critical nature of mass transit systems (airplanes, subways, etc.) affords those industries a much higher level of scrutiny then cars or medical devices ever will. But that's still no excuse for writing crappy code.

Even through drugs and airplanes need advance approval from authorities before being brought to the market, medical devices and software do not, at least in the United States.

  • This statement is not correct. All medical devices, including diagnostic and therapeutic software-only products, require FDA clearance to be sold in the US market (see the Class I & II link above).  There are many exemptions, but a 510k pre-market approval is generally a minimum requirement. After you receive approval the FDA can pull your device off the market (the dreaded “recall”) at any time due to complaints or unsatisfactory audit results.
  • The FDA QSR Subpart C (§ 820.30) looks a lot like DO-178B as quality system design controls go, but I'm sure the aviation standard enforcement is far more rigorous (well, at least I hope it is). It's true, there are no coding standards for medical device software.  Good companies set their own development standards and practices -- some even use static analysis! It's all the other companies that don't bother to do anything that you have to worry about.

I'm certain that static analysis technology has improved vastly in the four years since some of the articles below were written.  The challenge is that the complexity of medical device software and the systems they run on has also increased tremendously during that time. In particular, the explosion of high bandwidth wireless networks along with advances in handheld computing power and graphics capability (think iPhone/iPad, of course) is fundamentally changing the way medical devices will be developed and delivered to the market in the future.

Static analysis will remain a valuable tool for identifying critical software defects, but new methods will have to be developed for rooting out risks in the new network-connected, multi-touch world.

It's sad to say, but you should probably be more than "a little uncomfortable."

Other static analysis articles:

A Few Billion Lines of Code Later: Using Static Analysis to Find Bugs in the Real World
More Software Forensics and Why Analogies Suck
Medical Device Software Forensics
Pascal's 3 part static analysis series that starts here:
Guest Article: Static Analysis in Medical Device Software (Part 1) — The Traps of C

Personal Healthcare Products: This is what the future looks like.

Tuesday, January 4th, 2011

I'm jealous of companies that get to produce diagnostic medical devices without having to go through the FDA 510(k) process. For example, the iHealth BP3 blood pressure monitor is a high-tech looking device with a  free Apple application:

Hopefully they're using an approved non-invasive blood pressure (NIBP) device like the SunTech module.

The built-in ability to e-mail results to family or a physician seems useful, but posting your blood pressure on Facebook or Twitter?  I don't know...

Hat Tip: medGadget

Agile Software Development in Regulated Environments

Saturday, October 16th, 2010

As part of a series on High Assurance Agile Development in Regulated Environments is the article
Agile Software Development in Regulated Environments Example: Medical Devices. The purpose of this article and future posts is to introduce the FDA regulatory landscape and then

... see what we can do to “agilify” our practices under these standards as we move forward.

It's been three years since I wrote Agile development in a FDA regulated setting.  I'll be interested to see if the application of "agile, high assurance activities" in this environment -- and the associated issues -- have changed since then.

UPDATE (10/23/10): Can and should agile be used for medical device development? Absolutely!

UPDATE (11/27/10): More discussion here: Can Agile Software Methods be used in medical device software development?

UPDATE (11/28/10): Agile Medical Device Software Development?

UPDATE (12/17/10): GE Healthcare Goes Agile

UPDATE (1/5/11): Missed this one: Four Reasons Medical Device Companies Need Agile Development

Technical Debt in Medical Software

Wednesday, August 4th, 2010

Software development is software development. Most of the life cycle and quality issues faced in medical software are the same challenges for any software product. Technical Debt in Medical Software points out what technical debt is:

  • Complexity
  • Code Duplication
  • Documentation Debt
  • Testing Debt
  • Architectural Debt

A Martin Fowler article is referenced that nicely identifies the source of technical debt:

The benefits of paying down the debt are:

  • Increased R&D efficiency and improved time to market
  • Hitting commitment dates
  • Performance and technology upgrades

Of course if you don't want to pay it off, there's always the option to go bankrupt. This may have long-term advantages, but it will surely be a more expensive route. There is one statement in this regard that I think needs some qualification:

In this case the technical debt can be retired along with the legacy system, and like filing Chapter 11, you are no longer responsible to address all the sins of the past.

I know this refers to code sins, but just because you decide to do a re-write doesn't mean you no longer have responsibility for the legacy product. You still have customers using the old software that you're obligated to continue to support.  For FDA approved medical software, this is a legal requirement. Most of the time this means that the legacy code will need to be maintained and periodically updated in the field, sometimes even after the "new" product is released. This just makes the cost of bankruptcy even higher.

ISO 62304: The Harmonized Standard for Medical Device Software Development

Saturday, June 5th, 2010

The FDA approved ISO 62304 as a recognized software development standard in 2009. Developing Medical Device Software to ISO 62304 gives a nice overview.

Besides providing a globally accepted development process one of the other practical components is the assignment of a safety class to individual software items and units:

  • Class A: No injury or damage to health is possible
  • Class B: Non-serious injury is possible
  • Class C: Death or serious injury is possible

Each classification changes the required documentation for the assigned software.

These standards will become more widely known as the FDA moves to regulate the proliferation of medical applications for personal and home use, most notably software that runs on mobile devices. I've discussed this before in When Cell Phones Become Medical Devices. As noted more recently in FDA oversight may extend throughout health IT:

... an FDA director stated flatly: "Under the Federal Food, Drug and Cosmetic Act, HIT software is a medical device."

Broad FDA oversight at the QSR/62304 level will probably not happen, but change is certainly coming for many HIT companies.

The Elsmar Cove Forum IEC 62304 - Medical Device Software Life Cycle Processes has a lot of discussion on this topic. This is where I found a document checklist that is useful for understanding the process scope:

IEC62304_Checklist.xls (Excel spreadsheet)

UPDATE (9/9/10): IEC 62304 – The Basics

The Software Quality Balancing Act

Saturday, May 15th, 2010

Andrew Dallas's article Caution: V&V May Be Hazardous to Software Quality touches on a number of good points regarding software quality best practices.

Medical device software development V&V (also see here) and the documentation that goes with it have substantial costs. Any strategy that can reduce this overhead and still meet the necessary quality standards should be seriously considered.

The use of "incremental" software development approaches really refers to Agile methodologies.  I've talked about the use of Agile for medical device software development several times:

Most of the discussion revolves around the risks associated with this approach. The benefits of any process change have to be weighed against the possible risks that might be introduced.

Besides the importance of understanding what V&V documentation the FDA actually wants to see, Andrew makes a great point about producing quality software versus the V&V process (my highlight):

V&V is not software testing. Verification testing ensures specified requirements have been fulfilled. Validation testing ensures that particular requirements for a specific intended use can be consistently fulfilled.

Following the required FDA V&V processes alone is not sufficient to ensure software quality. You also have to adhere to software development best practices at all levels. For example, in addition to non-functional requirements there are many software quality factors that require careful design considerations and testing that you may decide are outside the scope of FDA reporting.  Deciding what to report and what to leave out is the balancing act.

To Validate and Verify: Software Issues Solved

Tuesday, April 6th, 2010

Yours truly was interviewed for this article:

To Validate and Verify: Software Issues Solved

"V&V" is one of those topics that should be simple to understand, but for some reason is the source of a lot of confusion. This is evident in the comments on Software Verification vs. Validation.

It is also interesting to note that the differing interpretations of these definitions results in a wide variety of V&V strategies and plans. From a regulatory point of view there is no single right or wrong way to do it. It's similar to the implementation of quality systems in general.  If you say you are going to do something you need to be able to prove that you're actually doing it.