Validation of Off-The-Shelf Software Development Tools

A reader asked me about OTS software tool validation. He says:

It seems to me that the editor and any other tool used to create the software is exactly that, a productivity tool. The end result (compiled binary installed on a validated PC configuration) is still going to go through verification and validation, therefore, it seems validating any of the items used to actually create the binary is unnecessary.

Any thoughts or guidance to help me understand this process?

This is a great question and the source of a lot of confusion.

The bottom line is that all third party tools (and libraries) used to construct or test FDA regulated software need to be validated.

You may think validating a compiler is unnecessary, but the FDA says otherwise -- section 6.3 of the FDA Guidance on General Principles of Software Validation discussion includes "off-the-shelf software development tools, such as software compilers, linkers, editors, and operating systems."

The form of the required documentation is detailed in the Off-The-Shelf Software Use in Medical Devices (PDF) guidance document.  Section 2.1 has the questions that the OTS software BASIC DOCUMENTATION needs to answer:

  1. What is it?
  2. What are the Computer System Specifications for the OTS Software?
  3. How will you assure appropriate actions are taken by the End User?
  4. What does the OTS Software do?
  5. How do you know it works?
  6. How will you keep track of (control) the OTS Software?
For most products (again, OTS tools and libraries, including open source products) this documentation is not as onerous as you might think.  #5 is where you apply the intended use validation to the specific product. I have done this for many products: Visual Studio,  Subversion/TortoiseSVN, NUnit, Log4Net, etc.  You also need to validate custom developed testing tools and fixtures.
Like it or not, this is the reality of developing FDA regulated software.
UPDATE (2/16/2012): See OTS/SOUP Software Validation Strategies.

Tags: ,

12 Responses to “Validation of Off-The-Shelf Software Development Tools”

  1. Matt says:

    Thanks Bob. Great reply. Unfortunately, not the answer I was hoping for! But at least it’s consistent with my fears.

    I looked through section 2.1 quite a bit. I’m very worried about all the computer specifications. Let’s say I want to use a package like Xmetal to create documentation. Every writer is using a different laptop with different specifications, does every possible combination have to be validated? What about updates? For every minor revision is re-validation necessary?

    I wish the FDA would publish some sample validation plans for well known pieces of software.

    A lot of their wording seems geared towards software actually running on a medical device. I’m more interested in productivity software used to create software that will run or used to create documents that the end users will have access to.

    In addition software 2.2 goes into hazards. If I’m validating a piece of software such as Adobe Professional to create PDF documents, there’s no potential for causing harm to a patient or operator of the medical device. Can I simply justify that there’s no risk and move on?

  2. Bob says:

    Matt,
    In my opinion, that’s a safe bet. Risk plays a big part in deciding where to expend OTS validation resources. Put yourself in the shoes of an FDA auditor looking over a design history file. Whether it’s a stand-alone device or a PC software product, only the components that may affect patient data integrity and presentation will be the ones worthy of scrutiny.

    Re: Computer requirements and updates. I think you’re talking about system validation of the product, not OTS software tool validation. Unfortunately, you do need to re-validate an updated product.

    The same is true for OTS tools – you have to weigh the benefits of upgrading to the latest and greatest version against the re-validation effort.

  3. very informative post.I have increase my knowledge. Thanks for sharing all….

  4. Yalna says:

    Thanks Bob for an informative post.

    I’ve got one question:

    I need to validate Visual Studio 2010 compiler and have no idea how to approach the task.

    How do you validate a compiler?

  5. Mitch says:

    Hi Bob,
    I was very disapointed the first time I read this requirements about OTS in the GPSV. However, we fixed this very easily by splitting OTS in two categories:
    -standalone OTS, like compilers, IDE (like visual studio), source control, database engines. We added a section in our software development plan where we have a list of our OTS, the justification why we chose them (usually one criterion is that it is widely used or open-source). We also added a section about looking periodically for updates of the versions of these OTS.
    -non standalone OTS, like libraires are verified when integrated in our software, with unit tests, data workflow tests and so on. For this second category, we fall into our ISO 13485 conception procedure. We consider that the OST is verified and validated when it is integrated in our device and when our device is itself verified and validated.
    Not necessary to pull one’s hair out with these GPSV requirements!
    Regards,
    Mitch.
    http://www.cm-dm.com

  6. A number of software development methodologies are in use. Our project management identifies the specific nature of the project at hand and then selects the best application development methodology.
    We pride ourselves on delivering projects on time, encouraging collaboration, promoting technical excellence and being continuously adaptable. Our projects use commonly accepted plan-driven software development methods that are, however, sufficiently flexible and adaptable to allow the developers to make late changes in the project specifications.
    Zion Technology specializes in customized software development for all your business needs. A number of software development methodologies are in use. Our project management identifies the specific nature of the project at hand and then selects the best applicable development methodology.

  7. Karl says:

    Hi Bob,
    I work for a company developing medical devices using application software to manage a robot. The application software runs on a regulat windows operating system and is programmed in .NET (thus using the .NET framework).

    As I understand 62304, both the .NET framework and the windows operating system are SOUPs and needs to be treated as such. But how on earth am I supposed to verify/validate Windows? It would been an exhaustive task that would take a million years!

    I have searched wide and far (i.e. googled) trying to find anyone who has actually done this but the only results I find are about OS in implants and other really HW close firmware OSs. Noone ever mentions application development in a windows environment.

    What is your experience on this? Have you ever heard about anyone that successfully treated Windows as a SOUP?

    Thanks for a great blog
    Karl

  8. SL says:

    Hi all,

    first of all thanks for the interesting post BOB.

    @Karl:
    Maybe GAMP5 (even if it comes from pharma) can give you a hint: GAMP knows 5 software categories. OSs fall into category 1 (infrastructure software) which is considered far from harming the patient.

    moving along this line you should declare exactly which version/configuration of windows you are using and checking this as a kind of incoming goods inspection. (you verify that you bought the right windows version)

    then you could check if windows was installed in the right way. (i guess errors during the installation would mean that you didn’t)

    finally windows is widely used so you could state that errors in the system would at somepoint get recognized by the community and by checking for updates (at least updates that affect your intended use) you can get those errors repaired.

    one other thing to consider are realtime requirements. if you have the need for a realtime OSS i guess this would be a critical point for validation.

    i excuse for my bad english (as i’m no native speaker) and am looking forward to hear if anybody thinks along the same line.

    greetings from austria

    as an addition: for all of you out there looking for validiation of software that is not itself a medical product but automates processes: consider having a look at the AAMI-TIR 36.

  9. […] last discussion of Off-The-Shelf software validation only considered the high-level regulatory requirements.  What I want to do now is dig deeper into […]

  10. Deepak says:

    hi bob,
    in my view, i would consider the decoposition OTS / SOUP software into categories this may include Product software and non prodcut software which mean any SOUP which will used and be part of medical device software, then that SOUP will be follow requriements as IEC 62304 design requirements of the SOUP. in case if the SOUP non product (means they are aided as support tools in development of medical software device and not part of it, then such SOUP tools can follow the CSV approach / GAMP

  11. We produce a software that is used primarily for ticketing and support. Both customer support and we also record a history of work done on specific devices etc; Our software is not used to operate or record results from any device, but we are getting asked the question, “Are we FDA Validated?” Do we need to be, according to the FDA???

  12. Nelson says:

    Hi Bob,

    We’re running into this issue with Microsoft Team Server. The quality people think it needs validating, but I’m convinced that the only portion of team server that needs validation is the build process, not the source repository or the ticket tracking system, since they do not contribute to the final product. I mean, where does it end? Do I need to validate the keyboard I’m typing on since I type code which ends up in the final product?

    Anyway, great discussion. I’ll stop ranting.

Leave a Reply