When Open-source can Kill or Cure

June 6th, 2012

The use of open-source software in medical devices has been a topic of discussion for many years. The Economist article Open-source medical devices: When code can kill or cure highlights continuing activity in the academic community and interest by the FDA in developing processes around its use.

One of the big unknowns in this area is how a community might be formed (Dreaming of Flexible, Simple, Sloppy, Tolerant in Healthcare IT  has some thoughts on this).

From the article:

Eventually, medical devices might evolve into collections of specialised (and possibly proprietary) accessories, with the primary computing and safety features managed by an open-source hub.

This is in reference to both hardware and software, but in either case one major challenge will be how to incentivize contributions.  Open-source means free to use and modify. If there is no financial gain to be had, other benefits for contributing need to be developed.  Also, proprietary and open-source in the same sentence seems like an oxymoron, so I'm not sure how that's going to work.

Another barrier would be liability risk. Let's say you contributed software to this hub and that component ended up in a device that harmed or killed someone.  All of the legal waivers, disclaimers, and releases in world wouldn't necessarily keep you out of a court room.

I don't think the basic arguments discussed in Open Source Medical Device Connectivity have changed a great deal.  At the end of the day the medical device manufacturer will ultimately be responsible for ensuring the safety and efficacy of their device.

Healthcare IT Q&A: R.I.P.

May 2nd, 2012

After close to two years of effort (see here) the Healthcare IT Stack Exchange Site is closing for good. HealthCare IT is closing:

it simply does not appear that this topic has a strong enough following on our network to support the site long-term

That says it all. 🙁

Design Patterns for EMR/EHR Performance

March 31st, 2012

Healthy Architectures - Using CQRS and Event Sourcing for Electronic Medical Records presents a couple of interesting patterns for the management of healthcare data. The two patterns are:

  • Command Query Responsibility Segregation (CQRS)
  • Event Sourcing (ES)

Here's another article that provides further clarification on the CQRS pattern and how it compares to ES and Task-Based UIs: CQRS, Task Based UIs, Event Sourcing agh!

These are high level design patterns that result in non-traditional and more complex system architectures when implemented.  The healthcare domain is complex and alternate approaches for providing robust solutions are worth consideration.

OTS/SOUP Software Validation Strategies

February 16th, 2012

My last discussion of Off-The-Shelf software validation only considered the high-level regulatory requirements.  What I want to do now is dig deeper into the strategies for answering question #5:

How do you know it works?

This is the tough one. The other questions are important, but relative to #5, answering them is pretty easy.  How to answer this question (i.e. accomplish this validation) is the source of a lot of confusion.

There are many business and technical considerations that go into the decision to use OTS or SOUP software as part of a medical device. Articles and books are available that include guidance and general OTS validation approaches. e.g. Off-the-Shelf Software: A Broader Picture (warning PDF) is very informative in this regard:

  • Define business’ use of the system, ideally including use cases and explicit clarification of in-scope and out-of-scope functionality
  • Determine validation deliverables set based on system type, system risk, project scope, and degree of system modification
  • Review existing vendor system and validation documentation
  • Devise strategy for validation that leverages vendor documentation/systems as applicable
  • Create applicable system requirements specification and design documentation
  • Generate requirements-traceable validation protocol and execute validation
  • Put in place system use, administration, and maintenance procedures to ensure the system is used as intended and remains in a validated state

This is great stuff, but unfortunately it does not help you answer question #5 for a particular type of software. That's what I want to try to do here.

OTS really implies Commercial off-the-shelf (COTS) software. The "commercial" component is important because it presumes that the software in question is a purchased product (typically in a "shrink-wrapped" package) that is designed, developed, and supported by a real company.  You can presumably find out what design controls and quality systems are in place for the production of their software and incorporate these findings into your own OTS validation.  If not, then the product is essentially SOUP (keep reading).

Contrast OTS with Software of Unknown Provenance (SOUP).  It is very unlikely that you can determine how this software was developed, so it's up to you to validate that it does what it's supposed to do.  In some instances this may be legacy custom software, but these days it probably means the integration of an open source program or library into your product.

This following list is by no means complete. It is only meant to provide some typical software categories and the strategies used for validating them.  Some notes:

  • I've included a Hazard Analysis section in each category because the amount of validation necessary is dependent on the level of concern.
  • The example requirements are not comprehensive. I just wanted to give you a flavor for what is expected.
  • Always remember, requirements must be testable.  The test protocol has to include a pass/fail criteria for each requirement. This is QA 101, but is often forgotten.
  • I have not included any example test protocol steps or reports.  If you're going to continue reading, you probably don't need help in that area.

Operating Systems

Examples:

  • Windows XP SP3
  • Windows 7 32-bit and 64-bit
  • Red Hat Linux

Approach:

  1. Hazard Analysis: Do a full  assessment of the risks associated with each OS.
    • Pay particular attention to the hazards associated with device and device driver interactions.
    • List all hazard mitigations.
    • Provide a residual Level of Concern (LOC) assessment after mitigation -- hopefully this will be negligible.
    • If the residual LOC is major, then Special Documentation can still be provided to justify its use.
  2. Use your full product verification as proof that the OS meets the OTS requirements. This has validity since your product will probably only be using a small subset of the full capabilities of the OS.  All of the other functionality that the OS provides would be out of scope for your product.
  3. This means that a complete re-validation of your product is required for any OS updates.
  4. There is no test protocol or report with this approach. The OS is considered validated when the product verification has been successfully completed.

Compilers

Examples:

  • Visual Studio .NET 2010  (C# or C++)
Approach:
  1. Hazard Analysis:
    • For a vast majority of cases, I think it is safe to say that a compiler does not directly affect the functioning of the software or the integrity of the data.  What a program does (or doesn't do) depends on the source code, not on the compiled version of that code.
    • The compiler is also not responsible for faults that may occur in devices it controls. The application just needs to be written so that it handles these conditions properly.
    • For some embedded applications that use specialized hardware and an associated compiler, the above will not necessarily be true. All functionality of the compiler must be validated in these cases.
  2. For widely used compilers (like Microsoft products) full product verification can be used as proof of the OTS requirements.
  3. Validation of a new compiler version , e.g. upgrading from VS 2008 to VS 2010: Showing that the same code base compiles and all Unit Tests pass in both can be used as proof. This assumes of course that the old version was previously validated.
  4. The compiler is considered fit for use after the product verification has passed so there is also no test protocol or report in this case.

Integrated Libraries

Examples:

Approach:
  1. Hazard Analysis: Both of these open source libraries are integrated into the product software.  The impact on product functioning, in particular data integrity, must be fully assessed.
  2. You first list the requirements that you will be using. For example, typical logging functionality that might include:
    • The logging system shall be able to post an entry labeled as INFO in a text file .
    • The logging system shall be able to post an entry labeled as INFO in a LEVEL column of a SQL Server database.
    • ... same for ERROR, DEBUG, WARN, etc.
    • The logging system shall include time/date and other process information formatted as "YYYY-MM-DD HH:MM:SS..." for each log entry.
    • The logging system shall be able to log exceptions at all log levels, and include full stack traces.
  3. For database functionality, listing basic CRUD requirements plus other specialized needs can be done in the same way.
  4. I have found that the easiest way to test these kinds of requirements is to simply write unit tests that prove the library performs the desired functionality.  The unit tests are essentially the protocol and a report showing that all asserts have passed is a great artifact.

Version Control Systems

Examples:

Approach:
  1. Hazard Analysis: These are configuration management tools and are not part of the product. As such, the level of concern is generally low.
  2. As above,  you first list the specific functionality that you expect the VCS to perform. Here are some examples of the types of requirements that need to be tested:
    • The product shall be able to add a directory to a repository.
    • The product shall be able to add a file to a repository.
    • The product shall be able to update a file in a repository.
    • The product shall be able to retrieve the latest revision of files and directories.
    • The product shall be able to  branch a revision of files and directories.
    • The product shall be able to merge branched files and directories.
  3. You then write a protocol that tests each one. This would include detailed instructions on how to perform these operations along with the pass/fail criteria for each requirement.

Issue Tracking Tools

Examples:

Approach:
  1. Hazard Analysis: These tools are used for the management of the development project. Again, the level of concern is generally low.
  2. You only need to validate the functionality you intend to use.  The features that you don't use do not need to be tested.
  3. You simply need to test the specific functionality.  Some example requirements -- the roles, naming conventions, and workflow will of course depend on your organization and the tool being used:
    • A User shall be able to create a new issue.
    • A User shall be able to comment on an issue.
    • A Project Manager shall be able to assign an issue to a Developer.
    • A Developer shall be able change the state of an issue to 'ready for test'.
    • A Tester shall be able to change the state of an issue to 'verified'.
    • The tool shall be able to send e-mail notifications when an issue has been modified.
    • An Administrator shall be able to define a milestone.
  4. A protocol with detailed instructions and pass/fail criteria is executed and reported on.

Validation is a lot of work but is necessary to ensure that all of the tools and components used in the development of medical device software meet their intended functionality.

Building Safety into Medical Device Software

January 7th, 2012

The article Build and Validate Safety in Medical Device Software takes a critical look at the current processes for medical device software and concludes:

The complexity of the software employed in many medical devices has rendered inadequate traditional methods (testing) for demonstrating their safety.

The article then provides examples of the types of analyses that can be performed to better ensure safety.

Interesting read.

Here are some references:

BohrBug: Not necessarily easy to find, but once discovered is reproducible.

Heisenbug: The ever-annoying bug that can not be reliably reproduced.

Spin: An open-source software tool for formal verification of distributed software systems.

SOPA and Internet Censorship

December 17th, 2011

I first heard about this from a system message on the Stack Overflow site. The post Protect intellectual property – but not like this explains their position (in particular, SOPA vs. DMCA) and has a lot of good links to more information.

SOPA-Rope-a-dope has a well written explanation of DNS blocking and DNSSEC.

This bill is a really bad idea. A lot of people in the know agree: An Open Letter From Internet Engineers to the U.S. Congress.

Congress to Resume SOPA Hearings Next Week (Wednesday 12/21) so it's not too late to help stop this bill. If you're like me and have always wondered why people contact their Congressperson, now we finally have a good reason to do so.  Go to Stop American Censorship and let your voice be heard.

From RWW Cartoon: SOPA Opera:

 

UPDATE (12/23/11): Bill that could 'break the Internet' delayed until 2012. Also see: What You Need to Know About SOPA in 2012

Validation of Off-The-Shelf Software Development Tools

November 12th, 2011

A reader asked me about OTS software tool validation. He says:

It seems to me that the editor and any other tool used to create the software is exactly that, a productivity tool. The end result (compiled binary installed on a validated PC configuration) is still going to go through verification and validation, therefore, it seems validating any of the items used to actually create the binary is unnecessary.

Any thoughts or guidance to help me understand this process?

This is a great question and the source of a lot of confusion.

The bottom line is that all third party tools (and libraries) used to construct or test FDA regulated software need to be validated.

You may think validating a compiler is unnecessary, but the FDA says otherwise -- section 6.3 of the FDA Guidance on General Principles of Software Validation discussion includes "off-the-shelf software development tools, such as software compilers, linkers, editors, and operating systems."

The form of the required documentation is detailed in the Off-The-Shelf Software Use in Medical Devices (PDF) guidance document.  Section 2.1 has the questions that the OTS software BASIC DOCUMENTATION needs to answer:

  1. What is it?
  2. What are the Computer System Specifications for the OTS Software?
  3. How will you assure appropriate actions are taken by the End User?
  4. What does the OTS Software do?
  5. How do you know it works?
  6. How will you keep track of (control) the OTS Software?
For most products (again, OTS tools and libraries, including open source products) this documentation is not as onerous as you might think.  #5 is where you apply the intended use validation to the specific product. I have done this for many products: Visual Studio,  Subversion/TortoiseSVN, NUnit, Log4Net, etc.  You also need to validate custom developed testing tools and fixtures.
Like it or not, this is the reality of developing FDA regulated software.
UPDATE (2/16/2012): See OTS/SOUP Software Validation Strategies.

Brain Wave Monitoring Through the Extremities

November 8th, 2011

The recent Time Magazine article Thought Control (subscription required) describes what is essentially another brain-computer interface.  What's novel about this device is that the EEG signal is monitored from dry electrodes on the arm or leg. The BodyWave® Brain Wave Monitoring (pdf) system developed by Freer Logic claims to allow measurement of brain wave activity away from the head:

BodyWave simply views brain energy as a field, collects the field energy as if the brain were a radio tower broadcasting from the brain and through the body.

For the purposes of teaching "stress control, increase attention, and facilitate peak mental performance", this may well be an adequate method.  Not having to wear the more traditional EEG head gear is certainly an advantage.  Providing reliable control of computer interaction tasks via either "mind reading" method is not likely to happen any time soon (see Turning the Mind into a Joystick).

Finally Launched: Healthcare IT Q&A

October 26th, 2011

The new Healthcare IT Stack Exchange site is now open to the public.

Hopefully this thing will take off.  So go search, ask, answer, up vote,  and (yes) down vote when necessary. Also, don't forget to tell your friends and colleagues.

We make our world significant by the courage of our questions and by the depth of our answers. -- Carl Sagan

Guest Article: RFID Systems in Healthcare Institutions

October 25th, 2011

Patient, medication, and equipment asset tracking are critical functions for any healthcare organization.  Yedidia Blonder of Vizbee RFID Solutions, a company providing RFID solutions for healthcare applications and other industries, provides an introduction to RFID technology and its benefits.

What healthcare executive wouldn’t want a system that:

  • Helps nurses locate necessary equipment in seconds?
  • Ensures that only the mother of a newborn or a nurse could remove that baby from the nursery?
  • Makes sure patients don’t wander into staff only areas?
  • Lists inventory of all the medications in a large medicine storage area in minutes?
  • Ensures even equipment distribution across wings and prevents theft?
  • Tracks disinfection patterns of employees?

Enter RFID.

RFID (radio frequency identification) is a technology in which radio waves emitted from electronic tags identify them uniquely. The tags are often used to pinpoint the location of the object, or person, to which the tag is attached. This is different than barcode technology, which is usually used to identify an object as belonging to a larger category without individual identification. Barcodes also need to be read one-by-one from very close proximity, whereas RFID readers can read many tags with a single pass of an RFID reader a few meters away.

How does RFID work?

First you need the tags. RFID tags can be split into two main categories: active tags and passive tags. The active tags are battery operated and transmit their data periodically to readers. Their reading distance varies between a few meters to hundreds. Passive tags are much smaller (sometimes like a paper sticker) and do not transmit their data until being interrogated by a reader in their proximity. The passive tags' reading distance can reach 2-3 meters. Passive tags are usually used for inventory purposes.

The readers consist of an RFID antenna connected to an RFID reader. They receive the data from the tags and then, in order to have a functioning system which will do all the above tasks, transmit the data to a software system which manages the received data.

When the system receives the data, it will both store it for immediate or later review by the healthcare staff, as well as act according to predefined rules set by the administrator. For example, in the case of preventing equipment theft, a rule could be set that if tags attached to pieces of expensive lab equipment go past the reader stationed near the exit to the lab, its signal will set off an alarm, alert important staff members, and lock the exits to that wing of the hospital.

How can RFID help a healthcare institution?

Keeping in mind the stunning figure of 15% of hospital equipment stolen annually as well as the damage that improper maintenance causes, RFID tracking can significantly diminish losses and increase efficient use of equipment. It can ensure that only the right person uses or moves any given piece of equipment, guarantee the correct quantities of a certain apparatus in a designated zone, enable the immediate and accurate location of any item, indicate which item is in use or available, and the list goes on.

RFID can also provide an accurate and comprehensive picture of the total amount of the organization’s inventory, including expiry dates and amount of usage, and provide real-time data on parameters such as temperature and moisture levels, providing alerts in the case of inappropriate conditions that could damage equipment and medications.

Add to this the capacity to track patient and staff movement and interactions with other people and objects - and your RFID healthcare system gives you your entire hospital at a glance, and alerts you to problems.

Implementing RFID systems.

RFID technology is also getting easier to customize. In the past, often RFID hardware would be programmed to work only with specific software. Recently, there have been advances in RFID technology enabling administrators to choose hardware and software independently according to the unique needs of each project. Parameterization tools built into the software can customize applications to specific projects while enabling the implementation of RFID projects in a very short time (days to weeks). You no longer have the time, expense and risk that come with developing software just for your project.

With RFID systems, managing healthcare institutions is getting easier, safer and more efficient.

UPDATE (9/8/2012) : Tim has written an excellent article on the subject:  RFID RTLS Update – Where to Start