Interoperability: Google Protocol Buffers vs. XML

Google recently open sourced Protocol Buffers: Google's Data Interchange Format (documentation, code download). What are Protocol Buffers?

Protocol buffers are a flexible, efficient, automated mechanism for serializing structured data – think XML, but smaller, faster, and simpler.

The documentation is complete and worth a quick read through. A complete analysis of PB vs. XML can be found here:  So You Say You Want to Kill XML.....

As discussed, one of the biggest drawbacks for us .NET developers is that there is no support for the  .NET platform. That aside, all of the issues examined are at the crux of why interoperability is so difficult. Here are some key points from the Neward post:

  1. The advantage to the XML approach, of course, is that it provides a degree of flexibility; the advantage of the Protocol Buffer approach is that the code to produce and consume the elements can be much simpler, and therefore, faster.
  2. The Protocol Buffer scheme assumes working with a stream-based (which usually means file-based) storage style for when Protocol Buffers are used as a storage mechanism. ... This gets us into the long and involved discussion around object databases.
  3. Anything that relies on a shared definition file that is used for code-generation purposes, what I often call The Myth of the One True Schema. Assuming a developer creates a working .proto/.idl/.wsdl definition, and two companies agree on it, what happens when one side wants to evolve or change that definition? Who gets to decide the evolutionary progress of that file?

Anyone that has considered using a "standard" has had to grapple with these types of issues. All standards gain their generality by having to trade-off for something (speed, size, etc.). This is why most developers choose to build proprietary systems that meet their specific internal needs. For internal purposes, there's generally not a need to compromise. PB is a good example of this.

This also seems to be true in the medical device industry.  Within our product architectures we build components to best meet our customer requirements without regard for the outside world. Interfacing with others (interoperability) is generally a completely separate task, if not a product unto itself.

Interoperability is about creating standards which means having to compromise and make trade-offs.  It would be nice if Healthcare interoperability could be just a technical discussion like the PB vs. XML debate. This would allow better integration of standards directly into products so that there would be less of the current split-personality (internal vs. external  needs) development mentality.

Another thing I noticed about the PB announcement -- how quickly it was held up to XML as a competing standard. With Google's clout, simply giving it away creates a de facto standard. Within the medical connectivity world though, there is no Google.

I've talked about this before, but I'm going to say it again anyway. From my medical device perspective, with so many confusing standards and competing implementations the decision on what to use ends up not being based on technical issues at all. It's all about picking the right N partners for your market of interest, which translates into N (or more) interface implementations. This isn't just wasteful, it's also wrong. Unfortunately, I don't see a solution to this situation coming in the near future.

Posted in EMR, Google, Interoperability, Open Source, Programming | Leave a comment

More Software Forensics and Why Analogies Suck

There's a recent article in the Baltimore Sun called Flaws in medical coding can kill which just rehashes static software  analysis (hat tip: FDA Trying to Crack Down on Software Errors).

I've discussed software forensics tools before. Yes, bad software has hurt and killed people, and there's no excuse for it.  I still don't think an expensive automated software tool is the silver bullet (which is implied by the article) for solving these problems.

But here's what really bugs me:

"If architects worked this way, they'd only be able to find flaws by building a building and then watching it fall down"

This is a prime example of why analogies suck.  The quote is supposed to somehow bolster the FDA's adoption of "new forensic technology". If you stop and think about it, it does just the opposite.

I guess you first have to consider the source --  a VP of Engineering for a forensic software vendor. This is exactly what a you'd expect to hear in a sales pitch.

What's truly ironic though is that a static analysis tool can only be used on source code! Think about it. Source code is the finished product of the software design and development process. Also, forensic science, by definition is the presentation of something that has already happened. It can only be done after the fact.

The logical conclusion you would draw from the analogy is that static analysis is probably useless because the building is already up!  If you step back and look at the full software quality process, this may well be true.

I'm not saying that static analysis tools don't have value. Like all of the other software tools we use, they have their place.

Just beware when you try to use an analogy to make a point.

UPDATE (7/5/08):

Here's another take on medical device bugs: When bugs really do matter: 22 years after the Therac 25.

UPDATE (7/16/08):
From Be Prepared: Software Forensics Gaining Steam at FDA, David Vogel of ­Intertech Engineering Associates says:

... that static tools are hyped to do more than they can actually deliver. “Static analysis looks for simple coding errors and does not apply heuristics to understand how it will perform dynamically because it is a static analysis tool”

I agree.

UPDATE (7/26/08):

Another reference : Are hospitals really safe?

UPDATE (9/16/08):

A couple more related articles:

Applying Static Analysis To Medical Device Software

Using static analysis to diagnose & prevent failures in safety-critical device designs

UPDATE (9/27/08):

Architecting Buildings and Software: Software architects are an important component in the creation of quality software and need to continue to refine and improve their role in the development process.  No matter how you try to bend and twist it though, the building analogy will always be problematic -- so why bother? Maybe that "intuitive understanding" of the construction industry just distracts us from being innovative about what's required to build software.

UPDATE (12/1/08): If Jeff wasn't a programmer he'd be a farmer: Tending Your Software Garden

Posted in FDA, Medical Devices, Software Quality | 3 Comments

One Reason Why Linux Isn’t Mainstream: ./configure and make

Bare with me, I'll get to the point of the title by the end of the post.

I primarily develop for Microsoft platform targets, so I have a lot of familiarity with Microsoft development tools and operating systems. I also work with Linux-based systems, but mostly on the systems administration side: maintaining servers for R&D tools like Trac and Subversion.

I recently had some interest in trying to use Mono running on Linux as .NET development platform.

This also allowed me to try Microsoft Virtual PC 2007 (SP1) on XP-SP3. I went to a local .NET Developer's Group (here) meeting a couple of weeks ago on Virtual PC technology. Being a Microsoft group most of the discussion was on running Microsoft OS's, but I saw the potential for using VPC running Linux for cross-platform development. My PC is an older Pentium D dual core without virtualization support, but it has 3Gig of RAM and plenty of hard disk space, so I thought I'd give it a try.

Download and installation of Ubuntu 8.04 (Hardy Heron) LTS Desktop on VPC-2007 is a little quirky, but there are many blog posts that detail what it takes to create a stable system: e.g. Installing Ubuntu 8.04 under Microsoft Virtual PC 2007. Other system optimizations and fixes are easily found, particularly on the Ubuntu Forums.

OK, so now I have a fresh Linux installation and my goal is to install a Mono development environment. I started off by following the instructions in the Ubuntu section from the Mono Other Downloads page. The base Ubuntu Mono installation does not include development tools. From some reading I found that I also had to install the compilers:

# apt-get install mono-mcs
# apt-get install mono-gmcs

So now I move on to MonoDevelop. Here's what the download page looks like:

Monodevelop Download

Here's my first gripe: Why do I have to download and install four other dependencies (not including the Mono compiler dependency that's not even mentioned here)?

Second gripe: All of the packages require unpacking, going to a shell prompt, changing to the unpack directory, and running the dreaded:

./configure
make

Also notice the line: "Compiling the following order will yield the most favorable response." What does that mean?

So I download Mono.Addins 0.3, unpack it, and run ./configure. Here's what I get:

configure: error: No al tool found. You need to install either the mono or .Net SDK.

This is as far as I've gotten. I'm sure there's a solution for this. I know I either forgot to install something or made a stupid error somewhere along the way. Until I spend the time searching through forums and blogs to figure it out, I'm dead in the water.

I'm not trying to single out the Mono project here. If you've even tried to install a Unix application or library you inevitably end up in dependency hell -- having to install a series of other packages that in turn require some other dependency to be installed.

So, to the point of this post: There's a lot of talk about why Linux, which is free, isn't more widely adopted on the desktop. Ubuntu is a great product -- the UI is intuitive, system administration isn't any worse than Windows, and all the productivity tools you need are available.

In my opinion, one reason is ./configure and make. If the open source community wants more developers for creating innovative software applications that will attract a wider user base, these have to go. I'm sure that the experience I've described here has turned away many developers.

Microsoft has their problems, but they have the distinct advantage of being able to provide a complete set of proprietary software along with excellent development tools (Visual Studio with ReSharper is hard to beat). Install them, and you're creating and deploying applications instantly.

The first step to improving Linux adoption has to be making it easier for developers to simply get started.

Posted in Open Source, Programming, Tools | 3 Comments

Connecting Computers to FDA Regulated Medical Devices

Pete Gordon asked a couple of questions regarding FDA regulations for Internet-based reporting software that interface with medical devices. The questions are essentially:

  1. How much documentation (SRS, SDS, Test Plan) is required and at what stage can you provide the documentation?
  2. How does the FDA view SaaS architectures?

The type of software you're talking about has no real FDA regulatory over site. The FDA has recently proposed new rules for connectivity software. I've commented on the MDDS rules, but Tim has a complete overview here: FDA Issues New MDDS Rule. As Tim notes, if the FDA puts the MDDS rules into place and becomes more aggressive about regulation, many software vendors that provide medical devices interfaces will be required to submit 510(k) premarket approvals.

Dealing with the safety and effectiveness of medical devices in complex networked environments is on the horizon. IEC 80001 (and here) is a proposed process for applying risk management to enterprise networks incorporating medical devices.  My mantra: High quality software and well tested systems will always be the best way to mitigate risk.

Until something changes, the answer to question #1 is that if your software is not a medical device, you don't need to even deal with the FDA. The answer to question #2 is the same. The FDA doesn't know anything about SaaS architectures unless it's submitted as part of a medical device 510(k).

I thought I'd take a more detailed look at the architecture we're talking about so we can explore some of the issues that need to be addressed when implementing this type of functionality.

mdds2.jpg

This is a simplified view of the way medical devices typically interface to the outside world. The Communications Server transmits and receives data from one or more medical devices via a proprietary protocol over whatever media the device supports (e.g. TCP/IP, USB, RS-232, etc.).

In addition to having local storage for test data, the server could pass data directly to an EMR system via HL7 or provide reporting services via HTTP to a Web client.

There are many other useful functions that external software systems can provide. By definition though, a MDDS does not do any real-time patient monitoring or alarm generation.

Now let's look at what needs to be controlled and verified under these circumstances.

  1. Communications interaction with proper medical device operation.
  2. Device communications protocol and security.
  3. Server database storage and retrieval.
  4. Server security and user authentication.
  5. Client/server protocol and security.
  6. Client data transformation and presentation to the user (including printed reports).
  7. Data export to others formats (XML, CSV, etc.).
  8. Client HIPAA requirements.

Not only is the list long, but these systems involve the combination of custom written software (in multiple languages), multiple operating systems, configurable off-the-shelf software applications, and integrated commercial and open source libraries and frameworks. Also, all testing tools (hardware and software) must be fully validated.

One of the more daunting verification tasks is identifying all of the possible paths that data can take as it flows from one system to the next. Once identified, each path must be tested for data accuracy and integrity as it's reformatted for different purposes, communications reliability, and security. Even a modest one-way store-and-forward system can end up with a hundred or more unique data paths.

A full set of requirements, specifications, and verification and validation test plans and procedures would need to be in place and fully executed for all of this functionality in order to satisfy the FDA Class II GMP requirements. This means that all of the software and systems must be complete and under revision control. There is no "implementation independent" scenario that will meet the GMP requirements.

It's no wonder that most MDDS vendors (like EMR companies) don't want to have to deal with this. Even for companies that already have good software quality practices in place, raising the bar up to meet FDA quality compliance standards would still be a significant organizational commitment and investment.

Posted in FDA, HL7, Medical Devices, Networking, Software Quality | Tagged , | 2 Comments

Moving Mountains With the Brain

In today's New York Times business section there's a piece called: Moving Mountains With the Brain, Not a Joystick. I've previously discussed both of the mentioned EEG-based headsets here.The article highlights some of the problems that this type of technology will face in the consumer marketplace:

“Not all people are able to display the mental activity necessary to move an object on a screen,” he said. “Some people may not be able to imagine movement in a way that EEG can detect.”

I agree. Even though Emotiv claims that "all 200 testers of the headset had indeed been able to move on-screen objects mentally" it's very doubtful that the device will have that level of success (100%!) with real gamers.They talk about the use of facial muscle activity (EMG) in addition to the EEG signal. With proper electrode placement, I think EMG holds far more promise for enhancing the gaming experience. Even EOG could be used effectively as a feedback and control mechanism. Reliable EEG processing for this purpose is still a long way off.

UPDATE (6/17/08): More of the same: No Paralysis in Second Life

UPDATE (6/29/08): Here's a pretty good description of how these devices are being used for control purposes:  OCZ's Neural Impulse Actuator (The flying car of control schemes).

UPDATE (7/21/08): An even more thorough evaluation: OCZ NIA Brain-Computer Interface. A generally positive and realistic assessment of the technology:

...the NIA isn't a replacement for traditional input methods, it is merely a powerful supplement.

brainwaves.jpg

Posted in EEG, HCI | 1 Comment

Goosh, a Google Command Line

For us old Unix hackers, Goosh, a Google Command Line is very cool.

Check in out here: Goosh.org.

Posted in Google | 2 Comments

.NET Console.Writeline Performance Issues

We ran into an interesting problem recently that I have not been able to find documented anywhere.

We're doing real-time USB data acquisition with .NET 2.0. The data bandwidth and processing isn't overwhelming. Specifically, we expect data packets at 50 Hz -- every 20 ms. Yet we were having horrible delay problems. In the end we found that Console.Writeline was the culprit!

To verify this a test program was written to measure the throughput of the following loop:

The length of mstr is varied and this loop is run for about 30 seconds. The results show the ms per message for increasing message lengths:

Console.Writeline Performance

Console.Writeline is surprisingly slow!

We use log4net for our logging. With the timestamp and class information, a log message is typically greater than 100 characters. A single log message introduces at least a 20 ms delay in that case, with additional messages adding that much more. Even though debug logging would not be included in the released version, these significant delays make development difficult.

Not only do you need to make sure that there are no Console.Writeline's in your real-time threads you also need to remove the console appender (<appender-ref ref="ConsoleAppender"/>) from the log4net configuration. The log4net file appenders do not cause noticeable delays.

Posted in .NET, Programming | 1 Comment

Selecting Books About Programming

This is tough to do. There are tons of technical books out there. Also, now that the Internet can instantly answer just about any question, the path of least resistance leads to arguments like this: Why I don’t read books.

There is no right or wrong when in comes to learning methods. It's a personal preference. I'm a book reader, but I can understand how Internet content (blogs, articles, etc.) has made it easy for people that don't like books (for whatever reason) to acquire relevant knowledge.

The availability of "Best" lists are abundant. For example:

OK, so once you get past the classics (Code Complete 2, The Pragmatic Programmer, Design Patterns, etc.) where do you go from there?

I will typically invest in books on emerging technologies that I want to fully understand (a recent one was Windows Presentation Foundation Unleashed -- highly recommended). The real challenge is finding a book that doesn't suck. Reviews and recommendations by other readers, like from the sites above, are the best resources.

My other vetting technique is standing (and reading) for long periods of time in the nerd-book section of a bookstore. There's no substitute for browsing the pages of a real book. The one that's still in your hand when your SO drags you towards the exit is probably the best one to buy.

Posted in Programming, Technology | 1 Comment

Drop, Spike… Huh?

I guess I didn't get the memo.

Here's the first couple of sentences from this post:

Several drops ago we introduced a ViewModel composition spike. The purpose of this spike was to introduce...

I don't know why, but this terminology caught me by surprise. My interpretations are:

  • Drop: a software release.
  • Spike: a quickly written new or enhanced feature used as a proof of concept (e.g. see here)

Drops are sometimes used in CodePlex-P&P projects and spikes are often referenced in the ALT.NET discussion group (searched for here).

Anyway, I like the sound of these. They're succinct and to the point. I foresee them becoming a common part of the developer's lexicon.

UPDATE: OK, so I'm not an Agile developer. In particular, I have not experience with XP (Extreme Programming), which defines a 'Spike': Create a Spike Solution

Posted in Programming | Leave a comment

Google Health Launches: More PHR for the masses.

It's finally here: Drumroll, Please: Google Health Launches!

If you use any of the Google applications (like Gmail), it's just as easy as all the others:

Google Health

It will be interesting to see if this and HealthVault have an impact on how patients interact with their medical service providers. The privacy and security issues are certain to remain a significant barrier to adoption. Only time will tell.

UPDATE (5/23/08): See Delving Into Google Health's Privacy Concerns

UPDATE (5/24/08): Apparently this Slashdot reference is "uninformed": Why Google Health and HealthVault are not covered by HIPAA.

UPDATE (7/6/08): I ran across this post that talks about Microsoft HealthVault security: You Will Never Get It Microsoft. Here's a quote from it:

Microsoft obviously think that I don't know how HealthVault works. I don't have to know how it works, I only know that it will and can be abused one day.

Posted in Google, Microsoft, PHR | Leave a comment