Archive for the 'Programming' Category

Page 2 of 5

Design Patterns for EMR/EHR Performance

Healthy Architectures – Using CQRS and Event Sourcing for Electronic Medical Records presents a couple of interesting patterns for the management of healthcare data. The two patterns are:

  • Command Query Responsibility Segregation (CQRS)
  • Event Sourcing (ES)

Here’s another article that provides further clarification on the CQRS pattern and how it compares to ES and Task-Based UIs: CQRS, Task Based UIs, Event Sourcing agh!

These are high level design patterns that result in non-traditional and more complex system architectures when implemented.  The healthcare domain is complex and alternate approaches for providing robust solutions are worth consideration.

Binary Waveform Data in SQL Server 2008

As Shahid points out in Consider MySQL ‘Archive’ storage engine to store large amounts of med device structured or waveform data, saving physiologic waveform data from a medical device in a MySQL database for archive purposes is a reasonable alternative to using flat files.

In SQL Server 2008 you can have it both ways.  In addition to saving binary data directly in the database you have the option to have a varbinary column stored as a file stream. From the article How to store and fetch binary data into a file stream column:

File stream data can be used from the .NET Framework using the traditional SqlParameter, but there is also a specialized class called SqlFileStream which can be used with .NET Framework 3.5 SP1 or later. This class provides mechanisms, for example, for seeking a specific position from the data.

There are pros and cons to this approach. The backup and transactional issues, along with the performance considerations, all have to be evaluated against your specific system requirements.  Having the SQL Server engine manage the database relationship to the binary files seems like a big advantage over maintaining flat files yourself.

Read the MSDN article FILESTREAM Storage in SQL Server 2008 for all the gory details.

UPDATE (3/25/11): Who’s Got Access to your FileStream Directories?

Technical Debt in Medical Software

Software development is software development. Most of the life cycle and quality issues faced in medical software are the same challenges for any software product. Technical Debt in Medical Software points out what technical debt is:

  • Complexity
  • Code Duplication
  • Documentation Debt
  • Testing Debt
  • Architectural Debt

A Martin Fowler article is referenced that nicely identifies the source of technical debt:

The benefits of paying down the debt are:

  • Increased R&D efficiency and improved time to market
  • Hitting commitment dates
  • Performance and technology upgrades

Of course if you don’t want to pay it off, there’s always the option to go bankrupt. This may have long-term advantages, but it will surely be a more expensive route. There is one statement in this regard that I think needs some qualification:

In this case the technical debt can be retired along with the legacy system, and like filing Chapter 11, you are no longer responsible to address all the sins of the past.

I know this refers to code sins, but just because you decide to do a re-write doesn’t mean you no longer have responsibility for the legacy product. You still have customers using the old software that you’re obligated to continue to support.  For FDA approved medical software, this is a legal requirement. Most of the time this means that the legacy code will need to be maintained and periodically updated in the field, sometimes even after the “new” product is released. This just makes the cost of bankruptcy even higher.

The Challenges of Developing Software for Medical Devices

Developing Software for Medical Devices – Interview with SterlingTech gives a good overview of the challenges that especially face young medical device companies. In particular (my emphasis):

Make sure that your company has a good solid Quality System as it applies to software development. Do not put a Quality System into place that you can not follow. This is the cause of most audit problems.

I couldn’t have said it better myself, though I think that focusing on the FDA may distract you from why you’re creating software quality processes in the first place. The real purpose of having software design controls is to produce a high quality, user friendly, robust, and reliable system that meets the intended use of the device.  If your quality system does that, you won’t have to worry about FDA audits.

Since Klocwork is a static analysis tool company I also want to point out a recent related article that’s worth reading — and trying to fully understand:

A Few Billion Lines of Code Later: Using Static Analysis to Find Bugs in the Real World

Note the user comment by Bjarne Stroustrup.

UPDATE (2/9/10): Here’s another good code analysis article:

A Formal Methods-based verification approach to medical device software analysis

Stackoverflow Overflow Update

In Stackoverflow Overflow I predicted 500,000 questions on 2/7/2010 at 5:31. When I checked (after the Superbowl — congratulations to NO!) at 19:42 this evening:

Not bad. Only 14 hours off on a three month linear extrapolation from only two weeks of data!

Stackoverflow Overflow

I really like SO. It’s quickly become an invaluable resource for finding and getting programming answers.

I generally browse the new questions to see if there’s anything I might be able to a answer.  Even when you find a question you already (think you) know the answer to, it still requires  some research.  Answering questions is a great exercise.

A couple of weeks ago I started to take notice of how fast the question count was increasing.  So I decided to track it and quantify the growth. Here’s what I found:


So on average there are about 1472 new questions every day! That means when I browse through my 50 or so new questions, there are more than 1400 other ones that I’m not seeing. And that’s if I visit every day.

Yeah, yeah — I know about searching, using the tags, unanswered questions, etc.  What’s remarkable to me is the shear volume of questions.  Two things strike me about this:

  1. There seems to be a lot of question duplication. You can see that when you go to a question and look at the related questions on the right hand column.
  2. My sense is that the number of views per question is also going down.  I don’t know for this sure, but it would follow that there must also be fewer answers per question.

These are just anecdotal observations on my part.  I didn’t mean this to be an exhaustive analysis of SO, and I’m not trying to be critical. It just seems to me that the “weight “of SO is bound to become an issue in the future.

SO is a great site and I’m sure Jeff and Joel are very aware of these patterns and are plotting solutions while the rest of us sleep.

So based on 14 days of data – 🙂 –  here’s about when the number of questions will hit future milestones:


We’ll have to wait and see if these predictions become true.

The Desperate Need for Simplicity

Ted Neward’s article “Agile is treating the symptoms, not the disease” touches on several important points about the software industry.

  • Modern software development tools and technologies require a significant learning curve.
  • Development methodologies (like Agile) exist for managing complexity, but do not reduce the load of these technologies.
  • In the last decade there has been no “Next Big Thing”, like Access was in the 90s.

What’s most interesting to me is:

We are in desperate need of simplicity in this industry. Whoever gets that, and gets it right, defines the “Next Big Thing”.

What’s true in the broader software world is also generally true in Healthcare IT.  In HIT there has never been an Access equivalent, just a lot of pieces and parts trying unsuccessfully to work together.

The need was touched on in Liberate the Data!.  Simplicity is desperately needed in order to create the “First Big Thing” for HIT interoperability.

UPDATE (10/14/09):  More commentary:

A .NET Application that Never Dies

29392-Live_forever_-sfullJeremy’s Graceful Shutdown Braindump should really include another use case. How do you create a .NET application that never shuts down? Ever!

This  is a common scenario for closed systems that only allow the user to interact with a predefined set of applications.  In other words, the user is never able to utilize any of the operating system functionality. In particular, they can not install new applications or update any software components.

This situation is related to the issues discussed in Medical Device Software on Shared Computers. Creating a closed Windows-based system is not an easy task. For our XP Embedded system here are some of the considerations:

  1. Prevent booting from a peripheral device (CD-ROM, USB stick, etc.)
  2. Prevent access to the BIOS so that #1 is enforced.
  3. Prevent plug-n-play devices from auto-starting installers.
  4. You can not run Explorer as the start-up shell — no desktop or start menu.
  5. Prevent Ctrl-Alt-Del from activating task manager options.
  6. Disable the Alt-Tab selection window so the user can not switch application focus.
  7. Ensure that the primary user interface application is always running.
  8. All UI components must exit without user interaction when the system is powered down.

One of the challenges for .NET applications is how to handle unexpected exceptions. What you need first is a way to catch all exceptions.  OK, so now you know your program is in serious distress. You may be able to recover some work (a la a “graceful shutdown”), but after that it’s not a good idea keep the application running.

That means you have to restart the program. For a WinForm application one option is:

Application.Restart() essentially calls Application.Exit() which tries to gracefully shutdown all UI threads.   The problem with that is the application may appear to be hung if you have background worker threads that are monitoring hardware devices that are not currently responding.

Another issue is when the .NET application is doing interop with COM components.  I’ve seen situations where all of the managed threads appear to exit properly via Application.Exit() but an un-managed exception (and error window) still occur.  This behavior is unacceptable.

The way to ensure that the application restarts properly (simplified):

The Environment.Exit() call is harsh, but it is the only way I know of that guarantees that the application really exits.  If you want a Windows Application event log and a dump of your application you can use Environment.FailFast() instead.

UPDATE (9/19/09): I ran across a post about COM object memory allocation in mixed managed/unmanaged environments: Getting IUnknown from __ComObject. As this article exemplifies, debugging COM objects under these circumstances is a real pain in the butt.  We used strongly-typed managed wrappers for our COM objects. Besides a .NET  memory profiler we just monitored overall allocations externally with Process Explorer. It may be undocumented and fragile, but at least it’s good to know that there is way to dig deeper if you need to.

2009 Ultimate Developer and Power Users Tool List for Windows

This is a great list. I linked to it in 2007, but somehow forget in 2008. Anyway, there’s probably at least one tool you may not have seen before that would be worth trying out.

Scott Hanselman’s 2009 Ultimate Developer and Power Users Tool List for Windows

Oh, and read through the comments. Everyone has their own favorites, and opinions.

Plunging into Web Development

ConanI’ve authored a few web sites. Nothing professionally though. I know just enough HTML, CSS, and JavaScript to be dangerous.

Now I’m faced with creating a customer-facing site that has (or will someday soon have) real requirements.

Here are a couple of the requirements I know so far:

  1. Relatively low volume traffic. The site will be public, but only registered users (customers) will have access.  No product pages, no shopping carts, no ads, no social networking. The front page is a login screen.
  2. Reliable and secure transport and storage of medical data.  At a minimum we must comply with HIPAA standards (privacy rules).

I don’t see web site development as really that different from building any other type of application. It’s all software. The architectural building blocks may be different, but the developer’s mind-set and methodologies for producing a quality product need to  be the same.

I haven’t gotten far enough along to really understand all of the deployment and maintenance issues. I’m thinking about them though. The same goes for testing. I can foresee development vs. production platform testing issues that will have to be carefully considered.

What I want to do is walk you through my rational for the selection of some of the major components and tools I’m considering using for this project.

Web Frameworks

Here’s a little historical perspective on selecting a web development framework:


Yep, that’s how it feels.  There are at least 100 options (plus a couple of my additions):

AgaviAIDA/Web | Ajile | Akelos | Apache ClickApache CocoonApache StrutsApache WicketAppFuseAraneaASP.NET MVC | Axiom Stack | BFCCakePHPCampingCatalystCherryPyCodeIgniterColdSpringCSLACppCMSDjangoDotNetNukeDrupal | ErlyWeb | eZ ComponentsFlex | FUSE | FuseboxGoogle Web ToolkitGrokGrailsHamletsHordeInterchangeItsNatIT Mill ToolkitJavaServer Faces | Jaxer | JBoss SeamKepler | Kohana | Lift | LISA | ManyDesigns PortofinoMasonMaypoleMach-IIMerbMidgardModel-GlueMonoRailMorfikNitroonTapOpenACSOpenLaszloOpenXava | Orbit | PEAR | Orinoco | PyjamasPylonsQcodoRadicoreReasonable Server FacesRIFERuby on RailsSeasideShale | Simplicity | SilverStripe (Sapphire)SmartClientSofiaSPIPSpringStripesSymfonyTapestryThinWire | Tigermouse | VaadinTurboGearsWavemakerweb2pyWebObjectsWebWork | Wigbi | YiiZendZK | Zoop | Zope 2Zope 3ztemplates


As a .NET developer, my first inclination was to look at ASP.NET MVC. The two most popular and active open source frameworks are  Ruby on Rails (RoR) and Django (Python-based). To be honest, I have not spent a lot of time investigating any of the others.

Why is it that I often find myself in this situation? It’s usually not 100, but there always seems to be multiple well developed solutions for these types of problems.  I ran into the same thing a couple of years ago when I was selecting an ORM for a .NET project.

All you can do is start by taking the advice of others (“most popular”) and give one or two a try.  Not only will you get a good sense of how well the framework meets your project requirements, since there will inevitably be problems or questions you’ll also be able to evaluate documentation and community activity.

It’s like making pasta — you throw a noodle against the wall and if it sticks, you’re done cooking.  Well, not really… but you know what I mean.


One of the major considerations is hosting. I’ve previously explored the three major cloud computing platforms.

  • Amazon EC2 would be overkill (see requirement #1). I don’t see a need for significant scale-up in the foreseeable future. Running a small on-demand EC2 instance 24/7 is more expensive (~$70/month) than just buying hosted services.  Also, supporting a complete OS platform is unnecessary work.
  • Microsoft Azure is currently in CTP (Community Technology Preview) and it’s still unclear what the pricing will be.
  • That leaves Google App Engine.  Based on the GAE Quotas, we would be able to operate under the limits for quite a while (exceeding the quotas would be a good thing).  That means GAE can provide us free hosting, which is hard to beat.

There are literally 100’s of hosting options, and most would meet our bandwidth and storage requirements at a nominal cost.  Independent of storage (see below) I guess I’m biased towards a cloud solution for two reasons:

  1. “Good Enough” isn’t Good Enough: I’ve been hosting this domain on a commercial site for about 6 years.  I’d classify my host as good enough for my personal use (family site, photo gallery, this blog, etc.).  If my hosting service went away tomorrow, no big deal. I backup everything regularly and could be up and running on a comparable host pretty quickly. But for business purposes that involve critical customer medical data, “good enough” and the possibility of the host disappearing just doesn’t cut it.
  2. Large Infrastructure: This is what makes a cloud solution so attractive. With any of the three cloud options you are buying into reliability and stability. They already have multiple data centers, security, and disaster plans in place.  You don’t have to worry about Amazon, Microsoft, or Google going away any time soon. Unless you have the resources to build it yourself, IMO using a cloud service is a good business decision.

So for now I’ll be using Google App Engine.

Data Storage

Now lets looks at requirement #2: reliable and secure data storage. At this time the best solution seems to be Amazon S3. Amazon has already put a lot of thought into this:  Creating HIPAA-Compliant Medical Data Applications with Amazon Web Services (warning: PDF).  S3 transfer and storage costs are very reasonable.  Paying only for what you use is a real benefit.

Both Google and Microsoft are very active in the Healthcare sector (Google Health and HealthVault) and I’m sure will soon have cloud storage offerings with similar features.

There are a number of web hosting sites that claim HIPAA data storage compliance, but most seem to just be using “HIPAA” as a marketing tool to attract medically related clients. I’d stay away from these.

Web Frameworks (part 2)

Deciding to use GAE quickly narrows the web framework choice down. GAE supports Python (w/ Django) and the Java 6 runtime environment. I do not believe that either ASP.NET or RoR are supported on GAE. Done deal — Django.

I know what you’re thinking.  There are many other Python-based web frameworks and even Java alternatives that I should be considering. That’s true, but Django is arguably the most popular and has a very active developers community. Also, there are several Google Code App Engine projects (see below) that support Django integration.

I did play around with RoR . The Ruby language itself is great. I love having five different ways to do the same thing. The RoR web framework is mature and has many of the same features as Django.

I looked at ASP.NET MVC, but only from a distance. Here’s a concise take from someone that recently jumped in: ASP.NET MVC Impressions after 1 week.

Development Environment

I initially setup a Windows-based Python/Django/GAE-SDK development environment but found it to be too clumsy.  I’ve settled into Ubuntu 9.04 running in a VirtualBox VM.

The Ubuntu Package Manager handled installation of all the necessary prerequisite components. Now that I think of it, I didn’t have to do a single ./configure and make. That’s progress!

I’m an old Unix hack and I quickly fell back into my first love : Emacs. After the nostalgia wore off, I needed to find a real development IDE.  There were two choices:

  1. Eclipse:  I tried using the PyDev plug-in along with some Django integration instructions I found. Google also provides some Eclipse integration, but being able to start the server and other functions from the IDE was not that important to me.  I’d rather use the command line. Also, Eclipse just seems like a real dog.
  2. Netbeans:  With the Python plug-in Netbeans works fine, so I’ll stick with it until something better comes along.

Django (Front-end)

The four features that make  Django attractive:

  • Object-relational mapper: Define your data models entirely in Python. You get a rich, dynamic database-access API for free — but you can still write SQL if needed.
  • Automatic admin interface: Save yourself the tedious work of creating interfaces for people to add and update content. Django does that automatically, and it’s production-ready.
  • Elegant URL design: Design pretty, cruft-free URLs with no framework-specific limitations. Be as flexible as you like.
  • Template system: Use Django’s powerful, extensible and designer-friendly template language to separate design, content and Python code.

Carefully walk through the four part Django tutorial. Beware: there are three versions of the tutorial (0.96, 1.0, and “Latest”). Make sure you’re using the desired one.

For Django integration with GAE I’m using app-engine-patch.  I had first tried Google App Engine Helper for Django, but I found that app-engine-patch works much better.

Data Integration (Back-end)

Getting data to and from the S3 server will be a critical component.  I have only started looking into this, but the Amazon documentation seems very good.  The Getting Started Guide examples are presented in multiple languages (PHP, C#, Java, Perl, Ruby, Python).  A Python interface to Amazon Web Services, Boto, also looks like it might be useful.

Amazon S3 POST is an efficient way to move data to S3:

S3 Post

The back-end will require much more investigation.

For the additional database needs (account management, logging, auditing, etc.) I’ll just use the GAE Datastore.


There’s a lot of “stuff” here. Investigating and evaluating it all plus making decisions is a daunting process.

The purpose of going through these selections is to reduce the number of variables so I could start concentrating on an architecture and design that will meet project requirements. There are still many unknowns though, and I’m sure there will be major bumps in the road that will cause me to change direction.

UPDATE (11/21/2010): Beware — you get what you pay for!: Goodbye Google App Engine (GAE)



Twitter Updates