Tag Archive for 'Amazon'

Plunging into Web Development

ConanI’ve authored a few web sites. Nothing professionally though. I know just enough HTML, CSS, and JavaScript to be dangerous.

Now I’m faced with creating a customer-facing site that has (or will someday soon have) real requirements.

Here are a couple of the requirements I know so far:

  1. Relatively low volume traffic. The site will be public, but only registered users (customers) will have access.  No product pages, no shopping carts, no ads, no social networking. The front page is a login screen.
  2. Reliable and secure transport and storage of medical data.  At a minimum we must comply with HIPAA standards (privacy rules).

I don’t see web site development as really that different from building any other type of application. It’s all software. The architectural building blocks may be different, but the developer’s mind-set and methodologies for producing a quality product need to  be the same.

I haven’t gotten far enough along to really understand all of the deployment and maintenance issues. I’m thinking about them though. The same goes for testing. I can foresee development vs. production platform testing issues that will have to be carefully considered.

What I want to do is walk you through my rational for the selection of some of the major components and tools I’m considering using for this project.

Web Frameworks

Here’s a little historical perspective on selecting a web development framework:

choosingwebframework

Yep, that’s how it feels.  There are at least 100 options (plus a couple of my additions):

AgaviAIDA/Web | Ajile | Akelos | Apache ClickApache CocoonApache StrutsApache WicketAppFuseAraneaASP.NET MVC | Axiom Stack | BFCCakePHPCampingCatalystCherryPyCodeIgniterColdSpringCSLACppCMSDjangoDotNetNukeDrupal | ErlyWeb | eZ ComponentsFlex | FUSE | FuseboxGoogle Web ToolkitGrokGrailsHamletsHordeInterchangeItsNatIT Mill ToolkitJavaServer Faces | Jaxer | JBoss SeamKepler | Kohana | Lift | LISA | ManyDesigns PortofinoMasonMaypoleMach-IIMerbMidgardModel-GlueMonoRailMorfikNitroonTapOpenACSOpenLaszloOpenXava | Orbit | PEAR | Orinoco | PyjamasPylonsQcodoRadicoreReasonable Server FacesRIFERuby on RailsSeasideShale | Simplicity | SilverStripe (Sapphire)SmartClientSofiaSPIPSpringStripesSymfonyTapestryThinWire | Tigermouse | VaadinTurboGearsWavemakerweb2pyWebObjectsWebWork | Wigbi | YiiZendZK | Zoop | Zope 2Zope 3ztemplates

YIKES!!

As a .NET developer, my first inclination was to look at ASP.NET MVC. The two most popular and active open source frameworks are  Ruby on Rails (RoR) and Django (Python-based). To be honest, I have not spent a lot of time investigating any of the others.

Why is it that I often find myself in this situation? It’s usually not 100, but there always seems to be multiple well developed solutions for these types of problems.  I ran into the same thing a couple of years ago when I was selecting an ORM for a .NET project.

All you can do is start by taking the advice of others (“most popular”) and give one or two a try.  Not only will you get a good sense of how well the framework meets your project requirements, since there will inevitably be problems or questions you’ll also be able to evaluate documentation and community activity.

It’s like making pasta — you throw a noodle against the wall and if it sticks, you’re done cooking.  Well, not really… but you know what I mean.

Hosting

One of the major considerations is hosting. I’ve previously explored the three major cloud computing platforms.

  • Amazon EC2 would be overkill (see requirement #1). I don’t see a need for significant scale-up in the foreseeable future. Running a small on-demand EC2 instance 24/7 is more expensive (~$70/month) than just buying hosted services.  Also, supporting a complete OS platform is unnecessary work.
  • Microsoft Azure is currently in CTP (Community Technology Preview) and it’s still unclear what the pricing will be.
  • That leaves Google App Engine.  Based on the GAE Quotas, we would be able to operate under the limits for quite a while (exceeding the quotas would be a good thing).  That means GAE can provide us free hosting, which is hard to beat.

There are literally 100’s of hosting options, and most would meet our bandwidth and storage requirements at a nominal cost.  Independent of storage (see below) I guess I’m biased towards a cloud solution for two reasons:

  1. “Good Enough” isn’t Good Enough: I’ve been hosting this domain on a commercial site for about 6 years.  I’d classify my host as good enough for my personal use (family site, photo gallery, this blog, etc.).  If my hosting service went away tomorrow, no big deal. I backup everything regularly and could be up and running on a comparable host pretty quickly. But for business purposes that involve critical customer medical data, “good enough” and the possibility of the host disappearing just doesn’t cut it.
  2. Large Infrastructure: This is what makes a cloud solution so attractive. With any of the three cloud options you are buying into reliability and stability. They already have multiple data centers, security, and disaster plans in place.  You don’t have to worry about Amazon, Microsoft, or Google going away any time soon. Unless you have the resources to build it yourself, IMO using a cloud service is a good business decision.

So for now I’ll be using Google App Engine.

Data Storage

Now lets looks at requirement #2: reliable and secure data storage. At this time the best solution seems to be Amazon S3. Amazon has already put a lot of thought into this:  Creating HIPAA-Compliant Medical Data Applications with Amazon Web Services (warning: PDF).  S3 transfer and storage costs are very reasonable.  Paying only for what you use is a real benefit.

Both Google and Microsoft are very active in the Healthcare sector (Google Health and HealthVault) and I’m sure will soon have cloud storage offerings with similar features.

There are a number of web hosting sites that claim HIPAA data storage compliance, but most seem to just be using “HIPAA” as a marketing tool to attract medically related clients. I’d stay away from these.

Web Frameworks (part 2)

Deciding to use GAE quickly narrows the web framework choice down. GAE supports Python (w/ Django) and the Java 6 runtime environment. I do not believe that either ASP.NET or RoR are supported on GAE. Done deal — Django.

I know what you’re thinking.  There are many other Python-based web frameworks and even Java alternatives that I should be considering. That’s true, but Django is arguably the most popular and has a very active developers community. Also, there are several Google Code App Engine projects (see below) that support Django integration.

I did play around with RoR . The Ruby language itself is great. I love having five different ways to do the same thing. The RoR web framework is mature and has many of the same features as Django.

I looked at ASP.NET MVC, but only from a distance. Here’s a concise take from someone that recently jumped in: ASP.NET MVC Impressions after 1 week.

Development Environment

I initially setup a Windows-based Python/Django/GAE-SDK development environment but found it to be too clumsy.  I’ve settled into Ubuntu 9.04 running in a VirtualBox VM.

The Ubuntu Package Manager handled installation of all the necessary prerequisite components. Now that I think of it, I didn’t have to do a single ./configure and make. That’s progress!

I’m an old Unix hack and I quickly fell back into my first love : Emacs. After the nostalgia wore off, I needed to find a real development IDE.  There were two choices:

  1. Eclipse:  I tried using the PyDev plug-in along with some Django integration instructions I found. Google also provides some Eclipse integration, but being able to start the server and other functions from the IDE was not that important to me.  I’d rather use the command line. Also, Eclipse just seems like a real dog.
  2. Netbeans:  With the Python plug-in Netbeans works fine, so I’ll stick with it until something better comes along.

Django (Front-end)

The four features that make  Django attractive:

  • Object-relational mapper: Define your data models entirely in Python. You get a rich, dynamic database-access API for free — but you can still write SQL if needed.
  • Automatic admin interface: Save yourself the tedious work of creating interfaces for people to add and update content. Django does that automatically, and it’s production-ready.
  • Elegant URL design: Design pretty, cruft-free URLs with no framework-specific limitations. Be as flexible as you like.
  • Template system: Use Django’s powerful, extensible and designer-friendly template language to separate design, content and Python code.

Carefully walk through the four part Django tutorial. Beware: there are three versions of the tutorial (0.96, 1.0, and “Latest”). Make sure you’re using the desired one.

For Django integration with GAE I’m using app-engine-patch.  I had first tried Google App Engine Helper for Django, but I found that app-engine-patch works much better.

Data Integration (Back-end)

Getting data to and from the S3 server will be a critical component.  I have only started looking into this, but the Amazon documentation seems very good.  The Getting Started Guide examples are presented in multiple languages (PHP, C#, Java, Perl, Ruby, Python).  A Python interface to Amazon Web Services, Boto, also looks like it might be useful.

Amazon S3 POST is an efficient way to move data to S3:

S3 Post

The back-end will require much more investigation.

For the additional database needs (account management, logging, auditing, etc.) I’ll just use the GAE Datastore.

Overwhelmed

There’s a lot of “stuff” here. Investigating and evaluating it all plus making decisions is a daunting process.

The purpose of going through these selections is to reduce the number of variables so I could start concentrating on an architecture and design that will meet project requirements. There are still many unknowns though, and I’m sure there will be major bumps in the road that will cause me to change direction.

UPDATE (11/21/2010): Beware — you get what you pay for!: Goodbye Google App Engine (GAE)

Kindle 2

Kindle 2I’m a big reader. When my family told me that they ordered a Kindle for me I was pretty excited. Well, that was over two months ago.  We were notified a couple of weeks ago they would be shipping a Kindle 2. It finally arrived!

The Kindle 2 hardware has a modern sleek look and feels great in your hands. At 10.2 ounces it’s lighter than most of the books I read.

Here are the things I like:

  • Electronic-ink: The displayed text looks like a real book and can be read anywhere you’d normally be reading.  You can also change the font size to your liking. Because there’s no back light, it’s easy on the eyes.
  • Whispernet: The Amazon broadband 3G network is subscription-free when you buy the device. This gives you access to the “Kindle Store” as well as the rest of the internet (beware: see Browser notes below). Being able to download purchased books and free previews is the ultimate in instant gratification. There are a number of free e-book sites (e.g. Feedbooks)  that can also be accessed directly from the Kindle.
  • Search, bookmarks, and annotations:  At first you might think that not being able to “flip” through the pages of a book and spot interesting content would be a negative. That may be true for some people. In the long term though, being able to easily find long forgotten content along with your own bookmarks and thoughts is a significant value-added.  I see these features as a real game changer.  Here’s a simple example: How many times have you purchased a book that you already own? With an e-reader, it won’t happen again.
  • Built-in Dictionary: I’m always looking up words. With the Kindle, you just point at the word and the definition appears at the bottom of the page. Sweet!
  • Personal files: Amazon provides an e-mail based service for converting common file formats (e.g. PDF, DOC, etc.) in to e-book files. This is very handy for keeping reference and personal material on the device.

Here are the things that I’ve found annoying:

  • Electronic-ink: Static pages are great, but all page turns have a noticeable “blink”. Apparently the screen must be blanked with all black before updated text is drawn. You do get used to it though.  The other major issue are the display problems with menus and cursor movement. The pop-up menu is sometimes not completely displayed. Also, you can end up with multiple pointers on the screen if you move the cursor to quickly.
  • 5-way Pointer and Page buttons: The 5-way pointer takes some getting used to. I accidentally purchased a book when I tried to move the pointer up but inadvertently pressed select. Fortunately, I would have bought the book anyway. The page buttons (Next, Prev, and Home) seem kind of tight — you have to snap down pretty hard to activate them. They look nice, but feel clunky.
  • Keyboard: It works, but the keys are very small and hard to use.
  • Web Browser: The “experimental” Basic Web browser is completely unusable! It doesn’t render anything properly and is impossible to navigate. On the Experimental page it says that it “Works best with web sites that are mostly text.” And what sites would those be? None!

The text-to-speech feature seems the work — the voice isn’t very natural though and would get on my nerves quickly. But that’s OK because I don’t plan on using this feature.

IMHO, the benefits of electronic book reading and content management far outweigh the annoyances of the Kindle. You’re not going to be browsing the Web or typing e-mails with this device.  The Kindle is for reading books! If you expect more than that you should consider purchasing a different device.

UPDATE (3/2/09): Here’s a thorough review of reading newspapers on the Kindle 2: Reading the New York Times on Kindle 2.

Exploring Cloud Computing Development

Cloud ComputingIt’s not easy getting your arms around this one. The term Cloud Computing has become a catch-all for a number of related technologies that have been used in enterprise-class systems for many years (e.g. grid computing, SOA, virtualization, etc.).

One of the primary concerns of cloud computing in Healthcare IT is privacy and security.  A majority of the content and comments in just about every article or blog post about CC, re: health data or not, deal with these concerns. I’m going to save that discussion for a future post.

I’m also not going to dig into the multitude of business and technical trade-offs of  these “cloud” options versus more traditional SaaS and other hybrid server approaches.  People write books about this stuff and there’s a flood of Internet content that slice and dice these subjects to death.

My purpose here is to provide an overview of cloud computing from a developers point-of-view so we can begin to understand what it would take to implement custom software in the cloud.  All of the major technical aspects are well covered elsewhere and I’m not going to repeat them here. I’m just going to note the things that I think were important to take into consideration when looking at each option.

Here‘s a simplified definition of Cloud Computing that’s easy to understand and will get us started:

Cloud computing is using the internet to access someone else’s software running on someone else’s hardware in someone else’s data center while paying only for what you use.

As a consumer, for example of a social networking site or PHR lets say, this definition fits pretty well.  There’s even an EMR that is  implemented in the cloud, Practice Fusion, that would fit this definition.

As a developer though,  I want it to be my software running in the cloud so I can make use of someone else’s infrastructure in a cost effective manner.  There are currently three major CC options.  Cloud Options – Amazon, Google, & Microsoft gives a good overview of these.

The Amazon and Google diagrams below were derived from here.

Amazon Web Services

Amazon Cloud Services

The Amazon development model involves building Zen virtual machine images that are run in the cloud by EC2. That means you build your own Linux/Unix or Windows operating system image and upload it to be  run in EC2. AWS has many pre-configured images that you can start with and customize to your needs. There are web service APIs (via WSDL) for the additional support services like S3, SimpleDB, and SQS.  Because you are building self-contained OS images, you are responsible for your own development and deployment tools.

AWS is the most mature of the CC options.  Applications that require the processing of huge amounts of data can make effective you of the AWS on-demand EC2 instances which are managed by Hadoop.

If you have previous virtual machine experience (e.g. with  Microsoft Virtual PC 2007 or VirtualBox) one of the main differences working with EC2 images is that they do not provide persistent storage. The EC2 instances have anywhere from 160 GB to 1.7 TB of attached storage but it disappears as soon as the instance is shut down. If you want to save data you have to use S3, SimpleDB, or your own remote storage server.

It seems to me that having to manage OS images along with applications development could be burdensome.  On the other hand, having complete control over your operating environment gives you maximum flexibility.

A good example of using AWS is here: How We Built a Web Hosting Infrastructure on EC2.

Google AppEngine

Google App Engine

GAE allows you to run Python/Django web applications in the cloud.  Google provides a set of development tools for this purpose. i.e. You can develop your application within the GAE run-time environment on our local system and deploy it after it’s been debugged and working the way you want it.

Google provides entity-based SQL-like (GQL) back-end data storage on their scalable infrastructure (BigTable) that will support very large data sets. Integration with Google Accounts allows for simplified user authentication.

From the GAE web site:  “This is a preview release of Google App Engine. For now, applications are restricted to the free quota limits.”

Microsoft Windows Azure

Microsoft Windows Azure

Azure is essentially a Windows OS running in the cloud.  You are effectively uploading and running  your ASP.NET (IIS7) or .NET (3.5) application.  Microsoft provides tight integration of Azure development directly into Visual Studio 2008.

For enterprise Microsoft developers the .NET Services and SQL Data Services (SDS) will make Azure a very attractive option.  The Live Framework provides a resource model that includes access to the Microsoft Live Mesh services.

Bottom line for Azure: If you’re already a .NET programmer, Microsoft is creating a very comfortable path for you to migrate to their cloud.

Azure is now in CTP and is expected to be released later this year.

UPDATE (4/27/09) Here’s a good Azure article:  Patterns For High Availability, Scalability, And Computing Power With Windows Azure.

Getting Started

All three companies make it pretty easy to get software up and running in the cloud. The documentation is generally good, and each has a quick start tutorial to get you going. I tried out the Google App Engine tutorial and had Bob in the Clouds on their server in about 30 minutes.

Bob's Guest Book

Stop by and sign my cloud guest book!

Misc. Notes:

  • All three systems have Web portal tools for managing and monitoring uploaded applications.
  • The Dr. Dobbs article Computing in the Clouds has a more detailed look at AWS and GAE development.

Which is Best for You?

One of the first things that struck me about these options is how different they all are.  Because of this, from a developer’s point-of-view I think you’ll quickly have a gut feeling about which one best matches your current skill sets and project requirements. The development components are just one piece of the selection process puzzle though. Which one you actually might end up using (it could very well be none) will also be based on all your other technical and business needs.

UPDATE (6/23/09): Here’s a good high level cloud computing discussion: Reflections on Executive Briefing Event: Cloud & RIA.  I like the phrase “Cloud Computing is Elastic” because it captures most the appealing aspects of the technology.  It’s no wonder Amazon latched on to that one — EC2.

Subscribe

Categories

Twitter Updates