EHR Usability

A few days ago, I wrote about Innovation, a term being overused in the EHR industry to the point where it lost all meaning. Here is another such term: Usability.

Just like Innovation, Usability is the weapon du jour against the large and/or established EHR vendors. After all, it is common knowledge that these “legacy” products all look like old Windows applications and lack usability to the point of endangering patients’ lives. On the other hand, the new and innovative EHRs, anticipated to make their debut any day now, will have so much usability that users will intuitively know how to use them before even laying their eyes on the actual product. With this new generation of EHR technology, users will be up and running their medical practice in 5 minutes and everybody in the office will be able to complete their tasks in a fraction of the time it took with the clunky, legacy EMRs built in the 90s. And all this because the new EHRs have Usability, not functionality, a.k.a. bloat, not analytical business intelligence and definitely not massive integration, a.k.a. monolithic. No, this is the minimalist age of EHR haiku. Less is better, as long as it has Usability.

Usability, according to the Usability Professionals Association, is “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use [ISO 9241-11]". Based on this definition, it stands to reason that any EHR prospective buyer should want a product with lots of Usability. Everybody wants to be effective, efficient and satisfied. So how does one go about finding such EHR?

Well, as always, CCHIT picked up the glove, and as always, CCHIT will be criticized for doing so. The 2011 Ambulatory EHR Certification includes Usability Ratings from 1 to 5 stars. The ratings are based on a Usability Testing Guide. Jurors are instructed to assess Usability of the product during and after the certification testing based on three criteria: Effectiveness, Efficiency and the subjective Satisfaction, as required by the ISO standard.  The tools for this assessment consist of 3 types of questionnaires:

• After Scenario Questionnaire (ASQ) –jurors rate perceived efficiency (time and effort), learnability, and confidence after viewing scenarios

       4 questions after each scenario –16 overall

• Perceived Usability Questionnaire (PERUSE)–jurors rate screen-level design attributes based on reasonably observable characteristics

       20 questions divided among each of the scenarios;

• System Usability Survey (SUS) –jurors rate the assessment of usability, and satisfaction with the application

       10 questions after all four scenarios have been demonstrated

The questions range from general subjective assessments in the ASQ, to very specific inquiries in PERUSE, like whether table headers are clearly indicative of the table columns content. Following the certification testing, results from all jurors are combined and weighted with more weight to specific answers and less to subjective overall impressions. The final result is the star rating, ranging from 1 to 5 Usability stars.

As of this writing, 19 Ambulatory EHRs have obtained CCHIT 2011 certification and all of them have been rated for Usability presumably according to the model described above. Of those, 12 achieved 5 stars, 6 have 4 stars and 1 has 3 stars. Amongst the 5 stars winners, one can find such “legacy” products as Epic, Allscripts and NextGen. The 4 and 3 stars awardees are rather obscure. So what can we learn from these results?

The futuristic EHR movement will probably dismiss these rankings as the usual CCHIT bias towards large vendors. Having gone through a full CCHIT certification process a couple of years ago, I can attest that the only large vendor bias I observed was in the functionality criteria, which seemed tailored to large products. Big problem. However, the testing and the jurors seemed very fair and competent. Looking at the CCHIT Usability Testing Guide, I cannot detect any bias towards any type of software. I would encourage folks to read the guide and form their own unbiased opinions. Are we then to assume that the 5 Stars EHRs have high Usability and therefore will provide satisfaction?

I don’t have a clear answer to this question. Obviously these EHRs have all their buttons and labels and text conforming to the Usability industry standards, and obviously a handful of jurors watching a vendor representative go through a bunch of preset tasks on a Webex screen felt comfortable that they understand and could use the system themselves without too much trouble. Many physicians feel the same way during vendor sales demos. However, efficiency and effectiveness can only be measured by repetitive use of the software in real life settings, for long periods of time and by a variety of users. Measuring satisfaction, the third pillar of Usability, is a different story altogether. There isn’t much satisfaction about anything in the physician community nowadays and when one is overwhelmed with patients, contemplating pay cuts every 30 days or so and bracing for unwelcome intrusion of regulators into one’s business, it’s hard to find joy in a piece of software, no matter how  well aligned the checkboxes are.

The bottom line for doctors looking for EHRs remains unchanged: caveat emptor. The footnote is that the bigger EHRs are as usable as the Usability standards dictate, just like they are as meaningful as the Meaningful Use standards dictate and when all is said and done it is still up to the individual physician user to pick the best EHR for his/her own Satisfaction.

EHR Adoption – Fix the Project Plan

HITECH was signed into law in February 2009 and the Meaningful Use clock starts in October 2011. We’re about half way through the process. How are we doing so far? Will we be ready in time? Everybody seems busy, none more so than ONC, and about $1.5 Billion has been awarded to States, Universities and Health Care organizations.

There are roughly five categories of grants: Interoperability, Adoption Assistance, Workforce Development, Pilots and HIT Research, all of them starting to take shape roughly at the same time. In parallel, ONC is working on the NHIN and NHIN-Direct interoperability frameworks, the EHR certification process is in final stages of definition, CMS is almost ready to finalize Meaningful Use criteria and NIST has finished a first draft of test procedures for whoever will be certifying EHRs during Stage 1 of Meaningful Use.

Arguably all the above are necessary endeavors towards the final goal of an EHR for every American by 2014. The only missing artifact is a Project Plan. A good old fashioned Project Plan with time-lines, milestones and above all dependencies of tasks on completion of other tasks. For example, EHR adoption assistance provided by Regional Extension Centers (REC) is dependent of the availability of a trained workforce, which in turn is dependent on the existence of training programs, which is dependent on the creation of a suitable curriculum. Not to mention that EHR vendors will also need to tap into properly trained cadres of implementers and trainers if they are to satisfy the astronomic increase in demand of their products.

While every single REC out there is issuing RFIs and RFPs for their short list of supported EHR products, the main requirement that an EHR be certified is dependent on a final definition of the certification process and on the accreditation of certification bodies, whose activities are dependent on a final version of NIST’s testing procedures, which is dependent on the definitive ruling on Stage 1 Meaningful Use criteria.

Anyone dealing with Project Management in daily life knows all too well what happens when you ignore dependencies. You miss your deadline. By a mile. And you go over budget. By a lot. And many times, the entire Project will be canceled as a result.

We, as a nation, cannot afford to cancel the HITECH Project. In a recent article, David Kibbe and Vince Kuraitis called for a Plan B. I’m an optimist. I still think that Plan A can be salvaged. However, it is imperative right now to reassess our time-lines and our milestones in view of the real status of each task, and its dependencies on all other tasks currently, and erroneously, proceeding in parallel.

Social Business

Last week my oldest son graduated from college. The football stadium was full of proud parents, most of whom watched the entire event through a camera lens. On the field, a strange medieval procession was taking place, complete with ancient looking robes, 70 lb pure silver staff, a chaplain and what looked to be a bunch of castles in the background; a typical graduation from a typical old and well-endowed University. That is until Muhammad Yunus took the stage.

I didn’t know much about Professor Yunus until that moment. He was an average looking middle age man, wearing an ill-fitting black gown and a funny looking cap. His voice is soft and he will not be remembered for his great orations. According to the graduating class of 2010 at Duke University, his speech was “yeah, OK, whatever”.  For me it was a Revelation; one of those moments that you remember for the rest of your life in great vivid detail. About five minutes into the commencement address, it wasn’t about my boy joining the ranks of educated people and it wasn’t about a parent’s pride and joy of seeing a child graduating with multiple majors and contemplating the great future of graduate school and a well-chosen career path. It was about poverty, hunger and misery everywhere, surrounding this school, these lucky children, on this glorious day, like a mighty ocean around a tiny shiny island.

Muhammad Yunus is out to eradicate poverty. As simple as that. As an economist from Bangladesh, he left his Ivory Towers and discovered a simple truth. Poverty does not reside in the poor. Poverty is arbitrarily imposed by our system on equally talented and creative people, who are denied the opportunities available to the lucky few. Professor Yunus’s life work is to bring those opportunities to the poor. He founded a bank for the poor, owned by the poor and servicing the poor with micro-loans, mainly to women, women who eventually live to witness their own child graduating from college with a professional degree.

Muhammad Yunus calls it a “Social Business” - a business whose purpose is not to amass profits for its founders and shareholders, but instead to better society and solve one social problem; a profitable and sustainable business, not a charity. His Grameen Bank for the poor is thriving in these times of global financial crisis, and expanding to other countries including the US. Muhammad Yunus is proposing to harness the power of the free market to solve the problems of poverty, hunger and inequality. In 2006, Muhammad Yunus was awarded the Nobel Prize for Peace for his work.

Why am I writing about this on a Healthcare Technology blog? After Dr. Yunus completed his commencement speech, and after the obligatory standing applause, he returned to the stage as if he forgot to say something, grabbed the microphone and said that he is now using the same principles to bring universal health care to Bangladesh, health care for every person in every village.

As we search for a solution to American Health Care problems and feverishly seek Innovation, Muhammad Yunus may already have the answers. His innovative Social Business approach, his new brand of a more humane form of capitalism, working with and within a free market system, could very well be the elusive missing piece of our own puzzle.

Innovation

If you follow the health IT media, you cannot escape the new and obligatory word, Innovation. Every self-respecting article, blog post, press release or casual comment on line is not complete unless some reference is made to Innovation, its derivatives (innovators, innovative, etc.) or compounds (foster innovation, disruptive innovation, etc.). By now I am ready to add Innovation to the infamous Do Not Use category, along with Synergy, Turn-Key and One-Stop-Shop, to name a few.

According to Merriam Webster Dictionary Innovation means the introduction of something new or a new idea, method, or device. That’s probably a bit too vague for us. The Business Dictionary has a more interesting definition of Innovation: Process by which an idea or invention is translated into a good or service for which people will pay. Since Health IT is a business, this definition makes more sense. It turns out that there are various types of Innovation. From a user point of view there is Evolutionary Innovation, which requires very little learning from the end user and not much change to routine, and there is Revolutionary Innovation which completely disrupts routine and requires learning new ways of doing things.

So are we Evolutionary or Revolutionary in Health IT? If you ask physicians, they will probably say that EHR is revolutionary, since it forces them to change their workflow and the learning curve is very steep. If, on the other hand, you take a step back, it is clear that workflow hasn’t changed much. Patients still make appointments, show up at the front desk, wait in waiting rooms, have nurses bring them to the exam room, vitals are taken, doctor steps in and out and it all ends with a claim to the insurance company. The only change is that paper has been replaced by a computer, and computers are ubiquitous in everyday life. All the talk about workflow redesign boils down to minor simplifications due to the fact that the chart is available to all simultaneously. EHRs are only incremental evolutionary innovations.

Well then, maybe we need a more Revolutionary Innovation, one for which doctors will be willing to pay. This is definitely the prescription from new entrants, or hopeful entrants, to the EHR market: the legacy EHR incumbents have failed and we need a slew of low priced new products, preferably fragmented into sub products, so that physicians can pick and choose from an ever increasing array of choices. We currently have several hundred EHRs to choose from. Maybe if we had several thousand modular choices, every doctor will be able to find a combination that fits his specific needs. It’s all about choices, or is it? There are numerous studies showing that over a certain threshold, more choices only slow down purchase rates and actually make shoppers disenchanted with their purchase. Maybe if we had just a handful of EHRs, things would be different. Maybe too much of this type of innovation is detrimental to an industry as a whole.

More recently our hopes have turned to Democratizing Innovation and hoping that Innovation will come from consumers armed with medical records. It is very likely that a healthy crop of consumer applications will be created to analyze all those medical records, provide advice, second opinions and even therapies outside the established medical settings. Some will be good and some will be harmful. Caveat emptor, as always, will be the rule and we still need to find out if this idea can be translated into a good or service for which people will pay. Yes, pay, either by hard cash or by bartering their private information for services.

To judge by current developments in Health IT, we will be witnessing both Revolutionary Innovations mentioned above, in a few short years. Will they revolutionize Medicine and Health Care? Not very likely. The most likely revolution will come from administrative simplifications, payment reform, creation of Medical Homes, education, medical research and eventually, technology inventions similar in magnitude to the silicon chip. To be sure, Health IT has a major role in all these upcoming changes, and Health IT will have to incrementally create a standardized Clinical Information Highway on the Internet to support change and improvement, but we will not be revolutionizing health care, anymore than desktop publishing has revolutionized literature or financial IT has revolutionized Wall Street.

And the Survey Says......

As I opened Healthcare IT News today, as I do every day, I was struck by the placement of the two top stories. One article describes how EHRs are linked to higher costs and worse outcomes, and the other presents a study showing how CPOE implementation decreases death rates. If you scroll down a bit you find the almost daily story about a lost hard drive and a security breach affecting thousands of patients. To the left is the top story of the day with a photo of Secretary Sebelius granting $220 million to BEACON communities to implement HIT from Maine to Hawaii.

So what is special about this page arrangement? Nothing. For many months now, we’ve been reading study upon study of how EHRs improve health care and how EHRs destroy the doctor patient relationship and adversely affect productivity. Some practitioners are on record saying that the EHR was the best decision they ever made and others are on record reporting harm to patients. I think my old friend, Dr. Graham Chiu, said it best:
“These sorts of studies are really low value in terms of evidence. The standard is of course double blind prospective placebo controlled studies which are not possible ... so we look at pre and post implementation studies for the evidence. However, the evidence ranges from increased mortality to decreased mortality ... so we need more studies.”

The most disturbing thing though is not the flurry of contradictory studies and surveys; it is the new underlying current that tends to regard folks who bring up concerns as getting in the way of progress. This goes for patient safety advocates, privacy groups, and most of all, physicians who are concerned with their ability to meet all the deadlines while maintaining financial solvency. I’m sure some are trying to derail the administration’s effort to reform health care and that probably goes beyond just EHRs and Health IT, but the vast majority is just plain and simple,  honestly and constructively concerned. While there are no valid reasons to keep medicine bound to paper charts, there are valid reasons to be concerned with transitions of such magnitude. Computerization of Medical Records does not only need to be done; it needs to be done right, and it needs to be done right the first time around.

I’m pretty sure that we can all agree that both patient safety and privacy are paramount and that both present major risks to the success of widespread adoption of HIT. Best practices indicate that, if a project is to be successful, the larger risks ought to be tackled first. Kudos to ONC for financing the SHARP research projects targeted at resolving such problems, but those are long term efforts and we also need some immediate, maybe even temporary, resolutions. Spilled, milk cannot easily be unspilled.

As everybody should have realized by now, we are about to change how medicine is practiced in this country. It’s not about where to place the buttons on the screen and it’s not about which encryption algorithm to use. It’s about what happens to you next time you get really sick. We MUST get this one right, and time IS of essence.

EHR Certification – A Delicate Balance

ARRA requires that EHR systems be certified. ONC published an IFR outlining certification criteria supporting Meaningful Use as defined by CMS. Later on ONC proposed rules for the creation of EHR certification authorities, and NIST is currently defining brand new testing procedures. In the past CCHIT certified EHRs and large portions of CCHIT certification criteria came from HL7’s work on various functional profiles.
All this means is that hundreds of functional points were required to be present, in very specific form, for an EHR system to achieve certification.

There are functional points mandating what data to capture, how it is recorded, how it is stored, how it is displayed and when it should display. Incorporating all these certification requirements into a software application created very comprehensive products with the ability to do many things, or as they came to be known “bloated EHRs”. But there is another, probably unintended, consequence when trying to enforce such incredible detail; the certification process was in effect dictating particular work flows and distinct user interfaces. That was the point where the grumblings about lack of EHR usability and complaints of EHRs not being built with physicians and patients in mind, started gaining steam.

I cannot count the times when my only response to a complaining physician was that the offending piece is required by CCHIT. Many vendors searched for places where the system need only have “the ability” to do something and created customizations allowing physicians to turn off said ability. Nothing more so than the endless, and mostly useless, alerts. At one point you could tell at first glance if an EHR was or was not CCHIT certified. All the certified ones had this strange little dropdown box next to each diagnosis in the problem list, where you could select if the problem was acute, chronic, resolved, bi-lateral, controlled, etc. I have never seen it used in real life. Does anybody think that unless we mark it chronic, there is a danger that a doctor will assume that DM is acute or bi-lateral?

Well, if it’s all so counterproductive, why are we certifying EHR technology? Honestly, I don't know that we should. The original thought was that certification will assure prospective buyers that the system is able and willing to do the job, thus separating the wheat from the chaff. Unfortunately, certification acted in a more simplistic way and separated the heavy from the lite, regardless of usability, value and stability.
The new thought at ONC is that certification should provide a guarantee that one can achieve Meaningful Use by employing a certified system. So instead of trying to dictate every detail, the ONC certification is all about the specific criteria of Meaningful Use; 25 criteria for Stage 1. As a result, some small uncertified vendors of very lite EHRs are expressing interest in the ONC certification, if the price is right, of course. CCHIT gracefully turned on a dime and created a much slimmer ARRA certification process. But is it possible that the slimming and trimming process is going too far?

Is it possible to certify too little?

It is probably too soon to tell if the new ARRA certification will be able to strike an acceptable balance between allowing vendors the freedom to innovate and ensuring that Meaningful Use can be achieved with a certified product. However, the proposed NIST test procedures indicate a possibility of certifying too little.
For example, certifying that a piece of software is capable of receiving electronic lab results in discrete format, while allowing the vendor to choose the structured format in which lab results are received could lead to the inability of future users of the software to meet Meaningful Use requirements, if the vendor selects anything other than HL7 for their preferred structured format. Laboratories do not send results out on a regular basis in either CCR or CCD, which are specifically allowed by NIST, or any format other than HL7. An EHR, or EHR module, capable of receiving lab results in a CCD format exclusively, may become ARRA certified, but its proud owner will be unable to become a Meaningful User.

A different and more troubling issue arises from looking at a requirement in a very narrow way and neglecting to account for ramifications not directly related to the particular Meaningful Use criterion. For example, testing for the ability to modify a medication list without concern for historical integrity of the list creates a strange and dangerous situation where adjustments made to the SIG (strength, dosage,etc.) of a particular prescription are allowed to overwrite the entire history of said prescription (think Warfarin). The narrow scope of testing that modifications are possible fails to assert the need for those modifications to be made in accordance to general principles enforcing the integrity of the medical record. Presumably this larger concern is left to vendor discretion. Existing vendors that certified their products under the old regime will probably provide a reliable product anyway, but new innovative vendors, writing software to published specifications and test procedures could easily create a certifiable, yet clinically unsafe product.

We are all aware that an over prescriptive certification process has not produced massive improvements in EHR offerings. The temptation to err on the side of oversimplification is large, particularly since the time is short and we must come up with a temporary certification process sooner rather than later. However, if we look at certification as a process designed to help vendors build better products and assist buyers in selecting tools to achieve Meaningful Use, then we must make sure that, while not overbearing, ARRA certification is indeed meaningful.