How do you measure Telehealth Success?

As the quote below highlights, measuring success can be a tricky business. Not all metrics we could collect are meaningful and some relevant aspects of telehealth success are difficult to measure accurately.

Not everything that can be counted counts.

Not everything that counts can be counted.

— William Bruce Cameron

In a recent conversation with a client of mine, the CEO of a mental health agency, we discussed his need for good measures for the C-suite to ascertain the health and success of their telehealth services. His goals? To gain more insight into this strategic priority and knowing whether, when, and where to allocate more resources.

As many of my articles have discussed, there are a multitude of components at play to make a telehealth service work. With telehealth being defined as “delivering care at a distance”, telehealth cannot be simplified to the mere video chat of a clinician with a patient (or client, in the mental health world).

Rather “telehealth” is the totality of the workflows and the workplaces, the technologies and the policies, the scheduling and the billing.

But how do you measure success?

Input, Output, Quality, and Outcome Measures

We’ll get back to measuring success in a minute, but let’s first briefly cover the basics.

Ultimately we are talking about measuring the performance of a telehealth service. There are a number of types of performance measures, some of which are useful and some are essentially meaningless (i.e., they don’t count – see the quote above).

Here are the four most relevant types of measuring telehealth performance:

• Input Measures – resources, e.g., time, staff, money, spent on delivering the service

• Output Measures – quantifiable results, e.g., the number of visits or patients seen

• Quality Measures – characteristics of the results, e.g. patient or physician satisfaction

• Outcome Measures – impact and value generated, e.g., improved health, reduced ED visits

The first two — Input & Output Measures — are relatively easy to come by and are what is most often measured. How many visits did we have? How many hours did staff spend on seeing patients? How much did we spend on telehealth licensing fees? How much revenue did we generate?

In one example of a telehealth outcome measure, one of the longest-running telehealth programs in the US continues to measure the number of miles saved by patients (and providers) that did not have to drive to their appointment. While quite an impressive metric (I think they are cumulatively well on their way to Mars), it bears by itself very little value. What can be derived from it, though, is driving time saved (and the resultant personal or communal productivity gain), gallons of gasoline saved, statistical car accidents avoided, etc. Which are the more meaningful outcome measures.

The latter two types of performance measures — Quality and Outcome — are, however, much more relevant to assessing the success of a telehealth service, though they require a little bit more involvement to collect.

The most useful examples of Telehealth Quality measures include the satisfaction of patients, of providers and of the staff (and we’ll dive deeper into these measures in the next article). The outcome measures can be about continuity of care or potentially avoidable utilization, such as an ER visit or an admission. Patient engagement is another outcome measure, as is the percentage of telemedicine visits that were reimbursed.

How do you measure success?

In the most basic definition in the context of today’s discussion, I simply define success as meeting or exceeding a pre-defined goal. Without targets, no success (or failure) can be declared.

Is 27 visits good? A Revenue of $3,853? 3.8 televisits per provider per day? A Net Promoter Score of 47?

We don’t know.

We can only measure success when we set ourselves a goal and then see if we can achieve it. But even setting goals is fraught with challenges. How do you know what a good goal is? How do you objectively and meaningfully set a goal (which is a topic for another article).

Action: The Ultimate Value of Measuring Performance

In my opinion there is only one valid business reason to justify the effort that goes into the collection, analysis, and reporting of metrics: Clarity of Action.

What I advise all of my clients as we establish performance dashboards is to have establish clarity around the following aspects for each metric:

• Ownership: who is the executive owner and who is the operational owner of that metric?
• Definition: how is the metric collected and analyzed? how often? how should it be reported?
• Expectations: what are the targets for acceptable performance? for success?
• Actions: what actions will be taken if the metric does not meet the target? what if it exceeds the target? what if it hits the target?

Especially the last distinction of pre-defining the action to be taken is what distinguishes a great organizational culture from a mediocre culture.

If there is no clear pre-defined action, the organization is wasting precious time and resources on the collection and analysis of the data, resulting in data fatigue. Any insights that could be gained from it are lost, since without clear action, not much attention is paid to the data.

Also, once you ask the question of “what actions will we take”, so-called “vanity metrics” quickly disappear, because they often don’t drive any meaningful action.

To use the example from above, a saving of “1 Million miles driven” is definitely a neat round number, but how would we know that it shouldn’t be 2 Million miles? Or that some visits should have occurred in person and led to negative health outcomes?

And where should we set our target and our resultant actions? 20,000 patient miles saved a month? Why not 18,000 or 24,000? And what actions will we take if we don’t meet the target – recruit patients who live farther away? Or, worse, ask those who live closer to come in, so they don’t count against our quota (i.e., maximizing miles saved per televisit)?

Key Success Metrics for Telehealth

Now that we laid the groundwork for creating a Telehealth Success Dashboard, I will focus my next article on developing a framework of metrics to include the most valuable and meaningful input, output, quality, and outcome measures.

Along with the pre-defined targets and actions to accompany each metric, that is how each and every C-suite should manage their telehealth services to enable the delivery of extraordinary care.

If you need to talk about your dashboard yet this week, connect with me through the means most convenient to you.

To receive articles like these in your Inbox every week, you can subscribe to Christian’s Telehealth Tuesday Newsletter.

Christian Milaster and his team optimize Telehealth Services for health systems and physician practices. Christian is the Founder and President of Ingenium Digital Health Advisors where he and his expert consortium partner with healthcare leaders to enable the delivery of extraordinary care.

Contact Christian by phone or text at 657-464-3648, via email, or video chat.