Or have you already hired Proof-of-Concepts?
As a German engineer it seems that I’ve been pre-wired to be relentlessly driven to optimize efficiency and effectiveness in everything I do. Having been involved in the design and implementation of technical solutions to improve service delivery for over 35 years, I’ve long been frustrated by the concept of “running a pilot”.
The name “pilot” here, as opposed to our mischievous-looking aircraft pilot in our image above, comes from the concept of atrial, and experiment or a prototype, that’s being tested before rolling it out to everyone.
Most of the time, pilots are (arbitrarily) time boxed (e.g., 4 weeks, 3 months, 100 days, etc.) or stop after a quantifiable number (e.g., number of clients, sales, etc.) has been reached. While those metrics are easy to measure, they are also quite irrelevant to evaluating the actual success of the solution or service you are piloting.
Using a Proof-of-Concept
A few years back I asked myself the simple question: “What is the purpose of a pilot?” And more importantly: “When is the pilot actually done?”
The answers are just as straightforward: The pilot should be done when we have confidence that the solution being “piloted” is ready to be deployed to more users — or when we know we need to go back to the drawing board to improve the solution.
Thus the purpose of the pilot is to confirm “that it works” — that users like and that it does what it’s supposed to do.
From a financial point of view, a solution that is in “pilot mode” does not create much value. The scope is typically very limited and very few patients have access to the solution. The objective therefore is to get beyond the pilot phase quickly, so that the solution can be offered to a broader range of patients, presumably creating financial (and clinical!) value.
So what if we were to flip the purpose of a “pilot” on its head? When the purpose no longer is “let’s test this out for 3 months and see what we got”, then the purpose can become: “How can we most quickly validate the reliability, usability, and value to the patients?”
Which led me to this definition of a Proof-of-Concept:
| The purpose of a Proof-of-Concept is to validate assumptions.
Identifying and Prioritizing Assumptions
We make assumptions about the world around us hundreds of times each day. Which is helpful, because if we had to measure and validate everything, we would get stuck in analysis paralysis.
We are therefore so used to making assumptions that most of the time during the design and development of a solution or service, such as a new telehealth services offering, we are not even aware that we are making assumptions.
In simple terms, an assumption is something you expect to be true: “The earth is round, the sky is blue and the stars come out at night.” It’s based on experience, past observations, and (most concerning) speculative guesses. But those assumptions may not be accurate: The earth is not perfectly round, the sky is rather clear and we know that the stars are there all day.
In order to run an efficient and effective Proof-of-Concept (PoC), the first step is to identify and document all assumptions of the solution. In the telehealth services world this includes assumptions around patient acceptance, patient engagement, equipment reliability, physician acceptance, reimbursement, etc.: They’ll love it, they want it, they’ll use it (and we get paid).
Prioritization is Key
If you’ve done a good job, you should have listed at least two dozen assumptions. In most circumstances it is not feasible to test all, nor is it desirable because many of the assumptions are connected and validating one will indirectly validate the related assumptions.
Assemble your multi-disciplinary team and find consensus on which assumptions would be valuable to each stakeholder to validate. This is important for future buy-in for the larger rollout. Next, make sure that the selected assumptions do indeed validate the primary objective of the project.
Validating Assumptions
Validating AssumptionsWith a set of defined and prioritized assumptions in hand, the next step is to identify which metric could best confirm the validity of the assumption.
E.g., for clinician acceptance it is a satisfaction score on a 5-point scale. For patient acceptance it could be a net promoter score. For utilization it would be a simple count.
Each metric must in turn have a clear definition, an associated owner, a collection mechanism, and guidelines for analysis and reporting. This is needed so that each owner knows how to collect, analyze, and present the metric.
Most importantly, each metric must have a goal, a target associated with it that basically indicates when the assumption has been confirmed. E.g., for clinician satisfaction you could set the goal at 4.5 on a 5-point Likert Scale or a net promoter score (NPS) of 60 or higher for patients’ “likelihood to recommend”).
Throughout the proof-of-concept, all metrics are periodically collected, analyzed and reported against the goals.
Once a significant number of metrics validate that the assumptions made have been proven, the proof-of-concept phase can come to an end and the deployment to more users can begin.
The Value of Proof-of-Concepts Pilots
One key benefit of this approach is that a proof-of-concept is done when all high-priority assumptions have been sufficiently validated, which oftentimes is much faster than a typical period for pilots.
Secondly, virtually at every stage of the Proof-of-Concept the team knows where they stand. They can confidently report on the % complete of the proof-of-concept — how many assumptions have been validated thus far? Are you closing the gap between current and targeted performance?
It is not uncommon for a proof-of-concept to wrap-up in a few weeks once all tweaks and improvements have been made to achieve the desired results, thereby accelerating the larger rollout (and value creation).
So the next time you want to launch a new telehealth service, employ a proof-of-concept rather than a pilot.
Look for next week’s article for a “from the trenches” for a real-life application of the “proof of concept” to (re-)launch a new primary telehealth service at a rural FQHC in Washington state.
Are you validating your assumptions when launching a new service? Or do you just put it out there and hope it’ll work and fix any problems along the way? Share your experience in the comments or reach out to me to discuss — I’d love to hear about it.
To receive articles like these in your Inbox every week, you can subscribe to Christian’s Telehealth Tuesday Newsletter.
Christian Milaster and his team optimize Telehealth Services for health systems and physician practices. Christian is the Founder and President of Ingenium Digital Health Advisors where he and his expert consortium partner with healthcare leaders to enable the delivery of extraordinary care.
Contact Christian by phone or text at 657-464-3648, via email, or video chat.
