I am relentlessly driven to optimize efficiency and effectiveness in everything I do. Having been involved in the design and implementation of technical solutions for close to 35 years, I’ve long been frustrated by the corporate concept of “running a pilot”.
The name pilot here, as opposed to our trustworthy-looking aircraft pilot in our image above, comes from the concept of a “pilot experiment” or prototype, that’s being tested to “steer” the direction of the final solution or service.
Most of the time, pilots are (arbitrarily) time boxed (e.g., 4 weeks, 3 months, 100 days, etc.) or stop after a quantifiable number (e.g., number of clients, sales, etc.) has been reached. While those metrics are easy to measure, they are also quite irrelevant to evaluating the actual success of the pilot.
Using a Proof-of-Concept
A few years back I asked myself the simple question: “What is the purpose of a pilot?” And most importantly: “When is the pilot actually done?”
The answers are just as simple: The pilot should be done when we have confidence that the solution being “piloted” is ready to be deployed to more users — or when we know we need to go back to the drawing board to redesign the solution.
Thus the purpose of the pilot is “that it works” — that users like and that it does what it’s supposed to do.
From a financial point of view, a solution that is in “pilot mode” does not create much value. The scope is typically very limited and very few users have access to the solution. The objective therefore is to get beyond the pilot phase quickly, so that the solution can be offered to a broader range of users, presumably for a fee of sorts.
So what if we were to flip the purpose of a “pilot” on its head? When the purpose no longer is “let’s test this out for 3 months and see what we got”, then the purpose can become: “How can we most quickly validate the reliability, usability, and value to the users?”
Which led me to this definitions of a Proof-of-Concept:
The purpose of a Proof-of-Concept is to validate assumptions.
We make assumptions about the world around us hundreds of times each day. Which is help, because if we had to measure and validate everything, we would get stuck in analysis paralysis.
We are therefore so used to making assumptions. Most of the time during the design and development of a solution or service, such as a new telehealth services offering, we are not even aware that we are making assumptions.
In simple terms, an assumption is something you expect to be true: “The earth is round, the sky is blue and the stars come out at night.” It’s based on experience, past observations, and speculative guesses. But those assumptions may not be accurate: The earth is not perfectly round, the sky is rather clear and we know that the stars are there all day.
In order to run an efficient Proof-of-Concept (PoC), the first step is to identify and document all assumptions of the solution. In the telehealth services world this includes assumptions around patient acceptance, patient engagement, equipment reliability, physician acceptance, etc.: They’ll love it, they want it, they’ll use it.
Next, the assumptions are prioritized to identify the set of the vital assumptions that are most meaningful to the primary objective of the solution.
With a set of defined and validated assumptions in hand, the next step is to identify which metric could confirm the validity of the assumption best.
Each metric must in turn have a clear definition, an associated owner, a collection mechanism, analysis guidelines, and reporting guidelines. Most importantly, each metric must have a goal associated with it that basically indicates when the assumption has been confirmed.
Throughout the proof-of-concept, all metrics are periodically collected, analyzed and reported against the goals.
Once a significant number of metrics validate that the assumptions made have been proven, the proof-of-concept phase can come to an end and the deployment to more users can begin.
Value of Proof-of-Concepts Pilots
One key benefit of this approach is that a proof-of-concept is done when all high-priority assumptions have been sufficiently validated, which oftentimes is much faster than a typical period for pilots.
Secondly, virtually at every stage of the Proof-of-Concept the team knows where they stand. They can confidently report on the % complete of the proof-of-concept.
It is not uncommon for a proof-of-concept to wrap-up in a few weeks once all tweaks and improvements have been made to achieve the desired results, thereby accelerating the larger rollout.
So next time you want to launch a new telehealth service, employ a proof-of-concept rather than a pilot.
Are you validating your assumptions when launching a new service? Or do you just put it out there and hope it’ll work and fix any problems along the way? Reach out to me to share your experience, I’d love to hear it.
Christian Milaster and his team optimize Telehealth Services for health systems and physician practices. Christian is the Founder and President of Ingenium Digital Health Advisors where he and his expert consortium partner with healthcare leaders to enable the delivery of extraordinary care.
Contact Christian by phone or text at 657-464-3648, via email, or video chat.
Christian Milaster optimizes Telehealth Services for health systems and physician practices as the interim Telehealth Program Director. He serves as a Digital Health & Telehealth Advisor to startups and established Health IT firms.
Christian is a Master Builder of Digital Health and Telehealth Programs and the Founder and President of Ingenium Digital Health Advisors, a boutique consultancy focused on enabling the effective delivery of extraordinary care through workflow optimization and the judicious use of technology.
Born, raised, and educated as an Engineer in Germany, Christian started his career at IBM Global Services before joining the Mayo Clinic in Minnesota, where he worked for 12 years in various roles before launching Ingenium in 2012.