Developing a new Clinical Skills Program? Start with a Pilot Study
Pilots are the best way to explore xapimed.
Healthcare organizations are complex. Starting with a small scale, well-defined pilot project is the best way to begin exploring xapimed’s ability to deliver on training and operational needs. What’s more, you’ll be testing its performance directly within the context of your organization.
We find that most clients are able to get their pilot projects off the ground with this five step framework.
How it Works
Assemble your team
Design, align, refine
Go live + troubleshoot on the fly
Reporting outcomes
Get your rollout on the radar
Step 1: Assemble your team
The people who participate in your pilots are the most valuable assets when it comes to
adopting new technology.
Regardless of the focus of your pilot project, you will need champions from your organization
who will contribute to, coordinate, and advocate for this important work. Connect with these
people early and clearly communicate why you want to run a pilot, the projected time
commitment, and the potential outcomes of the project, making sure you explain the value of
their input and how important their feedback will be to every stage of the process.
Staff champions may include people like subject matter experts, in-house trainers or
educators, instructional designers, and members of the management team.
Step 2: Design, align, refine
With your crew lined up, it is time to define, align and refine the pilot project.
Offering your ‘crew’ a sense of ownership over their part in your pilot process can help to promote ‘buy-in’. Engaged users will also often make observations and suggestions that add value to your pilot.
Design your pilot with input from your champion crew where appropriate. Co-designing projects is one of the fastest ways to encourage staff involvement.
- What problem are you solving?
- What is the goal of the pilot?
- What is the exact scope of this pilot?
- What is the timeline for the pilot?
- What milestones will there be?
- When will the pilot end?
- How will the pilot’s success be measured?
- What resources are needed to run this pilot?
- What is the budget for this pilot?
- Who will be impacted by this pilot? How?
Align your project to existing company strategic plans and objectives to help create a business case for your pilot. If your pilot is able to achieve its own purpose, and can additionally tick a few strategic ‘boxes’, the rationale for your work will have greater relevance and persuasive impact.
An easy way to craft a ‘punchy’ business case is to use the ‘SMART’ acronym. You want to show that this pilot is: Specific, Measurable, Attainable, Relevant and Time-limited.:
Refine your project once you have secured support for your pilot. You could consider the
following questions:
- Are there additional staff who need to be approached about this project?
- What roles are there, and who will fulfil these?
- What project management systems or tools are required?
- What equipment and resources need to be organised?
- What meetings and/or training sessions need to be scheduled?
- How will staff be onboarded to the tech?
- What risks need to be managed?
- How will data and feedback be gathering during the pilot?
Step 3: Go live + troubleshoot on the fly
Perfection is your enemy during a pilot – there is always a temptation to map out the perfect system to minimize potential problems. However, if you’ve been involved in a technology roll-out, you likely know that even the most carefully thought out plan rarely works out as you expected.
Troubleshooting issues on the fly allows you to craft a much more agile solution – rightly or wrongly anticipating ‘blockers’ to success has a less secure return on investment than responding directly to actual problems if/when they arise.
A pilot fast-tracks the process of accurately identifying the technical issues or usability hurdles which might affect the successful adoption of xapimed. This gives you a chance to work with our technical team to specifically address these ‘blockers’ with tailored solutions, knowing this work is ‘money in the bank’ for future, full-scale tech roll-out.
Sometimes this process unearths ‘friction points’ which cannot be addressed within the scope or timeline of the live pilot. For example, you may have opted for a manual username and password log-in process to offer your users a greater sense of security as they use the new technology. However, your users may tell you that having to remember a username and password is annoying. A single-sign-on will solve this friction, because it means your users won’t have to remember those details. In this case, you can note this feedback for future application, and thank pilot participants for raising this important user experience issue.
The take home message here is: get feedback early and often. By listening to feedback from pilot users about their experience with the technology, and acting upon this quickly, you will both improve their user experience (as pilot participants and tech testers!), and gain valuable insights which empower you for future roll-out.
To help you plan for iteration, PDSA Cycles might be a great concept you can borrow from the world of quality improvement. This is a four-step iteration management process which is designed to be repeated across several cycles. The following stages can be built into your timeline:
Step 4: Reporting outcomes
By the end of your pilot, you will have gathered a wealth of data which can be used to evidence pilot outcomes against the goals and purpose of the project.
This might include qualitative results from:
- User satisfaction surveys
- User comments and quotes from interviews
- Feedback from team leaders
You will also have quantitative data. This might be:
- Analytics and reports for data captured during the pilot
- Calculations of cost savings
- Rates of compliance
- Course completion data
You should be able to make comparisons from data before and after the pilot to demonstrate the deliverables made possible during the pilot. This might include improvements to:
- Patient safety or experience
- Staff engagement
- Clinical processes
Presenting your results to a range of stakeholders is an elegant way to close the ‘feedback loop’ of your pilot project. This process will likely include members of your team, leadership and pilot participants themselves – think of it as an excuse to plan a ‘roadshow’ to showcase your work.
To frame your findings, you could take inspiration from the following framework:
Baseline
What was the existing problem your pilot wanted to address? What baseline were you working with regarding your users and their knowledge, skills or attitudes as they related to the technology and the pilot’s goals?
Process
How quickly were users able to confidently use the technology? What sorts of issues did you encounter? Were these able to be solved? As the technology was mastered, was there also a corresponding improvement in knowledge, skills, attitudes or processes as these relate to your organization?
Results
What impact did the pilot project have upon the behaviour of users or upon the outcomes of your service? What were the final findings of your pilot relating to the use of technology? Did the pilot show that technology can be used to improve your service? Using this comparative data, how does this look as a ‘return on investment’?
Step 5: Get your rollout on the radar
Sow strategic seeds about the future use of xapimed ‘beyond the pilot’ while you have a captive, engaged audience. Use your presentation of pilot findings to show that, when your organization tackles a full xapimed rollout, this can be done with existing product knowledge, evidence-based methodology, and experience-based confidence.
Key question: How can the key learnings taken from this pilot project be applied to future work?
Pilot Projects: Ideas
If you already have a specific pilot project idea in mind and need help getting started, please don’t hesitate to let us know – we love to provide assistance to clients who are running xapimed pilots.
However, if you need some inspiration for a pilot project, here are some examples of projects that can be completed with a small cohort within 30 days.
Mock Code Blue
The benefit of using Mock Code Blues for a pilot is that these processes are mandated, already in place, and often involve copious amounts of spreadsheets and paperwork. The value of xapimed becomes quickly apparent in this audit process that deserves to be digitized.
A Mock Code Blue pilot project is the perfect way to see our Mock Code Blue audit tool in action, whilst also leading a quality improvement initiative at your site. By loading xapimed with real-life users from your organization, you’ll be able to both trial the new technology, and keep track of who has participated in mock code activities.
Who makes a good champion for this pilot?
Connect with the colleague who is responsible for running Mock Codes within your organization. By explaining that the pilot will use xapimed to collect time-stamped data on Mock Code Blue performance, you will quickly communicate the potential value of this pilot to their work.
Possible Quantitative Outcomes
With comparison data, you’ll be able to demonstrate the time savings created by eliminating the need to manually key hard copy data into spreadsheets.
By the end of your pilot, you’ll have tried xapimed ‘in the field’ and will know whether a full-scale roll-out is feasible within your setting. In the event that it is, you will be able to use data relating to any identified training or coverage ‘gaps’ to build a case for further technology-based innovation. Furthermore, you can see if pilot participants, whether trainees or supervisors, had a better experience of Mock Code Blue training using this approach.
Micro-learning Course
If delivering a more modern and flexible learning experience is part of your vision for xapimed, micro-learning courses can have a huge impact. What’s more, this little pilot project idea has the potential to get you your system administrator wings in our Checklists, Content, Pathways, Badges and the Skills Matrix.
Delivering small packages of information at repeated intervals is one way to support the long-term, durable retention of knowledge. This can be useful for training in new clinical skills, operational protocols, or strategic goals around company culture which might benefit from habituated behaviors. Targeted initiatives, like reducing incidents of C.Diff or CLABSI, offer a defined pilot scope with a clear link to quality improvement outcomes.
Who makes a good champion for this pilot?
An ‘instructional designer’ or ‘eLearning developer’ who loves mixing things up would be a great colleague to ask to create content for your pilot. They may well also supply a list of suggested users to undertake it. Because creating a training course for pilot participants using spaced repetition harnesses the known benefits to long-term retention of important knowledge and skills, your champion will appreciate your attempt to infuse technology roll-out with proven learning principles. If you need some examples of how microlearning can look in practice, the article ‘Effective learning: Twenty rules of formulating knowledge’ by Dr. Piotr Wozniak contains ideas for how the micro-learning activities of your pilot might be structured.
Possible Quantitative Outcomes
You’ll be able to see how your team members respond to small chunks of learning activities and how this affects the retention of new knowledge. Mapping both the reliability of their recall over time, and comparing baseline data to final assessment values at the end of your pilot will also show the overall value of a microlearning approach. You may also be able to link early improvement to the retention of knowledge to a corresponding improvement in patient outcomes.
By surveying your learners at the end of the pilot, you will understand if they prefer this method of learning delivery over the traditional LMS experience, and how this sense of engagement influences professional practice.
Hand Hygiene Audit
This is a great project to work with on your infection control team, and it has a huge pool of potential participants for you to choose from because good hand hygiene is everyone’s business in a healthcare setting.
Published studies suggest that, on average, compliance with hand hygiene is around 40% (WHO Guidelines on Hand Hygiene in Health Care). Getting your organization’s compliance rate above this is a strategic goal which directly improves patients’ lives; 1 in 25 patients is affected by a healthcare-associated infection every day.
Because we have the purpose-built Hand Hygiene app, a lot of the pilot design is pre-determined by our system – all you’ll need to do is assemble your pilot teams, determine the scope of your project, and get your hand hygiene observers onboarded.
Who makes a good champion for this pilot?
If your organization is already using general staff or trained volunteers to conduct audits, these will be your natural champions. However, finding yourself some new auditors to onboard to xapimed is also an excellent way to test the ongoing scalability of the approach, and to invest in staff upskilling as you do it. By showing your auditors the quality of the data you’ll be able to collect during the pilot, and how this reduces the time needed to collect and report on hand hygiene compliance, you’ll quickly get the attention of your pilot auditing team. If you need some examples of how microlearning can look in practice, the article ‘Effective learning: Twenty rules of formulating knowledge’ by Dr. Piotr Wozniak contains ideas for how the micro-learning activities of your pilot might be structured.
Possible Quantitative Outcomes
Depending on the scope of your Hand Hygiene Audit pilot project, you may be able to view audit analytics for an accurate picture of compliance at your organization, you may have insights into the coverage of your auditing activities, and you’ll be able to compare the quality of the data you’ve gathered against existing collection methods. In addition, you might wish to compare the labor costs of this process to your organization’s existing collection methods.
Apart from offering your staff in the field, direct experience with xapimed, you’ll also be able to get valuable feedback from your auditors about their experience of our platform. If xapimed makes their job easier (which is what we’re banking on!), this leaves your auditors with more time to address any patterns of behaviors which hinder compliance rates at your company.
Nursing Orientation Program
his is arguably the most complex of the pilot projects, but it has the potential for a fabulous return on investment. Indeed, there is a clear connection between a well-resourced and relevant orientation program with improved retention rates and greater staff satisfaction.
However, bringing new nurses into your organization can be difficult to manage because a comprehensive nurse orientation program will likely have many moving parts.
To keep this under control, managers and educators need a real-time view of the progress of a new nurse’s orientation process and the competencies they gain as a result. In a busy environment with staff handovers, workload pressures and competing reporting requirements, xapimed’s Checklists, Pathways, Badges and Skills Matrix features can offer a streamlined solution.
Rather than attempting to run a complete nurse orientation program as part of your pilot, the key is to keep the scope more defined. This can be done by identifying several competencies that need to be assessed as part of the nurse orientation process. It can be a good idea to choose competencies that are used universally across the nursing population within your organization.
Who makes a good champion for this pilot?
There’s a good chance you’ll be working with the Director of Nursing Education on a project like this. This person will likely also be able to connect you with the right educators and preceptors, as well as a small cohort of new nurses for this pilot project. This pilot is an opportunity to take your orientation programs and remove the ‘friction’ from them by automating processes which are better done by machines, like data reporting, the tracking of competencies, and the assigning of new orientation activities to learners. This means that the competencies and assessment activities around which you’ll build your pilot project can typically be entered from existing documentation. This is a chance for your Nursing Education champion to identify opportunities to add value to their existing program.
Possible Quantitative Outcomes
Managers and educators will be able to use to digital skills matrix to inform real-time training and assessment decisions in the flow of their work. You’ll be able to track the rate of progress through your existing nurse orientation program, using analytics to identify any ‘bottlenecks’ which deserve to be optimized. Similarly, you’ll be able to understand how educators, preceptors and mentors can use xapimed to support new nurses in their journey with your organization.
By using triggered questionnaires, you’ll be able to capture real-time feedback from both the new nurses navigating your organization, and the educators supporting them to do so. This information will help you to better understand the challenges your new nurses and educators face within their roles and environment so that you can make sure everyone gets the support they need to feel confident and excel at patient care.
