Data Collection

Data collection

While data collection can be costly in terms of time and money, team effort and focus should always remain on improvement rather than measurement. To track performance regularly and to advance PDSA cycles the team needs just enough data to know whether changes are leading to improvement. A sampling strategy that uses 20 randomly selected patient charts per month can be statistically appropriate as well as relatively quick and easy. To make the time commitment more manageable, 5 charts could be audited each week with the results rolled up into monthly reports.  The team should designate an individual or two to collect, collate, plot, and manage the data. Many improvement projects falter or die simply because data collection is inadequate.

The team should also choose between sampling active inpatients or recent discharges. The former approach may offer several real-time advantages. Providers can be alerted to process oversights, which might create moments to improve care as well as educate. In addition, by sampling active inpatients, insights into process barriers and valid reasons to amend the new process may emerge more readily. To lessen the burden of data entry you may want to consider using self-coding and scan-able forms.

Available data collection resources may dictate methods and definitions in any given medical center. Whatever method is chosen, consistency and usefulness are critical. It is usually helpful to pilot the metric definitions and steps in data collection to learn and solve stumbling blocks. In much the same way that you perform cycles of PDSA for your care delivery improvements, your team should go through several cycles of PDSA to perfect your performance tracking system.

Here is an example of how you might refine your VTE protocol and pilot it as audit tool: take your VTE protocol and apply it as an audit tool for 10-20 patients, using 3 independent reviewers. Compare notes: did you arrive at the same risk level? Did you agree on absence or presence of contraindications to pharmacologic  prophylaxis? Did you agree on bottom line decision (is the patient on adequate prophylaxis by our protocol)? This will also give you a gross estimate of your prevalence of adequate VTE prophylaxis, even with a sample of just 10-20 patients.

There are several issues that sequential pilots of your audit tool should help resolve.

How much leeway will you use in peri-operative or trauma settings about when it is OK to NOT be on pharmacologic prophylaxis?
Sample VTE Protocols can suggest some parameters.

What are the acceptable VTE prophylaxis options versus the preferred options for each level of VTE risk? 
Realize that when auditing, there will be VTE prophylaxis options that make sense to consider as ‘adequate’ even though they are not listed as ‘recommended’ in your VTE protocol. For example, when auditing you may accept UFH 7500 SQ q12 as acceptable prophylaxis in the moderate VTE risk patient, but not list it as an option on your VTE protocol because of the lack of pre-packaged syringes or clinical trials supporting that dose.

What patients will be included in your sampling strategy?
Depending on the scope of your efforts, you may or may not want to exclude:

• OB patients
• Psychiatric ward or Behavioral Health unit patients
• Patients hospitalized < 24 or 48 hours
• Younger patient populations

Which evaluation strategy (study design) should you use?
You could use a representative sample of patients at baseline, then re-assess a representative sample after you’ve gone through an extensive effort to improve VTE prevention. This “before- after” design is simple, but does not inform the process as it unfolds, and therefore this strategy is not recommended.

We recommend auditing a smaller number of patients each month on an ongoing basis. Your audit results can be tracked and trended in run charts.

There are several common sampling strategies:

• Convenience sampling – patients are selected by reviewers because they are available on the ward, but there otherwise there is no particular selection process. Convenience samples categorized by ward or service would be a common model. 
• Random sampling – all patients in a representative population are subject to selection. At UCSD we use this model. All patients over 18 and in house for > 24 hours are assigned a number, and an Excel random number generator (a free plug-in application) produces a list of 10 patients subject for review that day. The data collector goes to the first random patient generated for the audit. This has the advantage of giving you an accurate picture of the demographics and VTE risk in your institution. The main disadvantage is the potential that some small but important patient group will only be subject to a few audits. 
• Stratified random sampling – patients from several important patient groups are randomly sampled (e.g. medical versus surgical versus orthopedic, or critical care versus non-critical care). The advantage of this method is that you can target patients groups at higher risk for VTE or with other criteria important to your VTE prevention effort.  

Before piloting and finalizing your audit tool, it will be important to pilot and finalize your VTE protocol. Feedback from the VTE protocol pilot could change the metrics and the audit form.

Download the VTE Implementation Guide Snapshot