Data Collection

Data collection

While data collection can be costly in terms of time and money, the focus should remain on improvement rather than measurement. To track performance regularly and to advance PDSA cycles the team needs just enough data to know whether changes are leading to improvement. A sampling strategy that uses 20 randomly selected patient charts per month can be statistically appropriate as well as relatively quick and easy. To make the time commitment more manageable, 5 charts could be audited each week with the results rolled up into monthly reports. The team should designate an individual or two to collect, collate, plot, and manage the data. Many improvement projects falter or die simply because data collection is inadequate.

The team should also choose between sampling active inpatients or recent discharges. The former approach may offer several real-time advantages. Providers can be alerted to prophylaxis oversights, which might create moments to improve care as well as educate. In addition, by sampling active inpatients, insights into process barriers and valid reasons to amend the new process may emerge more readily. Self-coding and scan-able forms can lessen the burden of data entry.

Available data collection resources may dictate methods and definitions in any given medical center. Whatever method is chosen, consistency and usefulness are critical. It is usually helpful to pilot the metric definitions and steps in data collection to learn and solve stumbling blocks. In much the same way the team performs cycles of PDSA for care delivery improvements, it should go through several cycles of PDSA to perfect performance tracking system.

For example, to refine the VTE protocol and develop it as a valid audit tool, the team can apply the VTE protocol to audit 10-20 patients, using 3 independent reviewers. Did the reviewers arrive at the same risk level? Did they agree on absence or presence of contraindications to pharmacologic prophylaxis? Did they share the same conclusion about whether the patient was receiving adequate prophylaxis?

There are several issues that sequential pilots of the audit tool should help resolve.

How much time is acceptable in peri-operative or trauma settings for a patient not to be on pharmacologic prophylaxis? View sample VTE Protocols which can suggest some parameters.

What are the acceptable VTE prophylaxis options versus preferred options for each level of VTE risk?  Realize that when auditing, there will be VTE prophylaxis options that make sense to consider as “adequate” even though they are not listed as “recommended” in the VTE protocol. For example, the auditor may accept UFH 7500 SQ q12 as acceptable prophylaxis in the moderate VTE risk patient, even if it is not listed as an option on the VTE protocol because of the lack of pre-packaged syringes or clinical trials supporting that regimen.

What patients will be included in the sampling?
Depending on the scope of the initiative, it may make sense to exclude:

• Obstetric patients
• Psychiatry or behavioral health unit patients
• Patients hospitalized < 24 or 48 hours
• Younger patient populations

Which data collection strategy should the team use for performance tracking?
The team could look at a representative sample of patients at baseline, and then repeat with a representative sample after introducing the VTE protocol. This “before- after” approach is simple, but the data can be misleading. We have found that day-to-day variation in prevalence of VTE prophylaxis can be as wide as 35%. Such a wide range of variation indicates multiple sampling events are necessary to ensure accurate conclusions. Rather than using just several data points before an intervention, we recommend using at least 20 data points before an intervention and as many as necessary after the intervention to determine the new steady-state prevalence of prophylaxis. Results can be tracked and trended in run charts.

There are several common sampling strategies:

• Convenience sampling – patients are selected by reviewers because they are available on the ward, but otherwise there is no particular selection process. Convenience samples categorized by ward or service would be a common model.
• Random sampling – all patients in a representative population are subject to selection. At UCSD we use this model. All patients over 18 and in house for > 24 hours are assigned a number, and an Excel random number generator (a free plug-in application) produces a list of 10 patients subject for review that day. The data collector goes to the first random patient generated for the audit. This has the advantage of giving an accurate picture of the demographics and VTE risk in the institution. The main disadvantage is the potential that some small but important patient group will only be subject to a few audits.
• Stratified random sampling – patients from several important patient groups are randomly sampled (e.g. medical versus surgical versus orthopedic, or critical care versus non-critical care). The advantage of this method is the ability to target patient groups at higher risk for VTE, or with other criteria important to the VTE prevention effort.  

Before piloting and finalizing an audit tool, it will be important to pilot and finalize the VTE protocol. Feedback from the VTE protocol pilot could change the audit form.