In 1997, the Agency for Healthcare Research and Quality (AHRQ) created 12 Evidence-based Practice Centers (EPCs). The EPCs' mission is to “develop evidence reports and technology assessments on topics relevant to clinical, social science/behavioral, economic, and other health care organization and delivery issues.” The principal products of the EPCs are called evidence reports. EPC evidence reports are evaluations of the scientific literature on topics of interest to policymakers, clinicians, and other decisionmakers. The reports are requested by organizations representing these interested parties. Like any process, the EPC process is subject to improvement. The importance of process improvement to the EPC program is reflected both in the existence and the content of the annual AHRQ meeting with EPCs and partner organizations.

Two areas have been evident targets for improvement: process efficiency and stakeholder satisfaction (where stakeholders are both direct participants in the EPC effort as well as other parties with an interest in EPC products). This report describes a project initiated by the Duke Evidence-based Practice Center to contribute to the improvement of the EPC process, focusing on specific actions that could serve to improve process efficiency and stakeholder satisfaction. We organized our efforts around the concept of a “statement of work (SOW).” As a contractual document, the SOW is the tangible manifestation of the objectives of each EPC project, serving as a foundation for interaction between involved parties, including technical details such as budget, timeline, and deliverables.

The project proceeded in four steps. In Step 1 we interviewed participants in the EPC process in order to establish a deeper understanding of the current process. In Step 2 we reviewed the technical contracting literature relating to formulating SOWs for the production of policy reports and similar intellectual products, and policy literature related to the determinants of a successful policy analysis. In Step 3 we applied the Current Reality Tree (CRT) technique from the Theory of Constraints (TOC) to synthesize the information provided in the first two steps in a way that highlights core constraints (i.e., constraints that are the most important targets for improvement.) In the final step (Step 4) we reviewed the information collected in the interviews and literature review in the context of the TOC exercise. The objective was to identify promising solutions for the core constraints, which may serve as input to an action plan by AHRQ in collaboration with EPCs and partners. With these solutions in mind, we constructed a Future Reality Tree, an illustration of the impact of the proposed solutions on the EPC's SOW process.

We utilized the Theory of Constraints for the next steps in the project. Our interviews with EPC researchers, partners, and AHRQ allowed us to identify a common undesirable effect (UDE) related to EPC reports: a report that does not get used is unsuccessful and is deemed fundamentally undesirable. We formulated a current reality tree (CRT) working from this fundamental UDE at the top through proximate causes, and finally root causes. In the next stage we extracted a list of root causes from our interviews.

Following TOC methods, we next explored the relationship between these root causes in order to identify one (or two) major problems that lead to the rest, and ultimately result in a report not meeting the partner's needs, and therefore not being used. Of these, the first (partner does not know how to conceptualize and articulate needs, objectives, and specifications) rates as a core problem or the constraint that prevents an EPC from achieving its goal because it affects most other problems. This is the reason why specifications are insufficient in the first place, which leads to an underestimation of the volume of work, results in new cost and time requirements that lead to delays in the report, and finally results in the report not being used.

The interviews, literature search, and input from an EPC directors' meeting identified several potential injections/change strategies: Two injections: establish an ongoing relationship between partners, EPC, and AHRQ early in the process, and utilize a changes clause were deemed to be the most practical means to exert impact on the SOW process by addressing the core constraint, partner does not know how to conceptualize and articulate needs, objectives and specifications of the SOW.

With these concepts of UDEs, root causes, and injections we constructed the Future Reality Tree for the SOW process; the ideal process that EPCs should follow in order to develop evidence reports that meet the needs of the partners.