Background

Pharmacist-led programs and clinics have been integrated into primary and specialty care clinics in a variety of ways, for example, to improve diabetes outcomes via patient education and counseling. However, factors important to the implementation of different outpatient pharmacy models have not been well elucidated.

Objective

To identify provider- and health system-level drivers of implementation and sustainability of pharmacy-led programs in the outpatient setting.

Design

Qualitative study of key informants using semi-structured interviews of individuals working in various roles throughout a large health system, including ambulatory clinical pharmacists, pharmacy managers, medical directors and physician leaders, and operations and quality managers.

Participants

Key informants (n=19) with leadership roles in pharmacy programs and front-line experience providing integrated pharmacy care were selected purposively and with snowball sampling.

Approach

We coded the interviews using a codebook derived from the 2022 Consolidated Framework for Implementation Research (CFIR), which details various internal and external factors important for implementation.

Key results

We identified the following themes related to implementing ambulatory care pharmacy programs: (1) pharmacy programs varied in their level of embeddedness in the outpatient clinic, (2) establishing pharmacy program required leadership advocacy and coordination among stakeholders, (3) continued operations required integrated workflows and demonstrated value to the health system and clinicians, and (4) established revenue streams or added indirect value and continued improvement of integration sustained programs over time.

Conclusions

External policies and incentives such as new reimbursement codes and quality measurement programs that rely on pharmacy input play a significant role in shaping the design, implementation, and sustainability of health system outpatient pharmacy programs. Ensuring that quality metrics used in value-based contracts or programs demonstrate pharmacy benefits will be critical to supporting and growing pharmacy programs.

Background

Despite widespread adoption of patient-centered medical home (PCMH), little is known about why practices pursue PCMH and what is needed to undergo transformation.

Objective

Examine reasons practices obtained and maintained PCMH recognition and what resources were needed.

Design

Qualitative study of practice leader perspectives on PCMH transformation, based on a random sample of primary care practices engaged in PCMH transformation, stratified by US region, practice size, PCMH recognition history, and practice use of Consumer Assessment of Healthcare Providers and Systems (CAHPS®) PCMH survey.

Participants

105 practice leaders from 294 sampled practices (36% response rate).

Approach

Content analysis of interviews with practice leaders to identify themes.

Results

Most practice leaders had local control of PCMH transformation decisions, even if practices adopted quality initiatives under the direction of an organization or network. Financial incentives, being in a statewide effort, and the intrinsic desire to improve care or experiences were the most common reasons practice leaders decided to obtain PCMH recognition and pursue associated care delivery changes. Leadership support and direction were highlighted as essential throughout PCMH transformation. Practice leaders reported needing specialized staff knowledge and significant resources to meet PCMH requirements, including staff knowledgeable about how to implement PCMH changes, track and monitor improvements, and navigate implementation of simultaneous changes, and staff with specific quality improvement (QI) expertise related to evaluating changes and scaling-up programs.

Conclusion

PCMH efforts necessitated support and assistance to frontline, on-site practice leaders leading care delivery changes. Such change efforts should include financial incentives (e.g., direct payment or additional reimbursement), leadership direction and support, and internal or external staff with experience with the PCMH application process, implementation changes, and QI expertise in monitoring process and outcome data. Policies that recognize and meet the needs of on-site practice leaders will better promote primary care practice transformation and move practices further toward their PCMH transformation goals.

Objectives

To examine how primary care practices use the Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey and its patient-centered medical home (PCMH) items during their PCMH transition.

Study design

Qualitative study of practices' use of patient experience data during PCMH transformation, based on a random sample of primary care practices engaged in PCMH transformation, stratified by region, practice size, PCMH recognition history, and use of the CAHPS PCMH survey.

Methods

We interviewed 105 practice leaders from 294 sampled practices (36% response rate) and used content analysis to identify themes about uses of patient experience data for practice improvement during PCMH transformation.

Results

Patient experience data were used primarily to assess and track improvements toward PCMH goals and as quality improvement (QI) metrics. CAHPS measures were used most often to discuss best practices, share data with patient advisory councils, and improve provider performance. The CAHPS PCMH survey helped practices improve patient-centeredness, particularly in coordinating care and supporting patient self-management and communication. The CAHPS PCMH items that assisted practices most were about patient self-management and provider referrals. Most practice leaders using the CAHPS PCMH survey felt that its items were actionable for standardizing PCMH changes or making structural changes. Practices administering CAHPS surveys focused on a more diverse set of QI areas.

Conclusions

CAHPS surveys were considered actionable for PCMH transformation and used in standardizing and coordinating care. The CAHPS PCMH items were considered integral to the continuous QI needed for moving beyond formal PCMH recognition and maximizing transformation. This supports the National Committee for Quality Assurance's recommendation to administer the CAHPS or CAHPS PCMH survey for PCMH transformation.

Background: Surveys of internet panels such as Amazon’s Mechanical Turk (MTurk) are common in health research. Nonresponse in longitudinal studies can limit inferences about change over time. Objective: This study aimed to (1) describe the patterns of survey responses and nonresponse among MTurk members with back pain, (2) identify factors associated with survey response over time, (3) assess the impact of nonresponse on sample characteristics, and (4) assess how well inverse probability weighting can account for differences in sample composition. Methods: We surveyed adult MTurk workers who identified as having back pain. We report participation trends over 3 survey waves and use stepwise logistic regression to identify factors related to survey participation in successive waves. Results: A total of 1678 adults participated in wave 1. Of those, 983 (59%) participated in wave 2 and 703 (42%) in wave 3. Participants who did not drop out took less time to complete previous surveys (30 min vs 35 min in wave 1, P<.001; 24 min vs 26 min in wave 2, P=.02) and reported having fewer health conditions (5.88 vs 6.6, P<.001). In multivariate models predicting responding at wave 2, lower odds of participation were associated with more time to complete the baseline survey (odds ratio [OR] 0.98, 95% CI 0.97-0.99), being Hispanic (compared with non-Hispanic, OR 0.69, 95% CI 0.49-0.96), having a bachelor’s degree as their terminal degree (compared with all other levels of education, OR 0.58, 95% CI 0.46-0.73), having more pain interference and intensity (OR 0.75, 95% CI 0.64-0.89), and having more health conditions. In contrast, older respondents (older than 45 years age compared with 18-24 years age) were more likely to respond to the wave 2 survey (OR 2.63 and 3.79, respectively) and those whose marital status was divorced (OR 1.81) and separated (OR 1.77) were also more likely to respond to the wave 2 survey. Weighted analysis showed slight differences in sample demographics and conditions and larger differences in pain assessments, particularly for those who responded to wave 2. Conclusions: Longitudinal studies on MTurk have large, differential dropouts between waves. This study provided information about the individuals more likely to drop out over time, which can help researchers prepare for future surveys.

Background and objectives

Health care organizations track patient experience data, identify areas of improvement, monitor provider performance, and assist providers in improving their interactions with patients. Some practices use one-on-one provider counseling ("shadow coaching") to identify and modify provider behaviors. A recent evaluation of a large shadow coaching program found statistically significant improvements in coached providers' patient experience scores immediately after being coached. This study aimed to examine the content of the recommendations given to those providers aimed at improving provider-patient interactions, characterize these recommendations, and examine their actionability.

Methods

Providers at a large, urban federally qualified health center were selected for coaching based on Clinician and Group Consumer Assessment of Healthcare Providers and Systems (CG-CAHPS) patient experience scores (92 of 320 providers), shadowed by a trained peer coach for a half to full day and received recommendations on how to improve interactions with their patients. We coded 1082 recommendations found in the 92 coaching reports.

Results

Reports contained an average of 12 recommendations. About half encouraged consistency of existing behaviors and half encouraged new behaviors. Most recommendations related to behaviors of the provider rather than support staff and targeted actions within the examination room rather than other spaces (eg, waiting room). The most common recommendations mapped to behavioral aspects of provider communication. Most recommendations targeted verbal rather than nonverbal communication behaviors. Most recommendations were actionable (ie, specific, descriptive), with recommendations that encouraged new behaviors being more actionable than those that encouraged existing actions.

Conclusions

Patient experience surveys are effective at identifying where improvement is needed but are not always informative enough to instruct providers on how to modify and improve their interactions with patients. Analyzing the feedback given to coached providers as part of an effective shadow-coaching program provides details about implementation on shadow-coaching feedback. Recommendations to providers aimed at improving their interactions with patients need to not only suggest the exact behaviors defined within patient experience survey items but also include recommended behaviors indirectly associated with those measured behaviors. Attention needs to be paid to supplementing patient experience data with explicit, tangible, and descriptive (ie, actionable) recommendations associated with the targeted, measured behaviors. Research is needed to understand how recommendations are put into practice by providers and what motivates and supports them to sustain changed behaviors.

Background

The extent to which different measures of back pain impact represent an underlying common factor has implications for decisions about which one to use in studies of pain management and estimating one score from others.

Aims

To determine if different self-report back pain impact measures represent an underlying pain latent variable and estimate associations with it.

Method

Seven pain impact measures completed by Amazon Mechanical Turk adults are used to estimate internal consistency reliability and associations: Oswestry Disability Index (ODI), Roland-Morris Disability Questionnaire (RMDQ), short form of the Örebro Musculoskeletal Pain Questionnaire (OMPQ), Subgroups for Targeted Treatment (STarT) Back Tool, the Graded Chronic Pain Scale (GCPS) disability score, PEG (Pain intensity, interference with Enjoyment of life, interference with General activity), and Impact Stratification Score (ISS).

Results

The sample of 1,874 adults with back pain had an average age of 41 and 52% were female. Sixteen percent were Hispanic, 7% non-Hispanic Black, 5% non-Hispanic Asian, and 71% non-Hispanic White. Internal consistency reliability estimates ranged from 0.710 (OMPQ) to 0.923 (GCPS). Correlations among the measures ranged from 0.609 (RMDQ with OMPQ) to 0.812 (PEG with GCPS). Standardized factor loadings on the pain latent variable ranged from 0.782 (RMDQ) to 0.870 (ISS).

Conclusions

Scores of each measure can be estimated from the others for use in research.

Background and objectives

Delivering care as a patient-centered medical home (PCMH) is being widely adopted across the United States by primary care practices to better meet patient needs. A key PCMH element is measuring patient experience for practice improvement. The National Committee for Quality Assurance (NCQA) PCMH recognition program requires practices to both measure patient experience and engage in continuous practice/quality improvement to attain PCMH recognition and then throughout full PCMH transformation. The NCQA recommends but does not require that practices administer the Consumer Assessment of Healthcare Providers and Systems (CAHPS) clinician and group patient experience survey (CG-CAHPS) plus 14 CAHPS PCMH items, known as the CAHPS PCMH survey. We examine aspects of patient experience measured by practices with a varying number of years on their journey of PCMH transformation.

Methods

We randomly selected practices from the 2008-2017 NCQA directory of practices that had applied for PCMH recognition based on region, physician count, number of years and level of PCMH recognition, and use of the CG-CAHPS PCMH survey. We collected characteristics of the practices from practice leader(s) knowledgeable about the practice's PCMH history and patient experience data. We confirmed the patient experience surveys used during their PCMH history and requested copies of their non-CAHPS survey(s). For practices not administering the recommended CG-CAHPS survey (53/105 practices), we obtained and coded the content of their non-CAHPS surveys (68%; 36/53). We mapped the patient experience domains and specific measures to the CG-CAHPS survey (versions 2.0 and 3.0), CAHPS PCMH item set (versions 2.0 and 3.0), and the available CG-CAHPS supplemental items.

Results

Whether or not practices administered the CG-CAHPS items, most of them addressed topics contained in the CG-CAHPS survey such as Access to care, Provider communication, Office staff helpfulness/courteousness, Care coordination, and Shared decision-making. The most common CAHPS measures included were Office staff helpfulness/courteousness and Provider communication. Common non-CAHPS measures included were Ease of scheduling, Being informed about delays, and Provider helpfulness/courteousness.

Conclusion

NCQA PCMH practices included CAHPS items on their patient experience surveys even if they did not administer the full CG-CAHPS survey or the recommended CAHPS PCMH survey. To enhance the usefulness of patient experience surveys for practices undergoing PCMH changes, additional CAHPS measures could be developed related to key areas of PCMH change, including expanded access to care (ie, after-hours and weekend visits, ease of scheduling, being informed about delays), use of shared decision-making, and improvements in provider communication (ie, the provider is courteous, communication by other clinical staff members with the patient). These additional measures would assist practice leaders in capturing the breadth and depth of their PCMH transformation and its influence on providing more patient-centered care. Developing such items would help standardize the measurement of changes related to patient experience during PCMH transformation. Research is needed to determine whether a CAHPS survey is the best source of this information.

Objective

In this study, we examined the impact of a range of methods to improve data quality on the demographic and health status representativeness of Amazon Mechanical Turk (MTurk) samples.

Methods

We developed and field-tested a general survey of health on MTurk in 2017 among 5755 participants and 2021 among 6752 participants. We collected information on participant demographic characteristics and health status and implemented different quality checks in 2017 and 2021.

Results

Adding data quality checks generally improves the representativeness of the final MTurk sample, but there are persistent differences in mental health and pain conditions, age, education, and income between the MTurk population and the broader US population.

Conclusion

We conclude that data quality checks improve the data quality and representativeness.

Background

Researchers have implemented multiple approaches to increase data quality from existing web-based panels such as Amazon's Mechanical Turk (MTurk).

Objective

This study extends prior work by examining improvements in data quality and effects on mean estimates of health status by excluding respondents who endorse 1 or both of 2 fake health conditions ("Syndomitis" and "Chekalism").

Methods

Survey data were collected in 2021 at baseline and 3 months later from MTurk study participants, aged 18 years or older, with an internet protocol address in the United States, and who had completed a minimum of 500 previous MTurk "human intelligence tasks." We included questions about demographic characteristics, health conditions (including the 2 fake conditions), and the Patient Reported Outcomes Measurement Information System (PROMIS)-29+2 (version 2.1) preference-based score survey. The 3-month follow-up survey was only administered to those who reported having back pain and did not endorse a fake condition at baseline.

Results

In total, 15% (996/6832) of the sample endorsed at least 1 of the 2 fake conditions at baseline. Those who endorsed a fake condition at baseline were more likely to identify as male, non-White, younger, report more health conditions, and take longer to complete the survey than those who did not endorse a fake condition. They also had substantially lower internal consistency reliability on the PROMIS-29+2 scales than those who did not endorse a fake condition: physical function (0.69 vs 0.89), pain interference (0.80 vs 0.94), fatigue (0.80 vs 0.92), depression (0.78 vs 0.92), anxiety (0.78 vs 0.90), sleep disturbance (-0.27 vs 0.84), ability to participate in social roles and activities (0.77 vs 0.92), and cognitive function (0.65 vs 0.77). The lack of reliability of the sleep disturbance scale for those endorsing a fake condition was because it includes both positively and negatively worded items. Those who reported a fake condition reported significantly worse self-reported health scores (except for sleep disturbance) than those who did not endorse a fake condition. Excluding those who endorsed a fake condition improved the overall mean PROMIS-29+2 (version 2.1) T-scores by 1-2 points and the PROMIS preference-based score by 0.04. Although they did not endorse a fake condition at baseline, 6% (n=59) of them endorsed at least 1 of them on the 3-month survey and they had lower PROMIS-29+2 score internal consistency reliability and worse mean scores on the 3-month survey than those who did not report having a fake condition. Based on these results, we estimate that 25% (1708/6832) of the MTurk respondents provided careless or dishonest responses.

Conclusions

This study provides evidence that asking about fake health conditions can help to screen out respondents who may be dishonest or careless. We recommend this approach be used routinely in samples of members of MTurk.