The Help Me Grow Blog
Dec 18, 2015
All Measures Matter
Debating Long Term vs. Proximate Measures to
Evaluate Help Me Grow
I recently returned from Atlanta, where I took part in meetings exploring Help Me Grow replication in Georgia. During my time there I had the chance meet with staff at the Centers for Disease Control and Prevention (CDC).
While we already have a strong and productive partnership with the CDC National Center on Birth Defects & Developmental Disabilities, I welcomed the opportunity to explore a number of issues of common interest. These included such topics as early detection, system building, health equity, and evaluation strategies.
I was particularly excited to share the evolution of our thinking on Help Me Grow performance metrics, common indicators collected by all affiliates, and meaningful process and outcome measures. I was most eager to share our embracing of “proximate" measures and mediating factors to measure the impact of developmental interventions such as Help Me Grow. I enthusiastically anticipated the likelihood of CDC experts critiquing our methods and offering suggestions as to how to best overcome challenges and limitations.
I looked forward to their critique despite the fact that we have a great deal of confidence in focusing on the strengthening of protective factors as key mediators of the capacity of families to support their children’s healthy development. Recently, Help Me Grow evaluator Marcia Hughes and her University of Hartford colleagues submitted a manuscript on the topic to a respected and prominent peer-reviewed journal. Its acceptance further validates an evaluation strategy based on the Center for the Study of Social Policy’s Protective Factors Framework of their Strengthening Families initiative.
Yet despite this increasing conviction in the legitimacy of our approach, I am always compelled to share and even defend our rationale for employing measures more proximate than the long-term, highly desirable developmental outcomes that we ultimately seek.
In fact, my presentation of these ideas makes me sound more like a defense attorney than a doctor, as I am inclined to cite expert opinion as precedent for our actions. My particular favorites include a 2000 quote from the seminal publication, From Neurons to Neighborhoods: “[I]t is essential to consider the concept of a ‘moderator,’ a preexisting characteristic of families or children on which the impact of the treatment and magnitude of the treatment effect depend.”
I also am inclined to share a more recent quotation from the social policy guru Lisbeth (Lee) Schorr: “Obstacles to demonstrating the effectiveness of developmental interventions are best overcome by embracing theoretical connections established by research.”
Much to my relief, the CDC experts also embraced the use of such proximate measures. Not only that, they emphasized that such approaches are now well embedded within the early intervention literature. Indeed, we received the same message from the editor of the aforementioned peer review journal. This feedback from diverse sources further emboldens our efforts to operationalize protective factors as an evaluation strategy across our affiliate network.
At the same time, no one at the CDC recommended abandoning the quest to measure the long-term impact of our work to support vulnerable children’s developmental advances. Though we must not exclusively rely on developmental outcomes to demonstrate the efficacy of our interventions—far too many variables may undermine the impact of properly implemented early interventions—we must gather such long-term measures to help inform the success of our efforts.
Strategies are evolving to enable long-term measurement. A number of states, including our own Connecticut, are assigning “unique identifiers” to newborns. These identifiers will follow children through life and across various systems, including health, education, and social services. This tracking will ultimately enable correlational studies to examine the relationship between various interventions and long-term outcomes.
Another interesting strategy is Transforming Early Childhood Community Systems (TECCS). This national initiative developed through a partnership between the UCLA Center for Healthier Children, Families and Communities and the United Way Worldwide. Its goal is to improve the school readiness of young children.
A key component of TECCS is community measuring and mapping the developmental progress of young children at a population level and engaging diverse stakeholders in using this information to advocate for young children. Measurement and mapping with tools such as the Early Development Instrument (EDI) may help us examine the relationship between Help Me Grow penetration within a given jurisdiction and the evolution of children’s developmental trajectories.
As we confidently embrace protective factors as key proximate measures to confirm the efficacy of interventions, we will continue our long-term efforts to measure developmental outcomes. However, I happily conclude that my rigorous, proactive defense of proximate measures is no longer necessary. It may even raise suspicions, dare I remind listeners of Shakespeare’s Lady Macbeth: "The lady doth protest too much, methinks."
With respect to evaluating our impact on children’s optimal health development, the issue is not a matter of “either/or,” but rather that all measures matter.