The danger of asking the wrong questions and adopting the wrong outcome measures is an important consideration in our efforts to evaluate the efficacy of Help Me Grow. Our crucial task is to assure that we ask the right questions and adopt the right outcome measures for Help Me Grow as it promotes children’s healthy development.
How Superintendents Can Help
A recent National Public Radio report helped me understand anew the importance of appropriate and useful questions and measures. The report focused on a study from the Brookings Institution finding that student achievement (as measured by test scores) does not correlate with superintendent credentials, such as duration in office.
The broadcast cheekily implied that the “person at the top is almost irrelevant,” but my reaction was quite different. In fact, I wondered why any researcher would even suggest that the qualities of the superintendent might correlate with such a vulnerable and specific outcome as student test scores.
As we know, many factors affect students’ educational achievement, including family, school, and community. Studies have demonstrated that even characteristics that intuitively seem to support academic success, such as class size and facilities, are lacking clear research evidence of their impact.
In contrast, such characteristics as community and parental engagement, a proactive school board, and the extent to which schools are viewed as open and accessible do correlate with students’ academic outcomes. Thus, the recent study’s lack of a linkage between superintendents and students’ success doesn’t mean that superintendents are “paper pushing administrative overlords.” Instead, the study warrants criticism for asking the wrong question.
Here’s what I mean. Asking if superintendent characteristics correlate with students’ academic achievement is the wrong question. In reality, superintendent credentials are more likely to correspond to school-related characteristics that do affect the best student outcomes. These include the adequacy of school budgets, qualities of school-system governance and bureaucracy, the quality of teacher professional training, and student and teacher access to ancillary services and supports.
So what’s the right question? I would say this: What’s the relationship between superintendents’ priorities and the factors known to correlate with students’ success? I speculate that superintendents’ actions with regard to budgets, bureaucracy, training, and services are likely to contribute to student outcomes.
What This Insight Means for Help Me Grow
Much like academic success, developmental outcomes are influenced by a complex menagerie of factors and circumstances. The likelihood of any single intervention to correlate with children’s developmental advances is, at best, unlikely—even as broad as those facilitated by Help Me Grow.
This means that searching for a correlation between children’s developmental status (as measured by a population-based measure such as the Early Development Instrument) and Help Me Grow penetration within a community is likely to underestimate any Help Me Grow’s impact on children’s developmental status. It may even lead to the erroneous conclusion that Help Me Grow does not contribute to promoting children’s healthy development.
On the other hand, a proper outcome measure can demonstrate Help Me Grow’s success in promoting children’s optimal healthy development. What we need are those factors that a Help Me Grow intervention can affect and that are demonstrated, through high quality research, to correlate with developmental outcomes.
To this vital end, we are now partnering with the Center for the Study of Social Policy and embracing the Protective Factors Framework of their Strengthening Families initiative. This partnership follows many consultations and a thorough and careful review of the research literature.
Our partnership will allow us to continue to evaluate the impact of Help Me Grow on five key factors known to influence developmental outcomes.
Strengthening Families Five Protective Factors
- parental resilience
- social connections
- concrete support in times of need
- knowledge of parenting and child development
- social and emotional competence of children
Research shows that Help Me Grow linkages of children and their families to community-based programs and services has a positive impact on these five protective factors. We will continue to examine the utility of embracing the protective factors in both our program design and evaluation activities.
As the study of school superintendents illustrates, asking the wrong question can lead to erroneous conclusions regarding the importance of key factors. Worst case, the wrong questions lead us astray in our search for effective interventions. More proximate measures than student test scores and children’s developmental measures may be far more useful.
Such complex outcomes as student school success and children’s optimal healthy development demand our most careful, creative thinking. As always, we welcome your thoughts and suggestions.