Growing Together: Introducing the Impact & Network Performance Branch of the Help Me Grow National Center

BY: MELISSA MILLER

“Not everything that can be counted counts. Not everything that counts can be counted.”

– William Bruce Cameron, Sociologist, Informal Sociology: A Casual Introduction to Sociological Thinking

Having worked in the measurement and evaluation field for over a decade and a half, I feel like this quote aptly reflects a conundrum I am often faced with: measurement and evaluation are an imperfect science. When I joined the HMG National Center in January 2023, the Affiliate Network was in the throes of completing the annual Fidelity and Sustainability Assessments and questions were coming in from affiliates. Like many of you, I had to learn definitions and contextualize the wording used in the Assessments, while also recognizing that there is a degree of interpretation. My onboarding gave me a front row seat to the HMG Assessment experience.

Accurately measuring the impact and performance of any national initiative is challenging: there are always limitations, data gaps, or an asterisk to your findings. By default, the HMG Network experiences these same challenges. Like the opening quote, HMG systems also conduct important activities that are not feasible to measure.

However, albeit imperfect, we must strive to close the gap of ambiguity and try to produce precise measurement definitions, robust data collection strategies, and rigorous analytic plans.

Why? When we do so, our data can be illuminating; it can tell us important stories of our progress, help us identify gaps and areas of improvement, and reveal what is and what is not working. Having this information is critical to sustainability of our efforts: we need it to demonstrate our value and to advocate for continual and future funding of our work.

I am now beginning my sixth month at HMG and have had an opportunity to reflect on how to best help us leverage our data collection efforts, and especially improve the Assessment experience. I continue to be extremely humbled and invigorated to be part of this amazing community doing such important work for children. My hope for this blog post is to share our initial thoughts on the vision of our newly formed Impact and Network Performance branch, and also invite you to join an ongoing dialogue on how we can co-create and reimagine this body of work.

Who We Are

Our team is comprised of myself, Melissa Miller, MPH, PMP, as the Associate Director of Impact and Network Performance, and my colleague, Noshin Ahmed, MPH, as the Program Specialist of Impact and Network Performance, who is an amazing partner in executing this work.

Together, we bring broad expertise in quality improvement initiatives, survey development, measure development, data analytics, and evaluation. We also have experience applying these skills to different populations and settings. While our credentials are important (read about our backgrounds here), equally important are the values we will employ.

For all data we request and decisions we make, we promise to be:

Mindful of your constraints

Thoughtful in our requests

Collaborative in our processes

Mission-driven to produce findings that are actionable and maximally beneficial to the Affiliate Network

Our Work

The main purpose of our branch is to oversee the strategic development and implementation of a measurement framework to further validate the HMG Model and measure the impact and performance of the HMG Network. Traditionally, our branch’s primary responsibilities include maintaining, administering, and analyzing all Assessments collected from the Network, including the Fidelity, Sustainability and Mini Assessments. Over the next year and beyond, we will be dedicating significant efforts to improve this work and will be soliciting feedback from the Affiliate Network.

In the past couple of months, we have already made the following changes:

Draft Report “Preview Period”: New this year, we conducted a preview period for systems to review their draft Assessment report and provide edits to ensure we have high-quality data and that the data submitted is accurate.
Enhanced Reports: Our system-level reports have been “refreshed” and enhanced with additional data and features, such as: a customized resource list to support systems not in full implementation of a Core Component; Impact Indicator data; and SMART Aim & Trends.
Analysis Approach: New this year, we have applied more rigorous analytic methods to our analysis. We have included an additional quality check step where we validate the responses received, flag inconsistencies, errors, and data issues. We have also set up internal tools and queries to streamline and automate data analysis.
Your Feedback: Your feedback from the Assessment Experience Survey will help inform Assessment changes. We want to better understand experiences in aggregating data and submitting results to improve our future data collection efforts.

In addition to the Assessments, our branch’s work also touches all of the HMG National Center’s projects and initiatives since measurement and evaluation is a critical component of our work.

Our Approach

Being new in my role enables me to infuse creativity and new approaches to our work. At the same time, I am extremely respectful of the accomplishments to date.

We are taking a thoughtful, yet deliberate approach in reimagining changes to our current work and ensuring we incorporate the affiliate voice in designing our path forward.

We have already seen in your survey feedback that the Affiliate Network is experiencing significant constraints, ranging from staff turnover to funding. Better understanding the priorities and constraints that systems are experiencing related to data will ensure our data strategy is feasible, we receive quality data, and the analysis we produce is actionable.

The principles we will employ include:

Meaningful Measurement & Minimize Burden: All measures need to be tied to a purpose or goal. There is a saying that, “just because you can measure something, doesn’t mean you should.” This is how we will approach any revisions to our Fidelity and Sustainability Assessment processes and tools.
Leveraging Data to Accelerate Change: This is one of our strategic priorities as a National Center, and we want to align the reports and deliverables with data that helps you drive change in your communities.
Streamlining Data Collection: We want to explore ways to streamline the data collection process, potentially through different mechanisms or tools.
Mixed Methods: For evaluation, I am a big proponent of a mixed methods approach – collecting and analyzing both quantitative and qualitative data – to ensure a complete picture.

What to Expect

Optimization with respect to data collection continues to be our “North Star.”

Thus, our measurement and evaluation changes are rooted in minimizing burden to help the Affiliate Network achieve its maximum value.

Below are several upcoming activities where you can get involved and help chart the course:

Final Fidelity/Sustainability Assessment Report: Systems will receive final reports by June 16, 2023, inclusive of edits made during the preview period.
Mini Assessment: We are pausing Mini Assessment this year to give us additional time to incorporate feedback we have received from the Network in the recent Assessment Experience Survey and also conduct some targeted focus groups to dig deeper into themes that emerged. We still encourage systems to continue to query and review their data internally, to ensure they have the processes in the place to report on this data and to identify areas of improvement.
Experience Survey Findings: We will be sharing an aggregated high-level report on some of the key findings from the Experience Survey and potential changes based on your feedback.
Opportunities to Provide Feedback: We will be creating future opportunities for affiliates to provide input on our data collection efforts, so please let us know if you would like to be included in future focus groups and listening opportunities – our contact information is included below.
Listening Session: Join us for our branch’s listening session on August 16 for a conversation with Noshin and me where you can ask questions and share your priorities for HMG measurement.

Get to Know the New HMG National Center

In addition to these input opportunities, we will be holding a roundtable discussion on July 26 for affiliates to meet all of our branch leads. Open to the full HMG Network, we invite you all to come and share your thoughts on how the National Center can best apply our new capacity to support your efforts.

Following the roundtable, each branch at HMG National will hold a listening session where you can drop in and spend some more time chatting with specific branch leads and other affiliates who want to be part of the conversation:

• COMMUNICATIONS & NETWORK RELATIONS (AUGUST 8, 2-3PM ET)
• IMPACT & NETWORK PERFORMANCE (AUGUST 16, 2-3PM ET)
• IMPLEMENTATION & SYSTEM-BUILDING (AUGUST 22, 2-3PM ET)
• POLICY & COMMUNITY CHANGE (AUGUST 29, 2-3PM ET)

Feel free to reach out!
Connect with Melissa Miller at MMiller02@connecticutchildrens.org and
Noshin Ahmed at NAhmed@connecticutchildrens.org

Melissa Miller is the Associate Director of Impact & Network Performance at the Help Me Grow National Center.

Noshin Ahmed is the Impact & Network Performance Program Specialist at the Help Me Grow National Center.