The Carolina Across 100 (CX100) team works with cross-sector collaborative teams to positively impact local challenges such as youth workforce participation, mental health and suicide, and housing affordability. An important aspect of this work is a Shared Measurement System (SMS) to assess local impact. Over the course of CX100, our team has refined the co-design process to ensure the system is responsive to local needs, provides a straightforward data collection method for teams, and supports their cohort-wide goals. 

What is a Shared Measurement System?

An SMS is a tool to track progress and impacts over the full course of a program. We aim for the system to be a set of co-designed indicators that reflect the shared priorities across all community collaborative teams in a cohort. 

Some form of an SMS is used by multiple frameworks for collaboration, including Collective Impact, Results-Based Accountability, Adaptive Learning Systems, and the ncIMPACT Community Sync model. 

An SMS is not used to: 

• Compare collaboratives against one another 

• Determine the quality of collaboratives' work 

• Rank collaboratives 

• Punish collaboratives  

To date, an SMS has been used in multiple Carolina Across 100 (CX100) collaborative projects, including "Our State, Our Work," "Our State, Our Wellbeing," and "Our State, Our Homes." 

How do you co-design and implement an SMS?

The process for developing an SMS is highly collaborative and iterative.

When planning to launch a new program, we do extensive background research into the landscape of the topic, including interviews with key stakeholders on campus and beyond. By doing so, we start to understand what may be relevant to measure. 

At the beginning of a program, we ask teams to inventory the types of data they already collect, as well as potential activities they might pursue, to inform the first draft of the SMS. We look across the dozens of potential measures for every team and combine them into a few categories of draft indicators. This is then reflected to teams at an in-person forum and refined as they practice the data collection and reporting process. 

The first data collection period comes shortly after, a few months into the program, and allows us to identify and address any gaps in the system. An SMS Data Tracking Tool is distributed to assist with the process of gathering local data, as well as an SMS Frequently Asked Questions (FAQs). In addition to these resources, we provide regular technical support and feedback opportunities to understand what is working and not working for teams. After working through any initial kinks, we maintain consistent indicators throughout the remainder of the program to identify trends across cycles. 

During a data collection period, teams track information about the agreed-upon set of indicators. The project manager for each team then enters this data into a survey. The results of this survey are aggregated across all participating teams and reported back to teams, as well as external stakeholders. This process is visualized below and repeated every 3-6 months throughout the course of a program. 

The SMS co-design and implementation process.

Why is it valuable to use an SMS?

• Teams have different goals for their communities, but during a collaborative program, it is important to find commonalities to ensure the cohort is moving in one direction. 
• Creating an SMS requires the cohort to agree on a set of data points that will represent their goals. This process focuses and refines their approach. 

• An SMS captures information at the individual team level, allowing community collaboratives to understand their growth and progress. 
• Community organizations have limited resources. It’s important that teams have a way to assess how they are spending their precious time, human capital, and other resources. 
• An SMS can provide a chance to reflect, measure outcomes, and pivot approach. 

• An SMS allows the CX100 team to see where targeted additional technical assistance and other supports may be needed to reach cohort-level goals. 

• An SMS allows the CX100 team to learn what strategies and activities are being implemented across the state and share these promising practices widely.  

• An SMS allows the teams to capture what has happened during and because of the program.  
  • During the “Our State, Our Work,” program, the cohort was working to reconnect the more than 59,000 Opportunity Youth in their communities to work and school. They set a goal of reaching 6,400 by the end of 2024. Their SMS results reflected that they had reached over 15,800 young adults.  
  • During the “Our State, Our Wellbeing” program, the cohort’s efforts aimed to respond to the loss of life due to suicide in the state. North Carolina loses approximately 1,500 lives to suicide annually. The cohort identified shared strategies like awareness/stigma reduction, mental health training, and reducing means of suicide. By using an SMS throughout the program, the teams were able to report that 1,600+ mental health services were delivered in their communities to improve mental health and wellbeing. 

• An SMS cannot be a rigid, top-down system for evaluation. Instead, it must reflect the diversity of approaches across community teams while maintaining enough cohesion to tell a collective story. 
• By co-designing an SMS with community teams, the indicators reflect their goals and priorities, and the format is tailored to their needs and preferences. 

• The earlier they are exposed and able to contribute to its design, the more prepared and empowered they will be to manage their data.  
• It’s also imperative to have these discussions across the cohort, when possible, to bolster peer learning from shared experiences. 

• Local leaders often find it challenging to balance SMS efforts with other priorities. If it’s not meaningful and straightforward, it won’t be feasible to complete.  
• We aim to reduce barriers to participation by distributing resources like a data tracking tool and FAQs, as well as incorporating frequent technical assistance and feedback opportunities.