Measuring Transportation Outcomes

By Melissa Levy of Community Roots & Shanna Ratner of Yellow Wood Associates

Measurement is key to understanding how our programs are working. The You Get What You Measure approach to measurement:

• fuels continuous learning through reflection
• creates new information and new patterns of information flow
• captures results of risk & experimentation
• can lead to new & unprecedented conversations, particularly with information gatekeepers
• supports constructive self-organizing behavior
• allows us to test our assumptions about the way the world works
• helps us tell our stories.

Measurement is never neutral. It shapes what we see, reveals what we value, and directs the actions we take. In community and economic development work, which is often complex and relational, it’s especially important to choose a measurement approach that supports learning, not just reporting.

As we consider how to measure outcomes in the mobility management transportation space, we’re learning to move past the focus on measuring the intermediary outputs, such as the number of bus seats filled, the number of miles driven, and the number of people served, to measuring outcomes or the difference transportation makes in the lives of those who can access it. Unfortunately, most of the information we are generally asked to report does not tell us anything meaningful about outcomes. More importantly, it rarely gives us the information we really need to know to improve outcomes.

Much of what we’re trying to accomplish, including improving people’s lives, improving people’s health and facilitating their economic prosperity, result in observable behavioral changes. If it can be observed, it can be measured. In fact, the very efforts we make to measure behavioral outcomes can contribute to achieving these outcomes if we do it creatively, courageously and interactively. The You Get What You Measure approach offers the opportunity to learn how to collect information in ways that are generative rather than extractive.

"If you can't measure it, you can't improve it." – Peter Drucker (also attributed to Kelvin).

Generative vs. Extractive Measurement

What is the difference between generative and extractive measurement?

Extractive measurement is about “taking the data you need from people and places to meet external needs.” This is the dominant, traditional model of measurement used in program evaluation, economic development, philanthropy and government reporting.  It relies on predefined indicators and standardized data collection systems imposed by external entities. It requires reporters of information to gather information to meet external reporting requirements that too often does not contribute to the local capacity to learn how to improve outcomes. Externally imposed reporting requirements tend to overlook local context, relational change, and emerging opportunities. Data is extracted from workers on the ground but they are rarely invited to act as co-interpreters of the data to determine what it really means.

Extractive measurement often produces metrics that look objective but are not tied to local goals and conditions, miss lived experience, and fail to capture emerging change. For example, being asked to answer some version of the question “Did you do the things you said you would do in your proposal” (whether that is set up an on-call system, purchase a van, or whatever it may be) begs the question of “Did you make progress toward the goals you have established for your region?” (the goals being the conditions you are working to achieve such as access to employment, health care, etc.)  CCAM-TAC asks you to orient toward achieving your goals, even if it means changing what you do to get there because of what you learn along the way.

Generative measurement is about “creating insights with people who are doing the work and/or affected by it that help the system evolve to meet shared goals.” Instead of extracting data, it focuses on creating conditions where new understanding, meaning, and possibility emerge. Generative measurement treats measurement as a shared learning process that creates insights  along with the people doing the work. Instead of externally determined metrics, it emphasizes metrics developed in context, real-time analysis, reflection and adaptation. This approach helps teams spot patterns, understand what’s emerging, and adjust strategy as conditions change to smarter action. Generative measurement is collaborative, adaptive, and grounded in lived experience. It returns value to participants by strengthening relationships, clarifying direction and roles, and building capacity.

As a result, generative measurement helps people see:

• Are our assumptions correct?
• Are our actions taking us in the desired direction?
• What’s emerging?
• What’s shifting?
• How is capacity or connection growing?
• What experiments should happen next?

Generative measurement is measurement that produces shared value and knowledge in place not just collects data for the benefit of others.

"By believing passionately in something that still does not exist, we create it." - Franz Kafka

Extractive:

Like harvesting crops from a field you don’t tend and then leaving.

Generative:

Like tending a garden with community members, noticing new growth, learning together, and deciding what to plant next or how to amend the soil to improve yields for everyone.

"You get what you measure. Measure the wrong thing and you get the wrong behaviors." – Richard Hamming (mathematician)

Community‑driven strategies—like mobility management work—rarely follow a straight line. They rely on relationships, trust, experimentation, and local ownership. Generative measurement stays focused on the local goal, captures these dynamics of trust, collaboration, ownership and new opportunities and helps practitioners see patterns, learn quickly, and adjust in real time to improve the value of their contributions to the shared goal of better outcomes.