First and foremost, we believe Pay for Success projects should help improve the lives of low-income people in American cities. With all of the new projects coming onto the market, the first question that we ask is: what will the impact be?

We’ve launched our Pay for Success: To Invest or Not to Invest? series to walk others through how we decide which Pay for Success projects to consider investing in. Last week, we shared our initial screening criteria. Over the next few weeks, we’ll discuss each of our initial screening categories in more depth.

First and foremost, we believe PFS projects should help improve the lives of low-income people in American cities. With all of the new projects coming onto the market– tackling problems ranging from recidivism to green infrastructure– the first question that we ask is: what will the impact be? This assessment involves three major considerations: outcomes, scale and the target population.

Outcomes

We seek to invest in projects that improve the well-being of low-income people, which means paying for outcomes that clearly indicate people are on track for a better quality of life in the long-term. Take Denver’s Pay for Success Project for the Chronically Homeless, for example. When deciding which of the project’s two outcomes to invest in- housing or jail bed days- we chose to define success as a reduction in the number of jail-bed days, because we felt it was a more effective proxy for better life outcomes.

We believe the tool of PFS is most valuable when it serves as a mechanism to redirect government spending towards achieving outcomes. So if a project is primarily paying based on outputs instead of outcomes, it’s a deal breaker for us.

If a project is primarily paying for outputs instead of outcomes, it’s a deal breaker for us.

We understand that sometimes projects have to pay for early indicators that are outputs to mitigate risk for investors. In that case, the output must have a strong, evidence-based connection to longer-term life outcomes. Enrollment in a program, for example, is not a sufficient indicator. Just because a person enrolls in a treatment program, it’s no guarantee he or she will finish, let alone that completion will lead to an outcome like a decrease in recidivism. Instead, we might pay for an early indicator like treatment engagement, if there’s an evidence-backed connection between some number of hours of treatment and substantial reductions in recidivism.

Scale

We see PFS as a tool that should be intentionally used to scale the services and capacity of a service provider. To determine whether a project will accomplish this, one of the first questions we ask is whether it’s serving a significant proportion of the total eligible population. For example, if a project is serving 500 low-income mothers, and a total of 2,000 low-income mothers are eligible for treatment, then the project is actually reaching a quarter of the target population—that’s a big deal.

However, we’ve also invested in smaller-scale PFS projects because they’re helping build an evidence base by testing out a program or intervention with a new target population. Then, we assess whether anyone has thought about how to scale or replicate the program if it’s proven successful. We also consider a project’s replicability. As the PFS field grows, we don’t want every project to have to reinvent the wheel. While many elements need to be tailored to location, there are commonalities- like the transaction structure, loan documents and contracts from other projects- that can and should be replicated across PFS projects.

As the PFS field grows, we don’t want every project to have to reinvent the wheel.

Target Population

As we think about outcomes and scale, we remain centered on our mission of serving low-income people. That means keeping people’s lives at the center of our decisions. If a project is not paying for outcomes tied to people, it’s not a good fit for us. For example, we thought the recently closed DC Water Environmental Bond Project was an important and innovative project, but because the outcomes were tied to runoff reduction, it was not a match for our people-centered focus and we decided not to invest.

We also focus on how projects will impact people of color. While ideally we’d like to pay for outcomes that reduce racial disparities, a first step is considering whether a project recognizes existing racial disparities, and if it is disaggregating data by race or ethnicity. Due to its data-driven nature, we believe PFS should be a mechanism for understanding the disproportionate impacts of interventions. Let’s say that a PFS project hits its overall goal of reducing recidivism by 30%, but only sees a reduction among white participants and not the people of color who participated. While the project may be deemed “successful” overall for reaching its goal, it has actually increased racial disparities by failing to achieve results for the people of color. This hypothetical case illustrates the importance of tracking differences in outcomes by race. Without disaggregation, we wouldn’t know there was ever a difference in how the intervention landed.

Impact is a critical first screen, but next up, we consider a project’s innovation.