results4america.org
@Results4America
2. USE DATA & EVIDENCE
ARPA encourages local, state, and tribal governments to invest their ARPA funds in evidence-
based solutions backed by strong, moderate or preliminary evidence including the specific
dollar spending allocated to evidence-based investments in their Recovery Plans.
•
Implement evidence-based interventions identified by clearinghouses, such as Results
for America’s Economic Mobility Catalog.
•
Leverage results-based contracts to deliver better results and engage communities.
•
Define and prioritize evidence in grants and contracts.
•
Report spending on evidence-based interventions in Recovery Plan.
IMPLEMENTATION TIPS
The ARPA Reporting and Compliance Guidance defines three tiers of evidence:
STRONG
Strong evidence means the evidence base that can support causal conclusions for the specific
program proposed by the applicant with the highest level of confidence.
This consists of one or more well-designed and well-implemented experimental studies
conducted on the proposed program with positive findings on one or more intended outcomes.
MODERATE
Moderate evidence means that there is a reasonably developed evidence base that can support
causal conclusions.
The evidence base consists of one or more quasi-experimental studies with positive findings on
one or more intended outcomes or two or more non-experimental studies with positive findings
on one or more intended outcomes. Examples of research that meet the standards include: well-
designed and well-implemented quasi- experimental studies that compare outcomes between
the group receiving the intervention and a matched comparison group (i.e., a similar population that
does not receive the intervention).
PRELIMINARY
Preliminary evidence means that the evidence base can support conclusions about the program’s
contribution to observed outcomes.
The evidence base consists of at least one non-experimental study. A study that demonstrates
improvement in program beneficiaries over time on one or more intended outcomes or an
implementation (process evaluation) study used to learn and improve program operations would
constitute preliminary evidence. Examples of research that meet the standards include: (1) outcome
studies that track program beneficiaries through a service pipeline and measure beneficiaries’
responses at the end of the program; and (2) pre-and post-test research that determines whether
beneficiaries have improved on an intended outcome.