Evidence-Based Decision Making

Evidence-Based Decision Making web_admin
EBDM conference

What Is EBDM?

EBDM is a strategic and deliberate method of applying empirical knowledge and research-supported principles to justice system decisions made at the case, agency, and system level. The initiative team developed the EBDM framework, which posits that public safety outcomes will be improved when justice system stakeholders engage in truly collaborative partnerships, use research to guide their work, and work together to achieve safer communities, more efficient use of tax dollars, and fewer victims.

A Framework for Evidence-Based Decision Making in State and Local Criminal Justice Systems

The National Institute of Corrections (NIC), in partnership with the Center for Effective Public Policy, built the Evidence -Based Decision Making Initiative (EBDM) initiative to create game-changing criminal justice system reform.

EBDM is a strategic and deliberate method of applying empirical knowledge and research-supported principles to justice system decisions made at the case, agency, and system level and seeks to equip criminal justice local and state policymakers with the information, processes, and tools that will result in measurable reductions of pretrial misconduct, post-conviction reoffending, and other forms of community harm resulting from crime.

Introduction

Evidence-based decision making (EBDM) is the practice of using research findings to inform and guide decisions across the justice system. Some examples of such decisions include:

Phases

Phases of NIC’s Evidence-Based Decision Making (EBDM) initiative:

  1. Phase I Framework Development:

    Project partners worked with NIC and a multidisciplinary advisory committee to develop the Framework, with the intent to define risk and harm reduction as fundamental goals of the justice system, summarize the strongest of the evidence-based research, and outline a structure and set of principles for achieving EBDM in local justice systems.

    Read about Framework Development

  2. Phase II Planning Process:

    The initiative competitively selected and worked with seven sites as they engaged in a planning process to prepare to implement their local interpretation of the Framework. Their implementation plans were submitted to NIC in June 2011.

    Read about the Planning Process

  3. Phase III Implementation:

    Since August 2011, NIC has provided support to all seven sites in Phase III.

    Read about Implementation

  4. Phase IV Expansion to Statewide Structure:

    In September 2013, NIC entered into a cooperative agreement with the Center for Effective Public Policy to expand EBDM to the state level. Work under this phase of the Initiative includes the provision of technical assistance and the development of tools and protocols to expand EBDM to additional local counties and to state level policy groups within those states with existing EBDM local sites. In support of this work, NIC and the Center partnered with officials in the State of Wisconsin to develop and pilot a statewide Summit on EBDM in January 2014. The purpose of the EBDM Summit was to pilot test the Initiative’s state-level protocols.

    Read more about the results of the Summit

  5. Phase V Building EBDM Capacity at the Individual, Agency, and System Levels:

    In Phase V, states will begin the process of planning systemwide change strategies to achieve evidence-based decision making (EBDM) at the state level and in multiple local jurisdictions.

    Read about building capacity

NIC's Goal

Chart toward a more successful future

The goal of this initiative is to build a systemwide framework (arrest through final disposition and discharge) that will result in more collaborative evidence-based decision making and practices in local criminal justice systems. This effort is grounded in two decades of research on the factors that contribute to criminal reoffending and the methods the justice system can employ to interrupt the cycle of reoffense.

The initiative seeks to equip criminal justice policymakers in local communities with the information, processes, and tools that will result in measurable reductions of pretrial misconduct, post-conviction reoffending, and other forms of community harm resulting from crime.

Accession Number - 031461

Framework

Framework web_admin

In June 2008, the National Institute of Corrections (NIC) launched the “Evidence-Based Decision Making in Local Criminal Justice Systems” initiative. While first developed for local-level implementation, the initiative has since been expanded and adapted to state-level decision making, and is now known as the “Evidence-Based Decision Making in State and Local Criminal Justice Systems” initiative. The goal of the initiative is to build a systemwide framework (arrest through final disposition and discharge) that will result in more collaborative, evidence-based decision making and practices in local criminal justice systems

This document “A Framework for Evidence-Based Decision Making in State and Local Criminal Justice Systems” defines the core principles and action strategies that criminal justice policymakers may employ to reduce the harm to communities caused by crime.

The Framework identifies the key structural elements of a system informed by evidence. It defines a vision of safer communities. It puts forward the belief that risk and harm reduction are fundamental goals of the justice system, and that these can be achieved without sacrificing offender accountability or other important justice system outcomes.

The Framework both acknowledges the importance of the key premises and values underlying our criminal justice system and puts forward a set of principles to guide evidence-based decision making within that context. The principles themselves are evidence-based.

The Framework highlights some of the most groundbreaking research in the justice field—evidence that clearly demonstrates that we can reduce pretrial misconduct and offender recidivism. 

Phases

Phases web_admin
A woman at an EBDM conference

These are five phases of NIC’s Evidence-Based Decision Making (EBDM) initiative:

  1. Phase I Framework Development:

    Project partners worked with NIC and a multidisciplinary advisory committee to develop the Framework, with the intent to define risk and harm reduction as fundamental goals of the justice system, summarize the strongest of the evidence-based research, and outline a structure and set of principles for achieving EBDM in local justice systems.

    Read about Framework Development

  2. Phase II Planning Process:

    The initiative competitively selected and worked with seven sites as they engaged in a planning process to prepare to implement their local interpretation of the Framework. Their implementation plans were submitted to NIC in June 2011.

    Read about the Planning Process

  3. Phase III Implementation:

    Since August 2011, NIC has provided technical assistance to all seven sites:

    • Eau Claire County, Wisconsin
    • Mesa County, Colorado
    • Ramsey County, Minnesota
    • Grant County, Indiana
    • Yamhill County, Oregon
    • City of Charlottesville/County of Albemarle, Virginia
    • Milwaukee County, Wisconsin

    EBDM project staff will continue to work with the sites through the end of 2013 to revise their local work plans, assist them in implementing key change strategies, and collect data and other information to evaluate the effectiveness of these strategies in meeting their system wide scorecards.

    Read about Implementation

  4. Phase IV Expansion to Statewide Structure:

    In September 2013, NIC entered into a cooperative agreement with the Center for Effective Public Policy to expand EBDM to the state level. Work under this phase of the Initiative includes the provision of technical assistance and the development of tools and protocols to expand EBDM to additional local counties and to state level policy groups within those states with existing EBDM local sites. In support of this work, NIC and the Center partnered with officials in the State of Wisconsin to develop and pilot a statewide Summit on EBDM in January 2014. The purpose of the EBDM Summit was to pilot test the Initiative’s state-level protocols. Read more about the results of the Summit.

    Read more about the results of the Summit

  5. Phase V Building EBDM Capacity at the Individual, Agency, and System Levels:

    Under Phase V of EBDM, the selected state(s) will receive up to 15 months of intensive technical assistance (TA) to begin the work of EBDM at the state level and in multiple local jurisdictions. While Phase IV is focused on building capacity and preparing to expand EBDM efforts in the state, the goal of Phase V is to build capacity to make evidence-based decisions at the individual, agency, and system levels, and to develop plans for implementing systemwide change strategies that will align state and local officials/jurisdictions with one another and with the principles of EBDM. Phase V is a planning phase, similar to the planning process seven EBDM local sites engaged in during Phase II. Full implementation of EBDM strategies is anticipated in Phase VI (the implementation phase).

    Read about building capacity

Phase I - Framework Development

Phase I - Framework Development web_admin

Project partners worked with NIC and a multidisciplinary advisory committee to develop the Framework, with the intent to define risk and harm reduction as fundamental goals of the justice system, summarize the strongest of the evidence-based research, and outline a structure and set of principles for achieving EBDM in local justice systems.

Phase II - Planning Process

Phase II - Planning Process web_admin

Preparing to Implement the EBDM Framework (A Roadmap for Phase II)

Core Activities

Likely Action Steps

By the end of the Phase II, the EBDM Site will have…
Build a genuine, collaborative policy team.
  • Administer a policy team collaboration survey (one or more times).
  • Establish ground rules and operating norms.
  • Develop a shared vision statement.
  • Articulate roles and responsibilities of team members.
  • Develop “One Less” individual statements and a team document that reflects these statements.
  • Take other steps to build/enhance the collaborative climate of the policy teams.
  • a highly functioning collaborative policy team.
  • a shared vision for the criminal justice system.
  • a track record of meaningful team accomplishments.
Build individual agencies that are collaborative and in a state of readiness for change.
  • Administer an agency-based collaboration survey (one or more times).
  • Engage staff in the EBDM initiative in specific, purposeful ways (e.g., establish an internal working team to collect information, provide input, and assist in specific objectives).
  • Develop specific action items to address learnings from the survey.
  • agencies that demonstrate a collaborative climate and readiness for change.
  • an engaged staff that provides meaningful, ongoing input into evidence-based policy and practice changes.
Understand current practice within each agency and across the system.
  • Develop a system map.
  • Conduct policy/practice assessment around each decision point to determine the use of evidence-based practices/decision making and continuous quality improvement (CQI) competencies.
  • Gather baseline data.
  • Identify strengths/challenges and targets of change.
  • a full understanding of the basis upon which decisions are made at key points within and across agencies.
  • a set of agreed-upon strengths.
  • a set of agreed-upon targets for change.
Understand and have the capacity to implement evidence-based practices.
  • Administer a knowledge survey to the policy team and agency staff.
  • Assess staff skills in core competency areas.
  • Develop specific strategies to augment knowledge and competencies, where needed.
  • a common understanding of the research (and its limitations) across all relevant agencies/staff.
  • an understanding of the implications of these findings for future policy and practice.
Develop logic models.
  • Administer a knowledge survey to the policy team and agency staff.
  • Assess staff skills in core competency areas.
  • Develop specific strategies to augment knowledge and competencies, where needed.
  • a common understanding of the research (and its limitations) across all relevant agencies/staff.
  • an understanding of the implications of these findings for future policy and practice.
Establish performance measures, determine outcomes, and develop a system scorecard.
  • Agree on key definitions (e.g., “recidivism,” “probation violation”).
  • Develop scorecard items/outcomes.
  • Identify performance measures.
  • Assess data system capacity/collection methods.
  • Build capacity, where needed.
  • a set of agreed-upon performance measures that will enable an objective, empirical evaluation of the effectiveness of the justice system agencies in achieving their agreed vision.
  • benchmarks against which longer-term outcomes can be measured.
  • methods to collect and analyze data on an ongoing basis to inform policy and practice.
  • a systemwide scorecard
Engage and gain the support of a broader set of stakeholders and the community.
  • Conduct a public opinion survey.
  • Compile information/a clear set of messages the team and individual stakeholders can use to inform and engage the community.
  • Define the desired role of the community in justice system activities.
  • Identify individuals/groups within the community who are appropriate for outreach.
  • a strategy for engaging additional stakeholders and the community in meaningful dialogue about the vision/goals of the justice system, the state of knowledge and research, and the system’s performance in achieving these goals.
Engage and gain the support of a broader set of stakeholders and the community. Develop a strategic action plan for implementation.
  • Conduct an analysis of potential barriers to implementation.
  • Develop a plan of action for implementing specific policy and practice changes—who, what, when, where, how.
  • a clear, specific, measurable plan for implementing policy and practice changes that advance evidence-based decision making and further support the achievement of the justice system’s vision and goals.

Phase III - Implementation

Phase III - Implementation web_admin

Since August 2011, NIC has provided technical assistance to all seven sites:

EBDM project staff will continue to work with the sites through the end of 2013 to revise their local work plans, assist them in implementing key change strategies, and collect data and other information to evaluate the effectiveness of these strategies in meeting their system wide scorecards.

Phase IV - Expansion to Statewide Structure

Phase IV - Expansion to Statewide Structure web_admin

In September 2013, NIC partnered with the Center for Effective Public Policy to expand EBDM to the state level. Work under this phase of the Initiative includes the provision of technical assistance and the development of tools and protocols to expand EBDM to additional local counties and to state level policy groups within those states with existing EBDM local sites. In support of this work, NIC and the Center partnered with officials in the State of Wisconsin to develop and pilot a statewide Summit on EBDM in January 2014. The purpose of the EBDM Summit was to pilot test the Initiative’s state-level protocols. Click below for the results of the summit.

Phase V - Building EBDM Capacity at the Individual, Agency, and System Levels

Phase V - Building EBDM Capacity at the Individual, Agency, and System Levels web_admin

Introduction

For Phase V, National Institute of Corrections (NIC) sought applications for states to participate in this phase of the Evidence-Based Decision Making in State and Local Criminal Justice Systems Initiative. In Phase V, states began the process of planning systemwide change strategies at the state level and in multiple local jurisdictions. One state-level executive team and 4-6 local-level policy teams in each selected state are undergoing a planning process to implement changes—selected by the team—across a continuum of criminal justice decision points, as outlined in the Phase V Roadmap. All states involved in Phase IV of were eligible to apply to continue their work with NIC and its partners in Phase V of the Initiative. Three states--Indiana, Virginia, and Wisconsin--were selected to participate in Phase V of the EBDM Initiative.

EBDM Phase V

Under Phase V of EBDM, the selected state(s) will receive up to 15 months of intensive technical assistance (TA) to begin the work of EBDM at the state level and in multiple local jurisdictions. While Phase IV is focused on building capacity and preparing to expand EBDM efforts in the state, the goal of Phase V is to build capacity to make evidence-based decisions at the individual, agency, and system levels, and to develop plans for implementing systemwide change strategies that will align state and local officials/jurisdictions with one another and with the principles of EBDM. Phase V is a planning phase, similar to the planning process seven EBDM local sites engaged in during Phase II. Full implementation of EBDM strategies is anticipated in Phase VI (the implementation phase).

Sites

Sites web_admin

In August 2010, the Evidence-Based Decision Making in Local Criminal Justice Systems Initiative selected seven jurisdictions to serve as "EBDM sites" as part of Phase II of this initiative. They included:

These seven sites were selected as a result of their demonstration of collaboration among key policymakers, track record of success in previous high-impact initiatives, and commitment to using research to guide sound decision making.

The goal of Phase II was for the sites to develop implementation plans to specifically reduce the likelihood of pretrial misconduct, post-sentence reoffense, and other forms of community harm that result from crime.

With guidance from NIC, Initiative partners, and an assigned technical assistance provider, the sites engaged in a deliberate and strategic process to assess their current policy and practice and determine methods to more effectively integrate research into local policy and practice.

Pilot Sites

Pilot Sites web_admin

Charlottesville-Albemarle County, Virginia

Charlottesville-Albemarle County, Virginia web_admin

Charlottesville is an independent city located at the foothills of the Blue Ridge Mountains, adjacent to Albemarle County, in the Commonwealth of Virginia. As of 2009, the city proper had a population of 41,750. It is the county seat of Albemarle County, although the two jurisdictions are separate legal entities. Together, Charlottesville and Albemarle County are home to 98,970 citizens as well as students attending the University of Virginia.

This site is unique in that the work of the EBDM policy team encompasses two jurisdictions that routinely share resources and have a long history of collaboration in order to meet the needs of its citizens. The local and state probation offices have collaborated for the past seven years in the field of evidence-based practices, as both were selected original pilot sites for EBP within the Virginia Department of Criminal Justice Services (DCJS) and the Virginia Department of Corrections (VADOC). The EBDM policy team is the latest collaborative effort between the two localities.

EBDM Stakeholders

An EBDM Mission

The agencies in the Charlottesville-Albemarle criminal justice system seek to achieve justice and make communities safer by working closely together, applying the best-known research to policies and practices, listening to those affected by crime, and recognizing that every interaction can lead to improved outcomes.

The tagline for the effort in Charlottesville-Albemarle County is “Working together for a safer community, one person at a time.”

Due to the policy team’s large nature (i.e., having representatives from two localities for each discipline group), a steering committee was formed to guide the team’s activities. The steering committee, with representation from both the city and county but smaller in membership, is responsible for managing the many details of the work and for bringing work products to the full policy team for consensus building and final approval. The policy team includes:

  • a district court judge
  • a county public defender
  • city and county police chiefs
  • the probation and parole chief
  • the director of pretrial services
  • city and county Commonwealth attorneys
  • city and county sheriffs
  • city and county victim witness coordinators
  • the regional jail superintendent
  • the community services board director
  • the chief magistrate
  • the Thomas Jefferson area criminal justice planner

What stakeholders in Charlottesville-Albemarle County are saying about the EBDM Initiative:

The level of dedication and collaboration has been truly inspiring, and I am amazed how even those who were perhaps skeptical and somewhat reluctant to participate in this effort have embraced the idea of using evidence-based decision making when developing or changing policies. – District Court Judge Robert H. Downer

Both data and experience strongly suggest that every interaction that takes place within the criminal justice system—be it an interaction with a victim or offender—creates an opportunity to contribute to harm reduction. As criminal justice providers and members of a broader community, we cannot miss this opportunity to learn and improve upon our work based on the collection, analysis, and use of data and information. – Tim Longo, Charlottesville Police Chief

There is reason to believe that we can improve outcomes in criminal cases by utilizing evidence-based decision making at each stage of the process where discretion is exercised…This is true at the system level, when choosing among alternative polices, practices, and programs. It is also true at the individual level in the context of sentencing decisions or the consideration of appropriate alternatives to traditional prosecution. – Dave Chapman, Charlottesville Commonwealth Attorney

 

Harm Reduction Goals

Charlottesville-Albemarle County’s harm reduction goals include the following:

  • Reduce future criminal justice costs by improving effectiveness and reinvesting savings in further crime reduction activities.
  • Reduce rearrest rates as defined by “rearrest for a jailable offense” (offenders released from criminal justice supervision three years after discharge).
  • Increase the local community’s trust and confidence in the justice system by changing policies and practices that undermine the credibility of the justice system from the perspective of victims, offenders, and the public.

Portfolio

Charlottesville-Albemarle County’s activities

Download some of Charlottesville-Albemarle County’s products

Charlottesville-Albemarle County in the news.

For more information on the effort in Charlottesville-Albemarle, contact Stephanie Garbo at sgarbo@oar-jacc.org

Eau Claire County, Wisconsin

Eau Claire County, Wisconsin web_admin

Eau Claire County, Wisconsin, is 655 square miles in size, has a population of approximately 100,000 citizens, and serves as home to more than 20,000 students enrolled at the University of Wisconsin-Eau Claire and the Chippewa Valley Technical College.

In 2010, the Eau Claire County district attorney’s office opened 4,002 criminal files. Charges were filed in 3,336 of those files, with 837 felony cases and 2,499 misdemeanor cases. That same year, 5,336 individuals were booked into the Eau Claire County Jail and held for some time period, as opposed to being booked and released. Approximately 510 individuals are placed on probation each year in Eau Claire County.

EBDM Stakeholders

Vision for EBDM

Eau Claire County envisions a research-based justice system that results in less crime and fewer victims.

Eau Claire County will use coordinated leadership, community collaboration, and innovative criminal justice programs to enhance public safety.

The EBDM policy team was formed as a subcommittee of the Eau Claire County Criminal Justice Collaborating Council (CJCC), which was established by county board resolution in September 2006. The principal mission of the CJCC is to enhance public safety through system and community collaboration, to maintain and establish effective rehabilitation programs, and to foster innovative correctional programs. In addition, the CJCC is committed to providing the coordinated leadership necessary to establish and foster innovative corrections programs through process improvements.

The EBDM policy team is comprised of a variety of stakeholders, including:

  • the county administrator
  • three circuit court judges
  • a district attorney
  • a Wisconsin Department of Corrections representative
  • a state public defender
  • the county board
  • the county sheriff’s department
  • the chief of police
  • the district court administrator
  • the Department of Human Services
  • the CJCC coordinator
  • community members

What stakeholders in Eau Claire County are saying about the EBDM Initiative:

Working through the EBDM process has highlighted, for me, a heightened sense of the obstacles and challenges that not only face the court system(s) that I work with(in) but the very real potential that can be demonstrated through the systematic contemplation of the obstacles and challenges facing the Eau Claire County justice system. – Scott K. Johnson, Tenth District Court Administrator

We now openly discuss both our successes and failures and strive for continual improvement based on objective data. We have seen a broad acceptance of this change, and our employees have recognized the value of this type of policing strategy. –Chief of Police Jerry Matysik

Harm Reduction Goals

Eau Claire County’s harm reduction goals include the following:

  • Reduce the number of individuals who are convicted of crimes within three years of the completion of their criminal justice system contact.
  • More effectively allocate and use criminal justice system resources, as evidenced by reduced criminal caseloads and incarceration levels.

Portfolio

Eau Claire County’s activities

Download some of Eau Claire County’s products

Eau Claire County in the news

For more information on the effort in Eau Claire County:
Contact Tiana Glenna, CJCC Coordinator, at tiana.glenna@co.eau-claire.wi.us

Grant County, Indiana

Grant County, Indiana web_admin

Grant County, Indiana, is located 65 miles north of Indianapolis and is home to approximately 69,000 residents.

In 2009, the local criminal justice system had 3,850 jail bookings. Grant County’s courts and corrections department is known as a leader in the implementation of evidence-based practice. Representatives were invited by the National Institute of Corrections to attend the 2009 “Inter-site Summit,” where a select group shared their experiences with EBP implementation. Grant County is a pilot site for a University of Cincinnati probation supervision curriculum and is currently training staff to conduct and pilot the “Cognitive Self Change Community Model” developed for use with prison populations. Both the drug court and reentry courts have achieved recidivism reduction as compared with control groups.

EBDM Stakeholders

An EBDM Mission

The criminal justice system of Grant County promotes risk and harm reduction by utilizing collaborative decision-making and interventions founded on evidence-based research.

The EBDM team is a subgroup of the Community Corrections Advisory Board. This board, established by statute to advise local correctional programs, has been in existence since the early 1980s and has monitored a variety of grant-funded services over the years. The EBDM team is composed of the following members:

  • felony court judges
  • the county prosecutor
  • the jail administrator
  • the police chief
  • a victim advocate from the prosecutor’s office
  • the director of county correctional services
  • the director of community corrections
  • public defenders
  • representatives from the county fiscal body
  • a mental health agency representative
  • representative from the state judicial conference
  • the director of community programs at the Indiana Department of Corrections

What stakeholders in Grant County are saying about the EBDM Initiative:

This project promises to build upon existing EBP efforts and appeals to my desire to positively impact my community and to benefit personally and professionally from the best technical assistance in the field. – Cindy McCoy, Correctional Services Director

A relatively small jurisdiction can still be innovative and sophisticated enough to protect the public by seeking to reduce recidivism through the use of research-based practices. – Judge Jeffrey Todd

Since being assigned as the chief of police, I have been fascinated with the prospect of conducting business with a focus on a realistic and thoughtful assessment of crime and recidivism in our community based more on real evidence and less on what we suspect may be problems. – Police Chief David Gilbert

 

Harm Reduction Goals

Grant County’s harm reduction goals include the following:

  1. Reduce the use of jail for low risk, nonviolent, pretrial defendants by 10% over three years.
  2. On average, case processing from arrest to disposition will meet ABA standards: 9 months for felonies, 90 days for misdemeanors.
  3. Within 1 year, 70% of victims will report satisfaction with the court process.
  4. Within 3 years, new offense rearrests for probationers will be less than 40%.
  5. Within 3 years, improve housing stability, employment, and family functioning for probationers by 25%.

Portfolio

Download some of Grant County’s products

  1. Policy Team Charter
  2. Grant County Evidence-Based Practice®Evolution
  3. EBDM Logic Model
  4. EBDM Phase III Workplan

For more information on the effort in Grant County, contact Cindy McCoy at cmccoy@grantcounty.net

Mesa County, Colorado

Mesa County, Colorado web_admin

Mesa County (Grand Junction), Colorado, is located on Colorado’s western slope, approximately 250 miles west of Denver. The County encompasses 3,300 square miles; 74% of that territory consists of public land. The population of Mesa County is estimated at 153,712. Mesa County historically is an agricultural area that is rapidly becoming urbanized.

The county’s court system constitutes the 21st Judicial District of Colorado, one of several single-county judicial districts in the state. The office of the Mesa County sheriff is an elected position that has responsibility for the county jail, work release, and work-ender programs. The average daily population of the jail is 360, with approximately 7,000 new bookings and releases each year. The Criminal Justice Services Department is a county department that is responsible for community corrections, day reporting, a county-managed treatment facility, and electronic home monitoring services. Probation and parole are both state-managed functions, managed by separate entities.

EBDM Stakeholders

Vision for EBDM

The EBDM executive committee’s vision is one less crime, one less victim, and one less offender to create a safer community through the use of principles and practices of reliable evidence-based decision making.

The Mesa County Criminal Justice Leadership Group (CJLG) consists of 17 agencies. It was formed in 2010 (just prior to the EBDM Initiative) for the purpose of collaboratively making system changes to address jail overcrowding. The CJLG has served as the sounding board and oversight committee for the EBDM executive committee.

The EBDM executive committee is composed of the following stakeholders:

  • the chief judge
  • county judges
  • the deputy director of the Office of the Alternate Defense Counsel
  • the director of the Office of the Public Defender
  • the county sheriff
  • the chief of probation
  • the probation department supervisor
  • the district attorney
  • a deputy district attorney
  • the director of the Mesa County Criminal Justice Services Department
  • a judicial administrator
  • a county data analyst
  • the contracted local initiative coordinator

What stakeholders in Mesa County are saying about the EBDM Initiative:

This initiative has more potential to reach the common goals of less crime, a stable jail population, less fear, and increased community satisfaction in the justice system than anything I’ve seen in this community in the last 25 years. – Sheriff Stan Hilkey

A few years ago, we would not have seen an invitation to these types of meetings. Now we are present and fully involved. – Private defense attorney

Harm Reduction Goals

Mesa County’s harm reduction goals include the following:

  1. By 2015, 75% of all offenders will not recidivate within 12 months of successful completion of one of the primary sentencing options.
  2. By 2014, increase public safety by limiting pretrial misconduct of medium and high risk defendants to no more than a 5% failure to appear rate and a 20% new offense rate.
  3. Within 36 months, reduce the amount spent on low risk defendants and offenders in primary sentencing options by 33% so that financial and program resources can be better utilized.

Portfolio

Download some of Mesa County's products and read more about Mesa County in the news:

  1. Press Release: Mesa County Leads Innovation Behind Scientific Criminal Justice Pretrial Evaluations
  2. Policy Team Collaboration
  3. Chief Justice: Mesa County National Model

For more information on the effort in Mesa County, contact Sue Gormley at sgormley23@gmail.com.

Milwaukee County, Wisconsin

Milwaukee County, Wisconsin web_admin

Milwaukee County, Wisconsin, is located in the southeastern part of the state and has a population of approximately 950,000. The county includes the city of Milwaukee, which is the largest city in the state of Wisconsin and the 26th most populous city in the United States. The judicial system is comprised of the courts and the district attorney’s office, which work with law enforcement agencies to administer and enforce state and municipal law. The Milwaukee County Sheriff’s Office, the City of Milwaukee Police Department (MPD), and the Milwaukee Fire Department (MFD) are the three largest entities providing public safety services in the region.

EBDM Stakeholders

An EBDM Vision

By applying what the evidence tells us about what actually works in protecting the community and holding offenders accountable, Milwaukee County's criminal justice system will make the smartest possible use of its limited resources, continuously improving its performance against quantifiable goals and reinvesting the savings in programs that reduce crime in the first place.

The EBDMI policy team was formed in 2010 as a subset of the executive committee of the Milwaukee County Community Justice Council, formed in 2008. The team includes the following elected officials and stakeholders:

  • the county executive
  • the sheriff
  • a county board member
  • the district attorney
  • the city mayor
  • the chief judge of Milwaukee County Circuit Court, First Judicial District
  • the city’s chief of police
  • the State of Wisconsin first assistant public defender
  • the executive director of the Benedict Center
  • representatives from the Department of Corrections and the State Office of Justice Assistance
  • the United States Marshall of the Eastern District of Wisconsin (ex officio)
  • the court’s pretrial services coordinator
  • an inspector from the sheriff’s office
  • the presiding judge of the felony division of the Milwaukee County Circuit Court

The team is staffed by the coordinator of the community justice council, a deputy district attorney, and a public defender.

What stakeholders in Milwaukee County are saying about the EBDM Initiative:

The EBDM Initiative recognizes that “…true system change requires leadership from key policymakers, commitment throughout all levels of justice system organizations, and policy and practice alignment.” I couldn’t agree more. In the past, law enforcement in particular has been a closed society, silo-like endeavor. No longer. The EBDM Framework will further shape the work that we have already begun in interagency cooperation and systemic change, and as such its potential is truly exciting.
– Sheriff David Clarke, Jr.

From the perspective of community activists, change comes really, really hard in Milwaukee. So the breadth of collaborative action to initiate EBDM within every facet of the justice system is nothing short of awesome. System-encompassing evidence-based practice-already being implemented, solidly on the drawing board, or still in the alluring "what if" stage-underscores such a high degree of commitment that talk of becoming a model for the nation is serious talk.
– Kit McNally, Benedict Center Executive Director

Harm Reduction Goals

Milwaukee County’s harm reduction goals include the following:

  1. Reduce by 25% the number of people with mental health needs who lose their benefits due to being jailed or losing housing, and increase by 25% the number of individuals with mental health needs who are reconnected to the services they need within 20 days of arrest.
  2. Safely release and/or supervise 15% more pretrial detainees in the community rather than in jail, generating at least $1,000,000 in savings that can be reinvested in the community, and at the same time reduce by at least 40% the already low rates at which defendants waiting for trial fail to follow pretrial rules.
  3. Divert or defer prosecution in 10% more cases than we do currently, holding offenders accountable, compensating victims, and reducing recidivism, while generating at least $350,000 in savings that can be reinvested in the community.
  4. Demonstrate in a pilot project that by terminating probation as soon as an offender in need of treatment has received sufficient treatment, we can cut the cost of probation by at least 50% and at the same time reduce probation recidivism by 50%.

Portfolio

Read more about Milwaukee County’s activities

  1. Universal Screening/Praxis
  2. Milwaukee County Early Intervention Program: Presentation given at the National Symposium on Pretrial Diversion, May 30, 2012
  3. As part of Milwaukee County’s efforts to expand and strengthen their deferred prosecution agreement and diversion processes, they are planning to bring in some professionals to train their pretrial staff and outside treatment providers in cognitive behavioral treatment (CBT) approaches (i.e., Thinking for a Change, Moral Reconation Therapy). By providing CBT groups on-site, the county can guarantee that the appropriate participants receive the services that will address their underlying antisocial thoughts and belief systems. Milwaukee County hopes that this would lead to better recidivism outcomes and also increase buy-in from various stakeholders.

Download some of Milwaukee County’s products

  1. Phase III Application
  2. Scorecard

Read more about Milwaukee County in the news

  1. Video Clip: The Value of Criminal Justice Coordinating Councils in Wisconsin
  2. Online Article: The Return of a Very Bad Idea
  3. Blog: Evidence-Based Decision Making: The Increasing Use of Research in our Criminal Justice System
  4. Online Article: The Evidence Tells Us We are on the Right Track, The Third Branch, Wisconsin Court System Website
  5. Article: Blessings, and Other Metrics Worth Counting, Milwaukee Bar Association Messenger, Winter, 2011
  6. Article: Get Smart? Marquette Lawyer, Fall 2011
  7. Video Clip: Justice in Times of Transition and Tight Budgets

For more information on the effort in Milwaukee County:
Visit https://www.milwaukee.gov/EN/MCJC
Contact James Hiller at jhiller@publicpolicyforum.org
Contact Chief Judge Jeffrey Kremers at jeffrey.kremers@wicourts.gov

Ramsey County, Minnesota

Ramsey County, Minnesota web_admin

Ramsey County, Minnesota, has a population of 511,035. Its county seat is Saint Paul, which is also Minnesota's state capital.

Vision for EBDM

Ramsey County’s vision for the EBDM Initiative is:
One less crime.
One less victim.
One less offender.
A strategy for safer communities.

Ramsey County has 13 law enforcement agencies including the Ramsey County Sheriff’s Office (Minnesota’s First Law Enforcement Agency) and Saint Paul Police Department. In addition to providing law enforcement services similar to police departments (patrol, investigations, crime prevention, etc.) in seven cities, the Sheriff’s Office is also responsible for operating the 497-bed county jail (pre-trial), providing court services (court security, warrants, civil process), and protecting the county’s waterways. The County Attorney, the County’s top prosecutor, and the Sheriff, the County’s chief law enforcement officer, are elected every four years while Judges are elected every six years. Appointed criminal justice leaders include the Director of Community Corrections, the Chief Public Defender, City Attorneys, and City Chiefs of Police.

The Saint Paul Police Department, the county’s largest law enforcement agency, averages about 240,000 calls for service a year. The Ramsey County Sheriff’s Office averages approximately 23,000 bookings per year. Ramsey County is a Community Corrections Act county, meaning the county provides supervision (probation and parole) and correctional services. Community Corrections provided services to 24,885 adult offenders under supervision in 2009. Community Corrections is also responsible for operating the Ramsey County Correctional Facility, a 556-bed post-conviction institution, housing inmates serving sentences of one year or less.

EBDM Stakeholders

Prior to the EBDM Initiative, leaders would meet one-on-one or as part of specific projects with limited scope and complexity, but not as a whole to work on system-wide issues. The Ramsey County EBDM policy team includes the following stakeholders:

  • the county sheriff
  • the county attorney
  • a county commissioner
  • the chief judge of the Second Judicial District Court
  • the county community corrections director
  • the chief of the Saint Paul Police Department
  • the chief public defender of the Second Judicial District Court
  • the city attorney for the City of Saint Paul
  • the county pre-trial services executive director
  • the county chief deputy sheriff
  • the first assistant county attorney
  • a deputy city attorney for the City of Saint Paul
  • the executive director of the Office of Justice Programs
  • representatives from additional criminal justice and victim services organizations

What stakeholders in Ramsey County are saying about the EBDM Initiative:

When we can combine prosecutorial discretion with evidence-based decision making principles, we get better outcomes in the criminal justice system. – John Choi, Ramsey County Attorney

We owe it to the people we serve to be doing the smartest things with shrinking resources. The time is right. – Jeri Boisvert, Office of Justice Programs Executive Director

What I like about this initiative is that it is an external audit of our current practices. We need to move toward public justice; our jobs are larger than public safety. – Patrick Kittridge, Minnesota Second District Chief Public Defender

For me, it is about better outcomes. As public employees, it is our obligation to do our jobs the best we can, using research to do it. – Mary Pat Maher, Project Remand (Pre-Trial Services) Executive Director

We have come together in the past to successfully work on projects, but there has been no real integration of our work. We don't really understand the strategies that each of us employs and what we truly achieve. This is our chance to get it all out there and figure out if what we do truly drives public safety. – Carol Roberts, Ramsey County Community Corrections Department Director

Harm Reduction Goals

Ramsey County’s harm reduction goals focus on seven areas. These include increasing:

  1. community safety;
  2. collaboration;
  3. offender accountability;
  4. criminal justice system cost savings;
  5. swift, certain, and proportional responses to criminal behavior/misconduct;
  6. stakeholder support; and
  7. community awareness and support.

Portfolio

Read more about Ramsey County’s activities during Phase II:

  1. To kick off the Initiative, an EBDMI Awareness Event was held to increase awareness of evidence-based practices and evidence-based decision-making as well as to build support across all disciplines. The event was well attended by 225 people representing over 15 national, state, and local organizations.
  2. Members from every organization met to create a system map of the County’s Criminal Justice System. The map, which details how the majority of cases move through the system, is color-coded by organization and clearly identifies each decision point. Accompanying the system map is a document that briefly explains each organization’s role and involvement in the system. Key decision points are explained and additional information is provided regarding how and why decisions are made.
  3. Developed a report: Warrants in Ramsey County. This report provided detailed information (warrant types, offense levels, investigating agencies, and demographics) about active warrants and warrants issued in 2009.
  4. Developed a report: Court Case Overview. By providing a snapshot in time (offenses, charging methods, failures to appear, and dispositions), this report showed that many cases involve the courts.
  5. An evidence-based practices (EPB) knowledge survey was administered. The scores illustrated a basic understanding of EBP.
  6. Although funding limitations ruled out the development of a law enforcement diversion program at present, two successful local programs were analyzed.

Read more about Ramsey County’s current activities:

  1. Ramsey County is working to form the Criminal Justice Coordinating Council (CJCC). The CJCC will serve as a forum through which criminal justice, human service, community, and government organizations may promote best practice improvements in the criminal justice system that transcend various organizations and communities.
  2. While Ramsey County has already implemented many best practices during Phase II, additional projects, guided by best practices, were developed to further achieve various harm reduction goals. Throughout the next two years, Ramsey County will continue to implement existing and new projects.

For more information on the effort in Ramsey County, contact at Ryan O’Neill at Ryan.Oneill@co.ramsey.mn.us.

Yamhill County, Oregon

Yamhill County, Oregon web_admin

Yamhill County, Oregon, encompasses 718 square miles and is home to more than 99,000 residents. It is approximately 40 miles southwest of the heart of Portland.

The Yamhill County Jail can house 257 inmates, but most often operates under capacity. Yamhill County Community Corrections supervises 1,121 adult offenders on misdemeanor and felony probation, and on post-prison supervision. The Yamhill County criminal justice team has enjoyed a long history of close collaboration and innovation as evidenced by the establishment of a special management team prior to the EBDM Initiative. The purpose of this team was to identify methods to decrease the likelihood that mentally ill offenders would be housed in the local jail.

EBDM Stakeholders

Vision for EBDM

Yamhill County envisions a safer community where professionals work together utilizing data, research, and evidence-based practices in the criminal justice system.

Yamhill County will experience enhanced public safety, a reduction in the number of victims, greater offender accountability, and a reduced threat of harm through the appropriate application of proven practices at all phases of the criminal justice process.

Yamhill County stakeholders have a history of meeting weekly to work toward shared goals and system improvements. The EBDM policy team formed as a natural evolution of this work. The team is comprised of

  • the presiding judge
  • a county commissioner
  • the district attorney
  • the sheriff
  • a defense attorney
  • a victim advocate
  • the director of Health and Human Services
  • the director of community corrections

What stakeholders in Yamhill County are saying about the EBDM Initiative:

I am enthusiastic about the opportunity to take the hard work of our collaborative effort to a higher level. I believe this will be an opportunity not only for Yamhill County to improve our practices, but also to demonstrate an approach that can be a model for other counties, especially those of similar intermediate size and population. – Judge John L. Collins

EBDM will allow us to expand our evaluation of the entire criminal justice system and to make decisions that will reduce risk and harm to all of our citizens. The key policymakers in the county are already engaged and are committed to taking the next steps necessary. – Mary Stern, County Commissioner

Harm Reduction Goals

Yamhill County’s harm reduction goals include the following:

  • Increase victim and public safety by reducing pretrial misconduct, increasing court appearance rates, making informed release decisions, and providing effective conditions of release for individuals who can be managed safely in the community.
  • Reduce harm to defendants and their families due to unnecessary pretrial detention of individuals who can be safely managed in the community.
  • Achieve greater financial return on investment in treatment, rehabilitation, and alternatives to incarceration.
  • Reduce recidivism by 5% in community corrections felony populations over the next 48 months by improving early assessment practices and properly matching offenders to evidence-based programs based on their risk and need.
  • Increase the overall health and safety of our community by focusing on cost-effective, research-based principles to improve our response to, and reduce the involvement of, special needs individuals in the criminal justice system.

Portfolio

Yamhill County’s activities

  • The Yamhill County Policy Team meets monthly and is prioritizing its efforts on evidence-based sentencing, and evidence-based pretrial release and supervision.
  • Regarding pretrial services, Yamhill County is currently exploring the possibility of shifting this function to the Community Corrections Department. No final decision has been made.
  • Yamhill County was recently notified that it was selected as a participant in the Justice Reinvestment Initiative (JRI). Preliminary meetings have already occurred regarding the need to improve its data management systems so that progress can be analyzed as plans are implemented.
  • The policy team is also in the process of arranging local training. An expert from Multnomah County will be training Yamhill County officials on the use of the Virginia Pretrial Risk Assessment Instrument. A programming expert from Marion County will be offering parole and probation staff training on motivational and cognitive programs, and motivational group facilitation. The Community Corrections Manager and two parole and probation officers have recently begun EPICS training by the University of Cincinnati.
  • Community Corrections has recently hired a new facilitator for Moral Reconation Therapy. A meeting is being scheduled with all community corrections program staff to re-evaluate and modify the county’s current menu of motivational and cognitive programs.
  • Members of the Yamhill County Policy Team have given presentations on the EBDMI to the Multnomah County Local Public Safety Coordinating Council, the Oregon Community Corrections Directors Association, the Oregon Community Corrections Commission, and to the National Association of Counties in Portland, Oregon.

Download some of Yamhill County's products

Read about Yamhill County in the news

For more information on the effort in Yamhill County:
Visit http://www.co.yamhill.or.us/content/evidence-based-decision-making-initiative-ebdmi.
Contact Ted Smietana at smietant@co.yamhill.or.us.

State Sites

State Sites web_admin

EBDM in Wisconsin:
A Primer

Evidence-Based Decision Making (EBDM) is a disciplined approach to using data and research to inform and guide decision making across the criminal justice system. State and local criminal justice partners are working together to systematically use research to positively change criminal behavior. There is a growing body of research that informs criminal justice agencies how to increase performance and be more effective. Historically, systems lacked collaboration around a common set of goals and outcomes.

Download the Wisconsin Counties Primer

Read about Wisconsin's Story in the news:
Wisconsin Chosen as One of Three States in Nation for Criminal Justice Initiative

Virginia

Read about Virginia's Story in the news:
Richmond Department of Justice Services Joins Nationally Recognized Initiative

Starter Kit

Starter Kit web_admin

The Evidence-Based Decision Making (EBDM) in Local Criminal Justice Systems Starter Kit provides guidance to jurisdictions interested in implementing the EBDM Framework. The Starter Kit outlines a planning process that is informed by the experiences of the seven EBDM sites that participated in Phase II of the EBDM initiative.

Purpose of the Starter Kit

The Starter Kit is intended to help collaborative criminal justice policy teams build their capacity to engage in EBDM. It follows the EBDM “Roadmap,” a step-by-step process designed to assist local jurisdictions in preparing to implement the EBDM Framework.

Who Should Use the Starter Kit

The Starter Kit is designed for use by multidisciplinary criminal justice teams that have read and agreed to the basic premises of the Framework for Evidence-Based Decision Making in Local Criminal Justice Systems and that are committed to working together toward a justice system based on local data and empirical research. The team’s ultimate goal would be to achieve a reduction in the likelihood of pretrial misconduct, post-sentence reoffense, and other forms of community harm.

The Roadmap: Getting from Here to There

While the planning for the implementation of EBDM will not look the same across all local systems, the EBDM Starter Kit outlines a common, core set of activities designed to encourage or advance system alignment; system alignment is an essential foundation for EBDM. These core activities make up the EBDM Roadmap: 

  1. Build a genuine, collaborative policy team.
  2. Build individual agencies that are collaborative and in a state of readiness for change.
  3. Understand current practice within each agency and across the system.
  4. Understand and have the capacity to implement evidence-based practices.
  5. Develop logic models.
  6. Establish performance measures, determine outcomes, and develop a system scorecard.
  7. Engage and gain the support of a broader set of stakeholders and the community.
  8. Develop a strategic action plan for implementation.

A number of smaller steps are suggested under each core activity. To download a chart that summarizes these steps, click here.

It is important to note that these activities and steps typically do not occur in a linear fashion. Instead, teams should expect that they will conduct a number of activities concurrently, or they may need to revisit activities that have already been completed as a result of new learnings. You might say that there is not just one road, but many roads that lead to the final destination, and each team may take a different route to get there.

To begin the Roadmap, select one of the specific, core Roadmap activities in the navigation menu.

Additional Resources

Other products are available to supplement the EBDM Framework and the EBDM Starter Kit. User’s Guides specifically developed for defense attorneys, judges, pretrial justice professionals, and prosecutors will become available in 2012. The User’s Guides will summarize key research findings and describe the potential implications of EBDM for these disciplines.

1. Policy Team

1. Policy Team web_admin

Activity 1:
Build a genuine, collaborative policy team

Collaboration in the criminal justice system seeks to overcome the limitations of traditional and non-systemic approaches to criminal justice problem solving and solution development. It brings together stakeholders to share information, develop common goals, and jointly create policies to support those goals. “Stakeholders” are defined as those who influence and have an investment in the criminal justice system’s outcomes. These systemwide stakeholder groups are referred to as policy teams.

Ideally, policy teams are comprised of the criminal justice agencies and community organizations that impact, or are impacted by, decisions that will be made by the collaborative team. The specific composition of the collaborative team varies from jurisdiction to jurisdiction. Those with the positional power to create change within their own agencies and organizations are appropriate members of the collaborative team. The chief judge, court administrator, elected prosecutor, chief public defender and representative of the private defense bar, administrator of the community corrections agency, police chief and elected sheriff, pretrial administrator, victim advocates, local elected officials (i.e., city manager, county commissioner), service providers, and community representatives all play a part in the administration of justice and bring valuable information, resources, and perspectives to this collaborative endeavor.

Click on the documents in the navigation bar on the left for a set of activities designed to assist you in establishing a policy team and building a solid foundation for your collaborative team’s work together.

Elements of an EBDM justice system include:

  1. A highly functioning collaborative policy team;
  2. A shared vision for the criminal justice system; and
  3. A track record of meaningful team accomplishments.

1a: Conducting an EBDM Readiness Checklist

1a: Conducting an EBDM Readiness Checklist web_admin

Introduction

Jurisdictions interested in working towards building an EBDM justice system may want to begin by assessing their readiness for undertaking such work. The EBDM Readiness Checklist is designed to establish your “baseline” for working together and to help you explore a range of team-, policy-, and practice-related elements in your system. The results of the checklist will provide an assessment of your jurisdiction’s readiness to build capacity for implementing the EBDM Framework and can be used as a foundation for moving forward with the EBDM planning process.

Purpose

To measure and facilitate a dialogue among team members on the team’s readiness to begin working on building an EBDM justice system.

Participants

All policy team members should be involved in completing and debriefing the results of the Readiness Checklist.

Instructions

The checklist is designed to serve as a catalyst for discussion on the team’s capacity and willingness to adopt the EBDM Framework. While the checklist may be administered in a variety of ways, it is critical that the results are discussed and processed by the full policy team.

Administering the Checklist

Below are several approaches a jurisdiction might take to administer the checklist. In all cases, there are two important considerations:

  • Members should be encouraged to be as honest as possible. Candor will lead to the most accurate—and therefore helpful—results.
  • Teams might consider using the services of a neutral facilitator1 to debrief the results and implications of the checklist responses. Often, a neutral facilitator can help a group objectively engage in constructive dialogue and action planning, particularly around sensitive issues.

Methods of Administration

Some of the ways that jurisdictions may consider conducting the checklist include the following:

Administer the checklist during a meeting of the full team, using transponders to collect real-time, anonymous answers from individual team members. This approach will likely result in candid feedback due to the anonymity; will provide members with the opportunity to immediately see—through the visual results the transponder system offers—areas of agreement and diversity of view; and will create a forum for dialogue about a variety of team-, policy-, and practice-related issues.

The checklist may be distributed on paper for team members to complete individually either before or during a meeting, and responses can either be aggregated and reviewed or discussed in an open forum.

The survey may also be conducted online in advance of a policy team meeting using an online survey service (e.g., Survey Monkey). The results should then be tallied and discussed during the team meeting.

Discussing Responses

Team members should discuss together the results of the checklist; it will likely surface important areas of work for the team to undertake. These areas of work can be addressed through the various documents provided in this EBDM Starter Kit. For instance, the team should discuss

  • the level of policy-level collaboration in the jurisdiction and key stakeholders’ level of commitment to future collaboration;
  • the extent to which the team has effectively engaged justice system agency staff—and community members—in discussions regarding a vision for an EBDM justice system;
  • the extent to which policymakers and agency staff have the knowledge and skills necessary to implement evidence-based decisions;
  • the breadth and depth of evidence-based practices currently in place in the jurisdiction (e.g., use of assessment tools, targeting services to criminogenic needs); and
  • policymakers’ willingness to agree upon systemwide outcomes, and the jurisdiction’s ability to collect and analyze data to measure these goals.

The checklist is not intended to provide a list of items that must be addressed prior to a team starting the EBDM process. However, the checklist does include the core activities that a team will need to engage in for the successful implementation of the EBDM Framework. Therefore, if team members express serious concerns about certain items (particularly in terms of their commitment to working together), this may indicate that the time is not right to undertake this work and/or that further foundation-building is necessary before a jurisdiction is positioned to fully adopt the Framework.

 

1 Jurisdictions might explore whether technical assistance is available for this purpose.

Download the EBDM Readiness Checklist below.

1b: Administering a Collaboration Survey

1b: Administering a Collaboration Survey web_admin

Introduction

Research suggests that the presence of genuine collaborative relationships among members of a team is a key ingredient to the team's success in achieving its vision, mission, and goals. When specific conditions are successfully met, teams are most likely to be "highly effective." Research demonstrates that the most effective teams have eight common characteristics:

  • a clear and elevating goal;
  • a results-driven structure;
  • competent team members;
  • a unified commitment;
  • a collaborative climate;
  • standards of excellence;
  • external support and recognition; and
  • a principled leadership.

Too often groups do not take the time to assess their working relationships or consider how best to foster them to increase their likelihood of success.

Working Together: A Profile of Collaboration is a survey developed by David D. Chrislip and Carl E. Larson following extensive research on highly effective teams. The survey provides teams with a "baseline" regarding their level of collaboration. Its purpose is not to "rate" the team, but to identify the areas in which team members agree the team is strong and those areas where there is room for improvement.

Purpose

To utilize the Working Together survey to begin a conversation among team members—a conversation that can be continued in future meetings as necessary—that will surface areas of strength and those in need of improvement with regard to the level of collaboration within the team and that will result in agreements on how you can best position your team to be as effective as possible.

Participants

All policy team members should be involved in taking and debriefing the results of the Working Together survey.

Instructions

Conducting the Survey

  • Distribute the survey to policy team members and ask them to complete it individually. The survey may be completed on paper during a policy team meeting or prior to a meeting using an online survey method such as Survey Monkey.
  • The survey instrument can be downloaded here1.
  • If the survey is administered during a policy team meeting, the results can be tallied immediately so that the team can discuss them. Alternatively, if the survey is distributed ahead of time, the results should be tallied and then reviewed at the meeting.
  • For each question, consider including the following: total number of responses (N), minimum score, maximum score, mean, range, and the number of team members who indicated that they could not answer the question because the team was "too new" and therefore the question did not apply.

Scoring and Interpreting the Survey

N is the number of respondents who answered the question.

The minimum and maximum reflects the lowest and highest answers for the question and are used to calculate the range of scores.

The mean is the average of all the respondents' scores.

Lower mean scores (closer to 1) reflect a greater percentage of "mostly true" or "true" answers.

Higher mean scores (closer to 4) reflect a greater percentage of "mostly false" or "false" answers.

The range is calculated by subtracting the minimum score from the maximum score. The range reflects the variance in respondents' answers. The higher the range the lower the level of agreement between respondents; the lower the range the higher the level of agreement between respondents.

The column Too New to Answer reflects the number of respondents who felt they could not answer the question because the team had only recently begun working together.

Processing the Survey

Share a copy of the teams' Working Together survey results with the entire policy team.

Ask team members to discuss the results of the survey and identify the areas where the greatest strength is noted. A lower mean score (closer to 1) reflects a greater percentage of "True" or "Mostly True" answers (i.e., strengths).

Note these on a flip chart and use this as an opportunity to affirm the team regarding its strengths. Remind them that strengths should be drawn upon to address areas in need of improvement.

Next, ask team members to identify the areas where there seems to be the greatest opportunity for improvement. A higher mean score (closer to 4) reflects a greater percentage of "False" or "Mostly False" answers (i.e., areas in need of improvement).

Note these on the flip chart and take some time to process with the team areas in need of improvement. Do they reflect the fact that the team is new to working together? Do they reflect difficulties that have occurred in the past? Is it advisable to devote some future team time to identifying ways to specifically strengthen the team in these areas?

Challenges in collaboration that need to be addressed should be noted so that they are considered for inclusion in the team's action plan.2

Following Up

Re-administer the survey at future intervals (every six months is recommended for new teams) to determine changes in members' perceptions of the team's level of collaboration and to identify future action items.

1 An online version of the tool can be found at http://starterkit.ebdmoneless.org/wp-content/uploads/2016/01/EBDM-Policy-Team-Collaboration-Survey.pdf.

2 See 1l: Developing an Action Plan for your Policy Team.

1c: Creating a Vision for Your Policy Team

1c: Creating a Vision for Your Policy Team web_admin

Introduction

Clarity of outcome is critical to the success of any team. A fundamental starting place for a system of evidence-based decision making is agreement among key policymakers regarding their overarching purpose—or vision—for the justice system. It will likely take time and hard work to develop a vision for your team, and that vision may change over time. Nonetheless, it will be your vision that:

  • inspires your team to come to meetings when the press of other business grows strong;
  • guides your policy team's course of action; and
  • helps secure the support for change from your colleagues and community.

Definition of a "Vision"
A vision is a statement that paints a picture of the future that you hope to create (i.e., a statement of your preferred future). It captures your hopes of what will result from your team's efforts. It represents a future for which your policy team is willing to take responsibility for attempting to achieve.

Purpose

To facilitate a dialogue among team members that will result in a statement reflecting the vision for the justice system in your community

Participants

All policy team members should be involved in the development of your vision statement.

Instructions

  • Consider the following questions:
    1. If the criminal justice system in our jurisdiction were working ideally, what would be its characteristics?
    2. What results would it achieve?
    3. What values would guide how these goals are achieved?
  • Team members should take a few minutes to jot down their ideas about each of these questions. When they are finished, go around the room and record each person's first response on a flip chart. Go around the room again and note each person's second response. Continue this process until all ideas are recorded for for all three questions.
  • Review the ideas generated. Discuss each one and ensure that its meaning is clear. Eliminate duplications. To answer each question, develop a statement or set of statements that reflects the consensus of the team.
  • If multiple statements are produced, prioritize these by asking each member to rank order the statements, then tally the "votes" for each statement. Select the portions of the prioritized statements that resonate with team members.
  • From these, craft a single vision statement that reflects all of the agreed-upon ideas and comments.
  • This process may take some time; you may not be able to complete this work in one session or even as a full team. Often it is useful to form a subcommittee to craft a proposed vision statement based on the concepts/statements developed by the full team.

Tips

  • Remember that your vision statement should be energizing and inspiring and, when completed, describe your hopes for the future.
  • Avoid getting hung up on your current situation, limitations, or tasks needed to reach your vision. The vision statement is about where you want to go and why.
  • Make the creation of your vision statement an integral part of your early work together.
  • Resist the urge to "just get it done."
  • Keep it simple. Your vision statement should be easy to explain and easily understood by non-team members. Avoid jargon.
  • Keep it short. The details belong in your mission statement and goals.
  • Work on your vision statement until it truly represents the hopes of everyone on the team. It should be a powerful and compelling statement.
  • Use the vision statement as a touchstone for your ongoing efforts. Once it is completed, display your statement during each meeting.
  • Revisit your vision statement from time to time, and change it as your work together evolves.

1 Language adapted from Center for Sex Offender Management (CSOM), 2007.

 

Example:
Milwaukee County, Wisconsin, Vision and Mission

Our community vision is for greater accountability
in the criminal justice system
and better stewardship
of criminal justice resources.

By applying what the evidence tells us about what actually works in protecting the community and holding offenders accountable, Milwaukee County's criminal justice system will make the smartest possible use of its limited resources, continuously improving its performance against quantifiable goals and reinvesting the savings in programs that reduce crime in the first place.

When it comes to crime and punishment, we recognize that certain practices and traditions hold an intuitive appeal, but we are determined to judge everything we do in light of what solid research and objective evidence demonstrate to be potent and cost-effective. We also recognize that our efforts will be fruitless unless they are collaborative; the partners in this process pledge not merely their cooperation, but stand for the success of each other in achieving these ambitions.

Additional Resources/Readings

Center for Effective Public Policy (CEPP). (2005). A clear and elevating goal (vision).
Retrieved from https://cepp.com/wp-content/uploads/2020/03/Teamwork-Exercise-Developing-a-Clear-and-Elevating-Goal-VISION.pdf.

Center for Effective Public Policy (CEPP). (2005). Collaboration: A training curriculum to enhance the effectiveness of criminal justice teams.
Retrieved from https://cepp.com/wp-content/uploads/2020/03/Collaboration-A-Training-Curriculum-to-Enhance-the-Effectiveness-of-Criminal-Justice-Teams-2005.pdf.

National Institute of Corrections (NIC). (2006). Getting it right: Collaborative problem solving for criminal justice.
Retrieved from https://nicic.gov/getting-it-right-collaborative-problem-solving-crimina...

Center for Sex Offender Management (CSOM). (2007). Enhancing the management of adult and juvenile sex offenders: A handbook for policymakers and practitioners.
Retrieved from https://cepp.com/wp-content/uploads/2020/01/6-CSOM-Handbook.pdf.

1d: Conducting a Stakeholder Analysis

1d: Conducting a Stakeholder Analysis web_admin

Introduction

Through the process of working to become an evidence-based decision making criminal justice system, your team will develop a comprehensive understanding of your current criminal justice system and a vision of what that system could look like in the future. Without complete representation of the justice system’s stakeholders on your team, you will be unlikely to develop a complete understanding of your criminal justice system or implement changes for advancement. Therefore, it is vital that your team include all key stakeholders as early in the process as possible. In other words, to implement meaningful changes, you must have at the table from the outset all those who might be involved in the potential changes your team will identify. In most cases, the choice of policy team members will be obvious and will include, at a minimum

  • law enforcement officers;
  • pretrial officials;
  • victim advocates;
  • prosecutors;
  • defense attorneys;
  • jail administrators;
  • court administrators;
  • judges;
  • community supervision officers; and
  • city/county administrators.

Many others may also be part of your team, for example, human services professionals, treatment providers, state legislators, citizen representatives, and members of the faith community.

Purpose

To help your team consider all of the individuals who have a stake in the outcomes you seek to achieve and ensure that they are included in your efforts

Participants

All policy team members should be involved in analyzing stakeholders to be included on your team.

Instructions[1]

A chart is attached to assist the team in recording their responses to Steps 1 through 5.

Engaging the Defense Bar

While it may be relatively easy to determine which critical players should be added to your team, the challenge is devising a strategy to bring them on board. In one county, the chair of the policy team worked tenaciously to engage the private defense bar in the effort.

The chair reached out to them through multiple calls, made a point to answer any questions they had (and put in their hands relevant research), and changed the time of policy team meetings to lunchtime in order to accommodate their schedules. The defense bar began to participate in different aspects of the work, including attending training events, participating in meetings to map the current system, and joining work groups on pretrial, plea, and sentencing issues to ensure their perspectives were considered.

Through this multipronged and deliberate strategy, the chair was successful in bringing the defense bar onto the policy team to learn about EBDM alongside the other partners.
  1. Brainstorm a list of all agencies, organizations, and individuals that have a “stake” in criminal justice decision making in your jurisdiction.
  2. Organize the list in a logical fashion (e.g., group together those with influence over particular decisions, such as arrest, pretrial, community interventions, etc.).
  3. Review the list. Identify those stakeholders already on your team and those that are missing.
  4. Consider the advantages and disadvantages of including the individuals or agencies on your list. What can they add to the team? What are the possible consequences if they are not involved?
  5. For each identified stakeholder, determine a possible representative, considering the following questions:
    • Does the team need policy-level representation, front-line staff, or both to help advance evidence-based decision making?
    • Is there a particular person who is uniquely able to serve as a liaison between their constituency group and your collaborative team?
    • Who can provide a unique perspective on your work, enhancing it with new ideas or insights?
  6. Discuss strategies for adding new members to the team, and create a work plan to carry out these strategies.

Tips

  • Invariably, you will overlook someone along the way. Remain flexible and bring others onto your team as you move forward and as you deem it appropriate.
  • If the team is already sufficient in size, consider adding others the team feels strongly about—such as citizens, community members, and other non-criminal justice representatives—to subcommittees and working groups. This has the advantage of including others and gaining their input in structured ways, while not expanding the policy team to an unworkably large number.
  • Typically, the team will develop a lengthy list of possible team members through this analysis. The trick is to carefully select members to ensure that the team is not overly large or unworkable. Remember to consider two key factors when selecting team members: (1) their power and influence with their peers and the larger community; and (2) their openness to ideas and to new ways of looking at old problems.

Example:
EBDM Talking Points Used in One Jurisdiction

After conducting a stakeholder analysis, one jurisdiction decided to approach the local police chief in an attempt to solicit his participation in the initiative. The following are some talking points that the team chair used in his meeting.

  • What is the EBDM project?
    • The campaign is focused on “One less offender. One less crime. One less victim.”
    • The focus is on harm reduction goals, i.e., reduced victimization and increased public safety and community wellness.
    • Leadership and courage are required.
    • The campaign involves developing a set of goals common to the entire system in order to achieve greater outcomes.
    • The work is focused on three areas of importance:
      1. collaboration across agencies;
      2. implementation of EBP and best practices; and
      3. organizational development and enhancement of agency performance.
    • It is based on the assumption that implementing research and best practices systemwide will achieve greater results, such as fewer crimes, reduced erosion of property values, less money spent on criminal justice, increased sense of safety, less financial loss by victims, and greater confidence of citizens in the criminal justice system.
    • Law enforcement is a critical part of the system; the campaign can’t be as successful without its involvement.
    • Data can be used to determine “hot spots” where law enforcement intervention is most needed.
    • An actuarial tool, such as a brief screening instrument, can be used to inform cite versus detain decisions.
    • Law enforcement can serve as role models by attending offender graduation programs and affirming progress.
    • Law enforcement can participate, as guest speakers, in behavioral change programs.
    • Social learning training (e.g., role modeling, fairness, respect) can be integrated into arrest practices.
    • There would be greater cooperation with other parts of the criminal justice system.
    • Law enforcement can assist in developing a countywide vision for the criminal justice system.
    • What project goals/benefits are specific to law enforcement?

Additional Resources/Readings

Center for Effective Public Policy (CEPP). (2006). Getting it right: Collaborative problem solving for criminal justice.
Retrieved from https://nicic.gov/getting-it-right-collaborative-problem-solving-crimina...

Center for Sex Offender Management (CSOM). (2007). Enhancing the management of adult and juvenile sex offenders: A handbook for policymakers and practitioners.
Retrieved from https://cepp.com/wp-content/uploads/2020/01/6-CSOM-Handbook.pdf

Appendix:
Stakeholder Analysis Template

Stakeholder Analysis Template

[1] Adapted from the Center for Sex Offender Management (CSOM), 2007. (Teamwork Exercise 1) https://cepp.com/wp-content/uploads/2020/01/6-CSOM-Handbook.pdf.

1e: Establishing Team Leadership

1e: Establishing Team Leadership web_admin

Introduction

Successful justice system collaboration depends on effective leadership. While much has been written about the composition of a collaborative team and its role as a whole in carrying out the mutually established vision, mission, and goals for criminal justice system improvements in a local community, there has been less emphasis on the leadership of these teams. And yet, the critical role of the team leader is undeniable. Indeed, when Carl Larson and Frank LaFasto (1989) studied the work of groups from fields as diverse as business, sports, community development, and public health in order to determine what makes teams succeed, the presence of leadership was among the most important characteristics of effective teams.

Collaborative leadership is different from management leadership of an agency. We typically think of leadership in the context of managing an agency. In the criminal justice world, the jail administrator, chief judge, director of a public defender’s office, elected prosecutor, community corrections director or probation chief, sheriff, and human services director are the positional or hierarchical leaders. They manage agencies that provide key functions in the justice system. However, qualities long associated with top management of an agency (e.g., being fair, efficient, consistent) are not necessarily the same qualities needed to lead a collaborative team.

Common qualities of effective collaborative leaders include the following:[1]

Defining Collaborative Leadership

In their work Collaborative Leadership, David Chrislip and Carl Larson describe collaborative leadership in the following way:

“[Positional] leaders are those who articulate a vision, inspire people to act, and focus on concrete problems and results. [But] collaboration needs a different kind of leadership; it needs leaders who can safeguard the process, facilitate interaction, and patiently deal with high levels of frustration. Collaboration works when…leaders…keep the process going.”
–Chrislip & Larson, 1994, p. 42.
  • Willing to take risks. Great leaders are dissatisfied with the status quo and “business as usual.” They are willing to take risks because they understand that the benefits of success outweigh the setbacks that may be encountered on the way.
  • Eager listeners. Collaborative leaders are open to all viewpoints; they seek input of all kinds and from all places. It shapes their thinking, and they expect that fresh ideas and varying points of view will have a similar impact on other members of the EBDM team.
  • Passionate. One of the most important characteristics of a collaborative leader is a highly visible passion for the cause. As Larson and LaFasto wrote, “Their visions or intentions are compelling and pull people toward them. Intensity coupled with commitment is magnetic.”[2]
  • Optimistic. Great leaders are effective, at least in part, because of their attitude. They believe in the possible, and they encourage that optimism among all members of the policy team as well as among the managers and staff of the criminal justice agencies that will implement evidence-based improvements in policy and practice.
  • Able to share knowledge, power, and credit. Effective leaders are not concerned with garnering individual recognition for their work; instead, they choose to acknowledge the achievements of others and emphasize the success of the group over the success of specific individuals.
  • Garner the respect and trust of team members and have no hidden agendas.
  • Have adequate time to serve as the collaborative leader of the policy team and complete the required tasks of team chair.

In addition, effective leaders possess an important skill set that serves to facilitate and support the collaborative process. Common skills of effective collaborative leaders include the following:[3]

  • Political and substantive knowledge and skills. Effective leaders have sufficient expertise to a) assist with the identification of the key officials who will serve on the team, b) ensure that the vision, mission, and goals of the group are clearly defined, c) negotiate relationships and sensitive issues, and d) secure external support for the team’s work.

Specific actions leaders may take towards these ends include

  1. ensuring that the team is balanced in membership and includes all those who have a vested interest in the outcome of the work, whether they are initially on board or not.
  2. taking the initiative to expand their substantive knowledge of evidence-based practice literature so that they are able to guide and support the team’s work. While leaders may appropriately defer to the expertise of team members, they do not neglect their need to be knowledgeable.
  3. developing a keen understanding of the interests at stake—those who support the team’s vision, mission, and goals, and those who don’t and why—and devising strategies to advocate for and garner cooperation and resources from outside of the team.
  • Interpersonal knowledge and skills: Good leaders have the ability to work effectively with others. Key skill areas are consensus building, conflict management, the ability to build trust, and the ability to “read” individuals’ needs and to manage their strengths. Specific actions include:
    1. communicating their personal belief in the power of the collaborative process;
    2. communicating their personal commitment to the team and its work;
    3. demonstrating respect for the team as a whole and for its individual members;
    4. modeling the standards for individual and group interaction and behavior;
    5. consistently following through with commitments;
    6. sharing control of the team process, decision-making, and work;
    7. identifying the unique contributions of each team member and drawing upon these routinely;
    8. seizing the opportunity of conflict to surface and resolve hidden disagreements;
    9. developing and encouraging leadership qualities in other team members; and
    10. sharing opportunities to demonstrate leadership.
  • Process knowledge and skills: Effective leaders are also skilled in collaborative team management. This involves the ability to organize the team’s work activities and discussions, design and manage meeting agendas, and define work processes that will accomplish the team’s goals. Leaders demonstrate these abilities by
    1. focusing attention on both the team’s substantive work and its work processes;
    2. helping the team be clear about its shared vision, mission, and goals, and routinely revisiting these to maintain focus;
    3. conducting productive, goal-oriented meetings that start and end on time—using every meeting as an opportunity for meaningful exchange;
    4. helping the group understand the work processes of the “EBDM roadmap” and the strategic planning steps;
    5. ensuring group discussions stay on track—guiding the group toward consensus and action after full and appropriate discussion of issues;
    6. ensuring regular participation from all group members—drawing out less vocal members, balancing the contributions of more vocal members, encouraging diversity of opinion, and following up immediately on members’ absences from meetings; and
    7. ensuring that the team develops, monitors, and updates a specific work plan that is tied to its EBDM vision, mission, goals, and systemwide logic model and to the scorecard that defines the jurisdiction’s overall harm reduction goals.

Purpose

The purpose of this Starter Kit is to help teams identify the individuals who are best suited to assume responsibility for leadership of the teams by describing the common qualities and skills of effective team leaders and by guiding teams through an open and candid discussion that will result in consensus-based determination about who will fill the leadership role.

Participants

All policy team members should be involved in this discussion and the resulting decision.

Instructions

  1. Review the qualities and skills of collaborative leaders contained in this document.
  2. Discuss each key point and determine whether the group agrees that each quality/skill is an important attribute of your team’s leader. Make a list of the qualities that the group agrees upon.
  3. From this list, identify the skills/qualities the group deems most important.
  4. Identify the individual(s) who best fit these attributes. If there are several candidates, discuss the pros of each and form a consensus decision.

Tips

The following are some effective meeting techniques for EBDM policy team chairs:

  • Stay neutral.
  • Listen actively.
  • Ask questions for clarification. (“I’m not sure I understood what you just said. Can you say more about that?”)
  • Ask for examples.
  • Paraphrase. (“Let me make sure I got that right.”)
  • Synthesize ideas.
  • Stay on track.
  • Give and receive feedback.
  • Summarize or ask for summary. (“Can someone summarize what we just agreed upon?”)
  • Ask for feelings and opinions. (“John, how do you react to this?”)
  • Encourage participation. (“Before we go on, I’d like to hear from Elena.”)
  • Test for consensus. (“Before we move, let me check to make sure we are all in agreement.”)
  • Do a quick survey. (“Let’s go around the room and have everyone indicate whether or not they support this.”)
  • Initiate action. (“How would you suggest we proceed on this?”)
  • Explore an idea in more detail. (“In what other ways could we approach this problem?”)
  • Suggest a break.
  • Suggest a procedure. (“Would it help if we developed criteria to decide this issue?”)
  • Stop the action. (“Let’s stop the discussion for a moment and go back to where we started.”)
  • Reflect. (“Darius, I get the impression you have concerns about this.”)
  • Be supportive. (“Let’s give Mary a chance to explain her position before we comment.”)
  • Question and test assumptions. (“Your proposal assumes that we are not doing it correctly now. Is that right?”)
  • Check targets. (“Are we asking the right questions?”)
  • Confront differences. (“Jim, I am getting the impression you do not like the way this is going. Tell us what you are thinking.”)

Additional Resources/Readings

Carter, M. M. (2006). The importance of collaborative leadership in achieving effective criminal justice outcomes (text rev.).
Retrieved from https://cepp.com/wp-content/uploads/2020/01/6-The-Importance-of-Collaborative-Leadership.pdf

Chrislip, D. D., & Larson, C. E. (1994). Collaborative leadership: How citizens and civic leaders can make a difference. San Francisco, CA: Jossey-Bass.

Larson, C. E., & LaFasto, F. M. J. (1989). Teamwork: What must go right/what can go wrong. Newbury Park, CA: Sage Publications, Inc.

[1] Carter, 2006.

[2] Larson & LaFasto, 1989.

[3] Carter, 2006.

1f: Setting Ground Rules

1f: Setting Ground Rules web_admin

Introduction

Establishing ground rules[1] for how a team will work together is an essential step that teams should take before embarking on any collaborative endeavor. Developing ground rules sets an important foundation for how the team is expected to work together in order to accomplish its goals and serves as a reminder of the values the team intends to uphold.

Purpose

  • To help your team consider ground rules that might help you work collaboratively and remain focused on your work
  • To assist you in reaching consensus on the ground rules that will guide your work together
Sample Ground Rules
  • One person speaks at a time.
  • No side conversations are permitted.
  • No cheap shots are allowed.
  • War stories are limited.
  • The group works toward consensus.
  • Parochial interests are left at home.
  • A problem solving orientation is adopted.
  • Group members hold each other accountable.

(CEPP, 2006)

Participants

All policy team members should be involved in the development of your team’s ground rules.

Instructions

  1. Begin by asking each team member to consider the expectations they have of one another throughout this process. Have them jot down these ideas on a sheet of paper.[2]
  2. Using a round-robin approach, ask each participant to share one item on their list. Note these on a flip chart. Go around as many times as necessary until all items are recorded.
  3. Combine and clarify the suggestions.
  4. Discuss whether the suggested ground rules are
    • comprehensive—are any ground rules missing?
    • agreeable—do all team members consent to abide by these rules?
  5. Arrive at a consensus on the set of ground rules you will use to govern your work.
  6. Document your ground rules so that you have a written record.
  7. Consider laminating and posting these rules in your regular team meeting room. Review them before each meeting. Team members will then have a constant reminder of the rules under which the team agreed to operate.
  8. Periodically review the ground rules to ensure that they are still relevant to the team’s work and agreeable to all members (especially if new team members are added). Add new rules as a team, if the need arises.

Tips

Ground Rules in Action

In Ramsey County, Minnesota, the team decided to display a laminated poster of their ground rules during all policy team meetings. They referred to the rules periodically to ensure that all members were clear about the expectations for conduct.

In Grant County, Indiana, ground rules were referred to during key strategic planning sessions, particularly when the team was preparing to make critical decisions about the change targets they wanted to pursue.

When setting ground rules, consider the following questions:

  • Have you set an expectation for how much each team member is expected to contribute?
    • Who will be responsible for work products, doing “homework” between meetings?
    • Will members of the team hold each other accountable for completing assignments?
    • Will members be expected to attend all policy meetings? When is missing a meeting acceptable?
    • Will the use of proxies or designees be allowed?
  • Is there an expectation about how group decisions will be made?
    • Will decisions be made by consensus or majority vote?
    • What level of evidence or support will be needed to make a decision?
  • Is there clarity around how members will conduct themselves during team meetings?
    • Are members asked to refrain from sharing personal war stories?
    • How will team members deal with confrontation?
    • How do team members define “respect”?
  • How will information sharing and confidentiality be addressed?
    • What kinds of information will be shared with non-team members?
    • When will information be kept confidential within the policy team?
    • How will team members handle questions from the press?

Some other tips for creating ground rules include the following:

  • Foster a culture of honesty. Suggest that it is as dishonest for group members to “put up with” something they don't agree with, or can't live with, as it is to speak untruthfully.
  • Practice listening. Every voice deserves to be heard, even if people don't initially agree with the point of view being expressed.
  • Recognize the need for full participation. This includes both encouraging team members not to hold back and valuing the opinions of others.[3]
  • Everyone needs to take a fair share of the group work. This does not mean that everyone has to do the same thing. It is best when members of the group agree on how tasks will be allocated.
  • Remember that everyone brings different strengths. The work of a group can be achieved efficiently when tasks are allocated according to the experience and expertise of each member.
  • Cultivate philanthropy. Group work sometimes requires people to make personal needs and wishes subordinate to the goal of the group. This is all the more valuable when other group members recognize that this is happening.

Example:
Grant County, Indiana, Policy Team Ground Rules

The following ground rules and operating norms have been established, and team members have agreed to hold each other accountable for adherence to these rules and norms:

  • Decision-maker attendance is expected.
  • Be on time and prepared for meetings.
  • All opinions are valued. Critique the opinion, not the person.
  • Be candid but respectful.
  • Team decisions will be made by consensus while preserving the autonomy of individual office holders and department heads.
  • Membership may evolve and sub-committees may include non-members.
  • New members will only be added by consensus.
  • Members will consider data and unintended consequences when making decisions.

Example:
Mesa County, Colorado, Policy Team Ground Rules

  1. Agree on what we agree on.
  2. No one has veto power.
  3. No one vehemently opposes a decision; everyone has to be able to live with it.
  4. We have each others’ backs.
  5. Leave personal agendas at the door.
  6. Trust issues
    1. No personal details are shared, no one is critical about emotions, and storytelling is not allowed.
    2. Be clear about the purpose of information shared.
    3. Share only information meant to promote the purpose of this group.

Additional Resources/Readings

Center for Effective Public Policy (CEPP). (2005). Teamwork exercise: Developing ground rules.
Retrieved from https://cepp.com/wp-content/uploads/2020/03/Teamwork-Exercise-Discussion-of-Roles-and-Responsibilities.pdf

Center for Effective Public Policy (CEPP). (2005). Ten tips for effective team participation.
Retrieved from https://cepp.com/wp-content/uploads/2020/03/Signs-of-Effective-Meetings.pdf

Center for Effective Public Policy (CEPP). (2006). Getting it right: Collaborative problem solving for criminal justice. Retrieved from https://nicic.gov/getting-it-right-collaborative-problem-solving-crimina...

 

[1] Language adapted from Center for Effective Public Policy (CEPP), 2005.

[2] For more information on team member expectations, see 1g: Building a Collaborative Climate.

[3] From John’s Hopkins University: https://pages.jh.edu/virtlab/misc/Group_Rules.htm

1g: Building a Collaborative Climate

1g: Building a Collaborative Climate web_admin

Introduction

One of the eight characteristics of highly effective teams is that they operate within the context of a “collaborative climate.”[1] A collaborative climate is built upon a foundation of trust among members. Trust promotes efficient communication and coordination and allows team members to stay solution-focused, thereby improving outcomes. Trust is earned, over time, when the following conditions are true:

  • Honesty: Members operate with integrity and are truthful.
  • Openness: Members are willing to share and be receptive to new ideas.
  • Consistency: Members are predictable in their behaviors and responses.
  • Respect: Members treat others with dignity and fairness.

Purpose

Too often groups do not take the time to assess their working relationships or to consider how best to foster these relationships to increase their likelihood of success. The purpose of this document is to encourage teams to pay as much attention to this aspect of their work as they do to understanding evidence-based practice and decision making, and to put as much care into building and sustaining the team as they do in planning and implementing change initiatives.

Participants

All policy team members should be actively engaged in the process of building and sustaining a collaborative climate. However, it might be useful to identify a few team members who are particularly attuned to these kinds of issues to make regular observations of the team’s collaborative climate and to elicit feedback from members on how well the team is working together.

Instructions

Identify “Temperature Takers” to Monitor the Collaborative Climate

While it is clearly the responsibility of every member of the team to contribute to and thereby build a collaborative climate, some individuals are keenly adept at assessing climate and at recognizing and seizing opportunities to improve teamwork, while others may not be as strong in this area. Simply stated, some people almost instinctively notice when things are going well and when the collaborative climate is faltering. This may be reason to specifically identify one or more team members to monitor the collaborative climate, to initiate dialogue about how well the team is working together, and to identify and facilitate activities that are likely to promote trust and support a positive, collaborative climate.

Utilize Tools and Strategies for Monitoring the Climate

There are a variety of methods of assessing the level of collaboration among team members.

  1. The survey Working Together: A Profile of Collaboration is one method of objectively assessing members’ perceptions of how well the team is working together.[2] The initial administration of this survey will establish a baseline of data around the team’s functioning. Re-administer the survey periodically (e.g., every six months) to identify improvements or new challenge areas.
  2. Periodically review the team’s ground rules to discuss how well members are adhering to them.
  3. Provide team members with a list of expectations of their participation.[3]
  4. Consider asking team members to periodically assess themselves with respect to these expectations, to report back to the group the strength they believe they most consistently bring to the team and the one they most need to work on, or to ask team members to provide one another feedback on some or all of the individual expectations on the list.
  5. Periodically include an item on the team agenda that provides for an open dialogue about the team’s effectiveness working together.
  6. Alternatively, reserve time at the end of a team meeting to pose a set of structured questions such as the following. These questions can be answered anonymously (each member can write their responses (yes/no) on a sheet of paper and these can be collected and tallied) or members can provide their responses in an open forum.
    1. Generally speaking, are our meetings productive and helping us work towards our vision/mission?
    2. Do we work from agendas that provide structure and purpose to our meetings?
    3. Do all members actively participate in our meetings?
    4. Is there a sufficient level of trust among group members to allow for candid discussion?
    5. Are all members respectful, even when their perspectives or opinions differ?
    6. Do members feel “heard” when they share their views?
    7. Are all members equally committed to the vision/mission?
    8. Are all members equally contributing to the work of the team?
    9. Do members believe we can accomplish something together that we could not accomplish separately?
    10. Do members feel that the team is “in this together,” even when things become difficult?
    11. What can we be doing to further strengthen our team?
    12. Solicit the assistance of an outside, neutral individual to interview team members about how well the team is functioning, then report that information back to the group as a whole for consideration and action planning, where needed.

Additional Resources/Readings

Carter, M. (2005). Collaboration: A training curriculum to enhance the effectiveness of criminal justice teams. Retrieved from https://cepp.com/wp-content/uploads/2020/01/1-Collaboration-Curriculum.pdf

CEPP (2005). Collaborative Justice. Website. https://cepp.com/collaborative-justice/

Chrislip, D. D., & Larson, C. E. (1994). Collaborative leadership: How citizens and civic leaders can make a difference. San Francisco, CA: Jossey-Bass.

Larson, C. E., & LaFasto, F. M. J. (1989). Teamwork: What must go right/what can go wrong. Newbury Park, CA: Sage Publications.

Appendix:
Ten Tips for Effective Team Participation[4]

One of the hallmarks of a collaborative team is that every team member is equally important and the contribution of each member is necessary to the success of the team. These are some of the ways that individual team members can contribute to effective collaboration:

  1. Show up on time, turn your cell phone or pager to vibrate, and respond only to emergencies. Give the group your full attention.
  2. Show up consistently. When team members make an effort to be present, others understand that as a demonstration of commitment and are therefore more willing to make the commitment themselves.
  3. Adhere to whatever ground rules or standards are established by the group. If none are articulated, set your own standards high. Better yet, suggest that the group establish ground rules and standards.
  4. Be prepared and participate. If there is a group assignment (such as reading a report) or if you agree to perform a specific task for the group, do it. Take advantage of process activities to get to know other members of the team.
  5. Be aware of your body language. Don’t let your posture or facial expressions communicate a lack of interest or lack of respect for another team member’s contributions. Instead, listen actively and make an effort to understand what others are trying to say before responding.
  6. Stay focused on the vision and mission of the team. Leave personal agendas aside.
  7. Practice good conflict resolution skills. Conflict, dissent, and disagreement are essential to effective problem solving. Look for ways to use the information that comes out of conflict to improve the work you are doing together.
  8. Look for opportunities to exercise leadership.
  9. Challenge yourself. If you are typically one who dominates conversations, sit back and listen. If you are typically reluctant to share your ideas, try to speak up.
  10. Be willing to be held accountable. And be willing to hold others accountable. Holding each other accountable is a sign of respect and it demonstrates that group members have high expectations of each other.

These basic practices are fundamental to all kinds of group work, but are especially important when working in collaboration with others. Successful collaboration requires trust, and these practices help to build a climate of trust so that effective collaborative work can take place.

[1] The eight characteristics of highly effective teams identified by Larson and LaFasto (1989) are: a clear and elevating goal, a results-driven structure, competent team members, a unified commitment, a collaborative climate, standards of excellence, external support and recognition, and principled leadership.

[2] For more on the Working Together survey, see 1b: Administering a Collaboration Survey.

[3] For some ideas, see the Appendix.

[4] From https://cepp.com/wp-content/uploads/2020/03/Signs-of-Effective-Meetings.pdf.

1h: Establishing a Decision Making Process

1h: Establishing a Decision Making Process web_admin

Introduction

Agreeing to a decision making process is one of the first steps of a successful policy team. Decisions may be made by consensus, majority vote, or some other way. While consensus decision making is often the ideal method, it is easier said than done.

Consider that “consensus is not compromise, nor abdication, nor winning so that others lose. Rather, consensus is an agreement with others that may not be an ideal solution, but is a result that all can ‘live with.’”[1]

Purpose

To facilitate a dialogue among team members that will result in a process for team decision making.

Participants

All policy team members should be involved in establishing a decision making process for the team.

Instructions

  1. Have a discussion about how the team will make decisions before or while your team decides on its ground rules and operating norms.[2]
  2. Begin by asking your team members to think about how they would like group decisions to be made. Discuss the following questions:
    1. Will team members agree that decisions should be made by consensus (rather than by the group chair or by majority vote)?
    2. Will all—or only certain types of—decisions be made in this way?
    3. If consensus cannot be reached, what process will the team follow? (See below for a way to make non-consensus decisions.)
    4. What level of evidence or support will the team need to make a decision?
    5. What will happen when team members are absent from a meeting? Will decisions be made or held until all members are present?
  3. Once agreement is made on the issues above, designate someone to record the agreed-upon decision making guidelines. These should be included in your policy team’s charter, along with your team’s ground rules.

Possible Process for Making Non-Consensus Decisions

When consensus is not possible, your team may agree to a voting process. For example, policy team members might vote using their thumbs (or some other signal the team agrees to) to indicate

  • “I fully and completely support this decision” (thumbs up);
  • “I do not fully and completely support this decision (i.e., there are some things about it I would prefer were different) but I can live with it” (thumbs horizontal); or
  • “I cannot support this decision” (thumbs down).

The team can move forward with a decision as long as there are no “thumbs down’s.” That is, everyone on the team must completely support the decision or agree to “live with it” in order for the decision to be made.

In cases where one or more team member cannot support the decision (thumbs down), engage those team members in a dialogue about what the team would have to do in order to create a situation where everyone could live with the decision.

Tips

Consider the following tips when working with your team to come to agreement or to make a decision:

  • There may be some instances where different decision making methods are used. Ensure that team members are aware of the decision making “mode” prior to entering into discussions about decisions.
  • In order to arrive more quickly at consensus, use facts and information—not opinions or beliefs—as the basis of conversation. An alternative approach is to provide an allotted amount of time for team members to express opinions and beliefs; when time is up, restrict the discussion to factual information.
  • Recognize the difference between disagreement and conflict. Disagreements are healthy; they force the group to consider different options and select the best course of action. Disagreements turn to conflict when team members get emotionally attached to issues or positions, sometimes resulting in personal attacks on other team members. Encourage your team to “stay professional” by keeping their comments and criticisms focused on work items and processes, not the persons involved.[3]

Example:
Eau Claire County, Wisconsin, Decision Making Guidelines

The policy team has agreed on the following guidelines for decision making:

  • All members have equal status for the purposes of input and decision making.
  • Team members are allowed to send a designee if they are unable to attend a scheduled meeting. Communication of this must be made in writing to the chair prior to the scheduled meeting date.
  • Decisions will be reached by consensus whenever possible and team members will use the consensus decision rule:
    • I am at least 70% comfortable with or in favor of the decision, and I will be 100% committed to it.
    • I am 60% comfortable with the decision, but still have reservations and need more information. I am responsible for finding the information I need.
    • I am less than 50% comfortable with the decision. I am responsible for presenting an alternative solution.
  • Even though the EBDM policy team has no legal authority to force any changes in the criminal justice system at any of its points, justice stakeholders impliedly agree and consent that they will give great deference to a consensus decision of the entire EBDM policy team. Although individual justice stakeholders still retain their statutory and constitutional prerogative to exercise their discretion, they impliedly agree that if they decline to consider or implement a recommended practice or procedure, they will articulate clear and specific reasons explaining why, in good conscience, they will not consider or implement the consensus decision.

Example:
Milwaukee County, Wisconsin, Decision Making Guidelines

All members have equal status for the purposes of input and decision making.

Whenever possible, decisions will be reached by consensus, whether members are present at a scheduled meeting or not. If consensus cannot be reached, decisions shall be made as follows:

  • A proponent of the decision shall make a motion which shall be stated in a meeting agenda issued at least three business days in advance of a meeting.
  • The motion may be put to a vote at the meeting if a quorum is present; a quorum shall consist of at least eight members of the policy team.
  • A motion shall carry if a majority of the policy team members present at the meeting vote in favor.

Decisions of the policy team are subject to legal limitations on the authority of any agency with a representative on the policy team. However, a decision of the policy team binds the representative to make a good faith effort to use their authority to implement the decision or persuade their agency to authorize the necessary action or seek the necessary authority.

Additional Resources/Readings

CEPP. (2006). The role of facilitators and staff in supporting collaborative teams.
Retrieved from https://cepp.com/wp-content/uploads/2020/03/The-Role-of-Facilitators-and-Staff-in-Supporting-Collaborative-Teams.pdf.

CEPP. (2010). Coaching packet: Establishing a rational planning process.
Retrieved from https://cepp.com/wp-content/uploads/2015/12/Establishing-a-Rational-Planning-Process.pdf

Woodward, B. (1993). Establishing and maintaining the policy team. In National Institute of Corrections, The intermediate sanctions handbook: Experiences and tools for policymakers (pp. 27–34). Retrieved from

[1] National Institute of Corrections (NIC), 1993.

[2] See 1f: Setting Ground Rules.

[3] SMART Technologies, 2004.

1i: Developing a Mission for Your Policy Team

1i: Developing a Mission for Your Policy Team web_admin

Introduction

If the vision statement is a picture of the desired future, the team’s mission statement[1] describes what your team will do. The mission will define the team’s work together in a realistic and meaningful way; it will translate your overarching vision of the future into a shorter-term set of activities that can be more readily accomplished while bringing your community closer to the ideal you envision.

The mission should be concrete, represent tangible targets of change activity, and be connected firmly to achieving some part of the larger vision. When developing your mission, it is important to be clear about what you want to achieve through your work (e.g., what problems or issues your team wants to overcome). This will make it easier when your team begins to develop specific goals and objectives for achieving your mission.

Purpose

To facilitate a dialogue among team members that will result in a statement reflecting the mission for the justice system in your community

Participants

All policy team members should be involved in the development of your mission statement.

Instructions

The following steps will guide your team through the development of your mission statement:

  1. Ask each team member to consider the following questions:
    • What function does the team perform?
    • For whom does the team perform these functions?
    • Why do you exist as a team?

    Provide team members a few minutes to jot down their ideas about these questions. When they are finished, go around the room and record each person’s first response on a flip chart. Go around the room again and note each person’s second response. Continue this process until all ideas are recorded.

  2. Review the keywords generated. Discuss each one and ensure that its meaning is clear. Eliminate duplications. Develop a statement or set of statements that answers the questions and that reflects the consensus of the team.
  3. If multiple statements are produced, prioritize these by asking each member to rank order the statements; then tally the “votes” for each one.
  4. Follow the same process to answer the following questions: What must the team do to accomplish its vision? What are the team’s activities?
  5. For each question, develop a single statement (this may be one sentence or a full paragraph) that synthesizes the prioritized ideas into a mission statement. This process may take some time; you may not be able to complete this work in one session or even as a full team. You may decide it best to have one person or a subcommittee work on developing these statements between team meetings.

Tips

  • Try to make your mission clear enough to explain your work and purpose to a third party. Ask non-team members to read the mission and see if they understand what you are trying to accomplish.
  • Know that it is okay to let go of some key words or ideas that were shared early in the development process.
  • Rather than get overwhelmed by all the work that your team needs to do to accomplish its vision, focus on the work that your team needs to get done in the immediate future.
  • Revisit your mission periodically to determine whether the targets of change are still accurate or need to be revised.

Example:
Charlottesville–Albemarle County, Virginia, Mission Statement

“Working together for a safer community, one person at a time.”

The agencies in the Charlottesville-Albemarle Criminal Justice System seek to achieve justice and make communities safer by working closely together, applying the best known research to policies and practices, listening to those affected by crime, and recognizing that every interaction can lead to improved outcomes.

Additional Resources/Readings

Center for Effective Public Policy (CEPP). (2005). Collaboration: A training curriculum to enhance the effectiveness of criminal justice teams.
Retrieved from https://cepp.com/wp-content/uploads/2020/03/Collaboration-A-Training-Curriculum-to-Enhance-the-Effectiveness-of-Criminal-Justice-Teams-2005.pdf

Center for Effective Public Policy (CEPP). (2006). Getting it right: Collaborative problem solving for criminal justice. Retrieved from https://nicic.gov/getting-it-right-collaborative-problem-solving-crimina...

Center for Sex Offender Management (CSOM). (2007). Enhancing the management of adult and juvenile sex offenders: A handbook for policymakers and practitioners.
Retrieved from https://cepp.com/wp-content/uploads/2020/01/6-CSOM-Handbook.pdf

 

[1] Language adapted from CEPP, 2006 and CSOM, 2007.

1j: Creating a Charter for Your Policy Team

1j: Creating a Charter for Your Policy Team web_admin

Introduction

Developing a written charter is a critical step towards achieving an effective team. It facilitates dialogue among team members about the ways that a group of individuals can and should operate, and it documents these agreements. Doing so ensures that these agreements are understood by all, facilitates the orientation of new team members, and supports the accountability of individual team members, as well as the team as a whole.

The policy team charter should define the purpose of the team, outline its vision and mission, and explain how it will work together. The charter is important as it

  • defines common objectives and the shared intent of the team;
  • keeps the team focused and helps the team determine when it goes off track (i.e., when activities go beyond the scope of the team);
  • defines team boundaries and agreements; and
  • helps team members determine when to raise an issue.[1]

The charter serves as a roadmap for the team’s work together and should be referred to when “times get tough.” It should be developed jointly, in principle if not in practice, and be agreed upon by all team members. Signatures on the charter reflect members’ commitment to adhere to the charter. Although a charter is similar to a memorandum of understanding (or to an agreement), its distinction is that it contains more detailed information that a MOU/A typically does.

While charters may differ from team to team, generally the following elements are included:

  • the clear and elevating goal (vision) the team is striving to achieve;
  • the specific mission of the team during the current phase of work;
  • the goals and objectives the team has agreed to take on to achieve the mission;
  • team membership and agreed-upon roles and responsibilities;
  • the team’s agreed-upon operating norms/ground rules;
  • the team’s agreements around making decisions; and
  • the resources and support available to the team as it conducts its work.[2]

Purpose

To create a charter that will map the course for the policy team

Participants

While all policy team members should ultimately agree to the content of the charter and indicate their agreement by signing the document, a few team members may draft the charter based on the work of the team (i.e., they may take team products such as the vision, mission, ground rules, and list of roles and responsibilities and combine them to form a formal charter).

As the team’s work evolves, the charter should be updated. For instance, an initial charter is unlikely to contain a detailed work plan, team membership might change, or operating norms or team member roles might be expanded over time. In these cases, an amended charter would include an updated work plan, team membership list, set of operating norms, or clarification of roles.

Instructions

As your team determines what it wants to achieve and makes agreements about how it will work together, complete and/or update the appropriate sections in your team’s charter. Consider the example charter in this kit.

  1. In the section entitled “Team Membership,” explain how team members were chosen and list all the members of your team. You may also wish to state your team’s agreement on whether or not the membership may change over time, as well as the use (or not) of proxies or designees of team members. If you want to include a full list of contact information, this can be attached to the charter as a separate document.
  2. Once team membership has been established, one of the first activities of the team should be to discuss the reason for its creation. The first section of a charter should include an introduction to your team. It should state the “context,” that is, the problem that is being addressed and the results that you expect to achieve. Once your team has developed a shared vision for your work together, fill in your team’s vision statement.[3]
  3. Following a discussion with the team on its mission (i.e., the activities your team will engage in to meet its vision), include the mission statement in your team’s charter.[4]
  4. As your team begins to clarify how it will work together, include the following in the charter:
    1. your team’s agreement on how often and for how long the policy team will meet;
    2. the ground rules and operating norms on which your team has agreed;[5]
    3. under “Team Roles and Responsibilities,” any special roles of team members (e.g., chair, facilitator) and expectations for all team members (e.g., all team members will attend every meeting)[6]; and
    4. under “Decision Making Guidelines,” agreements made by your team regarding your decision making process.[7]
  5. In the section entitled “Policy Team Activities,” refer to your team’s work plan. When your team is satisfied with the objectives contained in the work plan, include it as an attachment to your charter.[8] Update this attachment as your team revises its work plan.
  6. Once the team has reviewed and approved its contents, be sure to have all team members sign the charter to indicate their agreement to adhere to it.

Example:
Yamhill County, Oregon, EBDM Policy Team Charter

Policy Team Vision: We envision a safer Yamhill County community, where professionals work together utilizing data, research, and evidence-based practices in the criminal justice system. Yamhill County will experience enhanced public safety, a reduction in the number of victims, greater offender accountability, and a reduced threat of harm through appropriate application of proven practices at all phases of the criminal justice process.

Policy Team Mission: Our mission is to collaboratively develop, by July 1, 2011, a strategic plan to implement proven, cost-effective system improvements.

Policy Team Values:

  • Safe and healthy communities
  • Public safety
  • Fairness
  • Justice, due process, and rule of law
  • Respect for the rights, needs, and concerns of people who are victims of crime
  • Respect for the rights of people accused of crimes
  • Enhanced collaboration
  • Cost efficient and effective practices
  • Competent and dedicated criminal justice professionals

Policy Team Activities: To ensure accomplishment of the policy team’s mission, an action plan has been developed, defining the objectives and action steps the policy team will undertake. The action plan will be reviewed at each policy team meeting to ensure that tasks are completed as anticipated and updated as needed. The action plan is attached as Appendix 1.

Meeting Frequency and Duration: To accomplish its work, the policy team will meet on a regular schedule. The first meeting of the policy team will be 10/14/10. Policy team meetings will be held twice per month through June, 2011. If accomplishment of the action plan requires more or less time, the policy team will readjust this schedule as necessary.

Ground Rules and Operating Norms: The following ground rules and operating norms have been established, and team members have agreed to hold each other accountable for adherence to these rules and norms:

  • Be courteous.
    • Don’t interrupt.
    • Don’t talk over each other.
    • Don’t check emails during meetings.
    • Take important phone calls outside of meetings.
  • Be candid, honest, and open.
  • Strive to be goal-oriented.
  • Bring your “best self” to meetings.
  • Be on time, participate, complete assigned “homework,” attend all meetings, and stay for the duration of all meetings.
  • Start/end meetings on time.
  • Agendas will be sent out in advance by email.
    • Each member may add items to the agenda.
  • At the end of each meeting, members will review how the meeting went and how well its members adhered to the ground rules and operating rules.
  • Judge Collins will facilitate all meetings.
  • As the local coordinator, Director Ted Smietana will schedule meetings and handle all necessary logistics in between meetings.
  • The policy team will agree to the type of information/messages that will be shared with the policy team member agencies to ensure consistency.
  • Team members will be transparent with their agencies regarding the Initiative.

Decision Making Guidelines: The policy team has agreed on the following guidelines for decision making:

  • All members have equal status for the purposes of input and decision making.
  • No proxy voting will be permitted.
  • Unanimity is preferred; otherwise, decisions will be reached by consensus whenever possible.
  • When consensus cannot be achieved, we will respect divergent views, seek common ground, seek technical guidance, and continue to strive for consensus.
  • Team members will remain cognizant of the impact of our decisions on individual agencies.
  • Team members have independence to make decisions for their own agency, while still adhering to the collective interest of the policy team.
  • Each meeting will be recorded and minutes will be shared with team members.

Team Membership: Team members have been selected based upon the desire to have all local and relevant criminal justice agencies involved and represented.

  • At least one committee member represents each of the following disciplines/interest areas:
    • Law Enforcement—Sheriff Crabtree, McMinnville Chief Ron Noble, and Newberg Chief Brian Casey
    • Jail Administration—Sheriff Crabtree
    • Judiciary—Presiding Judge John L. Collins
    • County Administration—Commissioner Mary Stern
    • Correctional Treatment—HHS Director Silas Halloran-Steiner
    • Prosecution—District Attorney Brad Berry
    • Defense—Attorney Carol Fredrick
    • Victim Services—Debra Bridges
    • Community Corrections—Director Ted Smietana
  • On an ongoing basis, the team will reconsider its composition and seek to add members to the team as additional expertise is required. Because the work of the team will be intensive and require the full engagement of all policy team members, proxies and designees will not be allowed.

Team Roles and Responsibilities: The following roles have been established to support the effective work of the policy team:

  • Chair: The role of Chair will be filled by Judge Collins. The Chair’s role is to facilitate and oversee each policy team meeting and ensure that the team’s mission and purpose is being fulfilled.
  • Coordinator: The role of Coordinator will be filled by Director Ted Smietana. The Coordinator’s role is to organize meetings and activities and to manage all logistics for the team.
  • Recorder: Kathe Bonfield, of Community Corrections, will record each meeting and produce detailed meeting minutes for all team members.
  • All team members have the following responsibilities:
    • Each team member will serve as a liaison to the constituency/agency/interest area they are representing, carrying key discussion points to these individuals and reporting feedback to the team.
    • Each team member will serve as a liaison to the community, communicating key decisions and operational changes, and reporting feedback to the team.
  • In addition, the following unique responsibilities have been agreed upon:
    • Mimi Carter, Principal and EBDM Initiative Yamhill County Site Coordinator, Center for Effective Public Policy: Ms. Carter will act as a meeting co-facilitator, provide technical assistance, and coach the policy team through the Initiative process so that objectives will be reached. She will also broker any additional technical assistance resources as needed, advocate for the team, and critique the team to assist in the achievement of all goals.
    • Carl Gordon, Ph.D. Local Citizen and former Bureau of Prisons Chief Psychologist: Dr. Gordon will provide additional local technical assistance and team support.

Additional Resources/Readings

DRM Associates. (2001). Team charter.

Mind Tools. (2010). Team charters: Getting your teams off to a great start. Retrieved from https://www.mindtools.com/pages/article/newTMM_95.htm

 

[1] DRM Associates, 2001.

[2] Mind Tools, 2010.

[3] See 1c: Creating a Vision for Your Policy Team.

[4] See 1i: Developing a Mission for Your Policy Team.

[5] For more information, see 1f: Setting Ground Rules.

[6] For more information, see 1k: Establishing Clear Roles and Responsibilities.

[7] See 1f: Setting Ground Rules.

[8] See 1l: Developing an Action Plan for Your Policy Team’s Work.

1k: Establishing Clear Roles and Responsibilities

1k: Establishing Clear Roles and Responsibilities web_admin

Introduction[1]

Teams function most efficiently when members share a common understanding of each others’ roles and responsibilities. Indeed, one of the reasons why teams fail is a lack of clarity among team members regarding their respective roles, responsibilities, and the expectations they hold of one another when working together to accomplish their vision, mission, goals, and objectives. When roles and responsibilities are clearly defined, team members are more productive. There is less duplication of effort; less confusion, disappointment, and frustration; and greater productivity. When roles and responsibilities are clearly defined, team members look beyond their own individual positions and learn to understand, respect, and value the unique contributions of one another, and they recognize that the overall success of the team is a function of shared responsibility and ownership.

Purpose

  • To support team members in articulating their specific roles and responsibilities with respect to the policy team and its work;
  • To encourage team members to articulate the expectations they hold of one another; and
  • To identify the leadership attributes and characteristics team members feel are needed to support the team’s performance.

Participants

All policy team members should be involved in the development of the team members’ roles and responsibilities.

Instructions

  1. Working individually, have team members write three brief statements:
    • a “job description” of what they perceive as their EBDM policy team duties and responsibilities;
    • what they expect other members to contribute to the team’s work; and
    • the type of leadership attributes and characteristics they feel are needed to support the team’s performance.
  2. When team members are finished writing their individual statements, have each member read their job descriptions and express their expectations of other team members.
  3. As team members report on their job descriptions and expectations of each other, make two lists on a flip chart:
    • a list of the roles and responsibilities expressed by team members, making note of those expressed by more than one member; and
    • a list of members’ expectations of other team members, again making note of those expressed by more than one member.
  4. As a team, review the final lists. Compare and contrast each member’s ideas about their roles and responsibilities and what they expect the other team members to contribute. Discuss any differences in opinion or observations.
  5. Come to an understanding and consensus on policy team members’ key roles and contributions to the team’s operation.
  6. Exchange comments about the role of leadership on the team, including who assumes or is expected to assume leadership roles (including the EBDM policy team chair, local coordinator, and others) and what is expected of those assuming these roles. Add these to the list as appropriate.
  7. Once the team comes to consensus on the final list of roles, responsibilities, and expectations, record these in the team’s charter.

Some common responsibilities of team members include

  • participating actively in all meetings (responsibility);
  • serving as note taker and preparing meeting records (role);
  • completing assignments between meetings (responsibility);
  • serving as spokesperson for the team with county administration (role); and
  • serving as liaison between agency staff and the team (responsibility).[2]

Some common responsibilities of the local coordinator may include

  • arranging appropriate space for team meetings, as well as equipment, food, and beverages;
  • preparing materials for team meetings;
  • drafting and sending correspondence;
  • taking meeting minutes and creating accurate meeting records;
  • collecting and synthesizing data and information to support members’ work; and
  • preparing reports and other documents as appropriate.

Some common responsibilities of the team’s chair may include

  • chairing team meetings, facilitating discussions, and assuring the team stays focused on stated goals;
  • working closely with the local Initiative coordinator and team leadership to prepare for and follow up on team meetings, and overseeing all aspects of the team’s work ;
  • regularly communicating with team members about the status and progress of the team’s work; and
  • assigning team members to work groups to address specific issues.

Tips

  • As team members consider their “job descriptions,” they should not forget to account for their professional background and experience and what specific role(s) they can perform. Also, members might consider what interpersonal skills, interests, and experiences they can contribute to the team’s work.
  • Consider the team’s goals and specific work activities over the coming months. Check to be sure that the list of team roles and responsibilities, expertise, and contributions align with expected work activities. If gaps are noted, refer to 1d: Conducting a Stakeholder Analysis.
  • Be as specific as possible to define roles and responsibilities and agree on what should be listed in the team’s charter.

Example:
Eau Claire County, Wisconsin, Established Team Member Roles and Responsibilities

The following roles have been established to support the effective work of the policy team:

  • The role of the EBDM policy team chair will be to
    • chair the EBDM policy team meetings;
    • preside over all meetings;
    • work with the local coordinator;
    • act as the lead facilitator and team builder; and
    • act as the chief spokesperson, be the public face of the team, and represent the team’s interests.
  • The role of the local coordinator will be to
    • adhere to the EBDM process as established by the EBDM Framework, the National Institute of Corrections, and the EBDM policy team;
    • schedule meetings and send out agendas;
    • oversee the implementation of all EBDM planning activities;
    • serve as the primary contact for the project with members, the chair, and others
    • maintain timelines established by the policy team;
    • develop and oversee the action plan; and
    • coordinate all EBDM policy team activities and work group activities as assigned.
  • All policy team members will
    • serve as a liaison to the constituency/agency/interest area they are representing, carry key discussion points to these individuals, and report feedback to the policy team;
    • support the requests of the policy team and the local coordinator in an efficient manner; and
    • abide by consensus authority rule.

Additional Resources/Readings

CEPP. (2005). Teamwork exercise: Discussion of roles and responsibilities.
Retrieved from https://cepp.com/wp-content/uploads/2020/03/Teamwork-Exercise-Discussion-of-Roles-and-Responsibilities.pdf

Scholtes, P. R., Joiner, B. L., & Streibel, B. J. (2003). The team handbook (3rd ed.). Madison, WI: Oriel Inc.

 

[1] Language adapted from CEPP, 2005.

[2] Scholtes, Joiner, & Streibel, 2003.

1l: Developing an Action Plan for the Policy Team’s Work

1l: Developing an Action Plan for the Policy Team’s Work web_admin

Introduction

After developing a vision and mission for your policy team, the team should determine the specific goals and objectives necessary to achieve each target of change in your mission. To recap:

  • Your vision is a statement about the clear and elevating goal your team is striving to achieve—a statement of your preferred future.
  • Your mission is a statement that describes the work your team intends to undertake during a specific period of time to make progress toward achieving your vision.
  • Your goals represent the accomplishments that must be realized to achieve your mission.
  • Your objectives are the steps, or tasks, that must be completed to meet your goals.

While the team should develop an action plan from the outset and include it in the charter, the plan will undoubtedly change as the team makes progress and learns more. As needed, update the work plan to keep it current, to pace work, and to gauge progress.

Purpose

To create an action plan that will outline the specific goals and objectives the team will undertake, the sequence and timing of activities, and the persons responsible for those tasks

Participants

All policy team members should be involved in the development of your action plan. (Individuals or groups of individuals within your policy team or outside of your policy team may be involved in achieving the objectives in your action plan.)

Instructions

The activities identified in this Starter Kit are intended to lay the groundwork for the implementation of the EBDM Framework.[1]

  1. Working together, refer to the activities listed on Getting from Here to There: The Roadmap for Preparing to Implement the EBDM Framework. Discuss each activity and ensure that all team members understand its purpose.
  2. Determine whether some of this work has been done before, in whole or in part.[2]
  3. Determine whether the goals listed in the Roadmap are the right goals for your policy team to achieve its mission, whether the goals should be changed somewhat, or whether new goals should be added.
  4. Agree upon the appropriate set of goals for your team’s work plan.
  5. Once this list is complete, place each goal in an order to reflect the sequencing of how these goals should be tackled.
  6. Next, consider each goal and identify the specific action steps (or objectives) that will be taken to achieve it. Determine the person(s) responsible for the task and a deadline for completion.
  7. Designate a team member who will create a written action plan for the decisions made during the action planning session. Refer to the action plan template in the Appendix as a guide for developing a formal, written action plan.

Tips

  • It may not be possible to forecast very specific steps for activities that will be accomplished in later months; try to develop in more detail the more immediate tasks (i.e., 3–4 months) that need to be accomplished.
  • Teams should revisit their action plan regularly to make revisions and adjustments as needed.

Example:
Ramsey County, Minnesota, Action Plan for the EBDM Planning Process (Abbreviated version of whole document)

 

Objective 1: Build a genuine, collaborative policy team

Implementation Steps (How)

Progress

By Whom &

With Whom

Target Date for Completion

Date Completed

Convene the full policy team.

First meeting held Nov. 8. Regular monthly meetings set up through Aug.

Local coordinator

10/15

11/8 & ongoing

Name a chair for the policy team.

Reviewed at Nov. 8 meeting.

Policy team

11/8

11/8

Draft and finalize a charter, which includes a vision, a mission, ground rules, and team responsibilities, for the policy team.

Reviewed at Nov. 8 and 15 meetings. Reviewed again at Dec. 13 meeting and unanimously approved.

Policy team

Dec

12/13

Train new members of the policy team.

Setting up training for Mar.

Training subcommittee & policy team

Mar.

 

At the end of each meeting, review how the meeting went and how well members adhered to ground rules and operating rules.

An evaluation will be administered and reviewed at each meeting.

Local coordinator

Start Nov. & monthly

11/8 & ongoing

Objective 2: Build individual agencies that are collaborative and in a state of readiness for change

Implementation Steps (How)

Progress

By Whom &

With Whom

Target Date for Completion

Date Completed

Provide regular updates to judges and to the Board of Commissioners.

Visit judges one-on-one to update them on the Initiative; provide monthly updates to the Board of Commissioners; hold Board workshops

Chair

Nov. & ongoing

Nov. & ongoing

 

 

Objective 3: Understand and have the capacity to implement evidence-based practices

Implementation Steps (How)

Progress

By Whom &

With Whom

Target Date for Completion

Date Completed

Plan a workshop for leadership and line staff in participating agencies.

Dec. 2010: Initial planning was begun. Target date for workshop is Mar. 9. Education/Building Skills subcommittee was formed. Workshop to include an awareness survey.

Policy team and Education/Building Skills subcommittee

Mar.

 

Form an Education/Building Skills subcommittee to develop skills in each specialty area to ensure that staff in each organization has a base level of competency.

Work with managers and then with staff to build skills. Training subcommittee was formed. Discuss how to gauge knowledge level, skill level, and cultural change issues.

Education/Building Skills subcommittee

Dec. & ongoing

Dec. & ongoing

 

 

Objective 4: Establish performance measurements/outcomes/system scorecard

Implementation Steps (How)

Progress

By Whom &

With Whom

Target Date for Completion

Date Completed

Agree on key definitions (e.g., “recidivism,” “probation violation”).

This work will start during the mapping process.

Policy team

Feb. & Mar.

 

 

Objective 5: Engage/gain support of our communities

Implementation Steps (How)

Progress

By Whom &

With Whom

Target Date for Completion

Date Completed

Name stakeholders/communities to be kept informed.

Discussed at Nov. policy team meeting.

Policy team

Nov. & ongoing

Nov. & ongoing

Develop a plan for communicating with the public.

Discuss how to address livability issues; how to talk about EBP. What does the community already know? Consider using the information from the public opinion survey.

 

Consider television, newspaper releases, public service announcements, town meetings, etc.

Chair

Jan. & ongoing

Jan. & ongoing

 

Additional Resources/Readings

CEPP. (2005). Collaboration: A training curriculum to enhance the effectiveness of criminal justice teams. Retrieved from https://cepp.com/wp-content/uploads/2020/03/Collaboration-A-Training-Curriculum-to-Enhance-the-Effectiveness-of-Criminal-Justice-Teams-2005.pdf

CEPP. (2006). Getting it right: collaborative problem solving for criminal justice. Retrieved from https://nicic.gov/getting-it-right-collaborative-problem-solving-crimina...

CSOM. (2007). Enhancing the management of adult and juvenile sex offenders: A handbook for policymakers and practitioners. Retrieved from https://cepp.com/wp-content/uploads/2020/01/6-CSOM-Handbook.pdf

Appendix:
Action Plan Template

GOAL #1 DATE TO BE COMPLETED BY WHOM WITH WHOM

GOAL #1

  • Specific task
  • Specific task
  • Specific task
  • Specific task
      

GOAL #2

  • Specific task
  • Specific task
  • Specific task
  • Specific task
      

GOAL #3

  • Specific task
  • Specific task
  • Specific task
  • Specific task
      

GOAL #4

  • Specific task
  • Specific task
  • Specific task
  • Specific task
      

ETC.

  • Etc.
  • Etc.
  • Etc.
      

 

[1] Please note that your policy team may or may not complete these activities in the same sequence as other sites.

[2] Work that has been completed before does not need to be redone unless it is dated, incomplete, or not fully encompassing of the EBDM initiative, or if all team members have not been a part of it. In these cases, adjustments may need to be made. Use the pre-existing work as a starting point, rather than starting from scratch.

1m: Managing the Policy Team: The Local Coordinator

1m: Managing the Policy Team: The Local Coordinator web_admin

Introduction

Since policymakers are busy people, designating an individual to manage and facilitate the collaborative policy team is crucial to its success. For the planning phase of this initiative, this responsibility may fall to a designated local coordinator, who may at times play multiple roles, including facilitator, staff support person, or other roles needed by the policy team. For example, the designated local coordinator might be responsible (either directly or indirectly) for

  • developing long- and short-term work plans, and determining how each meeting will fulfill them;
  • making sure that meetings occur when needed;
  • preparing materials for team meetings, most importantly, developing meeting agendas;
  • taking meeting minutes and creating accurate and helpful meeting records;
  • providing follow-up to meetings;
  • collecting and synthesizing data and information to support members’ work;
  • providing information (e.g., data, concerns about progress) to policy team members;
  • developing detailed proposals for action (based on team guidance); and
  • preparing reports and other documents.[1]

Purpose

To provide local coordinators with some tips and resources that will assist them in managing their policy teams[2]

Participants

This document is geared toward local coordinators. Local coordinators may share this information with others as appropriate.

Instructions

Role of the Local Coordinator

In Charlottesville-Albemarle County, Virginia, a steering committee was formed to guide the work of the 16-person policy team representing both city and county interests.

Due to the team’s size, it was agreed that the team’s local coordinator would serve as the “information gatekeeper” and main liaison with all parties involved in the work. The local coordinator served on both the steering committee and the policy team and was responsible for:

  • taking the lead on administrative tasks;
  • taking meeting minutes;
  • preparing materials for meetings;
  • ensuring tasks were completed fully and on time;
  • coordinating dates for meetings and timelines;
  • supporting the policy team’s work by collecting and organizing data where applicable; and
  • serving as a communication liaison with all parties.

Below are some suggestions for activities you might engage in during your first few meetings to set up your team for success.[3]

Consider devoting one team meeting to organizing yourselves and your work.

  • The genesis of the team: Discuss who decided to bring the group together and why.
  • Introduction: Have each member introduce themselves and share a bit about their current professional work, their professional or academic background, and maybe even a personal fact or two.
  • Team composition: Discuss the team’s composition (this will probably be the first of several times that you will talk about this topic), the reactions of individuals asked to participate, and those who might join the team in the future.
  • Team roles: Consider how best to fill the roles of chairperson, team coordinator, research coordinator, facilitator, and staff support, etc.
  • The team’s name: Agreeing on a title for the team can be fun and often begins to focus the group on its common purpose.
  • The establishment of subcommittees: Begin a discussion about how the team’s work will be organized, considering for a moment the potential number of team members and the scope of the tasks you will undertake. Discuss the pros and cons of establishing a core committee and subcommittees. Again, it may be too soon to make a final decision on these matters, but it is not too soon to begin thinking about them.
  • Communication method: Establish a communication method that will work best for all team members. Perhaps you will use the “grapevine” method, where each person is responsible for contacting another. Alternatively, you may designate one person to be the information gatekeeper; this person will then be responsible for keeping everyone else informed. Discuss telephone contacts, email, and paper correspondences. What works best for the majority of members?
  • Regular meeting times: There is no doubt that coordinating meeting times may be one of your most difficult tasks. It is often easiest if you agree in advance on a regular time—say the first and third Wednesday of every month from 10 a.m. to noon—and suggest that members permanently mark these meetings in their calendars.

Consider devoting one meeting to exchanging information about one another.

  • The agencies at the table: In advance of the meeting, assign each member with the task of preparing a presentation on their agency for the next meeting. Control the amount of time for each presentation. Have each member describe what the agency does in general and specifically. Also have each member discuss their agency’s interest in the team’s work and what the agency stands to gain from it.
  • The individuals at the table: In the same (or, depending upon the time, a different) meeting, have each member discuss the reasons they are interested in being on the team and what they have to gain from participation. Have each member discuss what it means to them to be a meaningful participant. Have each person on the team, one by one, tell every other member why they feel that member is important to the team.
  • Meaningful work: Have each member indicate what will keep them coming to the meetings and what will cause them to drop off the team. Note these responses and pay careful attention to them over time.

Consider using a meeting to discuss the external environment (or context) in which your work is taking place.

  • Identify external conditions: Have members brainstorm a list of all the current external conditions that might influence your work, for example, a highly publicized case, a budget crisis, or an upcoming election.
  • Identify and prepare for possible influences: For each condition identified, consider the possible influence this may have on your work. Make plans to respond to influences that seem particularly worrisome and continue to monitor the others. Revisit this discussion over time.

Tips

It is important to keep a watchful eye on the collaborative team over the course of your work together. The ideas below will go a long way in helping to keep your team together and performing well:

  • Build meeting agendas together: Seek suggestions for meeting agenda items from team members and attempt to include all items. This builds investment in the meeting’s content. The chair or coordinator, facilitator, and potentially the staff should work together to develop the next meeting’s agenda well in advance.
  • Make clear work assignments: Develop an action plan for all new work tasks identified. Be clear to identify the member or members who will take responsibility for each task, how it will be carried out, and when.
  • Keep meeting records: Designate an individual to create an official record of each meeting, or rotate this responsibility. Indicate those present, the items discussed, agreements made, and action items assigned. Attach a copy of the meeting agenda. Prepare and distribute copies of the meeting record in a timely fashion. The record will inform absent members of the team’s work and serve as a reminder of assigned tasks.
  • Establish and maintain legitimacy: Work to establish the team’s legitimacy among its colleagues and partner agencies. Failing to establish legitimacy in the first place—or losing it once you have it—will undercut your ability to effect change.
  • Address turnover in membership: Members should be ready to suggest replacements to the team if and when a member leaves. It is important to identify new members as quickly as possible. Invest as much time as is necessary to bring new members up to speed, and assist them in establishing rapport with the rest of the team.
  • Remain alert to conflicts: It is common for groups to experience conflict at one time or another. Stay alert to disagreements or misunderstandings that might damage relationships or jeopardize your work, and address them directly before problems escalate.
  • Produce regular products: Document the team’s activities as discrete pieces of work are concluded. Work products—whether they are flow charts, detailed reports, or summaries of interviews conducted—provide a sense of accomplishment in addition to serving as an historical record.
  • Build in early accomplishments: As you begin your work together, attempt to identify some small but meaningful problems you can resolve quickly. For example, as you develop your system map, you may identify an inefficiency that can be easily rectified. We are not suggesting tackling prematurely major system changes that require careful analysis and planning, but rather remedying small glitches in your system that can be resolved quickly and provide some early “wins” for the team.[4]

Here are some other tips for activities that are worth doing and repeating:[5]

  • Refer to the group’s vision often.
  • Regularly review the group’s mission, goals, and objectives.
  • Stay in touch with individual policy team members to keep them engaged; “interview” them about their hopes, concerns, goals, etc.[6]
  • Conduct and repeat team-building exercises as necessary.
  • Determine if your meetings are effective. See the Appendix: Signs of Effective Meetings.

Example: Ramsey County, Minnesota, Policy Team Meeting Survey

In Ramsey County, Minnesota, the local coordinator distributed a survey at the end of each policy team meeting in order to collect feedback from the policy team members on how the meeting went. The survey asked each member to jot down the following:

 

Plusses: What worked well

Wishes: What else can be done

 

 

 

 

 

 

 

While this is a very simple idea, this strategy helped the chair and the local coordinator regularly “take the temperature” of the engagement and satisfaction levels of team members and continually assess the effectiveness of the meetings.

Additional Resources/Readings

CEPP. (2005). Signs of effective meetings.
Retrieved from https://cepp.com/wp-content/uploads/2020/03/Signs-of-Effective-Meetings.pdf

———. (2006). The role of facilitators and staff in supporting collaborative teams.
Retrieved from https://cepp.com/wp-content/uploads/2020/03/The-Role-of-Facilitators-and-Staff-in-Supporting-Collaborative-Teams.pdf

CSOM. (2007). Enhancing the management of adult and juvenile sex offenders: A handbook for policymakers and practitioners.
Retrieved from https://cepp.com/wp-content/uploads/2020/01/6-CSOM-Handbook.pdf

McGarry, P. (1993). Essential ingredients for success. In National Institute of Corrections, The intermediate sanctions handbook: Experiences and tools for policymakers (pp. 21–26). Retrieved from

SMART Technologies. (2004). Using group process techniques to improve meeting effectiveness.

Woodward, B. (1993). Establishing and maintaining the policy team. In National Institute of Corrections, The intermediate sanctions handbook: Experiences and tools for policymakers (pp. 27–34). Retrieved from

Appendix:
Signs of Effective Meetings

There are several indicators teams can use to assess the effectiveness of their meetings. Some common signs of an effective meeting include the following:

  • There is a detailed agenda that describes what will be covered, the goal of the meeting, who will discuss each item, and a time estimate for how long each item will take.
  • Responsibilities for facilitator, timekeeper, and record keeper are assigned.
  • A set of ground rules is established and posted.
  • There is clarity about the mode of decision making that will be used.
  • There are periodic checks during the meeting to ensure that progress is being made.
  • There are clear strategies to resolve conflict.
  • A record of the meeting is kept and specific action items are outlined.

If any of the following signs are present, it may indicate that teams are convening ineffective meetings and should consider changes to help make their meetings more productive:

  • Meeting goals are unclear.
  • The meeting agenda is vague or nonexistent.
  • There are no time limits on discussions.
  • There is no process for working on important issues.
  • No one is facilitating the discussion.
  • Participants haven’t done their homework.
  • Discussions are unfocused.
  • There is a lack of closure to discussions.
  • Participants argue—rather than debate—different points of view.
  • Not all team members participate.
  • The meeting ends without action plans or next steps being developed.
  • There is an absence of any check-in about how the meeting went.

From: https://cepp.com/wp-content/uploads/2020/03/Signs-of-Effective-Meetings.pdf

 

[1] NIC, 1993.

[2] For additional guidance, see 1o: Creating Useful Meeting Records and 1n: Developing Meeting Goals and Agendas.

[3] CSOM, 2007.

[4] CSOM, 2007.

[5] NIC, 1993.

[6] For a list of interview questions, see NIC, 1993, p. 28.

1n: Developing Meeting Goals and Agendas

1n: Developing Meeting Goals and Agendas web_admin

Introduction

As the stewards of the policy team, the local coordinator and the team chair are responsible for ensuring that policy team meetings are productive. This means crafting meeting goals and an agenda that move the group forward in a deliberate fashion.

You cannot assume that just because the key policymakers are together in a room the team will naturally make progress. A clear set of goals and agenda for the meeting must be developed and circulated to the team prior to the meeting so that they understand the purpose and focus of the meeting and come prepared to work toward specific outcomes.

Purpose

To provide local Initiative coordinators with information and resources that will assist them in developing meeting goals and agendas

Participants

This document is geared toward local coordinators. Local coordinators may share this information with others (e.g., the team’s chair or another member who at times may assume responsibility for developing meeting goals and agendas) as appropriate.

Instructions

In establishing the goals for a meeting, consider the following questions:[1]

  1. What are the purposes of the meeting? What are you hoping to achieve in relation to the content and the process of the meeting?
    1. Content goals: What do you think is important for people to know, be exposed to, and have discussed by the end of the meeting? What decisions have to be made by the end of the meeting? What should be accomplished?
    2. Process goals: What messages and behaviors concerning the way (norms, structures) this group will/should operate do you want to establish or reinforce?
  2. Develop these into clear statements of the goals for the meeting.
  3. How do the meeting goals fit into the longer-term goals, mission, vision, etc., of the policy team?

In planning the agenda for a meeting, consider the following:[2]

  1. Consider the goals you developed above. Break these down into three categories:
    1. Information items: This is information that you need to share with policy team members (e.g., results from a study or article) that do not require discussion or feedback.
    2. Discussion items: These are the topics that are up for discussion, feedback, and/or development by team members (e.g., the team may discuss how arrest decisions are made).
    3. Decision items: The team should spend the majority of its time coming to agreement about certain topics or determining how to move forward (particularly once the team has been established and has been meeting regularly for some time). Strive to focus on decision items for two-thirds of your team meeting time.
  2. Determine the chronology of the agenda items:
    1. ls there a logical progression of activities? For example, do you move from general to specific, from specific to general, or back and forth?
    2. Do transitions from one agenda component to another seem smooth?
    3. Are the activities consistent with the process development of the group? For example, activities that generate heavy conflict or personal vulnerability should probably not be the first items on the agenda.
  3. The meeting leader does not have to take responsibility for all the items on the agenda. Assign other policy team members to lead different parts of the agenda as appropriate. (Make sure to tell them ahead of time what is expected of them!)
  4. You may begin a meeting by checking in with team members regarding their progress in accomplishing their “homework” since the last meeting. (This is where creating meeting records that remind members of their responsibilities comes in handy!) Regularly checking in with everyone about their progress will encourage accountability.
  5. Include as the final agenda item next steps and action planning. This will ensure that the group discusses what will happen in between this meeting and the next.
  6. Consider whether the agenda accomplishes your meeting goals.

Tips

When developing agendas, you might consider the following:

  • Is there a balance between lecture/presentations and participatory activities? Between “light” discussions and “deep” (e.g., potentially conflicting) discussions? In thinking about balance, be careful not to overload people with too much of one thing.
  • What materials will you need for the meeting? How will information be presented—verbally, on a flip chart, on handouts that can be read later?
  • Will the physical space and environment accommodate your goals and agenda? For example: Is there enough space to break into smaller groups? Is there wall space for hanging up flipchart paper? Can the tables and chairs be moved to meet your needs?
  • What do your team members want to cover during the meeting? Soliciting feedback or input from policy team members before finalizing the agenda is important to gaining their buy-in.
  • Is the agenda realistic in terms of timing? Remember: everything takes longer than you expect it to!

Example: Yamhill County, Oregon, Policy Meeting Goals and Agenda

Meeting Goals

The goals of today’s meeting are to

  1. continue work on and discussion about system mapping, key decision points, and use of evidence/information at decision points;
  2. begin to develop a communications/messaging strategy; and
  3. discuss upcoming work activities.

Agenda
7:00 a.m. Review of today’s meeting goals
7:05 a.m.

Mapping

  • Check in on key decision points and upcoming work group process
  • Check in on data collection strategy
  • Check in on resource inventory
  • Preliminary discussion: opportunities for advancement
8:00 a.m.

Communication/Messaging Strategy

  • Who are the key audiences?
  • What information do they receive now, and from whom?
  • What are the things they most need to know?
  • When do they need this information?
  • Who is best positioned to provide this information?
  • What are our “ground rules” about communicating?
8:50 a.m. Next steps
9:00 a.m. Adjourn

Next Meeting Dates: All meetings will be held in the Annex.

  • February 3, 7:30–9:00
  • February 15, 7:00–9:00
  • March 3, 7:30 – 9:00
  • March 15, 7:00–9:00
  • April 7, 7:30–9:00
  • April 19, 7:00–9:00
  • May 5, 7:30–9:00
  • May 17, 7:00–9:00
  • June 2, 7:30–9:00
  • June 21, 7:00–9:00

Additional Resources/Readings

CEPP. (2006). The role of facilitators and staff in supporting collaborative teams.
Retrieved from https://cepp.com/wp-content/uploads/2020/01/5-Role-of-Facilitators-and-Staff-in-Supporting-Collaborative-Teams.pdf

McGarry, P. (1993). Essential ingredients for success. In National Institute of Corrections, The intermediate sanctions handbook: Experiences and tools for policymakers (pp. 21–26). Retrieved from

SMART Technologies. (2004). Using group process techniques to improve meeting effectiveness. Retrieved from

Woodward, B. (1993). Establishing and maintaining the policy team. In National Institute of Corrections, The intermediate sanctions handbook: Experiences and tools for policymakers (pp. 27–34). Retrieved from

 

[1] NIC, 1993, p. 23.

[2] Ibid.

1o: Creating Useful Meeting Records

1o: Creating Useful Meeting Records web_admin

Introduction

Since policymakers are busy people, designating an individual to manage and facilitate the planning process is crucial to the team’s success. Local coordinators may at times (or always) assume responsibility for facilitation and other team support responsibilities, or they may designate and/or work with others who will take on these roles.

The regular recording of meetings ensures that policy teams are as productive as possible and is the responsibility of the team staff support person (or designee). Meeting records capture essential information and remind team members what happened during a meeting, including the decisions made. Records will serve to update absent team members of the work conducted during a missed meeting, and ensure that all team members are clear about their individual assignments and the team’s next steps.

Purpose

To provide local coordinators with information and resources about how to create useful meeting records

Participants

This document has been developed for local coordinators. Local coordinators may decide to share this information with others (e.g., a designated meeting recorder) as appropriate.

Instructions

The following are instructions for taking useful meeting minutes:[1]

  1. Record the date and time of the meeting, its purpose, attendees, and the meeting lead or chair name.
  2. Take notes however you feel comfortable (laptop, pen and paper, etc.).
  3. In your notes, provide space for filling in information during the meeting; label one section “Decisions Made” and one “Action Items.” Complete these sections as they come up during the meeting.
  4. Begin the record by stating the goals of the meeting. This will help frame the record.
  5. Do not take notes verbatim (there is no need for a transcript!); instead, provide an overview of the discussion items. Highlight the purpose of each discussion item and the salient points raised. Include decisions made and assignments (action items). For each action item that you list, record who is responsible for taking the next step and by when.
  6. Ask clarifying questions as needed throughout the meeting.
  7. After the meeting, type up (or clean up) your notes. Refer to the meeting record template in the Appendix. Make sure to
    1. attach, or include, a copy of the meeting goals and agenda;
    2. add page numbers to the document;
    3. write in the same tense throughout;
    4. remove individuals’ names from discussion points;
    5. add names and dates to action items;
    6. note at the end of the record the next meeting date(s); and
    7. place reference documents in an appendix or attach them to the record.
  8. Distribute the meeting notes as a draft document to the team. Ask team members to review the notes for accuracy and to provide any corrections to the record. Once the team approves the record, note it as a final document and circulate it again.

Tips

The following are tips for taking meeting minutes:

  • Don’t worry about keeping minutes in chronological order; sometimes discussions don’t happen that way. After the meeting, you may have to reorganize some information to make the record easier to read.
  • Use few adjectives and adverbs, and avoid personal observations; try to stay as objective and factual as possible.
  • Break up discussions or agenda items within the meeting record for ease of review. Headings and subheadings can be a helpful way to organize a record, and the use of introductory sentences such as “Next, the team discussed its vision for its work” can help organize the flow. Underneath each heading or subheading, list bullet points that outline what was discussed and decided.

Example: Ramsey County, Minnesota, Record of EBDM Policy Team Meeting

 

Evidence-Based Decision Making Policy Team:
Meeting Record

Agenda item:

Policy Team Membership Update

Discussion:
Carol will be meeting soon with the sheriff and the St. Paul police department chief. There is good indication that both are on board to join the policy team; the hope is that they attend the next policy team meeting. There was a discussion on bringing in a policy team member from the St. Paul city council and a suggestion that another city-elected official be drawn from the Ramsey County League of Local Governments.

 

Action items

Person responsible

Deadline

Reach out to St. Paul city council and Ramsey County League of Local Governments

Chris Crutchfield

Feb. 5, 2011

 

Agenda item:

Education/Building Skills Committee

Discussion:
Patrick gave an update on the Education/Building Skills Committee meeting. The Committee has defined the audience, discussed the meeting mechanics, and developed a rough draft of the agenda for a March 9 event. Speakers are still being sought. The meeting should give people an understanding of evidence-based decision making and what it means for Ramsey County. Judge Gearin suggested that media be brought in. There will be a panel with the key players from each agency. Becki reminded the group that we are in Phase 2, the planning phase, of the Initiative. This event should be thought of as an awareness-building event instead of training/education.

The editorial board from both newspapers should be invited to the March 9 event. Community media should be considered as well. The Coalition to Preserve Minnesota’s Justice System does media/public outreach. They would be interested in this. Members include justices and chiefs of police, and they have monthly meetings.

 

Action items

Person responsible

Deadline

Supply Connie or Chris with numbers of attendees from each agency.

Policy team members

1/25/11

Draft letter for Judge Gearin to send out to other chief judges.

Chris Crutchfield

1/25/11

Prepare a “Save the Date” message and mail it to Ramsey County criminal justice organizations.

Chris Crutchfield

1/21/11

Prepare a press strategy: includes preparing press releases, identifying a contact person for the press, determining who will be contacted (both newspapers, community media, Coalition to Preserve Minnesota’s Justice System), etc.

Chris Crutchfield

2/5/11

Prepare an Awareness Survey to distribute to event participants.

Becki Ney and Education/Building Skills Committee

2/20/11

 

Agenda item:

System Map Review

  
 

Discussion:

The system map was reviewed. It was suggested that a large map with numbers and time frames be on display at the March 9 event. The team needs to look at how “evidence-based” we are at each decision point and what should be done to make the process better. Once the maps are somewhat finalized, the original mapping group members should be brought back together to go over them. The Data Committee will be given the task of filling the data needs determined by going over the map. Areas that are starting to look like priority decision points are Court Case Processing, Pleas, Diversion, and Warrants. Outside technical assistance could be brought in to help with the Court Case Processing decision point.

The group also discussed diversion. Is diversion linked only to Project Remand? Are there programs attached to this? These questions should also be asked of the suburbs and employers. What is our potential in this area? We should look at diversion criteria, arresting criteria, and charging criteria.

Other areas to look into include the number of people charged with livability crimes, trespassing crimes, and disorderly conduct.

The chart below reflects issues and information needs that were brought up at this meeting.

 

System Map Review: Issues/Information Needs (Adult Offenders Only)

Police

Guidelines on who or who not to arrest; how are these decisions made (type of offense, behavior, etc.)?

Guidelines for establishing probable cause

Guidelines for issuing citations

Pretrial (Project Remand)

Return to court rate for each category and demographic

Appearance rates of detainees v. non-detainees (first appearance in court)

Diversion (no plea of guilty for Project Remand diversion)

  • Is it risk based?
  • How many people are diverted?
  • What are the criteria for diversion?

Warrants

When are warrants issued and for what reason? What are the number of

  • felonies;
  • misdemeanors;
  • no shows;
  • no reappearance; and
  • revocations of conditional release?

How long have warrants been outstanding?

Average number of warrants per person

Number of warrants v. number of people issued warrants

Charging

Need the payables list

What are the criteria for payables?

First Appearance

Number of guilty v. not guilty

% of offenders who plea at each point

What do early resolution cases look like compared to ones that go further?

Court

Number of pleas at each stage

Number of dismissed at each stage

Number of continuances and reasons; who requested continuance?

Length of time from start to resolution

Correctional Facility

How many people are at RCCF on livability crimes?

 

The meeting was adjourned.

Next Meeting: February 14, 2011, Noon–4:30, Room 41, Courthouse

Additional Resources/Readings

CEPP. (2006). The role of facilitators and staff in supporting collaborative teams.
Retrieved from https://cepp.com/wp-content/uploads/2020/01/5-Role-of-Facilitators-and-Staff-in-Supporting-Collaborative-Teams.pdf

McGarry, P. (1993). Essential ingredients for success. In National Institute of Corrections, The intermediate sanctions handbook: Experiences and tools for policymakers (pp. 21–26). Retrieved from

SMART Technologies. (2004). How to record useful meeting minutes. Retrieved from

Woodward, B. (1993). Establishing and maintaining the policy team. In National Institute of Corrections, The intermediate sanctions handbook: Experiences and tools for policymakers (pp. 27–34). Retrieved from

 

Appendix:
Meeting Record Template

Meeting Record Template

[1] SMART Technologies, 2004.

2: Readiness for Change

2: Readiness for Change web_admin

Activity 2:
Build individual agencies that are collaborative and in a state of readiness for change

For evidence-based decision making to be effective, it must occur with consistency throughout the justice system. That is, the reliance on evidence to inform decision making should occur at the system level, at the agency level, and at the case level.

Agency- and case-level alignment requires a specific focus on organizational development within each of the justice system agencies. This involves

  • reevaluating agency mission, goals, and values to support a vision that is shared by all the justice system stakeholders as well as by the agency's workforce;
  • reconsidering agency policy and practice in light of evidence-based knowledge;
  • in some instances, retooling organizational structure;
  • addressing, where necessary, organizational culture to align with a new vision, mission, and goals; and
  • providing new knowledge and skills for staff.

For these change efforts to take hold, they must prove themselves to be reliable and to better support staffs' ability to effectively carry out their duties. For example, if at the sentencing stage, objective data is provided to defense counsel, prosecutors, and judges that effectively informs and shapes the sentencing decision, decision makers will come to not only expect but also to rely on this information in the future. If, on the other hand, the information provided is neither useful nor reliable, the new approach of considering objective data will be abandoned and past practice will prevail.

Organizational change is not easy, nor is it always successful. According to experts

  • up to 85% of organizational change initiatives fail; and
  • up to 70% of these failures are due to flawed execution.1

Click on the document in the left menu bar for an activity designed to assist you in establishing EBDM consistently within individual agencies and across the criminal justice system.

Elements of an EBDM justice system include

  • agencies that demonstrate a collaborative climate and readiness for change; and
  • an engaged staff who provide meaningful, ongoing input into evidence-based policy and practice changes.

2a: Engaging and Readying Staff for Change

2a: Engaging and Readying Staff for Change web_admin

Introduction

For evidence-based decision making to achieve its optimum effect, it must occur consistently within individual agencies and across the criminal justice system. That is, the reliance on evidence to inform decision making should occur at the system level (the work the collaborative policy team is undertaking), at the agency level, and at the case level. This agency and case level “alignment” requires a specific focus on the individual agencies and the professionals within them. In the case of many agencies, it will likely involve

  • reevaluating the agency’s mission, goals, and values to determine the degree to which they align with and support the policy team’s system-wide vision;
  • reconsidering agency policy and practice in light of evidence-based knowledge;
  • retooling organizational culture, structure, and agency policy and practice where needed; and
  • providing new knowledge and skills for staff.

Staff engagement will be critical to making these efforts possible.

Purpose

To provide opportunities for professionals across the criminal justice system to become actively engaged in the process of building a criminal justice system that employs evidence-based decision making

Participants

All policy team members should be involved in engaging their agency’s staff in this work. How the work is conducted in each agency may vary, depending upon the size, structure and culture of the organization, as well as the extent to which evidence-based decision making has already been incorporated.

Instructions

  1. Devote a policy team meeting to the topic of engaging and readying staff for change. Review, and then consider, the relative merits of the strategies offered in the “Tips” section of this document. Keep in mind that some ideas will have merit across all agencies, some will make sense for only one or a few agencies, and some will not serve any agencies well.
  2. As a team, generate a list of additional strategies for engaging and readying staff for change. Assess the merits of these additional ideas. For each strategy that has merit, develop an action plan that considers the involvement of
    1. all agencies (those strategies that can be employed on a systemwide basis);
    2. a cluster of agencies; or
    3. single agencies.

    Consider establishing work groups that draw upon the talents of agency staff to carry out these action plans. This both expands the capacity of the policy team to accomplish work and builds agency staff engagement.

  3. Be sure that action plans include methods to monitor progress and assess the impact of the action plan. For instance, action plans might include routine report-backs to the policy team on activities and results. Action plans might also include objective measures to assess change over time. For instance, a simple ten-question survey administered to all agency staff at the beginning of the effort is an efficient way to capture baseline information. Repeating the survey at a later point in time will provide information about the effectiveness of the strategies adopted. The survey can be found in the Appendix. Alternatively, use the survey as a set of prompting discussion questions at an informal, “brown bag” lunch session.

Tips

Outreach Efforts

In Mesa County, Colorado, the policy team chair developed a standard presentation on EBDM and the policy team’s work, which she gave at “brown bag lunches” in different agencies to begin to build more interest in the Initiative.

The following are some possible strategies for engaging staff and readying them for potential changes that will result from involvement in the EBDM initiative:

  • Establish a county-wide web page to share information. Possible content includes information about the national EBDM initiative, the county’s involvement in the initiative (e.g., team members, action plan), and research findings.
  • Create a newsletter (electronic or paper) to share the kind of information described above.
  • Create research briefs that highlight specific empirical findings and their relevance to the county’s justice system.
  • Convene cross-disciplinary briefings to increase agencies’ knowledge about mission, roles, responsibilities, and activities to enhance understanding and foster new or improved working relationships.
  • Convene brown bag lunches within or across agencies to build stronger working alliances—particularly interdisciplinary alliances.
  • Circulate research articles and seek reaction and input from staff about their findings, the extent to which local policies and practices reflect these findings, and/or ideas about how to advance current practice using these research findings.
  • Administer surveys to staff to gauge their level of knowledge about particular matters and/or to seek their ideas, involvement, and input.
  • Meaningfully engage mid-level supervisors by asking them to lead certain portions of the agency’s EBDM efforts (e.g., convene a meeting of staff to examine the agency’s current mission statement and evaluate the extent to which it is consistent with the policy team’s vision for the local justice system; recommend modifications to the mission statement as appropriate; identify specific ways the agency’s work supports the system-wide vision; engage supervisors in the policy and practice analysis process; engage supervisors in the process of assessing line staff knowledge and skills around the core competencies of correctional practice).
  • Convene ad hoc work groups (within or across agencies; of mixed or same level of authority) to take on specific pieces of work in support of the policy team’s action plan.
  • Conduct briefings at staff meetings to share and dialogue about research findings, the EBDM initiative, and/or case study materials from colleague communities that have had success in risk and harm reduction efforts.
  • Hold brainstorming or focus group sessions to explore discreet ideas or problems (e.g., Which harm reduction measures should our community strive for? How might community members be more substantively educated about matters related to the justice system? How might certain kinds of information be shared with other system actors more effectively in the future?).
  • Prepare communications materials (e.g., vision statements, statement of values, research findings, etc.) and post visibly in agency lobbies, meeting rooms, and staff offices. Create a local “identity” related to the EBDM work.

Example 1:
Ramsey County, Minnesota, EBDM Awareness Event

On March 9, 2011, the Ramsey County, Minnesota, EBDM policy team hosted a half-day awareness event. The primary goal of the event was to raise awareness of the EBDM initiative among key stakeholders and their staff. More than 200 people attended the event from all levels of key agencies, including law enforcement, corrections, courts, public defender, city and county prosecution, county commission, and state criminal justice agencies. National experts Mark Carey and Frank Domurad, from The Carey Group, were faculty for the event; they presented information about the Framework and the rationale for it, as well as key principles of evidence-based practices.

Prior to the event, the Ramsey County EBDM Education Subcommittee took on the charge of identifying those to be invited, sending out “hold the date” announcements, certifying the event for Continuing Education Units (CEU), developing the goals and agenda, identifying faculty, and taking care of all the detailed planning and logistics for the event. EBDM staff also developed press releases and made sure that media were aware of the event. The event began mid-day with a luncheon, followed by a three-hour training session. The EBDM team’s map was blown up to wall size and prominently displayed in the meeting room as well as the “One Less…” logos. EBDM policy team members were on hand to answer any questions about the map and initiative. A Ramsey County EBDM brochure and EBP resource listing was provided to all those who attended.

During the event, participants were asked to complete a pre- and post-test to gauge their level of awareness and understanding of the EBDM initiative and the principles of evidence-based practices. The pre- and post-test were developed based on the awareness survey in the Starter Kit, and were tailored to their needs:

  • Prior to the event, 40% of participants said they were clear about (aware of) the EBDM initiative; after the event, 79% said they were clear about the initiative.
  • Prior to the event, only 28% of participants said they were very knowledgeable about evidence-based practices; after the event, 67% said they were very knowledgeable about evidence-based practices.

The event was well received by those in attendance and achieved the goals envisioned by the EBDM policy team. And as a result of the event, several participants signed up to receive regular updates about the project and volunteered to participate on various subcommittees. The EBDM policy team agreed to consider conducting a similar event later on in the effort to announce their implementation plan and to continue to gain the support and participation of all the agencies involved.

Example 2:
Charlottesville-Albemarle County, Virginia, Staff Engagement Strategy

Engaging Our Staff in the EBDM Initiative

Our overall goal is to engage the staff of the policy team members in the process of evidence-based decision making. The following is the proposed process for educating and engaging the staff:

  1. Set up meetings at each agency during a regularly scheduled staff meeting or a special staff meeting, with the purpose of addressing the EBDM process.
  2. Provide a one-pager on the EBDM initiative to the staff when inviting them to the meeting. Ask that they read the information and come to the meeting with ideas about what harm reduction means to them and what activities the policy team should adopt to address the areas of greatest concern.
  3. Invite someone from another discipline to present with you to show that this is a justice community initiative that all of us are participating in. It is important to demonstrate the collaboration around this process.

    Present the PowerPoint describing the Initiative in more detail. Engage staff in a discussion about how you, as a team, are attempting to make key decisions around evidence and research. Then

    1. share the mapping process, vision statement, decision point, etc.
    2. share the action items you are considering for the implementation phase;
    3. ask the staff for their input on the action items;
    4. ask them what evidence or research would help them do their job better; and
    5. ask them what they would propose if they could influence what the policy team was considering.

Documents to have at the session include

  • copies of the PowerPoint presentation;
  • one-page EBDM initiative summaries;
  • the vision poster (perhaps laminated…although this would not be a handout);
  • the One Less stakeholder brochures; and
  • top priorities, proposed activities/research items, and harm reduction measures that the policy team has tentatively selected.

Appendix A: 
Sample Survey on Staff Knowledge of and Interest in the EBDM Initiative

Our county is one of many communities across the country participating in the Evidence-Based Decision Making in Local Criminal Justice Systems Initiative. Please take a few minutes to answer the following eight questions regarding our county’s participation in this effort.

  1. Before receiving this survey, were you aware that our county was involved in this initiative?
    1. Yes, well aware
    2. Yes, vaguely aware
    3. No, not aware
  2. Have you seen any written information about this initiative (e.g., handouts, newsletters, memos, web pages)?
    1. Yes, I’ve seen a fair amount of written information about this initiative
    2. Yes, I’ve seen some written information about this initiative
    3. No, I have not seen any written information about this initiative
  3. To what extent do you feel you understand the purpose of this initiative and some of its potential outcomes?
    1. I feel clear about the purpose and potential outcomes of our participation in this initiative.
    2. I have a vague idea about the purpose and potential outcomes of our participation in this initiative.
    3. I feel unclear about the purpose and potential outcomes of our participation in this initiative.
  4. To what extent do you feel you would like more information about this initiative?
    1. I would like more information about this initiative and our county and agency’s involvement in it.
    2. I do not need more information about this initiative and our involvement in it.
  5. If you indicated in Question 4 that you would like additional information, through what means would you most prefer this information?
    1. Written information (e.g., memo, newsletter, or documents)
    2. Staff meeting
    3. Conversation with supervisor
    4. Observe a policy team meeting
    5. Other: ___________________________________
  6. To what extent do you feel involved in this initiative?
    1. I feel very involved in this initiative.
    2. I feel somewhat involved in this initiative.
    3. I do not feel involved in this Initiative.
  7. To what extent would you like to be involved?
    1. I would like to be more involved in this initiative.
    2. I feel sufficiently involved in this initiative.
    3. I do not want to be involved in this initiative.
  8. If you answered in Question 6 that you feel very or somewhat involved in this initiative, please indicate the ways in which you feel involved. (Multiple responses are possible.)
    1. I have read materials about this initiative.
    2. I have attended meetings of those leading this initiative in our county.
    3. I have been briefed by individuals involved in leading this initiative in our county.
    4. I have been asked to play a role in this initiative by participating in meetings, discussions, and/or work groups.
    5. I have been asked to provide input into this initiative by providing information and/or ideas.
    6. Other: ______________________________
  9. If you answered in Question 7 that you would like to be more involved, in what ways would you like to be more involved?
    1. I would like to read materials about this initiative.
    2. I would like to attend meetings of those leading this initiative in our county.
    3. I would like to be briefed by individuals involved in leading this initiative in our county.
    4. I would like to play a role in this initiative by participating in meetings, discussions, and/or work groups.
    5. I would like to provide input into this initiative by providing information and/or ideas.
    6. Other: ______________________________
  10. Please provide any additional information you would like us to know as it relates to this initiative.

3: Understand Current Practice

3: Understand Current Practice web_admin

Activity 3:
Understand current practice within each agency and across the system

Developing a vision of a justice system that results in harm and risk reduction—and engaging professionals and the larger community in that process—is one thing; building such a system is quite another. Such work begins with a clear understanding of how the justice system currently operates (i.e., the policies that guide the system, the practices that lead to its operation) and a working knowledge of research-supported approaches.

Developing this understanding involves a number of steps, including creating a system map, conducting assessments of current policy and practice around each of the key decision points in the criminal justice system, identifying the strengths of the current system and areas for potential improvement, and prioritizing targets for change.

Elements of an EBDM justice system include

  • a comprehensive understanding of the justice system, including the basis upon which decisions are made at key points within and across agencies;
  • a set of agreed-upon strengths and opportunities for change that will result in an increase in evidence-based decisions; and
  • a set of agreed-upon targets for change.

3a: Developing a System Map

3a: Developing a System Map web_admin

Introduction

One of the most fundamental ways to develop an understanding of a jurisdiction’s justice system is to develop a “system map.” Similar to an architectural diagram, a system map depicts the steps in the criminal justice process (i.e., processing of a case and the activities related to this), beginning with police contact and ending with the point in time when the case terminates. In addition to reflecting the key decision points in the system, the map reflects the decision makers at each key point and the amount of time it takes a case to move from one point to the next. It is also possible and desirable to document the volume of cases that flow through each process step and decision point. This may be accomplished first by noting estimated numbers and later by gathering data on a specified period of time to more precisely determine the flow and volume of cases and activities.

In the EBDM initiative, the system map will be the first step in developing a detailed understanding of each justice system decision point and of the evidence that informs these key decisions. Following the completion of the system map, a process to “dig deeper” into these decision points will be carried out. This more in-depth analysis will include an examination of the following as they relate to each decision point:

  • written policies;
  • the application of those policies to practice, as well as to other operational practices that are not formally articulated in policy;
  • the various types of data and information collected at each decision point;
  • the ways in which this data and information inform decisions;
  • the ways in which information is stored and shared; and
  • other factors related to using data, information, and evidence in the most efficacious ways.

Purpose

Mapping serves a variety of purposes:

  • It increases awareness of the ways in which the entire system “works” and how different parts of the system interact with one another. (Most people understand quite well their own “part” of the system but have a less detailed understanding of the other parts of the system.)
  • It brings together policymakers and agency staff to articulate the decisions they make, how they arrive at those decisions, and when (i.e., at what point in the process) decisions are made.
  • It surfaces areas of interest for further inquiry.
  • It can sometimes lead to recognition of quick solutions to bottlenecks or inefficiencies.

Participants

Building a system map is a time-consuming process. The most desirable method is for the entire policy team to be fully involved in its development. However, given that this process can take a full day or more, this may not be practical. A second, effective option is to convene a series of “work groups” to help with the construction of the map.

Work groups might be formed around the following decision points:

  1. arrest
  2. pretrial status
  3. charging/plea
  4. sentencing
  5. local institutional intervention(s)/local institutional release
  6. community intervention(s)
  7. violation response
  8. discharge from supervision

Work groups should include a range of individuals who have different roles and perspectives in the case and decision making processes. Different levels of decision making authority should be represented as well, including the highest decision maker (agency head), those who oversee the work being conducted (supervisors/managers), and those at the line level. In many cases, work groups will also include those who handle information flow and case files. Receptionists, clerks, and administrative staff often have the best “window” on how things actually get done; they should not be overlooked in this process.

In addition, it is strongly recommended that the persons involved in information technology (data collection and analysis) participate in work groups. As noted above, it will be important to affix data to the map at some point in the future. Involving those who will be responsible for collecting and analyzing that data will facilitate their understanding of what is needed when the time comes to collect this data.

Finally, it is highly desirable for one or more persons from the team (i.e., local coordinator and/or another member) to be present for the entire mapping process, if at all possible. While this is a major time investment, the opportunity for at least one member of the jurisdiction to observe and learn from the entire process should not be underestimated.

Instructions

  1. The policy team should have a discussion about mapping in advance of the mapping session:
    • Identify some of the issues and challenges the team most wants to learn about at each decision point to make sure these topics are addressed during the initial mapping session.
    • Identify the key individuals from each agency who have detailed knowledge about each decision point and make a plan for inviting them to attend each work group session. (See participants above.)
    • Identify the policy team member(s) who will attend the entire mapping session and who will be responsible, along with the local coordinator, for shepherding the mapping process for the team.
  2. The local coordinator should work with policy team members to craft an invitation to work group participants who will attend the mapping sessions. (An invitation template is provided in Appendix 1.) The invitation should outline the purpose of the mapping work group sessions, the date and two-hour work group session(s) they are asked to attend, and any additional information they should bring with them to the mapping work group session (i.e., information about time frames between decision points, easily accessible data about the volume of cases at decision points, maps they know about or that already exist and that can be folded into the mapping process). Also, consider making resource materials (see Chapter 13 of Getting it Right) available to work group participants in advance of the session so that they can be familiar with the process and come to the session prepared to map.
  3. The local coordinator may sit down with the team chair and/or the policy team to review the final list of persons who are planning to attend each session in order to ensure a comprehensive group of participants.
  4. Prior to the mapping sessions, it should be determined who will take notes during the session. It is recommended that at least one staff person assist the facilitator in capturing the discussion during the sessions. The team should also determine who will be responsible (facilitator, local coordinator, and/or other policy team members or staff) for putting all the “pieces” of the map together in one diagram in an easy-to-view format.
  5. Select a room with a considerable amount of wall space, where flip chart paper can be posted from end to end. Rooms with walls that do not permit (or are not conducive to) taping lots of paper to the wall should be avoided. The facilitator will need to be provided with a full pad of flip chart paper, a good set of flip chart markers, and a roll of masking tape.
  6. While not a necessity, it is recommended that jurisdictions bring in an outside facilitator to facilitate each mapping work group session. The session will begin with introductions of all participants and a brief overview by the facilitator of the purpose of the process. You may wish to provide a handout on the mapping process. (A sample handout is provided in Appendix 2.) Following this, the facilitator will guide the work group through the development of their component of the system map. (It is possible, albeit rare, that time will run out before the work is completed. In this case, the facilitator will dialogue with the work group about the best way to complete the map.)
  7. During or following each work group session, the facilitator should receive a complete list of the individuals participating in each session, including name, title, agency, and full contact information, for the purposes of seeking further information, receiving feedback on the accuracy of the map, and documenting those who contributed to its development.
  8. At the conclusion of the session, the facilitator should make a plan with the work group to review the final, transcribed portion of the work group’s portion of the map.
  9. Because of the complexity of the entire system and the desire to capture a “picture” of the system on paper, it is not possible to include on a flow chart every detail about a particular decision in the criminal justice system. For this reason, the person(s) transcribing the map and notes may wish to use footnotes that provide additional narrative. (See the example provided for an idea of how one jurisdiction captured its system map—and accompanying map notations—on paper.)
  10. Once the system map components have been reviewed by each work group, a final draft of the entire map should be presented to and reviewed by the policy team. For this meeting, a large share of the full team meeting time should be devoted to reviewing and discussing the map—first to ensure its completeness and accuracy, and then to discuss its implications.
  11. Once the mapping activity is fully completed, the team should prepare for the next, more in-depth step of the decision point analysis process.[1]

Tips

Mapping the Entire System

In Yamhill County, Oregon, the policy team created three separate maps in order to ensure that the justice system processes were adequately captured for the mentally ill, chemically dependent, and general offender population. To link the multiple maps, a process step was included in the main map that referenced a different track. The main and “mental health track” maps are provided.
  • For each work group session, carefully select a diverse group of individuals who have the most knowledge and perspective about both the formal policy/decision making and informal decision making that occurs at each decision point.
  • The number of individuals invited to attend each work group session should be considered—large enough to ensure representation, knowledge, and diversity, but not overly large as to be unworkable (ideally, about 10–15 people in each work group session).
  • Some individuals may have critical knowledge about more than one decision point and should be asked to attend more than one work group session, as appropriate. Likewise, some decision points may overlap more than others (for example, arrest and pretrial status or institutional and community interventions). In this case, consider whether it makes sense to combine decision point work groups into one group.
  • Consider sequencing work group sessions from the earliest decision point (arrest) to the latest decision point (discharge from supervision). For example, those invited to attend the work group session on the arrest decision point might be scheduled from 9:00–11:00 a.m.; pretrial status from 11:00 a.m.–1:00 p.m., and so forth.
  • Leave completed sections of the map hanging on the wall as each “new” work group convenes; visually, this will provide an overview of how each decision point “links” to the last one.
  • Assure that adequate time is devoted to each decision point; it may not be possible to complete an initial map of all of the decision points in one day.
  • Consider color coding the map or using sticky notes to flag questions that arise for further exploration, to identify specific processes for special populations (for example, domestic violence offenders) that may depart from the typical decision process, to identify gaps in current decision making, to note strategies that are being considered or implemented but that are not currently part of the decision making process, or to note other pertinent issues identified by the work group.
  • Consider conducting a “debriefing” session at the conclusion of the mapping work group sessions with the facilitator, local coordinator, and policy team members assigned to shepherding the mapping process. The debriefing session will provide an opportunity to discuss how well they think the mapping work group sessions captured information about each decision point, confirm a work plan for completing the map, identify an upcoming policy team meeting to discuss the map in more detail, and discuss any other issues or logistics that need to be resolved before moving forward.

 

[1] See 3b: Conducting a Policy and Practice Analysis.

3b: Conducting a Policy and Practice Analysis

3b: Conducting a Policy and Practice Analysis web_admin

Introduction

The development of a “system map” is a first step in developing a common understanding of how the criminal justice process “works” in a jurisdiction. When completed, the system map reflects the flow of cases through the justice process, key decision points, decision makers, and the volume of cases flowing through the system during a particular period of time. And while it can surface system strengths and challenges, it is the first step to—rather than a substitute for —a more in-depth examination of the policies and practices that underlie the workings of the system.

Following the completion of the system map, a process to “dig deeper” into these decision points should be undertaken. This more in-depth analysis includes an examination of the following as they relate to each decision point:

  • written policies;
  • the application of those policies to practice, as well as other operational practices that are not formally articulated in policy;
  • the various types of data and information collected at each decision point;
  • the ways in which this data and information inform decisions;
  • the ways in which information is stored and shared;
  • the extent to which evidence-based information and research is available and used to make decisions;
  • gaps and barriers that impede the use of evidence-based knowledge to inform these decisions; and
  • other factors related to using data, information, and evidence in the most efficacious ways.

Purpose

Most matters of public policy are guided in part by an array of formal policies contained in federal or state law, county code, case law, individual agency policy, or memoranda of understanding among multiple organizations. An assessment of current policy begins with the identification of these operating guidelines. Formal policy only begins to define the manner in which decisions are made and processes are carried out; in most communities, written policy guides only a small portion of activity. When formal policy leaves activities undefined, informal practices emerge to fill the gaps. Informal practices are sometimes developed with clear purpose and great care; in other instances, they simply evolve over time. Developing an understanding of informal practice is as critical as understanding current policy.

A policy and practice analysis serves to objectively assess the extent to which

  • clear, written policies dictate key decisions and practices;
  • policies are consistent with evidence-based knowledge;
  • policies are consistently carried out, consistent with one another, and support their stated purpose(s), agency missions, and jurisdiction vision;
  • gaps in policy result in discretionary practices, and the degree to which that discretion results in practice that is evidence-based, consistent, and in support of vision and mission; and
  • the collection, storage, sharing, and application of data and information support effective decisions.

Participants

A policy and practice analysis can be conducted by staff internal to the agency(ies) under examination, by staff from colleague agencies within the jurisdiction, and/or by external assessors. The latter approach offers the opportunity to bring to the jurisdiction the perspective of objective subject matter experts. While this might be the ideal approach, it may not always be possible.[1]

Instructions

Using Outside Experts to Conduct an Analysis

  1. Discuss the purposes of the policy and practice analysis. Make sure the team understands that the analysis will be conducted by individuals with expertise in specific areas who will assess the extent to which current policies and practices regarding key decision points are supported by research. Against that backdrop, engage the team in a discussion about their priorities for the analysis (i.e., What would you like to understand about these decision points? What would you like to have more information about? What (if any) change targets can you envision regarding the decision points? What information can be derived from the analysis to better inform the team about those potential targets?).
  2. Discuss with the policy team the process and timing of the analysis: who will conduct it, the expertise of the individuals involved, the anticipated duration, schedule of interviews and observations, and the method of recording and reporting the findings.
  3. Agree on an approach to informing affected agency staff of the purposes and timing of the assessment.
  4. Ask the outside experts to provide a report following the assessment that outlines key findings and specific recommendations for changes that the site can make to further align itself with the most current research. Ideally, a debrief session with the policy team will be conducted to enable the team to ask questions, deliberate over findings with the aide of subject matter experts, and develop strategies for action.

Process for Conducting a Policy and Practice Analysis through Self-Assessment

  1. Form subcommittees (or work groups) to conduct assessments of discreet areas (e.g., agencies, decision points, a combination of these).
  2. Across all subcommittees, develop a shared sense of purpose for this process—what information is to be collected and why, an agreed-upon set of “ground rules” for carrying out the process (e.g., ethical and professional behavior during and following the analytic process, confidentiality limits and expectations, etc.), the timeline, the final product subcommittees are to produce, and how and with whom these final products will be shared.
  3. Identify a “chair” for each subcommittee. This person should be responsible for calling together their subcommittee, facilitating their meetings, keeping the subcommittee on task, and sharing information with the policy team to ensure good communication, coordination, and information flow.
  4. Subcommittees should become familiar with the relevant research in their area(s) as well as emerging practices that have yet to be demonstrated empirically (if they do not already have familiarity in these areas). This knowledge will be the “backdrop” against which the analysis will be conducted.
  5. Be clear with subcommittees that part of their role is to educate the policy team members on their topic. The goal, in the end, is to have a common, core base of knowledge across all policy team members.
  6. Create mixed-discipline subcommittees. For example, do not assign only those team members who are involved in law enforcement to the cite/detain decision point. Doing otherwise will provide a unique opportunity for cross-disciplinary learning and for an “outsider” to ask questions that those entrenched in current processes may not consider.
  7. Early subcommittee meetings should be focused on developing a detailed work plan and timeline for conducting the analysis.
  8. Subcommittees should continue to meet through the information collection and analysis process to ensure that the work is being carried out as planned, data is recorded in a useful manner, and difficulties with the information collection and analysis process are addressed early.
  9. At the conclusion of the subcommittee’s work, it should be prepared to present its findings and recommendations to the entire team.[2]

Tips

Establishing a "Data Committee"

Many EBDM policy teams elect to form a “data committee” in addition to work groups tasked with examining decision points. These committees provide information and data support to the work groups throughout the mapping and policy and practice analysis processes. Although in some instances existing data systems may be limited in the amount and type of data that can be collected, the data committees seek to collect, at least, the following baseline information:

  • number of cases following each decision path on the system map;
  • average case processing time between steps;
  • average length of time/stay in pretrial and post-conviction disposition options;
  • characteristics/profiles of cases in disposition options (offense and offender characteristics, including available information on risk and needs);
  • success/completion rates for disposition options; and
  • failures to appear (FTAs), new arrests, and recidivism outcomes by disposition option.

For more information, see 3d Gathering Baseline Data.

Consider the following tips for collecting information effectively:

  • The quality of the information collected through this process will have tremendous bearing on the action planning decisions of the team. Seek quality information over quantity if time constraints are a factor. To that end, if it is not possible to thoroughly assess all decision points, agree with the policy team on the initial priorities for assessment.
  • The information gathering process should be used as an opportunity to educate those outside of the formal team about the vision, mission, and goals of the policy team, and to learn their perspectives on the strengths and gaps in the jurisdiction relative to evidence-based decision making. Doing so is likely to both elicit cooperation and engender long-term support for the team’s work.
  • If a team chooses to work together on all of the component sections rather than working in subcommittees, a work plan and timeline for each section should be developed. Dividing the work among individual group members and encouraging them to answer the questions independently—without at least processing them with the team—is not recommended. Doing so will provide only one person’s perspective, and is unlikely to give an accurate or comprehensive picture of policy and practice across the jurisdiction.
  • Do not rely solely on the expertise of policy team members and/or subcommittee members to conduct the analysis. Instead, collect multiple data sources to ascertain current policies and practices. Ideally, interview decision makers regarding current policies and practices; review documents (i.e., written policies, procedure manuals, programmatic materials, forms, and data screens); conduct individual interviews or focus groups with supervisors and staff; and observe the activities being assessed. This approach will provide the broadest view and most reliable information. (For instance, in assessing how pre-trial release recommendations are made, review written statutes, bail schedules, and agency policies; interview the pre-trial administrator, chief prosecutor, and bail commissioner; interview pre-trial release officers; and observe pre-trial interviews, verifications, the development of reports, and bail review hearings).
  • Expect variability in practice. It is not unusual to find that there is a lack of uniformity within a jurisdiction, with some practitioners or agencies doing things one way and others doing it completely differently.
  • In those cases where numerous individuals are involved in carrying out a particular function, committees might consider convening a focus group to understand the variety of approaches in use. For example, it might be helpful to identify those prosecutors handling violations of parole/probation cases and to interview them together in a focus group format to understand their management of these cases.

Example:
EBDM Work Group Charter for Assessing Evidence-Based Policy, Practice, and Decision Making

What are we trying to achieve through the work group process?

The role of the work group is to educate and advise the EBDM policy team on strategies that will result in the greater use of evidence (research) to support decision making consistent with the team’s vision for the justice system.

What do we mean by “evidence”?

In the justice system, the term “evidence” is used in a variety of ways. ”Evidence” can refer to items collected at a crime scene, eyewitness accounts, or security camera footage. These types of evidence are referred to as legal evidence. For the purposes of the EBDM initiative, the term “evidence” is used to describe findings from empirically sound social science research. The Framework refers to the results of this research as evidence-based policy and practice.

What is “evidence-based decision making”?

Evidence-based decision making is the use of evidence (as defined above) to inform decisions throughout the criminal justice system process at the case level (e.g., applying practices in light of offenders’ risk level), at the agency level (e.g., providing agency direction and support that results in probation officers spending 20 minutes or more in their offender contacts, and focusing those contacts on risk reduction techniques), and at the system level (e.g., working collaboratively to collect and analyze data to determine if systemwide policy decisions are resulting in the desired outcomes, and making adjustments as needed).

Who should be on the work group?

The work group should be composed of individuals who represent all agencies that affect or are affected by the decision points the group is tasked with addressing. The selected individuals should represent diverse experiences and points of view in order to ensure the broadest perspective on the topics discussed. The work group serves as a standing committee that seeks input from others (“ad hoc members”) as needed.

What is the role of the chairs or co-chairs?

Chairs or co-chairs are tasked with leading and organizing the work group. To that end, it is their responsibility to make certain that they and their group members are clear about their tasks and the work they are undertaking and, based upon this understanding, develop a work process that will be successful in accomplishing specific tasks within the designated time frame. It will be up to each set of work group chairs to determine how and when the work group will meet. The work group chairs will also be responsible for reporting progress to, and sharing their products with, the EBDM policy team.

What is the work group’s time frame?

Work groups should be prepared to complete their work by April 2011. If completion is possible sooner, this is desirable, but not if it means sacrificing quality of work.

What tasks are the chairs/co-chairs and their work groups expected to accomplish?

  1. Identify the agencies that/individuals who should be on the work group and secure their agreement to participate.
  2. Ensure that all work group members have read the EBDM Framework and EBDM project overview documents, and are familiar with relevant portions of the Mesa County system maps.
  3. Ensure that all work group members are aware of the EBDM policy team’s vision statement and have read the team’s charter.
  4. Ensure that all work group members are clear about the work group’s tasks and purposes.
  5. Create and manage a work plan (e.g., recruit members, establish a schedule of meetings that works for members for the next 2–3 months, clarify the tasks and work to be undertaken).
  6. Review and ensure that the system map and system map notations are complete and accurate.
  7. Identify all the decision points and decision options on the system map that are within the domain of the work groups. Ensure that relevant decision points are reflected by a diamond shape.
  8. Educate yourselves! Identify and review the evidence-based practice literature that most applies to the decision points for which your work group is responsible.
  9. Discuss the extent to which evidence-based information is available and used to inform each of these key decisions. For each decision point, ask yourself the following:
    1. What risk and harm reduction measures are we seeking to achieve (or contribute to in major ways) at this decision point?
    2. What guidance does the existing research literature have to offer at this decision point?
    3. What can be done to ensure that research informs decisions made at this point?
  10. Identify gaps and barriers that impede the effective use of evidence-based knowledge to inform these decisions (e.g., offender risk/need information is collected but not used to inform a particular decision; risk/need information is not collected; or research suggesting that individuals identified as low risk through the use of an actuarial risk tool is not used to inform the intensity of the applied criminal justice intervention).
  11. Identify further information needs that relate to #8 and that would inform the work group’s discussions (e.g., “We know that community corrections has risk/need information, but we need further information about how this information is used (i.e., consistently? accurately? appropriately?) and the extent to which it supports the outcomes we desire” or “We know that we have a variety of intervention options, but we do not know the criminogenic needs (risk factors) these interventions address or the risk levels of the individuals who are receiving these services”). Ask your work group’s data committee member to assist with getting answers to these questions.
  12. Identify how best the information in #9 would be collected. Can the work group do this, or someone else within the Mesa County justice system or county? Is support needed from an outside expert?
  13. Identify change targets and opportunities that will support the EBDM policy team’s efforts to achieve its vision.
  14. Keep the executive committee informed of your progress and of any issues/problems that need the input and advice of the policy team.

 

[1] Jurisdictions might explore whether technical assistance is available for this purpose.

[2] See 3e: Prioritizing Your Team’s Targets for Change.

3c: Creating a Resource Inventory

3c: Creating a Resource Inventory web_admin

Introduction

In addition to developing a system map and gathering information about the defendants/offenders in your system, conducting an assessment of the various services and resources available—within the criminal justice system, as well as from public sector entities in your community—will better inform your team’s policy and practice decisions.

Purpose

This Starter Kit is designed to provide instructions on how a jurisdiction might inventory and assess the resources available to defendants/offenders in their community and determine the strengths and best use of the current array of services, as well as gaps and overlaps in services.

Participants

Participants should include staff from the multiple criminal justice agencies represented on the policy team who are most knowledgeable about the resources, programs, and interventions available to offenders/defendants.

Instructions

  1. Conduct a brainstorming session to develop a list of all the resources in your jurisdiction currently available for defendants/offenders. Some of these resources will be within the criminal justice system (e.g., a life skills class offered by the probation department), some will be in other public sector agencies (e.g., a job training program in the county’s office of workforce development), and some will be in the private sector (e.g., specialized substance abuse or mental health treatment).
  2. Create a matrix to collect specific information about each resource and research each to gather pertinent information. At a minimum, identify the specific services available through each resource, including their admission and selection criteria, duration/length, capacity, the criminogenic needs that the program addresses, and a rating (such as one from the CPC, CPAI, or a similar entity). You may also provide a brief description of the service. See Appendix 1 for a template of such a matrix.
  3. Compile your findings into a report or chart after the resource inventory is complete. Review the findings with your team to assess the resource inventory for completeness and to identify the strengths in your current array of services, as well as overlaps and gaps. Use this as an opportunity to consider ways to improve upon this array. For instance, should resources be shifted from one service area where there is an excess in service (e.g., where resources are devoted to non-criminogenic needs) to other areas where there are gaps? Appendix 2 provides a list of criminogenic needs and considerations around dosage, intensity, and duration of programming by risk level for adult offenders.
  4. Consider developing a directory of your resources so you can share this information with pertinent stakeholders in your jurisdiction (e.g., probation officers, prosecutors, defenders, judges).

Example:
Yamhill County, Oregon, Correctional Programming Options

(Abbreviated version of full document)

Community-Based Programs

Program Name

Moral Reconation Therapy (MRT)

Brief Description

Evidence-based program that encourages clients to develop prosocial thoughts and behaviors

Based on Curriculum-Based Motivation Group by Anne Fields of Portland, OR (2004)

Admission & Selection Criteria

Offenders sentenced to 6 months or greater

High and/or medium risk on the LSI-R

Potential candidates are identified through monthly review of the sentenced populations’ risk factors

Only offered to male clients

Female clients complete Phase II AMCP in gender-specific group, as MRT not currently offered to female clients

Program Length, Cycle & Dosage

Groups typically meet once per week for 13 weeks

MRT consists of 13 chapters and homework assignments

Clients may not miss more than two MRT sessions; otherwise, they will be directed to start over with AMCP Orientation and Phase I

Length of program varies depending on client participation, motivation, and progress

Minimum total dosage = 19.5 hours class time; estimated at 13 groups (13 assignments) of 1.5 hours each

Program Capacity

Class size is 12 persons

Only one group offered at a time

Open group

Members start and finish individually

Criminogenic Needs Addressed

Primary = Antisocial personality/Attitudes

Secondary = Alcohol/Drug

CPC (or other) Rating

This program has not been evaluated

 

Community-Based Programs

Program Name

Ready to Work (RTW)

Brief Description

RTW is an ongoing job development program created to assist probation clients in developing the tools necessary to be successful in finding employment.

 

RTW includes a comprehensive intake interview, career identification, and goal setting; 40 hours of unpaid practical work experience; career development workshops; and educational opportunities.

Admission & Selection Criteria

Sentenced, unemployed offenders

High and/or medium risk on the LSI-R

Potential candidates are identified through monthly review of the sentenced populations’ risk factors and/or court/PO referral as sanction/intervention

Program Length, Cycle & Dosage

Four-tier system:

- Tier I (30 days): daily reporting

- Tier II (11 classes): work groups through job source

- Tier III (until employment gained): independent (report once weekly)

- Tier IV (90 days): job success

In each tier, clients participate in the program five days per week, eight hours per day, unless otherwise directed.

Program Capacity

20persons maximum for Tier I, 10 persons for Tier II, and 10 persons for Tier III

Daily reporting and/or 1–2 employment classes/appointments per week, depending on phase

Total 40persons at any one time, in all three phases combined

Criminogenic Needs Addressed

Primary = Education/Employment

Secondary = Antisocial patterns

CPC (or other) Rating

This program has not been evaluated

 

Additional Resources/Readings

CSOM. (2007). Section three: An overview of the steps in the planning and implementation process. Enhancing the management of adult and juvenile sex offenders: A handbook for policymakers and practitioners (pp. 29–56). Retrieved from https://cepp.com/wp-content/uploads/2020/01/6-CSOM-Handbook.pdf

McGarry, P., & Ney, B. (2006). Getting it right: Collaborative problem solving for criminal justice. (NIC Accession No. 019834). Retrieved from https://nicic.gov/getting-it-right-collaborative-problem-solving-criminal-justice

Appendix 1:
Resource Inventory Matrix Template

County Name: Correctional Programming Options

Jail-Based Programs

Program Name

Brief Description

Admission & Selection Criteria

Program Length, Cycle, & Dosage

Program Capacity

Criminogenic Needs Addressed

CPC (or other) Rating

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Appendix 2:
Risk and Criminogenic Need Considerations

Dosage, Intensity, and Duration by Risk Level for Adult Offenders[1]

dosage, intensity, duration

What Are the Criminogenic Needs and their Implications for Intervention?

While the literature has slightly different ways of expressing criminogenic needs, generally they fall into the eight areas noted below.[2]

Top 4 Criminogenic Needs

Criminogenic Need
Response
History of antisocial behavior
Build non-criminal alternative behavior in risky situations
Antisocial personality pattern
Build problem solving, self-management, anger management, and coping skills
Antisocial attitudes, cognition
Reduce antisocial thinking; recognize risky thinking and feelings; adopt alternative identity/thinking patterns
Antisocial associates, peers
Reduce association with antisocial others; enhance contact with prosocial others

 

Next Four Criminogenic Needs

Criminogenic Need
Response
Family and/or marital stressors
Reduce conflict; build positive relationships and communication
Lack of employment stability, achievement; lack of educational achievement
Increase vocational skills; seek employment stability; increase educational achievement
Lack of prosocial leisure activities
Increase involvement in and level of satisfaction with prosocial activities
Substance abuse
Reduce use; reduce the supports for substance-abusing lifestyle; increase alternative coping strategies and leisure activities

 

References:

Andrews, D. A. (2007). Principles of effective correctional programs. In L. L. Motiuk & R. C. Serin (Eds.), Compendium 2000 on effective correctional programming. Retrieved from http://www.csc-scc.gc.ca/text/rsrch/compendium/2000/chap_2-eng.shtml

Andrews, D. A., Bonta, J., & Wormith, J. S. (2006). The recent past and near future of risk and/or need assessment. Crime & Delinquency, 52(1): 7–27.

Bourgon, G., & Armstrong, B. (2005). Transferring the principles of effective treatment into a "real world" prison setting. Criminal Justice and Behavior, 32: 3–25.

[1] Bourgon & Armstrong, 2005; for more information see the Coaching Packet on Effective Case Management, available at https://www.yumpu.com/en/document/view/35063381/coaching-packet-effective-case-management-the-center-for-/32

[2] Andrews, 2007; Andrews, Bonta, & Wormith, 2006, p. 11; for more information see the Coaching Packet on Implementing Evidence-Based Practices, available at https://www.yumpu.com/en/document/view/35063381/coaching-packet-effective-case-management-the-center-for-/32

3d: Gathering Baseline Data

3d: Gathering Baseline Data web_admin

Introduction

The first step in any change initiative is deciding what you intend to change. As part of the EBDM initiative and designing the system-level logic model, policy teams should have already made some decisions about goals and objectives (i.e., what long-term changes they hope to achieve) and about the types of procedural and policy changes that need to occur in order to attain the goals and objectives. The next to step is to understand more about these changes by gathering baseline data. Baseline data for the EBDM initiative can be defined in three broad categories:

  • case processing data;
  • data about the offender population; and
  • harm reduction data.

Case processing data and data about the offender population provide critical information about the primary issues addressed with the EBDM initiative. This data is particularly useful in shaping the initiative and refining the approach laid out during the initial planning. Gathering data about the intended harm reduction outcomes lays the foundation for evaluating the effectiveness of the EBDM initiative, allowing for an assessment of how much change has occurred since implementation.

Purpose

This Starter Kit—which should be used following system mapping activities and the development of a system-level logic model—is designed to provide an overview of the types of data that a jurisdiction might want to collect to establish a baseline for the EBDM initiative. The focus is on case processing and offender population data.[1] Guidance on data collection strategies and analysis is provided, along with suggested next steps to prepare for the collection of harm reduction data. Using the information in this Starter Kit will help jurisdictions produce reports on current case processing policies and practices, what is known about the offender population, and what is known about the harms in your jurisdiction that are intended to be addressed through the EBDM initiative.

Participants

Partnering with Universities

The policy team in Grant County, Indiana, benefits from a longstanding relationship with a local professor at Indiana Wesleyan who conducts an ongoing evaluation of the drug and reentry courts.

Participants in the collection of baseline data should include representatives of the policy team to help articulate the actual data elements to be collected, as well as personnel in the various agencies/organizations from which the data will be drawn. Ideally, agency personnel would include those staff members responsible for designing or producing reports from data management systems. To the extent that new data will need to be collected, jurisdictions may wish to work with outside experts to help develop data collection instruments, codebooks, and data management systems. Local universities are a good resource for assistance with data collection, as is staff in county or agency IT departments.

Instructions

The idea of gathering data may seem straightforward at first blush: identify the data reports that you want about case processing or about the offender population and analyze them. This type of approach, however, is likely to prove unmanageable at best and meaningless at worst. Too much data will overwhelm the process; people simply cannot interpret a lot of data with no analytic frame of reference. Too little data will not yield reliable information about trends and patterns and is likely to raise more questions than it answers. These instructions are intended to help policy teams frame the data-gathering activities in terms of what to collect and how.

The first step in gathering data is to decide what you want to look at and what questions need to be answered. Once this has been done, it is important to decide on a methodology for data collection and then to collect and analyze the data.

Deciding What Data to Gather

As part of the system mapping and development of the logic model, you have undoubtedly highlighted areas in which you would like to have more information. These areas form the basis for the questions that you are hoping to begin answering with your baseline data. For example, why are there so many dismissals, why does it take so many months to adjudicate a case, why are so many defendants held pre-trial, who makes up the pretrial jail population, how many people in the jail have mental health challenges, etc.

In deciding what data to gather, it is important to define the exact data elements that you want to collect:

  • Be sure to define key terms. Some examples include the following:
    • What constitutes a case? Is a case counted by charge or by defendant? Is it counted consistently across the system?
    • How will recidivism be defined? Is it an arrest for a new offense, or can it include sanctions for technical violations of probation?
  • Specify the data parameters, such as the time period from which the data will be gathered:
    • Will you gather a year’s worth of data, or several months’?
    • Will you collect data related to offenders in the system on a given day, and if so what day?
    • Are you interested in new arrests of offenders 90 days, 6 months, 1 year, 3 years, etc. after they complete their sentence?

Potential Definition of a Case

An example of one way to define “case” for the purpose of data collection is by incident. In this instance, an individual person who is involved in the criminal justice is only counted once regardless of the number of criminal charges pending from a single arrest. A person involved in multiple incidents over time (e.g., three arrests in one year) would be counted as three “cases.”

Identify what the unit of analysis will be:

  • Will you count people, charges, cases, beds, offense types, dollars, etc.?

Case Processing Data: Information related to case processing can focus on a variety of factors related to the nature and type of cases in the system as well as the volume and flow of cases. Depending on your policy team’s goals and the capabilities of your data systems, you may want to collect case processing data throughout the “life” of a case, or as early as arrest, booking, and filing/charging. Types of information that might be sought include the following:

  • number of cases by case type;
  • number of pending cases;
  • age of pending cases;
  • number of cases at different stages in the case processing continuum;
  • number of cases that proceed or “fall out” by decision point;
  • number and type of dispositions by case type;
  • number and type of release decisions by case type;
  • average sentence length;
  • number of probation revocations for technical violations and for new offenses;
  • number of bench warrants issued;
  • number of continuances; and
  • length of time between initial appearance and disposition by case type.

A caution for collecting case processing data, however, is to ensure that a “case” is defined the same way by all agencies providing the information. For example, police may count arrests as “cases,” prosecutors may count charges as “cases,” and so on. To avoid confusion, the definition of a case should be consistent across agencies.

Offender Population Data: Because data gathering can be labor intensive, the policy team should keep the data collection focused on the types of offenders or activities that you are planning to include as part of the EBDM initiative. So, if the focus is largely on pre-trial release, your data collection should focus on which offenders get released and by what means, and on which offenders do not get released. Conversely, if the intent is to focus on medium and high risk offenders for risk needs assessment at sentencing, then your data collection should focus on medium and high risk offenders.

No matter where in the system your data gathering will focus or which types of offenders you might hone in on, some common types of data should be collected about the offender population:

Establishing a Data Committee

Prior to engaging in the EBDM initiative, Ramsey County, Minnesota, had established a Data Committee, which was composed of data, research, and IT staff from key criminal justice agencies, including the courts, corrections, human services, District Attorney’s office, and others. Historically, the Data Committee would be “activated” to assist with various cross-agency projects, and it was activated again for the EBDM initiative.

Throughout Phase II of EBDM, the Data Committee met regularly and assisted in defining the baseline data, key measures, and other data elements necessary to measure the County’s performance under EBDM. The committee reported regularly to the EBDM policy team on progress in the collection of data, assisted in the system mapping process, and gathered data to respond to policy team members’ questions.

The Data Committee was instrumental in identifying and resolving potential roadblocks in data collection and developing common definitions for measures (which differed across individual agency information systems) across all agencies.
  • demographic characteristics of offenders;
  • criminal histories; and
  • previous sanctions and sentence lengths.

Harm Reduction Data: The types of harm reduction data will depend on the specific harm reduction goals defined and desired as a result of EBDM. However, some general guidance includes

  • incorporating data from other governmental systems, as appropriate, to include as examples
    • the number of people engaged in mental health services outside of the criminal justice system; and
    • emergency room admissions.
  • conducting primary research on areas of interest, for example
    • victim satisfaction surveys;
    • analysis of cost-benefits; and
    • comparative analysis of justice spending versus non-justice spending.

Information Gathering and Analysis Methods

There are many different techniques and approaches to gathering information. Because the information being collected under the EBDM initiative will be largely quantitative (i.e., numeric), only those approaches that are appropriate for quantitative data are described. There are two major types of data collection methods:

  1. primary data collection: development of surveys, questionnaires, and data collection forms to collect information that does not already exist in another form
  2. secondary data collection: collection of information from pre-existing datasets and data sources, such as case management systems

For the most part, you will be relying on secondary data collection. However, if you decide that primary data collection is necessary, it is highly advisable to work with an outside expert—perhaps from a local university—to develop and test any data collection instruments to ensure that the information gathered is reliable and valid.

With secondary data collection, the method you select – and the amount of external assistance you may need – will largely depend on the data-generating capacity of your system and the areas on which you have decided you need more information. Certain types of data collection and analysis will be more appropriate for understanding baseline case processing information, whereas other methods may be more appropriate for understanding the offender population. Thus, being clear about what you want to gather information on before you determine how (and from where) you will gather it is critically important. With this in mind, there are several different approaches that may be used in your information gathering:[2]

  • Pipeline analysis, in which a specific cohort of arrestees is selected and data is collected on them through their passage into and out of the criminal justice system.[3] The unit of analysis is the individual; the analysis focuses on counting people. A pipeline approach is used to track the number of people in the system at every decision point and attrition at the point at which people exit the system. So, for example, a pipeline analysis starts with the number of people arrested, how many received citations, how many were booked into jail, how many were released after booking, how many were charged with offenses, how many were not charged, how many were convicted, how many were sent to diversion or deferred prosecution programs, how many were sentenced to jail, how many received suspended sentences, how many were placed on probation, etc. One way to think about this pipeline analysis is to determine the quantities of some or all of the process steps and decision points on your system map for a given period of time.
  • Time analysis, in which the unit of analysis is either the individual defendant/offender or the case. The focus of a time analysis is to understand the amount of time associated with different aspects of the system or how long a particular process takes. So for example, a time analysis might look at the number of days individuals are detained in jail pre-trial, elapsed time from charging to disposition, amount of time in diversion or under supervision, length of time between preliminary hearing and trial/plea, length of time between violation and court action on the violation, etc. Time analysis data can also be noted on the system map, reflecting the lapse of time between one step (or decision point) and another.
  • Jail analysis, in which the focus is to develop a thorough understanding of persons who are booked into the jail, the length of time they are in jail, and their status during the stay (e.g., pretrial, sentenced, DOC transfer, probation violation, etc.).[4]
  • Comparative analysis, which seeks to understand the differences between offenders in your population. For example, if you are concerned about compliance with terms of probation, this type of analysis will allow you to determine if there are differences between offenders’ outcomes based on risk level, types of conditions, types of present offense, prior criminal history, etc. Other questions that can be answered include, but are not limited to, issues related to who is placed on pretrial detention, the types and lengths of sentences/conditions, case outcomes, etc.

Tips

  • As you define what information you would like to gather, identify the source of the information, including specifics about which agency has the data, the name of the person who will generate the report, and the time frame for delivering the data to the policy team.
  • Be specific about the data to be collected. Identify if you want summary data, data aggregated into totals, or some other format; how you want the information broken down (e.g., by type of offense, by justice system status, etc.); the time frame from which the information is being drawn; etc. The more details in the information-gathering instructions, the better.
  • It is often useful to mock up an example of a report that shows how you want to see the information and to provide that information to whomever is gathering the data. Doing this will provide additional clarity about what the data should look like once it is generated or extracted from existing information systems.

Example:
Ramsey County, Minnesota, Baseline Data Collected on Warrants

baseline data

 

active warrants

Example:
Key Findings from a Jail Analysis in One County

One EBDM site enlisted the help of an outside expert to conduct an analysis of the jail population. The consultant looked at a sample that consisted of 10% of the inmates released from the jail during 2010 (N = 533). As a result of the analysis, a number of observations and recommendations for reducing the jail population were presented, including the following:

  • 60% of all jail bed days were being used by those who had a probation violation associated with their booking. Officials should examine current policies and procedures for dealing with probation violation cases with the goal of trying to reduce, if possible, the time required to resolve them.
  • Defendants released on signature bonds had an average length of stay of 7 days and consumed 87% of the jail bed days for the pretrial release population. Jail officials should determine how they can ensure that signature bond decisions will be made on the first day in order to have a significant impact on the overall jail population.
  • While Native Americans released during the pretrial period represented 4% of all those released pre-trial, they consumed 20% of all jail bed days used by all inmates released pretrial. Officials should look into whether this population has higher Failure to Appear (FTA) rates; if so, they may benefit from implementing court date reminder procedures, which have been very effective in reducing failure-to-appear rates in other jurisdictions.

A key finding from this analysis led to a recommendation to improve the collection of baseline data in the jail. The county jail, like many across the country, overwrites the legal status at booking (e.g., pretrial, probation violation, sentenced) when the status changes. Therefore, important data is lost regarding the reason individuals are initially booked into jail. This finding led to a relatively easy fix: developing a field for “legal status at booking” and a separate field for “current legal status.”

Example: Charlottesville-Albemarle County, Virginia, Steps to Calculate Baseline Data on Costs

Additional Resources/Readings

Carter, M., & Morris, L. (2007). Enhancing the management of adult and juvenile sex offenders: A handbook for policymakers and practitioners. Retrieved from https://cepp.com/wp-content/uploads/2020/01/6-CSOM-Handbook.pdf.

McGarry, P., & Ney, B. (2006). Getting it right: Collaborative problem solving for criminal justice (NIC Accession No. 019834). Retrieved from https://nicic.gov/getting-it-right-collaborative-problem-solving-crimina...

Rossman, S. B., & Winterfield, L. (2009). Measuring the impact of reentry efforts. Retrieved from https://cepp.com/wp-content/uploads/2015/12/Measuring-the-Impact-of-Reen...

 

[1] Harm reduction data is a third important component of information-gathering, specifically as it relates to outcome evaluations that are conducted; as such, harm reduction data are not discussed in detail in this document.

[2] The different approaches are not mutually exclusive; depending on what you are trying to understand, you may find that using two or more approaches provides a significantly clearer picture of the issues than using a single approach.

[3] McGarry & Ney, 2006.

[4] See McGarry & Ney (2006) for more detailed information about the analysis of the jail population.

3e: Prioritizing Your Team’s Targets for Change

3e: Prioritizing Your Team’s Targets for Change web_admin

Introduction

Considering the information collected through various policy and practice analyses is no easy feat. It is more than likely that your analyses will surface a variety of possible areas of improvement. This may present the team with some tough choices regarding its highest priorities for action. This document outlines a process your team might use for culling through the information collected through your policy and practice analyses and selecting your priority change targets.

Purpose

Reporting to the Policy Team: Work Group Findings

During their strategic action planning session during, Grant County, Indiana’s work groups were asked to report out on

  • their top 3–5 action items/change strategies for improving risk reduction outcomes;
  • the evidence-based research that supports the action/change and other supporting reasons for the proposal;
  • a brief analysis of the pros and cons of each action item;
  • action items/change strategies that were considered and then “taken off the table,” and why.

To agree, as a team, on the most significant opportunities to advance policy and practice to achieve the jurisdiction’s harm and risk reduction goals

Participants

All policy team members should be present and actively involved in considering work groups’/outside experts’ findings from the policy and practice assessments.

Instructions

  1. Plan a day-long strategic planning session (or a series of meetings) to conduct this work.[1]
  2. In advance of the meeting, ask policy team members to review the findings from work groups/outside experts regarding the extent to which policies, practices, and key decisions are supported/informed by research.
  3. At the beginning of the meeting, take a moment to agree as a team how decisions will be made for selecting the change strategies on which the team will move forward.
    1. Decide on the extent of agreement needed to make a decision. Will decisions be made by majority vote or through consensus? What does consensus mean? It may be unrealistic to assume that every team member will completely agree on the top change targets to pursue. One strategy for making decisions is for team members to agree to make an honest effort to understand what is being proposed and to determine whether each member “can live with it and support it.”
    2. Set criteria to determine which change targets will be prioritized. Some factors to consider may include: What is the potential impact of this change on risk and harm reduction? Do we agree the research support is strong enough to consider this change? Do we think this strategy is realistic, given what we know about our current system’s challenges?
  4. Once the team is clear on the way it will make decisions, work groups should report on their recommendations, on the level of research behind each recommendation, and on how the recommendations came about.
  5. For each work group recommendation or change strategy, consider it against the selection criteria and discuss whether team members agree that the change strategy is a priority. This may be accomplished through group discussion and/or ranking, scoring, or voting methods.
  6. If the team has already discussed its harm reduction goals, it will be necessary to consider these goals as part of the decision-making process for selecting change targets. (If the team has not yet determined its final harm reduction goals, it should do so as part of its efforts to establish performance measures and outcomes, and to develop a systemwide scorecard.[2])
  7. Once a set of change targets is formed, revisit them as a package and determine if anything is missing or if strategies need to be further developed. The team may decide that it wants the work groups that developed the recommendations to create more detailed descriptions of the scope of work and action steps to implement the proposed changes.[3]

Example:
Grant County, Indiana, Strategic Planning Session Goals and Agenda

Goals

  • The first goal is to propose, describe, and weigh the pros and cons of the top change strategies recommended by the various work groups.
  • The second goal is to prioritize the change strategies according to those that will produce the greatest impact and, as a second consideration, those most feasible to pursue in the next 18–24 months. Other change targets may be critical to long-term improvement in outcomes but will fall into a second phase of implementation.
Agenda
8:00 a.m. Welcome; agenda review
8:15 a.m. Review and discuss ground rules regarding how the EBDM policy team will make decisions about change targets
8:30 a.m. Grant County EBDM Vision Statement: The Critical Context for Choosing Change Targets
9:00 a.m.

Pretrial Work Group: Proposals for Top Change Targets

This will be the first of a series of reports from the work groups regarding proposed change targets and goals. Each work group will

  • describe each proposal, including the work group’s analysis of the pros and cons of the proposal
  • identify, with the participation of the full EBDM team, on flip chart paper
    • the agencies involved;
    • each proposal’s likely and logical contribution to harm reduction;
    • the estimated time required for implementation;
    • the supporting research, where possible; and
    • proposals that were considered and set aside, and the rationale for these decisions.

The full team is encouraged to ask clarifying questions, contribute to the discussion of pros and cons, and offer other strategies they would like to see considered at each decision point.
10:00 a.m. Break
10:15 a.m. Pretrial Work Group (continued)
11:00 a.m. Community Interventions Work Group: Proposals for Top Change Targets
12:00 p.m. Lunch
12:45 p.m. Violations Work Group: Proposals for Top Change Targets
1:45 p.m.

Change Targets on Other Decision Points

  • All team members will have the opportunity to nominate change targets that fall outside the purview of the three work groups.
2:15 p.m.

Review of the “Big Picture” of Change Targets and Setting Priorities

  • Prioritizing the change targets that will have the greatest impact on harm and risk reduction in Grant County
  • Understanding the change targets that are both foundational steps and feasible for implementation in the next 18–24 months

If consensus does not emerge from the day’s discussion, or possibly to support that consensus, members will use colored dots beside each change strategy listed on the flip chart paper:

  • red to indicate the most impactful change strategies; and
  • blue for those change strategies that would be most feasible within current resources.
3:00 p.m.

Action Planning for Next Steps

  • The session will close with a clear list of priorities for the next two months’ work (i.e., development of action plan, scorecard, logic model, communications strategy).
3:30 pm Adjourn

Example:
Milwaukee County, Wisconsin, EBDM Proposal Scoring Tool

Criteria

Scale

Weight of Overall Score

Integrality

  • How closely does this proposal tie into the vision statement (stewardship/reducing recidivism/collaboration/harm reduction, etc.)?

1 = whether it’s a good idea or not, it’s just not a strong example of EBDM principles at work

10 = epitome of EBDM principles at work, and especially useful in addressing the issues the criminal justice system in Milwaukee County needs to tackle

14.285%

Predictability

  • To what extent does existing research suggest the proposal will be successful?

1 = in essence, the proposal is a hypothesis that hasn’t been tested anywhere else we know of

10 = solid research shows this has been a winner in similar circumstances in other jurisdictions

14.285%

Novelty

  • How innovative is the proposal?

1 = it may be somewhat embarrassing to have to explain why we aren’t doing this already

10 = someday someone will call this “the Milwaukee ________”

14.285%

Supportability

  • To what extent do we have baseline data about the issue the proposal addresses, and to what extent do we have data collection systems in place that will help us track progress and success (or lack of it)?

1 = considerable effort will be needed to collect data about existing practices and the results of the project as we implement it

10 = current, easily accessible data about our practices already exists and data collection systems are already in place that we can use to track progress

14.285%

Impressiveness

  • How big a hit will this be if it’s successful?

1 = barely worth the effort

10 = candidates for public office will jockey to take credit for this idea

14.285%

Scorability

  • How measurable are the projected results? Can the results be evaluated in terms of our overall scorecard?

1 = the aspirations of the proposal are not quantified and the proposal makes no suggestion of how they might be

10 = the proposal contains a specific, quantified estimate of costs savings, reduction in recidivism, harm reduction, etc., and a firm methodology for conducting future measurements of actual performance against the estimate

14.285%

Feasibility

  • Do we have the financial and infrastructure capacity to implement the proposal immediately or must additional resources be sought?

1 = it is unlikely that necessary budgetary resources can be obtained, or necessary infrastructure developed, or both

10 = no additional budgetary resources or infrastructure are needed to implement the proposal

14.285%

 

[1] Consider the use of an outside facilitator to ensure that the meeting is as productive as possible.

[2] See 6a: Measuring Your Performance and 6b: Developing a Systemwide Scorecard.

[3] This activity may naturally lead to work on system- and/or case-level logic models. See 5a: Building Logic Models.

4: Capacity to Implement

4: Capacity to Implement web_admin

Activity 4:
Understand and have the capacity to implement evidence-based practices

For evidence-based decision making and practices to take hold, policymakers and professionals across the system must

What Do We Mean By "Research Evidence"?

In the justice system, the term "evidence" is used in a variety of ways. It can refer to items collected at a crime scene, eyewitness accounts, or security camera footage. These types of evidence are referred to as "legal evidence."

For the purposes of the EBDM initiative, however, the term "evidence" is used to describe findings from empirically sound social science research. The initiative refers to the results of this research as "evidence-based policy and practice."
  • share a common understanding of the key research findings underlying harm and risk reduction; and
  • possess a core set of skills, or competencies, to effectively apply these empirically based research findings to their day-to-day practices.

Through the assessment and bolstering of professionals' level of knowledge and skills, changes in attitudes, activities, and adaptation to new technologies occur, and pathways to constructive modifications in policy and procedure are opened—even in the face of unprecedented work demands and ever-tightening resources.

Elements of an EBDM justice system include

  • a common understanding of the research (and its limitations) across all relevant agencies/staff;
  • an understanding of the implications of these findings; and
  • a workforce skilled in effectively carrying out evidence-based practices.

4a: Understanding Your Agency: Conducting an EBP Knowledge Survey

4a: Understanding Your Agency: Conducting an EBP Knowledge Survey web_admin

Introduction

For a jurisdiction to be truly evidence-based, the professionals managing its justice system agencies[1]—and those working directly with the defendants and offenders in that system—should all share a common base of knowledge about key findings in the correctional literature. A knowledge survey can serve as a diagnostic tool to determine the level of current knowledge among these parties and potentially identify needs for knowledge enhancement. If it is determined that further knowledge development is needed, steps should be included in the jurisdiction’s strategic plan to enhance knowledge among key stakeholders and agency staff. This document provides a process for administering a knowledge survey and considering the implications of its results for the team’s future work.

Purpose

  • To assess the level of evidence-based knowledge among policy team members and of select groups of staff within their agencies.
  • To identify action steps to address gaps in knowledge as identified through the survey.

Participants

While the policy team will ultimately decide who will take the survey, it is recommended that knowledge survey participants include a sample of staff from all agencies represented on the team.

Instructions

Measuring the Impact of an Event

In Yamhill County, Oregon, the policy team used the EBP knowledge survey to measure changes in knowledge regarding EBP after a day-long awareness-building event for system stakeholders.

A greater percentage of respondents answered 17 out of 18 questions correctly following the event than prior to the event. These findings suggest that participation in this event had a positive impact on participants’ level of knowledge about various evidence-based practices.

  1. During a policy team meeting, discuss the reason(s) why a knowledge survey is important to the team’s work and how it will aid in the development of the team’s strategic plan.

    1. Determine what kind of knowledge (i.e., information that has been demonstrated to rise to the level of “evidence” and therefore be used to inform decision making) the team wants its members to know. From this feedback, either develop a knowledge survey or use the one in Appendix 1.
    2. If the policy team decides to develop its own knowledge survey, it is recommended that the survey be reviewed by a respected researcher and pilot tested with practitioners who are knowledgeable about the research before it is administered.
    3. Distribute the knowledge survey to policy team members.
    4. Survey respondents should not be coached in advance as to the specifics of the survey or possible answers. Results will only reflect accurate indicators of existing agency knowledge if the survey is administered without foreknowledge of content.
    5. Explain that not all agency staff will be surveyed. Instead, a selected group of policy and line level staff will be anonymously surveyed.
  2. Secure a commitment from each policy team member to fully participate in the survey process.
  3. Following the team meeting, discuss with each agency’s lead policymaker[2] which of their staff members should participate in the survey. It is recommended that only those who are in charge of policy decisions (i.e., directors/managers/supervisors) and those with direct contact with defendants, offenders, and victims be included in the survey.
  4. Calculate the number of staff persons who will participate in the survey, based on the criteria agreed to in #3. This will be the total survey “pool” for each agency. If possible, the survey should be sent to all individuals in this survey pool.
  5. The survey may be distributed in hard copy or through an automated online survey mechanism such as Survey Monkey.
  6. Communicate with agency staff in advance about the purpose of the survey. Some recommended email scripts are provided in Appendix 2.

  7. Once the survey results are compiled, review them during a policy team meeting.

    1. Determine how confident you are that the survey results reflect the knowledge of the pool of staff in each agency. Keep this in mind as you discuss the results. (For instructions on how to do this, see Appendix 3.
    2. Determine as a team the threshold for a satisfactory and unsatisfactory knowledge level.
    3. Determine whether the survey results were satisfactory (as determined by the team). If not, for which agencies were the results unsatisfactory?
    4. For those agencies with unsatisfactory knowledge survey scores, determine what action steps the team wants to take.
    5. If one or more action items are identified to enhance knowledge, consider, as part of your action plan, re-administering the knowledge survey after these efforts are completed to determine whether the knowledge gaps have narrowed. In this case, use the same process to determine the survey sample for re-administration as you used for the initial survey. The same individuals do not need to be surveyed; however, the same protocol for identifying the survey group must be followed.

Tips:

Using an EBP Survey with Agency Leaders

In Charlottesville-Albemarle County, Virginia, the director of the local community corrections agency used the knowledge survey with her statewide agency heads to test their knowledge of EBP principles and practices, and gave out prizes for the highest scores.
  • Make sure that survey participants understand the purpose of the survey. (See the email template below.)
  • Allow, at a minimum, two weeks (10 business days) for participants to complete the survey.
  • Send an email to the survey participants at least two days before the beginning of the two-week survey period to let them know that they are being asked to participate in the survey. (Make sure the survey is up and running early, in case participants want to fill out the survey immediately!) It is preferable that this email be sent by the agency policymaker or by another person in a leadership position. See Appendix 2 for ideas about what this email should contain.
  • Send out one reminder during the two-week survey period.
  • If, after ten business days, all of the surveys are not returned, send out another reminder indicating that, although the deadline has passed, it is important that each person participate and, therefore, the deadline has been extended for three more business days.

Additional Resources/Readings

The knowledge survey contains information collected from numerous research studies. See NIC’s Evidence Based Decision Making Framework[3] for research studies that include some of the key findings.

Appendix 1:
EBP Survey with Answer Key

 

Evidence-Based Practices Survey

Answer

1

True or false? Most offenders don’t handle stress well, so anxiety and stress reduction programs such as yoga and meditation are helpful in reducing the likelihood of rearrest.

F

2

Of the following, which is the most effective method to determine the likelihood that an individual will be rearrested?

  1. A validated assessment instrument
  2. Professional judgment by an individual with extensive experience
  3. None of the above; there is little evidence that an instrument or professional judgment predicts arrest patterns

A

3

Which of the following interventions best reduces recidivism in the long term?

  1. Boot camps
  2. AA
  3. Sanctions for non-compliance
  4. Programs that build thinking skills
  5. Victim empathy classes

D

4

To best achieve offender behavioral change, professionals should use social learning techniques that reduce recidivism. Which of the following is not a social learning technique?

  1. Modeling: demonstrating desired behavior
  2. Reinforcement: rewarding behaviors you want repeated
  3. Redirection: confronting and redirecting antisocial thinking
  4. Relationship: developing a meaningful working alliance with the offender
  5. Leadership: enhancing character through opportunities to demonstrate leadership
  6. Practice: teaching concrete problem solving skills by having the offender practice the desired skills

E

5

Which of the following statements is true?

  1. Validated risk assessment instruments predict the likelihood of reoffense better than professional judgment.
  2. Validated risk assessment instruments do not work as well as the judgment of an experienced, veteran professional.
  3. Validated risk assessment instruments are so consistently reliable that professional judgment should not be used to override the results of the instrument.

A

6

Which individual trait or circumstance, when present in a person’s life, does not likely contribute to whether that person commits a crime?

  1. Depression
  2. Friends who have low regard for the law
  3. Low victim empathy
  4. Unemployment

A

7

True or false? Placing offenders with low self-esteem in programs that increase their confidence does not reduce the likelihood of rearrest.

T

8

True or false? It is generally true that treatment does not work in reducing rearrest rates.

F

9

True or false? Lack of employment is among the top four influences that can lead an individual to commit a subsequent crime.

F

10

True or false? Lack of education is among the top four influences that can lead an individual to commit a subsequent crime.

F

11

Which statement is most accurate when trying to maximize the effectiveness of programming for medium and high risk offenders?

  1. Treatment should be intense over approximately a four-week period.
  2. Treatment is largely ineffective for this group of individuals.
  3. Treatment should last as long as possible—preferably 24 months or more.
  4. Approximately 200 hours of treatment time gets maximum effect.

D

12

True or false? Failure of a sex offender to register has a direct correlation to convictions for new sex crimes.

F

13

Which one of the following programs reduces recidivism over the long term?

  1. Gardening and horticulture
  2. Yoga
  3. Drum circles
  4. Lectures designed to give insight
  5. AA
  6. None of the above

F

14

True or false? It is better to invest in interventions for low risk offenders than for high risk offenders because their criminal tendencies are less hardened, or “fixed.”

F

15

True or false? Programs like “Scared Straight” and boot camps are particularly effective for youthful offenders between the ages of 16 and 25.

F

16

True or false? Jails and prisons are effective in changing future offender behavior after release if the conditions are severe enough that the offenders don’t want to return.

F

17

True or false? Giving offenders positive reinforcement and feedback when they exhibit prosocial behaviors supports positive changes in the future.

T

18

True or false? Of those probationers who are revoked from supervision and incarcerated, most are not revoked from supervision for new crime behavior.

T

19

True or false? Medium and high risk offenders benefit more from punishment than treatment.

F

20

Research suggests that the onset and use of drugs is different for female offenders than for male offenders. Which one of the following statements is not true about female offenders when compared to male offenders?

  1. Women describe the onset of drug use as sudden, rather than gradual.
  2. Women tend to report that their drug use begins as a result of a specific emotion or situation, such as depression or family problems.
  3. Women experience the adverse physical effects of alcohol more slowly than men.
  4. Women are more likely than men to be introduced to drugs by a sexual partner and to continue using drugs to maintain the relationship.
  5. Women who abuse drugs have higher rates of childhood physical and sexual abuse than men and non-substance-abusing women.

C

Appendix 2:
Email Templates for Conducting the Knowledge Survey

  • Email 1

    Send this email a few days before the survey period. (Make sure that the survey is up and running before you send this email!)

    Dear Colleague,

    We would like to ask you to complete a short, online survey. It should take no more than 10 minutes of your time. The purpose of this survey is to determine the extent to which staff in our organization share common knowledge about key justice practices that are supported by research.

    A number of policy level and line staff members have been selected to participate in this survey, including managers and those who work directly with defendants/offenders. Your answers will remain completely anonymous.

    To take the survey, please click on the following link: xxxx.

    The survey will be open for the next two weeks. If you have questions about how to take the survey, please contact Name and Email.

    Thanks in advance for your cooperation.

    Agency Leader Name

  • Email 2

    Send this reminder email about halfway through the survey period.

    Dear Colleague,

    If you have not completed the Evidence-Based Practices survey, please do so as soon as possible.

    To take the survey, please click on the following link: xxxx. Please complete this survey by: date.

    Thanks,

    Agency Leader Name

  • Email 3

    Send this reminder the day after the survey period ends.

    Dear Colleague,

    Although the deadline has passed, we feel it is important that each person participate in the Evidence-Based Practices survey and therefore the deadline has been extended for three more business days.

    If you have not already completed the survey, please do so by date by clicking here: xxxx.

    Thanks,

    Agency Leader Name

Appendix 3:
Determining the Minimum Number of Responses Desired for Accurate Inferences

In order to make more accurate inferences about the larger population, it is necessary that a certain percentage of the individuals in the target pool complete the survey. To determine the minimum number of responses necessary for confidence in the results, visit the following website: https://www.surveysystem.com/sscalc.htm#one. (This website provides a tool for determining survey sample sizes.)

Follow these instructions:

  1. Only use the box entitled “Determine Sample Size.”
  2. Click on the button marked “Confidence Level 95%.”
  3. Enter the number 10 in the “Confidence Interval” box.
  4. Enter the number of persons in the survey “pool” (as calculated in #4 above) in the “Population” box.
  5. Click “Calculate.” The number displayed is the minimum number of respondents that are necessary to get results that reflect the target population.

 

Once this process is completed for each agency, make a list of the survey targets for the policy team using a chart similar to the one below. This will help document the survey pool, the number of surveys received, and level of confidence in the results.

 

Agency

Total # of Staff in Agency

Total # of Staff in the Pool for Participation in the Survey

(from #4 above)

Minimum # of Responses Desired

# of Responses Received

Received More than the Minimum and Therefore Confident in Results?

Court

12

10

9

4

No

Prosecution

53

46

31

37

Yes

Police

324

217

67

121

Yes

 

[1] Throughout this document, the term “agency” is used to represent independent entities such as a police department, court, organization of public defenders, community corrections agency, etc.

[2] For the purposes of this document, the “lead policymaker” is the person on the policy team representing his/her agency (e.g., chief of police, chief public defender, chief judge).

[3] NIC's Evidence Based Decision Making Framework.

4b: Equipping Stakeholders to Apply Research Evidence

4b: Equipping Stakeholders to Apply Research Evidence web_admin

Introduction

A justice system that seeks to use evidence to guide decisions must have as its foundation a group of professionals (both policymakers and practitioners) who share a core base of knowledge about evidence-based practices and an appreciation for the usefulness of research to system, agency, and case-level decision making.

While agencies/individual professional groups necessarily promote knowledge and skill development that is unique to their disciplines (e.g., firearm instruction for law enforcement, interpretation of law for lawyers, safety practices for correctional staff), the research around evidence-based practices as it relates to risk and harm reduction is applicable to, and crosses, all disciplines. So too is the practice of using research to inform decisions—regardless of field or specific decision point.

These common areas of knowledge and the practice of applying research to decision making comprise a core EBP training curriculum and strategy that is appropriate for all justice system professionals. A core, systemwide evidence-based training strategy has several benefits. It

Developing a Learning Culture

Justice systems become learning cultures when they grow dependent on information, particularly empirically derived information. Policy debates that once centered on philosophical beliefs or one’s personal experiences are quite different when the core of the discussion is based on what the evidence does or does not suggest. Individual case decisions that are guided by science rather than intuition or past experience often result in different actions and, fortunately, better outcomes. Learning cultures begin with a common base of knowledge and an appreciation for science.
  • promotes the notion that criminal justice professionals are each an important part of a system, rather than operating in individual “silos” that have little bearing on one another;
  • opens lines of communication by creating a common language across discipline groups;
  • provides an opportunity for an open exchange of ideas, airing of concerns, and joint problem solving; and
  • creates a culture of knowledge development and growth—a learning system—whereby knowledge acquisition and its application are encouraged and reinforced. To this end, training and knowledge development is a dynamic process. It does not occur once or for one period of time and then come to an end. As systems grow and evolve, as new staff are hired, as experiences are acquired, and as new research emerges, learning cultures grow and evolve too.

Purpose

To equip professionals across the justice system with a core base of evidence-based knowledge and an appreciation for the application of research evidence to their decision making.

Participants

The policy team will ultimately take the lead in developing and delivering a core EBP training curriculum to local stakeholders. However, other staff may be brought in as needed (to assist with surveying, curriculum development, training, etc.).

Instructions

  1. Assess training needs and develop a knowledge baseline. The first step in developing and delivering a core EBP training curriculum is to assess knowledge regarding evidence-based practices across professional staff in all justice system agencies. This can be done fairly efficiently through the use of a brief survey or some other structured or semi-structured method.[1] The survey results will provide aggregated baseline data for each responding agency.[2]
  2. Review the knowledge survey results, identify information gaps, and build training plans. The results of the knowledge survey will provide clear and specific information about training needs across the system and within each discipline. This information should be used to guide the development of a systemwide training strategy and to inform agency-specific knowledge development efforts.
    1. In building a training strategy, keep in mind that although professionals across the system have a common need for core information, some will require more in-depth knowledge of some subjects while others may not.
    2. Also keep in mind that, in many areas, understanding the research and applying it requires the development of new skills. Skill-development training is quite different from knowledge-development training. It typically requires more time and always requires the opportunity to practice new strategies and techniques and receive feedback on the application of the new skills. The appendix illustrates some of the core EBP research topics and the stakeholder groups for whom each topic might be most relevant. Items in red are knowledge-based training objectives, while items in gray are skill-based training objectives.
  3. Customize the training to meet individual agency, as well as jurisdictional, needs. Development of a training strategy should take into account several questions:
    1. Should training delivery be directed to a single audience (e.g., judges), should it be directed to partially mixed audiences (e.g., judges, prosecutors, defenders), or should it be fully multidisciplinary? There are pros and cons to each choice. Keep in mind that when a core set of messages or base of knowledge is delivered, it is often beneficial for all stakeholders to receive this information at the same time. This helps support the goal of cross-agency collaboration.
    2. Should training be delivered by local staff/experts, external experts, or some combination of these? There are also pros and cons to each of these options. Regardless of the choice made, ensure that the selected trainers have sufficient knowledge and credibility with respect to the training topics that they will be received favorably by the audience.
    3. Should training cover basic information, advanced information, or both? The training needs assessment is likely to uncover some “unevenness” in the needs of cross-agency groups and within single-agency groups. Determining how to address individual needs within the context of a large number of potential trainees will be an important point to consider. Regardless of the final methods selected, be careful to avoid a “one size fits all” approach by assuming that what is right for one group will be right for all. It may be advisable to develop a customized needs approach—within the overall context of what is possible—by engaging the various stakeholder groups in discussions around how best to meet their needs.
  4. Conduct the training and then elicit feedback. Plan for the collection of post-training feedback prior to the training event. This could be in the form of re-administering the knowledge survey to determine if knowledge gaps were lessened (see below), an open discussion about the gaps filled by the training or lingering questions, and/or a written follow-up survey to determine if participants believe that the training goals were met and/or whether they identify additional training needs and, if so, which ones. Regardless of the method selected, feedback from the training will help determine where additional or more targeted training is needed. These needs should be incorporated into the systemwide training strategy.
    1. Re-administer the knowledge survey following the delivery of core EBP training material and compare the results with those of the initial survey to determine if knowledge levels improved sufficiently or if additional training is necessary. Then develop a “next stage” training plan, as needed.
    2. Identify methods to assess evidence-based knowledge over time and continue to build an ongoing, systemwide training strategy.

Example:
Knowledge Survey Summary Report from One County

The following chart is an excerpt from a larger report that summarizes the percentage of respondents in a select number of agencies who provided the correct answer to each question in an EBP knowledge survey.[3] This report highlights the differences between four agencies.

Question

Agency 1

Agency 2

Agency 3

Agency 4

All Respondents

 

N=20

N=16

N=22

N=106

N=228
  1. Most offenders don’t handle stress well, so anxiety and stress reduction programs such as yoga and meditation are helpful in reducing the likelihood of rearrest.

25%

88%

45%

59%

58%
  1. Of the following, which is the most effective method to determine the likelihood that an individual will be rearrested?

85%

100%

55%

39%

50%
  1. Which of the following interventions best reduces recidivism in the long term?

100%

94%

45%

67%

65%
  1. To best achieve offender behavioral change, professionals should use social learning techniques. Which of the following is not a social learning technique that helps reduce recidivism?

45%

38%

40%

13%

20%
  1. Which of the following statements is true about the use of validated risk assessment instruments to determine risk of reoffense?

90%

75%

62%

52%

59%
  1. Which individual trait or circumstance, when present in a person’s life, does not likely contribute to whether that person commits a crime?

65%

69%

43%

35%

45%
  1. Placing offenders with low self-esteem in programs that increase their confidence does not reduce the likelihood of rearrest.

5%

56%

33%

35%

34%
  1. It is generally true that treatment does not work in reducing rearrest rates.

95%

69%

81%

73%

75%
  1. Lack of employment is among the top four influences that result in whether an individual commits a subsequent crime.

10%

19%

30%

13%

18%
  1. Lack of education is among the top four influences that result in whether an individual commits a subsequent crime.

15%

63%

14%

21%

23%
  1. Which statement is most accurate when trying to maximize the effectiveness of programming for medium and high risk offenders?

21%

7%

10%

10%

10%
  1. Failure of a sex offender to register has a direct correlation to convictions for new sex crimes.

85%

38%

91%

31%

44%
  1. Which one of the following programs reduces recidivism over the long term?

55%

69%

38%

61%

57%
  1. It is better to invest in interventions for low risk offenders than for high risk offenders because their criminal tendencies are less hardened, or “fixed.”

90%

87%

60%

53%

56%
  1. Programs like “Scared Straight” and boot camps are particularly effective for youthful offenders between the ages of 16 and 25.

80%

81%

48%

47%

51%
  1. Jails and prisons are effective in changing future offender behavior after release if the conditions are severe enough that the offenders don’t want to return.

95%

94%

86%

62%

72%
  1. Giving offenders positive reinforcement and feedback when they exhibit prosocial behaviors supports positive changes in the future.

100%

100%

100%

85%

90%
  1. Of those probationers who are revoked from supervision and incarcerated, most are not revoked from supervision for new crime behavior.

75%

63%

71%

66%

64%
  1. Medium and high risk offenders benefit more from punishment than treatment.

95%

100%

91%

65%

71%
  1. Research suggests that the onset and use of drugs is different for female offenders than for male offenders. Which one of the following statements is not true about female offenders when compared to male offenders?

40%

31%

33%

20%

27%

Appendix:
Targeting Core EBP Knowledge Areas to Individual Stakeholder Groups

Targeting core ebp knowledge areas

[1] See 4a: Understanding Your Agency: Conducting an EBP Knowledge Survey for one example of a tool that can be used.

[2] See the Appendix for an illustration.

[3] For more information, see 4a: Understanding Your Agency: Conducting an EBP Knowledge Survey.

4c. Becoming a Better Consumer of Research

4c. Becoming a Better Consumer of Research web_admin

Introduction

The EBDM Initiative seeks to help local policy teams find and understand evidence-based knowledge about effective justice practices and to design more effective responses to defendants and offenders.[1] Many stakeholders already know how to find and use research; others will appreciate these tips regarding how to quickly access reliable research and how to review and understand the findings and their applications. The evidence or empirical studies will be drawn from many fields: evidence-based practices in criminal justice, behavioral health interventions, organizational development, leadership and management, effective collaboration processes, and cost–benefit analyses.

Purpose

Defining EBP

"Evidence-based practice is the use of direct, current empirical evidence to guide and inform effective and efficient decision making and supervision practices."

Broadly speaking, the goal of this document is to increase policy officials’ and practitioners’ skills in finding the research that matters and in understanding and translating empirical findings for their use in improving policy and practice. Specifically, this document offers

  • tips for finding research relevant to critical questions about evidence-based practice;
  • a list of searchable databases on criminal justice topics; and
  • advice on how to review and assess the quality of the findings in academic articles and the research literature.

Participants

This document was developed for EBDM policy teams, their work groups, and agency practitioners to enhance their ability to find and understand the best available research that may be applied to criminal justice problems and proposed solutions.

Instructions

Step 1: Look in the Right Places to Find the Evidence that Matters

Where should the discerning consumer begin the search for evidence-based policies and programs and answers to specific research questions? The answer is three-fold: the Web, written literature, and experienced colleagues from your local and state criminal justice systems and from national networks of professionals.

Websites that Filter the Information for You: Evidence-Based Program Databases

Websites designed specifically to summarize research in one or more criminal justice practice areas are an excellent place to begin the search for information on effective programs and policies. A growing number of government agencies, academic institutions, and professional groups maintain these databases as a service to criminal justice professionals and the public. These organizations

  • formulate evaluation criteria for assessing the strength of research findings;
  • employ experts to review multiple studies of research on programs in a single area; and
  • indicate which programs are shown to be effective (and at what level of rigor or confidence).

Some of these websites specialize in “systematic reviews” (also called meta-analytic reviews) of the literature regarding specific research questions and program areas. As the Center for Evidence-Based Crime Policy at George Mason University explains, systematic reviews “summarize the best available evidence on a specific topic using transparent, comprehensive search strategies to find a broad range of published and unpublished research, explicit criteria for including comparable studies, systematic coding and analysis, and often quantitative methods for producing an overall indicator of effectiveness.”[2]

A partial list of evidence-based program databases in criminal justice follows:[3]

A Website Caution

The consumer of website research summaries should be careful to not take the information at face value. Definitions as to what constitutes evidence, methodological soundness, and robust findings can vary significantly.

Furthermore, researchers do not always agree on what can be concluded from a research study. While some website authors make transparent attempts to give the user an accurate description of research findings, it is up to the user to exercise judgment. It is recommended that the user seek corroborating information to increase confidence in the relative strength of the research and its implications.
  • The Campbell Collaboration, The Crime and Justice Coordinating Group (CCJG) is an international network of researchers that prepares and disseminates systematic reviews of high-quality research on methods to reduce crime and delinquency and to improve the quality of justice. https://www.campbellcollaboration.org/
  • The Center for the Study of the Prevention of Violence, University of Colorado, maintains a website, Blueprints for Violence Prevention, on evaluated programs to prevent adolescent violence, aggression, and delinquency. https://www.ncjrs.gov/pdffiles1/ojjdp/187079.pdf
  • George Mason University’s Center for Evidence-based Crime Policy offers a number of services, including systematic reviews, research on crime and place, and a summary (matrix) of evidence-based policing practices.
  • Substance Abuse and Metal Health Services Administration’s (SAMSHA) National Registry of Evidence-based Programs and Practices (NREPP) provides a database of more than 190 interventions supporting mental health promotion, substance abuse prevention, and mental health and substance abuse treatment.
  • U.S. Department of Justice, Office of Justice Programs’ Crime Solutions’ website provides research on program effectiveness; easily understandable ratings (effective, promising, and no effects) that indicate whether a program achieves its goal; and key program information and research findings. https://crimesolutions.ojp.gov/

Websites that Provide Bibliographic Databases

These websites, which provide a listing of hundreds of studies, are often maintained by government agencies and universities. Prominent among these in the criminal justice field are the following:

Websites that Provide Summaries of Research and Practical Guidance

Some universities, state criminal justice agencies, and professional organizations also run websites that summarize the research on effective criminal justice practice and/or provide guidance to users. While not as extensive as bibliographic databases, these websites focus their publications on the critical issues of most concern to policymakers and practitioners. A partial list follows:

  • Center for Evidence-Based Crime Policy. https://cebcp.org/
  • Correctional Treatment Evaluations, Texas Christian University, Institute for Behavioral Research. This national research center for addiction treatment studies in community and correctional settings provides access to over 700 resources on its website. https://ibr.tcu.edu
  • National Implementation Research Network. This website contains research on the successful implementation of new processes within organizations and systems. http://nirn.fpg.unc.edu/
  • Stanford University, Evidence-Based Management. This website specializes in evidence directly related to the management of agencies. https://cebma.org/
  • University of Cincinnati School of Criminal Justice. This university-based site contains a number of research studies regarding the use of evidence in correctional interventions. https://cech.uc.edu/schools/criminaljustice.html
  • Washington State Institute for Public Policy. This website contains a number of helpful studies on what is or is not an effective intervention for reducing recidivism and costs. It is perhaps best known for its cost–benefit studies. http://www.wsipp.wa.gov/

Your Colleagues

Often an efficient way to check out the results of web-based and library searches is to ask experienced colleagues in your state and local jurisdiction and in national networks for recommendations regarding the latest and most reliable research. This strategy helps triangulate or hone in on the best studies.

Further, when identifying a journal article that appears useful but for which a subscription is required, contact colleagues at nearby colleges and universities and inquire about their ability to access the article from their library and provide a single copy for your review. (Be careful to not copy, distribute, or otherwise violate copyright laws.)

An increasing number of states support websites that summarize evidence-based research and practical guidance that is directly relevant to their criminal justice constituents and agencies. The websites may be hosted by a state criminal justice agency or university. Your colleagues will know how to access these sites.

Step 2: Evaluate Research Quality

What criteria should be used to decide if program evidence has been collected and analyzed according to high quality research standards? As Hess and Savadsky (2009) emphasize in their article “Evaluation Research for Policy Development,” all evidence is not created equally. Familiarity with a few key concepts can help policymakers wade through the growing body of information and make better-informed decisions about what is reliable. Following are a few tips about how to read the research literature and evaluate its quality:[4]

All Research Is Not Created Equal

"The golden rule here is to recognize that everything promoted as ‘research’ is not equally reliable or useful." – Hess & Savadsky, 2009
  • Understand the target population of the study and consider its relationship to the target population under consideration in your jurisdiction. Pay attention to sample size and sample selection. In general, larger samples provide more reliable data; however, there is no one hard and fast rule about sample size. The sample size may vary according to the purpose of the study, overall population, sampling error, and so forth.
  • Consider the context. What works in one place or for one population may not work for another (e.g., a study completed in a small, rural state with unique characteristics may not be applicable to a large, densely populated state with a different offender profile and justice system challenges). In addition, the context of one study cannot necessarily be transferred to other settings. An often-quoted study examined successful program results and found that 15% of the outcome was derived from the intervention itself (e.g., cognitive program, didactic intervention, or therapeutic community) and 30% from the working alliance with the individual providing the service.[5] However, the study was not carried out with correctional clients. The results could be valid across populations but until that hypothesis is tested, caution must be exercised about its applicability to the correctional population.
  • Be cautious about assertions of causality. Correlation does not mean causation; an intervention may be related to a certain outcome but may not be responsible for that outcome. For example, a significant portion of many communities’ offender population includes individuals with mental illness. A common assumption is that mental health treatment will reduce the likelihood of reoffense among this population. However, while a mental health condition should be treated, studies have shown that mental health treatment alone is unlikely to reduce recidivism.
  • Recognize that changes in implementation can change the outcomes of an intervention. For instance, an effective probation intervention that relies on officers proficient in motivational interviewing, case planning, and problem solving with clients may not work as well if delivered by staff who do not possess these skills.
  • Be sure the conclusions follow logically from the reported findings. The summaries or conclusions of some studies can be deceptive or take license in explaining the implications of findings. Consumers should look for research that “measures the impact of particular interventions on identifiable populations under controlled circumstances.”[6] These studies offer prescriptive guidance about actions that can be consistently replicated elsewhere.
  • The issue of confidence in results is important. The research consumer needs to know if the results of the intervention are “statistically significant.” This refers to the likelihood that a result is caused by something other than mere chance. In general, a 5% or power p-value is considered statistically significant. While policymakers may not want to dig through the statistical results’ section in great detail, it is useful to check whether the article mentions that the findings are statistically significant. Other issues such as whether the person(s) conducting the research study has a vested interest in the outcome of the study and whether the study was replicated elsewhere should also be considered.[7]

Additional Resources/Readings

Hess, F. M. & Savadsky, H. (2009). Evaluating research for policy development.

Fink, A. (2008). The research consumer as detective: Investigating program and bibliographic databases. Practicing Research: Discovering Evidence that Matters (pp. 33–64). Retrieved from https://www.sagepub.com/sites/default/files/upm-binaries/19270_Chapter_2...

Wampold, B. E. (2001). The great psychotherapy debate: Models, methods, and findings. Mahwah, NJ: Lawrence Erlbaum Associates.

[1] In Appendix 3 of the Framework for Evidence-Based Decision Making in Local Criminal Justice Systems, the Initiative provides a matrix of research findings on reducing pretrial misbehavior and offender recidivism. EBDM policy teams are encouraged to review this resource; however, the EBDM Research Matrix can only provide a snapshot of the research at one point in time, as new research is continually conducted. Therefore, this Starter Kit document is intended to provide EBDM policy teams with additional guidance on how to keep current with the research on EBDM.

[3] Adapted from Fink, 2008.

[4] Adapted from Hess & Savadsky, 2009.

[5] Wampold, 2001.

[6] Hess & Zavadsky, 2009.

[7] See Hess & Zavadsky (2009) for more information on how to be a good consumer of research.

5: Develop Logic Models

5: Develop Logic Models web_admin

Activity 5: Develop logic models

The development of a logic model is the critical next step in building a clear and specific understanding of how your system of evidence-based decision making (EBDM) will operate in the future. It is built upon—and in service of—a vision for the justice system, and it is informed by a careful analysis of current policy and practice in the context of evidence-based research. It reflects both the current inputs (i.e., resources) and activities that are supportive of the desired outcomes and reflective of the areas of advancement the policy team has identified. The logic model, therefore, describes currently available resources, activities that will be retained and those that will be changed or added, the outputs that these activities and changes will produce, and their intended long-term impacts.

The result of building a logic model is a picture that describes your theory of change—a local roadmap of the steps that need to be taken in order to produce your jurisdiction’s risk and harm reduction goals. A logic model provides a tool for managing the implementation and evaluation of EBDM activities.

Elements of an EBDM justice system include sound and testable system-level logic models.

5a: Building Logic Models

5a: Building Logic Models web_admin

Introduction

The development and use of a logic model is a critical step in understanding how evidence-based decision making (EBDM) will operate in a specific jurisdiction. A logic model helps lay out the shared understandings of what resources are available, what activities and changes will occur, what these activities and changes will produce, and what the intended long-term impacts of the initiative will be. The result of building a logic model is a picture that outlines the initiative’s theory of change, with a road map of what steps need to be taken in order to produce the desired impacts.

Logic models have six main components:

  • inputs, or resources, which represent the existing resources (both financial and human), policies, practices, facilities, and capabilities that a jurisdiction has in place to support the implementation of EBDM;
  • activities, which represent the specific strategies to be undertaken and implemented;
  • outputs, which specify the immediate results that occur as activities and strategies are implemented (e.g., changed policies and practices, adoption of new tools/protocols, number of people trained, number of cases in which risk assessments are administered);
  • outcomes, which serve as indicators that change is occurring at key decision points in the justice system as a result of the activities and which demonstrate that EBDM has been implemented at the system, agency, and case levels; and
  • impacts, which define the types of long-term results that are anticipated and that can be measured as a result of implementing EBDM.
The Logic Model as a Motivator

"The logic model, approached with integrity, encourages its developers to stretch their imagination and be accountable to their vision. For us, the logic model forced us to think harder and to be more specific about the results and how to measure them. It denied us the option of settling for platitudes or unquantified commitments."
–Policy Team Member, Milwaukee County, Wisconsin

In addition, because the logic model is intended to be a roadmap, the contextual conditions need to be considered. Contextual conditions represent the environment in which the local justice system operates and can include political, economic, social, cultural, or other factors.

For the EBDM initiative, the logic model should reflect implementation and desired change at the system level (i.e., a system logic model). The model that each jurisdiction develops will incorporate the resources, activities, etc. that are currently being used to reach the identified harm reduction goals as well as the new activities that are being planned.

Purpose

Building a logic model has two purposes:

  1. It helps facilitate the planning process by providing a mechanism for linking assumptions about how EBDM will work and the intended causal relationships between activities and impacts.
  2. It provides a tool for managing the implementation and evaluation of EBDM activities. Because EBDM can, and should, be implemented at multiple levels, separate logic models should be developed to represent EBDM at the system and agency levels.

The system-level logic model will provide an overall picture of the types of systemic activities and policy changes that will need to occur in order to achieve the jurisdiction-wide impacts that are expected with regard to harm reduction. The purpose of the agency-level logic model is to provide each entity and agency in the justice system with a plan for what activities the agency will need to undertake to move toward EBDM, what the outputs of these activities are, and how these will impact the stakeholders’ overall goal of harm reduction.

In addition to providing a graphic illustration of the causal relationship between activities and impacts, the component parts of the model also provide a sense of temporal order. In other words, the logic model can be used to show what activities or outputs need to occur before others can begin.

Participants

Initial work on the logic model—deciding what the jurisdiction hopes to accomplish—is a group discussion, ideally among the policy team. After these decisions have been reached, staff internal to the agency(ies), usually with some background in conceptualizing, planning, and implementing policy or program initiatives; staff with similar backgrounds from colleague agencies or county administration; or outside experts can develop the logic models, with input from the policy team. The instructions below assume that staff within agencies in the jurisdiction will develop the logic models.

Instructions

In general, the approach to developing a logic model—whether for the overall system or for an individual agency—is to work as a group to answer several critical questions related to what is hoped will be accomplished. The team discussion should result in answers to the following questions:

For the system model:

  • Why do you want to move toward an EBDM-based system? How will the jurisdiction benefit from an EBDM-based system (i.e., how will harm to jurisdictions be reduced)?
  • What significant changes do you expect from the implementation of EBDM in terms of system operation?
  • What types of information will convince you (and others, including the public) that positive change has occurred?
  • What are the possible unintended consequences, both positive and negative, of implementing EBDM?
  • What contextual (e.g., social, political, economic) conditions might facilitate or hinder your ability to achieve the types of impacts you’ve identified for both the system and the jurisdiction overall?

For the agency model:

  • What do you hope to accomplish as a result of implementing EBDM?
  • What outcomes does your agency need to achieve in order to contribute to the systemic impacts identified above?
  • What significant changes will occur within your agency as a result of the implementation of EBDM?
  • What types of information will convince you (and others) that you are achieving the outcomes that you’ve defined?
  • What are the possible unintended consequences, both positive and negative, of implementing EBDM?
  • What contextual (e.g., social, political, economic) conditions might facilitate or hinder your ability to achieve the types of impacts you’ve identified?

The answers to these questions form the basis for two of the logic model’s component parts: impact(s) and contextual conditions.

Good Impact and Outcome Statements

An example of a well-defined SMART impact is the following:
"75% of jail beds will be occupied by high risk offenders by 2013."
A well-articulated outcome has the same characteristics as an impact. An example of a good outcome statement is the following:
"The number of offenders who successfully complete their sentence or treatment will increase by 75% within one year."

Logic models are built from right to left—first you define the impacts, then the outcomes and outputs, followed by activities, and then the inputs. Contextual conditions are defined last or in tandem with the other components because they help you identify other factors that might need to be considered in order to achieve the intended results.

An easy way to think about the development of a logic model is to think in terms of “if…then…” statements. For example, if we want to achieve these harm reduction impacts (e.g., reduced costs), then we will need to accomplish these outcomes (e.g., cost-saving measures). If we want to achieve these outcomes, then we will need to accomplish these outputs (e.g., number of low risk offenders diverted from the system). If we want to produce these outputs, then we will need to implement a specific activity or set of activities (e.g., pretrial risk assessment tool). And finally, if we want to implement this activity, then we will need to draw on these types of inputs/resources (e.g., funding to purchase a risk assessment instrument).

The following instructions offer step-by-step guidance on the development of logic models.

Step-by-Step Instructions:

    1. Using the logic model table in Appendix 1, list the intended impacts and outcomes and define them according to the SMART principle:
      1. Be Specific.
      2. Make them Measurable (i.e., quantifiable).
      3. Be Action-oriented.
      4. Be Realistic.
      5. Articulate a Time in which the change will occur.[1]
    2. Define what short-term accomplishments (outputs) will be needed in order to produce the intended outcomes and impacts. For example, if your jurisdiction expects that a certain number of joint policy decisions will be adopted, then two outputs might be the number/percentage of meetings attended by each policymaker and the number of policy decisions discussed.
Defining Your Activities:
The Logic Challenge

Often there are preconceived ideas (because of funding opportunities, political will, or other reasons) about the specific activities that should be implemented. Be careful and realistic about the extent to which these activities will actually produce the intended outputs, outcomes, and impacts. As an example, consider the jurisdiction that wanted to decrease the amount of drug crime across the city. To do this, they decided to implement a truancy prevention program in one elementary school. By going through the process of linking activities, outputs, outcomes, and impacts, they would have readily highlighted the disconnect in the causal logic (i.e., what is the likelihood that a program at one elementary school will impact drug crime across the entire city?).
  1. For each output identified, define the activity that will produce it. For example, if the output is to have 100% of probation officers trained in the use of motivational interviewing techniques, then the activity might be to implement a motivational interviewing training program.
  2. As you define which activities will be implemented, make a list of available resources, including financial, human, and existing materials and policies, that will be used to facilitate implementation of the activities. Make note of resources that might be lacking, and consider adding activities to the model that would either produce the resources or develop the capacity needed.
  3. Once you complete the logic model table, make a list of the contextual conditions that are external to the justice system but that have an impact on its operation and ability to implement the planned activities or to achieve the desired outcomes and impacts.
  4. The next step is to transfer the contents of the logic model table to a logic model diagram. Laying out the diagram of the logic model will require additional consideration of how all the defined elements are logically related to each other; it may identify areas where the logic is flawed and additional work is required. The logic model diagram will also help identify any gaps that need to be filled.
    1. Appendices 2 and 3 illustrate a basic logic model structure, representing the inputs, activities, outputs, outcomes, and impacts of two specific strategies that might be part of a site’s implementation plan.
  5. Use the checklist in Appendix 4 to assess the quality of the draft logic model. Members of the policy team (or managers/line personnel in an agency) and others not involved in the development of the logic model should complete the checklist.
  6. Revise and finalize the logic model as required.

Tips

  • Logic models should be built during the planning process of the initiative to maximize their utility as a planning, management, and evaluation tool.
  • Horizontal arrows between components represent causal links; vertical arrows within components generally represent temporal order.
  • It may be useful to label each piece of information (i.e., each input, each activity, etc.) in the logic model table to make the transfer to the logic model diagram easier. One suggestion is to assign the first input the number “1.” Assign the number “1a” to the activity that is related to that input, “1b” to the output associated with the activity, and so forth. In the event that two or more elements flow from the previous one, then number these elements in a way that depicts their temporal order.
  • Logic models are not static. The logic model that you are preparing represents what you think and want to happen, not what will happen. The logic model should be thought of as a working model that you will periodically revisit and update as you move toward implementation.
  • To the extent the logic model consists of both activities that are already in place in support of the identified impacts and those that are being planned as part of the initiative, it may be useful to color code the planned activities to make clear the action items for moving forward (e.g., use black for components in place and red for proposed or new components).

Example:
Yamhill County, Oregon Logic Model

Logic Model

 

Additional Resources/Readings

W. K. Kellogg Foundation. (2004). Logic model development guide. Retrieved from https://www.wkkf.org/resource-directory/resources/2004/01/logic-model-development-guide

OJJDP. (n. d.). Performance measures: Logic model. Retrieved from https://ojjdp.ojp.gov/funding/grant-performance-measurement/overview

The Pell Institute. (2011). Using a logic model. Retrieved from http://toolkit.pellinstitute.org/evaluation-guide/plan-budget/using-a-logic-model/

CEPP. (2009). Measuring the impact of reentry efforts. Retrieved from https://cepp.com/wp-content/uploads/2015/12/Measuring-the-Impact-of-Reenty-Efforts.pdf

 

Appendix 1:
Logic Model Development Template

 

Inputs/Resources

Activities

Outputs

Short-Term Outcomes

Impacts

Contextual Conditions

Existing resources (both financial and human), policies, practices, facilities, and capabilities

Specific strategies to be implemented

Immediate results that occur as activities and strategies are implemented

Indicators, or benchmarks, that demonstrate changes are occurring as a result of the activities

Anticipated long-term harm reduction results

External factors that can facilitate or hinder the ability to implement the activity or achieve the intended outcomes and impacts

 

 

 

 

 

 

 

Appendix 2:
Sample of Partial System-Level Logic Model for Pretrial Risk Assessment

Appendix 2

 

Appendix 3:
Sample of Partial System-Level Logic Model for Using Risk Assessments to Inform Plea Negotiations

appendix 3

 

Appendix 4:
Logic Model Review Checklist

 

Review Questions

Yes

No

Comments

The identified intended impacts are realistic and attainable.

 

 

 

The impacts, outcomes, and outputs are quantifiable.

 

 

 

There is a logical, clear connection between the outcomes and impacts.

 

 

 

There is a logical, clear connection between the outputs and outcomes.

 

 

 

There is a logical, clear connection between the activities and outputs.

 

 

 

All available resources (inputs) needed for the activities have been accounted for in the model.

 

 

 

Possible contextual conditions that may affect the EBDM initiative and their impact have been identified.

 

 

 

The activities described in the model are realistically attainable.

 

 

 

The underlying assumptions for how the initiative will work are clearly discernible from the model.

 

 

 

There is consensus among the stakeholders that the model accurately describes how EBDM will achieve the desired results.

 

 

 

 

[1] See also 6a: Measuring Your Performance and 6b: Developing a Systemwide Scorecard for more information on developing SMART goals and objectives.

6: Performance Measures

6: Performance Measures web_admin

Activity 6:
Establish performance measures, determine outcomes, and develop a system scorecard

What Do We Mean by Outcomes?

"Outcomes," under a risk reduction model, are defined as:

  • decreases in the rate or severity of reoffense by offenders,
  • decreases in the harm caused to communities as a result of crime,
  • increases in the level of satisfaction with the justice system by victims, and
  • increases in the level of public confidence in the justice system.

Performance measurement facilitates an objective, empirical evaluation of the effectiveness of the justice system in achieving desired outcomes; it also facilitates an evaluation of the effectiveness of change strategies in contributing to those outcomes. Development of a systemwide scorecard—and reaching agreement on the methods to measure performance on scorecard items—ensures common agreement on the team's desired outcomes. It also provides a tool for engaging and educating professionals and community members around the goals and activities of the justice system.

Elements of an EBDM justice system include

  • a set of agreed-upon performance measures that will enable an objective, empirical evaluation of the effectiveness of justice system agencies in achieving their vision;
  • benchmarks against which longer-term outcomes can be measured;
  • methods to collect and analyze data on an ongoing basis to inform policy and practice; and
  • a systemwide scorecard.

6a: Measuring Your Performance

6a: Measuring Your Performance web_admin

Introduction

Performance measures are tools for managing the performance of an agency, organization, or even a system. Performance measures provide benchmarks about whether or not optimum performance by the criminal justice system (and the entities within it) is being realized and, more importantly, whether the system is achieving what it intends to achieve under the evidence-based decision making (EBDM) framework. The use of performance measures provides a way to understand quantitatively the business processes, products, and services in the justice system. In a nutshell, performance measures help inform the decision making process by ensuring that decisions are based on clearly articulated and objective indicators. Moreover, undertaking and institutionalizing performance measurement throughout the criminal justice system allows policy discussions and decisions to be “data-driven,” which in turn helps build the foundation for additional local evidence about what works.

In general, performance measures for the justice system fall into four categories:

  1. Effectiveness and the extent to which the intended outcomes are being produced
  2. Efficiency measures that demonstrate whether maximum outcomes are being produced at minimum cost
  3. Measures of satisfaction and quality to assess if the right processes are being used and the degree to which there is “satisfaction” with the processes[1]
  4. Timeliness in terms of the extent to which activities or processes occur within predefined time limits

Performance measurement is often confused with program evaluation because both attempt to capture quantitative information about desired goals and outcomes. Some key differences should be noted. First, program evaluation involves the use of specific research methodologies to answer select questions about the impact of a program. Performance measurement, on the other hand, is simply the articulation of performance targets and the collection/analysis of data related to these targets. Second, program evaluation is designed to establish causal relationships between activities and observed changes while taking into account other factors that may have contributed to or caused the changes. On the other hand, performance measurement simply provides a description of a change, but cannot be used to demonstrate causality. Third, program evaluations are usually one-time studies of activities and outcomes in a set period of time, whereas performance measurement is an ongoing process.

As you begin the process of defining performance measures, there are seven rules that need to be kept in mind. Performance measures should be

  1. Logical and related to goals
  2. Easy to understand
  3. Monitored regularly
  4. Readily accessible
  5. Based on specific benchmarks
  6. Quantified and measurable
  7. Defined with specific performance targets

Purpose

This starter kit is designed to help jurisdictions understand performance measures and to provide a guide for the development and implementation of performance measures systemwide. Information about the key steps in performance measurement is provided in addition to sample performance measures. It is important to note, however, that performance measures should be locally defined and driven; as such, the sample measures may or may not be relevant in a specific jurisdiction, depending on the focus of the local initiative. Finally, tips are offered for the implementation and use of performance measures.

Participants

Development of performance measures should involve a variety of stakeholders. At a minimum, the leadership of the various components of the justice system, along with some line level representatives, should be part of the process. The leadership can provide the broad systemic perspective about how the system should be performing under an EBDM initiative and how each agency/entity within the justice system contributes to overall system performance. The inclusion of line personnel, however, provides a different level of detail and, to some extent, a reality check about how the system is currently performing and what the capacity is for performance. Participants should also include representation from groups that have an interest in the justice system—city/county government budget officers and managers, health/mental health treatment providers, etc. The community and the media can also be important stakeholders to include as, ultimately, it is through these groups that performance is communicated and legitimacy is established. The point is that for performance measures to have validity (not necessarily in the statistical sense), they must be meaningful for others who judge the performance of the system.

Jurisdictions may wish to consider engaging an outside facilitator with experience in performance measurement to provide guidance and assistance through the process. Local universities are an excellent resource for finding this kind of assistance.

Instructions

To develop and implement performance measures, the stakeholders identified above should undertake four key steps:

  1. Identify the goals and objectives of the system under the EBDM framework.
  2. Determine what the key indicators of output and outcomes are and what type of data collection will be required.
  3. Begin the collection and analysis of the performance measures.
  4. Implement a reporting mechanism for communicating performance to stakeholders.

Detailed guidance for each of these steps is provided below.

  1. Identify the goals and objectives of the system under the EBDM framework

The first step for articulating performance measures is to define what is meant by “optimum performance,” i.e., establishing harm reduction goals and objectives for the criminal justice system. Several questions can help focus the discussion on what the jurisdiction hopes to accomplish:

  • How will the jurisdiction benefit as a whole (i.e., what are the intended harm reduction outcomes)?
  • How will the criminal justice system benefit from the movement to an EBDM-based system?
  • What is an EBDM system intended to achieve or produce?
  • What significant changes does the jurisdiction expect from the implementation of EBDM in terms of system operation?
    • How will the costs to operate the system change?
    • How will case processing change at point of entry into the system, during the adjudication process, while in corrections, and/or at point of release?
    • How will those in the system (victims, witnesses, and defendants) view the process?
    • How will EBDM impact those working in the system?
    • What types of information will convince you and others (including the public and funders) that the system is operating at an optimum level?
    • What types of information will convince you and others that the system is achieving what it is intended to achieve?

The answers to these questions need to then been articulated in terms of quantifiable goals and objectives. It is important to understand that goals and objectives are not synonymous. Goals represent the desired end result of the system. Objectives define the short-term indicators that demonstrate progress toward goal attainment and that describe who or what will change, by how much, and over what period of time. For example, broadly stated, one goal might be that the recidivism rate be no higher than 20%. An objective might be a 5% annual decrease in the percentage of offenders who commit new offenses in a three-year period.

Another important consideration in defining goals and objectives is adherence to the SMART principle:

  • Be Specific.
  • Make them Measurable (i.e., quantifiable).
  • Be Action-oriented.
  • Be Realistic.
  • Articulate a Time in which the change will occur.

Once goals and objectives have been defined, the stakeholders should compare them to the impacts and outcomes identified in the system-level logic model. Each goal and objective should align with the intended impacts and outcomes articulated in the logic model. Although there does not need to be complete overlap, there should be no contradictions.

  1. Determine what the key indicators of output and outcomes are and what types of data collection will be required

The second step in defining performance measures encompasses a number of activities:

  • determining what the key indicator data are for each goal and objective;
  • identifying where, or if, the data exist and, if not, whether the capacity exists for capturing the data;
  • refining the list of performance measures to represent a set of key indicators; and
  • establishing performance targets.

Well-articulated goals and objectives should lend themselves nicely to the identification of key indicator data. Using the worksheet in Appendix 1, jurisdictions will need to “break down” the goals and objectives into specific types of data that can be collected. Using the example from Step 1 above, the table below shows the goal, the objective, and the types of indicator data that are needed to measure performance:

 

Goal

Objective

Indicator Data

Our jurisdiction will have a recidivism rate of less than 20%.

5% annual decrease in the percentage of offenders who commit new offenses in a three-year period
  • Total number of offenders committing new offenses within three years
  • Total number of offenders released

 

As indicator data are being identified, jurisdictions should note if the data already exist; if so, they should identify who “owns” the data and, if not, they should determine whether the capacity for obtaining the data exists. To the extent that data is not already being collected or the capacity to collect the data does not exist, consideration should be given to the relative importance of the indicator. This next step in the process will help refine the list of performance measures.

Using Information Dashboards To Make Law Enforcement Decisions

Law enforcement has long understood the importance of routine performance measurement. By using the “dashboard” approach—that is, putting a spotlight on key information on a routine basis—law enforcement agencies around the country are using data to assess performance and adjust activities based on key outcome measures. Police Chief Bence Hoyle, of Cornelius, North Carolina, states that such dashboards should

  • identify and disseminate information about criminal activity to facilitate rapid intervention;
  • identify and disseminate information about crime to assist in long- and short-term strategic solutions;
  • allow agencies to research the key incident data patterns, such as modus operandi, repeat offender locations, or other related information, such as traffic stops near the scene, so suspects can quickly be identified;
  • provide data on the effectiveness of specific tactics, in near real-time, through focused views; and
  • support the analysis of workload distribution by shift and geographic area.

For more information, see "Dashboards Help Lift the Fog of Crime" at www.theomegagroup.com

An ideal performance measurement system must be manageable; as such, the number of performance measures for each goal and objective should be limited. Generally, there should be no more than three or four measures per goal or objective and, in fact, there may be fewer. Jurisdictions should aim to select those measures that are the strongest indicators of performance for which data already exist or for which the capacity for the data to be collected is in place. In refining the list, it is important to consider the following seven questions:

  1. Is the indicator logical and directly related to goals?
  2. Is the indicator easy to understand (i.e., would a reasonable person agree that the indicator accurately represents what it is intended to measure)?
  3. Can the indicator be monitored regularly?
  4. Is the data necessary for measurement readily available?
  5. Can the indicator be measured against a specific benchmark (i.e., is there a baseline against which performance can be assessed)?
  6. Is the performance indicator quantified and measurable?
  7. Can specific performance targets be set for the indicator in question?

The question of performance targets is a particularly important one and requires more than a simple “yes/no” answer. As the list of measures is refined, jurisdictions should begin thinking in terms of what the specific performance targets should be. In other words, what is the “magic number” that demonstrates optimum performance? For example, if the intent is to implement pretrial risk assessments in order to decrease jail operating costs, the performance target might be that 90% of release decisions are consistent with assessment results. The logic model may provide some guidance in answering this question.

  1. Begin the collection and analysis of the performance measures

Because performance measurement is an ongoing process, it is important to have a well-defined data collection plan in place prior to the actual collection of data. As shown in Appendix 2, the data collection plan should include the following:

  • data source: the name of the agency/person responsible for collecting the data and, if the data is already being collected, the name of the report or system from which the data is drawn; and
  • frequency of data collection: how often the data will be collected.

Once the data collection plan has been agreed upon by the key stakeholders and the agencies/persons that will be responsible for collecting the data, the jurisdiction should collect baseline data for each performance measure against which progress can later be measured.

It is rare that the data in raw form will be sufficient for assessing performance; quantitative analysis of the data is generally needed. The quantitative analysis will require basic statistical calculations such as ratios, percentages, percent change, and averages (mean, median, or mode). In some instances, depending on the measures selected, more complex statistics will be necessary and may require the involvement of persons with statistical analysis experience. Employees in the city/county manager’s offices may be resources, or even employees within criminal justice agencies that have analysis units. Local universities are also good resources for statistical analyses.

Sample Measures

The actual performance measures selected by the jurisdiction should be reflective of the goals and objectives that the stakeholders have identified as part of the EBDM initiative. The following list of possible performance measures are provided for illustrative purposes only:

  • XX% of low risk arrestees cited and released
  • XX% of defendants screened with a pretrial risk assessment tool
  • No more than XX% cases resulting in deviations for pretrial release from risk assessment results
  • XX% of jail beds occupied by low risk defendants awaiting adjudication
  • XX% of defendants/offenders with low risk assessment scores placed in diversion programs
  • Risk assessment information provided to judges in XX% of cases
  • XX% of cases in which sentencing conditions align with assessed criminogenic needs
  • XX% of offenders placed in interventions specifically addressing assessed criminogenic needs
  • XX% of offenders who commit new offenses in a three-year period
  • XX% of victims who report satisfaction with the handling of their cases
  1. Implement a reporting mechanism for communicating performance to stakeholders

Once the performance data is collected and analyzed, it should be reported to stakeholders in a clear and easily understood manner. Although there is no wrong or right way to report data, the following list of reporting formats should be considered:

  • Whenever possible, use graphic displays such as tables, bar charts, pie charts, or line charts.
  • In graphic displays, provide legends and labels to clearly identify the information.
  • Take care not to present too much information in a single graphic display.
  • Use short narrative descriptions to help the audience interpret the data.
  • Present both the performance measure (the target) (e.g., risk assessments provided to judges in 90% of cases) and the actual score (risk assessments provided to judges in 75% of cases).
  • Provide context for the interpretation that might include discussion of why performance targets were or were not met, how the current performance period compares to previous performance periods, or what recommendations for performance improvement can be made.

Jurisdictions should also establish a regular mechanism for communicating and discussing performance that includes target dates for the release of information. Possible mechanisms include

  • publication of a “scorecard,” “report card,” or “dashboard”;[2]
  • monthly, quarterly, or annual reports; and/or
  • performance meetings with stakeholders.

Tips

  • In deciding on the final list of performance indicators, make sure they are the best indicators of performance related to the specific goal or objective. Don’t “settle” on the easy indicators; instead, work toward a set of indicators that will provide the most compelling evidence of performance.
  • Make sure that indicators are clearly defined (e.g., how is recidivism being defined, or what constitutes a case—a defendant, a charge, or a case number?) and that there are specific guidelines in place for their collection. Refer to Appendix 3 for a list of definitions that you might draw from or at least use as a starting place for the development of your own definitions. It does not matter whether you use the provided definitions or definitions of your own. What matters is that your team agrees that these are the right terms and agrees on their meanings.
  • Consider “pilot testing” the performance measures by doing a preliminary data collection, analysis, and reporting to ensure that the data is interpreted consistently and that the performance measures actually measure what they are supposed to.
  • When data is being collected from multiple sources, consider the use of Memoranda of Understanding (MOUs) or some other form of agreement to ensure that it will be collected and reported in the manner specified and within the established time frames.
  • Use the performance measures to inform decision making. Where performance is lacking, dig deeper to understand why optimum performance is not being met and then make the appropriate adjustments.

Example:
Milwaukee County, Wisconsin,
EBDM Initiative Monthly Project Dashboard
(A Work in Progress)

In addition to developing a scorecard to communicate to the public its progress in reaching its harm reduction goals, Milwaukee County’s policy team is also developing a dashboard for use by its policy team and its more immediate stakeholders involved in the EBDM initiative.

The dashboard will serve as a managerial tool that provides a quick snapshot of the County’s progress towards its short-term goals.

example dashboard

 

Example:
Milwaukee County, Wisconsin, Harm Reduction Goals and Objectives

Goal: Reduce by 25% the number of people with mental health needs who lose their benefits due to being jailed or losing housing, and increase by 25% the number of individuals with mental health needs who are reconnected, within 20 days of arrest, to the services they require.

Measures:

  • Percentage of MPD officers with CIT training
  • Percentage of chronic consumers identified
  • EDs for CCs
  • Aggregate cost of EDs for CCs
  • Number of CCs in special needs pod
  • Aggregate cost of housing CCs in special needs pod

Goal: Safely release and/or supervise 15% more pretrial detainees in the community rather than in jail, generating at least $1,000,000 in savings that can be reinvested in the community and, at the same time, reduce by at least 40% the already low rates at which defendants waiting for trial fail to follow pretrial rules.

Measures:

  • Percentage of defendants considered for bail who are screened
  • Percentage of cases in which bail = Praxis recommendation
  • Pretrial jail bed days
  • Average length of stay
  • Average daily population (pretrial)
  • FTA rate
  • Rearrest rate

Goal: Divert or defer prosecution in 10% more cases than we do currently, holding offenders accountable, compensating victims, and reducing recidivism, while generating at least $350,000 in savings that can be reinvested in the community.

Measures:

  • Diversions/DPAs screened annually
  • Diversions approved annually
  • Diversions successfully completed annually
  • DPAs approved annually
  • DPAs successfully completed annually
  • Jail bed days avoided by successful diversion
  • Jail bed days avoided by successful DPAs
  • Arrests resulting in new charges during diversion period
  • Arrests resulting in new charges during DPA period

Goal: Demonstrate in a pilot project that by terminating probation as soon as an offender in need of treatment has received sufficient treatment, we can cut the cost of probation by at least 50% and at the same time reduce probation recidivism by 50%.

Measures:

  • Diversions/DPAs screened annually
  • Diversions approved annually
  • Diversions successfully completed annually
  • DPAs approved annually
  • DPAs successfully completed annually
  • Jail bed days avoided by successful diversion
  • Jail bed days avoided by successful DPAs
  • Arrests resulting in new charges during diversion period
  • Arrests resulting in new charges during DPA period

Additional Resources/Readings

Boone, H. N., Jr., & Fulton, B. (1996). Implementing performance-based measures in community corrections (NCJ 158836). National Institute of Justice Research in Brief. Retrieved from https://www.ncjrs.gov/pdffiles/perform.pdf

Boone, H. N., Jr., Fulton, B., Crowe, A. H., & Markley, G. (1995). Results-driven management: Implementing performance-based measures in community corrections. Lexington, KY: American Probation and Parole Association.

Bureau of Justice Statistics. (1993). Performance measures for the criminal justice system. Retrieved from https://www.bja.gov/evaluation/guide/documents/documentI.html

Dillingham, S., Nugent, M. E., & Whitcomb, D. (2004). Prosecution in the 21st century: Goals, objectives, and performance measures. Alexandria, VA: American Prosecutors Research Institute.

Hatry, H. P. (2007). Performance measurement: Getting results. Washington, DC: Urban Institute Press.

Hoyle, B. (2011). Dashboards help lift the ‘fog of crime.’ Retrieved from https://docplayer.net/13839392-Dashboards-help-lift-the-fog-of-crime-by-chief-bence-hoyle.html

National Center for State Courts. CourTools. Retrieved from https://www.courtools.org/

National Research Council. (2003). Measurement problems in criminal justice research. Washington, DC: National Academies Press.

Pennsylvania Commission on Crime and Delinquency: Office of Criminal Justice Improvements. Criminal justice performance measures literature review calendar years: 2000 to 2010. https://www.jrsa.org/pubs/sac-digest/documents/PA_PerformanceMeasures.pdf

Rossman, S. B., & Winterfield, L. (2009). Measuring the impact of reentry efforts. Retrieved from https://cepp.com/wp-content/uploads/2015/12/Measuring-the-Impact-of-Reenty-Efforts.pdf

Appendix 1:
Performance Indicator Worksheet

Harm Reduction Goal

Objective

Indicator Data

Sample: Our jurisdiction will have a recidivism rate of less than 20%.

5% decrease in the percentage of offenders who commit new offenses in a three-year period
  • Total number of offenders committing new offenses within three years
  • Total number of offenders released

 

 

 

 

 

 

 

 

 

 

Appendix 2:
Data Collection Plan Worksheet

Performance Measure

Data Source

Frequency of Data Collection

Sample: 90% of pretrial release decisions are consistent with assessment results

Pretrial Service Agency Bail Review Recommendation Forms

Monthly

 

 

 

 

 

 

 

 

 

Appendix 3:
Sample Glossary of Criminal Justice Terms

This glossary defines key terms that are commonly used in the criminal justice field.[3]

BJS Posts Online Tool For The Public to Calculate Recidivism Rates

The U.S. Bureau of Justice Statistics has posted an online analysis tool that allows users to calculate recidivism rates for persons released from state prisons. Recidivism rates may be generated for the entire sample of 35,000 released prisoners or for released prisoners with specific demographic, criminal history, and sentence attributes. The tool uses data collected by BJS on a sample of inmates released from state prisons in 1994 and followed for three years. A new BJS study on the recidivism of state prisoners released in 2005 is due next year.

BJS says the tool defines recidivism in a variety of ways and allows users to choose the measure that best fits their needs or to compare the various measures of recidivism for the same group of releases. As one example, more than 57% of white women between the ages of 21 and 25 who were released from prison were rearrested within three years, and 17% returned to prison.

The tool is available at http://bjs.gov
  • Community corrections: The supervision of offenders in the resident population, as opposed to their confinement in secure correctional facilities. The main types of community corrections supervision are probation, parole, and pretrial. Community corrections is also referred to as community supervision.*
  • Cost analysis: A type of economic analysis that provides a complete accounting of the costs related to a given policy or program. Cost analysis offers the most rudimentary cost information required by both decision makers and practitioners, and also serves as the foundation of all other economic analyses.[4]
  • Criminogenic: Attributes of offenders that are directly linked to criminal behavior, have predictive qualities (of a new offense), are dynamic or changeable in nature (such as employment and peer interaction), and therefore can be influenced through circumstances, programming, or changes in an offender’s attitude.
  • Data: A collection of observations or statistics used to measure and analyze interventions.
  • Data-driven: The use of regular and ongoing data collection and analysis to track performance and inform policy and practice.
  • Defendant: A person who has been formally charged with a crime.
  • Direct expenditure: All expenditures except those classified as intergovernmental. It includes "direct current expenditure" (salaries, wages, fees, and commissions and purchases of supplies, materials, and contractual services) and "capital outlays" (construction and purchase of equipment, land, and existing structures). Capital outlays are included for the year when the direct expenditure is made, regardless of how the funds are raised (for example, by bond issue) or when they are paid back.*
  • Evidence-based: Conclusions drawn from rigorous research studies that have been replicated numerous times with defined, measurable outcomes about the effectiveness of an intervention or process.
  • Failure to appear: A defendant’s absence for a scheduled court hearing when the defendant was notified in advance and deemed able to attend the hearing (e.g., the defendant’s absence was not a result of being held in confinement and not transported from jail to the hearing, hospitalized, etc.); “absence” from the hearing is defined as having not attended at all while court is in session (vs. late for the hearing).
  • Goal: The desired long-term result of an effort.[5]
  • Incarcerated population: The population of inmates confined in a prison or a jail. This may also include halfway houses, boot camps, weekend residential programs, and other facilities in which individuals are confined overnight.*
  • Institutional corrections: Secure correctional facilities. There are many different types of correctional facilities, operated by different government entities. Local jails are operated by county or municipal authorities, and typically hold offenders for short periods, ranging from a single day to a year. Prisons serve as long-term confinement facilities and are usually administered by the 50 state governments and the Federal Bureau of Prisons. Private correctional facilities also operate under contracts for a wide variety of local, state, and federal agencies. Other correctional facilities are operated by special jurisdictions, such as the U.S. Armed Forces, U.S. territories, and federal agencies such as Immigrations and Customs Enforcement (ICE).*
  • Jurisdiction: A unit of government or the legal authority to exercise governmental power. In corrections, it refers to the government (state, federal, local, or tribal) that has legal authority over an inmate. Prisoners under a given state's jurisdiction may be housed in another state or local correctional facility.*
  • Objective: Measurable, short-term indicators or benchmarks that indicate progress toward a goal is being made.[6]
  • Offender: A person convicted of a criminal charge.
  • Offense: An act or actions that constitute a violation of one or more criminal statutes. Such actions may result in an individual being charged and prosecuted, and result in a court disposition. Some offenses may not result in formal charges and may result instead in dropped charges, referral to a precharge diversion program, etc.
  • Operational capacity: The number of inmates that can be accommodated based on a facility’s size and space distribution, staff, existing programs, and services.*
Another Glossary

See also the Glossary of Terms and Phrases Relating to Bail and the Pretrial Release and Detention Decision, by the Pretrial Justice Institute:
www.pretrial.org
  • Performance measure: A quantifiable measure that is used to assess whether or not optimum performance is being achieved and to identify where adjustments in performance or strategy are necessary.
  • Recidivism: A measure of failure of an individual or group of individuals who have been or are under criminal justice authority. Individuals who have been charged with any new offense in any jurisdiction that proceeds past a probable cause hearing are considered to have “recidivated” unless those individuals are subsequently determined to be “not guilty.”
  • Research: The systematic collection and analysis of data, using scientific methods, to study the effect of an intervention.
  • Technical violation: A finding that an individual has not complied with a court-ordered condition (or, if this authority is delegated by the court to another entity such as pretrial justice or community supervision, a condition established by this entity) that does not constitute a new criminal offense. For the purposes of this definition, a finding of a positive (“dirty”) urine test is (or is not) considered evidence of the commission of a new criminal offense.
  • Victimization: The effect of a crime on an individual person or household. For personal crimes, the number of victimizations is equal to the number of victims involved. The number of victimizations may be greater than the number of incidents because more than one person may be victimized during an incident. For household crimes, each crime is assumed to involve a single victim, the affected household.*
  • Victimization rate: A measure of the occurrence of victimizations among a specified population. For personal crimes, this is based on the number of victimizations per 1,000 residents age 12 or older. For household crimes, victimization rates are calculated using the number of incidents per 1,000 households.*
  • Violation (any type): A finding that an individual has not complied with a court-ordered condition (or, if this authority is delegated by the court to another entity such as pretrial justice or community supervision, a condition established by this entity).
  • Violation (new crime): A finding that an individual has not complied with court-ordered conditions of community release by being arrested for (or being found guilty of) the commission of a new crime that occurred after being placed on supervision.

Glossary of Cost-Benefit Terms

Cost-Benefit Online Clearinghouse

The Cost-Benefit Knowledge Bank for Criminal Justice (CBKB) aims to broaden and deepen the understanding and use of cost-benefit analysis in criminal justice.

CBKB helps practitioners and jurisdictions build their capacity to conduct cost-benefit studies and apply cost-benefit analysis to policymaking. CBKB is a project of the Vera Institute of Justice and is funded by the U.S. Department of Justice’s Bureau of Justice Assistance.

Visit the CBKB at: https://www.vera.org/research/introduction-to-the-cost-benefit-knowledge-bank-for-criminal-justice

This glossary defines key terms that are commonly used in the field of cost-benefit analysis. These terms are taken in whole from the Cost-Benefit Knowledge Bank for Criminal Justice (CBKB)[7].

Average costs:
Total cost divided by the quantity of output. For example, the average cost of probation is calculated by dividing total probation department expenditures by the average probation population. See also marginal costs.
Benefit-cost analysis (BCA):
Refer to cost-benefit analysis (CBA).
Benefit-cost ratio (BCR):
A common means of reporting CBA results that is calculated by dividing total benefits by total costs. If the ratio is greater than one, it means that the benefits outweigh the costs. If it is less than one, then the costs outweigh the benefits. If it is equal to one, then the costs equal the benefits and the initiative breaks even.
Capital cost:
The cost of purchasing and/or developing tangible property, including durable goods, equipment, buildings, installations, and land. This cost includes any interest paid on the funds borrowed to finance a capital expense.
Contingent valuation:
A method that uses surveys to estimate the monetary value of something that is not commonly traded in the marketplace, such as environmental preservation or crime reduction. For example, a contingent valuation survey might ask individuals what they are willing to pay for a reduction in crime.
Cost analysis:
A type of economic analysis that provides a complete accounting of the costs related to a given policy or program. Cost analysis provides the most rudimentary cost information required by both decision makers and practitioners, and also serves as the foundation of all other economic analyses.
Cost-benefit analysis (CBA):
Also known as benefit-cost analysis (BCA). A type of economic analysis that compares the costs and benefits of policies and programs over a long-term period. The hallmark of CBA is that both costs and benefits are monetized, permitting the comparison of initiatives with different purposes and outcomes.
Cost-effectiveness analysis (CEA):
A type of economic analysis that compares the costs relative to the outcomes of programs and policies. Cost-effectiveness analysis indicates which option produces a desired outcome for the lowest cost.
Cost-of-illness approach:
A method that measures tangible victim costs, such as medical costs and lost earnings, using information from hospital databases and typical salary rates.
Direct costs:
Costs that are directly related to a specific activity. General categories of direct costs include but are not limited to salaries and wages, fringe benefits, supplies, contractual services, travel and communication, equipment, and computer use.
Fiscal impact analysis:
A type of economic analysis that comprehensively examines all governmental costs and savings that will result from a proposed policy or program. Referred to as a “fiscal note” when prepared by legislative staff to report the impact of draft legislation on the government budget.
Hedonic valuation:
A technique to estimate the dollar value of items that are not commonly traded in the marketplace by measuring their impact on the prices of other goods and services. Hedonic valuation can be used to estimate the value of crime by measuring how changes in crime rates affect local property values, for instance.
Indirect costs:
Also known as overhead. Indirect costs refer to central administrative expenses, such as accounting and legal services, that are necessary for the continued functioning of an organization but cannot be directly allocated to a specific activity.
Intangible costs:
Costs that cannot be measured directly in dollar terms. Examples of intangible costs include pain and suffering and lost confidence in the justice system.
Jury-compensation method:
A method to estimate the intangible costs of crime using the money awarded to victims by juries.
Marginal costs:
Used to describe the costs that are incurred because of changes in units of activity at the margin of an existing level of operation. Short-term marginal costs include those costs that change with a slight change in units of activity. Long-term marginal costs are costs that change as a result of more substantial changes in activity. Marginal costs are generally a more accurate measure to use in a cost-benefit analysis than average costs. See also average costs.
Monetize:
To convert something, for instance, program outcomes or intangible benefits, into dollar terms.
Net benefits:
Total benefits minus total costs. The net benefit is a common means of reporting CBA results.
Overhead:
Refer to indirect costs.
Per diem rate:
A daily allowable expense rate, for example, the daily rate for keeping people in prisons or jails.
Regression analysis:
A statistical technique used to model how changes in one or more variables, called independent variables, affect changes in a variable of interest, called the dependent variable. In CBA, this technique can be used to estimate marginal costs.
Tangible costs:
Costs that can be measured directly in dollar terms. Tangible costs to crime victims include medical expenses, property damage and loss, and lost wages.
Victim costs:
The monetary value of the physical, psychological, and financial harms experienced by crime victims. Victim costs typically include tangible and costs.

 

[1] Satisfaction can be measured on different levels but generally represents the satisfaction of justice system “consumers” such as victims, witnesses, and defendants. However, in certain instances, it may be desirable and important to measure satisfaction among those working in the justice system.

[2] For more information on developing a scorecard, see 6b: Developing a Systemwide Scorecard.

[3]Definitions noted with an asterisk (*) are drawn in whole or in part from BJS’s terms and definitions: http://bjs.gov/index.cfm?ty=tda.

[4] Cost-benefit analysis and cost effectiveness analysis are two types of cost analyses. See the glossary below for additional terms related to cost-benefit analysis.

[5] For purposes of this initiative, “goal” is synonymous with the term “impact” in the logic model.

[6] “Objective,” as it is used here, is synonymous with the term “outcome” in the logic model.

[7] See https://www.vera.org/research/making-sense-of-the-bottom-line-a-guide-to-reading-cost-benefit-analysis.

6b: Developing a System wide Scorecard

6b: Developing a System wide Scorecard web_admin

Introduction

Historically, criminal justice agencies and their allied partners have developed independent methods to describe and measure their performance. Police agencies report on crime trends, arrests made, and the elapsed time between calls to dispatch and the arrival of patrol cars on the scene of a crime, for instance; courts report on case processing, fines imposed and collected, and cases settled by plea, bench, and jury trial; probation agencies report on numbers of individuals supervised, assessments conducted, and cases closed by successful termination. Rarely if ever do justice systems report on their progress in achieving their harm reduction goals and objectives. Examples of system wide harm reduction goals and objectives include (but are not limited to)

  • reduced justice system costs as a result of a combination of activities that reduce the demand for jail beds and correctional staff, and the time associated with judicial processing. These activities may include conducting pretrial screening and diversion at police substations, establishing alternative responses to the acutely mentally ill, and addressing probation violators administratively rather than through the court system; and
  • increases in the success rate of offenders, as a result of: improving adherence to the risk principle at the arrest, pretrial, plea, sentencing, and supervision decision points; “matching” offenders to appropriate services (e.g., by prosecutors and defenders in plea negotiations, by judges at sentencing, jailers operating risk reduction programs, probation officers making referrals to community treatment programs); and employing professional skills to positively influence defendant and offender behavior.

Purpose

The purpose of developing a system wide scorecard is to “measure what matters.” While the measurement of “activities” (e.g., pre-plea assessments of defendants, quality assurance to determine whether risk tools are completed properly) and “outputs” (e.g., percent of professionals trained in the use of a new tool or methodology, percent of sentence conditions informed by risk/needs assessments) in the system logic model is important, these are means to an end, not the end themselves. Articulating the ends we seek to achieve—and measuring those—focuses attention on the work that is critical to achieving a jurisdiction’s vision of the justice system. It also equips leaders with statements of intent they can use to clearly communicate with community members and other stakeholders about the purposes and goals of the justice system.

Participants

This document was developed to assist EBDM policy teams in identifying the harm reduction goals they seek to achieve through their policy change work.  All policy team members should be involved to some extent in the development of your harm reduction goals and scorecard.

Instructions

  1. Working as a team, identify the evidence-based decision making changes that are under consideration.[1]  Using the logic model template, identify the “impacts” you want to achieve through these policy change initiatives. These impacts are your jurisdiction’s harm reduction goals.[2]
  2. List the goals on a flip chart. As a team, determine whether you have consensus around the importance of each goal. If not, work to achieve consensus.
  3. Examine the examples of scorecards contained in this kit. As a team, agree to adopt a design for your scorecard, either by selecting one of the templates provided or by creating your own. Include your “identity” on your scorecard. [3]
  4. Next, discuss and agree with your team how you will measure the system’s performance in regard to each of these harm reduction goals. These discussions may be lengthy and may require expert consultation from those within your agencies and system—particularly your research, planning, and information technology staff—and perhaps outside expertise.[4]
  5. Once the methods to collect and assess performance on your harm reduction goals are determined, be sure to collect baseline data.[5] Baseline data indicates your “starting place,” or basis of comparison.
  6. Finally, discuss how and when the scorecard data will be collected and used. Be clear and specific about this; there is no sense in establishing goals that will not be measured or in collecting data that will not be analyzed and examined for its implications. Perhaps the policy team will task specific individuals with collecting and analyzing performance measurement data and reporting this information back to the policy team on a quarterly basis. Results may be included in agencies’ annual reports or in periodic press briefings. Most importantly, if reported results are less than expected, it is critical that the policy team reexamine the conditions, assumptions, resources, activities, outcomes, and outputs related to the implemented policy and practice changes to determine why the expected results have not occurred, and that the team make appropriate modifications so that results do, in fact, improve over time.

Tips

  • Don’t attempt to develop a lengthy list of scorecard items. Agreeing on two, three or four significant goals that everyone is in full agreement with is superior to a laundry list of less significant accomplishments, or goals that do not have full support of the full team. In addition, as a part of your communications strategy, you won’t want the scorecard to be too lengthy, or to lack support of the full team.
  • Be clear regarding your definitions for key words. For example, “recidivism” is often defined in multiple ways. Refer to the starter kit on Measuring Your Performance for a list of definitions that you might choose to draw from, or at least use as a starting place for the development of your own definitions. Whether you use the provided definitions, or definitions of your own making does not matter; what matters is that you are clear on what you mean by these terms, and that your team is in agreement on these definitions.
  • Follow the SMART principle when developing goals for your scorecard:
  1. Be Specific
  2. Make them Measurable (i.e., quantifiable)
  3. Be Action-oriented
  4. Be Realistic
  5. Articulate a Time in which the change will occur
  • When you’ve completed your list of harm reduction goals/scorecard items, it should elicit a reaction of satisfaction. Ask your team, “Would you feel proud to have been a part of the achievement of these goals?” When everyone responds in the affirmative, chances are you’ve succeeded in the development of your scorecard. 

Example:
Eau Claire County, Wisconsin, System Scorecard

ebdm score card

 

Example:
Charlottesville-Albemarle County, Virginia, System Scorecard

ebdm scorecard nc

Example:
Mesa County, Colorado, Systemwide Scorecard

ebdm scorecard az

Additional Resources/Readings

NIC. (2010). Achieving, measuring, and maintaining harm reduction and advancing community wellness. A Framework for Evidence-Based Decision Making in Local Criminal Justice Systems (pp. 22).
Retrieved from https://cepp.com/wp-content/uploads/2018/10/A-Framework-for-Evidence-Based-Decision-Making-in-State-and-Local-Criminal-Justice-Systems.pdf

Minnesota Department of Administration. (2002.) Minnesota milestones: Measures that matter. Retrieved from https://mn.gov/admin/demography/data-by-topic/minnesota-milestones/

 

[1] For more information, see: 3e: Prioritizing Your Team’s Targets for Change.

[2] See 5a: Building Logic Models and 6a: Measuring Your Performance.

[3] For more on developing an identity, see 7a: Developing a Communications Strategy; Building Stakeholder and Community Engagement.

[4] See 6a: Measuring Your Performance.

[5] See 3d: Gathering Baseline Data.

7: Gain Support

7: Gain Support web_admin

Activity 7:
Engage and gain the support of a broader set of stakeholders and the community

Building awareness and understanding and soliciting support for EBDM from a broader set of stakeholders and from the community is critical to the achievement of successful systemwide change. Developing a communications strategy is therefore critical to achieving greater buy-in to the harm reduction outcomes that a policy team hopes to achieve.

Elements of an EBDM justice system include a strategy for engaging stakeholders within the justice system and those in the broader community in meaningful dialogue about the vision/goals of the justice system, the state of knowledge and research, and the local system’s performance in achieving these goals.

7a: Developing a Communications Strategy; Building Stakeholder and Community Engagement

7a: Developing a Communications Strategy; Building Stakeholder and Community Engagement web_admin

Introduction

The EBDM Initiative seeks to create a set of conditions under which harm and risk reduction are realized to their true potential. Agreement on a systemwide vision and methods to assess its achievement, collaboration at the policy level, and careful analysis and application of the research in ways that ensure evidence-based decision making and practice are all important but insufficient to the achievement of desired outcomes. Without the understanding and “buy-in” of stakeholders—both within the system and, as importantly, in the public—change of the order described in the Framework is unlikely to take root and flourish.

Purpose

Broadly, the purpose of developing a communications strategy is to facilitate understanding of, and support for, evidence-based decision making policies and approaches. The specific aims of a jurisdiction’s communications plan include the following:

  • To raise awareness and educate stakeholders about the value of evidence-based decision making as an enhancement to existing justice system practices.
  • To engage interest in, and support for, such an approach among those who oversee, work within, interact with, and/or are affected by the local criminal justice system.
  • To engage stakeholders in a purposeful way in the identification and/or implementation of harm reduction strategies that will support healthier communities.

Participants

This document was developed for EBDM policy teams (and/or their work groups) to advance their efforts to engage stakeholders—both internal and external to the justice system—in the EBDM Initiative and in jurisdictions’ broad harm reduction goals.

Instructions

Adopting a Consistent Message

While Milwaukee County, Wisconsin's communication plan involves outreach to a variety of audiences, including business leaders, citizens, elected officials, educators, and the media, a consistent message is communicated:

Our commitment to the discipline of EBDM will enable us to hold offenders accountable, reduce the overall crime rate and recidivism, and give taxpayers a better return on the dollars they invest in criminal justice.

To begin, consider the following questions to ensure a thorough understanding of the place from which your communications planning effort is starting:

  • Who are the audiences you are trying to reach? Consider those within the local and perhaps state justice system (e.g., policymakers, supervisors, and/or staff) and audiences external to the justice system (e.g., community leaders, the general public).
  • What information are they currently receiving?
  • Who communicates with these audiences regarding justice-system related matters in an official communications capacity (e.g., public information officers) and/or as part of their role (e.g., chiefs of police or district attorneys conduct routine roundtables with civic groups; probation officers and detectives are members of ad hoc public education committees that educate communities on offender reentry issues)?
  • For each audience identified, think about (and perhaps create a matrix that identifies) the following:
    • What are the one or two primary ways to reach each audience (e.g., newspaper article, radio broadcast, speech, on the Web)?
    • What do you want each audience to know?
    • What is their current base of knowledge—that is, where are you starting from? Is this a well-informed audience?
    • What is the audience’s perspective? For example, does this audience have a positive viewpoint on the topics you want to discuss?
    • Who is best positioned to communicate with this audience, and how?
  • In what ways are current communication efforts working effectively?
  • In what ways could or should these efforts be expanded?
  • Based upon the answers to these questions, and after reviewing the “Tips” section below, build an action plan for your communications strategy. It may include specific, one-time events for specific audiences (e.g., a presentation to the business leaders’ quarterly network meeting; a briefing of justice system professional staff on the EBDM Initiative) or a series of events for various audiences (e.g., a series of briefings over the course of six months with three specifically identified local journalists; a series of training events on specific topics for a multidisciplinary group of professional staff). It will likely include a mix of long-term, big-picture topics (e.g., how the justice system operates, strategic action plans being developed or underway) and specific event-related strategies (e.g., highlighting the story of an offender who successfully completed supervision, launching a new program or policy approach).

Tips

Grant County, Indiana's Core Message

EBDM is the thoughtful stewardship of the public's money and trust in operating an efficient and effective criminal justice system.
  • Consider crafting a set of communications messages. Possible examples include the following:
    • Our communities can do better (than a 67% failure rate); we can create safer communities; we can reduce harm; we can have fewer crimes and fewer victims.
    • A local criminal justice system informed by research can point the way because it places the highest premium on outcomes, on the individual and institutional actions that produce them, and on the careful, ongoing measurement of them.
    • An evidence-based approach should not replace discretion and judgment, but it can inform and guide that judgment to enhance the likelihood that desired outcomes will be achieved.
    • A common local vision, internal collaboration, interagency partnership, public involvement, and shared responsibility are indispensable building blocks for alleviating community harm.
    • Consider developing an identity. One resource is provided by the national EBDM Initiative team; jurisdictions may choose to adopt this identity or to develop their own.
      • The national EBDM Initiative team created an “interactive” graphic that encourages decision makers to “complete” its concept with one or more words capturing the forms of harm they, their staffs, and their communities most desire to reduce (i.e., the phrase “One less _____” accompanied by “A strategy for safer communities” as its tagline). The graphic was designed to stand alone as a deliberately incomplete thought to pique curiosity or, for particular audiences, to be filled in with words such as “victim,” “crime,” “inmate,” “offender,” “dollar spent,” “officer injured,” or “court case.”
    • Consider developing communications tools and materials, for example:
      • a scripted “elevator speech” incorporating the key messages. (An "elevator speech" is an overview of an idea for a product, service, or project. The name reflects the fact that it should be possible to deliver the speech in the time it would take for an elevator ride, that is, approximately 30 seconds.)
      • local criminal- and victim-focused case stories that have strong emotional impact.
      • well-designed, appealing pamphlets that replicate the elevator speech in bullet form and include human interest stories.
      • video clips by local champions that illuminate the aspirations of local policymakers, specific approaches or challenges, etc.
      • a presentation of the overall project to be used at stakeholders’ meetings (i.e., a core set of slides augmented by stakeholder-specific slides and jurisdictional findings from the assessment phase). The EBDM Initiative team has developed a “core training curriculum,” available on SharePoint, that can be tailored for local purposes.
      • stakeholder-specific material for staff on the elements of the Framework and the jurisdiction’s implementation plan that is applicable to their role in the justice system.
      • training materials for line staff that tie specific policy and procedure changes to specific research supporting such changes.
      • print communications (e.g., posters, banners, brochures, progress reports) directed at staff and displayed in offices. Examples include a “One Less” brochure or “One Less” posters that feature the name and photo of EBDM policy team members and their “One Less” aspirations. (See an example below.)
      • promotional items and giveaways for staff (e.g., t-shirts, coffee mugs, and/or pens) that encourage the Initiative and remind and excite staff about change.
National Survey on EBDM

In Phase I of the EBDM Initiative, the national Initiative team worked with Zogby International to develop and administer a national public opinion survey.

This tested survey offers a model that could be replicated at the local level and a set of findings against which local results can be compared. If the data align with the findings of the national poll, they will provide the impetus, as well as political coverage, for difficult decisions. Even in the event that the data do not align with national findings, they will become an integral part of the development of local messages.

A fact sheet that summarizes the findings of the national public opinion survey can be found here: fact sheet
  • Consider conducting a public opinion survey and/or focus groups.
    • Conduct a public opinion survey that measures citizens’ opinions on the justice system, its purpose, and the extent to which the system should rely on research, and citizens’ satisfaction with current justice system outcomes. For a list of questions that were used in a national survey, see the Appendix.
    • Using subject matter experts, convene focus groups with the general public to better understand their views on matters related to the justice system and evidence-based decision making and/or as a means to effectively communicate with and engage citizens on these matters.
  • Consider the development of a deliberate and purposeful public communications strategy using the media and other means.
    • Prepare news releases and Op-Ed pieces; talking points for speeches at local gatherings, professional conferences, radio talk or call-in shows, news conferences, one-on-one meetings and open houses at stakeholders’ offices, newspaper editorial board meetings, etc.; public service announcements; and print communications campaigns (e.g., posters, brochures, press kits, web-based reports).
  • Understand the research and collect data.
    • Examine the research on effective communication strategies and campaigns to determine how this body of knowledge can best inform and shape your own efforts.
    • Collect qualitative and quantitative information to determine the extent to which your communications tools and methods accomplish their intended purpose. One qualitative method of measurement would be to conduct a series of focus groups with local system stakeholders and the general public. Quantitative methods of measurement would involve pre- and post-testing of training modules, pre- and post-measurements of staff attitudes, or a fuller use of local public opinion polling. One possible strategy is to conduct a baseline poll at the launch of the communications strategy and then a second poll at a specified date in the future to measure change in both public and staff attitudes.

Example:
Eau Claire County, Wisconsin,
Article in Rotary Club Newsletter, January 10, 2011

Example

 

Additional Resources/Readings

 

Appendix:
EBDM Public Opinion Survey Questions

 

  1. Which of the following do you think should be the primary purpose of the criminal justice system?
    • Punishing those who commit crimes
    • Reducing the likelihood that convicted offenders will commit new crimes
    • Protecting the rights of people accused of crimes
    • Addressing the interests of victims of crimes
    • Not sure/Other
  2. Have you, a family member, or anyone you know ever served time in a jail or prison?
    • Yes
    • No
    • Not sure
  3. The criminal justice system should make neighborhoods safer?
    • Strongly agree
    • Somewhat agree
    • Somewhat disagree
    • Strongly disagree
    • Not sure
  4. The criminal justice system should increase the confidence of the public in the criminal justice system?
    • Strongly agree
    • Somewhat agree
    • Somewhat disagree
    • Strongly disagree
    • Not sure
  5. The criminal justice system should reduce taxpayers’ costs for public safety?
    • Strongly agree
    • Somewhat agree
    • Somewhat disagree
    • Strongly disagree
    • Not sure
  6. The criminal justice system should strengthen the well-being of offenders’ families?
    • Strongly agree
    • Somewhat agree
    • Somewhat disagree
    • Strongly disagree
    • Not sure
  7. If research consistently showed that there are ways other than jail to deal with people who are convicted of non-violent crimes that could reduce the chances they will commit new crimes, would that information make you more or less likely to support alternatives to jail?
    • Much more likely
    • Somewhat more likely
    • Somewhat less likely
    • Much less likely
    • Not sure
  8. If research consistently showed that there are ways other than jail to deal with people who commit violent crimes that could reduce the chances they will commit new crimes, would that information make you more or less likely to support alternatives to jail?
    • Much more likely
    • Somewhat more likely
    • Somewhat less likely
    • Much less likely
    • Not sure
  9. To what extent do you agree or disagree with this statement: “We should increase spending on approaches proven to reduce the chances that offenders will commit new crimes”?
    • Strongly agree
    • Somewhat agree
    • Somewhat disagree
    • Strongly disagree
    • Not sure
  10. When criminal justice officials make decisions, what should be the most important thing they rely on?
    • Research on what works in preventing crimes
    • Their professional experience
    • Their personal beliefs on what’s the right thing to do
    • Not sure/Other
  11. Knowing that research shows that about half of the people released from prison eventually go back to prison and about a third of the people on probation commit new crimes, to what degree do you think these results are acceptable?
    • Very acceptable
    • Somewhat acceptable
    • Somewhat unacceptable
    • Not at all acceptable
    • Not sure

Questions 12–17:
Doctors use research about risk factors to help identify which people are more likely to have a heart attack. Similar research about risk factors exists that helps identify which offenders are more likely to continue to commit crime. Should this research be used …

  1. By prosecutors when they decide what sentence to recommend to the judge?
    • Yes
    • No
    • Not sure
  2. By corrections officials when making decisions about release from jail or prison?
    • Yes
    • No
    • Not sure
  3. By defense attorneys when they are helping their clients?
    • Yes
    • No
    • Not sure
  4. By judges when deciding the appropriate sentence?
    • Yes
    • No
    • Not sure
  5. When deciding if a person should be released from jail on bail until their trial?
    • Yes
    • No
    • Not sure
  6. By the police when deciding to make an arrest?
    • Yes
    • No
    • Not sure
  7. Which of the following statements do you agree with more?
    Statement 1: "The most important thing in dealing with people who have committed a crime is to see to it that the punishment fits the crime."
    Statement 2: "The most important thing in dealing with people who have committed a crime is to do things that will reduce the chances they will commit future crimes."
    • Statement 1
    • Statement 2
    • Neither
    • Not sure/Other
  8. If two people were convicted of the same kind of crime, but one of them is more likely to commit crime in the future, what should happen in terms of sentencing?
    • Both people should be treated exactly the same.
    • The person more likely to commit a crime should be sentenced differently.
    • Not sure/Other
  9. To what extent do you agree or disagree with this statement: “Criminal justice officials should tell the public how well they are doing at reducing crimes”?
    • Strongly agree
    • Somewhat agree
    • Somewhat disagree
    • Strongly disagree
    • Not sure

8: Action Plan

8: Action Plan web_admin

Activity 8:
Develop a strategic action plan for implementation

The culmination of the process of building a justice system based on evidence-based decisions is the creation of a clear, specific, measurable plan for implementing the policy and practice changes that the policy team has agreed upon. This plan will serve as the new roadmap for the team as it begins to implement its steps to harm and risk reduction.

Elements of an EBDM justice system include a clear, specific, measurable implementation plan.

8a: Building a Plan for Implementation

8a: Building a Plan for Implementation web_admin

Introduction

Considerations for Developing Harm Reduction Goals and Objectives
  • How will the jurisdiction benefit as a whole (i.e., what are the intended harm reduction outcomes)?
  • How will the criminal justice system benefit from movement to an EBDM-based system?
  • What is an EBDM system intended to achieve or produce?
  • What significant changes do you expect from the implementation of EBDM in terms of system operation?
    • How will the costs to operate the system change?
    • How will case processing change at point of entry into the system, during the adjudication process, post-adjudication, and/or at point of release?
    • How will those in the system (i.e., victims, witnesses, and defendants) view the process?
  • How will EBDM impact those working in the system?
  • What types of information will convince you and others (including the public and funders) that the system is operating at an optimum level?
  • What types of information will convince you and others that the system is achieving what it is intended to achieve?

For more information, see 6a: Measuring Your Performance.

During the EBDM Initiative, your policy team has undertaken a number of preparation activities for implementing the Framework. These activities include

  • building a collaborative, multidisciplinary policy team;
  • preparing the team members’ individual agencies for change;
  • understanding current practice within each agency and across the system;
  • understanding and increasing your jurisdiction’s capacity to implement evidence-based practices;
  • developing logic models;
  • establishing common harm and risk reduction outcomes and performance measures (and displaying them on a system scorecard); and
  • developing plans for engaging broader support for the Initiative.

The culmination of these preparations leads your team to this final, but critically important, step: to develop a strategic action plan for implementation.

Purpose

To create a clear, specific, measurable plan for implementing the policy and practice changes that the policy team agrees will advance evidence-based decision making in your jurisdiction and that will support the achievement of the justice system’s vision and goals.

Participants

All policy team members should be involved to some extent in the development of your implementation plan, particularly in the development of harm reduction goals and objectives. After these decisions have been reached, staff internal to the agency(ies)—usually with some background in conceptualizing, planning, and implementing policy or program initiatives—and/or outside experts can assist in the development of the implementation plan, with guidance and input from the policy team.

Instructions

A number of preparation and self-assessment activities must occur simultaneously to lay the groundwork for implementing the EBDM Framework in a jurisdiction. These activities include developing harm reduction goals, objectives, and action steps; developing a systemwide logic model; drafting a communications strategy for gaining the buy-in of a broader set of stakeholders or the public; and creating a systemwide scorecard.[1] While every team will not develop its plan in the same way, the following steps are important to developing a comprehensive implementation plan:

  • Discuss and agree upon your team’s harm reduction goals, if your team has not come to some agreement on this already.[2]
  • Develop logic model(s). At a minimum, the team should develop a systemwide logic model that clearly outlines the path to achieving the team’s top harm reduction goals.[3] This activity will assist the team in developing many of the pieces of its implementation plan.
  • Develop objectives (which should be represented as outcomes in your logic model). Remember, while goals represent the desired end results of the system, objectives define the short-term indicators that demonstrate progress toward goal attainment and describe who or what will change, by how much, and over what period of time.
  • Define the action steps that will be necessary to achieve your harm reduction goals. (The major action steps can be found in the activities section of the logic model.)
    • Determine who from your jurisdiction will take the lead and who will need to be involved in these steps.
    • Determine the timing and sequence of these steps.
    • Consider any potential barriers to your work plan and strategize about how your team will overcome them. Barriers can be determined by considering the contextual conditions (i.e., the environment in which the local justice system operates, including political, economic, social, and cultural factors) that your team identified in your logic model.
    • Discuss how your team would like to engage a broader set of stakeholders and/or the public in EBDM, if you have not done so already.[4] Ensure that any agreements regarding this strategy are reflected in your work plan; these may encompass goals, objectives, and/or action steps, as appropriate.

The chart below displays these steps and indicates how these multiple activities might fit together.

Possible Steps to Developing an Implementation Work Plan

Step 1:
Develop Harm Reduction Goals

Develop the long-term harm reduction goals your team seeks to achieve.

 

Your harm reduction goals are recorded on your system scorecard.

Your harm reduction goals are the impacts on your logic model.

Your harm reduction goals are the “goals” on your work plan.

 

Harm Reduction Goal Example: Increasing the success rate of individuals who become involved in the justice system from the 2010 rate of x% to y% by 2014

Step 2:
Develop a Logic Model

After recording your harm reduction goals as the impacts on the logic model, follow 5a: Building Logic Models in order to determine the

  • short-term outcomes;
  • outputs;
  • activities;
  • inputs/resources; and
  • contextual conditions.

 

Once your logic model is complete, you can use the information it contains to build the rest of your work plan and scorecard.

Step 3:
Develop Objectives

Objectives define the short-term indicators that demonstrate progress toward attaining your harm reduction goals and describe who or what will change, by how much, and over what period of time.

 

Your objectives are the short-term outcomes on your logic model.

Your objectives are recorded as such on your work plan.

 

Objective Example: Decrease of X% in low risk defendants held in jail awaiting adjudication within X months

Step 4:
Develop Action Steps

Action steps are the “activities” on the logic model—the steps that must be taken to reach the objectives that will lead to your harm reduction goal. Since only major activities are likely included on the logic model, expand these—if and as needed—on your work plan to reflect all of the planned action steps.

 

Include as an action step on the work plan the development of agency-level logic models for all agencies significantly involved in the achievement of the objectives.

 

Actions Step Example: Train pretrial staff on use of assessment tool.

Step 5:
Determine Who Is Responsible/ Involved

Determine the person(s) responsible for accomplishing each action item, the person(s) responsible for decision making, needs related to resource allocation, and coordination with other entities. Record these assignments on the work plan.

 

Step 6:
Determine Timing and Sequencing

Define the timing and sequencing of the action steps. Record this information on the work plan.

Step 7:
Recognize Potential Barriers to Implementation

Consider the contextual conditions in your logic model and describe the potential barriers to implementation and strategies for addressing these barriers. Record these on the work plan.

Step 8:
Develop a Communications Strategy

If one or more harm reduction goals in your work plan do not include engaging new stakeholders, increasing support and engagement from the community, or communicating the jurisdiction’s harm reduction goals to the public, develop a strategy for doing so. Include it as an objective with action steps on the work plan.

 

Refer to 7a: Developing a Communications Strategy.

 

A template of a work plan is provided in the Appendix. It illustrates how the multiple elements of the work plan might be displayed in chart form.

Tips

  • It may not be possible to forecast the very specific steps for activities that will be accomplished in the later months; try to develop in more detail the more immediate tasks (i.e., 3–4 months) that need to be accomplished.
  • Teams may find that creating a visual timeline, separate from the work plan, is helpful in organizing the many anticipated tasks. An example of a timeline is provided.
  • If certain baseline data is not available, make sure to include in your work plan the anticipated steps your team will need to take to collect it.
  • Teams should revisit their implementation plans regularly to make revisions and adjustments as needed.

 

Example:
Mesa County, Colorado, Work Plan for Implementation (Excerpt from Full Document)

Arrest Decision

Harm Reduction Goal

By 2015, 75% of all offenders successfully completing sentences will not recidivate.

 

Objective 1

75% of staff trained will demonstrate a 50% increase from pretest to post test in knowledge and understanding of EBDM and Proxy Tool use.

Objective 2

Within 6 months of implementation, 95% of all arrest cases originating out of the Mesa County Sheriff’s Office will have a Proxy risk score in the narrative or on the summons.

Objective 3

For all arrestees who are assessed using the Proxy Tool by the Mesa County Sheriff’s Office, less than 20% of low risk offenders will be put in jail.

 

Date of Completion

Lead Person

Others Responsible

Resource Needs

Partner Coordination

Action Step 1

Incorporate C.R.S. 16-5-207(2) into existing Arrest Standards

August 1, 2011

Sheriff Stan Hilkey

Captain Steve Farlow, MCSO

Staff time

 

Action Step 2

Publish and implement new standards

Upon completion. Law in effect.

Sheriff Stan Hilkey

Captain Steve Farlow

Staff time

Grand Jct PD

Fruita PD

Palisade PD

Colorado State Patrol

Action Step 3

 

 

Develop training syllabus for patrol officers on EBDM and use of Proxy Tool

November 1, 2011

Sheriff Stan Hilkey

Bert Nieslanik

Staff time

 

Action Step 4

Develop Pocket Tool Proxy Instrument to be used by Patrol Officers

November 1, 2011

Sheriff Stan Hilkey

Bert Nieslanik

$ and staff time

 

Action Step 5

Develop Pre and Post Test on EBDM and Proxy use for Patrol Officer training

November 1, 2011

Sheriff Stan Hilkey

Bert Nieslanik, Jennifer Sheetz

Staff time

 

Action Step 6

Develop policy for Mesa County Sheriff’s Office to use Proxy Tool and produce score on all summons and arrest documents and cases

December 1, 2011

Sheriff Stan Hilkey

 

Staff time

 

Action Step 7

(Measurement)

Training, including pre and post testing, for all Patrol Staff in Mesa County Sheriff’s Office on EBDM, use of Proxy Tool, Proxy
Tool Pocket Guide, new MCSO policy, and implementation. Track #of Deputies trained

January 1, 2012–March 31, 2012

Sheriff Stan Hilkey

Bert Nieslanik

Staff time

 

Action Step 8

Implement use of Proxy Tool, as trained, by all Sheriff’s Office Patrol Staff

April 1, 2012

Sheriff Stan Hilkey

 

Staff time

 

Action Step 9

Develop training program for all newly hired MCSO staff and ongoing in-service training on EBDM and Proxy Tool use

April 1, 2012 and ongoing as needed

Sheriff Stan Hilkey

 

Staff time

Field Training Officers

Action Step 10

(Measurement)

Audit compliance and use of Proxy Tool by MCSO Patrol Deputies, produce data regarding % of summonses and arrest reports containing Proxy score

3, 6, & 12 months from implementation date

MCSO Compliance Officer Susan Redmond

Sheriff Stan Hilkey

Staff time and access to records

 

Action Step 11

Track # of defendants arrested, # of defendants issued summonses, # of deviations from risk results, # of defendants with new charges post arrest, and # of defendants with new charges post summons

April 1, 2012 and ongoing

Pretrial Services

 

Staff time and tracking tools

County Court, Sheriff’s Office

Action Step 12

For use of all other local law enforcement agencies, incorporate Proxy Tool use and scoring procedures into Mesa County Arrest Standards DRAFT document

October 1, 2012

Sheriff Stan Hilkey

Bert Nieslanik

Staff time

 

Action Step 13

Training of all Mesa County Patrol Officers from all agencies, including pre and post testing, on EBDM, use of Proxy Tool, Proxy
Tool Pocket Guide, recommended policy guidelines, and implementation. Track #of Officers trained

January 1, 2013–June 30, 2013

Sheriff Stan Hilkey and Staff

Bert Nieslanik

Staff time

Grand Junction Police, Fruita Police, Palisade Police, and Colorado State Patrol

Action Step 14

Implement new arrest standards with Proxy Tool use for all Mesa County Law Enforcement

July 1, 2013

All agencies

 

 

 

Action Step 15

Development of agency/case-level logic model

August 1, 2011

Sheriff Stan Hilkey

Executive Committee

 

 

Potential Barriers

Culture change of understanding EBDM and successful offender management post-arrest

Strategies to Address Barriers

Training, education, data collection

 

Pre-Sentence Investigations Report

Harm Reduction Goal

By 2015, 75% of all offenders successfully completing sentences will not recidivate.

 

Objective 1

Reduce the risk for future harm to members of our community by designing and implementing a PSIR that addresses criminogenic needs, thereby allowing informed sentencing decisions, reducing the likelihood of future offending.

 

Date of Completion

Lead Person

Others Responsible

Resource Needs

Partner Coordination

Action Step 1

Build in temporary compliance with HB 1180 to address criminogenic factors in the PSIR

Aug 1, 2011

Susan Gilbert

Janelle Carstens/Probation Supervisor

Staff/
Consultation time

DPS

Action Step 2

Develop and implement a training program to enhance awareness of HB 1180 and how the LSI is administered, scored, and incorporated into the temporary PSIR. This training will include a survey of stakeholders’ feedback on content preferences.

Aug 11, 2011

Probation

CJSD

Outside agency consultant

DCJ, NIC, DPS

Action Step 3

Implement temporary changes to PSIR

Aug 11, 2011

Probation

 

 

 

Action Step 4

 

Identify and establish a PSIR design workgroup

Aug 15, 2011

Susan Gilbert/

Probation

DA, bench, PD, CJSD, ADC, private defense bar

Consultation

DPS-SCAO

Action Step 5

 

 

Research and evaluate statute, HB 1180, Colorado Probation Standards, and survey feedback and define target population for PSIR

Sept 15, 2011

Chair of PSIR Design Group

Judges, DA, PD, Defense bar, Community Corrections, Probation

Consultation

DPS-SCAO

Action Step 6

 

Develop a draft of proposed changes to PSIR

Nov 1, 2011

PSIR Design Group

Local and State

Consultation

DPS-SCAO

Action Step 7

Present to stakeholders for feedback/approval

Nov 1, 2011

Susan Gilbert/Bert N./D.A.

 

 

N/a

Action Step 8

Update final version of PSIR

Dec 1, 2011

PSIR Design Group

 

Consultation

DPS-SCAO

Action Step 9

 

 

Train stakeholder staff on redesigned PSIR and how it can be applied to sentencing decisions

Feb 15, 2012

PSIR Design Group

 

 

 

Action Step 10

 

Implement pilot in Judge Bottger’s court for 6 months

Mar 1, 2012

Judge Bottger

 

 

 

Action Step 11

Develop agency/case-level logic model

August 1, 2011

Susan Gilbert

Executive Committee

 

 

Potential Barriers

 

 

The content of the PSIR is set by statute, although recent statute requires that the PSIR include criminogenic needs effective 8/10/2011. In addition, State probation has a standardized format that is currently being written into the new database for electronic dissemination. Getting approval by all stakeholders of the content will be a challenge as all parties have strong opinions regarding content based on their roles in the system. Lack of exposure and awareness of the LSI content, coupled with how it is completed, could be a significant barrier to constructive conversations regarding the content and acceptance of a revised PSIR and/or summary page. Another barrier is limited resources (manpower) to dedicate to meetings.

Knowledge of an offender’s risk level needs to be available for the court to determine who should receive a comprehensive risk/needs assessment and a PSIR.

Strategies to Address Barriers

 

Work with the Colorado Department of Probation Services to address any concerns with the format or the pilot court in an effort to develop a format that will be implemented statewide. Encourage representation and participation of all PSIR stakeholders in the design of the PSIR so it is perceived by all stakeholders as a useful tool for making informed decisions. Training regarding the LSI will be essential in getting buy-in.

Continue to include defense bar and DA in conversations, trainings, and work groups to build confidence in the LSI tool and in the content of the PSIR.

 

Pilot Courtroom

Harm Reduction Goal

By 2015, 75% of all offenders successfully completing sentences will not recidivate.

 

Objective 1

50% reduction in defendants appearing in pilot division who spend more than 7 days in pretrial custody within six months of implementation of CISPR pretrial assessment tool

 

Date of Completion

Lead Person

Others Responsible

Resource Needs

Partner Coordination

Action Step 1

 

 

Begin to consider results of CISPR pretrial assessment tool adopted via pretrial work plan in making release decisions

3/1/12

Bottger

DA, PD, ADC, private defense bar

Approved tool, agency to administer

CJSD (administering agency)

Action Step 2

 

 

Gather 2006 baseline data (pilot judge division) on time spent in pretrial custody, FTA, and reoffense rate for those released

5/1/12

Sheetz, Casselberry

Jail staff, court staff

Time

 

Action Step 3

 

 

Develop and implement plan to gather current data from pilot division on time spent by defendants in pretrial custody, FTA, and reoffense rate of those released

5/1/12

Sheetz, Casselberry

Jail staff, court staff

Time

 

Action Step 4

Compare results to baseline

7/1/12

Sheetz

 

 

 

Action Step 5

Consider results and implications, including changing tool and changing court practices

8/1/12

Bottger

DA, PD, ADC, private defense bar, CJSD, SO

 

 

Action Step 6

Develop agency/case-level logic model

8/1/11

Judge Bottger

Executive Committee

 

 

Potential Barriers

 

 

1) If the CISPR Tool is not available by March 1, 2012, this could require us to either delay use of a risk assessment tool or start with one tool and then switch to the Colorado tool when it is available, complicating outcome measurement.

2) We will need to overcome any reluctance on the part of defendants and their attorneys to submit to a pre-disposition assessment of any kind, even if it does not expressly call for incriminating information.

3) Timing of administration is critical. If administered before the defendant first appears in county court for video arraignment, the county court judge will have the benefit of the results. This will not only allow for earlier release, it will reduce the likelihood that the district court will significantly change the bond or bond conditions. Although such a change is not harmful per se, it could create the impression that the district court was critical of the county court’s bond decision.

4) A more liberal bond philosophy may result in fewer people entering substance abuse treatment as a bond condition.

Strategies to Address Barriers

 

1) Identify key players in CISPR development. See if district can help move along in any manner, including volunteering as a pilot district.

2) As CISPR has not been finalized, it is unknown what questions it will ask. Regardless, we will likely gain the confidence of the defense bar only through experience.

3) One solution is to get a commitment from the county court judges to follow EBDM principles in setting bond for felony defendants.

4) Monitor participation in “fast-track” meth treatment program.

Communications Strategy

 

Create an “interested players and parties” distribution list and send weekly short, yet informative, emails.

Utilize mesacourt.org website to post updates for parties to access.

Hold brown bag lunches to generally discuss pilot and ongoing results.

 

Objective 2

Within 12 months of implementation of sentencing guide, at least 30% of defendants sentenced to probation or community corrections will show a 10 point reduction in LSI score from sentencing to end of sentence

 

Date of Completion

Lead Person

Others Responsible

Resource Needs

Partner Coordination

Action Step 1

 

 

Encourage defendants and attorneys from the bench to have a completed LSI for every defendant before sentencing in pilot division

Ongoing

Bottger

Probation, defendants, defense counsel, DA’s

LSI, trained personnel to administer (already in place)

 

Action Step 2

Use results of LSI, including new summary sheet, to inform sentencing decisions (get low risk people out of system, impose conditions to address 1–2 greatest needs of rest)

Ongoing

Bottger

DA’s, defense counsel, defendants

 

 

Action Step 3

 

 

Use motivational interviewing techniques at sentencing

Ongoing

Bottger

PO’s trained in MI to monitor and suggest improvements

PO’s trained in MI (already in place); periodic judge training

 

Action Step 4

Gather research on value of periodic in-court reviews, including whom to include and how often

8/1/11

Bottger

Modley

Current research

 

Action Step 5

Conduct periodic in-court reviews of medium and high risk offenders, if supported by research

Ongoing

Bottger

DA’s, defense counsel, probation, CJSD

Time

 

Action Step 6

Re-administer LSI to offenders near end of sentence

Ongoing

Probation

Defendants, defense counsel

Trained PO’s (already in place)

 

Action Step 7

Gather baseline data on offenders sentenced in 2006

2/1/14

Casselberry, probation

 

Statistician

 

Action Step 8

Gather data on risk/needs of offenders sentenced during pilot project

2/1/14

Casselberry, probation

 

Statistician

 

Action Step 9

Compare data

4/1/14

Bottger

Casselberry, probation

Statistician

 

Potential Barriers

 

 

1) Because some members of the defense bar believe the LSI is biased and that results of any risk/needs assessment may portray some defendants in a less favorable light, some defendants will refuse to submit to an LSI or participate in the PSI process entirely.

2) Changing the format of the PSI is problematic because it has been standardized statewide.

3) Plea agreements that impose requirements inconsistent with LSI results

4) Lack of experience in MI techniques

5) We do not have a precise or measurable way to determine whether a sentence follows EBDM principles.

6) Outgoing risk/needs scores may be unavailable for some defendants sentenced in 2006.

Strategies to Address Barriers

 

1) Continue to meet with all parties to develop trust with one another on how the risk/needs information will be used. The courts should encourage participation in the PSI process and demonstrate how this information will be used in sentencing, proving that sentences will be imposed that match the risk/needs of the defendant. In addition, probation department will offer training to defense bar on LSI.

2) An acceptable alternative is to attach a face sheet to each PSI which gives the defendant’s risk level (low, medium, high) and identifies his or her top two to four criminogenic needs.

3) Persuading counsel and defendants to make open-ended plea agreements; rejecting agreements that impose conditions inconsistent with LSI results

4) Experience and training

5) Monitor research to see if anyone develops a way to determine whether a sentence follows EBDM principles.

6) Recognize limitations of data.

Communications Strategy

Distribute results as compiled to DA, PD, ADC, private defense bar, probation, CJSD, members of Executive Board, and Criminal Justice Leadership Council.

Consider public distribution.

 

Example:
Eau Claire, Wisconsin, Timeline for Implementation

timeline

 

Additional Resources/Readings

CSOM. (2007). Enhancing the management of adult and juvenile sex offenders: A handbook for policymakers and practitioners. Retrieved from https://cepp.com/wp-content/uploads/2020/01/6-CSOM-Handbook.pdf

CEPP. (2005). Collaboration: A training curriculum to enhance the effectiveness of criminal justice teams. Retrieved from http://cepp.com/wp-content/uploads/2020/03/Collaboration-A-Training-Curriculum-to-Enhance-the-Effectiveness-of-Criminal-Justice-Teams-2005.pdf

McGarry, P., & Ney, B. (2006). Getting it right: Collaborative problem solving for criminal justice. (NIC Accession No. 019834). Retrieved from https://nicic.gov/getting-it-right-collaborative-problem-solving-criminal-justice

Appendix:
Work Plan Template

Phase III Work Plan to Achieve Harm Reduction Goals

Harm Reduction Goal

 

 

Objective 1

 

 

Date of Completion

Lead Person

Others Responsible

Resource Needs

Partner Coordination

Action Step 1

 

 

 

 

 

 

 

Action Step 2

 

 

 

 

 

 

 

Action Step 3

 

 

 

 

 

 

 

Potential Barriers

 

 

 

Strategies to Address Barriers

 

 

 

Objective 2

 

 

Date of Completion

Lead Person

Others Responsible

Resource Needs

Partner Coordination

Action Step 1

 

 

 

 

 

 

 

Action Step 2

 

 

 

 

 

 

 

Action Step 3

 

 

 

 

 

 

 

Potential Barriers

 

 

 

Strategies to Address Barriers

 

 

 

[1] See 6b: Developing a Systemwide Scorecard.

[2] For more detailed information on this process, see the first step in 6a: Measuring Your Performance.

[3] See 5a: Building Logic Models.

[4] See 7a: Developing a Communications Strategy.

Videos

Videos web_admin

Wisconsin

Colorado

Products

Products web_admin

EBDM Readiness Checklist

The checklist is designed to serve as a catalyst for discussion on the team’s capacity and willingness to adopt the EBDM Framework. While the checklist may be administered in a variety of ways, it is critical that the results are discussed and processed by the full policy team.

Read more about the EBDM Readiness Checklist

EBDM Public Opinion Fact Sheet

"Criminal justice professionals have increasingly demonstrated an appreciation for and commitment to using research to guide policy and practice decisions. As part of an initiative designed to facilitate evidence-based decision making in local criminal justice systems, a national telephone survey was conducted1 to explore community members’ attitudes and expectations about the criminal justice system and, more specifically, the use of research to inform policy and practice decisions".

Read more the Fact Sheet

Resources

Resources web_admin

Primer

Case Studies

Stakeholder Briefs

User Guides

Related Work

Related Work web_admin
  • dosage probation

    Dosage Probation: Rethinking the Structure of Probation Sentences

    This monograph provides a policy and practice framework upon which a new model of supervision—"dosage probation"—can be constructed. It offers a review of evidence-based approaches to reducing recidivism in our communities, the most recent research on dosage, and its applicability to sentencing and community supervision practices. It describes the model's promise for increasing community safety through recidivism reduction, as well as achieving fiscal savings by reducing periods of supervision. Finally, the monograph offers a summary of the work of Milwaukee County's criminal justice stakeholders as they design and conduct the nation's first dosage probation experiment.

  • dosage probation

    Incarceration's Front Door: The Misuse of Jails in America

    Local jails, which exist in nearly every town and city in America, are built to hold people deemed too dangerous to release pending trial or at high risk of flight. This, however, is no longer primarily what jails do or whom they hold, as people too poor to post bail languish there and racial disparities disproportionately impact communities of color. This report reviews existing research and data to take a deeper look at our nation’s misuse of local jails and to determine how we arrived at this point. It also highlights jurisdictions that have taken steps to mitigate negative consequences, all with the aim of informing local policymakers and their constituents who are interested in in reducing recidivism, improving public safety, and promoting stronger, healthier communities.

  • Smart Supervision

    Smart Supervision

    The U.S. Department of Justice (DOJ), Office of Justice Programs (OJP), Bureau of Justice Assistance (BJA) is seeking applications for Smart Supervision: Reducing Prison Populations, Saving Money, and Creating Safer Communities. This program furthers the Department’s mission by providing resources to states, units of local government, and federally recognized Indian tribes to develop more effective and evidence-based probation and parole practices that effectively address individuals’ needs and reduce recidivism.

  • Evidence-Based Decision Making

    Toward Evidence-Based Decision Making in Community Corrections: Research and Strategies for Successful Implementation

    This special issue of Justice Research and Policy contains invited articles on community corrections, with special emphasis on successful implementation strategies. A common thread that runs through these articles relates to what is needed to better ensure fidelity to evidence-based practices in community supervision and treatment.

  • Measuring for Results

    Measuring for Results: Outcome and Performance Measures for Pretrial Diversion Field

    This publication outlines suggested outcome and performance measures and critical operational data for pretrial diversion programs. Its goals are to present clearly defined and easily calculable measures that pretrial diversion programs can use to gauge progress in achieving their mission and strategic goals, improve business decisions, and illustrate pretrial diversion’s value in an evidence-based criminal justice system.

  • Evaluation of Phase II

    Evaluation of Phase II Technical Assistance for Evidence-Based Decision-Making in Local Criminal Justice Systems

    This report describes the EBDM [Evidence-Based Decision-Making] Phase II technical assistance approach and presents findings and themes from the process evaluation and outcome assessment (conducted from October 2010 to February 2012) of the technical assistance delivered to the seven sites selected under Phase II of the EBDM initiative.

  • Evidence-Based Decision Making

    Evaluation Brief: Evidence-Based Decision Making in Local Criminal Justice Systems Initiative

    This brief covers results from the report "Evaluation of Phase II Technical Assistance for Evidence-Based Decision-Making in Local Criminal Justice Systems" by Janeen Buck Willison, Pamela Lachman, Dwight Pope, and Ashleigh Holand (issued June 2012)