Introduction

One of the most fundamental ways to develop an understanding of a jurisdiction’s justice system is to develop a “system map.” Similar to an architectural diagram, a system map depicts the steps in the criminal justice process (i.e., processing of a case and the activities related to this), beginning with police contact and ending with the point in time when the case terminates. In addition to reflecting the key decision points in the system, the map reflects the decision makers at each key point and the amount of time it takes a case to move from one point to the next. It is also possible and desirable to document the volume of cases that flow through each process step and decision point. This may be accomplished first by noting estimated numbers and later by gathering data on a specified period of time to more precisely determine the flow and volume of cases and activities.

In the EBDM initiative, the system map will be the first step in developing a detailed understanding of each justice system decision point and of the evidence that informs these key decisions. Following the completion of the system map, a process to “dig deeper” into these decision points will be carried out. This more in-depth analysis will include an examination of the following as they relate to each decision point:

• written policies;
• the application of those policies to practice, as well as to other operational practices that are not formally articulated in policy;
• the various types of data and information collected at each decision point;
• the ways in which this data and information inform decisions;
• the ways in which information is stored and shared; and
• other factors related to using data, information, and evidence in the most efficacious ways.

Purpose

Mapping serves a variety of purposes:

• It increases awareness of the ways in which the entire system “works” and how different parts of the system interact with one another. (Most people understand quite well their own “part” of the system but have a less detailed understanding of the other parts of the system.)
• It brings together policymakers and agency staff to articulate the decisions they make, how they arrive at those decisions, and when (i.e., at what point in the process) decisions are made.
• It surfaces areas of interest for further inquiry.
• It can sometimes lead to recognition of quick solutions to bottlenecks or inefficiencies.

Participants

Building a system map is a time-consuming process. The most desirable method is for the entire policy team to be fully involved in its development. However, given that this process can take a full day or more, this may not be practical. A second, effective option is to convene a series of “work groups” to help with the construction of the map.

Work groups might be formed around the following decision points:

1. arrest
2. pretrial status
3. charging/plea
4. sentencing
5. local institutional intervention(s)/local institutional release
6. community intervention(s)
7. violation response
8. discharge from supervision

Work groups should include a range of individuals who have different roles and perspectives in the case and decision making processes. Different levels of decision making authority should be represented as well, including the highest decision maker (agency head), those who oversee the work being conducted (supervisors/managers), and those at the line level. In many cases, work groups will also include those who handle information flow and case files. Receptionists, clerks, and administrative staff often have the best “window” on how things actually get done; they should not be overlooked in this process.

In addition, it is strongly recommended that the persons involved in information technology (data collection and analysis) participate in work groups. As noted above, it will be important to affix data to the map at some point in the future. Involving those who will be responsible for collecting and analyzing that data will facilitate their understanding of what is needed when the time comes to collect this data.

Finally, it is highly desirable for one or more persons from the team (i.e., local coordinator and/or another member) to be present for the entire mapping process, if at all possible. While this is a major time investment, the opportunity for at least one member of the jurisdiction to observe and learn from the entire process should not be underestimated.

Instructions

1. The policy team should have a discussion about mapping in advance of the mapping session:
   • Identify some of the issues and challenges the team most wants to learn about at each decision point to make sure these topics are addressed during the initial mapping session.
   • Identify the key individuals from each agency who have detailed knowledge about each decision point and make a plan for inviting them to attend each work group session. (See participants above.)
   • Identify the policy team member(s) who will attend the entire mapping session and who will be responsible, along with the local coordinator, for shepherding the mapping process for the team.
2. The local coordinator should work with policy team members to craft an invitation to work group participants who will attend the mapping sessions. (An invitation template is provided in Appendix 1.) The invitation should outline the purpose of the mapping work group sessions, the date and two-hour work group session(s) they are asked to attend, and any additional information they should bring with them to the mapping work group session (i.e., information about time frames between decision points, easily accessible data about the volume of cases at decision points, maps they know about or that already exist and that can be folded into the mapping process). Also, consider making resource materials (see Chapter 13 of Getting it Right) available to work group participants in advance of the session so that they can be familiar with the process and come to the session prepared to map.
3. The local coordinator may sit down with the team chair and/or the policy team to review the final list of persons who are planning to attend each session in order to ensure a comprehensive group of participants.
4. Prior to the mapping sessions, it should be determined who will take notes during the session. It is recommended that at least one staff person assist the facilitator in capturing the discussion during the sessions. The team should also determine who will be responsible (facilitator, local coordinator, and/or other policy team members or staff) for putting all the “pieces” of the map together in one diagram in an easy-to-view format.
5. Select a room with a considerable amount of wall space, where flip chart paper can be posted from end to end. Rooms with walls that do not permit (or are not conducive to) taping lots of paper to the wall should be avoided. The facilitator will need to be provided with a full pad of flip chart paper, a good set of flip chart markers, and a roll of masking tape.
6. While not a necessity, it is recommended that jurisdictions bring in an outside facilitator to facilitate each mapping work group session. The session will begin with introductions of all participants and a brief overview by the facilitator of the purpose of the process. You may wish to provide a handout on the mapping process. (A sample handout is provided in Appendix 2.) Following this, the facilitator will guide the work group through the development of their component of the system map. (It is possible, albeit rare, that time will run out before the work is completed. In this case, the facilitator will dialogue with the work group about the best way to complete the map.)
7. During or following each work group session, the facilitator should receive a complete list of the individuals participating in each session, including name, title, agency, and full contact information, for the purposes of seeking further information, receiving feedback on the accuracy of the map, and documenting those who contributed to its development.
8. At the conclusion of the session, the facilitator should make a plan with the work group to review the final, transcribed portion of the work group’s portion of the map.
9. Because of the complexity of the entire system and the desire to capture a “picture” of the system on paper, it is not possible to include on a flow chart every detail about a particular decision in the criminal justice system. For this reason, the person(s) transcribing the map and notes may wish to use footnotes that provide additional narrative. (See the example provided for an idea of how one jurisdiction captured its system map—and accompanying map notations—on paper.)
10. Once the system map components have been reviewed by each work group, a final draft of the entire map should be presented to and reviewed by the policy team. For this meeting, a large share of the full team meeting time should be devoted to reviewing and discussing the map—first to ensure its completeness and accuracy, and then to discuss its implications.
11. Once the mapping activity is fully completed, the team should prepare for the next, more in-depth step of the decision point analysis process.^[1]

Tips

Mapping the Entire System In Yamhill County, Oregon, the policy team created three separate maps in order to ensure that the justice system processes were adequately captured for the mentally ill, chemically dependent, and general offender population. To link the multiple maps, a process step was included in the main map that referenced a different track. The main and “mental health track” maps are provided.

• For each work group session, carefully select a diverse group of individuals who have the most knowledge and perspective about both the formal policy/decision making and informal decision making that occurs at each decision point.
• The number of individuals invited to attend each work group session should be considered—large enough to ensure representation, knowledge, and diversity, but not overly large as to be unworkable (ideally, about 10–15 people in each work group session).
• Some individuals may have critical knowledge about more than one decision point and should be asked to attend more than one work group session, as appropriate. Likewise, some decision points may overlap more than others (for example, arrest and pretrial status or institutional and community interventions). In this case, consider whether it makes sense to combine decision point work groups into one group.
• Consider sequencing work group sessions from the earliest decision point (arrest) to the latest decision point (discharge from supervision). For example, those invited to attend the work group session on the arrest decision point might be scheduled from 9:00–11:00 a.m.; pretrial status from 11:00 a.m.–1:00 p.m., and so forth.
• Leave completed sections of the map hanging on the wall as each “new” work group convenes; visually, this will provide an overview of how each decision point “links” to the last one.
• Assure that adequate time is devoted to each decision point; it may not be possible to complete an initial map of all of the decision points in one day.
• Consider color coding the map or using sticky notes to flag questions that arise for further exploration, to identify specific processes for special populations (for example, domestic violence offenders) that may depart from the typical decision process, to identify gaps in current decision making, to note strategies that are being considered or implemented but that are not currently part of the decision making process, or to note other pertinent issues identified by the work group.
• Consider conducting a “debriefing” session at the conclusion of the mapping work group sessions with the facilitator, local coordinator, and policy team members assigned to shepherding the mapping process. The debriefing session will provide an opportunity to discuss how well they think the mapping work group sessions captured information about each decision point, confirm a work plan for completing the map, identify an upcoming policy team meeting to discuss the map in more detail, and discuss any other issues or logistics that need to be resolved before moving forward.

 

---

[1] See 3b: Conducting a Policy and Practice Analysis.

Introduction

The development of a “system map” is a first step in developing a common understanding of how the criminal justice process “works” in a jurisdiction. When completed, the system map reflects the flow of cases through the justice process, key decision points, decision makers, and the volume of cases flowing through the system during a particular period of time. And while it can surface system strengths and challenges, it is the first step to—rather than a substitute for —a more in-depth examination of the policies and practices that underlie the workings of the system.

Following the completion of the system map, a process to “dig deeper” into these decision points should be undertaken. This more in-depth analysis includes an examination of the following as they relate to each decision point:

• written policies;
• the application of those policies to practice, as well as other operational practices that are not formally articulated in policy;
• the various types of data and information collected at each decision point;
• the ways in which this data and information inform decisions;
• the ways in which information is stored and shared;
• the extent to which evidence-based information and research is available and used to make decisions;
• gaps and barriers that impede the use of evidence-based knowledge to inform these decisions; and
• other factors related to using data, information, and evidence in the most efficacious ways.

Purpose

Most matters of public policy are guided in part by an array of formal policies contained in federal or state law, county code, case law, individual agency policy, or memoranda of understanding among multiple organizations. An assessment of current policy begins with the identification of these operating guidelines. Formal policy only begins to define the manner in which decisions are made and processes are carried out; in most communities, written policy guides only a small portion of activity. When formal policy leaves activities undefined, informal practices emerge to fill the gaps. Informal practices are sometimes developed with clear purpose and great care; in other instances, they simply evolve over time. Developing an understanding of informal practice is as critical as understanding current policy.

A policy and practice analysis serves to objectively assess the extent to which

• clear, written policies dictate key decisions and practices;
• policies are consistent with evidence-based knowledge;
• policies are consistently carried out, consistent with one another, and support their stated purpose(s), agency missions, and jurisdiction vision;
• gaps in policy result in discretionary practices, and the degree to which that discretion results in practice that is evidence-based, consistent, and in support of vision and mission; and
• the collection, storage, sharing, and application of data and information support effective decisions.

Participants

A policy and practice analysis can be conducted by staff internal to the agency(ies) under examination, by staff from colleague agencies within the jurisdiction, and/or by external assessors. The latter approach offers the opportunity to bring to the jurisdiction the perspective of objective subject matter experts. While this might be the ideal approach, it may not always be possible.^[1]

Instructions

Using Outside Experts to Conduct an Analysis

1. Discuss the purposes of the policy and practice analysis. Make sure the team understands that the analysis will be conducted by individuals with expertise in specific areas who will assess the extent to which current policies and practices regarding key decision points are supported by research. Against that backdrop, engage the team in a discussion about their priorities for the analysis (i.e., What would you like to understand about these decision points? What would you like to have more information about? What (if any) change targets can you envision regarding the decision points? What information can be derived from the analysis to better inform the team about those potential targets?).
2. Discuss with the policy team the process and timing of the analysis: who will conduct it, the expertise of the individuals involved, the anticipated duration, schedule of interviews and observations, and the method of recording and reporting the findings.
3. Agree on an approach to informing affected agency staff of the purposes and timing of the assessment.
4. Ask the outside experts to provide a report following the assessment that outlines key findings and specific recommendations for changes that the site can make to further align itself with the most current research. Ideally, a debrief session with the policy team will be conducted to enable the team to ask questions, deliberate over findings with the aide of subject matter experts, and develop strategies for action.

Process for Conducting a Policy and Practice Analysis through Self-Assessment

1. Form subcommittees (or work groups) to conduct assessments of discreet areas (e.g., agencies, decision points, a combination of these).
2. Across all subcommittees, develop a shared sense of purpose for this process—what information is to be collected and why, an agreed-upon set of “ground rules” for carrying out the process (e.g., ethical and professional behavior during and following the analytic process, confidentiality limits and expectations, etc.), the timeline, the final product subcommittees are to produce, and how and with whom these final products will be shared.
3. Identify a “chair” for each subcommittee. This person should be responsible for calling together their subcommittee, facilitating their meetings, keeping the subcommittee on task, and sharing information with the policy team to ensure good communication, coordination, and information flow.
4. Subcommittees should become familiar with the relevant research in their area(s) as well as emerging practices that have yet to be demonstrated empirically (if they do not already have familiarity in these areas). This knowledge will be the “backdrop” against which the analysis will be conducted.
5. Be clear with subcommittees that part of their role is to educate the policy team members on their topic. The goal, in the end, is to have a common, core base of knowledge across all policy team members.
6. Create mixed-discipline subcommittees. For example, do not assign only those team members who are involved in law enforcement to the cite/detain decision point. Doing otherwise will provide a unique opportunity for cross-disciplinary learning and for an “outsider” to ask questions that those entrenched in current processes may not consider.
7. Early subcommittee meetings should be focused on developing a detailed work plan and timeline for conducting the analysis.
8. Subcommittees should continue to meet through the information collection and analysis process to ensure that the work is being carried out as planned, data is recorded in a useful manner, and difficulties with the information collection and analysis process are addressed early.
9. At the conclusion of the subcommittee’s work, it should be prepared to present its findings and recommendations to the entire team.^[2]

Tips

Establishing a "Data Committee" Many EBDM policy teams elect to form a “data committee” in addition to work groups tasked with examining decision points. These committees provide information and data support to the work groups throughout the mapping and policy and practice analysis processes. Although in some instances existing data systems may be limited in the amount and type of data that can be collected, the data committees seek to collect, at least, the following baseline information: number of cases following each decision path on the system map; average case processing time between steps; average length of time/stay in pretrial and post-conviction disposition options; characteristics/profiles of cases in disposition options (offense and offender characteristics, including available information on risk and needs); success/completion rates for disposition options; and failures to appear (FTAs), new arrests, and recidivism outcomes by disposition option. For more information, see 3d Gathering Baseline Data.

Consider the following tips for collecting information effectively:

• The quality of the information collected through this process will have tremendous bearing on the action planning decisions of the team. Seek quality information over quantity if time constraints are a factor. To that end, if it is not possible to thoroughly assess all decision points, agree with the policy team on the initial priorities for assessment.
• The information gathering process should be used as an opportunity to educate those outside of the formal team about the vision, mission, and goals of the policy team, and to learn their perspectives on the strengths and gaps in the jurisdiction relative to evidence-based decision making. Doing so is likely to both elicit cooperation and engender long-term support for the team’s work.
• If a team chooses to work together on all of the component sections rather than working in subcommittees, a work plan and timeline for each section should be developed. Dividing the work among individual group members and encouraging them to answer the questions independently—without at least processing them with the team—is not recommended. Doing so will provide only one person’s perspective, and is unlikely to give an accurate or comprehensive picture of policy and practice across the jurisdiction.
• Do not rely solely on the expertise of policy team members and/or subcommittee members to conduct the analysis. Instead, collect multiple data sources to ascertain current policies and practices. Ideally, interview decision makers regarding current policies and practices; review documents (i.e., written policies, procedure manuals, programmatic materials, forms, and data screens); conduct individual interviews or focus groups with supervisors and staff; and observe the activities being assessed. This approach will provide the broadest view and most reliable information. (For instance, in assessing how pre-trial release recommendations are made, review written statutes, bail schedules, and agency policies; interview the pre-trial administrator, chief prosecutor, and bail commissioner; interview pre-trial release officers; and observe pre-trial interviews, verifications, the development of reports, and bail review hearings).
• Expect variability in practice. It is not unusual to find that there is a lack of uniformity within a jurisdiction, with some practitioners or agencies doing things one way and others doing it completely differently.
• In those cases where numerous individuals are involved in carrying out a particular function, committees might consider convening a focus group to understand the variety of approaches in use. For example, it might be helpful to identify those prosecutors handling violations of parole/probation cases and to interview them together in a focus group format to understand their management of these cases.

Example:
EBDM Work Group Charter for Assessing Evidence-Based Policy, Practice, and Decision Making

What are we trying to achieve through the work group process?

The role of the work group is to educate and advise the EBDM policy team on strategies that will result in the greater use of evidence (research) to support decision making consistent with the team’s vision for the justice system.

What do we mean by “evidence”?

In the justice system, the term “evidence” is used in a variety of ways. ”Evidence” can refer to items collected at a crime scene, eyewitness accounts, or security camera footage. These types of evidence are referred to as legal evidence. For the purposes of the EBDM initiative, the term “evidence” is used to describe findings from empirically sound social science research. The Framework refers to the results of this research as evidence-based policy and practice.

What is “evidence-based decision making”?

Evidence-based decision making is the use of evidence (as defined above) to inform decisions throughout the criminal justice system process at the case level (e.g., applying practices in light of offenders’ risk level), at the agency level (e.g., providing agency direction and support that results in probation officers spending 20 minutes or more in their offender contacts, and focusing those contacts on risk reduction techniques), and at the system level (e.g., working collaboratively to collect and analyze data to determine if systemwide policy decisions are resulting in the desired outcomes, and making adjustments as needed).

Who should be on the work group?

The work group should be composed of individuals who represent all agencies that affect or are affected by the decision points the group is tasked with addressing. The selected individuals should represent diverse experiences and points of view in order to ensure the broadest perspective on the topics discussed. The work group serves as a standing committee that seeks input from others (“ad hoc members”) as needed.

What is the role of the chairs or co-chairs?

Chairs or co-chairs are tasked with leading and organizing the work group. To that end, it is their responsibility to make certain that they and their group members are clear about their tasks and the work they are undertaking and, based upon this understanding, develop a work process that will be successful in accomplishing specific tasks within the designated time frame. It will be up to each set of work group chairs to determine how and when the work group will meet. The work group chairs will also be responsible for reporting progress to, and sharing their products with, the EBDM policy team.

What is the work group’s time frame?

Work groups should be prepared to complete their work by April 2011. If completion is possible sooner, this is desirable, but not if it means sacrificing quality of work.

What tasks are the chairs/co-chairs and their work groups expected to accomplish?

1. Identify the agencies that/individuals who should be on the work group and secure their agreement to participate.
2. Ensure that all work group members have read the EBDM Framework and EBDM project overview documents, and are familiar with relevant portions of the Mesa County system maps.
3. Ensure that all work group members are aware of the EBDM policy team’s vision statement and have read the team’s charter.
4. Ensure that all work group members are clear about the work group’s tasks and purposes.
5. Create and manage a work plan (e.g., recruit members, establish a schedule of meetings that works for members for the next 2–3 months, clarify the tasks and work to be undertaken).
6. Review and ensure that the system map and system map notations are complete and accurate.
7. Identify all the decision points and decision options on the system map that are within the domain of the work groups. Ensure that relevant decision points are reflected by a diamond shape.
8. Educate yourselves! Identify and review the evidence-based practice literature that most applies to the decision points for which your work group is responsible.
9. Discuss the extent to which evidence-based information is available and used to inform each of these key decisions. For each decision point, ask yourself the following:
   1. What risk and harm reduction measures are we seeking to achieve (or contribute to in major ways) at this decision point?
   2. What guidance does the existing research literature have to offer at this decision point?
   3. What can be done to ensure that research informs decisions made at this point?
10. Identify gaps and barriers that impede the effective use of evidence-based knowledge to inform these decisions (e.g., offender risk/need information is collected but not used to inform a particular decision; risk/need information is not collected; or research suggesting that individuals identified as low risk through the use of an actuarial risk tool is not used to inform the intensity of the applied criminal justice intervention).
11. Identify further information needs that relate to #8 and that would inform the work group’s discussions (e.g., “We know that community corrections has risk/need information, but we need further information about how this information is used (i.e., consistently? accurately? appropriately?) and the extent to which it supports the outcomes we desire” or “We know that we have a variety of intervention options, but we do not know the criminogenic needs (risk factors) these interventions address or the risk levels of the individuals who are receiving these services”). Ask your work group’s data committee member to assist with getting answers to these questions.
12. Identify how best the information in #9 would be collected. Can the work group do this, or someone else within the Mesa County justice system or county? Is support needed from an outside expert?
13. Identify change targets and opportunities that will support the EBDM policy team’s efforts to achieve its vision.
14. Keep the executive committee informed of your progress and of any issues/problems that need the input and advice of the policy team.

 

---

[1] Jurisdictions might explore whether technical assistance is available for this purpose.

[2] See 3e: Prioritizing Your Team’s Targets for Change.

Introduction

In addition to developing a system map and gathering information about the defendants/offenders in your system, conducting an assessment of the various services and resources available—within the criminal justice system, as well as from public sector entities in your community—will better inform your team’s policy and practice decisions.

Purpose

This Starter Kit is designed to provide instructions on how a jurisdiction might inventory and assess the resources available to defendants/offenders in their community and determine the strengths and best use of the current array of services, as well as gaps and overlaps in services.

Participants

Participants should include staff from the multiple criminal justice agencies represented on the policy team who are most knowledgeable about the resources, programs, and interventions available to offenders/defendants.

Instructions

1. Conduct a brainstorming session to develop a list of all the resources in your jurisdiction currently available for defendants/offenders. Some of these resources will be within the criminal justice system (e.g., a life skills class offered by the probation department), some will be in other public sector agencies (e.g., a job training program in the county’s office of workforce development), and some will be in the private sector (e.g., specialized substance abuse or mental health treatment).
2. Create a matrix to collect specific information about each resource and research each to gather pertinent information. At a minimum, identify the specific services available through each resource, including their admission and selection criteria, duration/length, capacity, the criminogenic needs that the program addresses, and a rating (such as one from the CPC, CPAI, or a similar entity). You may also provide a brief description of the service. See Appendix 1 for a template of such a matrix.
3. Compile your findings into a report or chart after the resource inventory is complete. Review the findings with your team to assess the resource inventory for completeness and to identify the strengths in your current array of services, as well as overlaps and gaps. Use this as an opportunity to consider ways to improve upon this array. For instance, should resources be shifted from one service area where there is an excess in service (e.g., where resources are devoted to non-criminogenic needs) to other areas where there are gaps? Appendix 2 provides a list of criminogenic needs and considerations around dosage, intensity, and duration of programming by risk level for adult offenders.
4. Consider developing a directory of your resources so you can share this information with pertinent stakeholders in your jurisdiction (e.g., probation officers, prosecutors, defenders, judges).

Example:
Yamhill County, Oregon, Correctional Programming Options

(Abbreviated version of full document)

Community-Based Programs
Program Name	Moral Reconation Therapy (MRT)
Brief Description	Evidence-based program that encourages clients to develop prosocial thoughts and behaviors Based on Curriculum-Based Motivation Group by Anne Fields of Portland, OR (2004)
Admission & Selection Criteria	Offenders sentenced to 6 months or greater High and/or medium risk on the LSI-R Potential candidates are identified through monthly review of the sentenced populations’ risk factors Only offered to male clients Female clients complete Phase II AMCP in gender-specific group, as MRT not currently offered to female clients
Program Length, Cycle & Dosage	Groups typically meet once per week for 13 weeks MRT consists of 13 chapters and homework assignments Clients may not miss more than two MRT sessions; otherwise, they will be directed to start over with AMCP Orientation and Phase I Length of program varies depending on client participation, motivation, and progress Minimum total dosage = 19.5 hours class time; estimated at 13 groups (13 assignments) of 1.5 hours each
Program Capacity	Class size is 12 persons Only one group offered at a time Open group Members start and finish individually
Criminogenic Needs Addressed	Primary = Antisocial personality/Attitudes Secondary = Alcohol/Drug
CPC (or other) Rating	This program has not been evaluated

 

Community-Based Programs
Program Name	Ready to Work (RTW)
Brief Description	RTW is an ongoing job development program created to assist probation clients in developing the tools necessary to be successful in finding employment.   RTW includes a comprehensive intake interview, career identification, and goal setting; 40 hours of unpaid practical work experience; career development workshops; and educational opportunities.
Admission & Selection Criteria	Sentenced, unemployed offenders High and/or medium risk on the LSI-R Potential candidates are identified through monthly review of the sentenced populations’ risk factors and/or court/PO referral as sanction/intervention
Program Length, Cycle & Dosage	Four-tier system: - Tier I (30 days): daily reporting - Tier II (11 classes): work groups through job source - Tier III (until employment gained): independent (report once weekly) - Tier IV (90 days): job success In each tier, clients participate in the program five days per week, eight hours per day, unless otherwise directed.
Program Capacity	20persons maximum for Tier I, 10 persons for Tier II, and 10 persons for Tier III Daily reporting and/or 1–2 employment classes/appointments per week, depending on phase Total 40persons at any one time, in all three phases combined
Criminogenic Needs Addressed	Primary = Education/Employment Secondary = Antisocial patterns
CPC (or other) Rating	This program has not been evaluated

 

Additional Resources/Readings

CSOM. (2007). Section three: An overview of the steps in the planning and implementation process. Enhancing the management of adult and juvenile sex offenders: A handbook for policymakers and practitioners (pp. 29–56). Retrieved from https://cepp.com/wp-content/uploads/2020/01/6-CSOM-Handbook.pdf

McGarry, P., & Ney, B. (2006). Getting it right: Collaborative problem solving for criminal justice. (NIC Accession No. 019834). Retrieved from https://nicic.gov/getting-it-right-collaborative-problem-solving-criminal-justice

Appendix 1:
Resource Inventory Matrix Template

County Name: Correctional Programming Options

Jail-Based Programs
Program Name	Brief Description	Admission & Selection Criteria	Program Length, Cycle, & Dosage	Program Capacity	Criminogenic Needs Addressed	CPC (or other) Rating

Appendix 2:
Risk and Criminogenic Need Considerations

Dosage, Intensity, and Duration by Risk Level for Adult Offenders^[1]

What Are the Criminogenic Needs and their Implications for Intervention?

While the literature has slightly different ways of expressing criminogenic needs, generally they fall into the eight areas noted below.[2]

Top 4 Criminogenic Needs

Criminogenic Need

Response

History of antisocial behavior

Build non-criminal alternative behavior in risky situations

Antisocial personality pattern

Build problem solving, self-management, anger management, and coping skills

Antisocial attitudes, cognition

Reduce antisocial thinking; recognize risky thinking and feelings; adopt alternative identity/thinking patterns

Antisocial associates, peers

Reduce association with antisocial others; enhance contact with prosocial others

 

Next Four Criminogenic Needs

Criminogenic Need

Response

Family and/or marital stressors

Reduce conflict; build positive relationships and communication

Lack of employment stability, achievement; lack of educational achievement

Increase vocational skills; seek employment stability; increase educational achievement

Lack of prosocial leisure activities

Increase involvement in and level of satisfaction with prosocial activities

Substance abuse

Reduce use; reduce the supports for substance-abusing lifestyle; increase alternative coping strategies and leisure activities

 

References:

Andrews, D. A. (2007). Principles of effective correctional programs. In L. L. Motiuk & R. C. Serin (Eds.), Compendium 2000 on effective correctional programming. Retrieved from http://www.csc-scc.gc.ca/text/rsrch/compendium/2000/chap_2-eng.shtml

Andrews, D. A., Bonta, J., & Wormith, J. S. (2006). The recent past and near future of risk and/or need assessment. Crime & Delinquency, 52(1): 7–27.

Bourgon, G., & Armstrong, B. (2005). Transferring the principles of effective treatment into a "real world" prison setting. Criminal Justice and Behavior, 32: 3–25.

---

[1] Bourgon & Armstrong, 2005; for more information see the Coaching Packet on Effective Case Management, available at https://www.yumpu.com/en/document/view/35063381/coaching-packet-effective-case-management-the-center-for-/32

[2] Andrews, 2007; Andrews, Bonta, & Wormith, 2006, p. 11; for more information see the Coaching Packet on Implementing Evidence-Based Practices, available at https://www.yumpu.com/en/document/view/35063381/coaching-packet-effective-case-management-the-center-for-/32

Introduction

The first step in any change initiative is deciding what you intend to change. As part of the EBDM initiative and designing the system-level logic model, policy teams should have already made some decisions about goals and objectives (i.e., what long-term changes they hope to achieve) and about the types of procedural and policy changes that need to occur in order to attain the goals and objectives. The next to step is to understand more about these changes by gathering baseline data. Baseline data for the EBDM initiative can be defined in three broad categories:

• case processing data;
• data about the offender population; and
• harm reduction data.

Case processing data and data about the offender population provide critical information about the primary issues addressed with the EBDM initiative. This data is particularly useful in shaping the initiative and refining the approach laid out during the initial planning. Gathering data about the intended harm reduction outcomes lays the foundation for evaluating the effectiveness of the EBDM initiative, allowing for an assessment of how much change has occurred since implementation.

Purpose

This Starter Kit—which should be used following system mapping activities and the development of a system-level logic model—is designed to provide an overview of the types of data that a jurisdiction might want to collect to establish a baseline for the EBDM initiative. The focus is on case processing and offender population data.^[1] Guidance on data collection strategies and analysis is provided, along with suggested next steps to prepare for the collection of harm reduction data. Using the information in this Starter Kit will help jurisdictions produce reports on current case processing policies and practices, what is known about the offender population, and what is known about the harms in your jurisdiction that are intended to be addressed through the EBDM initiative.

Participants

Partnering with Universities The policy team in Grant County, Indiana, benefits from a longstanding relationship with a local professor at Indiana Wesleyan who conducts an ongoing evaluation of the drug and reentry courts.

Participants in the collection of baseline data should include representatives of the policy team to help articulate the actual data elements to be collected, as well as personnel in the various agencies/organizations from which the data will be drawn. Ideally, agency personnel would include those staff members responsible for designing or producing reports from data management systems. To the extent that new data will need to be collected, jurisdictions may wish to work with outside experts to help develop data collection instruments, codebooks, and data management systems. Local universities are a good resource for assistance with data collection, as is staff in county or agency IT departments.

Instructions

The idea of gathering data may seem straightforward at first blush: identify the data reports that you want about case processing or about the offender population and analyze them. This type of approach, however, is likely to prove unmanageable at best and meaningless at worst. Too much data will overwhelm the process; people simply cannot interpret a lot of data with no analytic frame of reference. Too little data will not yield reliable information about trends and patterns and is likely to raise more questions than it answers. These instructions are intended to help policy teams frame the data-gathering activities in terms of what to collect and how.

The first step in gathering data is to decide what you want to look at and what questions need to be answered. Once this has been done, it is important to decide on a methodology for data collection and then to collect and analyze the data.

Deciding What Data to Gather

As part of the system mapping and development of the logic model, you have undoubtedly highlighted areas in which you would like to have more information. These areas form the basis for the questions that you are hoping to begin answering with your baseline data. For example, why are there so many dismissals, why does it take so many months to adjudicate a case, why are so many defendants held pre-trial, who makes up the pretrial jail population, how many people in the jail have mental health challenges, etc.

In deciding what data to gather, it is important to define the exact data elements that you want to collect:

• Be sure to define key terms. Some examples include the following:
  • What constitutes a case? Is a case counted by charge or by defendant? Is it counted consistently across the system?
  • How will recidivism be defined? Is it an arrest for a new offense, or can it include sanctions for technical violations of probation?
• Specify the data parameters, such as the time period from which the data will be gathered:
  • Will you gather a year’s worth of data, or several months’?
  • Will you collect data related to offenders in the system on a given day, and if so what day?
  • Are you interested in new arrests of offenders 90 days, 6 months, 1 year, 3 years, etc. after they complete their sentence?

Potential Definition of a Case An example of one way to define “case” for the purpose of data collection is by incident. In this instance, an individual person who is involved in the criminal justice is only counted once regardless of the number of criminal charges pending from a single arrest. A person involved in multiple incidents over time (e.g., three arrests in one year) would be counted as three “cases.”

Identify what the unit of analysis will be:

• Will you count people, charges, cases, beds, offense types, dollars, etc.?

Case Processing Data: Information related to case processing can focus on a variety of factors related to the nature and type of cases in the system as well as the volume and flow of cases. Depending on your policy team’s goals and the capabilities of your data systems, you may want to collect case processing data throughout the “life” of a case, or as early as arrest, booking, and filing/charging. Types of information that might be sought include the following:

• number of cases by case type;
• number of pending cases;
• age of pending cases;
• number of cases at different stages in the case processing continuum;
• number of cases that proceed or “fall out” by decision point;
• number and type of dispositions by case type;
• number and type of release decisions by case type;
• average sentence length;
• number of probation revocations for technical violations and for new offenses;
• number of bench warrants issued;
• number of continuances; and
• length of time between initial appearance and disposition by case type.

A caution for collecting case processing data, however, is to ensure that a “case” is defined the same way by all agencies providing the information. For example, police may count arrests as “cases,” prosecutors may count charges as “cases,” and so on. To avoid confusion, the definition of a case should be consistent across agencies.

Offender Population Data: Because data gathering can be labor intensive, the policy team should keep the data collection focused on the types of offenders or activities that you are planning to include as part of the EBDM initiative. So, if the focus is largely on pre-trial release, your data collection should focus on which offenders get released and by what means, and on which offenders do not get released. Conversely, if the intent is to focus on medium and high risk offenders for risk needs assessment at sentencing, then your data collection should focus on medium and high risk offenders.

No matter where in the system your data gathering will focus or which types of offenders you might hone in on, some common types of data should be collected about the offender population:

Establishing a Data Committee Prior to engaging in the EBDM initiative, Ramsey County, Minnesota, had established a Data Committee, which was composed of data, research, and IT staff from key criminal justice agencies, including the courts, corrections, human services, District Attorney’s office, and others. Historically, the Data Committee would be “activated” to assist with various cross-agency projects, and it was activated again for the EBDM initiative. Throughout Phase II of EBDM, the Data Committee met regularly and assisted in defining the baseline data, key measures, and other data elements necessary to measure the County’s performance under EBDM. The committee reported regularly to the EBDM policy team on progress in the collection of data, assisted in the system mapping process, and gathered data to respond to policy team members’ questions. The Data Committee was instrumental in identifying and resolving potential roadblocks in data collection and developing common definitions for measures (which differed across individual agency information systems) across all agencies.

• demographic characteristics of offenders;
• criminal histories; and
• previous sanctions and sentence lengths.

Harm Reduction Data: The types of harm reduction data will depend on the specific harm reduction goals defined and desired as a result of EBDM. However, some general guidance includes

• incorporating data from other governmental systems, as appropriate, to include as examples
  • the number of people engaged in mental health services outside of the criminal justice system; and
  • emergency room admissions.
• conducting primary research on areas of interest, for example
  • victim satisfaction surveys;
  • analysis of cost-benefits; and
  • comparative analysis of justice spending versus non-justice spending.

Information Gathering and Analysis Methods

There are many different techniques and approaches to gathering information. Because the information being collected under the EBDM initiative will be largely quantitative (i.e., numeric), only those approaches that are appropriate for quantitative data are described. There are two major types of data collection methods:

1. primary data collection: development of surveys, questionnaires, and data collection forms to collect information that does not already exist in another form
2. secondary data collection: collection of information from pre-existing datasets and data sources, such as case management systems

For the most part, you will be relying on secondary data collection. However, if you decide that primary data collection is necessary, it is highly advisable to work with an outside expert—perhaps from a local university—to develop and test any data collection instruments to ensure that the information gathered is reliable and valid.

With secondary data collection, the method you select – and the amount of external assistance you may need – will largely depend on the data-generating capacity of your system and the areas on which you have decided you need more information. Certain types of data collection and analysis will be more appropriate for understanding baseline case processing information, whereas other methods may be more appropriate for understanding the offender population. Thus, being clear about what you want to gather information on before you determine how (and from where) you will gather it is critically important. With this in mind, there are several different approaches that may be used in your information gathering:^[2]

• Pipeline analysis, in which a specific cohort of arrestees is selected and data is collected on them through their passage into and out of the criminal justice system.^[3] The unit of analysis is the individual; the analysis focuses on counting people. A pipeline approach is used to track the number of people in the system at every decision point and attrition at the point at which people exit the system. So, for example, a pipeline analysis starts with the number of people arrested, how many received citations, how many were booked into jail, how many were released after booking, how many were charged with offenses, how many were not charged, how many were convicted, how many were sent to diversion or deferred prosecution programs, how many were sentenced to jail, how many received suspended sentences, how many were placed on probation, etc. One way to think about this pipeline analysis is to determine the quantities of some or all of the process steps and decision points on your system map for a given period of time.
• Time analysis, in which the unit of analysis is either the individual defendant/offender or the case. The focus of a time analysis is to understand the amount of time associated with different aspects of the system or how long a particular process takes. So for example, a time analysis might look at the number of days individuals are detained in jail pre-trial, elapsed time from charging to disposition, amount of time in diversion or under supervision, length of time between preliminary hearing and trial/plea, length of time between violation and court action on the violation, etc. Time analysis data can also be noted on the system map, reflecting the lapse of time between one step (or decision point) and another.
• Jail analysis, in which the focus is to develop a thorough understanding of persons who are booked into the jail, the length of time they are in jail, and their status during the stay (e.g., pretrial, sentenced, DOC transfer, probation violation, etc.).^[4]
• Comparative analysis, which seeks to understand the differences between offenders in your population. For example, if you are concerned about compliance with terms of probation, this type of analysis will allow you to determine if there are differences between offenders’ outcomes based on risk level, types of conditions, types of present offense, prior criminal history, etc. Other questions that can be answered include, but are not limited to, issues related to who is placed on pretrial detention, the types and lengths of sentences/conditions, case outcomes, etc.

Tips

• As you define what information you would like to gather, identify the source of the information, including specifics about which agency has the data, the name of the person who will generate the report, and the time frame for delivering the data to the policy team.
• Be specific about the data to be collected. Identify if you want summary data, data aggregated into totals, or some other format; how you want the information broken down (e.g., by type of offense, by justice system status, etc.); the time frame from which the information is being drawn; etc. The more details in the information-gathering instructions, the better.
• It is often useful to mock up an example of a report that shows how you want to see the information and to provide that information to whomever is gathering the data. Doing this will provide additional clarity about what the data should look like once it is generated or extracted from existing information systems.

Example:
Ramsey County, Minnesota, Baseline Data Collected on Warrants

 

Example:
Key Findings from a Jail Analysis in One County

One EBDM site enlisted the help of an outside expert to conduct an analysis of the jail population. The consultant looked at a sample that consisted of 10% of the inmates released from the jail during 2010 (N = 533). As a result of the analysis, a number of observations and recommendations for reducing the jail population were presented, including the following:

• 60% of all jail bed days were being used by those who had a probation violation associated with their booking. Officials should examine current policies and procedures for dealing with probation violation cases with the goal of trying to reduce, if possible, the time required to resolve them.
• Defendants released on signature bonds had an average length of stay of 7 days and consumed 87% of the jail bed days for the pretrial release population. Jail officials should determine how they can ensure that signature bond decisions will be made on the first day in order to have a significant impact on the overall jail population.
• While Native Americans released during the pretrial period represented 4% of all those released pre-trial, they consumed 20% of all jail bed days used by all inmates released pretrial. Officials should look into whether this population has higher Failure to Appear (FTA) rates; if so, they may benefit from implementing court date reminder procedures, which have been very effective in reducing failure-to-appear rates in other jurisdictions.

A key finding from this analysis led to a recommendation to improve the collection of baseline data in the jail. The county jail, like many across the country, overwrites the legal status at booking (e.g., pretrial, probation violation, sentenced) when the status changes. Therefore, important data is lost regarding the reason individuals are initially booked into jail. This finding led to a relatively easy fix: developing a field for “legal status at booking” and a separate field for “current legal status.”

Example: Charlottesville-Albemarle County, Virginia, Steps to Calculate Baseline Data on Costs

Additional Resources/Readings

Carter, M., & Morris, L. (2007). Enhancing the management of adult and juvenile sex offenders: A handbook for policymakers and practitioners. Retrieved from https://cepp.com/wp-content/uploads/2020/01/6-CSOM-Handbook.pdf.

McGarry, P., & Ney, B. (2006). Getting it right: Collaborative problem solving for criminal justice (NIC Accession No. 019834). Retrieved from https://nicic.gov/getting-it-right-collaborative-problem-solving-crimina...

Rossman, S. B., & Winterfield, L. (2009). Measuring the impact of reentry efforts. Retrieved from https://cepp.com/wp-content/uploads/2015/12/Measuring-the-Impact-of-Reen...

 

---

[1] Harm reduction data is a third important component of information-gathering, specifically as it relates to outcome evaluations that are conducted; as such, harm reduction data are not discussed in detail in this document.

[2] The different approaches are not mutually exclusive; depending on what you are trying to understand, you may find that using two or more approaches provides a significantly clearer picture of the issues than using a single approach.

[3] McGarry & Ney, 2006.

[4] See McGarry & Ney (2006) for more detailed information about the analysis of the jail population.

Introduction

Considering the information collected through various policy and practice analyses is no easy feat. It is more than likely that your analyses will surface a variety of possible areas of improvement. This may present the team with some tough choices regarding its highest priorities for action. This document outlines a process your team might use for culling through the information collected through your policy and practice analyses and selecting your priority change targets.

Purpose

Reporting to the Policy Team: Work Group Findings During their strategic action planning session during, Grant County, Indiana’s work groups were asked to report out on their top 3–5 action items/change strategies for improving risk reduction outcomes; the evidence-based research that supports the action/change and other supporting reasons for the proposal; a brief analysis of the pros and cons of each action item; action items/change strategies that were considered and then “taken off the table,” and why.

To agree, as a team, on the most significant opportunities to advance policy and practice to achieve the jurisdiction’s harm and risk reduction goals

Participants

All policy team members should be present and actively involved in considering work groups’/outside experts’ findings from the policy and practice assessments.

Instructions

1. Plan a day-long strategic planning session (or a series of meetings) to conduct this work.[1]
2. In advance of the meeting, ask policy team members to review the findings from work groups/outside experts regarding the extent to which policies, practices, and key decisions are supported/informed by research.
3. At the beginning of the meeting, take a moment to agree as a team how decisions will be made for selecting the change strategies on which the team will move forward.
   1. Decide on the extent of agreement needed to make a decision. Will decisions be made by majority vote or through consensus? What does consensus mean? It may be unrealistic to assume that every team member will completely agree on the top change targets to pursue. One strategy for making decisions is for team members to agree to make an honest effort to understand what is being proposed and to determine whether each member “can live with it and support it.”
   2. Set criteria to determine which change targets will be prioritized. Some factors to consider may include: What is the potential impact of this change on risk and harm reduction? Do we agree the research support is strong enough to consider this change? Do we think this strategy is realistic, given what we know about our current system’s challenges?
4. Once the team is clear on the way it will make decisions, work groups should report on their recommendations, on the level of research behind each recommendation, and on how the recommendations came about.
5. For each work group recommendation or change strategy, consider it against the selection criteria and discuss whether team members agree that the change strategy is a priority. This may be accomplished through group discussion and/or ranking, scoring, or voting methods.
6. If the team has already discussed its harm reduction goals, it will be necessary to consider these goals as part of the decision-making process for selecting change targets. (If the team has not yet determined its final harm reduction goals, it should do so as part of its efforts to establish performance measures and outcomes, and to develop a systemwide scorecard.^[2])
7. Once a set of change targets is formed, revisit them as a package and determine if anything is missing or if strategies need to be further developed. The team may decide that it wants the work groups that developed the recommendations to create more detailed descriptions of the scope of work and action steps to implement the proposed changes.^[3]

Example:
Grant County, Indiana, Strategic Planning Session Goals and Agenda

Goals

• The first goal is to propose, describe, and weigh the pros and cons of the top change strategies recommended by the various work groups.
• The second goal is to prioritize the change strategies according to those that will produce the greatest impact and, as a second consideration, those most feasible to pursue in the next 18–24 months. Other change targets may be critical to long-term improvement in outcomes but will fall into a second phase of implementation.

Agenda
8:00 a.m.	Welcome; agenda review
8:15 a.m.	Review and discuss ground rules regarding how the EBDM policy team will make decisions about change targets
8:30 a.m.	Grant County EBDM Vision Statement: The Critical Context for Choosing Change Targets
9:00 a.m.	Pretrial Work Group: Proposals for Top Change Targets This will be the first of a series of reports from the work groups regarding proposed change targets and goals. Each work group will describe each proposal, including the work group’s analysis of the pros and cons of the proposal identify, with the participation of the full EBDM team, on flip chart paper the agencies involved; each proposal’s likely and logical contribution to harm reduction; the estimated time required for implementation; the supporting research, where possible; and proposals that were considered and set aside, and the rationale for these decisions. The full team is encouraged to ask clarifying questions, contribute to the discussion of pros and cons, and offer other strategies they would like to see considered at each decision point.
10:00 a.m.	Break
10:15 a.m.	Pretrial Work Group (continued)
11:00 a.m.	Community Interventions Work Group: Proposals for Top Change Targets
12:00 p.m.	Lunch
12:45 p.m.	Violations Work Group: Proposals for Top Change Targets
1:45 p.m.	Change Targets on Other Decision Points All team members will have the opportunity to nominate change targets that fall outside the purview of the three work groups.
2:15 p.m.	Review of the “Big Picture” of Change Targets and Setting Priorities Prioritizing the change targets that will have the greatest impact on harm and risk reduction in Grant County Understanding the change targets that are both foundational steps and feasible for implementation in the next 18–24 months If consensus does not emerge from the day’s discussion, or possibly to support that consensus, members will use colored dots beside each change strategy listed on the flip chart paper: red to indicate the most impactful change strategies; and blue for those change strategies that would be most feasible within current resources.
3:00 p.m.	Action Planning for Next Steps The session will close with a clear list of priorities for the next two months’ work (i.e., development of action plan, scorecard, logic model, communications strategy).
3:30 pm	Adjourn

Example:
Milwaukee County, Wisconsin, EBDM Proposal Scoring Tool

Criteria	Scale	Weight of Overall Score
Integrality How closely does this proposal tie into the vision statement (stewardship/reducing recidivism/collaboration/harm reduction, etc.)?	1 = whether it’s a good idea or not, it’s just not a strong example of EBDM principles at work 10 = epitome of EBDM principles at work, and especially useful in addressing the issues the criminal justice system in Milwaukee County needs to tackle	14.285%
Predictability To what extent does existing research suggest the proposal will be successful?	1 = in essence, the proposal is a hypothesis that hasn’t been tested anywhere else we know of 10 = solid research shows this has been a winner in similar circumstances in other jurisdictions	14.285%
Novelty How innovative is the proposal?	1 = it may be somewhat embarrassing to have to explain why we aren’t doing this already 10 = someday someone will call this “the Milwaukee ________”	14.285%
Supportability To what extent do we have baseline data about the issue the proposal addresses, and to what extent do we have data collection systems in place that will help us track progress and success (or lack of it)?	1 = considerable effort will be needed to collect data about existing practices and the results of the project as we implement it 10 = current, easily accessible data about our practices already exists and data collection systems are already in place that we can use to track progress	14.285%
Impressiveness How big a hit will this be if it’s successful?	1 = barely worth the effort 10 = candidates for public office will jockey to take credit for this idea	14.285%
Scorability How measurable are the projected results? Can the results be evaluated in terms of our overall scorecard?	1 = the aspirations of the proposal are not quantified and the proposal makes no suggestion of how they might be 10 = the proposal contains a specific, quantified estimate of costs savings, reduction in recidivism, harm reduction, etc., and a firm methodology for conducting future measurements of actual performance against the estimate	14.285%
Feasibility Do we have the financial and infrastructure capacity to implement the proposal immediately or must additional resources be sought?	1 = it is unlikely that necessary budgetary resources can be obtained, or necessary infrastructure developed, or both 10 = no additional budgetary resources or infrastructure are needed to implement the proposal	14.285%

 

---

[1] Consider the use of an outside facilitator to ensure that the meeting is as productive as possible.

[2] See 6a: Measuring Your Performance and 6b: Developing a Systemwide Scorecard.

[3] This activity may naturally lead to work on system- and/or case-level logic models. See 5a: Building Logic Models.