Designing vignettes for comparative research on breach
This post, reflecting on the progress made by the ‘Decision-Making and Supervision’ group at our recent Malta meeting, comes from Niamh Maguire.
The Malta meeting was a very productive and successful one for the members of the decision-making working group. The aim of our meeting in Malta was to develop a number of vignettes that could be used to examine the issue of breach in comparative contexts. Once developed the plan is to pilot the vignettes in a number of jurisdictions to test how useful vignette methodology could be for exploring the issue of breach in offender supervision comparatively. We managed to leave Malta with an outline of two comparative vignettes that we plan to pilot over the coming months in a number of different jurisdictions.
In Malta we built on the work that we had completed last October in Bratislava where we explored in-depth the potential for using vignettes as one part of a comparative methodology to examine decision making in the context of offender supervision in multiple jurisdictions in Europe. Whilst the focus in Bratislava was on understanding the advantages and disadvantages of vignette methodology, in Malta we agreed that while there are obvious drawbacks to using vignettes (particularly the inability to observe what goes on in practice first-hand), they offered us enormous advantages perhaps the most important of which was the ability to use a common scenario to explore decision making in many different contexts. Also highlighted was that fact that all research methodologies and methods have drawbacks and the important thing is to understand what these are and to make sure that our chosen methodology is appropriate for our research aims.
In preparation for our Malta meeting we worked on identifying the types of research data that could be collected in different jurisdictions as this would influence our choice of research methodologies. For example, in some countries access to court reports might not be problematic whereas access to the judiciary might be impossible. Representatives from each jurisdiction filled in a matrix designed to capture the possibilities in terms of research data and methods. We began our meeting in Malta with an overview of this matrix by Dr. Trevor Calalafo and it emerged that the most popular methods were interviews, focus groups and vignettes and that while access to the judiciary was possible in many countries it certainly was not easy.
We chose the issue of breach to focus on as our review of the literature showed that there was a dearth of research in Europe on this important issue in all decision-making three phases (Boone and Herzog-Evans 2013:85-86). We then had to decide which point in the decision-making process we would focus our efforts on. The decision-making group had already identified three stages in the decision making process relevant to offender supervision; the pre-trial phase, the sentencing phase and the release phase. Strong interest in the group emerged for exploring the sentencing and release phases and a sub-group was established to develop vignettes for each of these phases.
Before we split into our sub-groups we discussed our aims and objectives and the general approach we’d take to constructing the vignettes. We had to remind ourselves that our focus was on testing vignettes as a means studying breach comparatively and so the vignettes should be developed to give us maximum insight into the differences between jurisdictions in terms of procedures, processes and approach of the various decision makers.
If we want to compare approaches to breach in different systems then we needed to design something that would make sense and be comparable across a range of jurisdictions with very different laws, procedures, decision-makers and cultural contexts. Central to the design of the vignettes was figuring out what a realistic scenario was for each jurisdiction. Making the vignettes make sense in each jurisdiction meant designing a model and then identifying the various aspects of the model that will differ depending on the jurisdiction.
Our next consideration was the degree of detail to include in the vignettes; if we over-specify we risk getting responses so diverse that it will be difficult to distinguish between differences that are related to the detailed factors we have included and those related to differences between jurisdictions. Over-populating the vignettes also risks making them workable in some jurisdictions but not others. The balance between the degree of detail necessary and allowing the vignettes to remain vague in places was recognized as being really important. Although it was agreed that we would use vignettes as a starting point in a semi-structured interview, as a way to situate a discussion about breach in relation to a specific set of circumstances, it also emerged that we should not forget to can pinpoint a definite decision. Most important though is supplementing a rather broad and necessarily vague description in the vignette with a very detailed topic list to make sure that we collect the same kind of data and thus really have the opportunity to compare our data at the end.
The idea of seeing breach as a process involving many stages and many actors rather than just a one-stage event also emerged. We wanted to gain insight into the process of breach which involved capturing the initial decision of the immediate supervisor to report the breach of conditions, the decision of the next official actor (which could be a probation officer or a member of a local authority) and then the decision of the ultimate decision-maker in relation to the official response to the breach (which could be a judge, a probation board or another body depending on the jurisdiction). This approach is based on the idea that we cannot fully understand decisions at the breach stage without exploring the stages that precede it. Following discussion on this issue, we realized that the number of stages would differ depending on the jurisdiction but that there were likely to be at least two if not three stages. As a consequence we needed to make not just one but three vignettes per case to represent these different stages. Importantly, this meant capturing the perspectives of immediate supervisors, probation officers and ultimate decision-makers (in many cases judges).
As this point we split up into our sub-groups and the challenging but highly enjoyable job of actually writing up the vignettes and choosing the breach conditions began.
To be continued…!
Boone, M. and Herzog-Evans, M., (2013) Decision-Making and Offender Supervision in F. McNeill and K. Beyens (ed.s) (2013) Offender Supervision in Europe. Basingstoke, Hampshire: Palgrave Macmillan