Comparative research on decision-making: Reflections on the Bratislava meeting (from France)

This post comes from our colleague Prof Martine Herzog Evans…

Whilst my friends and colleagues were brainstorming about comparative research methodology in Bratislava on October 25-26, I was trying to listen to their presentations and debates via a rather unreliable skype connection. However I got the gist of what was said. Unfortunately I did not get a chance to participate in the ensuing debates and for this reason, I would like to make a series of observations and suggestions in the present post.

First about methodology

On Saturday morning Kristel Beyens and Niamh Maguire discussed methodology in sentencing research. Vignettes were mentioned as being useful tools; however there was much debate about their limitation in particular in terms of realism and reality.

For this reason, the common conclusion was that vignettes could not be stand-alone tool.

Much as I agree with this I would also like to say that there are also ways and means to make ‘vignettes’ closer to the truth. Usually, vignettes are fictitious cases; they are typically of a maximum of one page long. Indeed a lot is missed: files that courts (and other decision-makers) rely on; hearing and what goes on during them… and so on. One way around this to a certain extent is to create fictitious files rather than cases.

As an example, I am currently conducting a joint research with two psychologists from my Uni (Patrick Mollaret and Régis Lefeubvre) which focuses on values and principles of juges de l’application des peines (JAP).

We had much debates together about this technique. Having worked in sentences’ implementation for decades by now, I know too well that a huge amount of parameters lead to JAP decisions. There was no way, I told my psychologist colleagues that vignettes would reflect closely enough what JAP were used to. However they were used to building their studies like that and I gave in on two conditions: a) that the so-called vignettes would be as I mentioned above fictitious files; b) that the interview we would have based on the vignettes would strictly follow their scaled-questionnaire, but would also allow for discussion with JAP so that we would understand why they chose one way or another

My colleagues designed the scaled-questionnaire and I designed the fictitious files. It comprised : an extract of a PO report; an extract of the prosecutor’s requisitions; in one case an extract of the risk assessment made by an ‘expert’ and in all breach cases, an extract of the ruling which had granted the decision which was breached.

Four cases were designed with specific parameters as guidelines from the psychologists and based on what I knew was their typical situations and contexts and also the usual language of PO reports, prosecutors’ requisitions and experts’ assessments. All interviewees have so far confirmed that these were absolutely typical to a T. For each one of them the offender had been sentenced to imprisonment and was referred to the JAP for transformation of his sentence into a CSM. Then each offender breached. JAP are asked about their dealing with both granting and breaching.

So far I have used other methods with JAP research: ruling analysis (see below) + observation + interviews + interviews of those they work with + interviews of probationers (the latter ongoing). I have to say that vignettes have taught me a thing or two about JAP’s perceptions and about some of their biases (in particular with addicts: their understanding of addiction does not seem to be as good as their rulings had allowed me to believe).

Conclusive note on this first comment: working with scholars from a different field is particularly enlightening when one is designing research methods.

I would also like to mention a research tool which is seldom mentioned in the literature: quantitative and qualitative analysis of judicial rulings. Judges also write rulings to explain why and how they made their decisions. In French law it is called ‘motivation’ (the word is telling) and it is in my experience a very powerful research tool and a wonderful way of getting closer to why the decisions are made.

As an example, in my first bigger scale JAP research (soon published by l’Harmattan, French reentry courts: Mister Jourdain of desistance), I analysed 1,300 rulings. I looked for a great deal of factors… long story.

Indeed one finds in these rulings a lot of legal babble and to an extent, a lot of things which they are expected to do and say (postures) – depending of course on the legal systems and the demands they put on courts’ ‘shoulder’. But, and this is where being a lawyer is immensely useful,in order to use this particular tool, one needs to know what and where to look for in a ruling. In particular one needs to be able to distinguish, one the one hand, what is legally mandatory (and how much leeway there is in this respect – eg I analysed the use of guideline articles as opposed to specific technical articles) and, on the other hand, what is a mere description of the facts as presented by other parties (eg PO, prosecutor, attorney). One must also distinguish what has been proved and established (for instance the probation service report has checked whether the person actually has employment) and what has just been said (for instance the probation service just states that the offender said he had a job). Lastly, the essential part of rulings, the one which is the most promising in terms of empirical research is where the court’s own point of view appears – as opposed to the description of facts, opinions, and legal rules which apply.

For instance, in French rulings there are two to three (sometimes one, four) paragraphs which represent the core analysis and thoughts of the court situated right before the court states what the decision is. The analysis of these one to four paragraphs is an exceptionally rich empirical source.

These paragraphs convey judges’ views, appreciations, values, principles, on offenders, measures, contexts, issues (eg addiction, domestic violence). It explains the process of their thinking and not just the output (the decision which is made).

Like with vignettes it is even more powerful when interviews ensue where judges are asked questions about rulings. As an example I asked them for whom they wrote their rulings: for prosecutors (in case they appealed), for courts of appeal (in case there was an appeal and their ruling would be scrutinized), for society in general (because they rule in the name of the French people/or because they would be blamed if they made a mistake), or for offenders. I got a great variety of responses…. (do read my book : LoL!) and several of them responded that they also wrote for themselves… in order to check out that their first view of the case would still stand when they would have to objectively justify it in writing (in some cases it did not).

In conclusion I cannot imagine conducting a research on any judicial decision (and in particular in sentencing) or related issues such as release, bifurcation or breach without analysing rulings.

About things to study

In our book chapters each one of the COST work group leaders mentioned gaps in the literature. In our decision-making chapter, Miranda Boone and I pointed to a long list of them.

In the following paragraphs, I would like to insist on several re sentencing (but this could also be said about release and breach decisions, especially whenever a court is involved) and in particular:

Cognitive processes

As I mentioned in the book chapter with Miranda, there is an important US literature on the cognitive processes of judicial DM (see for ex.: Klein D. and Mitchell G. (eds.) (2010), The Psychology of Judicial Decision Making, Oxford University Press). To the point where the American Judges’ Association made recommendations based on what is now known (P. Casey, Judge K. Burke, Judge S. Leben, Minding the Court. Enhancing the Decision-Making Process. A White paper of the American Judges Association. Making Better Judges, October 1, 2012). In particular, they recommended ways of going back to the reflective brain rather than relying as most experts in a given field do, on their reflexive brain. Interestingly the reflexive brain is what explains this ‘hunch’ which Cyrus Tata was referring to on Friday in Bratislava, The reflexive brain is less sugar greedy and less tiring and works well enough most of the time. However, it is also open to biases and errors (and judges are no different: C. Guthrie, J. J. Rachlingski, A. J. Wistrich, ‘Inside the Judicial Mind’, Cornell Law Review, 2001, n° 86(4). Hence the recommendations made by the AJA: to use checklists, to eat enough and have small breaks, to sleep enough before a hearing, and so on.

As Cyrus recommended hunches are also important and should not to be ignored for being either uninteresting or interviewee’s defensive strategies. They can be and are also study objects. I would thus strongly recommend we also included cognitive psychologists and even neuroscientists (as a good proportion of this literature comes from neurosciences) to any ‘hunch exploration’ endeavour.

Collegial versus one judge courts

In the aforementioned literature, there are some papers on the difference between collegial courts and one judge courts. In collegial courts (e.g. V.A. Hettinger, S. A. Lindquist & L. Martinek, Judging on a Collegial Court: Influences on Federal Appellate Decision-Making, University of Virginia Press, 2007) there is a group factor and there are other relevant ones: the respective status of the judges, who has the more experience or charisma, who studies the files better and son on… The necessity to render a common judgement has lots of consequences (Malzmann F. Spriggs J. F. OO & Wahlbeck P. J., Crafting Law on the Supreme Court: The Collegial game, Cambridge Univ. press, 2000; Baum, op. cit.) and how people get along as well as organisational factors play a great part in their decision-making (Cohen J. M., Inside Appellate Courts: The Impact of Court Organization on Judicial Decision Making in the United-States Courts of Appeals, Univ. of Michigan press, 2002).

I may be wrong but I am not aware of such studies conducted in Europe. It clearly is a major issue

Courts of appeal

IT’s linked because courts of appeal are typically collegial, but they have particularities which lower courts don’t possess: judges are often older and more experienced; they may also be more consensual and (boring?) as in some legal systems they will have needed not to make any waves and to be friends with the right people in order to become appellate judges. They may also be particularly concerned by their public image and the desire to make “good justice” (Baum L., Judges and Their Audiences, Princeton Univ. press, 2006 ) and may thus be very sensitive to the risk of reversal by higher (supreme) courts( J. Barnes Bowie, D. R. Songer, ‘Assessing the Applicability of Strategic Theory to Explain Decision Making on the Courts of Appeals’, Political Research Quaterly, 2009, n° 62(2): 393-407).

It is conversely extremely difficult to approach them and to do research with them.

I do not think that we know much about how European Courts of Appeal judge (what are their values and views) compared to lower judges.


We have little knowledge about what type of attorney influence judges. What judges expect. What is a good attorney? Whether they can and under what conditions, reverse a case’s outcome.

We would need to know more about factors like their status (newbie… ordinary… Bar star…), their eloquence, their legal knowledge, their knowledge of the case or of their client? Others?

It would also be important to know what their clients expect? Is it mainly outcome related or is it more about the fact that he makes sure they’re given a legitimate/fair trial? My own hunch based on the legitimacy of justice literature is that attorneys also play their own part in the perception which offenders have about how legitimate justice is.

Similarly in jurisdictions where release and breach decisions are made by courts and probationers have a right to counsel, we know little if anything about the role played by attorneys in assisting the desisting process and, more immediately about their influence on the courts.

In my aforementioned JAP research I also interviewed attorneys (N32) and found that there may be a fascinating dichotomy in this respect between classic penal attorneys and holistic attorneys, who develop a form of therapeutic alliance with their clients. I am currently launching a specific research on attorneys in sentences’ implementation to test this typology.


It struck me that we were discussing sentencing as if it the sentencing process was the same everywhere. However there are fundamental differences and to name a few:

          Whether the legal system is adversarial or inquisitorial…. The balance of powers and the role of the judge are vastly different;

          Whether the court decides on the day of the trial or whether it postpones its decision. In the first case the trial is their last memory of the case before they decide; in the second, they can withdraw from the emotions of the trial, forget about what was said and rule more neutrally, but less humanly based on the file. In fact in some cases the court can decide between the two and the reasons are also interesting to analyse;

          And of course whether there is a hearing at all or not. In many situations where people are sentenced nowadays, there is none at all…

Martine’s suggestion

I have an idea which may be putting the cart before the horse (in French the cart before the ox). Nonetheless, here it is…

On Friday, we heard Reuben Miller telling us that reentry programmes, which being an outsider I imagined were wonderful, are in fact maintaining people in a poverty-exclusion web. He told us they were not so much about helping people desist (: finding a job, real housing…), but much more about asking people to admit they were ‘fucked up’ (one of his interviewees’ quote) and accepting the help required to ‘un-fuck’ themselves up.

Parallel to this, there is more and more criticism against the Drug Court movement (e.g. E. Miller, ‘Drugs, Courts and the New penology, 2009, Sandford Law and Policy Review, 2009, n° 20(2): 101-145) – to be fair to a great extent fuelled by the desire of policy makers to withdraw funding, but some of what critics say must be listened to: in some cases (in particular in the US? ) it seems, again, to be more about asking people to take responsibility for their actions, for them to change… and not always about really helping, supporting, or ‘accompanying’ as French probation practitioners say (suivre) them on their desistance journey.

All this makes me all the more confident that one of the core element about what practitioners, institutions and agencies do is their values, principles, perceptions.

As Cyrus pointed out, we cannot just analyse things from the top; we must also get down to street level as Reuben called it.

A mix of both is nonetheless probably essential.

For instance it is important to see that in the framework decision 947, it is allegedly about reinsertion… but really it was all about saving dimes, pennies or bucks in the first place. Once that is understood, to what extent does it impact on practitioners who will be implementing it?

Another example: in France there are beautiful legal reinsertion and resocialisation principles and guidelines. But in fact in the field social work has virtually disappeared (at least from official probation) as PO say a)’ it’s not my job to help’; b) ‘so long as offenders don’t want to stop we can’t do anything’; c) ‘our job is to be an interface between the judiciary and the rest of the… world, and other agencies’.

And then there is what I said above about cognition: what feeds the hunch is a mix of experience and shortcuts in practitioners’ brains and of biases, values and principles. This is why we need to know what the inner principles (and not only the legal or even the institutional principles – and perhaps we need to compare them) are.

Another reason why I am interested in this is that we have also heard M. Phelps in Liverpool and read Fergus (‘Community Sanctions and European Penology, in Daems and al., European Penology, Hart Publishing, 2013, 171-191: ‘) that there is a risk for CSM to become the mass punishment which follows – and/ or is parallel to mass incarceration. There are thus serious concerns about human rights in CSM.

There is some literature on values and principles in probation.

So far it has essentially been addressed by the criminology literature:

Ward and Connolly, Morals, Rights and Practice in the Human Services, 2008, Jessica Kingsely Publishers

Canton and Eadie, ‘Accountability, Legitimacy, and Discretion: Applying Criminology in Professional Practice, in  B. Stout, J. Yates and B. Williams, Applied Criminology, 2008, Sage

M. Nellis and L. Gelsthorpe, ‘Human Rights and the Probation Value Debate’, in Chui W. H. and Nellis M. (eds.), Moving Probation Forward: evidence, arguments and practice, London, Longman Pearson, 2003

Or in the social work literature:

Eg: Sarah BANKS, Practical Social Work, 4th ed, 2012, Palgrave

With all due respect most would have needed a legal touch when they have touched upon… legal matters.


My point is we need to cross methodologies here and in particular: criminology, social sciences, psychology , cognitive psychology, neurosciences and law.

How do we do this and would you like to join me in the study of values, goals, principles, guidelines, and deontology?

Comments are closed.


EU Flag COST is supported by the EU Framework Programme Horizon 2020


The views expressed on this website are the sole responsibility of the author and do not necessarily reflect the views of COST.