Experiencing Supervision: Surveys and Images

Like the others who have reported below, the Experiencing Supervision working group also had a great meeting in Belfast, and we also want to pass on our thanks to Nicola Carr for being such an excellent host.

Our group spent the first day together, so that the two sub-groups could contribute to and comment on each others work. One group is working on the ‘Eurobarometer’ — seeking to develop a survey instrument about the experience of supervision that can be used in different jurisdictions — both to help explore those experiences within jurisdictions and to compare results across jurisdictions. The project is led by Ioan Durnescu at the University of Bucharest, though the leaders on this meeting were Jelena Djoric and Gabriel Oancea. Prior to the meeting colleagues in Croatia, Italy and Romania had piloted tested the first version of the instrument and so they were able to update us on the strengths and weaknesses of the tool. We spent some time together working through it, seeking to improve items, and struggling with some tough issues, especially whether to not to try to develop the instrument for use across all forms of supervision, or to limit it to one or two. For the moment, we are sticking with the more ambitious route (of trying to cover all forms of supervision), but are trying to organise the instrument in modules or pathways, so that it doesn’t ask irrelevant questions of respondents. On the second day, the subgroup met in order to develop plans to test version two of the survey in a larger number of countries — probably adding Switzerland, Austria, Norway, Denmark, Serbia, England and Wales and the Netherlands to the existing three.

Our second group is working on the ‘Supervisible’ project. This is a ‘photovoice’ research project — seeking to collect and analyse images taken by supervises to represent their experiences of supervision; but it is also about directly addressing supervision’s invisibility in cultural terms. We hope to be able to curate an exhibition of the images at our final conference in Brussels on 10-11 March 2016. The project is led by Wendy Fitzgibbon at London Metropolitan University. Wendy was pleased to report that we have secured a small award from the Howard League for Penal Reform (in England and Wales) to cover the costs of a pilot in the UK (with me and Marguerite Schinkel handling the Scottish part of that). Both in England and in Scotland, ethical approval has been secured, and some complex access issues (in England) are being addressed. We are just about ready to start work with people with current or previous experience of supervision — and also with an art therapist (in England) and an artist (in Scotland). Progress had been more difficult to make in Germany, but colleagues from there and in Austria, England and Wales and Malta were able to make plans to try to extend the pilot to their countries.

We also had a lot of fun — and learned a lot (!) — by testing the process on ourselves. Basically, on the first day, we gave members of the working group about 45 minutes to go out and take one or two photos that represented the experience of supervision as they understand it. The image above is one of the photos that resulted. On the second day, the subgroup analysed the images we had taken before presenting them to the whole Action at the end of the meeting. We’d love to share more of them (and our analysis), but we don’t want to risk influencing the sorts of images that others who become involved in the project may take. Indeed, we found the whole process so fascinating that we are considering setting up a page on this blog and initiating a process via Twitter where anyone can post a picture and a caption, so that we can perhaps compare how people with different perspectives on supervision (practitioners, supervisees, judges, members of the general public, etc.) seek to represent it. So watch this space!

Overall, we had an excellent meeting, and left full of energy and ideas for the next stage of our work.

Multiple Methods in Malta or: the joys and pains of comparative research

This time, our COST caravan pitched the tents on the beautiful and hospitable island of Malta – Grazzi to Sandra, Mariella and Trevor! The spring sun was tempting, but we resisted and stayed indoors to continue the work we started in Bratislava last autumn. There, the different working groups had begun to develop ideas on how to actually study offender supervision comparatively. Meanwhile the multitude of methods under scrutiny is impressive: Supervising staff keeping diaries; giving supervisors and supervisees a ‘photovoice’; doing observations as well as  using vignettes are discussed and we will see pilotes in due course (some have been done already!).

Part of the Working Group on “Experiencing Offender Supervision” had decided to go quantitative. For the Malta meeting Jelena Zeleskov and Ioan Durnescu had prepared a good paper highlighting once more the relevance of aspects such as autonomy, the quality of relationship or legitimacy and fairness for compliance, for offender supervision as such and particularly for the experience of those undergoing it. They also made the connection to the supervisee’s human rights as laid down in various instruments – if we, for example, stress the relevance of the offenders’ “cooperation regarding interventions that affect them” (as in The Council of Europe’s Probation Rules) we should now how they perceive these interventions. But why choosing a quantitative approach? Well, at first there is a COST Action-inherent reason: So far this approach is missing and we promised to explore a variety of methods. Second, results may easier be generalized to the overall supervised population and, more important, a survey design with closed question makes it easier to obtain results comparable between jurisdictions and to process them. Thirdly, quantitative data sometimes seem to be harder to ignore for stakeholders – at least they have additional value when it comes to policy impact.

However, from Ioans and Jelenas paper and also from my own experience with comparative prison research using surveys it is quite clear how ambitious that task is and how stony the path to go. We all felt it when we were discussing the next steps. We even agreed quickly that we want to restrict the scope of the survey and we relatively easily reached consensus on the four dimensions we want to explore – the ‘general experience’ of being supervised (in a phenomenological way); the relationship with the main supervisor; legitimacy or fairness and the punitive content of supervision. But we found it extremely difficult to formulate proper questions. At least I had the impression of a Babylonian confusion which led us back to very basic questions about the nature of supervision, because everybody is necessarily travelling with his or her discipline, methods and domestic understanding of things in his or her luggage (and we could have done with a native speaker).

In all groups and in every comparative research we experience this kind of cross-country running, bumping into obstacles and going in circles – but hey, at least these are hermeneutic circles! Indeed in the end we did delineate a set of questions. I know that other groups make an effort to also describe the process of encircling the question and find a possible solution (or several), so these experiences are valuable in itself. They should be documented, for example for the training school. At the end of our four years we then could have something like a Vademecum (‘Va-de-me-cum’, lat., literally “go with me”), something like a “hitchhiker’s guide to comparative research on offender supervision”.

It could also warn possible users that sometimes doing comparative research and running in those circles is a painful thing. When I came back from one of our COST meetings I met a senior colleague who asked me about my current work. I said something about comparative research which prompted the reaction “O, so you are sure that you know your own stuff well enough?!” Well, I am not always sure. And in addition discussing punishment or supervision with sociologists from Belgium, lawyers from Greece, criminologists from Scotland or psychologists from Serbia may shatter your own beliefs and assumptions and somehow destabilize your scientific self. So why do comparative research in the criminal justice field?

I resort to one of the Mahatmas in this field, Hans-Heinrich Jescheck, who was the first director of the Max-Planck-Institute of Foreign and International Criminal law in Freiburg, Germany. Already in the 1950s he outlined why he thought comparative criminal justice research was important. The Max-Planck-Institute has two sections, one being the Criminal Law Section, one being the Criminology Section. So even if these thoughts relate to legal comparison, he took a socio-legal position and was aware of the necessity to include criminological perspectives. I am referring here to a paper written in 1974 on the comparative perspective in legal reform (criminal procedure, to be precise). He distinguishes four reasons or motives, obviously intertwined, for undertaking comparative research. The first is the need for “pure basic research”, ‘pure’ being a weak translation for the German ‘zweckfrei’ – meaning without immediate use and application. In that way comparative research simply seeks to gain new insights or knowledge (“Erkenntnis”). A later director of the Institute, Albin Eser, used the image of going to a museum (probably we must think of an ethnological museum) – looking at particularly interesting foreign norms or practices as if they were exhibits that attract us. In any case, Jescheck again, we must study law as cultural phenomenon in other countries and in doing so we study social problems the jurisdictions try to solve by applying the law.
A second reason to do it – and now already purpose and utility come into play – is that comparative research supports international cooperation and (this is my favourite!) helps to reduce intellectual arrogance and alleged superiority. It “culminates in carving out the communalities of all laws that, eventually, agree on the idea of justice”. It is interesting that he only looks at the communalities and that he is so optimistic about the common ground or a common good intention. Thirdly, comparative research helps interpreting domestic legislation, which must not only be read having in mind the historical path but usually has already taken in foreign influences. These thoughts can easily be transferred to criminal policies and programmes. Finally, comparative research serves, and is necessary for, legal and cultural progress. According to Jescheck, comparative legal studies may provide a supply of possible solutions (“Lösungsvorrat”) for social problems, already in a handy format – systematically organized and critically evaluated. The question remains whether law reform is always a remedy for social problems, nevertheless this thought again can be transferred to policies and practices. And again he takes a very optimistic perspective (also on the receptiveness of the legislator). To sum up: there is a whole range of good reasons to go on.

O yes, and there are some additional joys of comparative research: Travelling to interesting places; getting to know different cultures; meeting old friends and making new ones; eating, drinking and sometimes dancing with them; being challenged by other surroundings, concepts, languages and group dynamics are only some of them. All in all this helps to keep an open mind, welcoming new ideas …

Creative and comparative ways to study experiencing supervision

The Bratislava meeting was an opportunity for the Experiencing Supervision Working Group to review what has been done so far and look for a creative way of doing comparative work in the coming years.

Based on our literature review we noted that most of the methodologies employed in assessing the experiences of those under supervision are mostly narrative ones: interviews, questionnaires and so on. Although these tools can produce valuable knowledge about how different people perceive and interpret their experience they cannot capture aspects like context, structure and so on. Therefore, our group started to think of some alternative ways to grasp a more comprehensive image of supervision from the recipients point of view.

One way to proceed was, as suggested by Reuben Miller, an ethnographic approach. In his presentation Reuben emphasized the value of ethnography in capturing the interaction between individuals and social/penal/private institutions.

Another approach was suggested by our colleague Wendy Fitzgibbon. In her presentation, Wendy made a very strong case for the idea that ‘a picture is worth a thousand words’ (Brisbane, 1911). By visualizing a social situation the researcher together with the offender can construct and re-construct the feelings, the thoughts, the decisions and so on associated with that particular picture. Numerous options were discussed like:

• using digital versus traditional photography 
• limited or unlimited numbers of picture
• visual diaries
• art or documentary photography etc.

The decision of the group was to explore these idea further and come up with more concrete thoughts of how we can use this method in the comparative research. Wendy together with Lol will try to do this before the next meeting in Malta.    

Linked to these ideas, another option was suggested – ways of collecting accounts of  ‘A day on supervision’. What if we ask people under supervision in different places around Europe to record and describe the experienc of the same day in the calender? What would this method produce? What day? Who exactly should we ask? We noticed that experience is influenced by a number of factors. Shall we look only at those who have an interaction with their supervisor that day? Shall we aim at collect data from both supervised and supervisor? These are only a few questions that were debated in Bratislava. Fergus agreed to liaise with the other working groups and analyze the possibility of producing astudy of ‘one day on supervision’ as common enterprise across the working groups, taking more perspectives into account.

Another way to enhance the comparability of experiences is to ask the same questions to supervisees from different jurisdictions. This was the starting point for the idea of developing an European ‘barometer’ on experiencing supervision. In my presentation I introduced the five dimensions that might be the skeleton of the future tool: supervision as a general experience, supervision as a helpful experience, supervision as a human service experience, supervision as punishment and supervision as a rehabilitative experience. In developing these dimensions into questions European probation rules and the what works literature could be incorporated.

Different issues were discussed in connection to the ethics and the use of such an instrument. What if the countries will use this tool to pursue their political or ideological agenda? What if they will use it to scale probation services?

In the end it was agreed that together with some colleagues I will try to develop the tool further more and test it in a few jurisdictions. A new application might be needed to support a full development of such a tool.

The presence of our colleagues from WG 4 stimulated a lot all the discussions and we are looking forward to continue this format in the future.

Ioan Durnescu (Co-leader of WG 1 – Experiencing Supervision)

Experiencing supervision in England and Wales

The probation services of England and Wales have a long and distinguished history and have been extremely influential in the development of systems and practices of supervision in many jurisdictions, within Europe and beyond. Yet today, they are under extreme political pressure and it looks likely that much of their work will be privatised in the near future. But how much do we know about the lived experience of supervision in England and Wales. Our colleagues Lol Burke and Wendy Fitzgibbon have recently summarised the available evidence on this question for our working group on Experiencing Supervision. Their full report can be found on the Documents and Resources page and here: Experiencing supervision – England and Wales Country Report (FINAL)

But to whet, your appetite, here is an excerpt:

Many of the studies referred to in this overview attempt to give a voice which is usually silent in probation histories and dominated by the accounts of policymakers, academics and practitioners. However as Bailey (2007) contends, we need to understand the offender’s view and the insights provided by the desistance literature which recognises that the meaning and significance that offenders attach to life events or opportunities to desist from offending offer a valuable point of reference and their absence is a potential flaw in probation research that is only recently being redressed. Many of the studies discussed here have some methodological limitations, in that they are often based on small samples and are largely reliant on the respondents perception of events mainly obtained through interviews and questionnaires rather than observing the processes involved (see Fergus McNeill’s country report for similar findings in relation to Scottish studies). There is also a tendency to represent those subject to supervision as a homogenous group and therefore not be fully engaged with or sensitive to issues of diversity. So what lessons can be learnt from these attempts to throw light on the experiences of supervision?

• Conducting ‘consumer’ led research with those subject to supervision is not easy but it is worthwhile because it provides a potentially unique perspective from those involved in the process. Policy and practice in England and Wales, particularly since the mid 1990s, has very much taken a ‘top down’ approach driven by the findings of empirical and apparently value-neutral research primarily concerned with establishing an evidence base for the effectiveness of various accredited programmes to reduce reoffending. But as Hedderman et al note ‘outcome evaluations which rely exclusively or mainly on information in project databases and criminal records may not capture key elements which make an intervention “work”. Neglecting service users’ insights may lead to under-estimating resource needs, unrealistic target setting, and the eventual abandonment of promising ideas in favour of the next “new” magic bullet’ (2011, p.3). Listening to the experiences of those being supervised and using their insight to shape more responsive practices is intuitively positive but as Hughes notes; ‘these must be perceived as sincere. Approaches to seeking service user views, which are based on rigid impersonal processes may reinforce negative experiences’ (2012, p.62).

• Despite their differences in approach and focus, there is a remarkable consistency within the available research regarding those values that are viewed positively by those subject to supervisory sanctions. In short, this could be summed up as an encouraging and empathetic approach taken by the practitioner in the provision of advice and assistance that is viewed as practical and relevant by the individual being supervised. The value of this approach would seem to pertain regardless of broader policy changes aimed at enhancing probation’s credentials as a public protection agency and is still evidenced in the practice of many front-line staff (Fitzgibbon 2007, 2011).

• Legitimacy in terms of a personal attachment to a member of staff who in turn demonstrates concern for and personal interest in the individual supervised was seen to be the most effective means of promoting compliance. This would seem to support  the assertion by Canton (2007) that ‘people are more likely to comply with expectations on them and to accept decisions – even decisions that go against their own preferences – when they are persuaded that these are fair (have been fairly arrived at) and are reasonable’ (p.57).

• The relational aspects of supervision are at risk of becoming increasingly fragmented in England and Wales as the government intends to accelerate its plans to introduce a mixed economy of offender services involving the transfer of much of the work traditionally undertaken by the probation service to a range of private and voluntary sector providers. This will require more research across a range of providers to ensure that those subject to these services are properly heard and accounted for in terms of designing, developing and implementing services. This also raises fundamental issues in terms of legitimacy.

• There is clearly a need for more research into the lived experiences of supervision to counter some of the misconceptions of community sanctions held by many within society (even those within the criminal justice system). It also involves understanding supervision as a shared process involving a range of actors. Further research can assist in uncovering the ‘black box’ (Bonta et al, 2008) of supervision but it also needs to be sensitive to how supervision is experienced by different groups and the differential impact of the increasing armoury of community disposals.

• There appears to be something of a gap between people’s perceptions and the realities of supervision. Whilst public support for the probation service in England and Wales has declined (despite many changes within the organisational structures largely justified on the grounds of enhancing credibility with the public) there is still significant support for the principle of rehabilitation particularly where there is a reparative element. Media coverage of serious offences committed by those on supervision, though rare, has also served to damage probation’s reputation. However, as Maruna and King (2008) found in their survey of 1000 members of the public in England and Wales, there was widespread support for the idea of ‘redeemability’ (ie. The notion that everyone can change their ways and ‘make good’ can be powerful vehicles for increasing public confidence’ (p.344).

Kind of beautiful work… this probation thing: Experiencing supervision in Ireland and the Netherlands

Thanks to the diligence of Deirdre Healy (from University College Dublin) and Vera Hoetjes & Janine Plaisier (from Impact R&D in Amsterdam), we now have two more reports on experiencing supervision in the Documents and Resources section of the blog site, though you can also access them here:

Experiencing Offender Supervision in Ireland

Offender supervision in The Netherlands

For those too lazy even to follow those links (!), here are a couple of extracts, firstly from Vera and Janine’s report:

‘Literature on the experiences of (ex-) prisoners with probation and aftercare is not there for taking. The above findings are often found in a single paragraph in studies that count dozens of pages. The experiences of the probationer does not seem to be a priority, the effectiveness of sanctions in social terms (employment, housing, identification, social network) is clearly on the forefront. In their inspection reports of the different Dutch probation offices, the ISt designates the lack of evaluation that is done on the experiences of the probationer. They stress the importance of a structural policy on such evaluation, rather than the improvised evaluation that a few of the probation offices carry out after supervision.[1]

Confidence and information seem to be key words. This is consistent with literature on behavior (Van der Pligt, Koomen and van Harreveld, 2007, in: Plaisier and van Ditzhuijzen, 2009). Confidence is important because prisoners themselves often find that they could benefit from probation. But they do not experience this support as positive until they feel they can trust the organization:

“When I used to go to the probation service, I drank for Dutch courage. Now I have Roel. The threshold for contact is very low and he doesn’t condemn my behavior. Even if I do things that he is doesn’t particularly support. “(Help On Ex-Detainees Shoot Deficit. Care and Welfare Magazine, # 3, March 2009.)

“The Salvation Army (the favorite probation service of minors) focuses much more on helping young people, and less on controlling them, like the Probation Service does.” (Salvation Army understands offenders, scienceguide.nl, May 2, 2012.)

Forum visitor “Kwinten Tarantino” asks other visitors on Public Internet Forum “Fok” – Topic “Probation Supervision” to advice him on whether or not he should cooperate with probation supervision:

“Oh, I guess I’ll just go (to my first appointment with the probation services). Just read that they sometimes prematurely terminate probation if someone does well. I must admit that I could use some help in some areas. And a baffle like this does not sound bad … I hope the officer is someone like Charlie from Flodder. I would feel comfortable with that. And that he arranges housing for me at Zonnedael … (…)

Well, I went. A nice woman, I gotta say. It was a different woman from before. Apparently, one department makes the report and another department does surveillance. We’ve been talking about the offense and had I had to read and sign for some rules. The woman said that if I wouldn’t attend, I would get a warning and if I still would not come, then they would send it back to the judge and I’d have to go to jail. In my case that would be for 2 weeks. I’m not going to let that happen.

Kind of beautiful work, it seems to me, this probation thing. “

Information is important, because a negative experience with the probation often has its origin in poor information provision for convicts prior to the start of probation. This appeared from the process evaluation of the CoVa-2, but also from the problems with expectations on housing, that Exodus found itself confronted with.’

Deirdre’s report contains the following extract:

‘There is only limited information available about the experience of offenders on probation.  The most comprehensive study was conducted by Healy (2012a) who found that over 80% of probationers expressed positive attitudes towards probation supervision.  Furthermore, probation officers rated probationers highly on attitudes, attendance and engagement.  Probationers who were currently offending were less likely to express positive views as were probationers who perceived supervision to be oriented more towards surveillance than welfare.  Their main complaints were that probation appointments were inconvenient, that officers had too much control over their lives and that they received limited practical help (see also Durnescu, 2011 on the pains of probation). Overall, although most participants claimed that probation supervision had assisted them in resolving personal problems, few attributed desistance to their supervision experiences (see Farrall (2002) for similar findings in England and Wales). However, while probation supervision may not significantly affect behavioural outcomes, it can support personal efforts at change (Farrall, 2002; Healy, 2012a).

In a follow-up study conducted several years after the initial interviews, Healy (2012b) found that probationers largely retained their positive attitudes towards probation supervision.  In particular, they valued the practical assistance they had received from their probation officers, such as help with employment, addiction and housing.  They also positively recalled opportunities to exercise autonomy, participate in strong therapeutic relationships, and engage in meaningful rehabilitative activities during the supervision period.  However, many of the probationers began to experience personal difficulties after the supervision order ended but did not know how to access formal assistance outside the criminal justice system.  This highlights the need to develop a more effective and continuous system of formal social support that extends beyond the criminal justice system.

Finally, an in-house customer satisfaction survey conducted by the Probation Service which involved 150 probationers found that around 80% were satisfied with the quality of the service they received (Probation Service, 2011).  Although further independent research is required to confirm the results, the satisfaction ratings are in line with existing studies and are consistent with international evidence.  Typically, offenders on community service report more positive attitudes towards the criminal justice system than short-sentence prisoners (Killias et al., 2000).  Offenders on community sanctions also evidence high levels of compliance, along with significant reductions in criminal attitudes and personal problems (Rex et al., 2003).’

Interestingly, at first sight these findings are not so very different from my own, in my report on Experiencing Supervision in Scotland, though much work remains to be done in terms of comparing what we are discovering in each jurisdiction. If you want to hear the results of that work, you’ll need to book a slot at our first conference in Liverpool on 26th and 27th April… more of which to follow shortly.