Demonstrating impact

“The arts and cultural sector has huge potential to positively impact of the health and wellbeing of the population and provide innovative and creative pathways to reducing inequalities in England, but the sector, and commissioners, must be more effective at collecting and using data to demonstrate this in real time.”

Dr Justin Varney, National Lead for Adult Health and Wellbeing, Consultant in Public Health Medicine, Public Health England

For cultural organisations to succeed in winning contracts, they will need to provide evidence that their proposals can help deliver the outcomes that commissioners are looking for. Practices vary from one commissioner to another. They might be looking for a broad outcome, such as improving mental health and wellbeing within a particular demographic group. Or it may be a more specific outcome, such as reducing loneliness and social isolation amongst a community of older people, and a reduced reliance on the use of pharmaceutical drugs.

To ensure that you are measuring the right things, it’s helpful to understand the difference between outcomes and outputs. The following definitions, taken from SOLACE’s Guide to Commissioning and Sourcing (2012), are useful for this:

• An outcome is the overall end result, i.e. what happens to a service user or population as a consequence of what we do. 
• Outputs are activities conducted or products created that reach targeted participants, populations, specific audiences, decision makers or groups of individuals. Outputs are ‘what we do’ or ‘what we offer’.

Whilst it’s useful to bear this distinction in mind, it’s also useful to be aware that, sometimes, commissioners themselves are not totally clear on the distinction between the two!

…why limit things by only drawing on your own evidence bank?

Cartwheel Arts works with a measurement tool which is recognised by many commissioners in the field of mental health and wellbeing: the Warwick-Edinburgh Mental Well-being Scale (WEMWBS), for which there is a User Guide. This gives commissioners confidence in the organisation’s impact. Ian Mello, Head of Commissioning & Provider Management at Heywood, Middleton & Rochdale Clinical Commissioning Group, says: “WEMWBS is one of the tools we look for because it’s clinically validated nationally.”

However, Cartwheel uses other measurement tools as well, in what it describes as a “blended approach”. One of these is the Mental Health Recovery Star, which Cartwheel feels allows a more personal approach. This focuses on one of more than 20 policy applications available within the Outcomes Star; other applications include young people, long-term conditions and family support. The Star both measures and supports progress towards self-reliance, or other goals, and is designed to be integrated into activities that individuals are engaged in.

Because of the complexities of eliciting valid responses from some participants using some of these standardised measurement tools, Cartwheel also uses case studies of individuals and photographs of the creative work which they produce. Kerry Bertram, Project Co-ordinator, says: “We call it 3D statistics: it’s a richer resource with the numbers and headline figures as well as the emotional side and impact on people’s lives.”

‘Five Ways to Wellbeing’ is a framework for service specification and planning which is familiar to commissioners, with a strong link to delivering outcomes many are looking for. Rochdale Council has developed an online measurement system so that organisations can assess how well their services are delivering against the framework. As with WEMWBS and the Outcomes Star, arts and cultural organisations which adopt this measurement system will be using measures which are recognised more broadly, and help give their evidence of impact a degree of rigour.

To win a contract, commissioners will probably want to see evidence that the approach you propose will be effective, as well as have confidence that you will evaluate your activities robustly. The evidence you have from previous work may be fairly limited, or may not be relevant for the commission you are trying to secure. But why limit things by only drawing on your own evidence bank? If the type of activities or approach you propose has some similarities to work done by others which have a more extensive evidence base, then why not draw on this as well? Standing on the shoulders of others not only helps build your case, but also helps build the case for the wider sector.

The Cultural Commissioning Programme has pulled together some of the sites which collate evidence – including research reports by academic partners. We’ve also signposted to other organisations which provide invaluable sources of evidence in specialist areas. And we’ve highlighted a number of inspiring projects, many of which have useful evaluations, as a source of ideas and evidence for you to draw on in the areas of mental health and wellbeing, and older people.

The need for arts and cultural organisations to find ways of showing their value for money was discussed at a recent Cultural Commissioning Programme event with stakeholders and influencers from the fields of social care, health and public health. Cartwheel’s case study gives a straight-forward way of doing this. The organisation accounts for all of its expenditure (artists fees and costs of everything else involved, from employing an emotional support worker, undertaking wellbeing assessments of participants, through to final artistic product – whether an exhibition, a publication or a performance), and divides these by the number of participants, to get to a cost per head. This makes it easy for commissioners to compare costs with those of other providers. The inclusion of everything, in what is known as a full-cost recovery model, also enables the provider organisation to account for all of its expenses, and to use this as the basis of its bidding for contracts.

To see how your costs compare with those of other providers, take a look at the Unit Costs of Health & Social Care, a mine of information produced by the Personal Social Services Research Unit, and used by central and local government to develop policy and practice.

Jessica Harris is Project Manager for the Cultural Commissioning Programme, a three-year Arts Council England funded programme which supports the arts and cultural sector to engage with public service commissioning, and also works with commissioners to raise their awareness and understanding of how the arts and cultural sector can help deliver their outcomes. It is delivered by a partnership of National Council for Voluntary Organisations (lead partner), NPC and nef.

This is the second in a series of articles sponsored and contributed by the Cultural Commissioning Programme. The series explores the opportunities and challenges for the arts and cultural sector to engage in public service commissioning, and what this might mean in terms of strengthening engagement with audiences and diversifying income sources. It will highlight case studies of cultural organisations which are engaged in public service commissioning, to share their learning, and signpost readers to useful information, resources and support, including those in an online library, linked to topics explored in the case studies.

To receive regular updates, sign up to the Cultural Commissioning Programme eBulletin, or follow on Twitter @CultureComProg.

Cultural Commissioning Programme events

Build skills and knowledge to engage in public service commissioning

Phase 1: A two-day session is designed to help you understand and prepare to engage in commissioning
Phase 2: The second two-day session aims to help you understand how to demonstrate the impact of arts and cultural activities on public service outcomes.

Phase 1: Ipswich – 20 & 21 Jan
Phase 2: Ipswich – 19 & 20 March

Phase 1: Manchester – 29 & 30 Jan
Phase 2: Manchester – 9 & 10 March

Phase 1: Gravesend, Kent – 11 & 12 Feb
Phase 2: Gravesend, Kent – 26 & 27 March

Phase 1: London – 5 & 6 Feb
Phase 2: London – 12 & 13 March

Download the Learning programme brochure (pdf)