Outcomes: an approach to monitoring and evaluation

Everyone can agree that it is important to find out whether a particular service is effective. But while clients, frontline services, commissioners, and policy makers will share much common ground on the question of ‘what makes an effective service’ there will be important differences, e.g. because of different perspectives and different priorities.

SCSH is committed to promoting the use of outcome measures for monitoring and evaluation that all groups (service users, frontline services, commissioners, and policy makers) find meaningful, informative, and effective in both assessing and improving the quality of a service.

Why this matters even more in 2008

The ring-fenced Supporting People grant to local authorities is coming to an end from April 2008. From this point on, services previously funded by Supporting People will need to make a case for continued local authority funding alongside the very wide range of services focused across all groups. For many agencies it will be a real challenge to marshal good evidence on (1) the impact their service has on alleviating and preventing homelessness and (2) an account of the monetary benefit of their service. (See the Supporting People: Costs and Benefits below.)

As s a sector, there is a long way to go in routinely focusing on outcomes, both in terms of the impact of interventions and the potential costs savings. Estimating the impacts of intervention and understanding the costs and benefits in terms of time and money can be complex. Any approach to doing so will involve reaching a compromise between the resources allocated to measuring and the accuracy of the measurements.

SCSH's Work in this Area

SCSH held a meeting of key stakeholders in December 2007 to explore how the homelessnesss sector could take best advantage of the general move towards outcomes focused approaches. Ideas developed in that meeting, along with other work, will inform SCSH's future activity in this area during 2008. Our current plans are:

• to publish a briefing / mini-guide on the outcomes approach for monitoring and evaluating the effectiveness of homelessness services in the Scottish context
• to create a training / workshop event that will be delivered on a regional basis, with the aim of assisting local authorities and their partners to develop their understanding and practice

This page will be updated with more details during 2008. For more information contact Matt Elton (contact us).

Links / Publications on Outcomes

• Managing Outcomes: a Guide for Homelessness Organisations (40 pages - PDF). This is a very readable introduction and practical guide to the outcomes approach as it applies to homelessness agencies. It is published by the Charities Evaluation Service.
• Outcomes Star a graphical outcomes tool that can be used directly with clients - it has been developed, tested and refined by a number of agencies. The website includes detailed information for workers as well as managers. St. Mungo's have a briefing on some research evaluating the use of Outcomes Star (September 2007).
• Supporting People Outcomes Framework In 2007 the Scottish Government piloted an outcomes framework for monitoring and evaluating the impact of Supporting People funded projects. The evaluation of the pilot is due to be published in the first half of 2008. This document describes the outcomes model used in the pilot.
• Evaluation Support Scotland - a charity that works with voluntary sector organisations to help them measure the impact of their work. They run a range of training courses and can also work on a consultancy basis. See also their useful guides which are free to download.
• Inspiring Scotland 14:19 Fund Baseline Report An in-depth analysis of how Scotland is assisting young people make successful transitions into adulthood, with a focus on those at risk of being or being disengaged from education, training and employment. Includes useful material on outcomes. From the report: "during our research, [we] did not find any charities that could provide a systematic record of the effects of their services on young people's outcomes as they entered adulthood". (page 27)

Other Links on Monitoring and Evaluation Homelessness

Interim Guidance based on Evaluation of the Homelessness Prevention Innovation Fund Projects

The Scottish Government are currently evaluating the eight Innovation Fund projects to identify what practice works in the prevention of homelessness. The interim good practice guidance sets out some of the early lessons from the projects and shares emerging findings about how to evaluate homelessness prevention. Alternative dowload for interim good practice guidance.

What Happened Next?

This study follows 126 young people from 10 foyers in England. The study is distinctive in the attention it pays to assessing the profile of young people entering and leaving the foyers and also the local context, e.g. quality of local housing and availability of employment/training. You can read the executive summary or the full report.

Supporting People: Costs and Benefits

A report, published in December 2007, that attempts to measure the cost of the Supporting Programme and also place a monetary value on its impact. The report suggests an average unit cost of homelessness is around £5,000 and that where people are assisted by Supporting People, 40% of people who would otherwise have become homeless manage to avoid this outcome. While the report concludes that more money was saved by Supporting People than was spent, it also notes that the quality of information on both impact of interventions, their cost, and the costs of homelessness is often very weak.

Download the report.

SCSH have published a briefing outlining some estimates of the costs of tenancy failure.

Evaluation of Homelessness Prevention Activities in Scotland

A report on research, undertaken by Hal Pawson and colleagues from Heriot-Watt University, looks at a wide range of prevention activity currently being undertaken in Scotland and attempts to assess the impact of different approaches. Published March 28 2007 Download the report.

If you know of other recent publications on outcomes or monitoring & evaluation that should be included on this page please contact Matt Elton, Head of Youth Unit (contact us).

Last updated: February 22nd 2008