How much can data tell us? - Paul Hamlyn Foundation
In our latest issue of Trust & Foundation News, we spoke to members about the kinds of data they use and what it can help them to achieve, as well as some of its risks and limitations.
The third article in this series comes from Moira Sinclair, Chief Executive, Paul Hamlyn Foundation.
Use of data is essential for organisations who want to learn and improve what they do. Learning is a collective effort, and it is crucial to look at what has and hasn’t worked. We support grantees to collect, share, use and evidence data so they can test new approaches and evaluate and improve their practice. In turn, that gives us more information on what we are funding and allows us to analyse and make decisions on our grant-making.
The data we ask from grantees is not about reporting to us. It’s about creating a context and environment where we and our grantees are all engaged and committed to learning more about what we all do. We are committed to making the most of the data collected, and share key findings from our grant-making and research that we feel will be useful to others, including the thinking about what the data means we should do next. It is the data plus analysis and judgement that is really useful.
We employ a small Evidence and Learning team and use a variety of means, from surveys to independent evaluation of our funds and of the data our grantees create, and to the commissioning of quantitative research to see the bigger picture and qualitative to help us understand more about the issues.
This approach is underpinned by proportionate and tailored approaches. Depending on the type of grant, we might expect grantees to collect evidence of impact in a very light way, or where we think we may have more to learn, we expect more in terms of evaluation. We encourage organisations we fund to work with the people they support to co-design the evaluation of the programme and services – the experience and knowledge of those closest to the issue is really important for understanding issues and learning. Sometimes we will supplement the grant with additional funds to support that evaluation design and data collection.
If we genuinely want to create a culture of learning, trust in the relationship with grantees is essential. They need to be able to tell us what went wrong, as well as what went right, without being penalised. Our values include being open and transparent, so we strive to use that data in a way that respects the confidentiality of the grantee, but spreads the learning more widely, and we publish all our grant data on 360Giving.
We need to walk our talk. Grantee and applicant perception reports enable us to analyse systematically the experience of working with us, addressing the power dynamic through a completely anonymised survey once every three years, carried out by the Center for Effective Philanthropy. We publish those results and what we are going to do in response (see here). Sometimes we invite grantees with common interests to come together to discuss their programmes, so that is useful learning for us as well as individual grantees.
It can be difficult to collect data on the positive impact of the arts, particularly at programme level.
And it can be tough for any small organisation that has throughput of people and is trying to measure impact. But you can set up systems and create interesting proxy measures for some things where you are trying to describe impact. You can work with participants, their carers or teachers to reflect on the step change in them as a result of taking part in an activity.
And where we have had the ability to look across a cluster of organisations or a sectoral field, or an approach to participation and we have been able to gather data, that has enabled conversations with government and wider change. For example, the Our Museum programme placed communities at the heart of the participation agenda in museums. If you want to create a real change in relevance and connectivity, you need to have people in co-production, co-design, in governance and all sorts of other aspects of the museums. We gathered lots of evidence that demonstrated how that approach changes the relationship and creates benefits for museums, and through that we have been able to have really productive conversations with all of the bodies responsible for museums in the UK, we fed into Neil Mendoza’s review of museum practice, and now we can see things we view as important beginning to flow through in policy terms.
Paul Hamlyn Foundation
Other articles in this series
How much can data tell us? - Walcot Foundation
How much can data tell us? - Friends Provident Foundation
How much can data tell us? - Corra Foundation
How much can data tell us? - United St Saviour’s Charity
How much can data tell us? - Co-op Foundation
How much can data tell us? - Ada Lovelace Institute