It’s now six months since I joined the Fulfilling Lives national evaluation team. My background is in community and criminal justice research, so I have a degree of familiarity with the target group. I have evaluated initiatives to help re-engagement on release from prison, to provide pathways to education and ultimately employment, and to help with accommodation and independent living. Offenders often have chaotic lifestyles and multiple needs including experience of homelessness, alcohol and/or drug dependency, and/or mental health issues. You can often find childhood trauma, special educational needs or attachment issues as well. What struck me time and time again when listening to offenders and their workers tell me their stories was the lack of coherent support available to people who are in desperate need of help.
The latest national evaluation report on the Fulfilling Lives (Supporting People with Multiple Needs) programme, published today by CFE, shows continuing high demand for help. The 12 funded projects have successfully engaged with nearly 3,000 people affected by homelessness, substance misuse, offending and mental ill health.
Beneficiaries who remain with the programme show clear signs of progress, but this takes time and substantial resource. Project staff often need to spend extended periods of time with beneficiaries and have to be flexible to cope with chaotic lives. However some beneficiaries have needs for which they will always require support. What constitutes success varies from person to person and in many cases, success is about developing strategies, resilience and understanding to effectively manage their needs.
The report includes clear messages and promising practice, providing recommendations not just for those who work to support people with multiple needs, but also for service commissioners and funders.
Read the full report here: http://mcnevaluation.co.uk/wpfb-file/annual-report-2017-v1-final-pdf/
A key feature of Fulfilling Lives is that it is a learning programme; it aims to gather and share learning about what works, what doesn’t and what needs to change. A frequently used method to collect views and experiences from which to develop learning is the survey. In this blog, Kerry Dowding (Research and Evaluation Officer, Fulfilling Lives South East) shares her tricks and tips on how to get the most out of your survey design. Kerry has been creating surveys for the voluntary sector for the last eight years, and received a Masters of Research in 2016.
It’s 4.50pm, and you remember that you promised to send out a few survey questions by the end of the day. You scribble out some questions that roughly fit the bill, and send them off. The responses you get back are less than inspiring. So you end up going with the ideas you originally had for the project. We’ve all been there!
Why bother with survey methods?
A good survey can bring so much more to the table than this, if they are your method of choice (and they don’t suit every situation all of the time). A good survey can provide quality information to help us change systems and organisations, and to move forward in a way which is inclusive and co-produced. When done well, they are the gift that keeps on giving. Looking to get more from your surveys? Here are some hints and tips I’ve picked up along the way.
Before you start
- Why are you doing the survey in the first place? Write one sentence explaining the question you would like your survey to answer (and some smaller aims within that, which is fine – as long as they’re clear). Refer back to this and be strict – leave out anything that doesn’t help you to answer your question.
- How much time do you have to analyse this, really? If you have limited time, write fewer questions. Better to ask five questions that will be well used than twenty which you end up skimming over.
- Do you need to offer incentives? Some groups, like clients or volunteers, might be more likely to do your survey if there is an incentive. This is especially true if the survey is long, or they haven’t been in touch with the organisation for a while.
- How do you want to divide your results? You might like to split your data by area, hostel, gender or shoe size, to compare what different groups say in response to your questions. It’s important you think about this first so you include the questions you need. Just make sure these splits have a clear benefit to your work. For example, asking about shoe size is only really useful if it were clearly linked to something like NDT Which it’s not.
Question types and order
- Keep everyone in the loop. Tell them who you are, why you’re asking them questions, how long it will take, and what you’ll do with it all. It improves your response rate and is generally a good thing to do for people who are sharing their thoughts with you.
- Mix up open and closed questions. People can get bored with all open questions and provide poor quality answers. On the other hand, multi-choice ones don’t give you a lot of information to go on. Try putting open questions towards the end, so people get to them after they’ve been ‘warmed up’ by the easier ones.
- Throw in other types of question. Wake people up a bit by asking them to rank different options, use a slider to rate something from 0% to 100%, or use visual methods. There are lots of different types of question that are more fun to fill in (and analyse) than your regular.
- Don’t scare people off! Keep the sensitive questions for later in the questionnaire. Asking about someone’s criminal convictions or worst fears right off the bat can put people off.
A word on wording
- Be friendly and clear. Use clear, short words that everyone can understand. Think about trying to keep a warm, chatty tone (depending on your audience), and use plenty of examples for any tricky concepts – so there is no ambiguity about what you mean.
- Only one question per item! If you ask 3 things in one question, you don’t know which part of the question the person is answering. Asking ‘Do you feel happy, proud, confident and empowered through your work with Fulfilling Lives?’ in a yes/no question makes it unanswerable. A good rule of thumb is to banish the word ‘and’ from you questions as much as possible, so you can trust the answers you get.
- Keep an open mind. We often hope answers will be positive, but don’t let this optimism seep into your questions and make them look biased. Rather than asking if things are ‘better’ or ‘worse’ than before, just ask about what changes a person has noticed and see what they say.
- Make sure you cover all bases. There should always room for people to give an answer you didn’t expect, say they are unsure or surprise you. Sometimes the best answers refer to ideas you hadn’t even considered.
Getting it out there and making it useful
- When the results are in, get together with others who were involved in the survey to think about what to do next, and when. Set deadlines to act on what you’ve found, so it doesn’t fall down the list.
- Feedback to everyone. Make sure that those who took part know how it went – and any changes you’ve made as a result.
- Improve your method. Take five minutes to talk about how the survey was carried out and what you would do differently next time. This is important so that what you’ve learned isn’t lost.
- Share where you can. Think about if there is anyone else who might like to see the anonymised results/report that comes from the work. Data is an important product these days and being open (as long as it’s safe to do so) can improve relationships in the future.
It might not always be possible to do everything listed here all the time. However, finding ways to improve your survey work can have real and worthwhile benefits: reaching more people, getting better answers, and saving time by keeping focused. We’ve seen the results of this within our own organisation. As a project focused on learning and research, it’s important that we get the right information for ourselves, and to inform how we change systems. Try putting some of these ideas into your work locally, to banish the rushed survey questions forever!
References & further reading
Bryman, A. (2015). Social research methods. Oxford University Press. pp 231-267
Designing a Survey (January 19th, 2018). Retrieved from https://www.sciencebuddies.org/science-fair-projects/references/how-to-design-a-survey
Harrison, Chase. Harvard University Programme on Survey Research: Tip Sheet on Question Wording. (2007, November 17th) Retrieved from https://psr.iq.harvard.edu/files/psr/files/PSRQuestionnaireTipSheet_0.pdf
How to write in plain English. (2018, January 19th). Retrieved from http://www.plainenglish.co.uk/files/howto.pdf
Rea, L. M., & Parker, R. A. (2014). Designing and conducting survey research: A comprehensive guide. John Wiley & Sons.
Thomas, G. (2013). How to do your research project: A guide for students in education and applied social sciences. Sage. pp 207-17
At the centre of the Fulfilling Lives (Multiple Needs) programme is evaluation and here the workforce is on the frontline. For it is they who source, collate and complete the two key measures – the Homelessness Outcomes Star and the NDT Assessment – with the service beneficiaries. Frontline workers say something they enjoy about the programme is the move away from a purely target driven approach. To make the most of this we need to make sure that the essential evaluation is not a chore. Not tasks that are completed as an add-on to a day’s work but ones that are integral to the way of working alongside beneficiaries and peer workers.
Thus, we need to know what it is about the projects that captures the imagination and creativity of practitioners. Ask what is it they like about the work alongside beneficiaries which will secure as full an evaluation picture as possible? It is my contention that it is knowingly being part of a virtuous circle – one which takes theory into practice, evaluates that practice and thereby shapes a better informed and improved theory that can go on to influence wider practice, which is so rewarding. Pride – for both frontline worker and service user – is in curiosity satisfied. In the Fulfilling Lives (Multiple Needs) overall programme then, evaluation of just how each individual beneficiary achieves their aspirations should be accumulated, lead to system change and improve the lives of many people, families and communities with multiple and complex needs. Evaluation, and the curiosity that fuels it, are vital to generating that positive feedback loop on which the projects depend for their results.
In my experience curious practitioners have:
- Clarity of programme and project purpose
- Detailed understanding of their job role and responsibilities
- The skills and training required to make good use of the evaluation tools
- Time to collect the data and information needed
- An ability to make evaluation an engaging part of their relationship with beneficiaries
- Involvement in the analysis and assessment of data and information
- Encouragement to bring their wider knowledge and experience to support and/or challenge data and information
- Support to ask difficult questions about the reliability of data and information
- Knowledge and understanding of the common pitfalls in research and evaluation
- Feedback on what the data and information is revealing across projects
Essentially the outcomes of evaluations should help not hinder, be practical for practitioners and challenge in a way that stimulates further curiosity. Evaluations must be inquisitive about the interventions and approaches. The more we know about what works, and why, the better. In this way the best workforce can be recruited, trained, supported and retained. What follows is not just an effective service but one that is sustainable and replicable because it is founded on research. For, as we all know “research is formalised curiosity. It is poking and prying with a purpose”.
What we need to know from project beneficiaries, frontline workers and their managers is what will assist us all to undertake really effective evaluations? How can we make sure we make the best use of the evaluation tools we have? Do drop me a note at firstname.lastname@example.org or call on my mobile 07947 680 588.
 Zora Neale Hurston
I have blogged previously about purpose and it’s time to move on to the detail – where the devil lies (alongside salvation) . The question is how do we recruit, train, support and retain an effective workforce? A workforce with the knowledge, skills and experience to change lives and systems alongside people with complex needs including homelessness, offending, substance misuse and mental ill-health. One that supports people to overcome barriers to services and engagement. A workforce that is genuine and honest about the collaborative involvement of individuals with lived experience.
It is a big ask to come up with an exhaustive list of what the workforce does and how it goes about its tasks. Anyone that has spent time drawing up job descriptions knows the lengths you can go to trying to capture everything. Equally, they know the struggles of seeking to be both precise and concise. Perhaps that’s why so many job descriptions are rarely used in guiding day-to-day work and are only got out either when there is a vacancy or a workforce problem. In projects like Fulfilling Lives (Multiple Needs) the issues are compounded because many of the roles are evolving and breaking new territory.
Helen Gavaghan, Senior Engagement Worker at Inspiring Change Manchester (ICM), gives an inkling of some of the things workers do to support people to think about change. She says some ‘traditional’ approaches need challenging and staff freed up to be open about working with both their own and their client’s lived experience. She concludes by suggesting learning about what it is that staff are doing is collected and brought together as a tool-kit for other practitioners.
In the Fulfilling Lives (Multiple Needs) projects, when considering the learning that is being fed back overall, I feel there could be more detailed examination of what the frontline project workers do that makes a difference to people’s lives. I believe that there is some valuable learning that could be collected and shared and, if it isn’t, may be lost. General explanations of the approach, such as “building trusting relationships” and “avoidance of judgement” are fine – but I believe are in no way getting down to the specifics and nuances of the great work that is going on.
Vic Citarella delves further into the significance of ‘purpose’ for service users and the workforce in the Big Lottery Fulfilling Lives (Multiple Needs) projects.
Indications are emerging from evaluations that the ‘purpose’ of each Fulfilling Lives project is very important in generating value and ownership. For beneficiaries, engaging in meaningful activities appears to create a vital sense of purpose. Equally for the workforces involved feedback suggests that the ‘purposeful’ nature of job roles generates added value and personal motivation. In my last workforce blog, I suggested that the value stemmed from four sources:
- Meaningful service user engagement
- Concepts of open-endedness and persistence (turned into practice)
- The ideas around psychologically informed environments (PIE) and the like
- Systems Change
In focussing on workforce matters in this series of blogs I have therefore dug a bit deeper to address questions such as: what is the purpose of purpose? How is purpose agreed, described and refreshed? What is it about the four sources which users and workers value that drives them on to achieve their goals?
The National Expert Citizen’s Group is a group of individuals with lived experience of homelessness, substance misuse, mental ill health and offending behaviour. The group is comprised of individuals from the 12 funded project areas for the Big Lottery Fund’s ‘Fulfilling Lives: Supporting people with multiple needs’. CFE Research facilitates the group. Members from the group have been accepted to run a workshop at the 2017 Multiple Needs summit in Milton Keynes. Sarah Robinson, the group’s facilitator (and research manager on the programme’s National Evaluation), will be supporting two group members to lead and present at their first national conference.
Sarah Robinson (CFE) and Wayne Richmond and Nicola Plumb (Blackpool Fulfilling Lives) share their reflections on a National Expert Citizens’ Group Workshop
On Monday 28th and Tuesday 29th November 2016 CFE facilitated a two day workshop for the National Expert Citizens’ Group (NECG). The NECG is a group of individuals with lived experience of multiple needs; the group is formed by representatives from each of the 12 funded Fulfilling Lives project areas. The Monday was a training and planning day for the experts by experience. The day focused on training delegates to chair meetings/events and facilitate workshops at events. It was an opportunity to build the confidence and skills of the experts. The activities used to test their skills focused on deciding the direction of the NECG over the next 12 months.
In the first of a series of blogs, Vic Citarella considers the crucial role of the workforce in Fulfilling Lives for people with multiple and complex needs. Vic is keen to start a dialogue with projects on this topic. You can get in touch with him using the details below.
“The CFE and University of Sheffield 2nd annual report into the national evaluation of Fulfilling Lives: Supporting people with multiple needs programme has chapters on ‘interventions and approaches’ and on ‘working the frontline’. The report says it raises as many questions as it answers but without doubt it pinpoints the workforce and what they do as the mission critical factor in the projects. More is promised by way of research and future evaluation. That means, among other things, dialogue with the practitioners, the managers, the stakeholders and the customers of the services.
In the hunt for a bike for my daughter the typing of “kids bikes” in Google returned just shy of 78 million results in less than one second! After years in research I still find it extraordinary the number of sources of information that are returned when performing any internet search (“kids bike bells” a meagre 1.5 million returns in 0.76).
Chatting with friends over the weekend the topic turned to the works of Keanu Reeves (don’t ask) and his big break: the film Bill and Ted’s Excellent Adventure (1989!). We all agreed that the central philosophy of ‘Be excellent to each other’ was a pretty sound one.
CFE Research, working in partnership with the University of Sheffield, have published their latest report on the evaluation of the Big Lottery Fund’s ‘Fulfilling Lives: Supporting people with multiple needs’ programme.
Supporting those who are chronically excluded and disadvantaged – experiencing homelessness, mental ill health, substance misuse and reoffending – is a challenge that needs a different approach. The £112 million initiative is funding 12 projects across England to provide more person-centred and co-ordinated support.
Latest findings from the evaluation show that projects have made a good start in reaching those with greatest need. But continued effort is needed to reach ‘hidden’ and harder to reach groups such as women with multiple needs and people from Black and minority ethnic communities.
The initiative still has six years to run, and as such few beneficiaries have successfully moved on to date. But there are signs that those who remain on the programme are making progress, accepting help, engaging better with services and building trusting relationship. Frontline staff report that flexible, open-ended support that focuses on beneficiaries’ own priorities is key to making a difference.
The All Party Parliamentary Group on Complex Needs and Dual Diagnosis, is looking for examples of good practice in working with people with co-existing substance misuse and mental health issues, to inform an online ‘library’ for policy makers, commissioners, providers, academics, experts by experience and others to reference.
For anyone who is interested in either providing a case study of good practice or contributing to this work in general, please do get in touch with us at email@example.com
They are also planning on conducting a short survey to ascertain the current strengths, risks and opportunities facing services supporting people with multiple complex needs so please do look out for this in the next few weeks.
Sent on behalf of David Burrowes MP & Lord Victor Adebowale CBE
Co-Chairs of the APPG on Complex Needs and Dual Diagnosis
Turning Point provides the secretariat to the APPG on Complex Needs and Dual Diagnosis.
I am part of the National Expert Citizens Group (NECG). This is a group which draws people from all twelve Fulfilling Lives areas together to learn and try to influence on issues which are important to us. We all bring our own skills and expertise but the one thing that we have in common is that we all have lived experience of multiple needs.
On Friday 29th January we came together in Stoke-on-Trent to meet with Home Office representatives and Public Health England. On the agenda: The National Drug Strategy Review due to be published in March 2016.
On the 29th January, in Stoke on Trent, the National Expert Citizen’s Group of Fulfilling Lives were invited to review the National Drugs Strategy. The strategy focuses on three key themes: Reducing Demand, Restricting Supply and Building Recovery. The day focused on the third strand – Building Recovery. A member from the Home Office’s Drug Strategy Team provided an initial presentation to set the scene. The rest of the day focused on obtaining expert’s views and opinions, from their lived experience, on key aspects that are needed to inform the new strategy. The review was particularly interested in:
— Barriers to recovery
— How to reduce the stigma of substance misuse
— What works well in recovery – particularly around housing and employment
The day was recorded graphically and in notes.
The Home Office will be providing an annotated strategy after its publication later this year which marks which areas were changed/influenced by the views of the experts. A follow up session with the Home Office will occur at the May NECG meeting in Bristol.
Following the success of the Experts by Experience launch film in 2015, Birmingham Changing Futures Together decided it would be fitting to create a new film for 2016. This film features some new year’s resolutions, a variety of stories and successful journeys and recommendations for services for 2016.
Birmingham Changing Futures Together will continue to work with Experts by Experience to help improve services across Birmingham, and make the lives of their users more fulfilled.
They really hope you find it as rewarding to watch as they found it to make.
It’s coming up to the end of the year so I thought I would take some time to reflect and take stock. I started my role in January excited. The pioneering nature of the Fulfilling Lives programme interested me but above all was the hope that lives would be transformed both during the life of the project and in a real on-going way into the future. The opportunity to take any part in this, however small, seemed both a privilege and a responsibility.
PIElink is a rapidly growing community of practice with the aim of connecting people involved with addressing the psychological and emotional issues that go with homelessness, particularly through the development of ‘psychologically informed environments’ – PIEs.
They have just released their first newsletter, follow this link to find it.
If you want to find out more about PIElink then got to their website to get involved. http://pielink.net/
The event marked the first year of the WY-FI project. The audience saw videos made by beneficiaries of the Wakefield Navigator Team, presentations about the team’s achievements and testimonies from partners in Wakefield. Out Of Character Theatre showcased some of the work they had been doing across West Yorkshire to help WY-FI users tell their own stories of recovery and resilience
Read more about it here