Note: the following entry is missing citations. Please help us verify its content. 

Problems and Purpose

There is an established difficulty in evaluating area-based initiatives, even with substantial evaluation budgets. Showing difference over time has in the past proved very complex too, and change has often been difficult to evidence. The aim of the process was to evidence the early changes Thriving Places has initiated. In particular, it provided the Community Planning Partnership (CPP) with concrete examples to demonstrate how, or if, officers and partners are operating differently in Thriving Places and to show if a cultural shift has taken place in the engagement of local people in the development and delivery of services. The intention was to enable better evaluation of whether Thriving Places has resulted in a more flexible and empowered workforce, and whether it has led to the development and delivery of services that make use of locally available resources and assets. The goal was to improve understanding of whether real-world Thriving Places activities are contributing to the desired outcomes of the project, and if so, how they do it. Congruently, the over-arching aim was to support the services involved to develop as learning organisations. This process would ideally allow promising practice in one service, partnership or geographical area to be adapted by others.

Background History and Context

Towards the end of 2015, the Glasgow Community Planning Partnership asked What Works Scotland to work with officers across public services involved with Thriving Places, Glasgow’s ten-year area-based initiative. Thriving Places is a programme which aims to improve outcomes across nine specified geographic areas, each encompassing a community of approximately 10,000 people experiencing the highest levels of persistent multiple deprivation in Glasgow. Three of these areas embarked on their Thriving Places programme in 2014-15: Gorbals, Parkhead & Dalmarnock, and Ruchill & Possilpark. Subsequently, three further areas adopted the model in 2016.

The Thriving Places approach involves changing the process whereby resources are collectively allocated. It requires a long-term focus on partnership working, joint working at a very local community level, community capacity building and working with community anchors. Thriving Places focusses on co-production between communities and organisations, and intensive activity to build social capital and empower communities, which includes maximising the usage of assets in terms of buildings, organisations or people. The approach was motivated by evidence that the spatial concentration of poverty rather than for the effectiveness of locality-based initiatives. Essentially, if poverty is geographically bound, the response should be too.

Ten-year outcomes were sought for the Thriving Places project at its initiation, including:

• The creation of more resilient, sustainable communities which are stable, thriving and growing, and people are proud to live in.
• Communities have more aspiration and influence over the planning and commissioning of local services by CPP partners.
• Communities across the city work in partnership with CPP bodies to develop services for local residents.
• Levels of demand for particular local services shift, both up and down, as both needs, and awareness levels change.

The goal was to develop an evaluation framework for the Thriving Places programme. In fulfilling the request to develop and recommend options to evaluate Thriving Places, What Works Scotland facilitated an Evaluability Assessment process involving officers working in Thriving Places to develop and recommend options for effective evaluation.

For 2015/16, each of the three Thriving Places areas was allocated £35,000 towards staff and development costs. These funds were awarded to the appointed anchor organisation in each area and contributed to the creation of community organiser posts. This is the only current and anticipated ring-fenced CPP funding for Thriving Places. However, public services and third sector organisations in each area have devised and implemented a wide array of local activities, seeking to work in new ways and with communities, reflecting the philosophical and practical change that Thriving Places expects. 

Organizing, Supporting, and Funding Entities

The project was undertaken by What Works Scotland, which is funded by the Economic and Social Research Council and the Scottish Government.

Participant Recruitment and Selection

During the first element of the Evaluability Assessment process, staff and elected members with responsibility for leading Thriving Places locally, and staff and elected members with Glasgow-wide responsibilities for Thriving Places, were invited. Four workshops were facilitated by What Works Scotland, convened by one of the CPP partners: 

• 10 December 2015: Central leaders. Sixteen invitees; approximately seven attendees. 
• 11 December 2015: Local leaders. Twenty-four invited; approximately thirteen attendees.
• 29 January 2016: Central and local leaders. Approximately twenty attendees. 
• 22 June 2016: Central and local leaders. Approximately eleven attendees. 

Methods and Tools Used

The first phase of the evaluation assessment process was an exercise used to clarify the principles and ten-year outcomes on which Thriving Places is based. This was achieved through a series of structured workshops. Invitees included staff and elected members with responsibility for leading on Thriving Places locally, alongside staff and elected members with Glasgow-wide responsibilities for Thriving Places. The purpose of these workshops was to collectively model what Thriving Places is for and what it is expected to achieve. 

In order to judge the range of Thriving Places interventions that may be suitable for evaluation, all forty leaders who were invited to be involved in the EA process were asked to provide What Works Scotland with a short description of two or three activities that were happening or planned in relation to Thriving Places. They were asked to select activities that best exemplified the principles of the way in which they were approaching their Thriving Places work. These could be short-term, defined as up to 3 years - or long-term, anything up to 10 years. At the 29 January 2016 workshop, participants linked their proposed activities to one or more of the draft Thriving Places principles and outcomes.  

What Went On: Process, Interaction, and Participation

At the first two parallel workshops the participants were presented with the intended principles and outcomes of Thriving Places. These were drawn from a range of publications, and attendees were asked to reflect on the intended principles and outcomes, and whether this resembled what was actually being done. The discussion was used to produce an amended list, which was used to facilitate discussion at a further workshop held on Convene working group. This workshop brought both groups together to further discuss, amend and clarify the principles and outcomes. Following the 29 January workshop, the What Works Scotland team finalised the draft Principles and Ten-year Outcomes of Thriving Places based on those discussions.

The results of these discussion fell into three clusters of Principles, namely: services and communities working in partnership; mobilising communities, assets and resources; and monitoring outcomes and sharing the learning. The discussions also resulted in four clusters of Ten-year Outcomes: community changes; community mobilisation and public services change; learning from the Thriving Places programme; and measurable outcomes within Thriving Places. These reflected any differences between the recognised theory of how Thriving Places would work, and the actual practice of Thriving Places, as expressed by those leading the Thriving Places approach and participating in the evaluation assessment process. This final draft of the Principles and Ten-year Outcomes of Thriving Places was circulated to all participants for comment prior to the final workshop on 22 June 2016. It was then presented at the final workshop at which it was discussed and agreed through a facilitated process without further amendment.

Influence, Outcomes, and Effects

One of the main conclusions drawn was that extra investment in quantitative data collection appears to be of questionable value. What Works Scotland recommended instead that, given the volume and reach of current Thriving Places activities, additional evaluation effort should be invested in qualitative work to identify successful models of Thriving Place activity rather than in additional quantitative monitoring. The Community Planning Partnership should develop an understanding of the types of processes that lead to successful outcomes in Thriving Places through a purposive sample of case studies at macro, meso and micro levels. As Thriving Places is a ten-year programme, this formative evaluation process will inform the organisation as a whole for the rest of its duration. It will also be useful to practitioners working in Thriving Places areas by articulating the promising practices that have been identified in the early phase of the Thriving Places programme. 

The outcome of this conclusion was that a structured case study approach would be most effective in evaluating the performance of Thriving Places going forward. Each case study should use a context-specific, applied mix of research methods to evaluate the contribution of an activity to Thriving Places outcomes. Methods used to generate evidence may include: action research and other participatory methods involving officers, service users and Thriving Places residents; systematic approaches to monitoring; interviews and focus groups; surveys; diary keeping; observations; and documentary research. 

When systematically monitored through a case study approach using a range of research methods, a purposive sample of the exemplar activities provided through the evaluation assessment process provides short to medium-term indications of the types of intervention that successfully, or indeed unsuccessfully, fulfil the principles and outcomes of Thriving Places. This would provide formative evaluation data on a range of exemplary Thriving Places work within a short to medium-term period which would inform longer-term micro, meso and macro-level choices over what, how, who, where and when activities are done in Thriving Places. Importantly, finding out what does and doesn’t work would also fulfil Thriving Places obligations as a learning project.

Analysis and Lessons Learned

Conducting and coordinating a series of case studies is a skilled job which requires high-level research and facilitation skills. It must also implement a value-base that works with the ethos of Thriving Places. This involves collaboration with staff, service users and citizens of Thriving Places in developing, conducting and evaluating each case study. The time involved in conducting and coordinating each case study in Thriving Places would necessarily vary due to the diverse nature of the studies. What Works Scotland estimates that the resources needed to conduct and facilitate this type of research would be a University Scale 7 researcher, with a salary approximately £37-£40,000 p.a. full-time in the UK, employed at 50% FTE over two years. 

The experience of the What Works Scotland Collaborative Action Research programme, which currently operates with the four CPPs, showed that this process would also upskill the Thriving Places workers, service users and residents who were involved. This would manifest in the development of their research skills and awareness, their sensitivity to and ability to work with evidence, and their expertise in working together across professional jurisdictions. Additionally, the learning gained from the case studies can be adapted by services and partnerships across the Thriving Places areas, leading to the intervention being strongly evidence-informed as it seeks to improve social outcomes over the ten-year span. There was much to be learnt in a formative way from the existing Thriving Places activities that could be used to inform the further development and extension of the initiative to other areas. 

See Also

Evaluating the impact of participatory budgeting: How can Scotland learn from international evidence? 

Developing Glasgow’s Participatory Budgeting Evaluation Toolkit 

References

http://whatworksscotland.ac.uk/wp-content/uploads/2017/03/EvaluabilityAsessement.pdf 

External Links

http://whatworksscotland.ac.uk/

Notes

Lead Image: Evaluability Assessment of Thriving Places: a Report for Glasgow Community Planning Partnership/What Works Scotland https://bit.ly/2I4jAK6