Sustainability Assessment Framework to Evaluate Food Systems Performance

1. Introduction

In the context of rapid urbanisation, global resource depletion, climate change and staggering socio-economic inequalities (Godfray et al., 2010; OCDE, 2015); the development of more sustainable and just cities has become a priority in the agenda of governments, international institutions and practitioners around the world. In order to transform urban spaces, cities are mobilising food as a vehicle to deliver health, social, economic and environmental benefits (Moragues-Faus and Morgan, 2015). As exemplified by the more than 170 signature cities of the Milan Urban Food Policy Pact, an ever-increasing number of cities are using the convening power of food to bring together civil society, policy makers and the private sector in order to build more sustainable urban environments.

However, and despite the amount of urban food initiatives taking place, a key challenge remains around measuring the impact of these policy processes and specific projects in building more sustainable and fair cities. Scholars identify four main barriers to progress sustainable food systems conceptualisations and related assessments:

i) the lack of a clear definition of what is (and is not) sustainable;

ii) the low applicability of global conceptualisations to local decision-making communities;

iii) constraints caused by the accessibility of qualitative and quantitative data, and

iv) the diversity of approaches applied at the local level which prevents aggregation of results and measurement of global progress (Carlsson et al., 2017a; Tanguay et al., 2010).

Furthermore, many existing initiatives have focused either on national and international food systems evaluations which struggle to incorporate local-based specificities and contradictions, or on specific initiatives and places, with limited capacity to compare across sites. In other occasions, these exercises fail to address the systemic and interconnected nature of sustainability and food insecurity (Hiremath et al., 2013; Lynch et al., 2011). Nonetheless, numerous voices call for a holistic systems approach as the most effective strategy to deliver good food for all in a more sustainable manner (Gustafson et al., 2016; Ingram, 2011). Sustainability assessments are complex, and tend to be conducted by experts, however, sustainability transformations involve changes in a range of practices, and therefore, require a diversity of actors to participate.

In this paper, we aim to address these gaps by discussing the co-development of a sustainability assessment framework to evaluate food systems performance in UK cities. For that purpose, we have conducted a participatory action research process with the Sustainable Food Cities (SFC) Network – a network of over 55 cities in the UK that are building local partnerships and delivering food strategies guided by an SFC Award framework[1] covering all issues of the food system.

This is creating a unique community of practice driving food system change and a step forward in scaling up and out urban food strategies in a national context. The project aimed to develop a set of indicators that helps councils and community food initiatives to make the case for and measure their progress towards more sustainable and secure food systems. This exercise constitutes a crucial step at a time of austerity to maximise the efficiency of resource investment and improve sustainability and food security outcomes. The framework proposed here – and associated toolkit – builds on place-based experiences of cities developing sustainable food activities; incorporates the visions, needs and challenges of a wide range of practitioners; and, at the same time, takes a holistic approach to understand food system outcomes.

The resulting sustainability assessment framework aims to provide a template to evaluate progress in key sustainability dimensions, identify problematic areas and inspire action across civil society, policy makers and the private sector.

The remainder of the paper is organised as follows.

First, we discuss the results of a systematic literature review to provide an in-depth analysis of sustainable food assessments, including their benefits and limitations.

Second, we describe the methodological design of this action-research process.

Section three presents the sustainability assessment framework, including its main components and how they work together.

Section four discusses the results and insights gained from applying this framework to the city of Cardiff.

We conclude by reflecting on how this framework addresses key barriers to progress sustainable food assessments and raises new challenges and research questions.

2. Measuring success: literature review on sustainability and food system indicators

We have used a systematic literature review method to present the current state of art in developing indicators to measure sustainability in food systems. “Sustainability”, “food systems” and “indicators” were the keywords selected to search for relevant academic literature in Scopus. The search, which was restricted to the titles of the articles, their abstracts and the keywords identified, released 68 academic articles. The review of these papers was complemented with the analysis of a vast body of “grey” literature that we identified through a Google search (also using “Sustainability”, “food systems” and “indicators” as key words) or that was cited in the academic literature that we reviewed. We also included review papers on measuring sustainability in cities and food security indicators. All material was then analyzed to identify the main focus of food system assessments, their benefits and limitations.

2.1 What is assessed?

Indicators have been defined as “something used to show the condition of a system” (Feenstra et al., 2005). In the urban sustainability realm, indicators “provide information on the state, dynamics, and underlying drivers of urban systems, with specifications of time dimension, limits, or targets associated with them” (Huang et al., 2015). A recent review of sustainability assessments in the food sector revealed the diversity of purposes that these tools serve, including: research, providing policy advice, farm monitoring, farm advice, certification, self-assessment, landscape planning or consumer information (Schader et al., 2014). They also differ in their level of assessment – from focusing on the farm level to specific products – and scale, for example assessing regional, national or global dynamics (ibid).

Non-academic institutions have been particularly active in developing some of these frameworks. Of particular importance are the Sustainability Assessment of Food and Agriculture systems (SAFA Guidelines) developed by FAO (2014) which aims to harmonize existing approaches within a common framework that comprises four dimensions of sustainability – Good Governance, Environmental Integrity, Economic Resilience, and Social Well-Being – in order to evaluate the sustainability of farms or businesses along the food value chain.

An increasing number of exercises are evaluating food system’s performance in relation to different concepts, not only sustainability, such as sustainable nutrition security (Gustafson et al., 2016), food security (Ingram, 2011), resilience of the global food system (Seekell et al., 2017), agroecology (Hatt et al., 2016), sustainable intensification (Mahon et al., 2017) or sustainable diets (Johnston et al., 2014). While some of these more holistic accounts of food systems provide specific indicators (Gustafson et al., 2016), many initiatives just highlight areas of concern or elements to take into account.

Nevertheless, most of the papers reviewed fail to provide a comprehensive evaluation of the food system. On the one hand, an important part of this literature concentrates on just one activity or aspect of the food system for example farm workers’ health (Cross et al., 2009), organic agriculture (Underwood et al., 2011), meat consumption and production (Allievi et al., 2015), the performance of a specific product (Galli et al., 2017) or obesity (Delaney Burke, 2011). On the other hand, many works focus on just one dimension of sustainability, such as studies on planetary boundaries (Conijn et al., 2018), food ecological footprints (Kissinger, 2013; Menconi et al., 2013; Wright and Ostergård, 2016), or nutrition functional diversity (Luckett et al., 2015). Indeed, health and nutritional aspects are increasingly considered as a key topic in these frameworks (Dora et al., 2015), although not always in relation to all the other sustainability or food security dimensions. This is illustrated for example by Seconda’s et al., (2017) assessment of the sustainability of the Mediterranean diet through an individual behavior approach or Röös’ et al. (2015) assessment of the environmental impact and nutritional quality of diets.

A significant breadth of work on assessing sustainable food systems relates to the re-spatialisation of foodscapes and the development of more localized and proximate relationships in a specific area. The indicators proposed here range from food miles to the area of land actively farmed or the resilience of foodsheds to climate change challenges (Bele et al., 2018; Cleveland et al., 2015; Lengnick et al., 2015). This literature is by and large linked to the Alternative Food Networks scholarship (Goodman et al., 2012). As part of this body of work, different exercises have been conducted to evaluate the impact of specific initiatives on different sustainability dimensions from community supported agriculture to food initiatives in schools (Black et al., 2015; Peano et al., 2015; Torjusen et al., 2008). For example, Peano et al (2014) developed a set of sustainability indicators and applied it to a Slow Food project which focuses on developing farm-to-market systems for local, high-quality, sustainable products. Another relevant example is FOODSCALE, a method that provides a holistic sustainability assessment of public food procurement (Goggins and Rau, 2016).

Finally, there are few contributions around urban food system assessments. To date, an important part of the urban food literature has concentrated on evaluating the contribution of urban agriculture to sustainability (Hanson and Schrader, 2014). However, recent works are taking a more holistic perspective of urban food systems. For example, Landert et al. (2017) propose a framework to evaluate urban food system governance. Efforts have also been directed towards assessing and planning city-region food systems (Dubbeling et al., 2017). This joint effort by practitioners and academics shows the complexity of defining common indicators for different contexts as well as the lack of reliable data. Notwithstanding, Prosperi’s et al. (2016) recent review shows that there is an important body of work on sustainability and cities that can inform exercises to assess sustainable urban food systems (see for example (Hiremath et al., 2013; Science for Environment Policy, 2015).

2.2 What is the logical framework?

The selection of sustainability indicators generally follows a logical or conceptual framework and relies on a series of criteria for checking the availability and quality of the data. Building on Huang’s et al (2015) review of urban sustainability assessments, Prosperi et al (2015) identify three common types of conceptual frameworks used in food studies:

• The driving force-Pressure-State-Impact-Response framework (DPSIR) where indicators are identified and clustered according to pressures or/and driving forces, system state or/and impacts, and responses. An example is Johnston et al., (2014) examination of the determinants of sustainable diets and definition of a causal model from which to build indicators.
• Theme-based frameworks cluster indicators around a concept, for example the four dimensions of sustainability (environment, economy, society, and governance). There are many examples in the literature that depart from these dimensions (Landert et al., 2017) and even expand them, for instance adding cultural and food quality aspects (Peano et al., 2014). Similarly, Nasir et al., (2014) develop an index for measuring sustainability in food systems using three dimensions of food security: accessibility, availability and utilisation. By and large, all the exercises that aim to have a holistic perspective include social, health and wellbeing, economic, environmental and governance dimensions in their framework grouping them in different ways (see (Prosperi et al., 2015; Springer et al., 2015)).
• Goal- oriented approaches, which define an overall goal, desired outcomes and indicators to measure progress. Examples include Sekell’s (2017) attempt to define resilience global food system indicators or the definition of sustainable nutrition outcomes of food systems, and the identification of 7 metrics and associated indicators to achieve them (Gustafson et al., 2016).

These approaches result in different types of indicators which, according to the literature (Alrøe et al., 2017; FAO, 2014), can be by and large classified as:

• performance-based: aimed at measuring performance and setting up sustainability indicator targets and benchmarks to motivate the agents in the food system to contribute to the transformation.
• values-based: aimed at communicating and mediating sustainability values to enable coordinated and cooperative action to transform the food system.
• reflexive-based: grounded in both knowledge, the limits of knowledge, and values.
• practice-based: focus on prescribing the necessary tools and systems required to implement good practices and therefore are process rather than outcome oriented.

2.4 How are indicators selected?

The systematic review shows that indicators are selected by combining methods such as literature reviews, available data, benchmarks obtained from good practice and interviews with experts (Gustafson et al., 2016; Li-Yin Shen et al., 2011; Peano et al., 2014). The criteria to select indicators is a key aspect of these exercises, juggling aspects such as using publicly available data that allows cross-comparison or develop fine-grained analysis at the local level (Carlsson et al., 2017b). The SMART (Specific, Measurable, Accessible, Reliable, Timed) criteria constitutes a common departing point. However, Li-Yin Shen et al.’s (2011)’ review of urban sustainability indicators concludes that the selection of indicators should be done with the understanding of the needs where these are going to be applied. In this line, Landert et al. (2017) narrowed their selection from all food-related processes in the urban area to those processes that could be influenced by institutions. This was to ensure that the assessment results yielded practical suggestions for policy makers.

2.5 What are the benefits and limitations of these exercises?

Sustainability assessment exercises by and large contribute to a better understanding of the performance of specific initiatives and systems, as well as monitor and effectively influence the path towards sustainability (Peano et al., 2014). For example, these assessments can constitute a way of measuring urban sustainability  but also contribute to making the city’s sustainable development more visible and transparent; support comparisons, evaluation and prediction of trends; help construct and harmonise data banks; provide decision-making with relevant information, stimulate communication and promote citizen empowerment and participation (Hiremath et al., 2013). However, key challenges emerge when considering the integration of frameworks across scales and thematically, as well as critical reflections around whose knowledge, values and goals are reflected in these exercises, as discussed below.

National and international exercises such as the Food Sustainability Index that rely on widely available data have the capacity to shape our understanding of problems and approach to solutions. Nevertheless, it is increasingly challenging to align national and international exercises with “local-level specificity, as even within nations, different localities may experience significantly different cause-and-effect relationships between the ecological and social variables unique to each context” (Carlsson et al., 2017b: 3). Local level indicators therefore need to be co-produced with local actors and remain context-dependent. However, efforts to align local based exercise with more widely used frameworks are still relevant. This is the case of the use of SAFA guidelines to develop an urban food governance framework for Basel (Landert et al., 2017) or the establishment of connections between indicators used by different North American cities in their urban food strategies and the Sustainable Development Goals (Ilieva and T., 2017). Despite these efforts, aggregation still constitutes a key challenge. For example, Tanguay et al (2010) study the application of Sustainable Development Indicators (SDI) for 17 western cities. They find a lack of consensus in several steps in the creation of SDI that stems from the ambiguity in the definitions of sustainable development, objectives for the use of such indicators, the selection method and the accessibility of qualitative and quantitative data

The review of the literature also highlights the importance of developing holistic accounts of sustainability. Currently, most urban sustainability assessment methods fail to demonstrate sufficient understanding of interrelation and interdependencies of social, economic and  environmental considerations (Hiremath et al., 2013). The separation of these dimensions of sustainable urban development hinders the ability of most systems to accurately understand broad movements towards sustainability (Lynch et al., 2011). In this regard, sustainability pillars still constitute the main common framework. While some exercises reviewed here promote more complex conceptualizations of food systems and sustainable development interactions, they recognize the importance of working through key sustainability dimensions in order to align with existing discourses and improve communication (Carlsson et al., 2017b).

Finally, critical reflection on how knowledge is codified and used in sustainability assessment exercises is paramount to design tools to transform the food system. Working through sustainable food challenges and trade-offs necessarily requires addressing power imbalances which entail taking a participatory perspective, from problem definition to the identification of goals and development of solutions (Alrøe et al., 2017; Travers, 1997). Participatory processes can contribute to overcome data gaps, bridge different types of knowledge or resolve conflicts over values.

However, putting in practice the principles of sustainable assessment is highly complex, since many sustainability attributes are not (yet) measurable and “hard” methodologies need to be complemented by “soft” methodologies to identify critical issues and trade-offs across different types of stakeholders (Brunori et al., 2016; Brunori and Galli, 2016; Gasparatos, 2010).

In this line, there is a recent change in indicators exercises, from the study of expost evaluation of cities’ problems to focus on exante stage where indicators can be operationalized as planning tools and become instruments of policy (Hiremath et al., 2013). This coincides with an overall approach of providing not only assessment tools but means for improving the sustainability of urban food systems. In this context, we co-developed a sustainability assessment framework, involving academics and practitioners, with the aim of evaluating current foodscapes and providing tools to improve sustainability outcomes in specific places.

3. Methodology

Building on the lessons learned from the literature review, we designed a participatory-action research process to co-produce a place-based, holistic and action-oriented assessment framework. The process was led by academics and SFC practitioners, and involved different stakeholders in order to ensure a collective identification of goals and criteria to select indicators. The project was articulated around the following steps.

First, the review of both academic and “grey” literature led to identify the range of indicators used to measure the environmental, social and economic sustainability of urban food systems. The results of this literature review guided the design of four participative workshops enrolling 99 practitioners from 41 different places to define a vision for a sustainable food city as well as identify the most relevant indicators to guide action towards that goal. The workshops were organized geographically (Cardiff, London, Edinburgh and Liverpool) in order to grasp spatial issues that might affect involvement with urban food policies at the national level and maximize city governments’ and civil society organisations participation in the project.

A third step consisted of synthesizing the results of the workshops and verifying the feasibility and coherence of the selected indicators on the basis of the available literature and insights from practitioners. For that purpose, a draft toolbox was widely disseminated. This included conducting a webinar with 42 attendees. A meeting with 17 academics and practitioners was also organised in London to gather extra feedback. In total more than 70 participants provided comments on the draft toolbox.

Finally, we applied the framework to the city of Cardiff, actively involved in the SFC Network. This step included gathering information from different institutions and databases produced in the last five years to feed the indicators toolbox and gain additional insights. These activities have resulted in developing a conceptual framework and toolbox to measure progress towards sustainable food cities as described below.

4. Co-developing a place-based and systems approach to assess sustainable food cities

The literature review shows the usefulness of theme-based frameworks, particularly structured around the sustainability dimensions, which allows to establish linkages with other exercises and improves communication with a broad audience. We therefore used sustainability as the theme-based framework and combined it with a logical framework consisting of goals, outcomes and indicators that facilitated action (see Table 1).

From this starting point, and to co-develop a place-based and sustainability food systems framework, we started by asking practitioners in four workshops what does a sustainable food city mean to you. This open question is particularly important since sustainability is a contested concept constructed by multiple actors that hold different values and ethical perspectives shaped by specific socio-economic contexts and places (Sneddon et al., 2006), and therefore questions such as how sustainability outcomes are prioritized and by who emerge as key elements in defining what the right courses of action are judged to be (Garnett and Godfray, 2012). Responses were clustered around key sustainability dimensions, however, in every workshop this classification was open to contestation and consequently evolved, resulting in four broad themes: economy, health and social wellbeing, environment and governance.

Participants were then asked to define a broad goalunder each topic and then identify key related outcomes. These outcomes helped to select and ground potential indicators. This required a discussion on what are indicators and what are key criteria to select or discard them (see Table 4 below). The process of defining outcomes and indicators revealed close linkages between the four sustainability dimensions and therefore the need to consider them in conjunction. We developed further this toolbox by providing a set of activities – and associated evidence – that will help cities to progress on these indicators.

Table 1: Definitions of the conceptual framework

Term	Definition	Example
Goal	An overarching aim,	Heathy cities
Outcome	A state or position which is reached in order that the goal is achieved	Low incidence of diet-related illnesses
Indicator (outcome-based)	A measure of progress towards delivery of an outcome, that is, a change in a relevant and measurable parameter	Decrease in the incidence of diet-related illnesses
Activity-based indicators	Activities that can potentially contribute to improve indicators	Increase portions of vegetables in school meals

The analysis of the workshops’ results led to develop a sustainability assessment framework to measure progress in sustainable food cities. This framework consists of four thematic dimensions and associated goals, a selection of outcomes expressed through thematic (outcome-based) indicators, a broad range of activity-based indicators to guide action, and a set of evidence that links the impact of specific activities to the broader sustainability outcomes as well as illustrates best practice through case studies (see Figure 1). Below we present the process of developing this framework, as well as how its different elements work together.

Figure 1. SFCN Indicators Framework

4.1 Sustainability dimensions and goals

The framework includes three thematic dimensions: economic, environmental and health and social wellbeing. A fourth transversal element, governance encapsulates systemic changes highlighting the normative and therefore political dimension of the sustainable food city agenda. Under each dimension we defined goals by synthesizing the vision for a sustainable food city expressed by workshop participants. These dimensions and associated goals are intimately related to each other and reflect a holistic approach to food system reform to deliver long-lasting change (see (Moragues-Faus et al., 2013). These relationships are expressed more clearly through the levers for change (see examples below).

Table 2: Sustainability Dimensions, Goals and outcomes-based indicators

4.2 Outcomes-based indicators

The framework grounds these goals by selecting a range of outcomes-based indicators. (see Table 2). This selection relies first on the indicators identified by participants, second on the expert feedback on the draft toolbox, third on the indicators systematically collated from the literature (see appendix A) and fourth, on the criteria identified by workshop participants as important in the selection of indicators (see Appendix B).

An additional element that emerged through expert consultation was the importance of aligning these outcomes-based indicators with the priorities and language of local actors. For example, a public health officer should be able to identify shared goals and outcomes within the ‘Health & Social Wellbeing’ dimension (Table 2) such as the decrease in the number of people overweight or obese. This alignment facilitates collaboration between different stakeholders and allows to measure the collective impact of diverse actors working together.

4.3 Activity-based indicators and associated evidence

Finally, the framework identifies a set of activities that help to support progress towards achieving the outcomes-based indicators. The activities have been largely inspired by the SFC Award framework for action and are classified on the one hand according to their contribution to the three thematic dimensions that make up a vision for a sustainable food city, and on the other hand in relation to the type of tools and mechanisms employed to deliver them. This is to improve the practicality of the toolbox and facilitate its local application.

Table 3: Example of activity-based indicators and evidence

DIMENSIONS	GOVERNANCE
	HEALTH AND WELLBEING	ECONOMY	ENVIRONMENT
LEVERS FOR CHANGE     (Example of ACTIVITY-BASED INDICATORS under lever for change: Infrastructure & planning)	The Council works to prevent the development of food deserts (where people cannot access affordable healthy food within 500m) and food swamps (where the high street is dominated by fast food outlets).	The Council maps redundant retail and brownfield sites and makes them available to new food enterprises, for example through use of meanwhile and special leases and business rates reductions and holidays.	The Council maps green and brownfield sites that could be used for food growing, composting and local food processing and distribution and makes them available to local communities.
	The Council/city protects and/or re-establishes vital local sustainable food infrastructure, such as Grade 1 and 2 agricultural land, local processing and wholesale businesses, food hubs and distribution networks.   EVIDENCE [2] Grey and academic literature Ref 30. Campaign to Protect Rural England. 2012. ‘Mapping Local Food Webs Toolkit’. CPRE. The Food Webs Toolkit stresses the importance of maintaining and building strong local food infrastructures to create new jobs, and small businesses, to ensure that more money is spent and stays in the local economy, to reduce food miles and food waste and to secure better access to fresh, healthy and affordable food. Case study CS18. Brighton & Hove City Council holds in public ownership 11,923 acres of ‘downland’ farmland. A City Downland Advisory Board has been established to develop policy which supports a viable local farm economy, diversification such as eco-tourism reconnects farmers and city residents, and promotes sustainable food production.

Whilst the outcomes have been classified according to the main health and wellbeing, economic or environmental dimension they contribute to for practical reasons, the levers for change and associated activities draw attention to the interdependent and the interconnected nature of food systems. Hence, the conceptual framework conveys the need for a systems, place-based and partnership approach to deliver different sustainability outcomes by mobilising diverse levers and implementing a range of activities which have been collectively identified as underpinning a sustainable food city. In complex and adaptive systems such as the food system, this place-based and systems approach ensures adequate monitoring so that the unintended consequences of action in one area are accounted for and mitigated if necessary (Gustafson et al., 2016).

The framework also includes a wide range of academic evidence, grey literature and case studies that underpin with evidence the capacity of each activity to progress towards the outcomes-based indicators and goals identified in this framework (see Table 3).

5. Application of the framework in Cardiff

The framework was applied to Cardiff through the collection of secondary data. Data gaps were discussed further with key stakeholders from the city council (business development, waste management and public space/allotments officers), public health, the local food partnership and two main civil society organisations through 8 semi-structured interviews plus targeted requests of information. The application of the framework constituted a tool to connect different actors and link their actions, as recognised by interviewees who for example asked for additional information on some programmes they were unaware of. This exercise shows three main sets of results.

First, the framework allows to characterise Cardiff’s foodscape, providing information on its evolution and the possibility to compare key trends on food sustainability with other cities as well as national and international dynamics (see Appendix C). The governance indicators provide a qualitative account of how the city has developed a cross-sectoral partnership and related policy activities, including commitment from key stakeholders, number of organisations involved in the partnership, the delivery of a city-wide action plan and the active participation of the partnership in key policy spaces – including Wales and UK fora – which result in embedding food in multiple urban plans and programmes.

In terms of health-related outcomes, by and large Cardiff is experiencing negative trends, for example the percentage of adults eating five portions of fruit and veg showed a decrease from 36% in 2014/15 to 32% in 2016/7; also the percentage of 4/5 year olds that were recorded as obese or overweight increased from 26.1% to 27.1% for the same period. There are some positive trends around increase in dental health and also a reduction of one point in terms of percentage of population suffering from income deprivation.

Accessing secondary data on the economic and environmental dimension of the local food system proved extremely difficult. On the one hand, this is due to the lack of specific data on the food system, for example the living wage foundation does not include a classification of food businesses. On the other hand, data collection is costly and there are individual studies that shed light on a particular topic but do not track their evolution. However, the compilation of activity-based indicators under the different levers for change and goals allows to understand some of the initiatives and progress taking place in the city that so far are not monitored.  Furthermore, this lack of data was complemented with interviews.

In this regard interviewees highlighted a general increase in interest in sustainable and local food “led by the consumer” (interviewee 4) with an “increase in spend on food and drink offer which the council needs to respond appropriately to maintain employment levels in and around retail centres” (Interviewee 7). Local businesses are also increasingly “offering more than the living wage” although there are people “on zero-hour contracts and have no idea of the implications in terms of tax and national insurance” (Interviewee3). Nevertheless, there are some initiatives that are struggling, for example 14 food co-ops offering affordable fruit and veg across the city have closed in the last year (interviewee 5).

The environmental dimension shows positive trends for key measurements such as the reduction of CO2 emissions per capita or a 17% increase in the percentage of household and non-household waste prepared for reuse, recycling or composting in Cardiff; but there are important data gaps. Interviews on the environmental dimension of the city’s food system showed an increasing level of activity around tackling food-related waste, including plastics, through training, restrictions and campaigns. However, it also surfaced contradictions for example “recycling targets are weight based, if everyone stopped producing food waste we wouldn’t be getting the recycling in” (Interviewee 2) and specific targets will not be met; also there are unforeseen consequences of new projects that “are trying to do the “right thing” and move to bio-degradable packaging but it is actually worse for us, because our processors won’t accept it, (…) so it ends up in landfill” (interviewee 2).

Food growing is the other key aspect policy makers and civil society organisations have tackled in order to address the environmental dimension of sustainability. Key informants record an overall increase in number of sites and public engagement on food growing across the city. However, the lack of reliable measures together with changing levels of activity across sites shows once more the difficulty of monitoring progress.

The second set of results revolves around how the framework allows to identify which goals and tools are addressed by partnerships as well as key gaps. For example, in terms of activity-based indicators, Cardiff is clearly focusing its efforts on improving governance and health and wellbeing outcomes, but also develops activities that simultaneously address more than one sustainability dimension (see Table 4). Most of the activities implemented relate to establishing partnerships and collaborations, developing policies and strategies and setting up public services and related support programmes. There is a lack of activity around other key levers for change such as market-based mechanisms and infrastructure and planning.

Table 4 Results of application to Cardiff

Types of activity-related indicators		N activities achieved	N cross-cutting activities achieved	Total achieved	Potential N of activities (with cross-cutting)	% Achieved
By outcomes/ sustainability dimensions	Health	5	4	9	25	36%
	Economy	2	5	7	25	28%
	Environment	3	4	7	22	32%
By levers for change*	Partnership & collaboration	2			3	67%
	policies and strategies	4			7	57%
	Infrastructure and planning	1			9	11%
	Public services and support	4			9	44%
	Knowledge and awareness	4			12	33%
	Market-based mechanisms	1			7	14%

*There are no cross-cutting activities by levers of change

Finally, the application of the framework demonstrated the usefulness but also limitations of this exercise. While indicators where collectively selected by more than 100 practitioners, using metrics that are accessible, reliable, comprehensive and inexpensive to collect remains a key challenge. Furthermore, places measure similar processes differently which complicates comparability across a country or internationally.

As illustrated above in the case of food waste, some indicators present conflictions in terms of how they actually represent progress towards sustainability. It is paramount to acknowledge these trade-offs and use criteria for the selection of indicators that is most important for potential users. In this case, rather than developing a perfect assessment framework, the aim was to collectively develop a flexible toolbox that provides a common narrative to map and encourage action towards sustainability in the city.

In the UK case, some of the outcome-based indicators are more accessible than others; for example, it is easy to monitor the decrease in the number of overweight or obese people, and more difficult to measure changes in the number of jobs in the local food economy. It would be beneficial to break down indicators in terms of neighbourhoods, gender, age, income and ethnic background.

The analysis also reveals that despite progress in terms of the number of activities implemented in the city, there are multiple processes at play that can hinder progress towards more sustainable foodscapes. For example, despite the commitment of the City Council and the push towards incorporating more healthy options in school meals, vending machines and the high street, Cardiff is the UK city with the highest number of unhealthy take-aways per capita[3]. These overlapping dynamics highlight the need to measure progress and related activities that contribute towards sustainability goals, but also to monitor negative trends and developments that might be concurrent to these developments. Furthermore, they call for a cautious approach to establish causality relations between specific actions and sustainability outcomes.

6. Conclusions: assessment as a tool for food system transformation

This paper presents an innovative participative, systems and place-based approach to sustainable food system assessments that fosters city-to-city exchanges, provides sources of evidence to make the case for a holistic intervention in the food system and aligns goals and practices of different actors shaping the urban foodscape. This co-produced framework ultimaltely offers tools for practitioners to foster effective collective action across sectors towards sustainability. At the same time, the framework allows us to understand unique challenges faced by different cities and efforts to overcome them, providing a non-prescriptive and more grounded sustainability assessment of food systems. That is, rather than focusing on specific activities implemented or snapshots of data, this framework starts to capture how cities face a range of challenges and also have different assets and capabilities to build their own sustainability pathways.

Consequently, the framework proposed overcomes key limitations of previous sustainability assessment exercises identified in the literaturte review,  by providing a place-based and participative definition of sustainability aligned with global conceptualisations, widely applicable and relevant for local communities. However, a key challenge remains, mainly access to relevant data. In this framework, data gaps are partially overcome by mapping activity-based indicators which constitute a proxy to understand progress towards sustainability in the local foodscape.

While we have demonstrated that participatory, place-based and systems food assessments are necessary to understand urban food dynamics and design effective interventions, the development and application of this framework also reveals two new challenges.

First, the extensive participatory process tailored to SFCN members and allies failed to include the diversity of knowledge, practices and needs in the food system; particularly of underrepresented stakeholders, such as those experiencing food poverty and representatives of food businesses. Experiences of food poverty are seldom recognised in assessment exercises, indicators or mainstream sustainability discourses, and therefore their absence might limit the transformative capactity of these frameworks. Furthermore, within the same city there are a myriad of processes taking place, consequently it is paramount to reflect this diversity by breaking down measurements in terms of neighbourhoods, gender, age, income and ethnic background.

Secondly, the SFCN framework takes a flexible approach to the establisment of boundaries and scales, indeed the network works with regions, counties, cities, towns and boroughs. The framework therefore adapts to these administrative boundaries, taking a losely defined ‘local food system’ approach. However, this framework does not account for the interdependencies at play in the food system, that is, how proximate and distant foodscapes condition how food is produced, transformed, consumed and disposed in a particular place. Indeed, developing more sustainable foodscapes is a collective endeavour that requires cross-sectoral, cross-scalar and translocal efforts.

Our analysis calls for a more critical approach to sustainable food assessments that actively expands their transformative capacity by developing inclusive, participatory, place-based and systems approaches. Future transformative frameworks should actively address how they can reify, shift or empower particular framings and practices around food and sustainability; as well as explore further tools for policy makers and practitioners to support the creation of translocal alliances that collectively address multiscalar, synergistic and conflicting processes currently shaping our unsustainable food system.

References

Allievi, F., Vinnari, M. and Luukkanen, J. (2015), “Meat consumption and production – Analysis of efficiency, sufficiency and consistency of global trends”, Journal of Cleaner Production, Vol. 92, pp. 142–151.

Alrøe, H.F., Sautier, M., Legun, K., Whitehead, J., Noe, E., Moller, H. and Manhire, J. (2017), “Performance versus values in sustainability transformation of food systems”, Sustainability, Vol. 9 No. 3, p. 332.

Bele, B., Norderhaug, A. and Sickel, H. (2018), “Localized Agri-Food Systems and Biodiversity”, Agriculture, Vol. 8 No. 2, p. 22.

Black, J.L., Velazquez, C.E., Ahmadi, N., Chapman, G.E., Carten, S., Edward, J., Shulhan, S., et al. (2015), “Sustainability and public health nutrition at school: Assessing the integration of healthy and environmentally sustainable food initiatives in Vancouver schools”, Public Health Nutrition, Vol. 18 No. 13, pp. 2379–2391.

Brunori, G. and Galli, F. (2016), “Sustainability of Local and Global Food Chains: Introduction to the Special Issue”, Sustainability, Vol. 8 No. 12, p. 765.

Brunori, G., Galli, F., Barjolle, D., van Broekhuizen, R., Colombo, L., Giampietro, M., Kirwan, J., et al. (2016), “Are Local Food Chains More Sustainable than Global Food Chains? Considerations for Assessment”, Sustainability, Vol. 8 No. 12, p. 449.

Carlsson, L., Callaghan, E., Morley, A. and Broman, G. (2017a), “Food System Sustainability across Scales: A Proposed Local-To-Global Approach to Community Planning and Assessment”, Sustainability, Vol. 9 No. 6, p. 1061.

Carlsson, L., Callaghan, E., Morley, A. and Broman, G. (2017b), “Food system sustainability across scales: A proposed local-to-global approach to community planning and assessment”, Sustainability, Vol. 9 No. 6, p. 1061.

Cleveland, D.A., Carruth, A. and Mazaroli, D.N. (2015), “Operationalizing local food: goals, actions, and indicators for alternative food systems”, Agriculture and Human Values,  Vol. 32 No. 2, pp. 281–297.

Conijn, J.G., Bindraban, P.S., Schröder, J.J. and Jongschaap, R.E.E. (2018), “Can our global food system meet food demand within planetary boundaries?”, Agriculture, Ecosystems & Environment, Vol. 251, pp. 244–256.

Cross, P., Edwards, R.T., Opondo, M., Nyeko, P. and Edwards-Jones, G. (2009), “Does farm worker health vary between localised and globalised food supply systems?”, Environment International, Vol. 35 No. 7, pp. 1004–1014.

Delaney Burke, J. (2011), “Just Food: Obesity Trends Demand System Strategies”, American Journal of Lifestyle Medicine, 18 May.

Dora, C., Haines, A., Balbus, J., Fletcher, E., Adair-Rohani, H., Alabaster, G., Hossain, R., et al. (2015), “Indicators linking health and sustainability in the post-2015 development agenda”, The Lancet, Vol. 385 No. 9965, pp. 380–391.

Dubbeling, M., Santini, G., Renting, H., Taguchi, M., Lançon, L., Zuluaga, J., de Paoli, L., et al. (2017), “Assessing and Planning Sustainable City Region Food Systems: Insights from Two Latin American Cities”, Sustainability, Vol. 9 No. 8, p. 1455.

FAO. (2014), SAFA Sustainability Assessment of Food and Agriculture Systems—Guidelines Version 3.0, Rome.

Feenstra, G., Jaramillo, C., McGrath, S. and Grunnell, A.N. (2005), Proposed Indicators for Sustainable Food Systems, Portland.

Galli, F., Venturi, F., Bartolini, F., Gava, O., Zinnai, A., Chiara, S., Andrich, G., et al. (2017), “Shaping food systems towards improved nutrition: a case study on Tuscan Bread Protected Designation of Origin CASE STUDY International Food and Agribusiness Management Review”, International Food and Agribusiness Management Review, Vol. 20 No. 4, available at:https://doi.org/10.22434/IFAMR2015.0174.

Garnett, T. and Godfray, C. (2012), “Sustainable intensification in agriculture. Navigating a course through competing food system priorities”, Food Climate Research Network and the Oxford Martin Programme on the Future of Food, University of Oxford, UK.

Gasparatos, A. (2010), “Embedded value systems in sustainability assessment tools and their implications”, Journal of Environmental Management, Vol. 91 No. 8, pp. 1613–1622.

Godfray, H.C.J., Crute, I.R., Haddad, L., Lawrence, D., Muir, J.F., Nisbett, N., Pretty, J., et al. (2010), “The future of the global food system”, Philosophical Transactions of the Royal Society B: Biological Sciences, Vol. 365 No. 1554, pp. 2769–2777.

Goggins, G. and Rau, H. (2016), “Beyond calorie counting: Assessing the sustainability of food provided for public consumption”, Journal of Cleaner Production, Vol. 112, pp. 257–266.

Goodman, D., DuPuis, E.M. and Goodman, M.K. (2012), Alternative Food Networks: Knowledge, Place and Politics, Routledge, Oxon, New York.

Gustafson, D., Gutman, A., Leet, W., Drewnowski, A., Fanzo, J. and Ingram, J. (2016), “Seven Food System Metrics of Sustainable Nutrition Security”, Sustainability, Vol. 8 No. 3, p. 196.

Hanson, L.L. and Schrader, D. (2014), “Creating new urban spaces of sustainability and governmentality: An assessment of the development of a food and urban agriculture strategy for Edmonton, Canada”, Emerald Group Publishing Ltd., Vol. 14, pp. 191–214.

Hatt, S., Artu, S., Brédart, D., Lassois, L., Francis, F., Haubruge, É., Garré, S., et al. (2016), “Pour des systèmes agricoles et alimentaires durables: Le concept de l’agroécologie et comment il questionne les pratiques actuelles de recherche (synthèse bibliographique)”, FAC UNIV SCIENCES AGRONOMIQUES GEMBLOUX.

Hiremath, R.B., Balachandra, P., Kumar, B., Bansode, S.S. and Murali, J. (2013), “Indicator-based urban sustainability—A review”, Energy for Sustainable Development, Vol. 17 No. 6, pp. 555–563.

Huang, L., Wu, J. and Yan, L. (2015), “Defining and measuring urban sustainability: a review of indicators”, Landscape Ecology, Vol. 30 No. 7, pp. 1175–1193.

Ilieva and T. (2017) †”, Sustainability, Vol. 9 No. 12, p. 1707.

Ingram, J. (2011), “A food systems approach to researching food security and its interactions with global environmental change”, Food Security, Vol. 3 No. 4, pp. 417–431.

Johnston, J.L., Fanzo, J.C. and Cogill, B. (2014), “Understanding sustainable diets: A descriptive analysis of the determinants and processes that influence diets and their impact on health, food security, and environmental sustainability”, Advances in Nutrition, 5(4):418-429

Kissinger, M. (2013), “Approaches for calculating a nation’s food ecological footprint—The case of Canada”, Ecological Indicators, Vol. 24, pp. 366–374.

Landert, J., Schader, C., Moschitz, H. and Stolze, M. (2017), “A Holistic Sustainability Assessment Method for Urban Food System Governance”, Sustainability, Vol. 9 No. 4, p. 490.

Lengnick, L., Miller, M. and Marten, G.G. (2015), “Metropolitan foodsheds: a resilient response to the climate change challenge?”, Journal of Environmental Studies and Sciences, Vol. 5 No. 4, pp. 573–592.

Li-Yin Shen, a, J. Jorge Ochoa, a, *, Mona N. Shah, b, et al. (2011), “The application of urban sustainability indicators e A comparison between various practices”, Habitat International, Vol. 35, pp. 17–29.

Luckett, B.G., DeClerck, F.A.J., Fanzo, J., Mundorf, A.R. and Rose, D. (2015), “Application of the Nutrition Functional Diversity indicator to assess food system contributions to dietary diversity and sustainable diets of Malawian households”, Public Health Nutrition, Vol. 18 No. 13, pp. 2479–2487.

Lynch, A., Andreason, S., Eisenman, T., Robinson, J. and Steif, B. (2011), Sustainable Urban Development Indicators for the United States, Penn University.

Mahon, N., Crute, I., Simmons, E. and Islam, M.M. (2017), “Sustainable intensification – ‘oxymoron’ or ‘third-way’? A systematic review”, Ecological Indicators1 March.

Menconi, M.E., Stella, G. and Grohmann, D. (2013), “Revisiting the food component of the ecological footprint indicator for autonomous rural settlement models in Central Italy”, Ecological Indicators, Elsevier, Vol. 34, pp. 580–589.

Moragues-Faus, A. and Morgan, K. (2015), “Reframing the foodscape: the emergent world of urban food policy”, Environment and Planning A, Vol. 47 No. 7, pp. 1558–1573.

Moragues-Faus, A., Morgan, K., Moschitz, H., Neimane, I., Nilsson, H., Pinto, M., Rohracher, H., et al. (2013), Urban Food Strategies: The Rough Guide to Sustainable Food Systems, Document Developed in the Framework of the FP7 Project FOODLINKS http://www.foodlinkscommunity.net/fileadmin/documents_organicresearch/foodlinks/publications/Urban_food_strategies.pdf, available at:

Nasir, A., Toor, M.S. and Vatta, K. (2014), “Composite index for measuring sustainability of food systems in Punjab”, CURRENT SCIENCE, Vol. 106 No. 2,

OCDE. (2015), In It Together: Why Les Inequality Benefits All, available at: http://www.oecd-ilibrary.org/employment/in-it-together-why-less-inequality-benefits-all_9789264235120-en.

Peano, C., Migliorini, P. and Sottile, F. (2014), “A methodology for the sustainability assessment of agri-food systems: an application to the Slow Food Presidia project.”, Ecology and Society, Vol. 19 No. 4, p. art24.

Peano, C., Tecco, N., Dansero, E., Girgenti, V. and Sottile, F. (2015), “Evaluating the Sustainability in Complex Agri-Food Systems: The SAEMETH Framework”, Sustainability, Vol. 7 No. 12, pp. 6721–6741.

Prosperi, P.., Moragues-Faus, A.., Sonnino, R. and Devereux, C. (2015), Measuring Progress towards Sustainable Food Cities: Sustainability and Food Security Indicators, available at: access:

Röös, E., Karlsson, H., Witthöft, C. and Sundberg, C. (2015), “Evaluating the sustainability of diets-combining environmental and nutritional aspects”, Environmental Science & Policy, Vol. 47, pp. 157–166.

Schader, C., Grenz, J., Meier, M.S. and Stolze, M. (2014), “Scope and precision of sustainability assessment approaches to food systems”, Ecology and Society, Vol. 19 No. 3, p. art42.

Science for Environment Policy. (2015), “Indicators for sustainable cities.”, In-Depth Report 12. Produced for the European Commission DG Environment by the Science Communication Unit, UWE, Bristol., available at: http://ec.europa.eu/environment/integration/research/newsalert/pdf/indicators_for_sustainable_cities_IR12_en.pdf.

Seconda, L., Baudry, J., Allès, B., Hamza, O., Boizot-Szantai, C., Soler, L.G., Galan, P., et al. (2017), “Assessment of the sustainability of the Mediterranean diet combined with organic food consumption: An individual behaviour approach”, Nutrients, Vol. 9 No. 1, p. 61.

Seekell, D., Carr, J., Dell’Angelo, J., D’Odorico, P., Fader, M., Gephart, J., Kummu, M., et al. (2017), “Resilience in the global food system”, Environmental Research Letters, Vol. 12 No. 2, p. 025010.

Sneddon, C., Howarth, R.B. and Norgaard, R.B. (2006), “Sustainable development in a post-Brundtland world”, Ecological Economics, Vol. 57 No. 2, pp. 253–268.

Springer, N.P., Garbach, K., Guillozet, K., Haden, V.R., Hedao, P., Hollander, A.D., Huber, P.R., et al. (2015), “Sustainable sourcing of global agricultural raw materials: Assessing gaps in key impact and vulnerability issues and indicators”,.PLOS ONE, Vol. 10 No. 6, p. e0128752.

Tanguay, G., Rajaonson, J., Lefebvre, J., Indicators, P.L.-E. and 2010, U. (2010), “Measuring the sustainability of cities: An analysis of the use of local indicators”, Ecological Indicators, Vol. 10 No. 2, pp. 407–418.

Torjusen, H., Lieblein, G. and Vittersø, G. (2008), “Learning, communicating and eating in local food-systems: The case of organic box schemes in Denmark and Norway”, Local Environment, Vol. 13 No. 3, pp. 219–234.

Travers, K.D. (1997), “Nutrition Education for Social Change: Critical Perspective”, Journal of Nutrition Education, Vol. 29 No. 2, pp. 57–62.

Underwood, T., McCullum-Gomez, C., Harmon, A. and Roberts, S. (2011), “Organic Agriculture Supports Biodiversity and Sustainable Food Production”, Journal of Hunger & Environmental Nutrition, Vol. 6 No. 4, pp. 398–423.

Wright, C. and Ostergård, H. (2016), “Renewability and emergy footprint at different spatial scales for innovative food systems in Europe”, Ecological Indicators, 62, pp. 220–227.

---

[1] http://sustainablefoodcities.org/awards

[2] The evidence and case study presented here are to illustrate the functioning of the toolkit. Each activity has its own range of related evidence and case studies.

[3] See local campaign Peas Please on providing more healthy food (http://foodcardiff.com/update-on-peas-please-and-veg-power/) and also reports on increasing number of unhealthy take-aways (https://www.walesonline.co.uk/whats-on/food-drink-news/fast-food-outlets-cardiff-mapped-14864764 )