Article open access publication

When experts disagree: the need to rethink indicator selection for assessing sustainability of agriculture

Environment, Development and Sustainability, Springer Nature, ISSN 1387-585X

Volume 19, 4, 2017

DOI:10.1007/s10668-016-9803-x, Dimensions: pub.1000849818,



  1. (1) Wageningen University & Research, grid.4818.5
  2. (2) Aarhus University, grid.7048.b, AU
  3. (3) Centre for Sustainability: Agriculture, Food, Energy, Environment, University of Otago, Dunedin, New Zealand
  4. (4) Instituut voor Landbouw en Visserijonderzoek, grid.418605.e
  5. (5) University of Antwerp, grid.5284.b
  6. (6) AgResearch, grid.417738.e
  7. (7) Lincoln University, grid.16488.33
  8. (8) Landcare Research, grid.419186.3
  9. (9) French National Institute for Agricultural Research, grid.414548.8
  10. (10) Ministry for Primary Industries, grid.467701.3
  11. (11) National University of Chilecito, grid.441638.c
  12. (12) INRA, UMR 1121 Agronomie et Environnement, INRA-Université de Lorraine, BP20507, Colmar Cedex, France
  13. (13) University of Lorraine, grid.29172.3f
  14. (14) University of Otago, grid.29980.3a
  15. (15) The BHU Future Farming Centre, Lincoln, New Zealand
  16. (16) Knowledge Centre for Agriculture, grid.426594.8
  17. (17) University of Canterbury, grid.21006.35
  18. (18) Forschungsinstitut für Biologischen Landbau, grid.424520.5


Sustainability indicators are well recognized for their potential to assess and monitor sustainable development of agricultural systems. A large number of indicators are proposed in various sustainability assessment frameworks, which raises concerns regarding the validity of approaches, usefulness and trust in such frameworks. Selecting indicators requires transparent and well-defined procedures to ensure the relevance and validity of sustainability assessments. The objective of this study, therefore, was to determine whether experts agree on which criteria are most important in the selection of indicators and indicator sets for robust sustainability assessments. Two groups of experts (Temperate Agriculture Research Network and New Zealand Sustainability Dashboard) were asked to rank the relative importance of eleven criteria for selecting individual indicators and of nine criteria for balancing a collective set of indicators. Both ranking surveys reveal a startling lack of consensus amongst experts about how best to measure agricultural sustainability and call for a radical rethink about how complementary approaches to sustainability assessments are used alongside each other to ensure a plurality of views and maximum collaboration and trust amongst stakeholders. To improve the transparency, relevance and robustness of sustainable assessments, the context of the sustainability assessment, including prioritizations of selection criteria for indicator selection, must be accounted for. A collaborative design process will enhance the acceptance of diverse values and prioritizations embedded in sustainability assessments. The process by which indicators and sustainability frameworks are established may be a much more important determinant of their success than the final shape of the assessment tools. Such an emphasis on process would make assessments more transparent, transformative and enduring.


Research Categories

Main Subject Area

Links & Metrics

NORA University Profiles

Aarhus University

Danish Open Access Indicator

2017: Unused

Research area: Science & Technology

Danish Bibliometrics Indicator

2017: Level 1

Research area: Science & Technology

Dimensions Citation Indicators

Times Cited: 35

Field Citation Ratio (FCR): 19.54

Open Access Info

Green, Submitted