How Modelers Model: the Overlooked Social and Human Dimensions in Model Intercomparison Studies | There is a growing realization that the complexity of model ensemble studies depends not only on the models used but also on the experience and approach used by modelers to calibrate and validate results, which remain a source of uncertainty. Here, we applied a multi-criteria decision-making method to investigate the rationale applied by modelers in a model ensemble study where 12 process-based different biogeochemical model types were compared across five successive calibration stages. The modelers shared a common level of agreement about the importance of the variables used to initialize their models for calibration. However, we found inconsistency among modelers when judging the importance of input variables across different calibration stages. The level of subjective weighting attributed by modelers to calibration data decreased sequentially as the extent and number of variables provided increased. In this context, the perceived importance attributed to variables such as the fertilization rate, irrigation regime, soil texture, pH, and initial levels of soil organic carbon and nitrogen stocks was statistically different when classified according to model types. The importance attributed to input variables such as experimental duration, gross primary production, and netecosystem exchange varied significantly according to the length of the modeler’s experience. We argue that the gradual access to input data across the five calibration stages negatively influenced the consistency of the interpretations made by the modelers, with cognitive bias in “trial-and-error” calibration routines. Our study highlights that overlooking human and social attributes is critical in the outcomes of modeling and model intercomparison studies. While complexity of the processes captured in the model algorithms and parameterization is important, we contend that (1) the modeler’s assumptions on the extent to which parameters should be altered and (2) modeler perceptions of the importance of model parameters are just as critical in obtaining a quality model calibration as numerical or analytical details.

Albanito, Fabrizio | Mcbey, David | Harrison, Matthew | Smith, Pete | Ehrhardt, Fiona | Bhatia, Arti | Bellocchi, Gianni | Brilli, Lorenzo | Carozzi, Marco | Christie, Karen | Doltra, Jordi | Dorich, Christopher | Doro, Luca | Grace, Peter | Grant, Brian | Léonard, Joël | Liebig, Mark | Ludemann, Cameron | Martin, Raphaël | Meier, Elizabeth | Meyer, Rachelle | de Antoni Migliorati, Massimiliano | Myrgiotis, Vasileios | Recous, Sylvie | Sándor, Renáta | Snow, Val | Soussana, Jean-François | Smith, Ward, | Fitton, Nuala | University of Aberdeen | School of Biological Sciences, University of Aberdeen, Aberdeen, UK | Tasmanian Institute of Agriculture ; University of Tasmania [Hobart, Australia] (UTAS) | RITTMO Agroenvironnement (RITTMO) | Collège de Direction (CODIR) ; Institut National de Recherche pour l’Agriculture, l’Alimentation et l’Environnement (INRAE) | Indian Council of Agricultural Research (ICAR) | Unité Mixte de Recherche sur l'Ecosystème Prairial - UMR (UREP) ; VetAgro Sup - Institut national d'enseignement supérieur et de recherche en alimentation, santé animale, sciences agronomiques et de l'environnement (VAS)-Institut National de Recherche pour l’Agriculture, l’Alimentation et l’Environnement (INRAE) | Institute for BioEconomy [Sesto Fiorentino] (IBE | CNR) ; National Research Council of Italy | Consiglio Nazionale delle Ricerche (CNR) | Ecologie fonctionnelle et écotoxicologie des agroécosystèmes (ECOSYS) ; AgroParisTech-Université Paris-Saclay-Institut National de Recherche pour l’Agriculture, l’Alimentation et l’Environnement (INRAE) | Institut de Recerca i Tecnologia Agroalimentàries = Institute of Agrifood Research and Technology (IRTA) | Natural Resource Ecology Laboratory [Fort Collins] (NREL) ; Colorado State University [Fort Collins] (CSU) | Texas A and M AgriLife Research ; Texas A&M University System | Università degli Studi di Sassari = University of Sassari [Sassari] (UNISS) | Queensland University of Technology [Brisbane] (QUT) | Ottawa Research and Development Center | Agriculture and Agri-Food (AAFC) | Transfrontalière BioEcoAgro - UMR 1158 (BioEcoAgro) ; Université d'Artois (UA)-Université de Liège-Université du Littoral Côte d'Opale (ULCO)-Université de Lille-Institut National de Recherche pour l’Agriculture, l’Alimentation et l’Environnement (INRAE)-JUNIA (JUNIA) ; Université catholique de Lille (UCL)-Université catholique de Lille (UCL) | USDA-ARS, Northern Plains Agricultural Research Laboratory, Sidney, Montana | Cameron Ludemann Consulting | CSIRO Agriculture and Food (CSIRO) | Faculty of Veterinary & Agricultural Sciences, University of Melbourne, Parkville, Victoria | School of Geosciences, University of Edimburgh | Fractionnement des AgroRessources et Environnement (FARE) ; Université de Reims Champagne-Ardenne (URCA)-Institut National de Recherche pour l’Agriculture, l’Alimentation et l’Environnement (INRAE) | Centre for Agricultural Research [Budapest] (ATK) ; Hungarian Academy of Sciences (MTA) | AgResearch Ltd | Institute of Biological and Environmental Sciences, University of Aberdeen