Can model structure families be inferred from model output?

• Published in: Environmental modelling & software : with environment data news 2020-11, Vol.133, p.104817, Article 104817
• Main Authors: Remmers, Janneke O.E., Teuling, Adriaan J., Melsen, Lieke A.
• Format: Article
• Language: English
• Subjects: Correlation analysis; Dendrogram; Family trees; Inference; Model genealogy; Model structure uncertainty; Modular modelling framework; Modular structures; Parameter uncertainty; Sampling model space; Signatures
• ISSN: 1364-8152; 1873-6726
• DOI: 10.1016/j.envsoft.2020.104817

In model studies, a careful consideration of uncertainty is needed. Parameter and data uncertainty can be explored by sampling, but model structure uncertainty is more challenging to capture since the underlying hypotheses of many models are not directly clear. This study explores whether model structure can be inferred from model output. We created a dendrogram (family tree) based on model structure using a modular modelling framework. Subsequently, we created dendrograms based on model output. We determined the correlation between both dendrograms to analysed if model structure families could be inferred from model output. Results from this experiment over 671 climate instances showed that the performance of the inference depends on the type of output evaluated, and the climate. However, the performance of the inference is overall low, implying that model structure cannot be inferred from model output. These results demonstrate the need to further investigate opportunities to sample model space. •We test whether model structure families can be inferred from model output using a novel combination of approaches based on family trees (dendrograms).•Results show that model family trees based on model structure are generally different from model family trees based on signatures of model output.•The difference between both family structures depends on the signature evaluated and the climate where the models are applied.•The results of this study imply that we need to look for other ways to sample model space.