Adding imprecision to hypotheses: A Bayesian framework for testing practical significance in nonparametric settings

• Published in: International journal of approximate reasoning 2025-03, Vol.178, Article 109332
• Main Authors: Lassance, Rodrigo F.L., Izbicki, Rafael, Stern, Rafael B.
• Format: Article
• Language: English
• Subjects: Adherence; Bayesian nonparametrics; Goodness-of-fit; Link function; Pragmatic hypothesis; Quantile; Two-sample
• ISSN: 0888-613X
• DOI: 10.1016/j.ijar.2024.109332

Instead of testing solely a precise hypothesis, it is often useful to enlarge it with alternatives deemed to differ negligibly from it. For instance, in a bioequivalence study one might test if the concentration of an ingredient is exactly the same in two drugs. In such a context, it might be more relevant to test the enlarged hypothesis that the difference in concentration between them is of no practical significance. While this concept is not alien to Bayesian statistics, applications remain mostly confined to parametric settings and strategies that effectively harness experts' intuitions are often scarce or nonexistent. To resolve both issues, we introduce the Pragmatic Region Oriented Test (PROTEST), an accessible nonparametric testing framework based on distortion models that can seamlessly integrate with Markov Chain Monte Carlo (MCMC) methods and is available as an R package. We develop expanded versions of model adherence, goodness-of-fit, quantile and two-sample tests. To demonstrate how PROTEST operates, we use examples, simulated studies that critically evaluate features of the test and an application on neuron spikes. Furthermore, we address the crucial issue of selecting the threshold—which controls how much a hypothesis is to be expanded—even when intuitions are limited or challenging to quantify. •Accessible Bayesian nonparametric testing procedure compatible with MCMC methods.•Testing scheme applicable to both univariate and high dimesional data.•Experts' inputs help determine what are negligible deviations from a hypothesis.•Multiple testing reaches no contradictions as long as the hypotheses are nested.•Strategies for identifying the instances indistinguishable from the null in practice.