Joint Task Difficulties Estimation and Testees Ranking for Intelligence Evaluation

• Published in: IEEE transactions on computational social systems 2019-04, Vol.6 (2), p.221-226
• Main Authors: Zhang, Chi, Liu, Yuehu, Li, Li, Zheng, Nan-Ning, Wang, Fei-Yue
• Format: Article
• Language: English
• Subjects: Algorithms; Cognition; Estimation; Games; Intelligence tests; Intelligent systems; Numerical models; Probability distribution functions; Ranking; Statistical analysis; Task analysis; tasks evaluation; testees ranking; Testing
• ISSN: 2329-924X; 2329-924X; 2373-7476
• DOI: 10.1109/TCSS.2019.2899136

In this paper, we study the testing tasks evaluation and testees ranking problem, in which tasks have different difficulty levels, and testees have different capabilities.We assume that a testee may have a probability to pass a certain task so as to allow certain uncertainty. The goal of this problem is to simultaneously determine the relative difficulty level of each testing task and the relative capability of every testee, purely based on the test outcome. We design two models to solve this problem. The first one assumes that the test outcome follows a certain Bernoulli distribution; while the second one assumes that the test outcome follows a certain Bernoulli distribution with the beta distribution-type a priori knowledge. Then, we form the original problem into likelihood estimation problems and solve them by using coordinate descent algorithms. We show that the beta distribution-type a priori knowledge is needed, when we only carry out a limited number of tests due to time and financial budgets. All these findings are useful to intelligence tests. Finally, we discuss how to extend this statistical learning model for more general cases as well as in a specific case in the field of Computational Social Systems like artificial social cognition evaluation.