Automated assessment and feedback of open-response assignments remain a challenge in Computer Science despite recent milestones in fields as natural language processing. Even if we could make quality assessments with complete human supervision independence, many would argue against it. Competence assessment is a sensitive topic possibly impacting the issuance of a certificate that asserts that a student is ready for her insertion in the labour market or to continue her progress in the education system.

Despite the efforts on "opening" the black box of neural networks, current neural models are rarely equipped with logical narratives of the decision chains that lead them to a final prediction or classification. Nevertheless, transparency and explainability are desirable requisites for automated assessment systems.

For all the above, many researchers propose hybrid solutions combining the benefits of automation with human judgement. Probabilistic models of competence assessment join the benefits of automation with human judgement. In this work, two probabilistic models of peer assessment (PG1-bias and PAAS) are replicated and compared. We also present PG-bivariate, a model combining the approaches from the first two. 

Alejandra López de Aberasturi is a PhD candidate at the IIIA-CSIC.