Description
The use of valid and reliable psychometric tests in personnel selection is crucial, to identify candidates who will match best the organization and who will stay in the long run. This principle applies to all companies and all fields. In that regard, it is common practice to assess candidates’ Person-Organization fit (P-O fit; Chatman, 1989) using Situational Judgment Tests (SJTs) which consist in providing applicants with the description of relevant work-related situations to which they must respond (Lievens et al., 2008). However, although those tests have satisfactory psychometric qualities (e.g., Lievens et al., 2008; Sorrel et al., 2016; Bertrand et al., 2018), their low fidelity may limit their reliability and validity (Lievens & Patterson, 2011; Lievens et al., 2008).Gamification (i.e., the use of serious games) could be a venue to tackle this problem. Serious games (SGs) have good psychometric properties (e.g., Hernandez et al., 2017; Kato & Klerk, 2017; Levy, et al., 2016) and improve the validity of measurements while lightening the workload in the selection process, whose efficiency is improved by simultaneously attracting and selecting candidates early in the hiring cycle. Indeed, SGs also have the potential to represent a modern form of communication and marketing that participates in employer branding among young people. Hence, some authors have suggested that SGs can be considered as "a hybrid kind of recruitment and testing tools" (Derous & De Fruyt, 2016).
The aim of this research was to create and develop a SG based on a previous SJT as a hybrid tool for the recruitment and selection of military personnel at the Belgian Defence. This SG, called Do We Match (DWM), measures and provides players with immediate individual feedback on their performance during the game, that is the level of match between their personal values and the organizational values of the Defence.
To develop DWM, both front-end analysis and iterative design were used. Front-end analysis aims to describe the user, his interaction with the system and the context of use (Lee et al., 2017), while iterative design, or Scrum approach, is the process best suited to the development of software and consumer products, consisting of continuously planning, designing, developing, testing, deploying and revising the product until the final software is obtained for launch. In addition, common recommended principles for gamification were used, such as the creation and implementation of an efficient reward system (Robson et al., 2015). Finally, DWM has been qualitatively and quantitatively assessed through heuristic evaluation and usability testing. Heuristic evaluation focuses on user interface design and consists of 10 usability heuristics (Nielsen Norman Group, 2021) assessed independently by three evaluators before being discussed in common (Nielsen, 1993), while for the usability testing 20 participants representative of the future user were asked to interact with the SG to assess user experience on five dimensions (Moizer et al., 2019): gaming experience, learning experience, adaptivity, usability, and fidelity.
With regard to qualitative analysis, DWM received good inter-expert evaluation for all heuristics. Some of them were improved after inter-expert consensus to reach the end product: Visibility (i.e., showing system status, telling what’s happening), Error Prevention (i.e., helping users avoid making mistakes), and Error Recovery (i.e., helping users recognize, diagnose and recover from errors). For the quantitative analysis, each dimension was well rated by participants, particularly that of usability. Adaptivity, on the other hand, received lower scores. Overall, DWM showed good properties in terms of user interface design and multi-dimensional user experience, which are key determinants of SGs development and use.
Although the experts carried out the qualitative evaluation independently and as objectively as possible, a bias is still conceivable due to the fact that they were involved in the process of developing and creating the SG. This could have an impact on the results of the heuristic evaluation. Regarding the quantitative evaluation, some items had to be slightly adapted to remain relevant to DWM. Indeed, some items from the original scale were not well represented in the game. Furthermore, given that this evaluation took place before the change in Defence uniforms, and consequently the change in most of the illustrations in the game to remain faithful to reality, it would be necessary to conduct a second usability testing to ensure that there is no influence of the new illustrations on users' appreciation of certain items.
This paper will provide an overview and insight into the whole process of gamifying a SJT. This should enable researchers and practitioners to anticipate and address with more perspective the challenges inherent in the gamification of psychometric tests, specifically those reflecting the general difficulty of conciliating gaming and scientific interests.
Finally, this work has resulted in a complete functional tool that has been gamified in every possible way, combining both the ludic and scientific worlds with the collaboration of a company specialized in serious gaming. Do We Match is all the more innovative and unique in that it has been applied and designed for such a specific field and organization as Defence. Moreover, in a subsequent stage, DWM’s impact on recruitment and selection at Belgian Defence will be investigated using, respectively, an experimental and a longitudinal study. In doing so, the gamification of the SJT will be assessed for its added value in terms of predictive validity.
As regards relevance to the scope of this conference, labor shortage makes no exception and remains an important issue for Defence. Indeed, Belgian Defence is currently facing two major challenges: recruiting and improving its employer brand among young people, and performing accurate and rigorous personnel selection to retain applicants in the long run. This project is an attempt at a creative and innovative response to tackle these issues, with the development and implementation of a SG in its recruitment and selection processes. This game-based approach could simultaneously enhance the candidate's journey through the selection process and bolster the organization's appeal, attracting candidates and encouraging them to apply. Finally, in the future, DWM will contribute to research on applicant reactions towards game-based approaches.
| Période | 27 mai 2024 |
|---|---|
| Titre de l'événement | European Network of Recruitment and Selection Researchers |
| Type d'événement | Une conférence |
| Emplacement | Brussels, BelgiqueAfficher sur la carte |