Volume 34, Issue 1

Abstract

Samenvatting

In dit onderzoek staat de begripsvalidering van een virtual reality (VR) situationele beoordelingstest (of zogeheten situational judgment test; SJT) centraal. Voor zover bekend is dit de eerste keer dat VR-technologie wordt toegepast om een SJT te construeren. De VR-SJT beoogt het inzicht in effectief leiderschap te meten en rapporteert naast een totaalscore iemands scores op taaken persoonsgericht leidinggeven. De betrouwbaarheid en de begripsvaliditeit van het instrument werden onderzocht bij 121 assessmentkandidaten. De convergente validiteit werd bestudeerd met een video-SJT met een vergelijkbare meetpretentie. Verder werd onderzocht hoe de VR-SJT verband houdt met cognitieve capaciteiten en persoonlijkheid en werd de ervaring van deelnemers bestudeerd. De convergente en discriminante validiteit werden aangetoond voor persoonsgericht leidinggeven. De VRSJT vertoonde geen relatie met cognitieve capaciteiten en vertoonde een onverwachte negatieve relatie met de persoonlijkheidsfactoren integriteit, extraversie en verdraagzaamheid. Tot slot werd gevonden dat deelnemers de VR-SJT ervaren als leuk, duidelijk, realistisch, modern en relevant. De video-SJT, die op deze aspecten eveneens positief werd beoordeeld, werd door de VR-SJT overtroffen op de aspecten ‘leuk’ en ‘modern’.

Loading

Article metrics loading...

/content/journals/10.5117/GO2021.1.006.RIJS
2021-02-01
2024-03-29
Loading full text...

Full text loading...

/deliver/fulltext/09215077/34/1/GO2021.1.006.RIJS.html?itemId=/content/journals/10.5117/GO2021.1.006.RIJS&mimeType=html&fmt=ahah

References

  1. Aguinis, H., Henle, C. A., & Beaty Jr, J. C.(2001). Virtual reality technology: A new tool for personnel selection. International Journal of Selection and Assessment, 9, 70-83. doi:10.1111/1468-2389.00164
    [Google Scholar]
  2. Anderson, N., Salgado, J. F., & Hülsheger, U. R.(2010). Applicant reactions in selection: Comprehensive meta-analysis into reaction generalization versus situational specificity. International Journal of Selection and Assessment, 18, 291-304. doi:10.1111/j.1468-2389.2010.00512.x
    [Google Scholar]
  3. Bailenson, J.(2018). Experience on demand: What virtual reality is, how it works, and what it can do. Retrieved from https://books.google.nl/books?hl=nl&lr=&id=2fkqDwAAQBAJ
    [Google Scholar]
  4. Campion, M. C., Ployhart, R. E., & MacKenzie Jr, W. I.(2014). The state of research on situational judgment tests: A content analysis and directions for future research. Human Performance, 27, 283-310. doi:10.1080/08959285.2014.929693
    [Google Scholar]
  5. Catano, V. M., Brochu, A., & Lamerson, C. D.(2012). Assessing the reliability of situational judgment tests used in high-stakes situations. International Journal of Selection and Assessment, 20, 333-346. doi:10.1111/j.1468-2389.2012.00604.x
    [Google Scholar]
  6. Chan, D., & Schmitt, N.(1997). Video-based versus paper-and-pencil method of assessment in situational judgment tests: Subgroup differences in test performance and face validity perceptions. Journal of Applied Psychology, 82, 143-159. doi:10.1037/0021-9010.82.1.143
    [Google Scholar]
  7. Christian, M. S., Edwards, B. D., & Bradley, J. C.(2010). Situational judgment tests: Constructs assessed and a meta-analysis of their criterion-related validities. Personnel Psychology, 63, 83-117. doi:10.1111/j.1744-6570.2009.01163.x
    [Google Scholar]
  8. Cook, M.(2016). Personnel selection: Adding value through people – A changing picture (6th ed.). Chichester, UK: Wiley-Blackwell
    [Google Scholar]
  9. De Meijer, L. A., Born, M. P., Van Zielst, J., & Van der Molen, H. T.(2010). Constructdriven development of a video-based situational judgment test for integrity. European Psychologist, 15, 229-236. doi:10.1027/1016-9040/a000027
    [Google Scholar]
  10. De Leng, W. E., Stegers-Jager, K. M., Born, M. P., & Themmen, A. P.(2018). Integrity situational judgement test for medical school selection: Judging ‘what to do’ versus ‘what not to do’. Medical Education, 52, 427-437. doi:10.1111/medu.13498
    [Google Scholar]
  11. De Leng, W. E., Stegers-Jager, K. M., Husbands, A., Dowell, J. S., Born, M. P., & Themmen, A. P. N.(2017). Scoring method of a situational judgment test: Influence on internal consistency reliability, adverse impact and correlation with personality?Advances in Health Sciences Education, 22, 243-265. doi:10.1007/s10459-016-9720-7
    [Google Scholar]
  12. Derue, D. S., Nahrgang, J. D., Wellman, N. E. D., & Humphrey, S. E.(2011). Trait and behavioral theories of leadership: An integration and meta-analytic test of their relative validity. Personnel Psychology, 64, 7-52. doi:10.1111/j.1744-6570.2010.01201.x
    [Google Scholar]
  13. De Vries, R. E.(2012). Personality predictors of leadership styles and the selfother agreement problem. The Leadership Quarterly, 23, 809-821. doi:10.1016/j.leaqua.2012.03.002
    [Google Scholar]
  14. De Vries, R. E., Ashton, M. C., & Lee, K.(2009). De zes belangrijkste persoonlijkheidsdimensies en de HEXACO persoonlijkheidsvragenlijst. Gedrag & Organisatie, 22, 232-274.
    [Google Scholar]
  15. De Vries, R. E., Wawoe, K. W., & Holtrop, D.(2016). What is engagement? Proactivity as the missing link in the HEXACO model of personality. Journal of Personality, 84, 178-193. doi:10.1111/jopy.12150
    [Google Scholar]
  16. Fleishman, E. A.(1953). The description of supervisory behavior. Journal of Applied Psychology, 37, 1-6.
    [Google Scholar]
  17. Frauendorfer, D., Schmid Mast, M., Nguyen, L., & Gatica-Perez, D.(2014). Nonverbal social sensing in action: Unobtrusive recording and extracting of nonverbal behavior in social interactions illustrated with a research example. Journal of Nonverbal Behavior, 38, 231-245. doi:10.1007/s10919-014-0173-5
    [Google Scholar]
  18. Gilliland, S. W.(1993). The perceived fairness of selection systems: An organizational justice perspective. Academy of Management Review, 18, 694-734. doi:10.5465/amr.1993.9402210155
    [Google Scholar]
  19. Guenole, N., Chernyshenko, O. S., & Weekly, J.(2017). On designing construct driven situational judgment tests: Some preliminary recommendations. International Journal of Testing, 17, 234-252. doi:10.1080/15305058.2017.1297817
    [Google Scholar]
  20. Hausknecht, J. P., Day, D. V., & Thomas, S. C.(2004). Applicant reactions to selection procedures: An updated model and meta‐analysis. Personnel Psychology, 57, 639-683. doi:10.1111/j.1744-6570.2004.00003.x
    [Google Scholar]
  21. Hersey, P., & Blanchard, K. H.(1969). Life cycle theory of leadership. Training & Development Journal, 23(5), 26-34.
    [Google Scholar]
  22. Hersey, P., Blanchard, K. H., & Johnson, D. E.(2007). Management of organizational behavior (9th ed.). Upper Saddle River, NJ: Prentice hall.
    [Google Scholar]
  23. Hiemstra, A. M. F., Cassel, T, Born, M. Ph., & Liem, C. C. S.(2020). De (on)mogelijkheid van machine learning voor het verminderen van bias en discriminatie bij personeelsbeslissingen. Gedrag & Organisatie, 33, 279-299.
    [Google Scholar]
  24. Judge, T. A., Piccolo, R. F., & Ilies, R.(2004). The forgotten ones? The validity of consideration and initiating structure in leadership research. Journal of Applied Psychology, 89, 36-51. doi:10.1037/0021-9010.89.1.36
    [Google Scholar]
  25. Kanning, U. P., Grewe, K., Hollenberg, S., & Hadouch, M.(2006). From the subjects’ point of view: Reactions to different types of situational judgment items. European Journal of Psychological Assessment, 22, 168-176. doi:10.1027/1015-5759.22.3.168
    [Google Scholar]
  26. Langer, M., Konig, C. J., & Krause, K.(2017). Examining digital interviews for personnel selection: Applicant reactions and interviewer ratings. International Journal of Selection and Assessment, 25, 371-382. doi:10.1111/ijsa.12191
    [Google Scholar]
  27. Liem, C. C., Langer, M., Demetriou, A., Hiemstra, A. M., Wicaksana, A. S., Born, M. P., & König, C. J.(2018). Psychology meets machine learning: Interdisciplinary perspectives on algorithmic job candidate screening. In J.Escalante, S.Escalera, I.Guyon, X.Baró, Y.Güçlütürk, U.Güçlü, & M.van Gerven (Eds.), Explainable and interpretable models in computer vision and machine learning (pp. 197-253). doi:10.1007/978-3-319-98131-4
    [Google Scholar]
  28. Lievens, F.(2017). Construct-driven SJT’s: Toward an agenda for future research. International Journal of Testing, 17, 269-276. doi:10.1080/15305058.2017.1309857
    [Google Scholar]
  29. Lievens, F., & Coetsier, P.(2002). Situational tests in student selection: An examination of predictive validity, adverse impact, and construct validity. International Journal of Selection and Assessment, 10, 245-257. doi:10.1111/1468-2389.00215
    [Google Scholar]
  30. Lievens, F., & De Soete, B.(2011). Instrumenten om personeel te selecteren in de 21ste eeuw: Onderzoek en praktijk. Gedrag & Organisatie, 24, 18-42.
    [Google Scholar]
  31. Lievens, F., & Motowidlo, S. J.(2016). Situational judgment tests: From measures of situational judgment to measures of general domain knowledge. Industrial and Organizational Psychology, 9, 3-22. doi:10.1017/iop.2015.71
    [Google Scholar]
  32. Lievens, F., Peeters, H., & Schollaert, E.(2008). Situational judgment tests: A review of recent research. Personnel Review, 37, 426-441. doi:10.1108/00483480810877598
    [Google Scholar]
  33. Lievens, F., & Sackett, P. R.(2006). Video-based versus written situational judgment tests: A comparison in terms of predictive validity. Journal of Applied Psychology, 91, 1181-1188. doi:10.1037/0021-9010.91.5.1181
    [Google Scholar]
  34. McCarthy, J. M., Bauer, T. N., Truxillo, D. M., Anderson, N. R., Costa, A. C., & Ahmed, S. M.(2017). Applicant perspectives during selection: A review addressing “So what?,” “What’s new?,” and “Where to next?”. Journal of Management, 43, 1693-1725. doi:10.1177%2F0149206316681846
    [Google Scholar]
  35. McDaniel, M. A., Hartman, N. S., Whetzel, D. L., & Grubb III, W. L.(2007). Situational judgment tests, response instructions, and validity: A meta-analysis. Personnel Psychology, 60, 63-91. doi:10.1111/j.1744-6570.2007.00065.x
    [Google Scholar]
  36. McDaniel, M. A., List, S. K., & Kepes, S.(2016). The “hot mess” of situational judgment test construct validity and other issues. Industrial and Organizational Psychology, 9, 47-51. doi:10.1017/iop.2015.115
    [Google Scholar]
  37. Motowidlo, S. J., Dunnette, M. D., & Carter, G. W.(1990). An alternative selection procedure: The low-fidelity simulation. Journal of Applied Psychology, 75, 640-647. doi:10.1037/0021-9010.75.6.640
    [Google Scholar]
  38. Mroz, J. E., Allen, J. A., Verhoeven, D. C., & Shuffler, M. L.(2018). Do we really need another meeting? The science of workplace meetings. Current Directions in Psychological Science, 27, 484-491. doi:10.1177/0963721418776307
    [Google Scholar]
  39. Muñoz-Saavedra, L., Miró-Amarante, L., & Domínguez-Morales, M.(2020). Augmented and virtual reality evolution and future tendency. Applied Sciences, 10(1), 322. doi:10.3390/app10010322
    [Google Scholar]
  40. Odermatt, I., König, C. J., Kleinmann, M., Nussbaumer, R., Rosenbaum, A., Olien, J. L., & Rogelberg, S. G.(2017). On leading meetings: Linking meeting outcomes to leadership styles. Journal of Leadership & Organizational Studies, 24, 189-200. doi:10.1177/1548051816655992
    [Google Scholar]
  41. Olaru, G., Burrus, J., MacCann, C., Zaromb, F. M., Wilhelm, O., & Roberts, R. D.(2019). Situational Judgment Tests as a method for measuring personality: Development and validity evidence for a test of dependability. PloS one, 14(2), 1-19. doi:10.1371/journal.pone.0211884
    [Google Scholar]
  42. Olson-Buchanan, J. B., & Drasgow, F.(2006). Multimedia situational judgment tests: The medium creates the message. In J. A.Weekley & R. E.Ployhart (Eds.), Situational judgment tests: Theory, measurement, and application (pp. 253-278). Retrieved from https://books.google.nl/books?id=vz4zjg5jzTIC&pg
    [Google Scholar]
  43. Oostrom, J. K., Born, M. P., Serlie, A. W., & Van der Molen, H. T.(2011). A multimedia situational test with a constructed-response format. Journal of Personnel Psychology, 10, 78-88. doi:10.1027/1866-5888/a000035
    [Google Scholar]
  44. Oostrom, J. K., Born, M. P., Serlie, A. W., & Van der Molen, H. T.(2012). Implicit trait policies in multimedia situational judgment tests for leadership skills: Can they predict leadership behavior?Human Performance, 25, 335-353. doi:10.1080/08959285.2012.703732
    [Google Scholar]
  45. Oostrom, J. K., De Soete, B., & Lievens, F.(2015). Situational judgment testing: A review and some new developments. In I.Nikolaou & J. K.Oostrom (Eds.), Employee recruitment, selection, and assessment: Contemporary issues for theory and practice (pp. 172-189). Retrieved from https://books.google.nl/books?hl=nl&lr=&id=FGtKCAAAQBAJ&oi
    [Google Scholar]
  46. Oostrom, J. K., De Vries, R. E., & De Wit, M.(2019). Development and validation of a HEXACO situational judgment test. Human Performance, 32, 1-29. doi:10.1080/08959285.2018.153985
    [Google Scholar]
  47. Op de Beek, M. R., Oostrom, J. K., & Born, M. P.(2011). De webcamtest als voorspeller van professioneel gedrag. Gedrag & Organisatie, 24, 257-285.
    [Google Scholar]
  48. Piccolo, R. F., Bono, J. E., Heinitz, K., Rowold, J., Duehr, E., & Judge, T. A.(2012). The relative impact of complementary leader behaviors: Which matter most?The Leadership Quarterly, 23, 567-581. doi:10.1016/j.leaqua.2011.12.008
    [Google Scholar]
  49. Richman-Hirsch, W. L., Olson-Buchanan, J. B., & Drasgow, F.(2000). Examining the impact of administration medium on examinee perceptions and attitudes. Journal of Applied Psychology, 85, 880-887. doi:10.1037/0021-9010.85.6.880
    [Google Scholar]
  50. Ryan, A. M., & Derous, E.(2019). The unrealized potential of technology in selection assessment. Journal of Workand Organizational Psychology, 35, 85-92. doi:10.5093/jwop2019a10
    [Google Scholar]
  51. Sharma, S., Gangopadhyay, M., Austin, E., & Mandal, M. K.(2013). Development and validation of a situational judgment test of emotional intelligence. International Journal of Selection and Assessment, 21, 57-73. doi:10.1111/ijsa.12017
    [Google Scholar]
  52. Steiner, D. D., & Gilliland, S. W.(1996). Fairness reactions to personnel selection techniques in France and the United States. Journal of Applied Psychology, 81, 134-141. doi:10.1037/0021-9010.81.2.134
    [Google Scholar]
  53. Thompson, G., & Glasø, L.(2015). Situational leadership theory: A test from three perspectives. Leadership & Organization Development Journal, 36, 527-544. doi:10.1108/LODJ-10-2013-0130
    [Google Scholar]
  54. Thompson, G., & Vecchio, R. P.(2009). Situational leadership theory: A test of three versions. The Leadership Quarterly, 20, 837-848. doi:10.1016/j.leaqua.2009.06.014
    [Google Scholar]
  55. Tiffin, P. A., Paton, L. W., O’Mara, D., MacCann, C., Lang, J. W., & Lievens, F.(2020). Situational judgement tests for selection: Traditional vs construct-driven approaches. Medical Education, 54, 105-115. doi:10.1111/medu.14011
    [Google Scholar]
  56. Van der Maesen | Koch HRM-advies.(2011a). Handleiding SQ-leidinggeven. Retrieved from www.t-station.nl/upload/files/handleiding_sqleidinggeven_september_2011.pdf
    [Google Scholar]
  57. Van der Maesen | Koch HRM-advies.(2011b). Internet cognitieve capaciteitentest hoog niveau. Retrieved from van www.t-station.nl/internet-cognitieve-capaciteiten-test-hoog-icct-hv-nl-e-116.html
    [Google Scholar]
  58. Van Gelder, J. L., Martin, C., Van Prooijen, J. W., De Vries, R., Marsman, M., Averdijk, M., ... Donker, T.(2018). Seeing is believing? Comparing negative affect, realism and presence in visual versus written guardianship scenarios. Deviant Behavior, 39, 461-474. doi:10.1080/01639625.2017.1407106
    [Google Scholar]
  59. Weekley, J. A., & Jones, C.(1997). Video-based situational testing. Personnel Psychology, 50, 25-49.
    [Google Scholar]
  60. Weiner, E. J., & Sanchez, D. R.(2020). Cognitive ability in virtual reality: Validity evidence for VR game-based assessments. International Journal of Selection and Assessment, 28, 215-235. doi:10.1111/ijsa.12295
    [Google Scholar]
  61. Whetzel, D. L., & McDaniel, M. A.(2009). Situational judgment tests: An overview of current research. Human Resource Management Review, 19, 188-202. doi:10.1016/j.hrmr.2009.03.007
    [Google Scholar]
  62. Woods, S. A., Ahmed, S., Nikolaou, I., Costa, A. C., & Anderson, N. R.(2020). Personnel selection in the digital age: A review of validity and applicant reactions, and future research challenges. European Journal of Work and Organizational Psychology, 29, 64-77. doi:10.1080/1359432X.2019.1681401
    [Google Scholar]
http://instance.metastore.ingenta.com/content/journals/10.5117/GO2021.1.006.RIJS
Loading
/content/journals/10.5117/GO2021.1.006.RIJS
Loading

Data & Media loading...

Keyword(s): multimedia testing; personnel selection; situational judgment test; technology; virtual reality

Most Cited Most Cited RSS feed