Content validity of the Virtual Neurocognitive Tracking Test for child population

Gloria Andrea Rico Velasco, Andres Grisales, Juan Toro Arias, Paola Benavides

Abstract


Objective: This research aims to validate the content of the Screening Test (PRANI) for children, focusing on the development of cognitive tests tailored to the Spanish-speaking population in Latin America.


Method: In this non-experimental, instrumental study, the psychometric properties of a new cognitive test were analyzed, focusing on theoretical validation rather than direct child population application. A panel of nine Child Neuropsychology experts assessed 149 items of a virtual neuropsychological test across five cognitive processes. They used a Likert-type scale for evaluation, focusing on criteria like relevance and writing quality. Content validity was established using the Aiken V coefficient and confidence intervals, emphasizing expert consensus in test development and refinement.


Results: The study resulted in the consolidation of an instrument, reducing the test length from 149 to 119 items by eliminating 30 elements. Significant variations were observed between the initial version of the instrument and the refined version after the expert ratings analysis, which evaluated cognitive processes such as attention, memory, language, visuoconstructive skills, executive functions, and academic skills. A comprehensive assessment of the instrument's content validity was obtained before and after the judges' analysis. Notably, the initial mean values of the Aiken V coefficients were high, both globally and by process and component.


Conclusions: This study highlights the importance and efficacy of developing culturally adapted cognitive assessment tools for children in Latin America, particularly in the Colombian child population. The findings emphasize the necessity for specific cognitive tests for this demographic, providing a reliable and validated framework for their implementation.


Keywords


Psychometrics; Cognitive function; Clinical Psychology; Preschool children; Screening; instrument validation.

Full Text:

PDF

References


Alcauter, S., Lin, W., Smith, J. K., Short, S. J., Goldman, B. D., Reznick, J. S., Gilmore, J. H., & Gao, W. (2014). Development of thalamocortical connectivity during infancy and its cognitive correlations. The Journal of Neuroscience: The Official Journal of the Society for Neuroscience, 34(27), 9067–9075. https://doi.org/10.1523/jneurosci.0796-14.2014

Alex, A. M., Aguate, F., Botteron, K., Buss, C., Chong, Y.-S., Dager, S. R., Donald, K. A., Entringer, S., Fair, D. A., Fortier, M. V., Gaab, N., Gilmore, J. H., Girault, J. B., Graham, A. M., Groenewold, N. A., Hazlett, H., Lin, W., Meaney, M. J., Piven, J., … the ENIGMA ORIGINs group. (2023). A global multicohort study to map subcortical brain development and cognition in infancy and early childhood. Nature Neuroscience, 1-11. https://doi.org/10.1038/s41593-023-01501-6

Anderson, P. (2002). Assessment and Development of Executive Function (EF) During Childhood. Child Neuropsychology, 8(2), 71–82. https://doi.org/10.1076/chin.8.2.71.8724

Anderson, P. J., & Burnett, A. (2017). Assessing developmental delay in early childhood -concerns with the Bayley-III scales. Clinical Neuropsychologist, 31(2). https://doi.org/10.1080/13854046.2016.1216518

Aiken, L. (1980). Content Validity and Reliability of Single Items or Questionnaire. Educational and Psychological Measurement, 40(4), 955-959. https://doi.org/10.1177/001316448004000419

Aiken, L. (1985). Three Coefficients for Analyzing the Reliability and Validity of Ratings. Educational and Psychological Measurement, 45(1), 131-142. https://doi.org/10.1177/0013164485451012

AERA, APA, & NCME. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.

Ato, M., López, J. J., & Benavente, A. (2013). Un sistema de clasificación de los diseños de investigación en psicología. Anales de Psicologia, 29(3). https://doi.org/10.6018/analesps.29.3.178511

Aygun, B., Parker, J., Freeman, M. B., Stephens, A. L., Smeltzer, M. P., Wu, S., Hankins, J. S., & Wang, W. C. (2011). Neurocognitive screening with the Brigance Preschool screen-II in 3-year-old children with sickle cell disease. In Pediatric Blood and Cancer, 56(4). https://doi.org/10.1002/pbc.22833

Bayley, P. J., & Squire, L. R. (2007). The Neuroanatomy and Neuropsychology of Declarative and Nondeclarative Memory. https://doi.org/10.1007/978-3-540-45702-2_1

Benítez, M. T., González, L., Méndez, S., Mora, C., Ordoñez, A., Pedrero, E. J., & Rodríguez, R. (2018). Uso y abuso de las Tecnologías de la Información y la Comunicación por adolescentes. Revista Del Centro Reina Sofía Sobre Adolescencia y Juventud, 2(9).

Buelow, J. M., Perkins, S. M., Johnson, C. S., Byars, A. W., Fastenau, P. S., Dunn, D. W., & Austin, J. K. (2012). Adaptive functioning in children with epilepsy and learning problems. Journal of Child Neurology, 27(10). https://doi.org/10.1177/0883073811432750

Butterworth, G. (2014). Principles Of Developmental Psychology. In Principles of Developmental Psychology. https://doi.org/10.4324/9781315804675

Casarin, F. S., Wong, C. E. I., Parente, M. A. de M. P., de Salles, J. F., & Fonseca, R. P. (2012). Comparison of Neuropsychological Performance between Students from Public and Private Brazilian Schools. The Spanish Journal of Psychology, 15(3). https://doi.org/10.5209/rev_sjop.2012.v15.n3.39386

Daraki, V., Roumeliotaki, T., Koutra, K., Georgiou, V., Kampouri, M., Kyriklaki, A., Vafeiadi, M., Papavasiliou, S., Kogevinas, M., & Chatzi, L. (2017). Effect of parental obesity and gestational diabetes on child neuropsychological and behavioral development at 4 years of age: the Rhea mother–child cohort, Crete, Greece. European Child and AdolescentPsychiatry, 26(6). https://doi.org/10.1007/s00787-016-0934-2

De Mello, C. B., Brunoni, L. R. R., Pilla, A. L., Taddei, J. A. A. C., Barbosa, T., Sinnes, E. G., Rodrigues, C. C., Miranda, M. C., Muzskat, M., & Bueno, O. F. A. (2012). A model for pediatric and neuropsychological screening assessment of children with learning disabilities. Dementia e Neuropsychologia, 6(1). https://doi.org/10.1590/s1980-57642012dn06010004

Delgado-Rico, E.; Carretero-Dios, H. & Ruch, W. (2012). Content validity evidences in test development: an applied perspective. International Journal of Clinical and Health Psychology España, 12(3):449-460. https://doi.org/10.5167/uzh-64551

Diniz, A., Pocinho, M. D., & Almeida, L. S. (2011). Cognitive abilities, sociocultural background and academic achievement. Psicothema, 23(4).

Eslinger, P. J., Marlowe, W. B., & Biddle, K. R. (2022). Development of Prefrontal Cortex. En Encyclopedia of Behavioral Neuroscience, 2nd edition (pp. 410–417). Elsevier. https://doi.org/10.1016/B978-0-12-819641-0.00071-2

Espino, Y. C., Vázquez-Justo, E., Fernandes, S., & Rocha, N. S. (2019). Uso de estrategias visuales y verbales en adultos con quejas de memoria. Neuropsicologia Latinoamericana, 11(2).

Gómez, F. M. (2004). Early experiences and environment: their impact on neurodevelopment and the creation of mental processes. Vertex (Buenos Aires, Argentina), 15(56), 105–114.

García Molina, A., Enseñat Cantallops, A., Tirapu Ustárroz, J., & Roig Rovira, T. (2009). Maduración de la corteza prefrontal y desarrollo de las funciones ejecutivas durante los primeros cinco años de vida. Revista de Neurología, 48(08). https://doi.org/10.33588/rn.4808.2008265

Groppell, S., Soto-Ruiz, K. M., Flores, B., Dawkins, W., Smith, I., Eagleman, D. M., & Katz, Y. (2019). A rapid, mobile neurocognitive screening test to aid in identifying cognitive impairment and dementia (braincheck): Cohort study. JMIR Aging, 2(1). https://doi.org/10.2196/12615

Hambleton, R. K. (1980). Test score validity and standard setting methods. En R. A. Berk (Ed.), Criterion-Referenced Measurement: The State of the Art. Baltimore: Johns Hopkins University Press.

Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel Psychology, 28,563–575. https://doi.org/10.1111/j.1744-6570.1975.tb01393.x

Lipsitt, L. P., & Lewis P. Lipsitt. (2019). Learning and Memory in Infants. Merrill-Palmer Quarterly, 36(1).

Mas Salguero, M. J. (2019). Detección de trastornos del neurodesarrollo en la consulta de Atención Primaria. Congreso De ActualizacionPediatria 2019, 3(0).

Medina Alva, M. del P., Caro Kahn, I., Muñoz Huerta, P., Leyva Sánchez, J., Moreno Calixto, J., & Vega Sánchez, S. M. (2015). Neurodesarrollo infantil: características normales y signos de alarma en el niño menor de cinco años. Revista Peruana de Medicina Experimental y Salud Pública, 32(3). https://doi.org/10.17843/rpmesp.2015.323.1693

Mesulam, M.-M. (2000). Attentional networks, confusional states, and neglect syndromes. In Principles of behavioral and cognitive neurology (2nd ed.) (pp. 149-173). Oxford University Press. https://doi.org/10.1093/oso/9780195134759.003.0003

Merino-Soto, C., & Livia-Segovia, J. (2009). Intervalos de confianza asimétricos para el índice la validez de contenido: un programa Visual Basic para la V de Aiken. Anales de Psicología / Annals of Psychology, 25(1), 169–171. Retrieved from: https://revistas.um.es/analesps/article/view/71631

Onnis, L. (2017). Caregiver communication to the child as moderator and mediator of genes for language. Behavioural Brain Research, 325. https://doi.org/10.1016/j.bbr.2017.02.003

Onnis, L., Truzzi, A., & Ma, X. (2018). Language development and disorders: Possible genes and environment interactions. Research in Developmental Disabilities, 82. https://doi.org/10.1016/j.ridd.2018.06.015

Penfield, R. D., & Giacobbi, Jr., P. R. (2004). Applying a Score Confidence Interval to Aiken’s Item Content-Relevance Index. Measurement in Physical Education and Exercise Science, 8(4), 213–225. https://doi.org/10.1207/s15327841mpee0804_3

Peralta-Cuji, I. J., Cobos-Cali, M. E., & Ochoa-Arévalo, V. F. (2021). Systematic review of the literature on child neuropsychological evaluation in Ecuador. Revista Ecuatoriana de Neurologia, 30(1), 125–134. https://doi.org/10.46997/REVECUATNEUROL30100125

Pedrosa, I.; Suárez-Álvarez, J.; & García-Cueto, E. (2013). Evidencias sobre la validez de contenido: avances teóricos y métodos para su estimación. Acción Psicológica, 10(2), 3-18. http://dx.doi.org/10.5944/ap.10.2.11820

Penfield, R. & Giacobbi, P. (2004). Applying a Score Confidence Interval to Aiken's Item Content-Relevance Index. Measurement in Physical Education and Exercise Science, 8(4), 213-225. https://doi.org/10.1207/s15327841mpee0804_3

Qin, S., Cho, S., Chen, T., Rosenberg-Lee, M., Geary, D. C., & Menon, V. (2014). Hippocampal-neocortical functional reorganization underlies children’s cognitive development. Nature Neuroscience, 17(9), 1263–1269. https://doi.org/10.1038/nn.3788

Rasgado-Toledo, J., Lizcano-Cortés, F., Olalde-Mathieu, V. E., Licea-Haquet, G., Zamora-Ursulo, M. A., Giordano, M., & Reyes-Aguilar, A. (2021). A Dataset to Study Pragmatic Language and Its Underlying Cognitive Processes. Frontiers in Human Neuroscience, 15. https://doi.org/10.3389/fnhum.2021.666210

Resolución Número 8430 de 1993 del Ministerio de Salud, por la cual se establecen las normas científicas, técnicas y administrativas para la investigación en salud (4 de octubre de 1993). https://www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/DE/DIJ/RESOLUCION-8430-DE-1993.PDF

Rico, G.A., Benavides, G.P. & Utria, O.E. (2021). Diseño y validación de contenido de una prueba de tamizaje neuropsicológico digital para niños entre 6 a 7 años. Tesis Psicológica, 16(2), 48-66. https://doi.org/10.37511/tesis.v16n1a3pp. 48-66

Riggins, T., Geng, F., Botdorf, M., Canada, K., Cox, L., & Hancock, G. R. (2018). Protracted hippocampal development is associated with age-related improvements in memory during early childhood. NeuroImage, 174. https://doi.org/10.1016/j.neuroimage.2018.03.009

Romero-Ayuso, D., Toledano-González, A., Segura-Fragoso, A., Triviño-Juárez, J. M., & Rodríguez-Martínez, M. C. (2020). Assessment of sensory processing and executive functions at the school: Development, reliability, and validity of epyfei-escolar. Frontiers in Pediatrics, 8. https://doi.org/10.3389/fped.2020.00275

Rosselli Cock, M., & Ardila, A. (2016). Historia de la neuropsicología infantil. EduPsykhé: Revista de Psicología y Psicopedagogía, 15(1). https://doi.org/10.57087/edupsykhe.v15i1.3904

Rosselli, M. (2015). Desarrollo Neuropsicológico de las Habilidades Visoespaciales y Visoconstruccionales. Revista Neuropsicología, Neuropsiquiatría y Neurociencias, Enero-Diciembre, 15.

Shah, M., Shandilya, A., Patel, K., Mehta, M., Sanghavi, J., & Pandya, A. (2023). Neuropsychological detection and prediction using machine learning algorithms: a comprehensive review. Intelligent Medicine. https://doi.org/10.1016/j.imed.2023.04.003

Shaw, G. A., Dupree, J. L., & Neigh, G. N. (2020). Adolescent maturation of the prefrontal cortex: Role of stress and sex in shaping adult risk for compromise. Genes, Brain and Behavior, 19(3), e12626. https://doi.org/10.1111/gbb.12626

Silva, P. A., Williams, S., & McGee, R. (1987). A Longitudinal Study of Children With Developmental Language Delay At Age Three: Later Intelligence, Reading And Behaviour Problems. Developmental Medicine y Child Neurology, 29(5). https://doi.org/10.1111/j.1469-8749.1987.tb08505.x

Sireci, S., & Faulkner-Bond, M. (2014). Validity evidence based on test content. Psicothema, 26(1), 100–107. https://doi.org/10.7334/psicothema2013.256

Sorge, A., Balzarotti, S., Ruzzon, D., Burdese, C., & Saita, E. (2023). Understanding the impact of prison design on prisoners and prison staff through virtual reality: a multi-method approach. Mediterranean Journal of Clinical Psychology, 11(2). https://doi.org/10.13129/2282-1619/mjcp-3808

Takio, F., Koivisto, M., & Hämäläinen, H. (2014). The influence of executive functions on spatial biases varies during the lifespan. Developmental Cognitive Neuroscience, 10, 170–180. https://doi.org/10.1016/j.dcn.2014.09.004

Tipper, S. P., Bourque, T. A., Anderson, S. H., & Brehaut, J. C. (1989). Mechanisms of attention: A developmental study. Journal of Experimental Child Psychology, 48(3). https://doi.org/10.1016/0022-0965(89)90047-7

Tomai, M., Langher, V., Martino, G., Esposito, F., Ricci, M. E., & Caputo, A. (2017). Promoting the development of children with disabilities through school inclusion: clinical psychology in supporting teachers in Mozambique. Mediterranean Journal of Clinical Psychology, 5(3). https://doi.org/10.6092/2282-1619/2017.5.1671

Urbina, S. (2014). Essentials of Psychological Testing (Second Edition). New Jersey: Wiliey & Sons.

Wolf, K., y Pfeiffer, T. (2014). The development of attentional resolution. Cognitive Development, 29(1). https://doi.org/10.1016/j.cogdev.2013.09.004

Woodburn, M., Bricken, C. L., Wu, Z., Li, G., Wang, L., Lin, W., Sheridan, M. A., & Cohen, J. R. (2021). The maturation and cognitive relevance of structural brain network organization from early infancy to childhood. NeuroImage, 238(118232), 118232.https://doi.org/10.1016/j.neuroimage.2021.118232

Woodward, L. J., Lu, Z., Morris, A. R., & Healey, D. M. (2017). Preschool self-regulation predicts later mental health and educational achievement in very preterm and typically developing children. Clinical Neuropsychologist, 31(2). https://doi.org/10.1080/13854046.2016.1251614




DOI: https://doi.org/10.13129/2282-1619/mjcp-3934

Refbacks

  • There are currently no refbacks.