RL-5: The Future Is Here: Analyzing NAEP Process Data Using R

Instructors:  

Emmanuel Sikali, U.S. Department of Education; (course co-director) 

Ruhan Circi, American Institutes for Research; (course co-director) 

Fusun Sahin, American Institutes for Research; 

Xiaying Zheng, American Institutes for Research;  

Juanita Hicks, American Institutes for Research;  

Soo Youn Lee, American Institutes for Research;  

Tiago A. Caliço, American Institutes for Research 

Fee: $35

This course will introduce the unique features of National Assessment of Educational Progress (NAEP) process data to researchers and provide necessary guidance on how to appropriately prepare and analyze this new data type. NAEP process data consist of time-stamped records of student actions (e.g., highlighter use, calculator use, answer selection, item navigation) within the test delivery system. These time-stamped records are collected at every part of the NAEP assessment (i.e., tutorial, cognitive section, and student questionnaire) across multiple subject areas (e.g., mathematics, reading, science). For this course, participants will use a NAEP mathematics process data file. For each student in this file, there are individual actions that the student made, while interacting with a) the items presented, b) the physical computer, and c) the testing environment.

 The course will provide participants with hands-on practice training in analyzing NAEP process data using the R statistical analysis language. Participants will learn how to create new variables from process data, such as item response time, and how to conduct analysis tailored to various research questions, through a mixture of instructors’ demonstrations of data analyses and visualization. The course is designed for advanced graduate students as well as researchers in academia, the private sector, government, and nonprofit organizations who are interested in learning how to integrate response process data into their research. Participants should have at least basic knowledge of the R programming language, as well as of statistical techniques including statistical inference and clustering. Participants need access to R and RStudio. To familiarize yourself with some aspects of the NAEP assessment and get a general idea of some of the process data elements that are collected during the assessment, please explore the NAEP Grade 8 Mathematics Tutorial using this link here. Detailed instructions will be sent to participants before the course.