(Post last updated July 21, 2021)
Review panel summary
The Redox Concept Inventory (ROXCI) is an 18-item, multiple-choice instrument designed to assess students' conception related to oxidation-reduction processes within the domain of general chemistry instruction at tertiary level institutions in the United States. The item distractors are designed based on students' alternative conceptions identified by the developers. Every item on the ROXCI is accompanied by a confidence tier, allowing the administrator to measure how confident respondents are in their answers. Expert judgment from graduate students and faculty was used both in the development and refinement of the ROXCI items [1] to provide evidence in support of test content validity. Semi-structure interviews with students were used as evidence for response process validity [1]. There is some evidence that increased ROXCI scores are associated with more instructional exposure as supported by increased scores for students in the second semester compared to the first semester. Therefore, it is reasonable to use ROXCI to differentiate between groups with different levels of exposure to general chemistry instruction. Detailed investigations of the relations between confidence and content-based scores within the ROXCI measures were conducted [1, 2], which provides some validity support with respect to relations with other variables. Evidence for internal structure derived from CFA loading exists for a shortened, 14-item version of the instrument [3]. However, the ROXCI developers [1, 2] made explicit claims that factor analysis would not be appropriate to make claims on the internal structure of the instrument due to the nature and format of the instrument and concept inventories in general. Evidence based on single administration reliability of data produced by ROXCI is supported by coefficient alpha, McDonald's Omega and Coefficient H [1-3], whereas evidence based on test-retest reliability is supported by Spearman rank correlation coefficient. Psychometric characteristics of items, both from classical test theory and Rasch analysis, are available [1,3].
Recommendations for use
The ROXCI instrument can be used to identify the prevalence of students’ misconceptions related to oxidation-reduction processes within the instructional domain of general chemistry. If students’ reported confidence is deemed important, the ROXCI items can be administered alongside the confidence tier. If the unidimensionality of the instrument is deemed important, a shortened, 14-item version of the instrument [3] can be administered after reviewing omitted items for coverage of instructionally important concepts. While the instrument was developed with a confidence tier accompanying every question, the panel concluded that the instrument can be used without the confidence tier if confidence levels are outside of the interest of potential adopters. The panel evaluated evidence for validity based on test content, response process, relations to other variables, and internal structure and found it sufficient. However, due to the nature of cross-sectional data collected for general chemistry 1 and 2, the evidence that students’ ROXCI score will increase as a result of increased instructional exposure was discussed by the panel as being weak. The panel also evaluated evidence for reliability based on single administration reliability and in test-retest conditions and found it sufficient.
Details from panel review
The panel agreed that there is sufficient evidence presented for test content validity, response process validity, and some evidence based on the relation to other variables. The construct of redox reactions pertaining to general chemistry instruction is adequately represented with ROXCI items; however this construct will manifest differently when moving to the organic chemistry domain. Therefore, the panel cautions potential adopters who are planning to use ROXCI for subjects with substantial levels of instructional exposure to organic chemistry. Claims about internal structure are justified and a suitable solution for 1-factor CFA was found by eliminating items with lower loadings [3]; however, this approach may lead to construct underrepresentation.
References
[1] Brandriet, A. R., & Bretz, S. L. (2014). The Development of the Redox Concept Inventory as a Measure of Students’ Symbolic and Particulate Redox Understandings and Confidence. Journal of Chemical Education, 91(8), 1132–1144. https://doi.org/10.1021/ed500051n
[2] Brandriet, A. R., & Bretz, S. L. (2014). Measuring meta-ignorance through the lens of confidence: Examining students’ redox misconceptions about oxidation numbers, charge, and electron transfer. Chemistry Education Research and Practice, 15(4), 729–746. https://doi.org/10.1039/C4RP00129J
[3] Jin, Y., Rodriguez, C. A., Shah, L., & Rushton, G. T. (2020). Examining the Psychometric Properties of the Redox Concept Inventory: A Rasch Approach. Journal of Chemical Education, 97(12), 4235–4244. https://doi.org/10.1021/acs.jchemed.0c00479