Redox Concept Inventory

ROXCI

OVERVIEW

Overview

Listed below is general information about the instrument.

Summary
Original author(s)	Brandriet, A.R., & Bretz, S.L.
Original publication	Brandriet, A.R., & Bretz, S.L. (2014). The development of the redox concept inventory as a measure of students' symbolic and particulate Redox understandings and confidence. Journal of Chemical Education, 91(8), 1132-1144.
Year original instrument was published	2014
Inventory
Number of items	18
Number of versions/translations	1
Cited implementations	3
Language	English
Country	United States
Format	Multiple Choice Response Scale
Intended population(s)	Students Undergraduate
Domain	Cognitive
Topic	Redox

EVIDENCE

Evidence

The CHIRAL team carefully combs through every reference that cites this instrument and pulls all evidence that relates to the instruments’ validity and reliability. These data are presented in the following table that simply notes the presence or absence of evidence related to that concept, but does not indicate the quality of that evidence. Similarly, if evidence is lacking, that does not necessarily mean the instrument is “less valid,” just that it wasn’t presented in literature. Learn more about this process by viewing the CHIRAL Process and consult the instrument’s Review (next tab), if available, for better insights into the usability of this instrument.

Information in the table is given in four different categories:

General - information about how each article used the instrument:
- Original development paper - indicates whether in which paper(s) the instrument was developed initially
- Uses the instrument in data collection - indicates whether an article administered the instrument and collected responses
- Modified version of existing instrument - indicates whether an article has modified a prior version of this instrument
- Evaluation of existing instrument - indicates whether an article explicitly provides evidence that attempt to evaluate the performance of the instrument; lack of a checkmark here implies an article that administered the instrument but did not evaluate the instrument itself
Reliability - information about the evidence presented to establish reliability of data generated by the instrument; please see the Glossary for term definitions
Validity - information about the evidence presented to establish reliability of data generated by the instrument; please see the Glossary for term definitions
Other Information - information that may or may not directly relate to the evidence for validity and reliability, but are commonly reported when evaluating instruments; please see the Glossary for term definitions

Publications:	1	2	3
General
Original development paper	✔
Uses the instrument in data collection	✔	✔
Modified version of existing instrument
Evaluation of existing instrument	✔	✔	✔
Reliability
Test-retest reliability	✔
Internal consistency	✔	✔	✔
Coefficient (Cronbach's) alpha	✔	✔	✔
McDonald's Omega			✔
Inter-rater reliability
Person separation
Generalizability coefficients
Other reliability evidence	✔
Validity
Expert judgment	✔
Response process	✔
Factor analysis, IRT, Rasch analysis			✔
Differential item function
Evidence based on relationships to other variables	✔
Evidence based on consequences of testing
Other validity evidence
Other information
Difficulty	✔	✔	✔
Discrimination	✔
Evidence based on fairness
Other general evidence	✔		✔

REVIEW

Review

DISCLAIMER: The evidence supporting the validity and reliability of the data summarized below is for use of this assessment instrument within the reported settings and populations. The continued collection and evaluation of validity and reliability evidence, in both similar and dissimilar contexts, is encouraged and will support the chemistry education community’s ongoing understanding of this instrument and its limitations.
This review was generated by a CHIRAL review panel. Each CHIRAL review panel consists of multiple experts who first individually review the citations of the assessment instrument listed on this page for evidence in support of the validity and reliability of the data generated by the instrument. Panels then meet to discuss the evidence and summarize their opinions in the review posted in this tab. These reviews summarize only the evidence that was discussed during the panel which may not represent all evidence available in the published literature or that which appears on the Evidence tab.
If you feel that evidence is missing from this review, or that something was documented in error, please use the CHIRAL Feedback page.

Panel Review: Redox Concept Inventory

(Post last updated July 21, 2021)

Review panel summary

The Redox Concept Inventory (ROXCI) is an 18-item, multiple-choice instrument designed to assess students' conception related to oxidation-reduction processes within the domain of general chemistry instruction at tertiary level institutions in the United States. The item distractors are designed based on students' alternative conceptions identified by the developers. Every item on the ROXCI is accompanied by a confidence tier, allowing the administrator to measure how confident respondents are in their answers. Expert judgment from graduate students and faculty was used both in the development and refinement of the ROXCI items [1] to provide evidence in support of test content validity. Semi-structure interviews with students were used as evidence for response process validity [1]. There is some evidence that increased ROXCI scores are associated with more instructional exposure as supported by increased scores for students in the second semester compared to the first semester. Therefore, it is reasonable to use ROXCI to differentiate between groups with different levels of exposure to general chemistry instruction. Detailed investigations of the relations between confidence and content-based scores within the ROXCI measures were conducted [1, 2], which provides some validity support with respect to relations with other variables. Evidence for internal structure derived from CFA loading exists for a shortened, 14-item version of the instrument [3]. However, the ROXCI developers [1, 2] made explicit claims that factor analysis would not be appropriate to make claims on the internal structure of the instrument due to the nature and format of the instrument and concept inventories in general. Evidence based on single administration reliability of data produced by ROXCI is supported by coefficient alpha, McDonald's Omega and Coefficient H [1-3], whereas evidence based on test-retest reliability is supported by Spearman rank correlation coefficient. Psychometric characteristics of items, both from classical test theory and Rasch analysis, are available [1,3].

Recommendations for use

The ROXCI instrument can be used to identify the prevalence of students’ misconceptions related to oxidation-reduction processes within the instructional domain of general chemistry. If students’ reported confidence is deemed important, the ROXCI items can be administered alongside the confidence tier. If the unidimensionality of the instrument is deemed important, a shortened, 14-item version of the instrument [3] can be administered after reviewing omitted items for coverage of instructionally important concepts. While the instrument was developed with a confidence tier accompanying every question, the panel concluded that the instrument can be used without the confidence tier if confidence levels are outside of the interest of potential adopters. The panel evaluated evidence for validity based on test content, response process, relations to other variables, and internal structure and found it sufficient. However, due to the nature of cross-sectional data collected for general chemistry 1 and 2, the evidence that students’ ROXCI score will increase as a result of increased instructional exposure was discussed by the panel as being weak. The panel also evaluated evidence for reliability based on single administration reliability and in test-retest conditions and found it sufficient.

Details from panel review

The panel agreed that there is sufficient evidence presented for test content validity, response process validity, and some evidence based on the relation to other variables. The construct of redox reactions pertaining to general chemistry instruction is adequately represented with ROXCI items; however this construct will manifest differently when moving to the organic chemistry domain. Therefore, the panel cautions potential adopters who are planning to use ROXCI for subjects with substantial levels of instructional exposure to organic chemistry. Claims about internal structure are justified and a suitable solution for 1-factor CFA was found by eliminating items with lower loadings [3]; however, this approach may lead to construct underrepresentation.

References

[1] Brandriet, A. R., & Bretz, S. L. (2014). The Development of the Redox Concept Inventory as a Measure of Students’ Symbolic and Particulate Redox Understandings and Confidence. Journal of Chemical Education, 91(8), 1132–1144. https://doi.org/10.1021/ed500051n

[2] Brandriet, A. R., & Bretz, S. L. (2014). Measuring meta-ignorance through the lens of confidence: Examining students’ redox misconceptions about oxidation numbers, charge, and electron transfer. Chemistry Education Research and Practice, 15(4), 729–746. https://doi.org/10.1039/C4RP00129J

[3] Jin, Y., Rodriguez, C. A., Shah, L., & Rushton, G. T. (2020). Examining the Psychometric Properties of the Redox Concept Inventory: A Rasch Approach. Journal of Chemical Education, 97(12), 4235–4244. https://doi.org/10.1021/acs.jchemed.0c00479

Redox Concept Inventory

General

Reliability

Validity

Other information

Panel Review: Redox Concept Inventory

CHIRAL

ChemEd X

NSF