(Post last updated June 23, 2022)
Review panel summary
The College Chemistry Self-Efficacy Scale (CCSS) is a 21-item, Likert-scale instrument designed to measure university students’ self-efficacy beliefs regarding their ability to perform activities/tasks required in their chemistry courses. The instrument comprises three subscales - self-efficacy for cognitive skills, self-efficacy for psychomotor skills, and self-efficacy for everyday applications. It has been evaluated with students enrolled in first-year undergraduate chemistry courses in Turkey [1], South Africa [2], and the United States [3]. Originally developed for the assessment of students enrolled at an English-speaking university in Turkey, test content validity was provided by experts in chemistry, chemistry education, educational measurement, and English language for appropriateness and clarity of items [1]. The developers provide evidence of internal structure validity through exploratory factor analysis [1]; the three-factor structure has been replicated in a second context, though some items were removed from this analysis [2]. In multiple studies, evidence based on relations to other variables is provided by the relation between CCSS scores and subscale scores and course performance, which is supported by theory and prior research [1-3]. In one context, chemistry majors were observed to have higher CCSS scores than non-majors [1]; therefore, the CCSS may be able to differentiate between major and non-major students, though this has not been demonstrated beyond the development context. In multiple studies, coefficient alpha has been used to estimate single administration reliability for the CCSS subscales [1-3].
Recommendations for use
Based on evidence of validity and reliability, the CCSS can be used to measure chemistry-specific self-efficacy for students in first-year undergraduate chemistry courses. The CCSS can be interpreted as simultaneously measuring three dimensions/facets of chemistry self-efficacy. It should be administered intact, as a 21-item instrument, as there is no evidence that the data generated by administering subscales separately can be validly interpreted.
Details from panel review
The CCSS developers and users have presented evidence of validity and reliability of data generated using the instrument in first-year undergraduate chemistry settings in multiple countries. While the developers subjected the instrument to expert review, they (and the reviewers) suggest further validation, particularly for student populations from English-speaking countries [1]. Additionally, some items were modified based on expert feedback, and these items were not reviewed again (i.e., no evidence of test content validity for these modified items) [1].
Some evidence of internal structure validity lacked detail. For example, while the instrument was developed to measure four dimensions of college chemistry self-efficacy, the developers used exploratory factor analysis (as opposed to confirmatory factor analysis) to investigate factor structure [1]. Two of the four hypothesized factors collapsed into a single factor (self-efficacy for cognitive skills). The developers describe this as reasonable and supported by theory. The factor structure was replicated with a second sample [1]. In a second study [2], authors again used exploratory factor analysis, despite the evidence for a factor structure presented by the developers. In that study [2], the three-factor structure mirrored that suggested by developers but was composed of only 12 of the original 21 items; it is unclear when and why 9 items were removed from the factor analysis.
Coefficient alpha was reported as an estimate of single administration reliability in all three studies; alpha values were ≥ 0.82 in [1], ≥ 0.68 in [2], and ≥ 0.74 in [3]. In one study [3], though the researchers did administer the full CCSS, they did not provide an alpha value (nor any other analysis, evidence of validity and reliability) for one of the subscales (self-efficacy for psychomotor skills) because it was deemed irrelevant in their context, as the course was separate from the lab component.
References
[1] Uzuntiryaki, E., & Çapa Aydın, Y. (2009). Development and validation of chemistry self-efficacy scale for college students. Research in Science Education, 39(4), 539-551. https://doi.org/10.1007/s11165-008-9093-x
[2] Ramnarain, U., & Ramaila, S. (2018). The relationship between chemistry self-efficacy of South African first year university students and their academic performance. Chemistry Education Research and Practice, 19(1), 60-67. https://doi.org/10.1039/C7RP00110J
[3] Graham, K. J., Bohn-Gettler, C. M., & Raigoza, A. F. (2019). Metacognitive training in chemistry tutor sessions increases first year students’ self-efficacy. Journal of chemical education, 96(8), 1539-1547. https://doi.org/10.1021/acs.jchemed.9b00170