(Post last updated June 23, 2022)
Review panel summary
This 6-item Likert scale instrument is adapted from the original 12-item Self-efficacy for Cognitive Skills scale from the three-factor College Chemistry Self-Efficacy Scale (CCSS). This reduced scale was designed to measure university students’ self-efficacy beliefs regarding their ability to perform cognitive skills required in their chemistry courses. It has been evaluated for first-year undergraduate chemistry courses in institutions across the United States [1-4], including a Hispanic-serving institution [3] and reformed courses [4].
The developers present evidence related to response process validity, based on interviews with undergraduate chemistry students in a first-semester general chemistry laboratory course. The developers also report evidence of internal structure validity, based on confirmatory factor analyses. A 1-factor, 6-item model for the revised scale is supported by model fit indices in the acceptable range; this factor structure is further supported in additional confirmatory factor analyses with alternate samples in the development study [1] and has been replicated in two additional studies [3, 4]. Evidence of measurement invariance validity supports the use of the instrument to compare scale scores across time, institutional settings, gender, and underrepresented minority status groups [4]. In multiple studies, evidence based on relations to other variables is provided by the relation between scale scores and course performance [2, 3]. In one context, chemistry majors were observed to have higher scale scores than non-majors [1]; this scale may be able to differentiate between major and non-major students, though this has not been demonstrated beyond the development context. The developers have also hypothesized and explored relations between scale scores and interest in chemistry through multiple regression and path analysis, though this relation has not been theoretically supported and might require further investigation [2]. In multiple studies, single administration reliability was estimated using coefficient alpha [1-3] and McDonald’s omega [4].
Recommendations for use
This reduced scale is designed to be used as a 6-item scale to measure students’ chemistry-specific self-efficacy for Cognitive Skills. There is evidence to support that the instrument generates valid and reliable data when used to assess students in first-year undergraduate courses in the United States. However, for contexts unlike those in these studies, users are encouraged to further investigate the validity and reliability of data generated using the scale.
Details from panel review
Initially, the scale was administered as an 8-item scale, but two items were dropped based on response process interview data and modification indices for the items. Though the internal structure validity evidence is consistent across multiple studies and samples, the error estimates (RMSEA) are also consistently higher than recommended [1, 3, 4]. Likewise, while Naibert et al. [4] suggest that evidence of measurement invariance validity supports the use of the instrument to compare scale scores across gender and underrepresented minority status groups, the change in RMSEA values are slightly outside the recommended range.
Across studies, coefficient alpha values were ≥ 0.82 [1], ≥ 0.85 [2], and ≥ 0.88 [3]; in one study, authors reported McDonald’s omega ≥ 0.83 [4] as evidence of single administration reliability.
References
[1] Ferrell, B., & Barbera, J. (2015). Analysis of students' self-efficacy, interest, and effort beliefs in general chemistry. Chemistry Education Research and Practice, 16(2), 318-337. https://doi.org/10.1039/C4RP00152D
[2] Ferrell, B., Phillips, M. M., & Barbera, J. (2016). Connecting achievement motivation to performance in general chemistry. Chemistry Education Research and Practice, 17(4), 1054-1066. https://doi.org/10.1039/C6RP00148C
[3] Moreno, C., Pham, D., & Ye, L. (2021). Chemistry self-efficacy in lower-division chemistry courses: changes after a semester of instruction and gaps still remain between student groups. Chemistry Education Research and Practice, 22(3), 772-785. https://doi.org/10.1039/D0RP00345J
[4] Naibert, N., Duck, K. D., Phillips, M. M., & Barbera, J. (2021). Multi-institutional Study of Self-Efficacy within Flipped Chemistry Courses. Journal of Chemical Education, 98(5), 1489-1502. https://doi.org/10.1021/acs.jchemed.0c01361