(Post last updated June 2, 2021)
Review panel summary
The ASCIv2 is an instrument that measures two aspects of students’ attitudes toward chemistry: Intellectual Accessibility and Emotional Satisfaction [1]. It consists of 8 semantic differential items, four on each scale. The instrument has been used in both paper and online formats and the time needed for completion is less than 5 minutes. The instrument has been used in a variety of courses and evidence for the validity and reliability of data from this instrument has been reported from students in general chemistry [1-7, 9-10], organic chemistry [8, 11], and second year inorganic chemistry [12] courses in the United States. Data have also been collected with students in Australia [3, 7, 9], Saudi Arabia [7], and Qatar [10] using the instrument in English throughout.
Confirmatory factor analysis has been used to produce abundant evidence related to the internal structure [1-4, 7, 11] of data from the instrument. These analyses indicate that the two subscales are robust and function well in multiple settings, although a confirmatory factor analysis of data collected in Saudi Arabia [7] did have poor fit. Two studies conducted student interviews providing limited response process evidence [7, 9] (see recommendations below). The instrument has been used extensively in studies with other measures of affective traits [5] and course achievement [1, 2, 4, 11] providing evidence for relations to other variables validity. Additional evidence for these relations include reports indicating the instrument detecting differences in attitudes of students from one country compared to other countries [3] and in course pedagogy/format [8-10]. Evidence for measurement invariance has been found [11] suggesting that data from the instrument works consistently across student cohorts. There is limited evidence to support test-retest reliability [2, 4]. Evidence for single administration reliability of both subscales is based on coefficient alpha values, ranging from 0.56 – 0.85 [1-10, 11], and omega values of 0.69 – 0.81 [11].
Recommendations for use
Given the abundant evidence supporting the proposed internal structure of data from the subscales on the ASCIv2, it can be used to measure two aspects of students’ attitude towards chemistry: Intellectual Accessibility and Emotional Satisfaction. To increase response process validity evidence, additional student interviews are recommended, especially for studies using the instrument with non-native English speakers. For studies seeking to compare attitudes across groups, across time, or in alternate delivery formats, it is recommended that measurement invariance be investigated further.
Details from panel review
The internal structure of data from the ASCIv2 has been thoroughly investigated and appears to be well-supported [1-4, 7, 11]. The subscales, while not developed from a strong theoretical stance, have been mapped onto an existing theoretical framework [1, 11]. When data were collected from students in Saudi Arabia [7], the confirmatory factor analysis model fit was not as good, and one item was identified as being problematic. With only two reports of use in non-native English-speaking countries [7, 10], if the instrument is used in this type of setting a thorough investigation of the internal structure and the response process of the data is warranted.
Student attitudes, on one or both subscales, have been found to positively correlate with ability/achievement [1, 2, 4, 11]. The instrument has also been used to investigate the relation between student attitude and course delivery, with attitudes in a flipped course improving more than in a traditional course [8] and no difference between online versus face-to-face delivery [12]. The instrument has detected higher perceived Intellectual Accessibility from students in courses using POGIL pedagogy compared to students in a traditional course [9, 11].
Scalar measurement invariance has been found across groups (Black female students versus all other students), thereby supporting the comparison of their ASCIv2 scores [11]. No evidence for configural invariance across time (i.e., pre to post) was found, therefore, the comparison of scores in this manner is currently unsupported [4]. More work in measurement invariance is needed to better understand which comparisons would be supported.
Evidence for reliability is largely based on reports of coefficient alpha [1-10, 12], with one study reporting a value of omega [11]. While alpha is a widely used measure of single administration reliability, there is no evidence that the data from this instrument meet the criteria for its use [11].
The instrument has been used in both paper and online administrations. One study indicated that the descriptive statistics did not differ when used online versus paper [2]. However, more work is needed in this area to demonstrate the equivalent functioning of the instrument in these different environments.
References
[1] Xu, X., & Lewis, J. E. (2011). Refinement of a chemistry attitude measure for college students. Journal of Chemical Education, 88(5), 561–568. https://doi.org/10.1021/ed900071q
[2] Brandriet, A. R., Xu, X., Bretz, S. L., & Lewis, J. E. (2011). Diagnosing changes in attitude in first-year college chemistry students with a shortened version of Bauer’s semantic differential. Chemistry Education Research and Practice, 12(2), 271–278. https://doi.org/10.1039/c1rp90032c
[3] Xu, X., Southam, D. C., & Lewis, J. E. (2012). Attitude toward the subject of chemistry in Australia: an ALIUS and POGIL collaboration to promote cross-national comparisons. Australian Journal of Education in Chemistry, 72, 32–36.
[4] Brandriet, A. R., Ward, R. M., & Bretz, S. L. (2013). Modeling meaningful learning in chemistry using structural equation modeling. Chemistry Education Research and Practice, 14(4), 421–430. https://doi.org/10.1039/c3rp00043e
[5] Chan, J. Y. K., & Bauer, C. F. (2014). Identifying at-risk students in general chemistry via cluster analysis of affective characteristics. Journal of Chemical Education, 91(9), 1417–1425. https://doi.org/10.1021/ed500170x
[6] Cracolice, M. S., & Busby, B. D. (2015). Preparation for college general chemistry: more than just a matter of content knowledge acquisition. Journal of Chemical Education, 92(11), 1790–1797. https://doi.org/10.1021/acs.jchemed.5b00146
[7] Xu, X., Alhooshani, K., Southam, D., & Lewis, J. E. (2015). Gathering psychometric evidence for ASCIv2 to support cross-cultural attitudinal studies for college chemistry programs. In Affective Dimensions in Chemistry Education; Kahveci, Orgill, Eds.; Springer-Verlag: Berlin; pp 177–194. https://doi.org/10.1007/978-3-662-45085-7_9
[8] Mooring, S. R., Mitchell, C. E., & Burrows, N. L. (2016). Evaluation of a flipped, large-enrollment organic chemistry course on student attitude and achievement. Journal of Chemical Education, 93(12), 1972–1983. https://doi.org/10.1021/acs.jchemed.6b00367
[9] Vishnumolakala, V. R., Southam, D. C., Treagust, D. F., Mocerino, M., & Qureshi, S. (2017). Students’ attitudes, self-efficacy and experiences in a modified process-oriented guided inquiry learning undergraduate chemistry classroom. Chemistry Education Research and Practice, 18(2), 340–352. https://doi.org/10.1039/c6rp00233a
[10] Vishnumolakala, V. R., Qureshi, S. S., Treagust, D. F., Mocerino, M., Southam, D. C., & Ojeil, J. (2018). Longitudinal impact of process-oriented guided inquiry learning on the attitudes, self-efficacy and experiences of pre-medical chemistry students. Qscience Connect, 2018(1), 1. https://doi.org/10.5339/connect.2018.1
[11] Rocabado, G. A., Kilpatrick, N. A., Mooring, S. R., & Lewis, J. E. (2019). Can we compare attitude scores among diverse populations? An exploration of measurement invariance testing to support valid comparisons between black female students and their peers in an organic chemistry course. Journal of Chemical Education, 96(11), 2371–2382. https://doi.org/10.1021/acs.jchemed.9b00516
[12] Nennig, H. T., Idárraga, K. L., Salzer, L. D., Bleske-Rechek, A., & Theisen, R. M. (2019). Comparison of student attitudes and performance in an online and a face-to-face inorganic chemistry course. Chemistry Education Research and Practice, 21(1), 168–177. https://doi.org/10.1039/c9rp00112c