(Post last updated February 15, 2023)
Review panel summary
The Students’ Motivation Towards Science Learning (SMTSL) is a 35-item, 6-factor instrument developed to assess students’ motivation towards science learning [1]. The full 35-item inventory[1-3], as well as shortened adaptations [4-5] have been administered to junior and senior high school students (grades 7-11) in Taiwan using both a 5-point [1-4] and 4-point [5] Likert-type response scale. The total SMTSL score and subscale scores (e.g., self-efficacy) have been used to evaluate students’ science learning motivation in relation to their learning styles [2] and conceptual change [3]. The measure has also been used to explore the effect of classroom interventions on students’ motivation by evaluating pre-post scores [2,4] and through comparisons between control and experimental groups [2,4,5]. Evidence based on test content was provided during the development of the instrument as subscale themes were constructed from literature searches and prior qualitative studies on students’ motivation related to science learning [1]. Additionally, feedback about the items themselves was collected from a panel of experts including six science teachers, three educational psychologists, and five science educators; although no details about how this feedback was used for item development are available [1]. Factor analysis of the 35-item measure provided support for 6 factors: self-efficacy, active learning strategies, science learning value, performance goal, achievement goal, and learning environment stimulation [1]. However, details regarding the type of factor analysis are not included and as a result the evidence based on internal structure of the subscale scores is limited. While total SMTSL scores were calculated and included in multiple analyses [1-5], there was no evidence provided to support the use of a total score. Evidence based on relations to other variables was provided through correlational analysis between the total and subscale score and a measure of students’ science attitude, as well as with students’ science achievement scores [1]. Additionally, significant differences in total and subscale scores were found between groups of high, moderate, and low motivated students, as identified by the classroom teachers [1]. Evidence of single administration reliability was provided through reporting coefficient alpha values for the individual subscales ranging from 0.70 - 0.87 [1,3,4]. Coefficient alpha was also reported for the total SMTSL score [1,3-5], although no evidence was provided to support the use of a total score.
Recommendations for use
The SMTSL has been used to explore students’ science learning motivation in junior and senior high schools in Taiwan. Although some validity evidence has been provided, details related to this evidence are limited. Evidence based on internal structure provided some support for the presence of 6 factors [1], which would allow subscale scores (i.e., self-efficacy, active learning strategies, science learning value, performance goal, achievement goal, and learning environment stimulation) to be reported. While a SMTSL total score has been reported [1-5], no evidence was provided to support the use of a total score. Therefore, the use of a total score is discouraged in future studies. Some studies reported using a Chinese language version of the survey [4,5]; however, it is unclear in which language the original survey items were developed and administered, although an English language version was included in the appendix of the original development paper [1]. Given the limited evidence supporting the item wording and item groupings, future users are encouraged to provide validity evidence based on response process and internal structure for data collected with this instrument. Additionally, there is currently no evidence to support the use of a shortened 4-factor version of this instrument. Therefore, the SMTSL should only be administered in its entirety (i.e., 35 items), unless a user plans to conduct factor analyses to show that a reduced number of items still group into the same categories.
Details from panel review
Some aspects of validity and reliability evidence have been reported for data collected with the SMTSL; however, much of the evidence provided lacks the details needed to sufficiently support the use of this instrument. For evidence based on test content, although a panel of experts were consulted regarding the items [1], it is unknown what type of feedback the developers received from the experts and how any feedback might have been used to make modifications to the items and/or measure. Although factor analysis was used to analyze the data to provide evidence based on internal structure [1], it is unclear what type of factor analyses was performed and the specifications that were used. The results from factor analyses of the data collected with the full 35-item survey indicated that items loaded onto 6 separate factors [1]; however, factor analyses were not reported for the 4-factor shortened version [4] or to support a single SMTSL total score from the full [1-3] or shortened [4-5] versions. Correlations between subscale (e.g., self-efficacy) means ranged from 0.09-0.51 [1], providing some support for independent but related factors. Some of the studies included data from interviews which were designed to learn more about students’ motivation [2,3,5], but the details provided about these interviews did not indicate that any evidence based on response process was gathered. Evidence based on relations to other variables was explored during development by evaluating the relation between total and subscale scores and students’ attitude toward science and their science achievement [1]. Additionally, significant group differences on total and subscale scores were found between students identified as high, moderate, and low motivation by teachers [1], although the guidelines on how teachers grouped these students was not reported. Group differences were explored in multiple studies, both between different classes [2,4] and between control and experimental groups with the introduction of classroom interventions [2,4,5]; however, there was no indication that evidence to support measurement invariance had been collected. Evidence of single administration reliability was provided through reporting coefficient alpha values for each subscale [1,3,4]. This was done using both individual responses and the class mean as units of analysis [1], although it is unclear how the latter value was calculated. Coefficient alpha was also reported for a SMTSL total score of the full [1,3] and shortened versions [4,5], although no evidence to support the unidimensionality of a total score was provided.
References
[1] Tuan, H.-L., Chin, C.-C., & Shieh, S.-H. (2005). The development of a questionnaire to measure students' motivation towards science learning. Int. J. Sci. Educ., 27(6), 639-654.
[2] Tuan, H.-L., Chin, C.-C., Tsai, C.-C., & Cheng, S.-F. (2005). Investigating the effectiveness of inquiry instruction on the motivation of different learning styles students. Int. J. Sci & Math. Educ., 3, 541-566.
[3] Tseng, C.-H., Tuan, H.-L., & Chin, C.-C. (2010). Investigating the Influence of Motivational Factors on Conceptual Change in a Digital Learning Context Using the Dual‐Situated Learning Model. Int. J. Sci. Educ., 32(14), 1853-1875.
[4] Lin, C.-Y. & Wu, H.-K. (2021). Effects of different ways of using visualizations on high school students’ electrochemistry conceptual understanding and motivation towards chemistry learning. Chem. Educ. Res. & Pract. 22, 786-801.
[5] Lin, C.-Y. & Tsai, C.-C. (2021). The Effect of a Pedagogical STEAM Model on Students’ Project Competence and Learning Motivation. J. Sci. Educ. & Tech. 30, 112-124.