(Post last updated July 21, 2021)
Review panel summary
The Structure of Motion and Matter (SAMM) survey focuses on the implicit assumption students have about the particulate nature of matter. It is a 15 minute open-ended survey with three main items that have three to five tasks per item. A scoring scheme is available via an Excel spreadsheet for easy and reliable scoring. To minimize variance in scoring, the spreadsheet features a series of yes/no questions about the items in particular and then, through a sequence of programmed functions, an implicit assumptions’ category will be assigned for the concept tested. The SAMM has been evaluated with a diverse population of students ranging from middle school through the fourth year of college with students enrolled in science courses in the United States [1-3]. A Flesch-Kincaid test of readability provided evidence that the SAMM survey is appropriate for a 4th grade reading level and above [1]. It should be noted that there is a version of both the survey and the scoring scheme in Spanish that are available [1]. Test content validity evidence was established through consultation with 26 graduate students and 3 faculty members who were referred to as content experts [1]. Evidence of response process validity was shown through eliciting student comments on the clarity of instructions and items, focus groups, and student interviews [1]. There is some evidence of validity when investigating relations with other variables as it was shown that transfer of knowledge to other contexts was possible by relating the average scores on common exam questions to that of the students’ performance on the administration of the SAMM as a post-test [4]. It was also suggested that the SAMM survey results could be related to the participant’s exposure to science instruction [1]. Test-retest reliability was conducted to provide evidence of reproducible results using students from a psychology course; however, it was not conducted on the intended audience of the survey [1]. Interrater reliability was also calculated which suggests reliability of the scoring scheme [1].
Recommendations for use
The SAMM survey is designed to assess students’ implicit assumptions about four particular nature of matter concepts: the structure of the substance, the structure of the medium, the origin of the motion of particles, and the trajectories of particles [1]. The authors came up with this grouping of concepts and due to the format of open ended scoring, no evidence was provided for if these concepts support the internal structure of the survey. However, there is much supporting evidence that this instrument produces reliable data and valid inferences that is intended for a variety of age and skill levels. The panel recommends that before use, the content and difficulty of the items be evaluated to determine if they are appropriate for the intended audience.
Details from panel review
The panel discussed that there was no statistical evidence provided to support the suggested internal structure of the survey [1]. The survey and scoring scheme; however, were built around research conducted on students’ understanding and implicit assumption about the particulate nature of matter. Implicit assumptions about the four concepts of the structure of the solute, the structure of the solvent, the origin of motion of the particles, and the trajectory of the particles were chosen as the area of study because of previous research on cognitive constraints from a learning progression on understanding matter. While it is understood that factor analysis may be less meaningful with their intended open-ended scoring scheme, providing some additional evidence for the internal structure besides the implicit assumptions analysis through grounded theory would help to further support the scores derived from SAMM data. The developers have provided evidence for test-retest reliability using students enrolled in a psychology course that was chosen because of the belief that it was unlikely they would learn any of the content featured in the SAMM during the two week span between taking the survey. However, there is a need for more evidence that the test-retest reliability of the survey extends to students in the sciences as well [1].
References
[1] Stains, M., Escriu-Sune, M., Molina Alvarez de Santizo, M. L., & Sevian, H. (2011). Assessing Secondary and College Students’ Implicit Assumptions about the Particulate Nature of Matter: Development and Validation of the Structure and Motion of Matter Survey. Journal of Chemical Education, 88(10), 1359–1365. https://doi.org/10.1021/ed1002509
[2] Sevian H., & Stains M. (2013) Implicit Assumptions and Progress Variables in a Learning Progression About Structure and Motion of Matter. In: Tsaparlis G., Sevian H. (eds) Concepts of Matter in Science Education. Innovations in Science Education and Technology, vol 19. Springer, Dordrecht. https://doi.org/10.1007/978-94-007-5914-5_4
[3] Stains, M., & Sevian, H. (2015). Uncovering Implicit Assumptions: A Large-Scale Study on Students’ Mental Models of Diffusion. Research in Science Education, 45(6), 807–840. https://doi.org/10.1007/s11165-014-9450-x
[4] Sevian, H., Hugi-Cleary, D., Ngai, C., Wanjiku, F., & Baldoria, J. M. (2018). Comparison of learning in two context-based university chemistry classes. International Journal of Science Education, 40(10), 1239–1262. https://doi.org/10.1080/09500693.2018.1470353