Particulate Nature Of Matter And Chemical Bonding Diagnostic Instrument

(Post last updated 28 December 2024)

Review panel summary   
The Particulate Nature of Matter and Chemical Bonding Diagnostic Instrument (PNM-CBDI) is a 10 item instrument with five items assessing students’ conceptions of the particulate nature of matter and five items assessing chemical bonding [1]. It has been used with high school students in Singapore [1] and general chemistry students in a U.S. college [3]. The items used are two-tiered, where the student selects an answer to the question and then selects a reason for their answer. Most items for the PNM-CBDI were taken from existing instruments, with a few developed for this work. The items are intended to identify misconceptions held by students. Single administration reliability evidence is based on coefficient alpha values of 0.66 [1] and 0.62 [3] when the 20 items (10 answers plus 10 reasons) are analyzed separately. When the answers and reason are scored together, resulting in a 10-item set, alpha decreases to 0.42 [3]. Weak evidence of internal structure validity, for either a one- or two-factor solution, was provided using confirmatory factor analysis [3]. Some validity evidence in the form of relation to other variables has been found. In one study, students in higher grades had greater overall scores than students in lower grades [1]. Scores on the assessment, when SAT Math was accounted for, were higher for students that had taken a preparatory chemistry course than for students that had not [3].

Recommendations for use   
According to the validity evidence found to date, there is no support for the proposed two-factor structure (i.e., students’ conceptions of particulate nature of matter and students’ understanding of chemical bonding). The lack of validity evidence supporting the proposed factors, and the alpha values suggests that this instrument is more akin to a content assessment. Without further psychometric exploration, the instrument is best used as an item-by-item diagnostic tool for the presence of common misconceptions, and not as a tool to measure the proposed cognitive constructs of conceptions of particulate nature of matter and students' understanding of chemical bonding.

Details from panel review   
The PNM-CBDI was developed to measure two proposed cognitive constructs: students’ conceptions of particulate nature of matter and students’ understanding of chemical bonding. The development process followed a four-step process: (1) Identification of propositional knowledge statements, (2) Development of concept map(s), (3) Relating of propositional knowledge to the concept map(s), and (4) Validation of the content (by two academics not in the research team) [2]. One of the concept maps was developed for this instrument (particulate nature of matter) and the other (concepts of chemical bonding) was modified from a concept map from previous study. The items developed were not developed specifically for this instrument, rather they were taken from other instruments. The development paper does not describe in detail the relationship between the selected items and the concept maps [1]. Fit statistics from confirmatory factor analysis models of the proposed two-factor model were acceptable, the item loadings were problematic with items only weakly loading on their proposed factor [3]. Additionally, there was high correlation, > 0.83, between the factors suggesting redundancy in the two factor solution. Fit statistics for the one factor model were slightly worse, again the items only weakly loaded on the one factor [3].

References

[1] Jazilah Othman , David F. Treagust & A. L. Chandrasegaran (2008). An Investigation into the Relationship between Students’ Conceptions of the Particulate Nature of Matter and their Understanding of Chemical Bonding. Int. J. Sci. Educ., 30(11), 1531-1550.

[2] Heredia, K., Xu., X., & Lewis, J. E. (2012). The application and evaluation of a two-concept diagnostic instrument with students entering college general chemistry. Chem. Educ. Res. & Pract., 13, 30-38.