Skip to main content

Particulate Nature Of Matter Assessment

ParNoMA

OVERVIEW
Overview
Listed below is general information about the instrument.
Summary
Original author(s)

  • Yezierski, E.J., & Birk, J.P.
Original publication

  • Yezierski, E.J., & Birk, J.P. (2006). Misconceptions about the particulate nature of matter. Using animations to close the gender gap. Journal of Chemical Education, 83(6), 954-960.
Year original instrument was published 2006
Inventory
Number of items 20
Number of versions/translations 2
Cited implementations 3
Language
  • English
Country United States
Format
  • Open Ended
Intended population(s)
  • Students
  • Middle School
  • High School
  • Undergraduate
  • Secondary School
Domain
  • Cognitive
Topic
  • Nature of Matter
Evidence
The CHIRAL team carefully combs through every reference that cites this instrument and pulls all evidence that relates to the instruments’ validity and reliability. These data are presented in the following table that simply notes the presence or absence of evidence related to that concept, but does not indicate the quality of that evidence. Similarly, if evidence is lacking, that does not necessarily mean the instrument is “less valid,” just that it wasn’t presented in literature. Learn more about this process by viewing the CHIRAL Process and consult the instrument’s Review (next tab), if available, for better insights into the usability of this instrument.

Information in the table is given in four different categories:
  1. General - information about how each article used the instrument:
    • Original development paper - indicates whether in which paper(s) the instrument was developed initially
    • Uses the instrument in data collection - indicates whether an article administered the instrument and collected responses
    • Modified version of existing instrument - indicates whether an article has modified a prior version of this instrument
    • Evaluation of existing instrument - indicates whether an article explicitly provides evidence that attempt to evaluate the performance of the instrument; lack of a checkmark here implies an article that administered the instrument but did not evaluate the instrument itself
  2. Reliability - information about the evidence presented to establish reliability of data generated by the instrument; please see the Glossary for term definitions
  3. Validity - information about the evidence presented to establish reliability of data generated by the instrument; please see the Glossary for term definitions
  4. Other Information - information that may or may not directly relate to the evidence for validity and reliability, but are commonly reported when evaluating instruments; please see the Glossary for term definitions
Publications: 1 2 3

General
Original development paper ✔
Uses the instrument in data collection ✔ ✔ ✔
Modified version of existing instrument ✔
Evaluation of existing instrument ✔ ✔ ✔

Reliability
Test-retest reliability
Internal consistency
Coefficient (Cronbach's) alpha ✔ ✔
McDonald's Omega
Inter-rater reliability
Person separation
Generalizability coefficients
Other reliability evidence

Vailidity
Expert judgment ✔
Response process ✔
Factor analysis, IRT, Rasch analysis ✔
Differential item function
Evidence based on relationships to other variables ✔ ✔
Evidence based on consequences of testing
Other validity evidence

Other information
Difficulty
Discrimination
Evidence based on fairness
Other general evidence
Review
DISCLAIMER: The evidence supporting the validity and reliability of the data summarized below is for use of this assessment instrument within the reported settings and populations. The continued collection and evaluation of validity and reliability evidence, in both similar and dissimilar contexts, is encouraged and will support the chemistry education community’s ongoing understanding of this instrument and its limitations.
This review was generated by a CHIRAL review panel. Each CHIRAL review panel consists of multiple experts who first individually review the citations of the assessment instrument listed on this page for evidence in support of the validity and reliability of the data generated by the instrument. Panels then meet to discuss the evidence and summarize their opinions in the review posted in this tab. These reviews summarize only the evidence that was discussed during the panel which may not represent all evidence available in the published literature or that which appears on the Evidence tab.
If you feel that evidence is missing from this review, or that something was documented in error, please use the CHIRAL Feedback page.

Panel Review: Particulate Nature of Matter Assessment (ParNoMA)

(Post last updated June 24, 2022)

Review panel summary

The Particulate Nature of Matter Assessment (ParNoMA) consists of 20 multiple choice items and includes 5 different concepts about particles (size, weight, composition, phase change, and energy) [1, 2]. The instrument was designed to assess a student's conceptual understanding regarding the particulate nature of matter (PNM), especially as it relates to phases of matter and phase change topics. The instrument has been evaluated with middle/high school students and students enrolled in college general chemistry [1-3]. The instrument is used as a pre- and post-test. Student scores are reported as the percent of the number of questions the participants correctly answered out of the total score [1, 2]. The authors did not differentiate the student scores at different academic levels [1, 2].

Several aspects of validity and reliability have been assessed for the data generated by the ParNoMA. This instrument is based on a previously unpublished assessment developed by the authors. Evidence in support of the test content validity was reviewed by a small sample of experts [1]. Faculty and graduate students provided feedback regarding the correctness of the answers to the items at the development stage. During pilot testing undergraduate students provided feedback on the clarity of the items and this was used to provide some evidence for the response process validity of ParNoMa data [1, 2]. Evidence in support of the internal structure of ParNoMa data was reported through factor analysis. The analysis of the 20 items [3] suggested three factors (size of particles, composition of molecules, and particle motion and energies) according to the eigenvalues and the shape of the scree plot. Each factor consisted of 4-9 questions [3]. This three-factor finding was different from the initial design of the instrument of five factors. The authors of the study mentioned that the results were conceptually consistent with the initial design and justified the results of factor analysis by combining factors [3]. The instrument was used in evaluating the effectiveness of a treatment (intervention) showing significant gain score between the researched groups.

In terms of evidence supporting reliability, coefficient alpha was used to estimate the single administration reliability for the total score of the ParNoMA [1-3] but not for the subscales. A small sample of students were asked to justify their responses after they completed the pretest and ¾ of the students’ responses matched their pencil-and-paper test, providing further response process validity evidence.

Recommendations for use

The ParNoMA was designed to assess students’ conceptual understanding regarding the PNM (phases of matter and phase change topics) [1]. As developed, this instrument was designed to be used to assess students’ conceptual understanding of the PNM using a total score, however additional evidence of internal structure is suggested, as was acknowledged by the authors [3]. Less confidence would be warranted for interpreting the subscales, with the exception of the grouping (size of particles, composition of molecules, and particle motion and energies) found in a later study [3]. However, to date no evidence in support of the reliability of any subscales has been shown. Studies provide some evidence that the ParNoMa produces valid and reliable data from the sample of college students [2, 3].

Details from panel review

The ParNoMA developers gathered and reported several aspects of validity and reliability evidence for the instrument [1-3]; however some of the evidence was found to be lacking or is not supported by the reported details. For example, the test content and the response process validity is not well supported by the reported details to assess this evidence or for alignment of the instrument with the theories mentioned by the authors. For the internal structure validity, the authors of the study mentioned that the results were conceptually consistent with the initial design and justified the results of factor analysis by combining factors [3] though evidence was lacking to support this action. Coefficient alpha has been used to estimate single administration reliability for the whole instrument [1-3] but not for the original subscales nor those found through factor analysis, which leaves a lack of support for the interpretation of the data generated by the instrument's subscales.

References

[1] Yezierski, E.J., & Birk, J.P. (2006). Misconceptions about the particulate nature of matter. Journal of Chemical Education, 83(6), 954-960. https://doi.org/10.1021/ed083p954

[2] Bridle, C.A., & Yezierski, E.J. (2012). Evidence for the effectiveness of inquiry-based, particulate-level instruction on conceptions of the particulate nature of matter. Journal of Chemical Education, 89(2), 192-198. https://doi.org/10.1021/ed100735u

[3] Tang, H., & Abraham, M.R. (2016). Effect of computer simulations at the particulate and macroscopic levels on students’ understanding of the particulate nature of matter. Journal of Chemical Education, 93(1), 31-38. https://doi.org/10.1021/acs.jchemed.5b00599

Versions
Listed below are all versions and modifications that were based on this instrument or this instrument were based on.
Instrument has been modified in:
Name Authors
Particulate Nature Of Matter Concept Inventory

  • Ozalp, D., & Kahveci, A.
Chemical And Physical Change Assessment

  • Christian, B.N., & Yezierski, E.J.
Citations
Listed below are all literature that develop, implement, modify, or reference the instrument.

  1. Yezierski, E.J., & Birk, J.P. (2006). Misconceptions about the particulate nature of matter. Using animations to close the gender gap. Journal of Chemical Education, 83(6), 954-960.

  2. Tang, H., & Abraham, M.R. (2016). Effect of Computer Simulations at the Particulate and Macroscopic Levels on Students' Understanding of the Particulate Nature of Matter. Journal of Chemical Education, 93(1), 31-38.

  3. Bridle, C.A., & Yezierski, E.J. (2012). Evidence for the effectiveness of inquiry-based, particulate-level instruction on conceptions of the particulate nature of matter. Journal of Chemical Education, 89(2), 192-198.