Skip to main content

Scale Concept Inventory

SCI

OVERVIEW
Overview
Listed below is general information about the instrument.
Summary
Original author(s)

  • Gerlach, K., Trate, J., Blecking, A., Geissinger, P., & Murphy, K.
Original publication

  • Gerlach, K., Trate, J., Blecking, A., Geissinger, P., & Murphy, K. (2014). Valid and Reliable Assessments To Measure Scale Literacy of Students in Introductory College Chemistry Courses. Journal of Chemical Education, 91(10), 1538-1545.
Year original instrument was published 2014
Inventory
Number of items 40
Number of versions/translations 1
Cited implementations 2
Language
  • English
Country United States
Format
  • Response Scale
Intended population(s)
  • Students
  • Undergraduate
  • Graduate
Domain
  • Cognitive
Topic
  • Size
Evidence
The CHIRAL team carefully combs through every reference that cites this instrument and pulls all evidence that relates to the instruments’ validity and reliability. These data are presented in the following table that simply notes the presence or absence of evidence related to that concept, but does not indicate the quality of that evidence. Similarly, if evidence is lacking, that does not necessarily mean the instrument is “less valid,” just that it wasn’t presented in literature. Learn more about this process by viewing the CHIRAL Process and consult the instrument’s Review (next tab), if available, for better insights into the usability of this instrument.

Information in the table is given in four different categories:
  1. General - information about how each article used the instrument:
    • Original development paper - indicates whether in which paper(s) the instrument was developed initially
    • Uses the instrument in data collection - indicates whether an article administered the instrument and collected responses
    • Modified version of existing instrument - indicates whether an article has modified a prior version of this instrument
    • Evaluation of existing instrument - indicates whether an article explicitly provides evidence that attempt to evaluate the performance of the instrument; lack of a checkmark here implies an article that administered the instrument but did not evaluate the instrument itself
  2. Reliability - information about the evidence presented to establish reliability of data generated by the instrument; please see the Glossary for term definitions
  3. Validity - information about the evidence presented to establish reliability of data generated by the instrument; please see the Glossary for term definitions
  4. Other Information - information that may or may not directly relate to the evidence for validity and reliability, but are commonly reported when evaluating instruments; please see the Glossary for term definitions
Publications: 1 2

General
Original development paper ✔
Uses the instrument in data collection ✔ ✔
Modified version of existing instrument ✔
Evaluation of existing instrument ✔

Reliability
Test-retest reliability ✔
Internal consistency
Coefficient (Cronbach's) alpha
McDonald's Omega
Inter-rater reliability
Person separation
Generalizability coefficients
Other reliability evidence

Validity
Expert judgment ✔
Response process
Factor analysis, IRT, Rasch analysis
Differential item function
Evidence based on relationships to other variables ✔
Evidence based on consequences of testing
Other validity evidence

Other information
Difficulty
Discrimination
Evidence based on fairness
Other general evidence
Review
DISCLAIMER: The evidence supporting the validity and reliability of the data summarized below is for use of this assessment instrument within the reported settings and populations. The continued collection and evaluation of validity and reliability evidence, in both similar and dissimilar contexts, is encouraged and will support the chemistry education community’s ongoing understanding of this instrument and its limitations.
This review was generated by a CHIRAL review panel. Each CHIRAL review panel consists of multiple experts who first individually review the citations of the assessment instrument listed on this page for evidence in support of the validity and reliability of the data generated by the instrument. Panels then meet to discuss the evidence and summarize their opinions in the review posted in this tab. These reviews summarize only the evidence that was discussed during the panel which may not represent all evidence available in the published literature or that which appears on the Evidence tab.
If you feel that evidence is missing from this review, or that something was documented in error, please use the CHIRAL Feedback page.

Panel Review: Scale Concept inventory (SCI)

(Post last updated June 14, 2022)

Review panel summary

The Scale Concept Inventory (SCI) is a 40-item 5-point Likert-type scale instrument developed by the same authors who developed the Scale Literacy Skills Test (SLST). The developers suggest that the SCI can be used to assess students' misconceptions related to scale. The authors described that during the development of the SLST, evidence of the need to develop a companion test addressing misconceptions was shown. In support of test content validity evidence, the developers describe creating the items from the literature and interviews with students [1]. However, a detailed process of the development was not described in the manuscript and this instrument was not available in the publications, nor in the supplemental information. Relation to other variables validity evidence was collected by correlation studies between the SCI and variables including ACT Math and final exam scores, where the correlations were significant [1, 2]. The panel found no reported reliability evidence.

Some descriptive item statistics are reported in the supplemental materials, but not the items themselves [1]. The panel found it difficult to evaluate this instrument further without a deeper understanding of the items.

Recommendations for use

The SCI has been used in preparatory and general chemistry courses [1, 2]. It has been used as a companion to the Scale Literacy Skills Test (SLST) instrument, and often used as a component of what the authors call the Scale Literacy Score (SLS) [1, 2]. However, the SCI on its own has been shown to correlate to measures of performance, such as final exam scores, at a significant level.

It is unclear that this instrument is needed as a companion to the SLST as intended. It could be used as a stand-alone instrument to assess similar but separate aspects of scale literacy skills and misconceptions of such. It is hard to evaluate how to use this instrument or further provide a recommendation for use without more understanding of the items and their internal structure.

Details from panel review

The authors provided evidence of relation to other variables validity in the form of correlation analyses with a variety of scores including final exam, ACT Math, and others. Correlation analysis between the SCI and the course final exam scores was 0.402 for general chemistry, and 0.250 for preparatory chemistry [1]. All of these correlations were significant, however, no estimates of effect size were reported [1]. The authors used the SCI and the SLST combined into one weighted score (SLS), which was found to be a predictor of final exam scores. Given that the correlations with other variables for the SCI (~0.2-0.3) are lower than those for the SLST (~0.4-0.5), and the correlations do not improve markedly when using the combined SLS (~0.4-0.5) [1 supporting information], there was not enough evidence to support using both instruments. Given the higher correlations, the SLST may be the preferred choice for scale literacy testing.

The limitations of this instrument noted by the panel include the sparse information about test content validity, lack of evidence supporting internal structure validity, as well as a lack of evidence supporting reliability.

References

[1] Gerlach, K., Trate, J., Blecking, A., Geissinger, P., & Murphy, K. (2014). Valid and reliable assessments to measure scale literacy of students in introductory college chemistry courses. Journal of Chemical Education, 91(10), 1538-1545. https://doi.org/10.1021/ed400471a

[2] Trate, J.M., Geissinger, P., Blecking, A., & Murphy, K.L. (2019). Integrating scale-themed instruction across the general chemistry curriculum. Journal of Chemical Education, 96(11), 2361-2370. https://doi.org/10.1021/acs.jchemed.9b00594

Versions
Listed below are all versions and modifications that were based on this instrument or this instrument were based on.
Instrument is derived from:
Name Authors
Scale Activity

  • Gerlach, K., Trate, J., Blecking, A., Geissinger, P., & Murphy, K.
Citations
Listed below are all literature that develop, implement, modify, or reference the instrument.

  1. Gerlach, K., Trate, J., Blecking, A., Geissinger, P., & Murphy, K. (2014). Valid and Reliable Assessments To Measure Scale Literacy of Students in Introductory College Chemistry Courses. Journal of Chemical Education, 91(10), 1538-1545.

  2. Trate, J.M., Geissinger, P., Blecking, A., & Murphy, K.L. (2019). Integrating Scale-Themed Instruction across the General Chemistry Curriculum. Journal of Chemical Education, 96(11), 2361-2370.