Skip to main content

Assessment On Protein Structure & Enzyme Inhibition

OVERVIEW
Overview
Listed below is general information about the instrument.
Summary
Original author(s)

  • Villafañe, S.M., Heyen, B.J., Lewis, J.E., Loertscher, J., Minderhout, V., & Arnold, Murray T.
Original publication

  • Villafañe, S.M., Heyen, B.J., Lewis, J.E., Loertscher, J., Minderhout, V., & Arnold, Murray T. (2016). Design and testing of an assessment instrument to measure understanding of protein structure and enzyme inhibition in a new context. Biochemistry and Mo
Year original instrument was published 2016
Inventory
Number of items 8
Number of versions/translations 1
Cited implementations 1
Language
  • English
Country United States
Format
  • Multiple Choice
  • True/False
  • Open Ended
Intended population(s)
  • Students
  • Undergraduate
Domain
  • Cognitive
Topic
  • Biochemistry
Evidence
The CHIRAL team carefully combs through every reference that cites this instrument and pulls all evidence that relates to the instruments’ validity and reliability. These data are presented in the following table that simply notes the presence or absence of evidence related to that concept, but does not indicate the quality of that evidence. Similarly, if evidence is lacking, that does not necessarily mean the instrument is “less valid,” just that it wasn’t presented in literature. Learn more about this process by viewing the CHIRAL Process and consult the instrument’s Review (next tab), if available, for better insights into the usability of this instrument.

Information in the table is given in four different categories:
  1. General - information about how each article used the instrument:
    • Original development paper - indicates whether in which paper(s) the instrument was developed initially
    • Uses the instrument in data collection - indicates whether an article administered the instrument and collected responses
    • Modified version of existing instrument - indicates whether an article has modified a prior version of this instrument
    • Evaluation of existing instrument - indicates whether an article explicitly provides evidence that attempt to evaluate the performance of the instrument; lack of a checkmark here implies an article that administered the instrument but did not evaluate the instrument itself
  2. Reliability - information about the evidence presented to establish reliability of data generated by the instrument; please see the Glossary for term definitions
  3. Validity - information about the evidence presented to establish reliability of data generated by the instrument; please see the Glossary for term definitions
  4. Other Information - information that may or may not directly relate to the evidence for validity and reliability, but are commonly reported when evaluating instruments; please see the Glossary for term definitions
Publications: 1

General
Original development paper ✔
Uses the instrument in data collection ✔
Modified version of existing instrument
Evaluation of existing instrument ✔

Reliability
Test-retest reliability
Internal consistency
Coefficient (Cronbach's) alpha
McDonald's Omega
Inter-rater reliability ✔
Person separation
Generalizability coefficients
Other reliability evidence

Vailidity
Expert judgment ✔
Response process ✔
Factor analysis, IRT, Rasch analysis
Differential item function
Evidence based on relationships to other variables ✔
Evidence based on consequences of testing
Other validity evidence

Other information
Difficulty
Discrimination ✔
Evidence based on fairness
Other general evidence
Review
DISCLAIMER: The evidence supporting the validity and reliability of the data summarized below is for use of this assessment instrument within the reported settings and populations. The continued collection and evaluation of validity and reliability evidence, in both similar and dissimilar contexts, is encouraged and will support the chemistry education community’s ongoing understanding of this instrument and its limitations.
This review was generated by a CHIRAL review panel. Each CHIRAL review panel consists of multiple experts who first individually review the citations of the assessment instrument listed on this page for evidence in support of the validity and reliability of the data generated by the instrument. Panels then meet to discuss the evidence and summarize their opinions in the review posted in this tab. These reviews summarize only the evidence that was discussed during the panel which may not represent all evidence available in the published literature or that which appears on the Evidence tab.
If you feel that evidence is missing from this review, or that something was documented in error, please use the CHIRAL Feedback page.

Panel Review: Assessment on Protein Structure & Enzyme Inhibition

(Post last updated June 16, 2022)

Review panel summary

The Assessment on Protein Structure & Enzyme Inhibition is an eight-item assessment instrument with a combination of true/false, multiple choice, and open-ended items. The instrument was designed to assess students’ understanding of biochemistry concepts. It has been evaluated with undergraduate students enrolled in introductory biochemistry courses in seven U.S. institutions, including four small, private four-year colleges, two private masters-granting universities, and one public masters-granting university [1]; additionally, one of the seven institutions is classified as a Hispanic-Serving Institution (HSI). Several aspects of validity and reliability have been assessed for the data generated by the instrument. In terms of validity, test content evidence was provided by 30 biochemistry faculty members (five of which were extensively involved in the item generation) in a biochemistry Process Oriented Guided Inquiry Learning (POGIL) community, all with a minimum of five years of teaching experience [1]. Additionally, topics/subject matter essential to be included in an undergraduate biochemistry degree plan were initially selected for instrument items based on recommendations and guidelines from the American Society for Biochemistry and Molecular Biology (ASBMB) [1]. Items were revised based on reviews by two independent raters upon initial collection of data [1]. All eight items on the instrument have also been assigned to levels of Bloom’s taxonomy by a panel of 18 biochemistry faculty members [1]. Following initial collection of data, revisions were made to items based on students’ responses to the open-ended items, as well as instructor feedback, and this was used to provide some evidence in support of response process validity [1]; however, as students were not interviewed in order to clarify details within their open responses, it is difficult to evaluate this evidence. In terms of reliability, a sample of 36 responses were evaluated with Cohen’s kappa to provide evidence for interrater reliability for open response items through two independent raters [1].

Recommendations for use

The Assessment on Protein Structure & Enzyme Inhibition was designed to assess students’ understanding of biochemistry concepts [1]. Items have been scored as correct or incorrect for the true/false and multiple choice; scoring for the open-response items uses a developed rubric [1], which assigns items a score of 0 (incorrect), 1 (partially correct), or 2 (correct). In many instances, the evidence provided in the literature has limited details provided; therefore, the continued generation of evidence of validity and reliability of the data generated via this instrument is recommended.

Details from panel review

The developers of the Assessment on Protein Structure & Enzyme Inhibition instrument gathered and reported some aspects of validity and reliability evidence for the instrument [1]; however, some of the evidence provided is limited since a lack of details have been reported. For example, for interrater reliability, there was limited detail provided on the development of the rubric used to evaluate students’ responses to the open-response items. In terms of internal structure validity, future studies might involve exploratory evaluations of how individual items are related, especially given the group scoring based on Bloom’s levels. In terms of relation to other variables, there is currently no documented evidence regarding how the instrument is related to variables such as performance, achievement, and/or progression in the course content area, which may provide further support for the data from this instrument.

References

[1] Villafañe, S.M., Heyen, B.J., Lewis, J.E., Loertscher, J., Minderhout, V., & Arnold Murray, T. (2016). Design and testing of an assessment instrument to measure understanding of protein structure and enzyme inhibition in a new context. Biochemistry and Molecular Biology Education, 44(2), 179-190. https://doi.org/10.1002/bmb.20931

Versions
This instrument has not been modified nor was it created based on an existing instrument.
Citations
Listed below are all literature that develop, implement, modify, or reference the instrument.

  1. Villafañe, S.M., Heyen, B.J., Lewis, J.E., Loertscher, J., Minderhout, V., & Arnold, Murray T. (2016). Design and testing of an assessment instrument to measure understanding of protein structure and enzyme inhibition in a new context. Biochemistry and Mo