Ionisation Energy Diagnostics Instrument

(Post last updated 28 December 2024)

Review panel summary   
The Ionisation Energy Diagnostics Instrument (IEDI) is a 10-item, 2-tiered, multiple-choice diagnostic instrument, with both an answer tier and a corresponding reasoning tier. The IEDI was designed to assess the alternative conceptions related to ionization of A-level students (16 to 19 years old, high school students) in Singapore, as well as explore their understanding of the trend of ionization energy across the periodic table [1-3]. The first part, or “answer tier”, of each of the 10 items consists of a multiple-choice content question, with 2-3 answer options. The second part of each item, or “reason tier”, contains a set of 4-5 possible reasons for the answer to the answer tier. Incorrect reasons (distractors) were developed using the literature, interviews with experts and students, and open-response tests. Initially, a concept map of ionization energy and corresponding propositional knowledge statements were developed using the literature and two chemistry textbooks [1]. The concept map and propositional knowledge statements were subsequently reviewed by 13 experienced A-level chemistry teachers and two tertiary chemistry educators, who agreed that these components were accurate and relevant.

The development of the 10-item IEDI consisted of two main phases [1]. In Phase 1, Version 1 of the instrument was developed based on the concept map and propositional knowledge statements. Version 1 was administered to 18 Grade 11 students, and another six Grade 11 students were interviewed in pairs using the instrument as the interview protocol. Based on evidence collected, a revised Version 2 of the instrument was administered to 146 Grade 11-12 students at four schools. Version 2 was then subsequently revised to Version 3, which was reviewed by four A-level chemistry teachers who agreed that the items were consistent with requirements of the A-level syllabus for ionization energy. Version 3 was then administered to 130 Grade 12 students from three schools, where 11 of these students were interviewed using the instrument as the interview protocol to determine whether any item was ambiguous and to probe the reasons for their answers. In Phase 2, 283 Grade 11-12 students were given the Version 3 10-item instrument, with five experienced A-level teachers and two tertiary chemistry educators providing expert feedback. The final version of the IEDI was administered to Grade 11 (N = 777) and Grade 12 (N = 202) students from 17 institutions in Singapore (N = 979 total); 32 Grade 11-12 students were additionally interviewed.

Several aspects of validity and reliability have been assessed for the data generated by the IEDI [1-3]. Interviews were conducted to provide evidence based on test content, with A-level teachers consulted in both phases of instrument development and tertiary-level chemistry teachers consulted in the second phase. Evidence based on response process validity was gathered via Grade 11-12 student interviews, conducted at several time points throughout instrument development as well as after the instrument was finalized. In terms of evidence for reliability, coefficient alpha was reported to estimate single administration reliability, with a single value of 0.52 reported for all 10 IEDI items (N = 979). For item difficulty, facility indices (FI) were reported, ranging between 0.05 - 0.48. For item discrimination, discrimination indices (DI) ranged between 0.11 - 0.67, with eight items showing good discrimination (DI > 0.30). Reliability of data generated by the IEDI has only been reported within reference [1].

Recommendations for use   
The IEDI was intended to measure alternative conceptions related to ionization and understanding of the trend of ionization energy across the periodic table, originally developed within the context of A-level students (16 to 19 years old, high school students) in Singapore [1-3]. It is important for potential users of the IEDI to consider this development within a highly specific context, as the implications for validity and reliability of data may not directly translate to other contexts. Future researchers and educators should provide evidence of validity and reliability in new contexts when possible. Additionally, while the IEDI was thoughtfully developed in several stages, using expert reviewers (evidence based on test content validity) and student interviews (evidence based on response process validity) to revise and pilot several versions of the instrument to its final stage, the developers reported a single IEDI score for the 10 items without evidence of internal structure validity. It is also currently unclear how the IEDI score is calculated (i.e., does each tier have associated points for answer/reasoning?). Therefore, future users should consider the various ways in which items from a 2-tier assessment can be scored and the implications of each method.

Details from panel review   
It is worth noting that reference [1] is the initial detailed development of the IEDI, written as a dissertation/thesis, while references [2] and [3] are peer-reviewed publications of the same data reported in reference [1]. Reference [2] contains more detailed information on the qualitative interview data collected with both students and educators, while reference [3] contains more detailed information on the quantitative data collected.

References

[1] Tan, D. K. C., Khang, G. N., Sai, C. L., Taber, K. (2005). DEVELOPMENT OF A TWO-TIER MULTIPLE CHOICE DIAGNOSTIC INSTRUMENT TO DETERMINE A-LEVEL STUDENTS’ UNDERSTANDING OF IONISATION ENERGY, National Instituion of Education, Nanyang Technological University, Singapore.

[2] Tan, K.C.D., Taber, K.S., Goh, N.K., Chia, L.S. (2005). The Ionisation Energy Diagnostic Instrument: A Two-Tier Multiple-Choice Instrument to Determine High School Students’ Understanding of Ionisation Energy. Chem. Educ. Res. & Pract. 6(4), 180-197.

[3] Taber, K.S., Tan, K.C.D. (2007). Exploring Learners’ Conceptual Resources: Singapore A Level Students’ Explanations in the Topic of Ionisation Energy. Int. J. Sci. Math. Educ., 5(3), 375-392.