Postsecondary Instructional Practices Survey

(Post last updated June 16, 2022)

Review panel summary

The Postsecondary Instructional Practices Survey (PIPS) was developed as a self-report instrument that would allow researchers, faculty developers, and others to learn about the way that faculty members teach. The PIPS lists 24 instructional practices and asks faculty members to report the extent to which these practices describe their teaching on a 5-point scale [1]. There is evidence to support that the test content of the PIPS adequately reflects its intended broad range of instructional practices, with a group of experts reviewing the items in the development phase. Additional evidence to support the interpretation of PIPS scores based on test content was collected in a pilot sample of postsecondary instructors, which is valuable because this is the intended audience. There is some concern in the overall evidence for the internal structure of the PIPS data. In the original development of the instrument, there were two different factor structures and associated scoring schema proposed: a two-factor solution containing instructor-centered and student-centered practices, and a five-factor solution containing student–student interactions, content delivery practices, formative assessment, student–content engagement, and summative assessment [1]. Subsequent studies utilized the five-factor model instead of the two-factor model more often [2-4]; the two-factor model was only used in one other study [4]. None of the other studies conducted any analyses to support the internal structure of the data collected with the instrument. While all studies using the PIPS provided coefficient alpha values as evidence in support of single administration reliability for the entire scale and each subscale used, this information is not sufficient to support the consistency of scores since the number of factors is not well-established. There is also evidence to support PIPS scores’ relations to other variables; scores on the PIPS have been found to be related to course and instructor characteristics, such as the number of students enrolled in a course and the number of years that a faculty member has been teaching, as well as years at their own institution [1].

Recommendations for use

The context of application for the PIPS is recommended by the instrument developers to be used across disciplines. There is evidence of its use in chemistry-specific environments [2, 3], however, neither of these studies had a large enough sample from which to draw psychometric evidence. Therefore, additional psychometric evidence supporting the data generated by PIPS would be warranted before its wide use to measure instructional practices. The PIPS is only available in English; while not translated, the instrument has been used in a Chinese population with an available translator for ad-hoc needs [4], however, there is currently no evidence to support using the instrument in another language.

Details from panel review

The questions on the PIPS were designed to measure four general areas of instruction: instructor-student interactions, student-content interactions, student-student interactions, and assessment. However, after administering the instrument, the developers conducted both exploratory factor analysis and confirmatory factor analysis, finding evidence for both a two-factor and a five-factor (where formative and summative assessment are separate) internal structure. It is uncommon to use multiple factor models with data generated from one instrument without robust justification.

References

[1] Walter, E.M., Henderson, C.R., Beach, A.L., & Williams, C.T. (2016). Introducing the Postsecondary Instructional Practices Survey (PIPS): A concise, interdisciplinary, and easy-to-score survey. CBE–Life Sciences Education, 15(4), ar53. https://doi.org/10.1187/cbe.15-09-0193

[2] Barlow, A., & Brown, S. (2020). Correlations between modes of student cognitive engagement and instructional practices in undergraduate STEM courses. International Journal of STEM Education, 7, 1-15. https://doi.org/10.1186/s40594-020-00214-7

[3] Houseknecht, J.B., Bachinski, G.J., Miller, M.H., White, S.A., & Andrews, D.M. (2020). Effectiveness of the active learning in organic chemistry faculty development workshops. Chemistry Education Research and Practice, 21, 387-398. https://doi.org/10.1039/C9RP00137A

[4] Du, X., Kolmos, A., Hasan, M.A., Spliid, C.M., Lyngdorf, N.E.R., & Ruan, Y. (2020). Impact of a PBL-based professional learning program in Denmark on the development of the beliefs and practices of Chinese STEM university teachers. International Journal of Engineering Education, 36(3), 940-954.