Finney, Wells & Henning call for Program Theory and Implementation Fidelity in Assessment 

Summary: In this NILOA article, Finney, Wells, & Henning emphasize the importance of articulating program theory (why/how your programming should work) and collecting implementation fidelity data (what programming students actually received). They situate program theory and implementation fidelity within the typical outcomes assessment cycle. They then provide guidance on the process of building should-be-effective programming and assessing how well that programming was implemented. 

Abstract:

On their own, student learning and development outcomes assessment data have limited utility for improving programming. We believe outcomes data should not be collected until two fundamental questions can be answered: “Why should this programming result in the desired outcome?” (i.e., program theory) and “Was the intended programming actually experienced by students?” (i.e., implementation fidelity). Some assessment professionals may find this proclamation radical. Our call is fueled by the creation of unjustified programming and curriculum, coupled with the collection of outcomes data that are not used for improvement efforts. We contend that it is only after program theory is articulated that faculty and student affairs professionals can collect relevant, useful outcomes data. Moreover, valid inferences from outcomes data are contingent on knowing what programming students experienced. This “expanded” assessment practice has potential to afford better-designed, more impactful, research-informed programming to students. As our students have opportunities to engage in well-implemented, should-be-effective programming, their learning should demonstrably improve. Thus, we call for professional standards and professionals themselves to integrate program theory and implementation fidelity into outcomes assessment practice.

Finney, S.J., Wells, J.B., & Henning, G.W. (2021). The need for program theory and implementation fidelity in assessment practice and standards (Occasional Paper No. 51). Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA).

For more information on program theory and implementation fidelity, please see resources on the SASS website.