What We're Learning

What constitutes learning improvement on a college campus? And how do we provide evidence for it? It depends who you ask. JMU uses a more narrow definition (still in a working draft stage): 

Strong evidence, from direct measures, supporting substantive learning improvement due to program modifications. This program responded to previous assessment results, made curricular and/or pedagogical modifications, RE-assessed, and found that student learning improved. The rationale and explanation of the modifications leading to the change are clearly laid out. The methodology is of sufficient strength that most reasonable alternative hypotheses can be ruled out (e.g., sampling concerns, validity issues with instrument or student motivation). In essence, the improvement interpretation can withstand reasonable critique from faculty, curriculum experts, assessment experts, and external stakeholders.

(Fulcher, Sundre, Russell, Good, & Smith, 2015)

Other institutions and organizations take a broader approach. In April 2017, national leaders in higher education assessment discussed the complexity of defining 'improvement.' Not surprisingly, little consensus was reached during their brief, introductory, discussion.

We oftentimes use the word improvement when we really mean change. For example, if a program changes the structure and content of course(s) to include more writing practice, many would say, "the program made improvements to the writing assignments." What we're learning is that not every change leads to student learning improvement. Baseline data collection and follow-up assessment provides evidence for actual writing skill development. 

Many institutions of higher education require (or encourage) academic programs to submit annual assessment reports. Although we condone annual data collection and program review, we recognize that restructuring an entire program to accommodate new modes of teaching a student learning outcome is an in-depth process; one that's effects can certainly not be captured in a one year assessment cycle. When speaking to programs about LID, we try to keep the big picture in mind. See Learning Improvement Stages for more information.

Although a national definition of learning improvement may not have emerged at the national summit, leaders were in consensus about the benefits of joining the efforts of faculty and educational development offices with assessment offices on campus. Simply changing the assessment of a program does not change what students know, think, or are able to do. Instructors participating in the LID project are asked to change the way information is taught. Educational development offices like JMU's CFI provide support for institutional change and invaluable resources for academic programs. 

See Learning Improvement Stages for more information regarding our suggested next steps to learning improvement by design.