Dr. John Hathcoat is an Assistant Professor in Graduate Psychology and Associate Director of University Learning Outcomes Assessment in the Center for Assessment and Research Studies at James Madison University (JMU). Hathcoat has a background in educational research and evaluation. He engages in research on a variety of topics including validity theory and applications of performance assessment. Recently Hathcoat’s advisees, psychological sciences master’s student, Nikole Gregg, and assessment and measurement doctoral student, Courtney Sanders, had the opportunity to speak with him about his current assessment projects.

Currently, Hathcoat works with student affairs and general education programs at JMU on student learning outcomes assessment. Earlier this year, he also began working with the Multi-State Collaborative to Advance Learning Outcomes Assessment (MSC). The MSC is a partnership between the Association of American Colleges & Universities (AAC&U), the State Higher Education Executive Officer’s association (SHEEO), and 13 state higher education systems. In a recent conversation with JMU students, Dr. Kate McConnell, Senior Director for Research and Assessment at AAC&U, highlighted the importance of collaboration and the contributions she believes Hathcoat can make to the MSC initiative.

Hathcoat’s collaboration with the MSC began as a result of concerns he raised last year in the Chronicle of Higher Education’s article on measuring student learning. The MSC uses different assignments to measure a common standard and Hathcoat questioned the validity of this process. AAC&U recognized the merit of Hathcoat’s concerns along with his expertise in validity theory and performance assessment and decided to partner with him.

Hathcoat characterizes himself as a “supportive skeptic” of the MSC. He insists higher education researchers and stakeholders must be extremely cautious when making broad claims about data from the MSC before justification of those claims through research. However, he also believes that the MSC is an exciting initiative because it provides the opportunity to collect data on a largescale from a variety of institutions and also aims to address important research questions. Some questions Hathcoat plans to help the MSC address include: What are the best sampling strategies to collect students’ assignments?, When is it justifiable to compare institutions?, and When is it appropriate to make broad claims about how well a state is doing across multiple institutions?

This year (2016-17) JMU decided to participate in the MSC along with other schools in Virginia, and Hathcoat is the project lead for JMU. He believes JMU is in a prime position to participate in this initiative because of the existing infrastructure to support assessment on campus. He asserts, “We have assessment day, multiple graduate assistants, and faculty who assist with assessment in various capacities… and [the MSC] could be a way to supplement things we currently have.”

Hathcoat believes he and his research team can contribute to the MSC in two areas: 1) investigating the validity of the inferences about the scores obtained from the VALUE rubrics, and 2) examining the characteristics of various assignments collected from institutions. Investigating the validity of inferences will help the MSC determine whether comparisons between institutions are an appropriate use of the scores. Hathcoat hopes examining characteristics of the assignments will help the MSC determine how to define assignment quality, which can later be incorporated into assessment design workshops and screening procedures.

In addition to his assessment work at JMU and with the MSC, Hathcoat also trains many graduate students to work in higher education and measurement fields. When asked about the skills and values he wants to students to have, Hathcoat focused on three areas. He stressed continuous learning, communication, and the importance of valuing skeptical, critical thought for the advancement of knowledge. The fields of educational and psychological testing continue to advance at a rapid pace, and Hathcoat contends that courses taken during an academic program can familiarize students with specific topics, but it is more important that students learn how to learn. When students graduate and begin their professional careers, they will be confronted with situations that can’t be found in a textbook. Hathcoat claims it is therefore important that students are creative, flexible, and able to learn continuously as their professions require.

Hathcoat also highlighted the importance of communication skills. That is, students should learn how to convey technical information to individuals who may have expertise in other areas. Clear and effective communication is necessary to accomplish initiatives at an institution, organization, or politically. Finally, Hathcoat emphasized that he wants students to leave JMU with a critical, but open mind to new ideas and insights. He stated, “The people who critique my ideas, my views, my positions, are more of a friend to me than those who do not provide such feedback”. Hathcoat maintains that students should be open to constructive criticism because it can help them refine, and ultimately improve their thinking and rationale.

Hathcoat underscored the importance of evidence-based decision making and being cautious about the inferences we make from data throughout the conversation. He is optimistic about the work the MSC is doing and the influence that he and faculty members from other institutions can have on the ongoing initiative. Hathcoat is excited about the movement in higher education to collaborate with other institutions and states to advance student learning outcomes assessment and he hopes that he and his students can make valuable contributions to the assessment work at JMU and nationally.

See the associated interview with AAC&U’s Kate McConnell: link

Back to Top