Skip to Main Content

2017

You are in the main content


Courtney Sanders, a doctoral student in the assessment and measurement program, interviewed Dr. Ross Markle and Dr. Javarro Russell from Educational Testing Service (ETS). Markle and Russell are senior research and assessment directors in ETS’s higher education division. It is their job to work with institutions to help them better understand ETS’s assessments and how to use assessment results. Prior to their roles at ETS, both Markle and Russell graduated from JMU’s assessment and measurement Ph.D. program. Throughout the interview, they shared their experiences at ETS and goals of their current projects.

Russell’s role focuses largely on student learning outcomes, while Markle’s primary focus is student success. ETS is generally known as a standardized testing organization (e.g., they own the GRE, and do a great deal of operational work on the SAT and other assessments owned by the College Board), however, Markle and Russell are trying to shift the focus of ETS within colleges and universities to one of collaboration. They are working with institutions to build assessment processes and help faculty and staff use the data from assessments.Russell

They have faced two major challenges in their work. First, internally, they have helped shift the culture at ETS to focus more on a process of student learning outcomes assessment, rather than simply developing and providing tests. Second, Markle and Russell are working to change the external perception of ETS at colleges and universities. While ETS has done great work in creating high quality standardized tests, the higher education division is now focused on helping institutions use the results from those tests to make appropriate inferences.

ETS has also launched a new wave of student learning outcome assessments via the HEIghten suite. The HEIghten assessments include critical thinking, quantitative literacy, and written communication. Assessments related to civic competency & engagement and intercultural competency & diversity are expected to be released in 2017. ETS has recently partnered with JMU to pilot some of the HEIghten tests. Markle and Russell want to ensure institutions are benefitting from their assessments, so they decided to partner with institutions that have the infrastructure in place to be successful. JMU not only has a strong assessment culture and processes in place, but also an interest in making improvements to student learning based on data.

Russell asserts that “people are looking for proof that [student learning outcomes assessment] works.” He hopes that showing them proof from JMU and additional institutions will help others be more engaged in the process. Russell recognizes it may be difficult for other universities to exactly duplicate JMU’s assessment processes. However, he believes many best practices from JMU can translate to other institutions. Markle added that case studies from colleges and universities such as JMU are necessary. That is, providing examples will show others that ETS’ philosophy of using results for student learning improvement can be successfully applied. Markle and Russell have started to receive positive feedback from institutions. These institutions have recognized that ETS creates high quality assessments. Institutions have also expressed appreciation for Markle and Russell’s ability to communicate and engage with faculty beyond just the technical aspects of the assessments. They hope that positive results come from these institutions about use of results from the assessments.

In addition to the collaboration on the HEIghten assessments, ETS has also hired Kristen Smith (third-year assessment and measurement doctoral student) as an intern. This internship occurred as a result of conversations between ETS and Dr. Sara Finney, who is a JMU faculty member and ETS’ GRE Technical Advisory Committee chair. During her internship, Smith is working with Russell and Markle to develop user guides for some of the HEIghten tests. The guides help institutions use the tests to think about students and the whole process of assessment – from student learning outcomes to using results to increase student success. Working with Smith, they also hope to create an exemplar report that addresses best practices in using the assessment, displaying data, and a rationale for explaining differences in students’ scores on the assessment.

When asked how their experiences as graduate students at JMU helped prepare them for their current work, both Markle and Russell emphasized the communication skills they honed while they were in the assessment and measurement doctoral program. Markle also discussed the importance of quality in his work. That is, he learned that it is best to spend more time and resources on processes on the front end, rather than try to fix things later on.

Markle

Markle and Russell also offered advice for current graduate students in the assessment and measurement program. Markle highlighted the importance of refining skills and experiences while in graduate school, because it will benefit students’ professional careers later. He also mentioned that students should stay abreast of what is going on nationally in education and measurement fields. Having current knowledge will enable graduates to be in a stronger position to best help and learn from others they work with. Russell described the importance of taking a step back, listening to faculty, and letting them know you are working with them to accomplish a common goal. He has found this strategy useful in his consultations with others throughout his career.

Throughout the conversation, Markle and Russell stressed the importance of collaboration and using available resources. They underscored the work JMU has done in relation to assessment practice and how this couples nicely with the concept of use of results they are currently trying to advocate with institutions. Their emphasis on information sharing and relationship building are insights Sanders is sure to remember as she looks to further her career in educational assessment. 

See the associated interview with JMU’s Dr. Sara Finney.