James Madison University
Office of Institutional Research

Research Notes

Volume 16, Number 1

March, 2002

Performance Measures In Higher Education, Virginia, And JMU

The importance of collecting and reporting data about the performance of higher education institutions has increased dramatically in the last ten years. According to the State Higher Education Executive Officers (SHEEO), at least 21 states require some type of annual accountability reports that include performance measures. A report by the Nelson Rockefeller Institute of Government indicated that 28 states, including Virginia, had or were intending to adopt some form of budgeting based, in part, on performance measures.

The emergence of the importance of performance measures is predictable. State governments, taxpayers, students, and parents demand evidence that their funds are spent efficiently and on the most important outcome--learning. If a parent is charged $10,000 or more per year in Virginia to educate his or her son or daughter at a public four-year institution, it is only natural that evidence of quality is desired. For FY 2002 the General Assembly approved more than $1.4 billion in General Fund appropriations for higher education instruction and student aid. It is understandable that the Commonwealth expects evidence of performance. The trend is for increased accountability information.

If defined well, performance measures may provide taxpayers, students, and parents some useful information upon which to make informed decisions about college choice. Performance measures can add rationality to the ways in which college choice and funding decisions are made.

The major concern with most performance measures is that they tend to focus on data which can easily be obtained. Data such as graduation and retention rates, percent of dollars spent on instruction, credit hours taught per faculty member, and percentage of class sections taught by full-time faculty are relatively easy to obtain and report, but do not focus on the purpose of higher education--learning. Seldom can one find learning measures in lists of state-wide performance measures. National ratings like U.S. News & World Report do not focus on learning. Instead they focus on input measures and the reputation of institutions as measures of institutional quality. If an institution enrolls high quality students (as measured by SAT scores), has significant resources, and is perceived by other presidents as being a high quality institution, it will be ranked at or near the top of U.S. News & World Report's listings.

Performance Measures In Virginia

In 1999 the Governor's Blue Ribbon Commission on Higher Education recommended that the State Council of Higher Education for Virginia (SCHEV) develop a means to report on the performance of higher education at each public institution. SCHEV, in cooperation with the institutions, developed Reports of Institutional Effectiveness (ROIE). Mandated by law, ROIE focuses on:  accessibility (percentage of applicants admitted, institutional size, and student demographics); affordability (cost of tuition and fees, percentage/number of students awarded financial aid); quality (associated measures of quality such as faculty credentials and class size, and, in some cases, direct measures of student learning); and efficiency (appropriations and expenditures, management standards).

ROIE measures consist of three sections on their Web site: profile measures (descriptive statistics about the institution); system-wide measures (fourteen measures of operational efficiency and academic quality); and  institution-specific measures (eleven measures contributing special insights and context to the understanding of the institution and what it values).  You are invited to visit the ROIE Web site listed below.

One of the major differences between JMU's institution-specific measures and those of the other institutions is that JMU's measures focus on learning. JMU's measures include:

  • Information-Seeking Skills Test
  • Basic Technological Skills Test
  • Writing
  • American History and Government
  • Critical Thinking Test
  • Oral Communication Test
  • Wellness and Human Development
  • Arts and Humanities
  • Quantitative Reasoning Test
  • Natural World/Science Test 
  • Global Experience Test

JMU Performance Measures

JMU continues to collect and report on a variety of performance measures through different media. The Center for Assessment and Research Studies (CARS) provides learning outcome data for the ROIE. The Office of Institutional Research collects profile and system-wide data for ROIE and other data for internal monitoring, including a quarterly report card of performance measures. The Division of Institutional Effectiveness maintains a University Portfolio, a Web site that contains a profile of JMU and evidences of JMU's effectiveness.

In the future it is expected that divisions and units will be expected to develop and post performance measures. These may be incorporated into the JMU Planning Database for ease of collection and dissemination to the university community.

In summary, performance measures will be an important aspect of the measurement of effectiveness of higher education for many years. JMU, especially with its learning measures, is one of the leaders in the collection and dissemination of performance measures. The reader is encouraged to visit the links below to view the different types of performance measures for JMU and other institutions.

SCHEV Reports Of Institutional Effectiveness (JMU only) http://research.schev.edu/roie/four_year/JMU/body.asp?m=1&p=1&s=1&i=1
ROIE (all public institutions) http://research.schev.edu/roie/
JMU Portfolio http://centennial.jmu.edu/portfolio/index.shtml
Quarterly Report Of JMU Performance Measures http://www.jmu.edu/instresrch/Performancemeasures/index.htm
JMU Peer Measures http://www.jmu.edu/instresrch/peers/faculty_salaries&group_comparison.htm
State Accountability Reports http://www.sheeo.org/account/acct-reports.htm
Washington Monthly: Playing With Numbers: How U.S. News Mismeasures Higher Education And What We Can Do About It http://www.washingtonmonthly.com/features/2000/0009.thompson.html
National Center For Educational Statistics http://nces.ed.gov/ipeds/cool/InstDetail.asp?UNITID=232423
Nelson Rockefeller Institute of Government http://www.rockinst.org/quick_tour/higher_ed/current_projects.html

Back to Research Notes

Back to Facts and Figures