Starting A Doctoral Program in Assessment and Measurement

To assist other universities interested in starting a doctoral program in assessment and measurement, we have compiled a list that we hope will guide others in their endeavors. We begin this document with a brief description of the characteristics of the institution offering this program. We then move on to some recommendations in the areas of faculty-student involvement and leadership, academic requirements, graduate student opportunities, and issues around funding.

Putting Things in Context

What are some general suggestions for institutions offering the doctoral program?

• Offer advanced assessment techniques (i.e., IRT, generalizability theory, computer-based testing) practiced in a broad range of areas such as student affairs, student development, general education, and academic departments.
• Offer opportunities for applied experiences that are closely aligned with institutional assessment efforts.
• Develop strong relationships with local businesses, hospitals, and social science organizations to provide alternative practicum experiences in different accountability contexts (e.g., healthcare, business, non-profit).
• Develop contacts with other university faculty and staff who are interested in applied assessment.

Recommendations

What are some recommendations for faculty and doctoral program leaders?

• Participate in frequent conversations with institutions offering similar programs in measurement and higher education policy. Many new lessons are learned each year when starting a doctoral program. Time and effort can be saved if this information is shared.
• Develop a clearly stated mission, focus, and learning objectives. Faculty and staff should have a shared vision for where the program is headed and assumptions concerning what will be learned should be clarified through written learning objectives for each class.
• Encourage an apprenticeship model of training. Faculty members should be willing to work with doctoral students and share their area of expertise within and without the classroom setting. Be willing to include students in your work, research, and consulting relationships when at all possible.
• Expose students to off campus consulting experiences, as an effective model of a consulting relationship.
• Maintain forward thinking in technological applications. Many new technological applications can be applied to assessment and measurement. For example, seek out information on the newest software for processes such as qualitative data collection, online surveys, and online assessments with various media.
• Engage students in thinking about the research agenda for the assessment and measurement field. This is a young field without a well-defined research agenda. Faculty should assist students in evaluating the current research trends and challenge them to consider new directions for research.
• Model the role of effective researchers. For example, faculty should talk about research issues and discuss scholarly journals with students.
• Expose students to local and national professional organizations (e.g., National Council on Measurement in Education, American Association of Higher Education, American Psychological Association, National Alliance of Business).
• Begin planning early for establishing internship sites. Often the potential site requires several months to obtain approval and funding for the internship position.
• Consider the quality of experience and supervision that different internship sites can provide.
• Develop specific policies and procedures regarding all aspects of the doctoral program (i.e., admissions, academic progress, evaluation procedures, dismissals). Do not be afraid to create policies for the worst-case scenario. Revisit your policies and procedures to discuss what is working and what could use revision. Discussing these issues with other similar programs could be helpful as well.
• Establish evaluative procedures for professional development.
• Publish a student handbook explicitly stating the policies, procedures, and evaluative processes of the program. Annually revise and clarify procedures and expectations outlined in the student handbook.
• Document decisions and their rationale as policies and procedures are established for the doctoral program. As the policies and procedures are revisited or as program leadership changes, rationale for prior decisions may be helpful.
• Develop policies for graduate students working with assessment data to protect student confidentiality. Typically, it is helpful to have the student check with a faculty member before any assessment information is exported to an outside source.
• Develop explicit policies regarding distribution of reports authored by students-in-training.
• Have a specific recruitment plan. Determine what type of students you are targeting and where they are located. If you are only accepting students with specific Master's degrees, learn what institutions in your area offer these degrees.
• Emphasize and evaluate interpersonal communication skills; assessment practitioners deal with individuals, groups, and a variety of audiences.

What are the academic requirements of the institution offering this doctoral program?

• Interpersonal communication in course work and applied experiences; evaluate out-of-class interactions every semester.
• Course work in classical test theory.
• Course work involving modern measurement techniques including item response theory, generalizability theory, and structural equation modeling.
• Course work in test development.
• Course work that involves understanding how students learn including cognitive development.
• Exposure to various theories of college impact including psychosocial and other student development theories.
• Knowledge of the assessment practitioner's role in consultation.
• Relationships with other departments to find common interests.
• Research agendas for the assessment and measurement field.
• Application of assessment data to course work.
• Development and utilization of computer based tests.
• Public policy issues of accountability including the origin and nature of mandates and state policies.

What opportunities are available for graduate students?

• Supervised learning using the scholar-practitioner model.
• In-depth and applied experiences working with faculty across the campus.
• Participation in local and national workshops and conferences.
• Interactions with other graduate students on campus.
• Independent and collaborative research with faculty.
• Training in advanced measurement and statistics software (i.e. item response theory, generalizability software, structural equation modeling software).
• Development of a professional portfolio of accomplishments, processes, and products.
• Provisions made for student led workshops in item writing, standard setting, and student development assessment.
• Meetings with assessment liaisons for each department.
• Involvement in the administration of an annual Alumni Survey.
• Sharing of information concerning the strategies of assessment implementation. Educate students about the history of the university's assessment plan.
• Assisting university departments with developing the assessment portion of their annual reports.
• Discuss state assessment and accreditation requirements and discussion of the university-wide impact.
• Expose students to meetings with stakeholders such as parents and government officials.

What are funding considerations?

• Allot money for students' participation in and travel to local and national professional conferences.
• Determine availability of graduate assistantships. Availability of graduate assistantship often is the deciding factor for a student to select a graduate program.
• Plan for graduate student office space, computers, and software.
• Allot time/money for program faculty member to supervise the student internship process. Student supervision onsite and at external sites takes a great deal of faculty time, energy, and funding.

Supported in Part with a Grant from the Fund for the Improvement of Postsecondary Education