Standard 4 Addendum:
Evidence for the OnSite BOE Team to validate during the onsite visit:
4.3 The BOE team indicated that the AFIs from our previous visit continue. The first two requests for more information specifically focus on the continuing AFIs and address areas of concern related to continuing to meet the standard.
1. Unit programs inconsistently use rubrics that delineate and assess candidates' knowledge and skills necessary to work effectively with culturally and linguistically diverse children and those with exceptionalities.
Rationale: Rubrics for a variety of programs either lack criteria related to working effectively with diverse learners and/or focus on the process of writing, preparing and presenting projects but not on specific criteria related to working with diverse learners, including those that are ELLs or SWEs. Many rubrics lack criteria related to working effectively with ELLs or SWEs.
As part of the state program review process, all programs identified those courses and field experiences that addressed competencies related to working with diverse learners, including students who are identified as English Language Learners and/or as having a disability, as defined in the VDOE professional studies requirements. The matrices that were submitted as part of that program approval process are in Exhibit 1.3.a: Virginia Department of Education Endorsement Competencies. The review process required programs to submit syllabi with evidence that these competencies were covered in the referenced courses. Links to those syllabi can be found in in IR Exhibit 4.3.b. The syllabi in these exhibits were those submitted as part of the program approval process. Updated syllabi will be available to the BOE team during the site visit.
In IR Exhibit 4.3.c, aggregated data for program key assessments are provided. Each of these key assessments measure candidates' abilities to impact student learning by designing instructional environments that are responsive to identified needs and differences for students. In addition to the program key assessments listed on the chart and displayed in the data tables for IR Exhibit 4.3.c, Key Assessment Data (Initial); Key Assessment Data (advanced), all initial programs measure candidates' competencies in a clinical setting using the Assessment of Student Teaching (ST-9). Aggregated data for items on the ST-9 identified by the unit as measuring candidates' diversity competencies are shown in IR Exhibit 4.3.c: Clinical Experience Data.
While these key assessments are important to our assessment of candidates' knowledge and skills, they provide only a snapshot of our work. Across the unit, programs address competencies related to diversity, including working with students who are English Language Learners and Students with Exceptionalities in a number of classes and field experiences. For example, the foundation course for initial programs in the unit, EDUC 300 Foundations of American Education, is our candidates' first opportunity to explore their own personal identity and perspectives on race, ethnicity, socio-economic, second language learners and other aspects of diversity and is required of all candidates in the initial licensure programs. Standard 4 Addendum Exhibit 1: Syllabi, Assignments and Rubrics presents how competencies related to diversity are addressed and assessed in EDUC 300 (taken by all candidates in initial licensure programs) for one specific assignment. As can be seen in the course syllabus, a wide range of course assignments designed to support candidates' understanding of diversity provide insights into understanding our unit philosophies and practices for supporting all learners. The visiting BOE team would be welcome to visit with the faculty who teach this critical course to gain more insights and understanding of the course goals and expectations we have of candidate performance.
Across the unit there are other courses that have targeted assignments designed to develop candidates' knowledge and skills related to working with diverse learners and a sample of those courses is also listed in Standard 4 Addendum Exhibit 1 Syllabi, Assignments and Rubrics. Representative samples of the courses woven throughout our programs include EDUC 310, a course required of our middle, secondary and exceptional education programs, ECED 511, a required course in our Early Childhood MAT program, MAED 628 a required course in the advanced math specialist program, and MSSE 625, a required course in our secondary education licensure areas are also included in Standard 4 Addendum Exhibit 1: Syllabi, Assignments and Rubrics.
An example of how courses across the curriculum provide the foundations and building blocks for the candidates in acquiring the knowledge, skills and dispositions to work with diverse learners can be presented by describing the Middle and Secondary programs. The TWS key assessment serves as a capstone opportunity for candidates to demonstrate their knowledge and skill level. Courses leading up to the development and implementation of the TWS provide the foundational knowledge and skills.
For example, all middle and secondary education candidates enroll in MSSE 370 General Methods. One of the assignments is the development of a lesson plan to meet the diverse student needs, specifically the needs of ELLs, struggling learners, advanced students and students with special needs. This simulated activity requires the candidates to apply pedagogical knowledge to their content and the accompanying rubric assesses their appropriate application, understanding, and synthesis. Standard 4 Addendum Exhibit 2: MSSE 370 provides the components of this activity for review, including the rubrics.
These skills are extended in EXED 520 Differentiation and Collaboration. The range of topics covered in the class is extensive, addressing the strategies of universal design to meet the needs of all learners. Again there is a specific focus on students with special needs, requiring the candidate to integrate learning across content and contexts. An integral component of the final project is the development of an ELL and exceptionality accommodations plan. Standard 4 Addendum Exhibit 3: EXED 520 provides documentation of the specific expectations for the candidates and the rubric used to evaluated their performance.
The TWS is a comprehensive assessment of the candidates' abilities to fully integrate the knowledge and skills acquired over their programs in middle and secondary education. It is designated as a key assessment and is aligned with not only our conceptual framework but also professional SPAs demonstrated in Standard 1 Addendum Exhibit 4 Alignments-Initial Programs. Standard 4 Addendum Exhibit 4: TWS Alignments show the alignments of each of the rubric components with spa standards.
Advanced programs also integrate opportunities for candidates to explore their perceptions, understandings and practices in supporting diverse learners. The advanced program in School Psychology is an example of how advanced programs support their candidates meeting the diversity competencies of the unit. As articulated in the School of Psychology Handbook, The James Madison University School Psychology Program's Culturally Competent Practitioner Initiative (CCPI) was developed in response to calls by the major national professional organizations of school psychologists (National Association of School Psychologists [NASP] and the American Psychological Association [APA]) to train professionals who are equipped to provide services to children and families whose cultural beliefs, values, and expectations are different from the mainstream. (Standard 4 Addendum Exhibit 5: School of Psychology Handbook) The curriculum of the School Psychology program integrates experiences across the curriculum that support candidates academic and clinical knowledge, skills and dispositions in working with children and families whose culture is other than their own.
Rationale: The number of students evaluated on instruments related to the diversity outcomes is frequently less than 15 and sometimes less than five. It is not clear if the number of students evaluated on each assessment represents a substantive sample of the candidates enrolled in each program. The reports of candidates who met or exceed criteria are confounded by instruments lacking criteria related to working effectively with diverse learners, especially ELLs and SWEs.
Data reported in the key assessments reflect the numbers of candidates that were evaluated in any given semester. The linkage between courses and key assessment results in the evaluation of cohorts of candidates at any given time and does not reflect the total numbers of candidates in the programs. For example, in IR Exhibit 4.3.c, aggregated data for unit and program key assessments are provided. The unit assessment data would reflect ALL candidates in any given program for a specific time period on the UNIT instrument, the ST-9. Program assessment data are reported each semester only on those candidates who were evaluated during that specific semester using the PROGRAM key assessment.
Candidates progress through the program in cohort groups. Key assessments are often linked to specific classes and the size of the individual sections of a class varies depending upon the size of the cohort. Some cohorts may have an enrollment of 5 in one class; others may have three sections of 25 candidates per section. For the table used for IR Exhibit 1.3.c-h, the number in each cell represents the candidates in each program cohort who attempted the key assessment during the specified semester. All candidates complete the key assessments that are embedded in the class in which they are enrolled.
4.4 The following responses provide information and more evidence for the Onsite BOE Team to validate during the onsite visit
1. Design, implementation, and evaluation of curriculum and experiences: What are the assessments that are being developed to evaluate candidates meeting the new diversity outcomes? When are they implemented?
As referenced in the IR, two diversity competencies were identified for use across the unit: 1) Our candidates use the latest diversity, equity and social justice research, including culture, socio-economic status and language; 2) Our candidates create learning opportunities that support intellectual, social, and personal development.
To assess the first competency, the diversity committee recommended administering a cognitive (knowledge-based) test that will evaluate whether candidates learned the key principles of current research in the field. The development of this instrument is underway and will likely be ready for piloting in fall 2012 and spring 2013.
The unit's diversity task force proposed to PECC that the Multicultural Dispositions Index (MDI; Thompson, 2011) be administered via Qualtrics to assess competency 2. The MDI was piloted with 53 candidates at the beginning of EDUC 300 in the spring 2012 semester. In addition to answering the items, respondents were invited to provide feedback on the relevance and clarity of the questions. Feedback data were aggregated and items were modified. These individuals will receive a post-test in December 2012 via Qualtrics. At that point the pre- and post-test data can be reviewed to see whether the instrument can pick up change over time. To preserve confidentiality while allowing for matching pre- and post-test scores, candidates were asked to provide their first pet's name as their User ID. There were no repeated names in the 53 pilot cases.
The PECC supported the recommendation to use the MDI and beginning Fall 2012. The candidates in initial licensure programs will provide responses to the questions on the MDI four times during their program: admission, EDUC 300, once prior to beginning student teaching and once at the end of their program. The Director of Assessment will send an email containing the survey link to all candidates who are registered for the course. Taking the survey will be a course requirement; their responses will be anonymous but the system will provide information about which candidates completed the survey and which did not.
Advanced programs will administer the assessment three times: admission, midpoint in the program and at the end of the program.
Copies of the MDI as well as the adaptations to the form that we will implement can be found in the Standard 4 Addendum Exhibit 6: MDI.
2. Design, implementation, and evaluation of curriculum and experiences: Clarify the ways in which and degree to which candidates in programs that have professional reviews develop and meet diversity outcomes.
Diversity competencies of candidates who complete programs that have professional reviews are measured across the programs' integrated curriculum. Assurances that candidates are meeting respective professional expectations for work with diverse populations are provided in the accreditation documentation. In addition, because the faculty who teach in those programs are members of the PECC, they are actively engaged in the conversations surrounding the development and assessment of diversity outcomes for our candidates.
For example, working with students with special needs is emphasized in the Music Education program. Each semester candidates have to opportunity to learn from Dr. Alice Hammel, a nationally recognized leader in the are of music for students with special needs. Dr. Hammel is in residence at JMU one week each semester and works with all the music methods courses. The candidates are required to select a concept to teach, create a learning sequence for the concept, and for each step, demonstrate how they would differentiate for a student for whom the goal might not be appropriate, identifying possible accommodations for all students who might need them. Hammel's work is not restricted to this one class or experience but woven throughout the curriculum in MUED 271, 273, 150, 380 and 371.
Similar examples are evident in Art, School Counseling, Music and Communication Science and Disorders.
3. Experiences working with diverse students in P - 12 schools: Verify how the unit ensures that advanced candidates work with diverse students.
The majority of our advanced programs are delivered to professionals who are already in the field. The diversity of the classrooms and divisions in which they teach reflect the diversity of the communities in which they are housed. We primarily deliver our advanced programs to teachers who are working in regional schools division. In our service area, these schools typically reflect a broad range of diversity as reflected in Standard 4 Addendum Exhibit 7: School Summaries so the practicing teachers are exposed to diverse learners in their own classrooms and schools. A concerted effort is made to ensure that all candidates have the opportunity to demonstrate the knowledge, skills and dispositions necessary for working with diverse students in classroom based assignments and projects. The competencies adopted by the unit are being incorporated into the experiences and projects.
Evidence of the programs' commitment to providing diverse experiences is present in the assignments and rubrics used to evaluate those experiences. The School Psychology program requires the candidates to complete a comprehensive yearlong internship. Clear expectations for the candidate are articulated in the School Psychology Handbook, along with the rubrics used to evaluate their performance.
The Reading Specialist program focuses on supporting the development of literacy in diverse learners. A required internship conducted during the summer ensures that the candidates have experience working with a broad range of learners.
In the School Leadership project candidates in the advanced program for Educational Leadership, complete a school-based internship. The candidates are required to have experiences in two of three school levels - not including the level in which they are currently employed, and central office. Specific expectations for documenting the experience, including working with all diverse students and creating a learning environment to support all learners, are articulated in the documents for both of these programs and other advanced programs in Standard 4 Addendum Exhibit 8: Advanced Programs Diversity Experiences.
4. Experiences working with diverse students in P - 12 schools: Review files/records that demonstrate that the unit tracks the placement of candidates to ensure that all candidates work with diverse learners.
The Education Support Center has been utilizing a system of placing candidates for the past several years that take into consideration the diversity of each of our school partner divisions. As stated in our Student Teaching handbook:
"We are committed to providing for all candidates rich experiences in diverse settings to support their success in working with all students. Candidates can expect to be placed in a range of classrooms and educational settings throughout their professional education program that reflect the demographics of our communities and beyond. All locations used for placements are categorized based on characteristics that reflect diversity, such as socio-economic levels, rural vs. urban designation and diversity of cultures and languages present in the schools. Every attempt is made to provide opportunities for each candidate to experience as broad a spectrum of experiences as possible. Some of the placements are available within our local city school division and others may be in communities within a commutable distance. Therefore, candidates need to be aware that travel to schools outside of the immediate community may be required for their program." (http://www.jmu.edu/coe/esc/student_teaching/PoliciesProcedures.shtml#StudentTeacherPlacementProcess)
Updated demographic tables based on the U.S. census data are used to support the decision-making process. Data were gathered from the VDOE 2001-2012 Fall Membership Report and the 2011-2012 School Nutrition Program Statistics & Reports. An electronic file that describes school diversity is maintained. Each building is ranked with high/medium/low diversity designations (originally determined by race, ethnicity, exceptional student population, and free/reduced lunch rates). Diversity is defined as the percentage of nonwhite students. Socio-economic ratings reflect the percentage of students on Free and Reduced Lunch. Ranking are determined on the following percentages:
0-20%= Low (L)
21-49%= Middle (M)
50-100%= High (H)
Candidate placements are noted in our database for purposes of tracking and making decisions about placement options. Each student placement is determined based on ST request at time of application, consideration of prior field experiences for practicum in terms of both school/division and grade/content area. This prior field placement information shows up on the database; we create a working spreadsheet document to manage placements each semester. Every attempt is made to place students in with different divisions and grade-level experience in order to provide a broad and diverse range of placements for each candidate. Standard 4 Addendum Exhibit 7: School Summaries presents this data.
A demonstration of the range of placements for candidates currently engaged in clinical practice is found in Standard 4 Addendum Exhibit 9: Diversity of Candidate Placements.
5. Experiences working with diverse faculty: Obtain a table that includes data regarding faculty diversity over at least three years to demonstrate "increase."
Increases in the diversity of the faculty at JMU and across the unit have been modest, despite recruitment activities. Standard 4 Addendum Exhibit 10: Faculty Diversity presents a snapshot of faculty demographics in 2011-12 as compared to 2004-05. Recognizing that the faculty demographics at JMU in general and specifically within the unit are not significantly different than they were seven years ago, the unit has taken steps to provide opportunities for candidates and faculty to work with more diverse colleagues. As mentioned in the IR, we are making movement in our goal of strengthening and expanding on a collaborative relationship with Morgan State University. Ideally, we will be creating opportunities for teaching and learning together. We are hopeful that we will be able to develop a partnership that will result in our hiring an ABD graduate student from Morgan State who will teach with us and whom we will support through the completion of a dissertation.
What is not reflected in the diversity data for our faculty is the wealth of professional and personal experiences that our faculty bring to the classroom and the schools in which they work. Faculty ethnicity and race is only one measure. Our faculty represents backgrounds that reflect the full socioeconomic spectrum. They bring to the table experiences in low performing schools, rural and urban teaching environments, international teaching experiences in third world countries and work with children and families of all races, ethnicity and socioeconomic levels. We have faculty members who were raised in countries other than the United States and faculty members who regularly travel to third world countries and across Europe, working with schools, teachers and students and that work is all reflected in their perspectives and strategies for preparing effective teachers. Faculty engagement in out-of-classroom experiences reflect a commitment to social justice and the belief that all students can learn. Consulting, volunteering, and personal passions result in our faculty reaching out to the underserved student, teachers and school divisions. As articulated in the IR, and demonstrated in Standard 4 Addendum Exhibit 11: Faculty Experiences in Diverse Settings, our service to communities across the Commonwealth as well as the personal and professional dedication of the unit faculty to teach all learners is reflected in experiences beyond demographics.
6. Experiences working with diverse candidates: Verify evidence that demonstrates that candidates from diverse groups serve on committees and projects.
The strategies used in all of our classes across the unit actively engage ALL candidates in their learning. Large group, small group, and individual projects are integral to the programs. Class presentations often are used as a way of observing not only individual candidate's understanding and application of the knowledge and skills, but also a demonstration of candidate dispositions, collaboration and teamwork.
The unit's candidates represent multiple ethnic/racial groups as well as socioeconomic levels. During the onsite visit, conversations with our candidates regarding opportunities for them to work together and with faculty will provide evidence for responding to this question.
7. Design, implementation, and evaluation of curriculum and experiences: Review rubrics for the Teacher Work Sample and the Inquiry Project in the Early Childhood initial programs.
The Teacher Work Sample is integral to the Middle and Secondary Education programs assessment of their candidates. The development of this comprehensive assessment has engaged the faculty for several years and reflects the best thinking and planning on how to most effectively evaluate candidate knowledge and skills in teaching all students. The components and all developed rubrics incorporate our conceptual framework, state competencies and professional standards.
The Inquiry Project in Early Childhood is the capstone experience and provide the candidates with the opportunity to apply the knowledge, skills and dispositions necessary to be an effective teacher through the development, implementation and analysis of a classroom based inquiry project. During their internship semester, candidates work closely with the cooperating teacher to conduct the inquiry project in the classroom and present their findings to program faculty at the conclusion of the project.
8. Design, implementation, and evaluation of curriculum and experiences: Clarify what are the MIE programs in English, Math, and Social Studies on the Key Assessment-Impact on Student Learning Table and how these differ from the SEED programs.
The licensure regulations in Virginia designate middle education as a discrete licensure level and require that candidates demonstrate competence in one area of concentration; English, Math, Social Studies or Science. At James Madison, our candidates must complete TWO areas of concentration in their IDLS (Interdisciplinary Liberal Studies) degree. Secondary education candidates complete an undergraduate degree in one discipline (Math, English, Social Studies or an area of Science - biology, chemistry, physics, earth science)
The Teacher Work Sample serves as the basis for the Key assessments in the Department of Middle, Secondary and Mathematics Education for both middle and secondary licensure areas.
Faculty cross over between middle and secondary education and as such, the framework, descriptions and rubrics used for the TWS are similar. IR Exhibit 1.3.c demonstrates the alignment of the unit and program key assessment for initial licensure.