UNC-GA Teacher and School Leader Quality Research Initiative
In partnership with the UNC General Administration (UNC GA) and the fifteen UNC system institutions, the EPIC research team conducts evidence based research on teacher and school leadership quality in North Carolina, with a particular focus on how preparation influences performance. Notable research projects from this collaboration include: teacher portals analyses, UNC teacher preparation program analyses, an evaluation of the North Carolina Teaching Fellows program, and the effects of the UNC Masters of School Administration program on student performance. Additional focuses include the examination of on-the-job learning, attrition of early-career teachers, and investigating how principal characteristics and training experiences influence their leadership performance. Future research agenda items include a predictive validity study of edTPA, continued evaluation research of the North Carolina Principal Fellows program, and examining the influence of non-cognitive characteristics on teacher performance. Findings from this research are regularly presented to the Deans of the Schools, Colleges, and Departments of Education at the 15 UNC system institutions, North Carolina private and independent colleges and universities, and the North Carolina Legislature.
Predictive Validity Study
Researchers at EPIC, in collaboration with the UNC General Administration and the National Council on Teacher Quality (NCTQ), have conducted a study to assess whether teacher preparation programs with higher NCTQ overall ratings or that meet NCTQ standards produce more effective novice teachers, as measured by value-added to student achievement and teacher evaluation ratings. Results indicate that there is not a strong relationship between NCTQ’s ratings and measures of teacher performance. Moving forward, results suggest several directions for improving the performance of teacher preparation program graduates: (1) ensuring that teacher preparation programs systematically and routinely obtain data on teacher candidate and graduate performance; (2) setting higher standards for admission into teacher preparation programs; and (3) when input and process based standards are used to evaluate teacher candidates and the delivery of teacher training, adding measures of quality to those of quantity
UNC edTPA Pilot NCTQ
The UNC edTPA Pilot is a voluntary-multi-campus effort to develop formative applications of edTPA. The Pilot grew out of the need to identify a valid and reliable instrument for the assessment of teacher candidates in the UNC system and to generate accurate and useful feedback to guide improvement of student teachers’ performance over time. By selecting this instrument campus partners aim to:
- accurately predict candidates’ later on-the-job performance, as measured by academic value added measures;
- provide a basis for faculty to make improvements in teacher education programs; and
- contribute to decisions on the preparation, development and recommendation of student teachers for licensure.
The researchers at EPIC have partnered with the UNC General Administration to host training events and meetings designed to support institutions at various stages of edTPA implementation and conduct research to assess the reliability, measurement properties, and predictive validity of edTPA scores.
Gear Up Research Project
This research project will inform the scope of work to evaluate the effectiveness of the GEAR UP North Carolina (GU-NC) intervention, and to identify the critical elements of a successful GEAR UP (GU) model. The research team will conduct quantitative analysis to examine longitudinal trends among GU students, parents, and schools over the course of the grant and utilize a quasi-experimental study design to provide a causal estimate of the intervention on core national and state-level outcomes. A mixed-methods approach will provide a deeper understanding of the statewide variation across program implementation, as well as to identify the “active ingredients” within this multi-component intervention.
Data Sharing Initiative
Researchers at EPIC, in collaboration with UNC General Administration, have launched an initiative to provide each of the 15 UNC system institutions and their colleges of education individual-level data on program graduates. Such data include graduates’ teaching credentials, classroom and school working environment characteristics, and outcomes of interest (teacher value-added and persistence). This data sharing initiative is at the forefront of researcher-practitioner collaborations and holds the promise to improve the quality of teacher preparation by facilitating evidence-based changes to teacher preparation practices. Beyond data provision, EPIC plans to seek additional funding to fully support campuses’ use of these externally-provided data.
Recent Graduate Study
Researchers at EPIC are coordinating the development of a new survey instrument that will be administered to first year teachers in North Carolina. Survey responses will allow UNC teacher preparation programs to determine how their recent graduates perceive their program’s ability to prepare them to become an effective teacher and thereby enable programmatic improvements based on respondents’ perceptions of preparation quality.
EPIC’s research provides evidence based comparisons of teacher effectiveness across different routes or “portals” into the teaching profession. Recent findings indicate that, in comparison to in-state publicly prepared undergraduates, teachers who are prepared out of state are significantly less effective in elementary school mathematics, elementary school reading, and high school tested subjects. The elementary school findings are particularly noteworthy given that teachers prepared out of state are the largest source of elementary teachers within North Carolina’s public schools. In comparison to in-state publicly prepared undergraduates, the research findings also reveal that alternatively prepared teachers are significantly less effective in high school. These findings are notable given that alternatively prepared teachers are the largest source of teachers within NC public high schools. Finally, in comparison to in-state publicly prepared undergraduates, Teach For America corps members are significantly more effective in elementary grades mathematics, middle grades mathematics and science, and all high school tested subjects than NC public schools teachers who are traditionally prepared.
An examination of the NC Teaching Fellows program revealed that the program attracts individuals into the teaching profession with substantially higher levels of academic achievement (SAT scores, high school GPA) than other in-state prepared teachers. Additional findings showed that NC Teaching Fellows are significantly more effective than other in-state prepared, out of state prepared, and alternative entry teachers in elementary school mathematics, middle school mathematics, and all high school tested subjects, combined. Finally, compared to other in-state prepared teachers, NC Teaching Fellows are significantly more likely to remain for at least five years in North Carolina public schools.
Stayers and Leavers
The EPIC research team conducted a rigorous analysis designed to investigate teachers’ on-the-job learning and differential attrition in their early-career years. Results indicated that teachers exhibit significant amounts of on-the-job learning in their first three years of teaching. Beyond three years of teaching, however, growth in teacher effectiveness diminishes rapidly. In addition, teachers who exit North Carolina public schools early in their careers are less effective, on average, than teachers who stay. Comparing teachers who remain in the classroom to those who leave the profession, the EPIC research team found that a large majority of the average returns to teacher experience are attributable to teacher on-the-job learning, rather than the differential attrition of less effective teachers.
College Star Evaluation
College STAR (Supporting Transition, Access, and Retention: A UNC System Project Supporting Students with Learning Differences) was developed with the vision of UNC becoming “the first public university system intentionally prepared to provide access to higher education and to foster academic success for students with learning differences.” Beginning in 2011, College STAR was designed to provide holistic support through multiple points of influence at the student, faculty, and institutional level to create an environment that is responsive to the needs of students who learn differently. Now in its third phase, EPIC is conducting an evaluation of the impacts of College STAR across three campuses, Appalachian State University, East Carolina University, and Fayetteville State University. A mutual goal of the program implementers and evaluation team is to establish a framework where successful program elements and processes can be shared with other state universities, and common metrics and measures can be utilized to enhance the fidelity and generalizability of evaluation findings.
Race to the Top (RttT) Evaluation
The Education Policy Initiative at Carolina (EPIC) is a member of the Consortium for Educational Research and Evaluation–North Carolina (CERE-NC) which includes the Friday Institute for Educational Innovation at North Carolina State University and the SERVE Center at the University of North Carolina at Greensboro. CERE–NC was contracted to conduct an independent evaluation of initiatives funded by the state’s $400,000,000 Race to the Top federal grant.
The RttT Evaluation examines the effectiveness of more than ten initiatives ranging from tracking the inequitable distribution of teachers and leaders across the state, to teacher support programs for beginning teachers in North Carolina’s lowest performing schools, and professional development for the new standards and assessments being implemented across North Carolina. The evaluation includes short-term formative assessment as well as summative impact assessments to determine the relative and cumulative contributions of these initiatives over time, and to inform future funding and policy decision for education reform in North Carolina.
The Race to the Top grant spans from 2010-201, with the possibility of a one year extension to June 2015. Impact measures of interest include student level measures of achievement scores, graduation rates, and college readiness; teacher level measures of effectiveness using Value Added Models (VAM) and classroom observations, and indicators from statewide surveys of principals and teachers across multiple initiatives. The RttT evaluation includes projects such as:
North Carolina New Teacher Support Program (NC NTSP)
The NC NTSP initiative provides a comprehensive beginning teacher support program – institutes, instructional coaching, and professional development – to first, second, and third year teachers in the state’s lowest five percent achieving schools. To determine the quality and intensity of NC NTSP services and ascertain whether the NC NTSP significantly influences beginning teacher self-efficacy, retention, and effectiveness, the EPIC research team serves as independent evaluators of the program. In this effort the evaluation team uses multiple measures to gauge program effectiveness, including teacher surveys, CLASS™ observations, teacher evaluation ratings, teacher retention, and teacher value-added estimates in order to compare the performance of participating NC NTSP teachers with that of novice teachers in similar non-participating, high-need schools.
The Omnibus Teacher and Principal Survey
The Omnibus survey was developed and piloted by the Race to the Top evaluation team in 2010-11, and will be administered through the 2013-14 academic year. The purpose of the survey is to capture responses spanning multiple initiatives within a representative sample of schools across the state, and to provide measures of school context and intermediate educational outcomes related to student learning. The survey sample consists of principals, assistant principals, and teachers from 366 schools and has three administration waves in the spring of each year. The schools were selected based on probability sampling stratified among District and School Transformation (DST) schools, other lowest achieving schools, STEM schools, rural schools, and then from the remaining NC public schools.
Part of the North Carolina Race to the Top effort is focused on implementing professional development to the state’s 100,000 teachers and 2,400 principals. In partnership with the William and Ida Friday Institute for Educational Innovation, the EPIC research team conducts various evaluations that focus on the effectiveness and usefulness of professional development within North Carolina LEAs and schools. These evaluation components are as follows:
- Leadership in Practice (DLP): The DLP initiative employs a series of professional development meetings for North Carolina’s principals to develop key leadership traits and learn how to build positive school climates. As part of this evaluation, the EPIC research team observes DLP sessions and leads the qualitative analysis of principal satisfaction surveys.
- Longitudinal PD: The EPIC research team conducts teacher, principal, and LEA Coordinator interviews to gauge the impact of professional development across multiple years. The evaluation is focused on assessing information about the types of professional development utilized and perceptions of their effectiveness, at the district and school level.
- Online Professional Development (OPD): EPIC researchers lead the qualitative analysis, including open-ended responses, to assess perceptions of the usefulness of online professional development tools that are disseminated to educators across the state.
- Statewide Face-to-Face Professional Development Formative Evaluation: The EPIC research team contributes to the observation of Summer Institutes, the Regional Education Service Alliance (RESA), and conducts interviews with RESA Directors to determine the effectiveness of face-to-face professional development to teachers, principals, LEA coordinators, and district leaders.
Statewide Strategic Staffing
The EPIC research team serves as the independent evaluators of the State Strategic Staffing initiative (SSS), a program intended to provide Local Education Agencies (LEAs) with vouchers to recruit highly effective teachers into lower-performing schools. To determine program effectiveness, EPIC researchers conduct interviews with teachers, principals, LEA and state officials, as well as analyzes teacher value-added data.
Teacher and Leader Effectiveness Evaluation
The evaluation of the Race to the Top’s Teacher and Principal Evaluation initiative examines the validity and reliability of new, independently-developed components added to the NC Educator Evaluation System (EES). The implementation of the new Educator Evaluation System components are being examined to determine if the new value-added measures provide independent information that allows for meaningful assessment of teachers’ and administrators’ effectiveness. In partnership with the SERVE Center at UNC–Greensboro, EPIC researchers are conducting extensive fieldwork including classroom observations and interviews. Qualitative analyses, including teacher and principal interviews, are being conducted by EPIC researchers to assess the impact of the new evaluation process on educators’ attitudes and practices. Quantitative analysis is being conducted to assess the correlation between value-added measures and other measures of teacher effectiveness.
RttT Overall Evaluation
The EPIC research team is leading the RttT Overall Impact evaluation including a comparative cost analysis of each initiative, and will lead the quantitative analysis of the summative impact of the full RttT grant period on primary outcome measures including student achievement, graduation rates, and school performance ratings.
Double Dosing of Mathematics in North Carolina
This project explores the set of intervention strategies commonly referred to as “double-dosing”, where students with performance deficiencies or advanced ability in a tested subject (generally mathematics or reading) are given extended instructional time in the subject. This study aims to identify how double dosing in Mathematics is being implemented within North Carolina. Specifically, this study seeks to develop a typology of double dosing and to apply that typology to North Carolina education data. From this application, the study will examine the extent to which double dosing is used, the types of double dosing, and the characteristics of the schools, teachers, and students involved in double dosing interventions. While these practices appear to be increasingly common, relatively little is known about the ways double dosing is being implemented and the effect it has on students. This study will help to inform this line of research by providing a preliminary analysis of double dosing practices in North Carolina.
East Carolina University- Teacher Quality Research Partnership Grant (ECU TQP)
The EPIC research team was contracted to serve as the external evaluators of the College of Education (COE) at East Carolina University (ECU) multiyear U.S. Department of Education grant titled “Teacher Quality and Student Achievement: A Comprehensive Data-Driven School-University Approach to P-16 Reform.” The goal of the grant is to reform ECU’s teacher preparation program and to improve teacher quality in two high-need rural school districts near ECU. The initiative includes five interrelated areas of reform: recruitment and retention of qualified teacher candidates, curriculum, clinical practice, beginning teacher support programs, and local school reform. ECU’s reform efforts include four primary objectives: (1) Improve recruitment and retention of underrepresented groups; (2) Enhance the quality of beginning teacher performance by increased use of instructional strategies; (3) Improve the academic achievement of students in the ECU Teacher Education Program; and (4) Improve the achievement of students taught by graduates of ECU’s Teacher Education Program. Impact measures of interest include student level measures such as achievement scores; teacher level measures of effectiveness using Value Added Models and classroom observations using CLASS™, and an analysis of the ECU TQP enrollment data.
New Schools Learning Lab Initiative
The primary goal of the New Schools Evaluation is to look at the effect of the Learning Lab Initiative on student outcomes. The evaluation also looks at the effects on student outcomes of Early College High Schools and Redesign High Schools that are part of the New Schools network. Student outcomes examined in the evaluation include standardized test scores, 9th grade retention, graduation and dropout rates, UNC system enrollment and Associate’s degree completion rates.