The purpose of this research is to analyze current VAM data and determine if there is a need for a more comprehensive research framework that will extend Florida’s current evaluation model to consider socio-economic and demographic implications.  America's youth now face competition and challenges in a global economy that have never before existed.  According to the Program for International Student Assessment (PISA), American students ranked 14th in reading, 17th in science, and 25th in math (Pisa, O. E. C. D., 2012).

 

Only 78% of American student's complete high school on time and the numbers are even lower for Blacks and Latinos.  Recent reforms and decisions made by education and policy leaders in Florida, particularly high stakes testing and the accountability that accompanies them, have placed excessive pressure on school leaders, teachers, and students.  This research aspires to clarify if schools' socio-economic and ethnographic standings have statistically significant effects on teacher's VAM scores (for technical details, see appendices).  Subsequently, this body of research will address the essential need for a more comprehensive and effective teacher evaluation platform.

 

Research Technique

 

This quantitative secondary data research will use a three Factor Fixed-effects Analysis of Variance (ANOVA) Model to study the effect of each factor (race, migrant status, socioeconomic status) on the continuous dependent variable (teacher value-added scores), as well as the effects of interactions between factors on the continuous dependent variable (DV).  The Factorial ANOVA model used (three independent variables) will test for both main effects and interactions.  Using three factors will provide a more precise estimate of error variance and increase power while simultaneously providing greater generalizability (Lomax & Hahs-Vaughn, 2013).  Independence, homogeneity of variance, and normality are assumed with the model.

 

Research Questions

 

The following hypothesis will be tested:

 

There is no mean difference in Florida teacher’s value-added scores (DV) based on the socio-economic status (IV) of schools.

 

There is no mean difference in Florida teacher’s value-added scores (DV) based on student race (IV).

 

There is no mean difference in Florida teacher’s value-added scores (DV) based on school’s migrant population (IV).

 

There is no mean difference in Florida teacher’s value-added scores (DV) based on the socio-economic status (IV) of schools and student race (IV).

 

There is no mean difference in Florida teacher’s value-added scores (DV) based on the socio-economic status (IV) of schools and a school’s migrant population (IV).

 

There is no mean difference in Florida teacher’s value-added scores (DV) based on race (IV) and a school’s migrant population (IV).

 

There is no mean difference in Florida teacher’s value-added scores (DV) based on socio-economic status (IV), race (IV) and a school’s migrant population (IV).

 

Population, Setting and Sample

           

This research will use the entire population of teachers in Florida that received a VAM score during the 2013 – 2014 school year, approximately 78,448 records.

​

Variables and Measures

 

The independent variables are the socio-economic status (category 1 - 0%-50% and category 2 - 51%-100%), the migrant status of the school (category 1 - 0%-50% and category 2 - 51%-100%), and the racial makeup of each school during the 2013-2014 school year.  The independent variables are categorical (socio-economic status converted to dichotomous); which meets the assumption for running three-way analysis of variance which is an extension of the one-way ANOVA that examines the influence of three different categorical independent variables on one continuous dependent variable.  The three-way ANOVA aims at assessing the main effect and interaction of each independent variable.

​

There is one dependent variable.  The dependent variable is the combined VAM score of Florida school districts in the 2013-2014 school year.  VAM scores can be used as an ordinal scale of measurement because they can be ranked.

 

Instrumentation & Data Collection Procedures

 

There is no evidence of documentation of reliability or validity measurements performed by the Department of Education. Data in this research study included nominal (dichotomous) and interval variables from the data collection process from the Florida Department of Education’s Data Warehouse Integrated Education Data System (IEDS) and the Jacksonville Times Union database. The Florida Department of Education’s Education Data Warehouse (EDW) stores enrollment, financial aid, student demographics, test scores, awards, employment information, curriculum, and certification information.  The EDW integrates data extracted from multiple source systems and provides a single repository of data concerning students served in the K-20 public education system. Students can be tracked across delivery systems over time, provide trend analysis and provides education leaders and researchers with the information and tools necessary to make informed, data-driven decisions (FLDOE, 2016). In February of 2013, the Florida Times-Union filed a lawsuit requesting teacher Value-Added Model (VAM) scores.  In November of 2013, an appeals court ruled the data was public record and is currently available to the public on a dedicated database website.

​

Teachers will be removed if either free and reduced lunch rates or VAM scores are not available during the reporting period. The unit of analysis for this research will be teacher VAM scores.  The confidence level used will be 95% resulting of a significance level of alpha 0.05.  

 

Institutional Review Boards are in place to protect the welfare of research participants through compliance with federal regulations governing the protection of human subjects.  All human subject research requires review and approval by an Institutional Review Board (IRB) prior to subject recruitment and data collection. Prior to the collection of data for this research, an IRB application will be completed and no data will be collected until the IRB application has been approved.

 

Statistical Test

 

The test used to calculate the data will be a three-factor Analysis of Variance. Statistics for each variable will be based on valid data for that pair with any missing values treated as such.  The dependent and independent variables, mean, standard deviation and number of variables will also be presented in table form.  

 

Assumptions

​

Assumption #1: The dependent variable is measured at the continuous level Assumption #2: The three independent variables should each consist of two or more categorical, independent (unrelated) groups.

 

Assumption #3: Independence of observations

 

Assumption #4: There should be no significant outliers

​

Assumption #5: The dependent variable should be approximately normally distributed for each combination of the groups of the three independent variables. The Shapiro-Wilk test of normality will be used in SPSS to test for normality.

 

Assumption #6: Homogeneity of variances for each combination of the groups of the three independent variables. Levene's test for homogeneity of variances will be used to fulfill this assumption.

Independence implies that each observation is selected without regard to any other observation sampled. Homogeneity of variance is met when the variances of the dependent variable for the two samples are the same.  Finally, the assumption of normality is met when the dependent variable is normally distributed for each sample (Lomax & Hahs-Vaughn, 2013). Each assumption is addressed in the final report.  

 

Data Analysis

 

Data will be imported into the Statistical Package for the Social Sciences (SPSS) for analysis and summary. A three-way ANOVA compares the mean differences between groups that have been split into three independent variables.  The primary purpose of a three-way ANOVA is to understand if there is an interaction between the three independent variables on the dependent variable.

 

Institutional Review Board

 

This research meets the Institutional Review Board (IRB) definition of both “human subjects” and “research” and thus, is not exempt from the application process. The IRB on-line training through the Collaborative Institutional Training Initiative has been completed and once this proposal is approved, an IRB application will be submitted through IRBnet.  No data will be collected until the IRB application has been approved.

​