22S:30/105 Statistical Methods and Computing
Public versus Private School Test Performance
May 5, 2008
Lewis Mann
Annie Temple
Tony Hubbard
Analysis of public and private test scores controlling for possibly confounding variables
One-Variable Comparisons: CHI-square tests using SAS
- direct comparisons of test scores with one variable
| Grouped according to: | Math | Reading | Notes |
|
|
|
| Public | (275.51, 276.49) | (260.67, 261.33) | work |
| Private | (290.03, 293.97) | (280.85, 283.15) | |
Note: We could only perform this comparison on data for which we we could find individual sample sizes.
Data from Nation's Report Card
Two-variable Comparisons: Excel Whisker Plots
- side-by-side plot of mean test score +/- standard error
- public and private school student scores are compared, grouping by different possibly confounding variables
Note: Some subjects and years were not available in the original data.
Our data came from the data explorer on the National Assessment of Educational Progress (NAEP). The NAEP collects data from the tests mandated by the No Child Left Behind Act, in place since 2001. They also have data from before this time, but it is much less complete. Using this data, reports are issued regularly on the status of the American education system by the NAEP and other organizations. Their website allows individuals to examine mean test scores as well as percent proficient among 4th, 8th, and 12th graders for several different years. The scores come from a series of standardized tests, topics of which include math, reading, writing, science, history, civics, economics, and geography. Several of these topics are only covered on the 12th grade exam. Reading, writing, and math, can be further broken down into subcategories such as reading for literary experience versus reading for information. The students and schools also provided demographic information about their school, community, and the family life and involvement of students. Our main focus was the difference between public and private school tests scores. Because of the large amount of data, we chose to focus on a smaller subset which we thought would be representative of these differences. We only looked at data for the 8th grade, and only for the subjects of math, reading, and writing, because these were the most complete data sets. We did not look at any of the subcategories. We performed cross-tabulation, to see if there were any reasons why private schools score differently than public. We divided schools into groups based on Title I (federal funding for low-income areas), free-lunch program, and parental involvement, for example.
The NAEP website only provides test score means (or percent proficient) and standard errors. It does not make sample size publicly available. This information is restricted and only released to approved researchers. Analysis and manipulation of the data without the sample size was difficult, as many forms of analysis that we wanted to perform, such as Chi-square and ANOVA testing require this information. We were able to find sample sizes for public and private schools, but the numbers were not divided into subcategories for each variable. The NAEP data explorer will not show results if the sample size is too small by their standards, so dividing groups by more than one variable creates a lot of groups whose sample size is too small to report. Because of this, we weren’t able to perform the full analysis that we had initially proposed. Also, in our project proposal, we wanted to examine data regarding the amount of money spent per student, however this information is not publicly available on the NAEP website. We used Title I funding, presence of computers in the home, and free-school lunch as indicators of financial status to compensate for this.
Upon analysis of whisker plots and raw data, it is conclusively evident that 8th grade private school students score better than public school students on standardized tests for math, reading, and writing, regardless of any confounding variables that we examined. Without the appropriate data, we could not technically provide sufficient statistical evidence that this is the case, but the information we do have is very persuasive. Visually, it does appear that each of the confounding variables we chose do have an effect on student test scores, but these do not account for the differences in scores between public and private schools.
The way that the NAEP gathers its data has been scrutinized, and many people find that standardized testing is not an accurate indicator of student learning and knowledge. For example, it does not measure improvement of individuals over time, nor does it predict their learning potential. Some people also believe that there is a socioeconomic bias to these tests. Another critique is that it forces educators to “teach to the test” and neglect other important areas of study.