School Improvement Hinges on Access to Student Data
The state should know lots about those students: their standardized-test scores, whether they voted, their criminal records, their income, etc. The state replies that it does not have this information collected in a manner that is accessible. And, to add insult to injury, the state explains that it would not release the information anyway because of privacy concerns.
You decide to proceed with pre-K in your district regardless, but, so that future researchers can learn something, you ask the state if you can assign pupils to the pre-K class through a random lottery given that there will not be enough spots for everyone. The state refuses. A local education researcher asks if you can work together to at least keep track of key administrative data for the children within and without the program. To do that, you need help from the state, but again the state refuses.
That scenario is neither fanciful nor uncommon. Despite some improvements, many states do not maintain the data in a usable manner that education researchers need, much less do they use program rollouts as a regular opportunity to conduct controlled experiments. On the one hand, this failure makes sense. Organizing and managing administrative data is not costless, especially if privacy concerns are properly taken into account. Furthermore, as a matter of practical politics, not much of a constituency exists for the collection of good data that will yield conclusions many years in the future-a time frame that for most politicians or administrators makes no sense.
At the same time, the failure to generate high-quality data is untenable. Education is by far the biggest expenditure made by state and local governments. The cost of collecting good data and making the information available is not even a rounding error compared with state and local education budgets. It would be one thing if educational researchers were doing well enough with the data they have, but the expert consensus is that they are not. And this is not because there is a lack of researchers or analytic methods. (Indeed, something of a revolution is going on in the social sciences when it comes to the use of administrative data. For example, a much-celebrated recent study by Raj Chetty and colleagues demonstrated that social mobility in the United States depends greatly on where a child grows up, based on careful analysis of years of tax-return data.)
If education research is not to be left behind, states need to devote resources to collecting the data and making those data available to researchers. There are models for doing so. In some Nordic countries, each citizen is given an administrative-record number that is used throughout the government. When a researcher requests data, the government provides the data, but using a different set of numbers to protect privacy. That arrangement has enabled education research that would be difficult or impossible to carry out elsewhere, such as studying the effect of publicly provided day care on labor-market outcomes decades later&-just the question our hypothetical superintendent was hoping to answer.
"If education research is not to be left behind, states need to devote resources to collecting the data and making those data available to researchers."
The policy prescription is clear: States should aim to collect and disseminate first-rate educational data. A good place to start on this project is with the checklist provided by the nonprofit Data Quality Campaign, which emphasizes the record-keeping arrangements needed to track students over time and across administrative databases.
States should also organize themselves so that opportunities for controlled experiments are not squandered. In many cases, states already roll out programmatic changes in pieces; it would not cost much more to do so in a manner that enables credible inferences about the reform's effects.
What if a state refuses to take reasonable steps to assess the effectiveness of its biggest outlay? Can the states be forced into self-reflection? We think the answer is yes.
Virtually every state constitution provides for a system of free public schools. Most states have been sued under those provisions, with the plaintiffs claiming that states are not distributing funds equitably or just not spending an adequate amount. Plaintiffs have a mixed record in such suits, though it should be noted that many states changed their educational system because of the threat of such a lawsuit.
As a result, there are many states where the constitutional provision concerning education has been litigated and in which the courts have held that the state has a legally enforceable "duty of care" with respect to education. We argue in a forthcoming law review article that if this duty of care means anything, it must at least mean that states take reasonable efforts to enable the assessment of how their public education systems are performing. That is, leaving aside whether states must spend more money or spend more fairly, they must at least have some reasonable system in place to assess their compliance with the constitutional command to provide a decent public education.
Especially in states with courts that have proven willing to impose dramatic solutions, such as spending and other mandates, we think that even the threat of litigation should motivate state officials to provide education researchers the data they need. To be clear, our vision of states' duty of care with respect to education wouldn't turn children into lab rats. States would still have to protect student records from privacy-compromising disclosures, and state officials-not researchers-would continue to set priorities. But whatever else the states may owe to disadvantaged children in particular, at least the states must make it possible to learn whether their efforts to better educate those children are doing any good.
[Cross-posted from Education Week]