The schools reflect their students far more than the students reflect the school (I think the impact ratio is > 10:1). I moved into an area with an excellent school disctrict when my kids were in school - but I did not fully trust the schools to do their job. We monitored their learning and at times supplemented it. I gave school administrators grief to get the kids into appropriate classes - and when I could not, I oversaw home-study/correspondence classes. I was a hands-on and involved parent - but I did not do the work for them, I made sure that they did it themselves and mastered the material.
Once you have a classroom and a teacher, the next most important thing is not devices, but the absence of disruptive and disaffected students who disrupt the learning by everybody else in the classroom. And money doesn't affect that directly.
The better students will be at least partial autodidacts - they will learn if given a chance. When I went to school many of the better students were not well off - they were the children of the Holacaust survivors, many of whose parents were running small shops. Many were both bright and hard working - and their home culture supported academics.
There is an even shorter iron law in social science than the ones Emil quotes: "If it can be selection effects, it is selection effects".
I think this one fits very well with your post. You mention three primary influences, and notably *all* of them are selection effects. The most obvious is, good schools are such because they have good students. Put the same students in other schools, and they will perform quite well regardless, especially longterm. Any intervention that attracts them will have spuriously positive results.
The next one is, good schools have good parents. Highly involved & supportive parents can make a large difference, and though this tends to wash out over adulthood later on (in contradiction of the progressive assumption of compounding), it nevertheless is easily measurable in the child test scores.
Finally, good schools will have a low number of disruptive students. Again, this tends to wash out over time, but the child test scores will be negatively influenced even for the good students.
In general tbf I've gotten quite cynical about learning approaches and education spending. Simple, direct instruction to good students, with good parents and little disruption will easily outperform ANY complicated learning design or ultra-high education spending with average students/parents.
economists, so ignorant of higher maths like measure theory or harmonic analysis, chose to gave up pursuing mechanisms of human action and instead decided to output econometric slop 24/7
while thankfully it's easy to debunk their slop given the knowhow, the whole sea of econometric slop needs to be done away with
1983? Probably not relevant considering econometrics at that time was mostly simple regression. The real issue is the lack of sociobiological theory in econometrics. It's always the studies aiming to prove that man is arbitrary and cotrolled by his environment that end up p-hacked and null. And it's always biological controls that are missing -- innate intelligence, race, talent
because those fields advance the theory of Stochastic differential equations
I myself have developed a new model of price action which derives the Fokker Planck equation among other stuff from the microfoundations of a limit order book
my cheekily named post "Austrian Econophysics" describes the entire setup, though the real paper is some 80 pages long (and does not reference Rothbard Mises et al)
On top of everything else, the more funding, the more corruption, at least in U.S. public primary and secondary schools. (And in the universities, though they're not in question here.) So the more funding, the less effective the funding is.
Plus, schools and educational administrations that are funded on a per-student basis will be very reluctant to let high-achieving students move through at their own pace and graduate early--which can cripple or wreck the lives of even high-achieving students.
Most of the effects are tiny but still not 0 and sometimes there is p-hacking but not always. My problem with these studies is not even this, but that the effects usually start working several years after the reforms. If the reform's effects take 5-6 years to kick in (see e.g. Figure 2, right side in what I linked), aren't those selection effects? If the school becomes better funded, maybe it attracts smarter students over time. To be fair, apparently it also takes several years for funding to increase (see your figure) but then these studies are much more correlational/cross-sectional and less quasi-experimental than they look. There is a court order and bad school gets more money, smarter parents notice this and some decide to send there kids there for the nice new gym or whatever, ultimately improving the student pool, but this takes several years and absolutely leaves room for selection effects.
Evidence from gifted/accelerated programs shows a dramatically different picture. High-ability learners are far more "resource-responsive" — they convert enriched, accelerated, or specialized environments into large, lasting gains that average students do not.
Every Chinese city has schools for high-ability learners...
While I'm also quite suspicious of the results here - of course high funding attracts the best of the best, and likewise high entrance barriers outright ensure that only they can enter - but I agree that if anything, the evidence points in the direction that better funding *widens* the gap, instead of *closing* it.
Once you accept that some people are just more industrious, smart, ... etc. than others, the reason is obvious: These good attributes allow you to use resources more effectively and so you benefit more from them. In the end, you only get equality by, as the saying goes, chopping the tall grass.
I think this is largely correct. Having high end resources only benefits those who will use them and are able to use them, widening the gaps. You could take pity on my pathetic garage and give me a 10 million dollar car shop and my vehicles will be in roughly the same state of repair (possibly worse if I get to playing around) but someone who cares about cars and maintenance gets their hands on it and they will be fixing everyone's car.
If a huge example is needed, basically everyone in the developed world has access to the internet, but it isn't clear that people are less ignorant of all the important subjects.
That's the study of what happens after court ordered funding increases. You just say it's a complex study. Yes, and looking at it, I sympathize with you. But they are economists, not ed school people, so we can trust them prima facie (and their funding is NSF, not Dept of Ed.). And they thank some good economists, e.g. Larry Katz.
One probably it looks like they haven't addressed is lags. We wouldn't expect much change in the first year of paying teachers more, only later when they hire better teachers, for example. There are lots of other potential pitfalls. Their techniques are not advanced, tho, not fancy.
Given that results of complex studies change dramatically when different research teams are asked to analyze the same data to answer the same question, there is plenty of room for doubt. What they should do is release the data and have 10 other teams analyze it and see if they agree.
Economists have always been pretty good at sharing their data used in a paper. We're shocked at fields where this is not true. As of about a decade ago, journals even require this, and require the computer code too. So probably someone could get it for this QJE paper. Of course, replication is a big project, too big for me, for examlpe, to want to spend time on. If anyone has trouble geting the Jackson et al data, tho, let me know-- I'm a senior economists, and I bet I could gte it from them. erasmuse61@gmail.com
it is funny how this whole area (of wishful thinking) seeks to squeeze out a statistically significant finding and declare success. how about an effect that has any practical meaning?! Effect sizes matter--esp when it is my money paying for it. we all know what the issue is driving educational performance. it is more than skin deep.
The schools reflect their students far more than the students reflect the school (I think the impact ratio is > 10:1). I moved into an area with an excellent school disctrict when my kids were in school - but I did not fully trust the schools to do their job. We monitored their learning and at times supplemented it. I gave school administrators grief to get the kids into appropriate classes - and when I could not, I oversaw home-study/correspondence classes. I was a hands-on and involved parent - but I did not do the work for them, I made sure that they did it themselves and mastered the material.
Once you have a classroom and a teacher, the next most important thing is not devices, but the absence of disruptive and disaffected students who disrupt the learning by everybody else in the classroom. And money doesn't affect that directly.
The better students will be at least partial autodidacts - they will learn if given a chance. When I went to school many of the better students were not well off - they were the children of the Holacaust survivors, many of whose parents were running small shops. Many were both bright and hard working - and their home culture supported academics.
There is an even shorter iron law in social science than the ones Emil quotes: "If it can be selection effects, it is selection effects".
I think this one fits very well with your post. You mention three primary influences, and notably *all* of them are selection effects. The most obvious is, good schools are such because they have good students. Put the same students in other schools, and they will perform quite well regardless, especially longterm. Any intervention that attracts them will have spuriously positive results.
The next one is, good schools have good parents. Highly involved & supportive parents can make a large difference, and though this tends to wash out over adulthood later on (in contradiction of the progressive assumption of compounding), it nevertheless is easily measurable in the child test scores.
Finally, good schools will have a low number of disruptive students. Again, this tends to wash out over time, but the child test scores will be negatively influenced even for the good students.
In general tbf I've gotten quite cynical about learning approaches and education spending. Simple, direct instruction to good students, with good parents and little disruption will easily outperform ANY complicated learning design or ultra-high education spending with average students/parents.
damn that's depressing. for these kids, is school just daycare that reduces criminality while allowing their parents to work?
perhaps. but it doesn't reduce the former and I see little evidence of the latter.
economists, so ignorant of higher maths like measure theory or harmonic analysis, chose to gave up pursuing mechanisms of human action and instead decided to output econometric slop 24/7
while thankfully it's easy to debunk their slop given the knowhow, the whole sea of econometric slop needs to be done away with
You would like Ed Leamer’s book “Specification Searches” or his AER article “ Let’s take the con out of econometrics”
I've been developing new foundations and rigorous derivations of models
https://x.com/BajiquanArmorer/status/1983298572205601102
1983? Probably not relevant considering econometrics at that time was mostly simple regression. The real issue is the lack of sociobiological theory in econometrics. It's always the studies aiming to prove that man is arbitrary and cotrolled by his environment that end up p-hacked and null. And it's always biological controls that are missing -- innate intelligence, race, talent
there is already research showing that time preference is a two factor construct of conscientiousness and neuroticism, and this was done by MRI scans
the research is still burgeoning but it's there
utility theory simply needs sociobiological foundations and SDE extensions (I have been working on both)
How would measure theory or harmonic analysis produce models with measurable parameters? They couldn't even do that with calculus.
I've been developing new foundations and rigorous derivations of models
https://x.com/BajiquanArmorer/status/1983298572205601102
because those fields advance the theory of Stochastic differential equations
I myself have developed a new model of price action which derives the Fokker Planck equation among other stuff from the microfoundations of a limit order book
my cheekily named post "Austrian Econophysics" describes the entire setup, though the real paper is some 80 pages long (and does not reference Rothbard Mises et al)
On top of everything else, the more funding, the more corruption, at least in U.S. public primary and secondary schools. (And in the universities, though they're not in question here.) So the more funding, the less effective the funding is.
Plus, schools and educational administrations that are funded on a per-student basis will be very reluctant to let high-achieving students move through at their own pace and graduate early--which can cripple or wreck the lives of even high-achieving students.
Jackson 2018 is my favorite review on this topic, still relevant:
https://www.nber.org/system/files/working_papers/w25368/w25368.pdf
Most of the effects are tiny but still not 0 and sometimes there is p-hacking but not always. My problem with these studies is not even this, but that the effects usually start working several years after the reforms. If the reform's effects take 5-6 years to kick in (see e.g. Figure 2, right side in what I linked), aren't those selection effects? If the school becomes better funded, maybe it attracts smarter students over time. To be fair, apparently it also takes several years for funding to increase (see your figure) but then these studies are much more correlational/cross-sectional and less quasi-experimental than they look. There is a court order and bad school gets more money, smarter parents notice this and some decide to send there kids there for the nice new gym or whatever, ultimately improving the student pool, but this takes several years and absolutely leaves room for selection effects.
That 96 page behemoth paper spent some time trying to rule this out as well.
Evidence from gifted/accelerated programs shows a dramatically different picture. High-ability learners are far more "resource-responsive" — they convert enriched, accelerated, or specialized environments into large, lasting gains that average students do not.
Every Chinese city has schools for high-ability learners...
While I'm also quite suspicious of the results here - of course high funding attracts the best of the best, and likewise high entrance barriers outright ensure that only they can enter - but I agree that if anything, the evidence points in the direction that better funding *widens* the gap, instead of *closing* it.
Once you accept that some people are just more industrious, smart, ... etc. than others, the reason is obvious: These good attributes allow you to use resources more effectively and so you benefit more from them. In the end, you only get equality by, as the saying goes, chopping the tall grass.
I think this is largely correct. Having high end resources only benefits those who will use them and are able to use them, widening the gaps. You could take pity on my pathetic garage and give me a 10 million dollar car shop and my vehicles will be in roughly the same state of repair (possibly worse if I get to playing around) but someone who cares about cars and maintenance gets their hands on it and they will be fixing everyone's car.
If a huge example is needed, basically everyone in the developed world has access to the internet, but it isn't clear that people are less ignorant of all the important subjects.
You don't really address the Jackson et al (2016) https://gsppi.berkeley.edu/~ruckerj/QJE_resubmit_final_version.pdf?utm_source=substack&utm_medium=email .
That's the study of what happens after court ordered funding increases. You just say it's a complex study. Yes, and looking at it, I sympathize with you. But they are economists, not ed school people, so we can trust them prima facie (and their funding is NSF, not Dept of Ed.). And they thank some good economists, e.g. Larry Katz.
One probably it looks like they haven't addressed is lags. We wouldn't expect much change in the first year of paying teachers more, only later when they hire better teachers, for example. There are lots of other potential pitfalls. Their techniques are not advanced, tho, not fancy.
Given that results of complex studies change dramatically when different research teams are asked to analyze the same data to answer the same question, there is plenty of room for doubt. What they should do is release the data and have 10 other teams analyze it and see if they agree.
https://www.emilkirkegaard.com/p/different-researchers-same-dataset-and-questions-what-happens
Economists have always been pretty good at sharing their data used in a paper. We're shocked at fields where this is not true. As of about a decade ago, journals even require this, and require the computer code too. So probably someone could get it for this QJE paper. Of course, replication is a big project, too big for me, for examlpe, to want to spend time on. If anyone has trouble geting the Jackson et al data, tho, let me know-- I'm a senior economists, and I bet I could gte it from them. erasmuse61@gmail.com
Well, why don't you find a copy of their analysis code and data? Then give it a try. Then at least we have results from 2 teams.
it is funny how this whole area (of wishful thinking) seeks to squeeze out a statistically significant finding and declare success. how about an effect that has any practical meaning?! Effect sizes matter--esp when it is my money paying for it. we all know what the issue is driving educational performance. it is more than skin deep.