We all know that "publish or perish" is stupid. The premier example of Goodhart's Law: “When a measure becomes a target, it ceases to be a good measure.” Why can't our highly paid administration understand this?
Say what now?
So the only way to get a correction for a paper is if the author is willing to publicly admit they messed up? Something that an unethical researcher is very unlikely to do.
but all flaws are issues, later reported issues are right next to the paper, heck there could even badges for publication and review status...
a woman may dream...
With this, science will probably lose trust even more in the coming years.
> The amount of energy needed to refute bullshit is an order of magnitude bigger than that needed to produce it.
Gish galloping bad faith trolls aren’t new. LLMs shape their BS into fluffy BS that isn’t particularly more effective. But now, We Have The Technology, refuting a pile of poo semi-accurately should be cheap (or at least getting cheaper).
I don’t need an LLM on my phone that can do tax law in Georgia the country. But an “AI Assistant” that could highlight logical fallacies, shifting goalposts, non-responsive dialog, rhetorical obfuscations, etc, would be useful online, at the bar, and work (ie when HR tries to “HR” you, but also is lying and obfuscating about it).
We already have models and people that bullshit. Maybe refutation models are the cure… Chinese needle snakes to catch the lizards, Gorillas to catch the snakes…
> The above story came from my occasional collaborator Andy King [...]
I'll add that the reaction of most of academia will be "It's in a management journal - of course it's nonsense."
Many such cases of this, it seems.
It was never between the left and the right or any other false dichotomies, but always between the Epstein-class and the actual human beings.
The question now is that do the normal people realize and act on the fact that the elevator to Epstein class was never working. Or even better, they don't want to become the zillionaire class husk of a human.
Now the right all around the world is hijacked by narcissistic greed that punishes any voicing of conservative moral.
In the US some republicans are daring to challenge the extreme narcissistic greed and a lot more are thinking about this privately.
I also mentioned the other dichotomies and perhaps the "right" could be hopelessness and the the "left" false hope.
As in "no point trying to curb the emissions or addressing any social causes, because the zillionaires choke hold of the planet" vs. the "eternal green growth and economy will save us and make us rich".
The Trump humangod class is not the right IMO.
I believe that power (via money or absolute power) corrupts and thus me must find a way to prevent individuals from becoming human-gods.
The left (without the "") pretends to know this but ends up being corrupted anyway and the right (the one that has some moral and spine left) seems to believe that they will not be corrupted by power.
Hope that explains it a bit better.
Computer science is very close to math and should be even easier to verify, but there's plenty of dubious results published every year, simply because it's more profitable to game the system. For example, I'd wager that 50%+ of academic claims related to information security are bogus or useless. Similarly, in the physics-adjacent world of materials science, a lot of announcements related to metamaterials and nanotech are suspect.
Most scientific research represents about the same amount of improvement over the state of the art as the shitty web app or whatever that you're working on right now. It's not zero, but very few are going to be groundbreaking. And since the rules are that we all have to publish papers[*], the scientific literature (at least in my field, CS) looks less like a carefully curated library of works by geniuses, and more like an Amazon or Etsy marketplace of ideas, where most are crappy.
[* just like software engineers have to write code, even if the product ends up being shitty or ultimately gets canceled]
Neither of us are going to be changing how the system works, so my advice is to deal with it.
I used to work for the leading statistical expert witness in the country. Whenever I read something like this:
> The empirical strategy in Eccles, Ioannou, and Serafeim (2014) rests on a demanding requirement: the “treated” and “control” firms must be so closely matched that which firm is treated is essentially random. The authors appear to recognize this, reporting that they used very strict matching criteria “to ensure that none of the matched pairs is materially different.”
I just assume they kept trying different "very strict matching criteria" until they got the matches they wanted. Which is basically what we did all day to support our client (usually big auto or big tobacco). We never presented any of the detrimental analyses to our boss, so he couldn't testify about them on the stand even if asked.
Although in this case it sounds like the authors couldn't even do that, and just fudged the data instead.
Math (and theoretical adjacents like TCS) claim not to make any fundamental claims about the actual world (compared to 17th century philosopher-mathematicians like Leibniz), and physics studies the basest of, well, physical phenomenon.
I don't even know how you would begin actually rigorously studying sociology unless you could start simulating real humans in a vat, or you inject everybody with neuralink. (but that already selects for a type of society, and probably not a good one...)
To be clear, I don't think all sociological observations are bad. However, I tend to heavily disregard "mathematical sociological studies" in favor of just... hearing perspectives. New ones and unconventional ones especially, as in a domain where a lot of theories "seem legit", I want to just hear very specific new ways of thinking that I didn't think about before. I find that to be a pretty good heuristic for finding value, if the verification process itself is broken.
Annals of Mathematics once published a supposed proof (related to intersection bodies IIRC) for a statement that turned out to be false, and it was discovered only by someone else proving the opposite, not by someone finding an error.
If it doesn't have "science" in the name, it's a science
If it has the suffix "logy", it's a semi-science
If it has the word "science", it's nothttps://news.ycombinator.com/item?id=46752151
(2 months ago, 374 comments)
There’s no accountability for junk science, especially if it props up the political status quo.
I suspect what it narrowly shows though is that this isn't the same category of error as what's being discussed here.
[1] https://www.mercatus.org/research/policy-briefs/debt-and-gro...
But for the millenniums preceding that, it was the reverse, practice and observation drove theory, and I wonder if we are going back to that and practice and once again dominate how we discover new things as a civilization.
This seems to be the key part. Are you sure that's true?
In other news, (a) apparently you can now submit URLs with anchors to HN, previously a perennial problem; (b) this submission anchors to a comment that just says "I will try this. Suggestions welcome" with no further context.
Ironically, (b) was exactly why (a) was disallowed for the longest time. Anchors are usually a mistake by the submitter, since whatever's being anchored to usually has a permalink. Except Github. Hello, Github comments.
In the academic circles I frequent, it's not true. Any one journal might reject the good stuff, but it doesn't take more than a few applications to find a journal who recognizes it, and the cost of producing the research is so high that with the current career incentives it'd be ridiculous not to continue submitting. That does mean that journal "quality" matters less than you might think, but I don't think anyone's surprised by that notion either.
Errors the other direction are more common. I'll state that as an easily verified fact, but people like fun stories, so here's an example:
One professor I worked with had me write up a bunch of case studies of some math technique, tried to convince me that it was worth a paper, paid somebody else to typeset my work, and told me to compensate him if I wanted my name on the "paper." I didn't really; it was beneath any real mathematician; but there now exists some journal which has a bastardized, plagiarized version of my work with some other unrelated author tacked on available for the world to see [0], and it's worth calling out that nothing about the "paper" is journal-worthy. It's far too easy to find a home for academic slop, and I saw that in every field I spent any serious amount of time in.
[0] https://www.m-hikari.com/ams/ams-2019/ams-9-12-2019/p/jabbar...
I'm just relieved you can submit anchored URLs now. I once stayed up for a few hours trying to submit some work I made as a github comment only to be disappointed that it would always redirect to the toplevel issue.
This isnt a new thing though.
Cantor: https://en.wikipedia.org/wiki/Controversy_over_Cantor%27s_th... they didnt just reject him, they basically publicly beat him down, and drove him away from math and into depression.
David Bohm: https://en.wikipedia.org/wiki/Quantum_potential spent years on the outside for having his ideas on this.
Geoffrey Hinton: was considered a quack and an outsider for YEARS because of his ideas on AI... the breakthrough he spawned was done on a shoestring of a budget (read: home pc).
Edit: I forgot John Yudkin: Pure White and Deadly, talking about how bad sugar is for you in 1972...
Rejected by the mainstream academics, and in a brutal way, happens a LOT more than we think.
Her advisor, Suhadolnik, was a gigantic asshole and paid no price whatsoever for it. University of Pennsylvania demoted her and denied her tenure and nobody involved paid any price for that. etc.
It reminds me of something my dad said while watching Generation Kill - a TV show adapted from the written work of an embedded journalist in Iraq. The show, made by Americans, depicts the US armed forces as ramified through with bumbling fools seeking glory with a few competent people in there. So we finish watching the series and my dad says "Only the Americans would make a show like this" and it's somewhat[1] true. I think perhaps that being able to create a machine that tells you the truth is crucial to success and I feel that the US's peak period as unipolar hegemon (Gulf War I to the end of Obama I) this was more the case than it is today, though this is more of a feeling than anything I have verified.
It also reminds me of an old sort of censorship, one which George Orwell talks about in regards to Animal Farm[2] - a book that was criticized because it perhaps harmed the greater cause of communism. There's too much to quote in his essay because I find the whole thing worthy of reading, but here's one bit:
> Both publicly and privately you were warned that it was ʻnot doneʼ. What you said might possibly be true, but it was ʻinopportuneʼ and played into the hands of this or that reactionary interest.
...
> Is every opinion, however unpopular – however foolish, even – entitled to a hearing? Put it in that form and nearly any English intellectual will feel that he ought to say ʻYesʼ. But give it a concrete shape, and ask, ʻHow about an attack on Stalin? Is that entitled to a hearing?ʼ, and the answer more often than not will be ʻNoʼ. In that case the current orthodoxy happens to be challenged, and so the principle of free speech lapses.
There is even today an orthodoxy of sorts and if you were to contradict it, it is considered sinful to say so. I'm Indian so perhaps it is safe for me to use this as a race of choice but what if it were found that Indians actually are less smart than, say, White people. Could such a thing be published if it were true? People often say "what are you going to do with that information?" and somehow I don't share that view that all science must necessarily immediately deliver applied benefit. Knowing is good for its own sake. Truth is good for its own sake. Or at least that's what I believe.
I suppose I'll only know through the period of my own life whether this belief is adaptive. Who knows, a present or future power might be one formed entirely through inaccurate data and information[3], and we might be as Orks and painting things red might make them faster because we believe it so in sufficient numbers.
0: Obviously there are limits. Eli Lilly benefits from GLP-1RA drugs working well but they do in fact work well.
1: Others obviously also make fun of themselves, but something like In The Loop parodies specific people more than the whole machine and its participants. Generation Kill feels much more real a depiction of large organizations and their incentive mechanisms - especially how they grind forward and get the outcomes they want despite everything else. Perhaps my least favourite parts were the emotional-breakdown bits at the end, which I've since found out that the participants themselves said were invented for TV.
2: https://www.marxists.org/archive/orwell/1945/preface.htm
3: Open societies like ours have the problem that external misdirection leaks into internal data but perhaps with sufficient computerization we can keep separate truth and propaganda within the structure of government
This is an interesting thought experiment.
I'd like to think that if such a thing was discovered, we would investigate why this is the case. I'd like to believe that we wouldn't just accept this as some kind of de-facto truth, and start treating Indians as lesser
I'm an idealist I guess though
(if not trying to highlight that particular comment on it)
If there was a time, it surely wasn't the time of pregnant men, math is racist, and acronyms are White supremacy culture.
- The consolidation of media (& social media in general) is about making money from outrage (emotions)
- Anti Vax (& other) movements is about people only receptive to people saying what they already feel (feelings)
- Accountability is gone because people care about being on the winning team and being "right".
Reason, Logic, and Evidence seems completely replaced by propaganda and mistrust of experts (fueled by the propaganda), but it's all rooted in comfort in people's own emotional validation.
I think it is the exact opposite. Now that anyone in the world can create and share "media", professionals trying to make high brow media cannot compete with the emotional reaction slop that the other 8 billion people put out.
Look at what is popular on Reddit, Youtube, Twitter, TikTok, and now even the federal US government targets the same lowest common denominator. Even Fox News and ESPN cannot compete.
The supply of media sellers is the most unconsolidated it has ever been, with millions of random people recording their own faux outrage and uploading it daily for others to mindlessly consume.
But I wouldn't exactly call "professionals trying to make high brow media" exclusive alternative to Reddit, Youtube, Twitter, TikTok.
A lot of the propaganda (and sane washing) is coming from mass media, too. I feel like the only "legit" media outlets are like Reuters, AP, and some international ones, I guess.
I remember when I was a teenager there were lawsuits about trying to teach creationism in school. My entire life conservatives have been arguing against climate change.
It's not that outrage or unfounded opinions were new, or the masses were never fooled or taken advantage of before. It's that the mechanism for social consensus is rapidly shifting.
> I remember when I was a teenager there were lawsuits about trying to teach creationism in school. My entire life conservatives have been arguing against climate change.
And yet the consensus about climate change and in particular support for policies that address it is very strong.
https://climatecommunication.yale.edu/visualizations-data/yc...
60-70% is far far higher than most politicians win elections by. They'll call 5low-something% a landslide. They push policies and laws that are far less popular than that, claiming popular mandate.
And yet there are a bunch of people fixated on the idea that it is a disadvantaged (poorer, less educated) minority of average citizens of the country who are orchestrating some evil battle against it. Rather than seeing the obvious that the ruling class is as always pushing divide and conquer techniques, shifting blame, and turning people on one another. A good example of the emotional mechanism of social consensus.
If you asked "Do cigarettes contribute to lung cancer", you'd expect 95%+. Our evidence for climate change is on-par with that, and yet the rich have run a wildly successful campaign to cast doubt on it for years.
If people really appreciated the gravity of it, we would not have trump, a demonstrably anti-climate president who has rolled back green policies and slowed decarbonization, and even ran on it. Apparently spiting the "other side" is more important than our planet's long term habitability.
I don't think it's unique to the people of today that people in groups do dumb emotional shit. That's kind of the point of Mark Twain's "The Mysterious Stranger".