Not entirely about Coronavirus!
Bad science, learning effectively, and Coronavirus simulations.
You don't want a criminal journal, you want a criminal journal by Andrew Gelman: My favorite statistics blogger Andrew Gelman calls out a criminology journal for negligent behavior. I’m impressed by Andrew Gelman’s courage to consistently call out behavior like this. I also want to signal-boost people like James Heathers who have less job security than Gelman and also do great work discovering bad science.
I’ve rattled on about this sort of scientific malpractice enough that some of my friends are probably thinking, “why do you care so much? Sturgeon's Law says that 90% of all stuff is bad. Why expect science to be different?” Many reasons but here are the two most salient:
Science matters. Think of how many companies design their training programs, individuals their diet programs, governments their education programs based on science like this.
To give a concrete example, say a state government designs their education program based on Visible Learning and it turns out that the criticisms in that post’s comments are accurate. That’s 10s of thousands of students who learned less than they would’ve and maybe failed a class or standardized test because of bad science.
Goo in the gears (Gelman's take). One idiosyncratic mental model I have for organizational and field-level productivity is that you want to avoid goo in the gears as much as possible.
By goo in the gears, I mean factors which generally slow things down and reduce productivity. In the company context, goo in the gears often shows up as bad process or incompetent individuals who drain lots of other people’s time with either make-work (in the manager case) or endless requests for help (in the subordinate case).
In the science context, one major source of goo in the gears especially for more junior people (like me) is zombie papers. That is, papers which many people know are wrong but which remain in high status journals because of behavior like that of the criminology journal. These sorts of papers not only waste valuable researcher time but also can actively mislead researchers into pursuing the wrong experiments/analyses/etc. based on their results, reducing the overall productivity of the field.
Crowding out of good work. Gelman also talks about this fairly frequently. For every Brian Wansink (article) or Phil Zimbardo (article), there’s 10s (100s?) of currently no-name grad students doing good work (real science) who will not get good jobs or positions because they refused to exaggerate their findings or rush their analyses.
Augmenting long-term memory by Michael Nielsen: I’ve probably read this article 5 times now and I can’t emphasize enough how beneficial its recommendations have been for me. In it, Michael Nielsen (former physics professor and researcher at Y-combinator research) discusses how he uses Anki, a spaced repetition system, to understand a new field with high retention in the long-term.
For a few years now, I’ve been of the view that memorization is overly maligned by technical types and Nielsen captures the reasons why. As I mentioned above, famous scientists, physicists in particular, like Feynman and Einstein have been historically outspoken against rote memory as negatively impacting creative thinking so Nielsen’s status as a successful physicist lends him credibility here.
I keep meaning to write something longer about my use and experience with Anki, but I’m not sure how to balance sharing a personal perspective with giving information that’ll be useful to others.
Finally, if you enjoyed this and are interested in mathematics and mathematical cognition, I highly recommend Michael Nielsen’s follow-up post on using spaced repetition to ‘see through’ a piece of mathematics.
Outbreak by Kevin Simler: I had to include something related to Coronavirus! Simler (who I’ve met!) provides a series of progressively more complicated interactive simulations of disease spread with knobs the reader can tweak to understand how a virus’ (not specific to Coronavirus) spread depends on factors like transmission rate, travel radius, and daily encounters. What I really liked about this is that it empowers the reader to answer their own questions rather than just feeding them information.
If you play with Simler’s simulations and find yourself wanting more knobs and potentially a more complicated model to mess around, with I recommend trying out Gabriel Goh's interactive simulation. Goh uses a SEIR (Susceptible → Exposed → Infected → Removed) model, which is apparently quite similar to the state-of-the-art models used by epidemiologists.