to the right music, and your age will change. Apparently, this is scientifically proven. Psychological Science reports a study where subjects were asked to give their birthday after listening to different songs; listening to the Beatles ("When I'm 64") actually reduces your age...
Huh? The paper (by Simmons et al.) is actually meant to show how easy it is to get even absurd hypothesis above the threshold for statistically significant rejection. In this case, the researchers a. kept adding to the sample until the "right" result emerged b. used a variety of control variables, only one of which actually produced the desired significance level on the coefficient for the main exogenous variable. It is not quite as wild as the IG Nobel - award winning study demonstrating (with MRI imaging) that there was brain activity in a dead salmon, but it is close...
The authors also propose an interesting list of "best practices" to avoid this kind of data mining. Among them is one that economic history journals would do well to implement - the requirement to deposit the data used for a study in an online archive, where other researchers can try to replicate the results. Most leading economics journals already have such a policy; but economic history journals - despite their emphasis on data (or because of it?) - do not require the posting of data, descriptions, or do-files.
Huh? The paper (by Simmons et al.) is actually meant to show how easy it is to get even absurd hypothesis above the threshold for statistically significant rejection. In this case, the researchers a. kept adding to the sample until the "right" result emerged b. used a variety of control variables, only one of which actually produced the desired significance level on the coefficient for the main exogenous variable. It is not quite as wild as the IG Nobel - award winning study demonstrating (with MRI imaging) that there was brain activity in a dead salmon, but it is close...
The authors also propose an interesting list of "best practices" to avoid this kind of data mining. Among them is one that economic history journals would do well to implement - the requirement to deposit the data used for a study in an online archive, where other researchers can try to replicate the results. Most leading economics journals already have such a policy; but economic history journals - despite their emphasis on data (or because of it?) - do not require the posting of data, descriptions, or do-files.
No comments:
Post a Comment