While at the University of Iowa, Ziliak became friends with his
dissertation adviser,
Deirdre McCloskey. He and McCloskey shared an interest in the fields of rhetoric and statistical significance — namely how the two concepts merge in modern economics. Ziliak had discovered one big cost of the "significance mistake" early on in his job with Workforce Development, in 1987. By U.S. Department of Labor policy he learned he was not allowed to publish black youth unemployment rates for Indiana's labor markets: "not statistically significant," the Labor Department said, meaning the p-values exceeded 0.10 (p less than or equal to 0.10 was the Labor Department's bright line cut-off for publishing estimates). In their paper, "The Standard Error of Regressions," McCloskey and Ziliak argue that
econometrics greatly over-values and misuses statistical significance testing —
Student's t-test. They claim
econometricians rely too heavily on
statistical significance, but too little on "actual" economic significance. The paper also reviews and critiques over 40 years' worth of published papers in economic
journals on their use of statistical significance. In a reply to critics, Ziliak and McCloskey did a follow-up study of the 1996 research and claimed that the significance problem had grown even larger, causing false inferences and decisions in from 70% in the 1980s to 80% of the 1990s articles published in the
American Economic Review. "Size Matters: The Standard Error of Regressions in the American Economic Review" was presented by Ziliak at the 2004 meetings of the
American Economic Association. The article and a reply to critics ("Significance Redux") were published in a special issue of the Journal of Socio-Economics. Published cooperatively at the same time in
Econ Journal Watch (2004), "Size Matters" maintains its rank as one of the top-most downloaded articles in that journal's history. Ziliak was a lead author on the twenty-four statistician team which crafted in 2015-2016 the "
American Statistical Association Statement on Statistical Significance and P-Values," edited by Ronald Wasserstein and Nicole Lazar. His article "How Large are Your G-values? Try Gosset's Guinnessometrics When a Little 'p' Is Not Enough" was published in a follow-up special issue of
The American Statistician (2019 73 sup1), a major re-think of statistical testing, estimation, and reporting in "A world beyond p<0.05" for which Ziliak also served as associate editor. ==
The Cult of Statistical Significance==