Statistical inference

Statistical inference is the process of using data analysis to infer properties of an underlying distribution of probability. Inferential statistical analysis infers properties of a population, for example by testing hypotheses and deriving estimates. It is assumed that the observed data set is sampled from a larger population.

Inferential statistics can be contrasted with descriptive statistics. Descriptive statistics is solely concerned with properties of the observed data, and it does not rest on the assumption that the data come from a larger population. In machine learning, the term inference is sometimes used instead to mean "make a prediction, by evaluating an already trained model"; in this context inferring properties of the model is referred to as training or learning (rather than inference), and using a model for prediction is referred to as inference (instead of prediction); see also predictive inference.

Introduction

Statistical inference makes propositions about a population, using data drawn from the population with some form of sampling. Given a hypothesis about a population, for which we wish to draw inferences, statistical inference consists of (first) selecting a statistical model of the process that generates the data and (second) deducing propositions from the model.

Konishi & Kitagawa state, "The majority of the problems in statistical inference can be considered to be problems related to statistical modeling". Relatedly, Sir David Cox has said, "How [the] translation from subject-matter problem to statistical model is done is often the most critical part of an analysis".

The conclusion of a statistical inference is a statistical proposition. Some common forms of statistical proposition are the following:

• a point estimate, i.e. a particular value that best approximates some parameter of interest;
• an interval estimate, e.g. a confidence interval (or set estimate), i.e. an interval constructed using a dataset drawn from a population so that, under repeated sampling of such datasets, such intervals would contain the true parameter value with the probability at the stated confidence level;
• a credible interval, i.e. a set of values containing, for example, 95% of posterior belief;
• rejection of a hypothesis;
• clustering or classification of data points into groups.

History

Al-Kindi, an Arab mathematician in the 9th century, made the earliest known use of statistical inference in his Manuscript on Deciphering Cryptographic Messages, a work on cryptanalysis and frequency analysis.

See also

• Algorithmic inference
• Induction (philosophy)
• Informal inferential reasoning
• Population proportion
• Philosophy of statistics
• Predictive inference
• Information field theory

Notes

Template:NoteFoot

References

Sources

• Bandyopadhyay, P. S.; Forster, M. R., eds. (2011), Philosophy of Statistics, Elsevier .
• Bickel, Peter J.; Doksum, Kjell A. (2001). Mathematical statistics: Basic and selected topics. 1 (Second (updated printing 2007) ed.). Prentice Hall. ISBN 978-0-13-850363-5. MR 443141. 
• Cox, D. R. (2006). Principles of Statistical Inference, Cambridge University Press. ISBN 0-521-68567-2.
• Fisher, R. A. (1955), "Statistical methods and scientific induction", Journal of the Royal Statistical Society, Series B, 17, 69–78. (criticism of statistical theories of Jerzy Neyman and Abraham Wald)
• Freedman, D. A. (2009). Statistical Models: Theory and practice (revised ed.). Cambridge University Press. pp. xiv+442 pp. ISBN 978-0-521-74385-3. MR 2489600. 
• Freedman, D. A. (2010). Statistical Models and Causal Inferences: A Dialogue with the Social Sciences (Edited by David Collier, Jasjeet Sekhon, and Philip B. Stark), Cambridge University Press.
• Hampel, Frank (Feb 2003). The proper fiducial argument. http://e-collection.library.ethz.ch/eserv/eth:26403/eth-26403-01.pdf. 
• Hansen, Mark H.; Yu, Bin (June 2001). "Model Selection and the Principle of Minimum Description Length: Review paper". Journal of the American Statistical Association 96 (454): 746–774. doi:10.1198/016214501753168398. JSTOR 2670311. MR 1939352. http://www.stat.berkeley.edu/webmastr/users/binyu/ps/mdl.ps. 
• Hinkelmann, Klaus; Kempthorne, Oscar (2008). Introduction to Experimental Design (Second ed.). Wiley. ISBN 978-0-471-72756-9. https://books.google.com/books?id=T3wWj2kVYZgC. 
• Kolmogorov, Andrei N. (1963). "On tables of random numbers". Sankhyā Ser. A. 25: 369–375. MR 178484.  Reprinted as Kolmogorov, Andrei N. (1998). "On tables of random numbers". Theoretical Computer Science 207 (2): 387–395. doi:10.1016/S0304-3975(98)00075-9. MR 1643414. 
• Konishi S., Kitagawa G. (2008), Information Criteria and Statistical Modeling, Springer.
• Kruskal, William (December 1988). "Miracles and statistics: the casual assumption of independence (ASA Presidential Address)". Journal of the American Statistical Association 83 (404): 929–940. doi:10.2307/2290117. JSTOR 2290117. 
• Le Cam, Lucian. (1986) Asymptotic Methods of Statistical Decision Theory, Springer. ISBN 0-387-96307-3
• Moore, D. S.; McCabe, G. P.; Craig, B. A. (2015), Introduction to the Practice of Statistics, Eighth Edition, Macmillan.
• Neyman, Jerzy (1956). "Note on an article by Sir Ronald Fisher". Journal of the Royal Statistical Society, Series B 18 (2): 288–294. doi:10.1111/j.2517-6161.1956.tb00236.x. JSTOR 2983716.  (reply to Fisher 1955)
• Peirce, C. S. (1877–1878), "Illustrations of the logic of science" (series), Popular Science Monthly, vols. 12–13. Relevant individual papers:
  • (1878 March), "The Doctrine of Chances", Popular Science Monthly, v. 12, March issue, pp. 604–615. Internet Archive Eprint.
  • (1878 April), "The Probability of Induction", Popular Science Monthly, v. 12, pp. 705–718. Internet Archive Eprint.
  • (1878 June), "The Order of Nature", Popular Science Monthly, v. 13, pp. 203–217.Internet Archive Eprint.
  • (1878 August), "Deduction, Induction, and Hypothesis", Popular Science Monthly, v. 13, pp. 470–482. Internet Archive Eprint.
• Peirce, C. S. (1883), "A Theory of probable inference", Studies in Logic, pp. 126-181, Little, Brown, and Company. (Reprinted 1983, John Benjamins Publishing Company, ISBN 90-272-3271-7)
• Freedman, D.A; Pisani, R.; Purves, R.A. (1978). Statistics. New York: W. W. Norton & Company. https://archive.org/details/statistics0000free. 
• Pfanzagl, Johann; with the assistance of R. Hamböker (1994). Parametric Statistical Theory. Berlin: Walter de Gruyter. ISBN 978-3-11-013863-4. MR 1291393. 
• Rissanen, Jorma (1989). Stochastic Complexity in Statistical Inquiry. Series in Computer Science. 15. Singapore: World Scientific. ISBN 978-9971-5-0859-3. MR 1082556. 
• Soofi, Ehsan S. (December 2000). "Principal information-theoretic approaches (Vignettes for the Year 2000: Theory and Methods, ed. by George Casella)". Journal of the American Statistical Association 95 (452): 1349–1353. doi:10.1080/01621459.2000.10474346. JSTOR 2669786. MR 1825292. 
• Traub, Joseph F.; Wasilkowski, G. W.; Wozniakowski, H. (1988). Information-Based Complexity. Academic Press. ISBN 978-0-12-697545-1. 
• Zabell, S. L. (Aug 1992). "R. A. Fisher and Fiducial Argument". Statistical Science 7 (3): 369–387. doi:10.1214/ss/1177011233. JSTOR 2246073. 

Further reading

• Casella, G., Berger, R. L. (2002). Statistical Inference. Duxbury Press. ISBN 0-534-24312-6
• Freedman, D.A. (1991). "Statistical models and shoe leather". Sociological Methodology 21: 291–313. doi:10.2307/270939. JSTOR 270939. 
• Held L., Bové D.S. (2014). Applied Statistical Inference—Likelihood and Bayes (Springer).
• Lenhard, Johannes (2006). "Models and Statistical Inference: the controversy between Fisher and Neyman–Pearson". British Journal for the Philosophy of Science 57: 69–91. doi:10.1093/bjps/axi152. http://www.stats.org.uk/statistical-inference/Lenhard2006.pdf. 
• Lindley, D (1958). "Fiducial distribution and Bayes' theorem". Journal of the Royal Statistical Society, Series B 20: 102–7. 
• Rahlf, Thomas (2014). "Statistical Inference", in Claude Diebolt, and Michael Haupert (eds.), "Handbook of Cliometrics ( Springer Reference Series)", Berlin/Heidelberg: Springer. http://www.springerreference.com/docs/html/chapterdbid/372458.html
• Reid, N.; Cox, D. R. (2014). "On Some Principles of Statistical Inference". International Statistical Review 83 (2): 293–308. doi:10.1111/insr.12067. 
• Young, G.A., Smith, R.L. (2005). Essentials of Statistical Inference, CUP. ISBN 0-521-83971-8

External links

Template:Wikiversity

• MIT OpenCourseWare: Statistical Inference
• NPTEL Statistical Inference, youtube link
• Statistical induction and prediction

Template:Portal bar