Occam's razor (also written as Ockham's razor and in Latin lex parsimoniae, which means 'law of parsimony') is a problem-solving principle devised by William of Ockham (c. 1287–1347), who was an English Franciscan friar and scholastic philosopher and theologian.
The principle states that among competing hypotheses that predict equally well, the one with the fewest assumptions should be selected. Other, more complicated solutions may ultimately prove to provide better predictions, but—in the absence of differences in predictive ability—the fewer assumptions that are made, the better.
The application of the principle can be used to shift the burden of proof in a discussion.
(read with links & references in Wikipedia)
To quote Isaac Newton, "We are to admit no more causes of natural things than such as are both true and sufficient to explain their appearances. Therefore, to the same natural effects we must, as far as possible, assign the same causes."
Bertrand Russell offers a particular version of Occam's razor: "Whenever possible, substitute constructions out of known entities for inferences to unknown entities."
Around 1960, Ray Solomonoff founded the theory of universal inductive inference, the theory of prediction based on observations; for example, predicting the next symbol based upon a given series of symbols. The only assumption is that the environment follows some unknown but computable probability distribution. This theory is a mathematical formalization of Occam's razor.
Another technical approach to Occam's razor is ontological parsimony.
The widespread layperson's formulation that "the simplest explanation is usually the correct one" appears to have been derived from Occam's razor.