Certainty In Mathematics and Physics 

Rigorous proof (of the kind that supposedly distinguishes mathematics from physics) resides only within a formal system. Each theorem of a formal system can be viewed as just a single data point that either does or does not contradict some other theorem of the system. After we have explored a given formal system for a long time we may feel very confident that it is consistent but, needless to say, no finite number of contradictionfree theorems can constitute a proof of consistency. Our confidence in PA, ZFC, or any other formal system is necessarily based on an incomplete induction. It's always possible that a given formal system could exhibit an inconsistency at some point. In fact, it's been suggested that every formal system, if pressed far enough, is inconsistent. Nothing guarantees us the existence of a consistent formal system with enough complexity to encompass arithmetic. 

On the other hand, the existence of a contradiction in a formal system need not completely vitiate the system. The only operative consistency in any system, mathematical or physical, is local. We might define a metric on the space generated by the axioms of a system, and find that there is consistency within a certain region of that space, even though there may be global inconsistencies. Everyone is familiar with a similar situation in physics, such as the success of Newtonian mechanics in the quasistatic limit of low relative velocities. However, we shouldn't overlook examples in mathematics, such as the naive set theory of Cantor, Frege, et al, that was developed extensively over a period of many years, with many meticulously proven theorems, before it was recognized that the system itself was illfounded and inconsistent (unless restrictions analogous to the quasistatic stipulation for Newtonian mechanics) are imposed. 

The Winter 97 issue of Mathematical Intelligencer contained an article by Dan Velleman discussing Wiles' proof of Fermat's Last Theorem (FLT), and pointed out that what has really been proven (assuming Wiles' proof has no flaw) is not "FLT" but rather "If ZF is consistent then FLT". Of course, even though our belief in ZF is based on incomplete induction, very few mathematicians think it's worth worrying about. Admittedly a generation of competent mathematicians were deceived by naive set theory, but we tend to think that our thought processes have subsequently become infallible. 

Regarding the contrast between physical theories and mathematical theories, we note that a physical theory such as Newtonian mechanics can also be construed reductively as a purely mathematical theory, and as such it may be internally consistent, even if its physical validity is in doubt. However, I'd venture to say that Newtonian mechanics is mathematically complex enough that even its purely mathematical consistency can't be rigorously proven except perhaps within a formal system whose consistency is even more dubious. 

Still, a physical theory is by definition a theory of knowledge that associates (or identifies) some of its elements with observable quantities. That identification is part of the theory, and is what makes it a physical rather than a purely mathematical theory. Newtonian mechanics, interpreted as a physical theory, does assert a correspondence between its elements and certain measureable quantities, and in this sense it fails (as a physical theory, not necessarily as mathematical theory) when pressed to extremes of relative velocities. Whether it also fails at some point as a purely mathematical theory is a separate (and interesting) question. 

Perhaps a better example would have been the theory of general relativity, which predicts  and breaks down at  singularities in the spacetime field. In this case the breakdown is both physical and mathematical. 

In response to my assertion that our confidence in PA, ZFC, or any other formal system is necessarily based on an incomplete induction, Torkel Franzen replied "There is nothing necessary about it. ...it's doubtful whether anybody actually accepts those theories on the basis of inductive evidence." However, this claim is refuted by Velleman, who wrote 

It might be thought that the consistency of ZF is sufficiently wellestablished that it is not worth worrying about. After all, mathematicians have been working with ZF for most of this century, and no contradictions have been discovered yet. 

This is virtually the definition of incomplete induction. Still, Franzen argues that this all misses the point, because "The consistency of arithmetic is a consequence of the truth of its axioms." One is tempted to add that the "truth" of arithmetic's axioms is a consequence of their consistency. But seriously, labelling a set of axioms as "true" doesn't constitute a rigorous formal proof of their consistency. For example, Frege and others regarded the axioms of naive set theory as manifestly true, even more so than the axioms of arithmetic... but they were wrong. 

In response to my suggesting the possibility that every formal system, if pressed far enough, would be fond to be inconsistent, and that nothing guarantees us the existence of a consistent formal system with enough complexity to encompass arithmetic, Franzen replied "This is like a suggestion that 0 and 1 might turn out to be identical if we look hard enough." However, it's worth noting that when it was realized the axioms of naive set theory were flawed and led to contradictions, we did not conclude (as we were strictly entitled to do) that 0 and 1 were identical. Rather we concluded that the axioms of set theory were flawed and needed to be modified. Some proofs, and even some theorems, were rendered invalid in the new system, but happily we were able to preserve the distinctness of 0 and 1. 

In response to Velleman's assertion that what Wiles has really been proven is not {FLT} but rather {If ZF is consistent then FLT}, Franzen asked "What does "really" mean here?" Let me try to explain. People who believe that ZF is manifestly true and consistent tend to regard the two statements {FLT} and {If ZF is consistent then FLT} as equivalent. As such they tend to say that Wiles has proved FLT. The word "really" was intended to remind those people that the two statements are not equivalent, and that Wiles has proved the latter, not the former. This is essentially just a paraphrase of Velleman's article, which concludes 

For those who believe in the existence of the universe of all sets, and who believe that the ZF axioms are true statements about this universe, there is no reason to doubt the reliability of Wiles' proof. But for those who are skeptical about the existence of such a universe, the question remains: Should we be convinced by Wiles' proof that no counterexample will ever be found, or merely that if ZF is consistent then no counterexample will be found? 

Of course, one could argue that it's a bit unfair to single out Wiles' proof to highlight this clarification, since the same qualifier could be applied to every mathematical proof carried out within the ZF framework − which is essentially all of mathematics. On the other hand, the issue 
becomes more relevant as the "size" and complexity of a proof increases. The more thoroughly a proof exercises all the remotest resources of our axiomatic system, the more likely it is (arguably) to be at risk if a subtle flaw or inconsistency in that axiomatic system ever came to light. 
