Mathematical Background: Foundations of Infinitesimal Calculus second edition

Mathematical Background: Foundations of Infinitesimal Calculus second edition

 

by

K. D. Stroyan

x

y

y=f(x)

dx

dy

δx

ε

dx

dy

Figure 0.1: A Microscopic View of the Tangent

Copyright c

1997 by Academic Press, Inc. - All rights reserved.

Typeset with AMS-TEX

i

Preface to the Mathematical Background

We want you to reason with mathematics. We are not trying to get everyone to give formalized proofs in the sense of contemporary mathematics; ‘proof’ in this course means ‘convincing argument.’ We expect you to use correct reasoning and to give careful explanations. The projects bring out these issues in the way we find best for most students, but the pure mathematical questions also interest some students. This book of mathematical “background” shows how to fill in the mathematical details of the main topics from the course. These proofs are completely rigorous in the sense of modern mathematics – technically bulletproof. We wrote this book of foundations in part to provide a convenient reference for a student who might like to see the “theorem - proof” approach to calculus. We alsowroteit for the interestedinstructor. In re-thinking the presentationof beginning calculus, we found that a simpler basis for the theory was both possible and desirable. The pointwise approach most books give to the theory of derivatives spoils the subject. Clear simple argumentslike the proofof the FundamentalTheoremat the startof Chapter 5 below are not possible in that approach. The result of the pointwise approach is that instructors feeltheyhaveto either bedishonestwithstudents ordisclaimgoodintuitiveapproximations. This is sad because it makes a clear subject seem obscure. It is also unnecessary – by and large, the intuitive ideas work provided your notion of derivative is strong enough. This book shows how to bridge the gap between intuition and technical rigor. A function with a positive derivative ought to be increasing. After all, the slope is positive and the graph is supposed to look like an increasing straight line. How could the function NOT be increasing? Pointwise derivatives make this bizarre thing possible - a positive “derivative” of a non-increasing function. Our conclusion is simple. That definition is WRONG in the sense that it does NOT support the intended idea. You might agree that the counterintuitive consequences of pointwise derivatives are unfortunate, but are concerned that the traditional approach is more “general.” Part of the point of this book is to show students and instructors that nothing of interest is lost and a great deal is gained in the straightforward nature of the proofs based on “uniform” derivatives. It actually is not possible to give a formula that is pointwise differentiable and not uniformly differentiable. The pieced together pointwise counterexamples seem contrived and out-of-place in a course where students are learning valuable new rules. It is a theorem that derivatives computed by rules are automatically continuous where defined. We want the course development to emphasize good intuition and positive results. This background shows that the approach is sound. This book also shows how the pathologies arise in the traditional approach – we left pointwise pathology out of the main text, but present it here for the curious and for comparison. Perhaps only math majors ever need to know about these sorts of examples, but they are fun in a negative sort of way. This book also has several theoretical topics that are hard to find in the literature. It includes a complete self-contained treatment of Robinson’s modern theory of infinitesimals, first discovered in 1961. Our simple treatment is due to H. Jerome Keisler from the 1970’s. Keisler’s elementary calculus using infinitesimals is sadly out of print. It used pointwise derivatives, but had many novel ideas, including the first modern use of a microscope to describe the derivative. (The l’Hospital/Bernoulli calculus text of 1696 said curves consist of infinitesimal straight segments, but I do not know if that was associated with a magnifying transformation.) Infinitesimals give us a very simple way to understand the uniform

ii

derivatives, although this can also be clearly understood using function limits as in the text by Lax, et al, from the 1970s. Modern graphical computing can also help us “see” graphs convergeas stressed in our main materials and in the interesting Uhl, Porta, Davis, Calculus & Mathematica text. Almost all the theorems in this book are well-known old results of a carefully studied subject. The well-known ones are more important than the few novel aspects of the book. However, some details like the converseof Taylor’s theorem – both continuous and discrete – arenotsoeasytofindintraditionalcalculussources. Themicroscopetheoremfordifferential equations does not appear in the literature as far as we know, though it is similar to research work of Francine and Marc Diener from the 1980s. We conclude the book with convergence results for Fourier series. While there is nothing novel in our approach, these results have been lost from contemporary calculus and deserve to be part of it. Our development follows Courant’s calculus of the 1930s giving wonderful results of Dirichlet’s era in the 1830s that clearly settle some of the convergence mysteries of Euler from the 1730s. This theory and our development throughout is usually easy to apply. “Clean” theory should be the servant of intuition – building on it and making it stronger and clearer. There is more that is novel about this “book.” It is free and it is not a “book” since it is not printed. Thanks to small marginal cost, our publisher agreed to include this electronic text on CD at no extra cost. We also plan to distribute it over the world wide web. We hope our fresh look at the foundations of calculus will stimulate your interest. Decide for yourself what’s the best way to understand this wonderful subject. Give your own proofs.

Contents

Part 1 Numbers and Functions Chapter 1. Numbers 3 1.1 Field Axioms 3 1.2 Order Axioms 6 1.3 The Completeness Axiom 7 1.4 Small, Medium and Large Numbers 9 Chapter 2. Functional Identities 17 2.1 Specific Functional Identities 17 2.2 General Functional Identities 18 2.3 The Function Extension Axiom 21 2.4 Additive Functions 24 2.5 The Motion of a Pendulum 26 Part 2 Limits Chapter 3. The Theory of Limits 31 3.1 Plain Limits 32 3.2 Function Limits 34 3.3 Computation of Limits 37 Chapter 4. Continuous Functions 43 4.1 Uniform Continuity 43 4.2 The Extreme Value Theorem 44

iii

iv Contents

4.3 Bolzano’s Intermediate Value Theorem 46 Part 3 1 Variable Differentiation Chapter 5. The Theory of Derivatives 49 5.1 The Fundamental Theorem: Part 1 49 5.1.1 Rigorous Infinitesimal Justification 52 5.1.2 Rigorous Limit Justification 53 5.2 Derivatives, Epsilons and Deltas 53 5.3 Smoothness ⇒ Continuity of Function and Derivative 54 5.4 Rules ⇒ Smoothness 56 5.5 The Increment and Increasing 57 5.6 Inverse Functions and Derivatives 58 Chapter 6. Pointwise Derivatives 69 6.1 Pointwise Limits 69 6.2 Pointwise Derivatives 72 6.3 Pointwise Derivatives Aren’t Enough for Inverses 76 Chapter 7. The Mean Value Theorem 79 7.1 The Mean Value Theorem 79 7.2 Darboux’s Theorem 83 7.3 Continuous Pointwise Derivatives are Uniform 85 Chapter 8. Higher Order Derivatives 87 8.1 Taylor’s Formula and Bending 87 8.2 Symmetric Differences and Taylor’s Formula 89 8.3 Approximation of Second Derivatives 91 8.4 The General Taylor Small Oh Formula 92 8.4.1 The Converse of Taylor’s Theorem 95 8.5 Direct Interpretation of Higher Order Derivatives 98 8.5.1 Basic Theory of Interpolation 99 8.5.2 Interpolation where f is Smooth 101 8.5.3 Smoothness From Differences 102 Part 4 Integration Chapter 9. Basic Theory of the Definite Integral 109 9.1 Existence of the Integral 110

Contents v

9.2 You Can’t Always Integrate Discontinuous Functions 114 9.3 Fundamental Theorem: Part 2 116 9.4 Improper Integrals 119 9.4.1 Comparison of Improper Integrals 121 9.4.2 A Finite Funnel with Infinite Area? 123 Part 5 Multivariable Differentiation Chapter 10. Derivatives of Multivariable Functions 127 Part 6 Differential Equations Chapter 11. Theory of Initial Value Problems 131 11.1 Existence and Uniqueness of Solutions 131 11.2 Local Linearization of Dynamical Systems 135 11.3 Attraction and Repulsion 141 11.4 Stable Limit Cycles 143 Part 7 Infinite Series Chapter 12. The Theory of Power Series 147 12.1 Uniformly Convergent Series 149 12.2 Robinson’s Sequential Lemma 151 12.3 Integration of Series 152 12.4 Radius of Convergence 154 12.5 Calculus of Power Series 156 Chapter 13. The Theory of Fourier Series 159 13.1 Computation of Fourier Series 160 13.2 Convergence for Piecewise Smooth Functions 167 13.3 Uniform Convergence for Continuous Piecewise Smooth Functions 173 13.4 Integration of Fourier Series 175

-4 -2 2 4

w

-4

-2

2

4

x

Part 1

Numbers and Functions

2

CHAPTER 1 Numbers This chapter gives the algebraic laws of the number systems used in calculus.

Numbers represent various idealized measurements. Positive integers may count items, fractions may represent a part of an item or a distance that is part of a fixed unit. Distance measurements go beyond rational numbers as soon as we consider the hypotenuse of a right triangle or the circumference of a circle. This extension is already in the realm of imagined “perfect” measurements because it corresponds to a perfectly straight-sided triangle with perfect rightangle, or a perfectly round circle. Actual realmeasurements arealwaysrational and have some error or uncertainty. The various “imaginary” aspects of numbers are very useful fictions. The rules of computation with perfect numbers are much simpler than with the error-containing real measurements. This simplicity makes fundamental ideas clearer. Hyperreal numbers have ‘teeny tiny numbers’ that will simplify approximationestimates. Direct computations with the ideal numbers produce symbolic approximations equivalent to the function limits needed in differentiation theory (that the rules of Theorem 1.12 give a direct way to compute.) Limit theory does not give the answer, but only a way to justify it once you have found it.

1.1 Field Axioms

The laws of algebra follow from the field axioms. This means that algebra is the same with Dedekind’s “real” numbers, the complex numbers, and Robinson’s “hyperreal” numbers.

3

4 1. Numbers

Axiom 1.1. Field Axioms A “field” of numbers is any set of objects together with two operations, addition and multiplication where the operations satisfy: • The commutative laws of addition and multiplication, a1 + a2 = a2 + a1 & a1 ·a2 = a2 ·a1 • The associative laws of addition and multiplication, a1 +(a2+a3)=(a1+a2)+a3 & a1·(a2·a3)=(a1·a2)·a3 •The distributive law of multiplication over addition, a1 ·(a2 + a3)=a1·a2+a1·a3 •There is an additive identity, 0, with0+a=afor every number a. • There is an multiplicative identity, 1, with1·a=afor every number a 6=0. •Each number a has an additive inverse, −a, witha+(−a)=0. •Each nonzero number a has a multiplicative inverse, 1 a, witha·1 a=1. A computation needed in calculus is Example 1.1. The Cube of a Binomial

 

(x +∆x)3 =x3+3x2∆x+3x∆x2+∆x3 =x3+3x2∆x+(∆x(3x +∆x))∆x

We analyze the term ε = (∆x(3x +∆x)) in differentiation. Thereadercouldlaboriouslydemonstratethatonlythefieldaxiomsareneededtoperform the computation. This means it holds for rational, real, complex, or hyperreal numbers. Here is a start. Associativity is needed so that the cube is well defined, or does not depend on the order we multiply. We use this in the next computation, then use the distributive property, the commutativity and the distributive property again, and so on.

(x+∆x)3 =(x+∆x)(x +∆x)(x +∆x) =(x+∆x)((x +∆x)(x +∆x)) =(x+∆x)((x +∆x)x+(x+∆x)∆x) =(x+∆x)((x2 + x∆x)+(x∆x+∆x2)) =(x+∆x)(x2 + x∆x + x∆x +∆x2) =(x+∆x)(x2 +2x∆x+∆x2) =(x+∆x)x2+(x+∆x)2x∆x +(x+∆x)∆x2) . . .

The natural counting numbers 1,2,3,...have operations of addition and multiplication, but do not satisfy all the properties needed to be a field. Addition and multiplication do satisfy the commutative, associative, and distributive laws, but there is no additive inverse

Field Axioms 5

0 in the counting numbers. In ancient times, it was controversial to add this element that could stand for counting nothing, but it is a useful fiction in many kinds of computations. The negative integers −1,−2,−3,...are another idealization added to the natural numbers that make additive inverses possible - they are just new numbers with the needed property. Negative integers have perfectly concrete interpretations such as measurements to the left, rather than the right, or amounts owed rather than earned. The set of all integers; positive, negative, and zero, still do not form a field because there are no multiplicative inverses. Fractions,±1/2,±1/3, ...are the needed additional inverses. When they are combined with the integers through addition, we have the set of all rational numbers of the form ±p/q for natural numbers p and q 6= 0. The rational numbers are a field, that is, they satisfy all the axioms above. In ancient times, rationals were sometimes considered only “operators” on “actual” numbers like 1,2,3,.... The point of the previous paragraphs is simply that we often extend one kind of number system in order to have a new system with useful properties. The complex numbers extend the field axioms above beyond the “real” numbers by adding a number i that solves the equation x2 = −1. (See the CD Chapter 29 of the main text.) Hundreds of years ago this number was controversial and is still called “imaginary.” In fact, all numbers are useful constructs of our imagination and some aspects of Dedekind’s “real” numbers are much more abstract than i2 = −1. (For example, since the reals are “uncountable,” “most” real numbers have no description what-so-ever.) The rationals are not “complete” in the sense that the linear measurement of the side of an equilateral right triangle (√2) cannot be expressed as p/q for p and q integers. In Section 1.3 we “complete” the rationals to form Dedekind’s “real” numbers. These numbers correspond to perfect measurements along an ideal line with no gaps. The complex numbers cannot be ordered with a notion of “smaller than” that is compatible with the field operations. Adding an “ideal” number to serve as the square root of−1 is not compatible with the square of every number being positive. When we make extensions beyond the real number system we need to make choices of the kind of extension depending on the properties we want to preserve. Hyperreal numbers allow us to compute estimates or limits directly, rather than making inverse proofs with inequalities. Like the complex extension, hyperreal extension of the reals loses a property; in this case completeness. Hyperreal numbers are explained beginning in Section 1.4 below and then are used extensively in this background book to show how many intuitive estimates lead to simple direct proofs of important ideas in calculus. The hyperreal numbers (discovered by Abraham Robinson in 1961) are still controversial because they contain infinitesimals. However, they are just another extended modern number system with a desirable new property. Hyperreal numbers can help you understand limits of real numbers and many aspects of calculus. Results of calculus could be proved without infinitesimals, just as they could be proved without real numbers by using only rationals. Many professors still prefer the former, but few prefer the latter. We believe that is only because Dedekind’s “real” numbers are more familiar than Robinson’s, but we will make it clear how both approaches work as a theoretical background for calculus. There is no controversy concerning the logical soundness of hyperreal numbers. The use ofinfinitesimals inthe earlydevelopmentofcalculus beginning with Leibniz, continuing with Euler, and persisting to the time of Gauss was problematic. The founders knew that their use of infinitesimals was logically incomplete and could lead to incorrect results. Hyperreal numbers are a correct treatment of infinitesimals that took nearly 300 years to discover.

6 1. Numbers

With hindsight, they also have a simple description. The Function Extension Axiom 2.1 explained in detail in Chapter 2 was the missing key.

Exercise set 1.1 1. Show that the identity numbers 0 and 1 are unique. (HINT: Suppose 00 + a = a. Add −ato both sides.) 2. Show that 0·a =0. (HINT: Expand0+b a·awith the distributive law and show that0 ·a+b = b. Then use the previous exercise.) 3. The inverses −a and 1 a are unique. (HINT: Suppose not, 0=a−a=a+b. Add−a to both sides and use the associative property.) 4. Show that −1·a =−a. (HINT: Use the distributive property on 0 = (1−1)·a and use the uniqueness of the inverse.) 5. Show that (−1)·(−1) = 1. 6. Other familiar properties of algebra follow from the axioms, for example, if a3 6=0and a4 6=0, then a 1+a 2 a 3 =a 1 a 3 +a 2 a 3 , a 1·a 2 a 3·a 4 =a 1 a 3· a 2 a 4 & a 3·a 46=0

1.2 Order Axioms Estimation is based on the inequality ≤ of the real numbers.

Oneimportantrepresentationofrationalandrealnumbersisasmeasurementsofdistance along a line. The additive identity 0 is located as a starting point and the multiplicative identity 1 is marked off (usually to the right on a horizontal line). Distances to the right correspond to positive numbers and distances to the left to negative ones. The inequality < indicates which numbers are to the left of others. The abstract properties are as follows. Axiom 1.2. Ordered Field Axioms A a number system is an ordered field if it satisfies the field Axioms 1.1 and has a relation < that satisfies: • Every pair of numbers a and b satisfies exactly one of the relations a = b, a<b, orb<a •If a<band b<c, thena<c. •If a<b, thena+c<b+c. •If 0 <aand 0 <b, then0<a·b. These axioms have simple interpretations on the number line. The first order axiom says that every two numbers can be compared; either two numbers are equal or one is to the left of the other.

The Completeness Axiom 7

The second axiom, called transitivity, says that if a is left of b and b is left of c, then a is left of c. The third axiom says that if a is left of b and we move both by a distance c, then the results are still in the same left-right order. Thefourthaxiomisthemostdifficultabstractly. Allthecompatibilitywithmultiplication is built from it. The rational numbers satisfy all these axioms, as do the real and hyperreal numbers. The complex numbers cannot be ordered in a manner compatible with the operations of addition and multiplication. Definition 1.3. Absolute Value If a is a nonzero number in an ordered field, |a| is the larger of a and −a, that is, |a|= a if −a<aand |a|=−a if a<−a. We let|0 |=0.

Exercise set 1.2 1. If 0 <a, show that −a<0by using the additive property. 2. Show that 0 < 1. (HINT: Recall the exercise that (−1)·(−1) = 1 and argue by contradiction, supposing 0 < −1.) 3. Show that a·a>0for every a 6=0. 4. Show that there is no order < on the complex numbers that satisfies the ordered field axioms. 5. Prove that if a<band c>0, thenc·a<c·b. Prove that if 0 <a<band 0 <c<d, thenc·a<d·b.

1.3 The Completeness Axiom

Dedekind’s “real” numbers represent points on an ideal line with no gaps.

The number √2 is not rational. Suppose to the contrary that √2=q/r for integers q and r with no common factors. Then 2r2 = q2. The prime factorization of both sides must be the same, but the factorization of the squares have an even number distinct primes on each side and the 2 factor is left over. This is a contradiction, so there is no rational number whose square is 2. A length corresponding to √2 can be approximated by (rational) decimals in various ways, for example, 1 < 1.4 < 1.41 < 1.414 < 1.4142 < 1.41421 < 1.414213 < .... There is no rational for this sequence to converge to, even though it is “trying” to converge. For example, all the terms of the sequence arebelow1.41422 < 1.4143 < 1.415 < 1.42 < 1.5 < 2. Even without remembering a fancy algorithm for finding square root decimals, you can test

8 1. Numbers

the successive decimal approximations by squaring, for example, 1.414212 =1 . 9999899241 and 1.414222 =2.0000182084.

It is perfectly natural to add a new number to the rationals to stand for the limit of the better and better approximations to √2. Similarly, we could devise approximations to π and make π the number that stands for the limit of such successive approximations. We would like a method to include “all such possible limits” without having to specify the particular approximations. Dedekind’s approach is to let the real numbers be the collection of all “cuts” on the rational line.

Definition 1.4. A Dedekind Cut A “cut” in an ordered field is a pair of nonempty sets A and B so that: • Every number is either in A or B. • Every a in A is less than every b in B. We may think of√2 defining a cut of the rationalnumbers where A consists of allrational numbers a with a<0 ora2<2 andBconsists of all rational numbers b with b2 > 2. There is a “gap” in the rationals where we would like to have √2. Dedekind’s “real numbers” fill all such gaps. In this case, a cut of real numbers would have to have √2 either in A or in B.

Axiom 1.5. Dedekind Completeness The real numbers are an ordered field such that if A and B form a cut in those numbers, there is a number r such that r is in either A or in B and all other the numbers in A satisfy a<rand in B satisfy r<b.

In other words, every cut on the “real” line is made at some specific number r, so there are no gaps. This seems perfectly reasonable in cases like √2 andπwhere we know specific ways to describe the associated cuts. The only drawback to Dedekind’s number system is that “every cut” is not a very concrete notion, but rather relies on an abstract notion of “every set.” This leads to some paradoxical facts about cuts that do not have specific descriptions, but these need not concern us. Every specific cut has a real number in the middle.

Completeness of the reals means that “approximation procedures” that are “improving” converge to a number. We need to be more specific later, but for example, bounded increasing or decreasing sequences converge and “Cauchy” sequences converge. We will not describe these details here, but take them up as part of our study of limits below.

Completeness has another important consequence, the Archimedean Property Theorem 1.8. We take that up in the next section. The Archimedean Propertysays precisely that the real numbers contain no positive infinitesimals. Hyperreal numbers extend the reals by including infinitesimals. (As a consequence the hyperreals are not Dedekind complete.)

Small, Medium and Large Numbers 9 1.4 Small, Medium and Large Numbers

Hyperreal numbers give us a way to simplify estimation by adding infinitesimal numbers to the real numbers.

We want to have three different intuitive sizes of numbers, very small, medium size, and very large. Most important, we want to be able to compute with these numbers using the same rules of algebra as in high school and separate the ‘small’ parts of our computation. Hyperreal numbers give us these computational estimates. Hyperreal numbers satisfy three axioms which we take up separately below, Axiom 1.7, Axiom 1.9, and Axiom 2.1. As a first intuitive approximation, we could think of these scales of numbers in terms of the computer screen. In this case, ‘medium’ numbers might be numbers in the range -999 to + 999 that name a screen pixel. Numbers closer than one unit could not be distinguished by different screen pixels, so these would be ‘tiny’ numbers. Moreover, two medium numbers a and b would be indistinguishably close, a ≈ b, if their difference was a ‘tiny’ number less than a pixel. Numbers larger in magnitude than 999 are too big for the screen and could be considered ‘huge.’ The screen distinction sizes of computer numbers is a good analogy, but there are difficulties with the algebra of screen - size numbers. We want to have ordinary rules of algebra and the following properties of approximate equality. For now, all you should think of is that ≈ means ‘approximately equals.’ (a) If p and q are medium, so are p+ q and p·q. (b) If ε and δ are tiny, so is ε +δ, that is,ε≈0 andδ≈0 implies ε + δ ≈0. (c) If δ ≈0 andqis medium, then q·δ ≈0. (d) 1/0 is still undefined and 1/x is huge only when x≈0. You can see that the computer number idea does not quite work, because the approximation rulesdon’talwaysapply. If p = 15.37andq =−32.4,then p·q =−497.998≈−498,‘medium times medium is medium,’ however, if p = 888 and q = 777, then p·q is no longer screen size... The hyperreal numbers extend the ‘real’ number system to include ‘ideal’ numbers that obey these simple approximationrules as well as the ordinary rules of algebra and trigonometry. Very small numbers technically are called infinitesimals and what we shall assume that is different from high school is that there are positive infinitesimals. Definition 1.6. Infinitesimal Number A numberδin an ordered field is called infinitesimal if it satisfies 1 2 > 1 3 > 1 4 >···> 1 m >···>|δ| for any ordinary natural counting number m =1,2,3,···. We writea≈band say a is infinitely close to b if the number b−a ≈0 is infinitesimal. This definition is intended to include 0 as “infinitesimal.”

10 1. Numbers

Axiom 1.7. The Infinitesimal Axiom The hyperreal numbers contain the real numbers, but also contain nonzero infinitesimal numbers, that is, numbers δ ≈0, positive, δ>0, but smaller than all the real positive numbers.

This stands in contrast to the following result. Theorem 1.8. The Archimedean Property The hyperreal numbers are not Dedekind complete and there are no positive infinitesimal numbers in the ordinary reals, that is, if r>0is a positive real number, then there is a natural counting number m such that 0 < 1 m <r.

Proof: We define a cut above all the positive infinitesimals. The set A consists of all numbers a satisfying a<1/m for every natural counting number m. The setBconsists of all numbers b such that there is a natural number m with 1/m < b. The pair A, B defines a Dedekind cut in the rationals, reals, and hyperreal numbers. If there is a positive δ in A, then there cannot be a number at the gap. In other words, there is no largest positive infinitesimal or smallestpositivenon-infinitesimal. This is clear because δ<δ+δand2δ is still infinitesimal, while if ε is in B, ε/2 <εmust also be in B. Since the real numbers must have a number at the “gap,” there cannot be any positive infinitesimal reals. Zero is at the gap in the reals and every positive real number is in B. This is what the theorem asserts, so it is proved. Notice that we have also proved that the hyperreals are not Dedekind complete, because the cut in the hyperreals must have a gap. Two ordinary real numbers, a and b, satisfya≈bonly if a = b, since the ordinary real numbers do not contain infinitesimals. Zero is the only real number that is infinitesimal. If you prefer not to say ‘infinitesimal,’ just say ‘δ is a tiny positive number’ and think of ≈ as ‘close enough for the computations at hand.’ The computation rules above are still important intuitively and can be phrased in terms of limits of functions if you wish. The intuitive rules help you find the limit. The next axiom about the new “hyperreal” numbers says that you can continue to do the algebraic computations you learned in high school. Axiom 1.9. The Algebra Axiom (Including < rules.) The hyperreal numbers are an ordered field, that is, they obey the same rules of ordered algebra as the real numbers, Axiom 1.1 and Axiom 1.2.

The algebra of infinitesimals that you need can be learned by working the examples and exercises in this chapter. Functional equations like the addition formulas for sine and cosine or the laws of logs and exponentials are very important. (The specific high school identities are reviewed in the main text CD Chapter 28 on High School Review.) The Function Extension Axiom 2.1 shows how to extend the non-algebraic parts of high school math to hyperreal numbers. This axiom is the key to Robinson’s rigorous theory of infinitesimals and it took 300 years to discover. You will see by working with it that it is a perfectly natural idea, as hindsight often reveals. We postpone that to practice with the algebra of infinitesimals. Example 1.2. The Algebra of Small Quantities

Small, Medium and Large Numbers 11

Let’s re-calculate the increment of the basic cubic using the new numbers. Since the rules of algebra are the same, the same basic steps still work (see Example 1.1), except now we may take x any number and δx an infinitesimal. Small Increment of f[x]=x3 f[x+δx]=(x+δx)3 = x3 +3x2δx+3xδx2 + δx3 f[x+δx]=f[x]+3x2 δx+(δx[3x + δx]) δx f[x+δx]=f[x]+f0[x]δx+ε δx with f0[x]=3x2and ε =(δx[3x + δx]). The intuitive rules above show that ε≈0 whenever x is finite. (See Theorem 1.12 and Example 1.8 following it for the precise rules.) Example 1.3. Finite Non-Real Numbers The hyperreal numbers obey the same rules of algebra as the familiar numbers from high school. We knowthat r+∆ >r, whenever∆ > 0 is anordinarypositivehighschoolnumber. (See the addition property of Axiom 1.2.) Since hyperreals satisfy the same rules of algebra, we also have new finite numbers given by a high school number r plus an infinitesimal, a = r + δ>r The number a = r + δ is different from r, even though it is infinitely close to r. Since δ is small, the difference between a and r is small 0 <a−r=δ≈0 ora ≈ r but a 6= r Here is a technical definition of “finite” or “limited” hyperreal number. Definition 1.10. Limited and Unlimited Hyperreal Numbers A hyperreal number x is said to be finite (or limited) if there is an ordinary natural number m =1,2,3,··· so that |x| < m. If a number is not finite, we say it is infinitely large (or unlimited).

Ordinary real numbers are part of the hyperreal numbers and they are finite because they are smaller than the next integer after them. Moreover, every finite hyperreal number is near an ordinary real number (see Theorem 1.11 below), so the previous example is the most general kind of finite hyperreal number there is. The important thing is to learn to compute with approximate equalities. Example 1.4. A Magnified View of the Hyperreal Line Of course, infinitesimals arefinite, since δ ≈0 implies that|δ| < 1. The finite numbers are not just the ordinary real numbers and the infinitesimals clustered near zero. The rules of algebrasaythatifweaddorsubtractanonzeronumberfromanother,theresultis adifferent number. For example, π−δ<π<π+δ, when 0<δ≈0. These are distinct finite hyperreal numbers but each of these numbers differ by only an infinitesimal, π ≈ π + δ ≈ π −δ. If we plotted the hyperreal number line at unit scale, we could only put one dot for all three. However, if we focus a microscope of power 1/δ at π we see three points separated by unit distances.

12 1. Numbers

X 1/d

Pi Pi + dPi - d

Figure 1.1: Magnification at Pi The basic fact is that finite numbers only differ from reals by an infinitesimal. (This is equivalent to Dedekind’s Completeness Axiom.) Theorem 1.11. Standard Parts of Finite Numbers Every finite hyperreal number x differs from some ordinary real number r by an infinitesimal amount, x−r ≈0 or x ≈ r. The ordinary real number infinitely near x is called the standard part of x, r = st(x).

Proof: Suppose x is a finite hyperreal. Define a cut in the real numbers by letting A be the set of all real numbers satisfying a ≤ x and letting B be the set of all real numbers b with x<b. BothAand B are nonempty because x is finite. Every a in A is below every b in B by transitivity of the order on the hyperreals. The completeness of the real numbers means that there is a real r at the gap between A and B. We must havex≈r, because if x−r>1/m, say, thenr+1/(2m) <xand by the gap property would need to be in B. A picture of the hyperreal number line looks like the ordinary real line at unit scale. We can’t draw far enough to get to the infinitely large part and this theorem says each finite number is indistinguishably close to a real number. If we magnify or compress by new number amounts we can see new structure. You still cannot divide by zero (that violates rules of algebra), but if δ is a positive infinitesimal, we can compute the following:

−δ, δ2,

1 δ

What can we say about these quantities? The idealization of infinitesimals lets us have our cake and eat it too. Since δ 6= 0, we can divide by δ. However, since δ is tiny, 1/δ must be HUGE. Example 1.5. Negative infinitesimals In ordinary algebra, if ∆ > 0, then−∆ < 0, so we can apply this rule to the infinitesimal number δ and conclude that −δ<0, since δ>0. Example 1.6. Orders of infinitesimals

In ordinary algebra, if 0 < ∆ < 1, then 0 < ∆2 < ∆, so 0 <δ2<δ. We want you to formulate this more exactly in the next exercise. Just assume δ is very small, but positive. Formulate what you want to draw algebraically. Try some small ordinary numbers as examples, like δ =0 . 01. Plot δ at unit scale and place δ2 accurately on the figure. Example 1.7. Infinitely large numbers

Small, Medium and Large Numbers 13

For real numbers if 0 < ∆ < 1/n then n<1/∆. Since δ is infinitesimal, 0 <δ<1/n for every natural number n =1 ,2 ,3,...Using ordinary rules of algebra, but substituting the infinitesimal δ, we see thatH=1/δ > n is larger than any natural number n (or is “infinitely large”), that is, 1 < 2 < 3 <...<n<H, for every natural number n. We can “see” infinitely large numbers by turning the microscope around and looking in the other end. The new algebraic rules are the ones that tell us when quantities are infinitely close, a ≈ b. Such rules, of course, do not follow from rules about ordinary high school numbers, but the rules are intuitive and simple. More important, they let us ‘calculate limits’ directly. Theorem 1.12. Computation Rules for Finite and Infinitesimal Numbers (a) If p and q are finite, so arep+qand p·q. (b) If ε and δ are infinitesimal, so is ε+δ. (c) If δ ≈0 and q is finite, then q·δ ≈0. (finite x infsml = infsml) (d) 1/0 is still undefined and 1/x is infinitely large only when x≈0. To understand these rules, just think of p and q as “fixed,” if large, and δ as being as small as you please (but not zero). It is not hard to give formal proofs from the definitions above, but this intuitive understanding is more important. The last rule can be “seen” on the graph of y =1/x. Look at the graph and move down near the values x ≈0.

x

y

14 1. Numbers

justifies the local linearity of x3 at finite values of x, that is, we have used the approximation rules to show that f[x+ δx]=f[x]+f0[x]δx+ε δx with ε ≈0 whenever δx≈0 andxis finite, where f[x]=x3and f0[x]=3x2.

Exercise set 1.4 1. Draw the view of the ideal number line when viewed under an infinitesimal microscope of power 1/δ. Which number appears unit size? How big does δ2 appear at this scale? Where do the numbers δ and δ3 appear on a plot of magnification 1/δ2? 2. Backwards microscopes or compression Draw the view of the new number line when viewed under an infinitesimal microscope with its magnification reversed to power δ (not 1/δ). What size does the infinitely large number H (HUGE) appear to be? What size does the finite (ordinary) number m = 109 appear to be? Can you draw the number H2 on the plot? 3. y = xp ⇒ dy = pxp−1 dx, p =1,2,3,... For each f[x]=xp below: (a) Compute f[x+ δx]−f[x] and simplify, writing the increment equation: f[x+δx]−f[x]=f0[x ]·δx+ε·δx =[term in x but not δx]δx+[observed microscopic error]δx Notice that we can solve the increment equation for ε = f[x+δx]−f[x] δx −f0[x] (b) Show that ε ≈ 0 if δx ≈0 and x is finite. Does x need to be finite, or can it be any hyperreal number and still have ε≈0? (1) If f[x]=x1, thenf 0[x]=1x0=1and ε =0. (2) If f[x]=x2, thenf 0[x]=2xand ε = δx. (3) If f[x]=x3, thenf 0[x]=3x2 and ε = (3x+δx)δx. (4) If f[x]=x4, thenf 0[x]=4x3 and ε = (6x2+4xδx+ δx2)δx. (5) If f[x]=x5, thenf 0[x]=5x4 and ε = (10x3+10x2δx+5xδx2 + δx3)δx. 4. Exceptional Numbers and the Derivative of y = 1 x (a) Let f[x]=1/x and show that f[x + δx]−f[x] δx = −1 x(x+δx) (b) Compute ε = −1 x(x + δx) + 1 x2 = δx· 1 x2(x+δx) (c) Show that this gives f[x+ δx]−f[x]=f0[x ]·δx+ε·δx when f0[x]=−1/x2.(d) Show that ε ≈0 provided x is NOT infinitesimal (and in particular is not zero.)

Small, Medium and Large Numbers 15 5. Exceptional Numbers and the Derivative of y =√x (a) Let f[x]=√xand compute f[x+ δx]−f[x]= 1 √x+δx+√x (b) Compute

ε =

1 √x + δx+√x −

1 2√x = −1 2√x(√x+δx+√x)2 ·δx

(c) Show that this gives

f[x+ δx]−f[x]=f0[x ]·δx+ε·δx

when f0[x]= 1 2√x. (d) Show that ε ≈0 provided x is positive and NOT infinitesimal (and in particular is not zero.) 6. Prove the remaining parts of Theorem 1.12.

16 1. Numbers

CHAPTER 2 Functional Identities In high school you learned that trig functions satisfy certain identities or that logarithms have certain “properties.” This chapter extends the idea of functional identities from specific cases to a defining property of an unknown function.

The use of “unknown functions” is of fundamental importance in calculus, and other branches of mathematics and science. For example, differential equations can be viewed as identities for unknown functions. One reason that students sometimes have difficulty understanding the meaning of derivatives or even the general rules for finding derivatives is that those things involveequations in unknown functions. The symbolic rules for differentiation and the increment approximation defining derivatives involve unknown functions. It is important for you to get used to this “higher type variable,” an unknown function. This chapter can form a bridge between the specific identities of high school and the unknown function variables from rules of calculus and differential equations.

2.1 Specific Functional Identities All the the identities you need to recall from high school are:

(Cos[x])2 +(Sin[x])2 = 1 CircleIden Cos[x+y] = Cos[x]Cos[y]−Sin[x]Sin[y] CosSum Sin[x + y] = Sin[x]Cos[y]+Sin[y]Cos[x] SinSum bx+y = bx by ExpSum (bx)y = bx·y RepeatedExp Log[x·y] = Log[x]+Log[y] LogProd Log[xp]=pLog[x] LogPower

but you must be able to use these identities. Some practice exercises using these familiar identities are given in main text CD Chapter 28.

17

18 2. Functional Identities 2.2 General Functional Identities

A general functional identity is an equation which is satisfied by an unknown function (or a number of functions) over its domain.

The function

f[x]=2x

satisfies f[x + y]=2 (x+y)=2x2 y=f[x ] f[y], so eliminating the two middle terms, we see that the function f[x]=2xsatisfies the functional identity

f[x + y]=f[x ]f[y](ExpSum)

It is important to pay attention to the variable or variables in a functional identity. In order foranequationinvolvingafunctionto beafunctionalidentity, theequationmustbevalidfor all values of the variables in question. Equation (ExpSum) above is satisfied by the function f[x]=2xfor all x and y. For the function f[x]=x, it is true thatf[2+2] = f[2]f[2], but f[3+1]6= f[3]f[1], so = x does not satisfy functional identity (ExpSum). Functional identities are a sort of ‘higher laws of algebra.’ Observe the notational similarity between the distributive law for multiplication over addition, m·(x + y)=m·x+m·y and the additive functional identity

f[x + y]=f[x]+f[y](Additive)

Most functions f[x] do not satisfy the additive identity. For example, 1 x + y 6= 1 x + 1 y and √x+y 6=√x+√y The fact that these are not identities means that for some choices of x and y in the domains of the respective functions f[x]=1/x and f[x]=√x, the two sides are not equal. You will show below that the only differentiable functions that do satisfy the additive functional identity are the functions f[x]=m·x. In other words, the additive functional identity is nearly equivalent to the distributive law; the only unknown (differentiable) function that satisfies it is multiplication. Other functional identities such as the 7 given at the start of this chapter capture the most important features of the functions that satisfy the respective identities. For example, the pair of functions f[x]=1/x and g[x]=√xdo not satisfy the addition formula for the sine function, either. Example 2.1. The Microscope Equation

The “microscope equation” defining the differentiability of a function f[x] (see Chapter 5 of the text), f[x+ δx]=f[x]+f0[x]·δx+ε·δx(Micro)

General Functional Identities 19

with ε ≈ 0 ifδx ≈ 0, is similar to a functional identity in that it involves an unknown function f[x] and its related unknown derivative function f0[x]. It “relates” the function f[x] to its derivative df dx = f0[x]. You should think of (Micro) as the definition of the derivative of f[x] at a givenx, but also keep in mind that (Micro) is the definition of the derivative of any function. If we let f[x] vary over a number of different functions, we get different derivatives. The equation (Micro) can be viewed as an equation in which the function, f[x], is the variable input, and the output is the derivative df dx. To make this idea clearer, we rewrite (Micro) by solving for df dx:)

df dx

= f[x+ δx]−f[x] δx −ε

or

df dx

= lim ∆x→0

f[x+ δx]−f[x] ∆x If we plug in the “input” function f[x]=x2into this equation, the output is df dx =2x. If we plug in the “input” function f[x] = Log[x], the output is df dx = 1 x. The microscope equation involves unknown functions, but strictly speaking, it is not a functional identity, because of the error term ε (or the limit which can be used to formalize the error). It is only an approximate identity. Example 2.2. Rules of Differentiation The various “differentiation rules,” the Superposition Rule, the Product Rule and the Chain Rule (from Chapter 6 of the text) are functional identities relating functions and their derivatives. For example, the Product Rule states: d(f[x]g[x]) dx = df dx g[x]+f[x]dg dx We can think of f[x] andg[x] as “variables” which vary by simply choosing different actual functions for f[x] andg [ x]. Then the Product Rule yields an identity between the choices of f[x] andg[x], and their derivatives. For example, choosing f[x]=x 2and g[x] = Log[x] and plugging into the Product Rule yields d(x2 Log[x]) dx =2xLog[x]+x2 1 x Choosing f[x]=x3and g[x] = Exp[x] and plugging into the Product Rule yields d(x3 Exp[x]) dx =3x2Exp[x]+x3Exp[x] If we choose f[x]=x 5, but do not make a specific choice for g[x], plugging into the Product Rule will yield d(x5g[x]) dx =5x4g[x]+x5dg dx The goal of this chapter is to extend your thinking to identities in unknown functions.

20 2. Functional Identities

Exercise set 2.2 1. (a) Verify that for any positive number, b, the function f[x]=bx satisfies the functional identity (ExpSum) above for all x and y. (b) Is (ExpSum) valid (for all x and y) for the function f[x]=x 2or f[x]=x 3? Justify your answer. 2. Define f[x] = Log[x] where x is any positive number. Why does this f[x] satisfy the functional identities f[x·y]=f[x]+f[y](LogProd) and f[xk]=kf[x](LogPower) where x, y, andkare variables. What restrictions should be placed on x and y for the above equations to be valid? What is the domain of the logarithm? 3. Find values of x and y so that the left and right sides of each of the additive formulas for 1/x and √x above are not equal. 4. Show that 1/x and √x also do not satisfy the identity (SinSum), that is, 1 x + y = 1 x √y +√x 1 y is false for some choices of x and y in the domains of these functions. 5. (a) Suppose that f[x] is an unknown function which is known to satisfy (LogProd) (so f[x] behaves “like” Log[x], but we don’t know if f[x] is Log[x]), and suppose that f[0] is a well-defined number (even though we don’t specify exactly what f[0] is). Show that this function f[x] must be the zero function, that is show that f[x]=0for every x. (Hint: Use the fact that 0∗x =0).(b) Suppose that f[x] is an unknown function which is known to satisfy (LogPower) for all x>0and all k. Show that f[1] must equal 0, f[1] = 0. (Hint: Fixx=1, and try different values of k). 6. (a) Let m and b be fixed numbers and define f[x]=mx+b Verify that if b =0, the above function satisfies the functional identity f[x]=xf[1](Mult) for all x and that if b 6=0,f[x ]will not satisfy (Mult) for all x (that is, given a nonzero b, there will be at least one x for which (Mult) is not true). (b) Prove that any function satisfying (Mult) also automatically satisfies the two functional identities f[x+y]=f[x]+f[y](Additive) and f[xy]=xf[y](Multiplicative) for all x and y.

The Function Extension Axiom 21

(c) Suppose f[x] is a function which satisfies (Mult) (and for now that is the only thing you know about f[x]). Prove that f[x] must be of the form f[x]=m·x , for some fixed number m (this is almost obvious). (d) Prove that a general power function, f[x]=mxk where k is a positve integer and m is a fixed number, will not satisfy (Mult) for all x if k 6=1, (that is, ifk6=1, there will be at least one x for which (Mult) is not true). (e) Prove that f[x] = Sin[x] does not satisfy the additive identity. (f) Prove that f[x]=2x does not satisfy the additive identity. 7. (a) Let f[x] and g[x] be unknown functions which are known to satisfy f[1] = 2, df dx(1) = 3, g(1) =−3, dg dx(1) = 4. Leth ( x)=f[x ] g[x ] . Compute dh dx(1). (b) Differentiate the general Product Rule identity to get a formula for d2(fg) dx2 Use your rule to compute d2(h) dx2 (1) if d2(f) dx2 (1) = 5 and d2(g) dx2 (1) =−2, using othervalues from part 1 of this exercise.

2.3 The Function Extension Axiom

This section shows that all real functions have hyperreal extensions that are “natural” from the point of view of properties of the original function.

Roughly speaking, the Function Extension Axiom for hyperreal numbers says that the natural extension of any real function obeys the same functional identities and inequalities as the original function. In Example 2.7, we use the identity, f[x + δx]=f[x ]·f[δx] with x hyperreal and δx≈0 infinitesimal where f[x] is a real function satisfying f[x+y]= f[x]·f[y]. The reason this statement of the Function Extension Axiom is ‘rough’ is because we need to know precisely which values of the variables are permitted. Logically, we can express the axiom in a way that covers all cases at one time, but this is a little complicated so we will precede that statement with some important examples. The Function Extension Axiom is stated so that we can apply it to the Log identity in the form of the implication (x>0&y>0)⇒Log[x] and Log[y] are defined and Log[x·y] = Log[x]+Log[y] The natural extension of Log[·] is defined for all positivehyperreals and its identities hold for hyperreal numbers satisfying x>0 andy>0. The other identities hold for all hyperreal x and y. To make all such statements implications, we can state the exponential sum equation as (x = x & y = y) ⇒ ex+y = ex ·ey

22 2. Functional Identities

The differential

d(Sin[θ]) = Cos[θ] dθ is a notational summary of the valid approximation Sin[θ + δθ]−Sin[θ] = Cos[θ]δθ+ε·δθ where ε ≈ 0 whenδθ ≈ 0. The derivation of this approximation based on magnifying a circle (given in a CD Section of Chapter 5 of the text) can be made precise by using the Function Extension Axiom in the place where it locates (Cos[θ+δθ],Sin[θ+δθ]) on the unit circle. This is simply using the extension of the (CircleIden) identity to hyperreal numbers, (Cos[θ + δθ])2 +(Sin[θ + δθ])2 = 1. Logical Real Expressions, Formulas and Statements Logical real expressions are built up from numbers and variables using functions. Here is the precise definition. (a) A real number is a real expression. (b) A variable standing alone is a real expression. (c) If E1,E2,···,En are a real expressions and f[x1,x2,···,xn] is a real function of n variables, then f[E1,E2,···,En] is a real expression. A logical real formula is one of the following: (a) An equation between real expressions, E1 = E2. (b) An inequality between real expressions, E1 <E 2,E 1≤E 2,E 1>E 2,E 1≥E 2, or E 16=E 2. (c) A statement of the form “E is defined” or of the form “Eis undefined.” Let S and T be finite sets of real formulas. A logical real statement is an implication of the form, S ⇒T or “whenever every formula in S is true, then every formula in T is true.” Logical real statements allow us to formalize statements like: “Every point in the square below lies in the circle below.” Formalizing the statement does not make it true or false. Consider the figure below.

x

y

Figure 2.1: Square and Circle

The Function Extension Axiom 23

Theinsideofthesquareshowncanbedescribedformallyasthesetofpointssatisfyingthe equationsintheset S ={0≤ x,0≤y,x≤1.2, y ≤1.2}. Theinsideofthecircleshowncan be defined as the set ofpoints satisfying the singleequation T ={(x−1)2+(y−1)2 ≤1.62 }. This is the circle of radius 1.6 centered at the point (1,1). The logical real statement S ⇒T means that every point inside the square lies inside the circle. The statement is true for every real x and y. First of all, it is clear by visual inspection. Second, points (x,y) that make one or more of the formulas in S false produce a false premise, so no matter whether or not they lie in the circle, the implication is logically true (if uninteresting). The logical real statement T ⇒ S is a valid logical statement, but it is false since it says every point inside the circle lies inside the square. Naturally, only true logical real statements transfer to the hyperreal numbers. Axiom 2.1. The Function Extension Axiom Every logical real statement that holds for all real numbers also holds for all hyperreal numbers when the real functions in the statement are replaced by their natural extensions. The Function Extension Axiom establishes the 5 identities for all hyperreal numbers, because x = x and y = y always holds. Here is an example. Example 2.3. The Extended Addition Formula for Sine

S ={x = x,y = y}⇒T={Sin[x] is defined , Sin[y] is defined , Cos[x] is defined , Cos[y] is defined , Sin[x + y] = Sin[x] Cos[y]+Sin[y] Cos[x ] } The informal interpretation of the extended identity is that the addition formula for sine holds for all hyperreals. Example 2.4. The Extended Formulas for Log We may take S to be formulas x>0, y>0 andp=pand T to be the functional identities for the natural log plus the statements “Log[ ] is defined,” etc. The Function Extension Axiom establishes that log is defined for positive hyperreals and satisfies the two basic log identities for positive hyperreals. Example 2.5. Abstract Uses of Function Extension There are two general uses of the Function Extension Axiom that underlie most of the theoretical problems in calculus. These involve extension of the discrete maximum and extension of finite summation. The proof of the Extreme Value Theorem 4.4 below uses a hyperfinite maximum, while the proof of the Fundamental Theorem of Integral Calculus 5.1 uses hyperfinite summation. Equivalence of infinitesimal conditions for derivatives or limits and the “epsilon - delta” real number conditions are usually proved by using an auxiliary real function as in the proof of the limit equivalence Theorem 3.2.

24 2. Functional Identities

Example 2.6. The Increment Approximation Note: The increment approximation f[x+ δx]=f[x]+f0[x]·δx+ε·δx with ε ≈0 forδx≈0 and the simpler statement δx≈0 ⇒ f0[x]≈ f[x]+δx)−f[x] δx are not real logical expressions, because they contain the relation ≈, which is not included in the formation rules for logicalrealstatements. (The relation≈does not apply to ordinary real numbers, except in the trivial case x = y.) For example, if θ is any hyperreal and δθ≈0, then Sin[θ + δθ] = Sin[θ] Cos[δθ]+Sin[δθ] Cos[θ ]

by the natural extension of the addition formula for sine above. Notice that the natural extension does NOT tell us the interesting and important estimate Sin[θ + δθ] = Sin[θ]+δθ·Cos[θ]+ε·δθ with ε ≈ 0 whenδθ ≈0. (I.e., Cos[δθ]=1+ιδθ and Sin[δθ]/δθ ≈ 1 are true, but not real logical statements we can derive just from natural extensions.)

Exercise set 2.3 1. Write a formal logical real statement S ⇒ T that says, “Every point inside the circle of radius 2, centered at(−1 ,3) lies outside the square with sides x =0,y=0,x=1, y=−1. Draw a figure and decide whether or not this is a true statement for all real values of the variables. 2. Write a formal logical real statement S ⇒ T that is equivalent to each of the functional identities on the first page of the chapter and interpret the extended identities in the hyperreals.

2.4 Additive Functions

An identity for an unknown function together with the increment approximation combine to give a specific kind of function. The two ideas combine to give a differential equation. After you have learned about the calculus of the natural exponential function in Chapter 8 of the text, you will easily understand the exact solution of the problem of this section.

Additive Functions 25

In the early 1800s, Cauchy asked the question: Must a function satisfying f[x+y]=f[x]+f[y](Additive) be of the form f[x]=m·x? This was not solved until the late 1800s by Hamel. The answer is “No.” There are some very strange functions satisfying the additive identity that are not simple linear functions. However, these strange functions are not differentiable. We will solve a variant of Cauchy’s problem for differentiable functions. Example 2.7. A Variation on Cauchy’s Problem

Suppose an unknown differentiable function f[x] satisfies the (ExpSum) identity for all x and y, f[x+y]=f[x ]·f[y] Does the function have to be f[x]=bx for some positive b? Since our unknown function f[x] satisfies the (ExpSum) identity and is differentiable, both of the following equations must hold: f[x+ y]=f[x ]·f[y] f[x+δx]=f[x]+f0[x]·δx+ε·δx We let y = δx in the first identity to compare it with the increment approximation, f[x+ δx]=f[x ]·f[δx] f[x+δx]=f[x]+f0[x]·δx+ε·δx so f[x]·f[δx]=f[x]+f0[x]·δx+ε·δx f[x][f[δx]−1] = f0[x]·δx+ε·δx f0[x]=f[x ]f[δx]−1 δx −ε or f0[x] f[x] = f[δx]−1 δx −ε with ε ≈ 0 whenδx ≈ 0. The identity still holds with hyperreal inputs by the Function Extension Axiom. Since the left side of the last equation depends only on x and the right hand side does not depend on x at all, we must have f[δx]−1 δx ≈k, a constant, orf[∆x]−1 ∆x →kas ∆ x → 0. In other words, a differentiable function that satisfies the (ExpSum) identity satisfies the differential equation df dx = kf What is the value of our unknown function at zero, f[0]? For any x and y = 0, we have f[x]=f[x+0]=f[x]·f[0] so unless f[x] = 0 for all x, we must havef[0] = 1. One of the main morals of this course is that if you know: (1) where a quantity starts,

26 2. Functional Identities

and (2) how a quantity changes, then you can compute subsequent values of the quantity. In this problem we have found (1) f[0] = 1 and (2) df dx = kf. We can use this information with the computer to calculate values of our unknown function f[x]. The unique symbolic solution to

f[0] = 1 df dx = kf

is

f[x]=ekx The identity (Repeated Exp) allows us to write this as

f[x]=ekx=(ek)x=bx

where b = ek. In other words, we have shown that the only differentiable functions that satisfy the (ExpSum) identity are the ones you know from high school, bx. Problem 2.1. Smooth Additive Functions ARE Linear H Suppose an unknown function is additive and differentiable, so it satisfies both

f[x+ δx]=f[x]+f[δx](Additive)

and f[x+ δx]=f[x]+f0[x]·δx+ε·δx(Micro) Solve these two equations for f0[x] and argue that since the right side of the equation does not depend on x, f0[x] must be constant. (Or f[∆x] ∆x → f0[x1] and f[∆x] ∆x → f0[x2], but sincethe left hand side is the same, f0[x1]=f0[x2].) What is the value of f[0] if f[x] satisfies the additive identity? The derivative of an unknown function f[x] is constant and f[0] = 0, what can we say about the function? (Hint: Sketch its graph.) N A project explores this symbolic kind of ‘linearity’ and the microscope equation from another angle.

2.5 The Motion of a Pendulum

Differential equations are the most common functional identities which arise in applications of mathematics to solving “real world” problems. One of the very important points in this regard is that you can often obtain significant information about a function if you know a differential equation the function satisfies, even if you do not know an exact formula for the function.

The Motion of a Pendulum 27

For example, suppose you know a function θ[t] satisfies the differential equation d2θ dt2 = Sin[θ[t]] This equation arises in the study of the motion of a pendulum and θ[t] does not have a closed form expression. (There is no formula for θ[t].) Suppose you know θ[0] = π 2. Then the differential equation forces d2θ dt2 [0] = Sin[θ[0]] = Sin[π 2 ]=1 We can also use the differential equation for θ to get information about the higher derivatives of θ[t]. Say we know that dθ dt[0] = 2. Differentiating both sides of the differential equation yields d3θ dt3 = Cos[θ[t ]]dθ dt by the Chain Rule. Using the above information, we conclude that d3θ dt3 [0] = Cos[θ[0]]dθ dt [0] = Cos[π 2 ]2 = 0

Problem 2.2. H Derive a formula for d4θ dt4 and prove that d4θ dt4 [0] = 1. N

28 2. Functional Identities

-4 -2 2 4

w

-4

-2

2

4

x

Part 2

Limits

30

CHAPTER 3 The Theory of Limits The intuitive notion of limit is that a quantity gets close to a “limiting” value as another quantity approaches a value. This chapter defines two important kinds of limits both with real numbers and with hyperreal numbers. The chapter also gives many computations of limits.

A basic fact about the sine function is

lim x→0

Sin[x] x

=1

Notice that the limiting expression Sin[x] x is defined for 0 <|x−0| < 1, but not if x = 0. The sine limit above is a difficult and interesting one. The important topic of this chapter is, “What does the limit expression mean?” Rather than the more “practical” question, “How do I compute a limit?” Here is a simpler limit where we can see what is being approached.

lim x→1

x2 −1 x−1

=2

While this limit expression is also only defined for 0 < |x−1|, orx6= 1, the mystery is easily resolved with a little algebra,

x2 −1 x−1

=

(x−1)(x+1) (x−1)

= x+1

So,

lim x→1

x2 −1 x−1

= lim x→1 (x +1)=2 The limit limx→1(x + 1) = 2 is so obvious as to not need a technical definition. If x is nearly 1, then x +1 is nearly 1+1 = 2. So, while this simple example illustrates that the original expression does get closer and closer to 2 as x gets closer and closer to 1, it skirts the issue of “how close?”

31

32 3. The Theory of Limits 3.1 Plain Limits

Technically, there are two equivalent ways to define the simple continuous variable limit as follows.

Definition 3.1. Limit Let f[x] be a real valued function defined for 0 < |x−a| < ∆ with ∆ a fixed positive real number. We say lim x→a f[x]=b when either of the equivalent the conditions of Theorem 3.2 hold.

Theorem 3.2. Limit of a Real Variable Let f[x] be a real valued function defined for 0 < |x−a| < ∆ with ∆ a fixed positive real number. Let b be a real number. Then the following are equivalent: (a) Whenever the hyperreal number x satisfies 0 < |x − a|≈0, the natural extension function satisfies f[x]≈ b (b) For every accuracy tolerance θ there is a sufficiently small positive real number γ such that if the real number x satisfies 0 < |x−a| <γ, then | f[x ]−b |<θ

Proof: We show that (a) ⇒ (b) by proving that not (b) implies not (a), the contrapositive. Assume (b) fails. Then there is a real θ>0 such that for every real γ>0 there is a real x satisfying 0 < |x−a| <γand |f[x]−b|≥θ. Let X[γ]=xbe a real function that chooses such an x for a particular γ. Then we have the equivalence

{γ>0}⇔{X[γ] is defined ,0 <|X[γ]−a| < γ,|f[X[γ]−b|≥θ}

By the Function Extension Axiom 2.1 this equivalence holds for hyperreal numbers and the natural extensions of the real functions X[·] andf[ · ]. In particular, choose a positive infinitesimal γ and apply the equivalence. We have 0 <|X[γ]−a| <γand |f[X[γ]−b|>θ and θ is a positive real number. Hence, f[X[γ]] is not infinitely close to b, proving not (a) and completing the proof that (a) implies (b). Conversely, suppose that (b) holds. Then for every positive real θ, there is a positive real γ such that 0 < |x−a| <γimplies |f[x]−b| <θ. By the Function Extension Axiom 2.1, this implication holds for hyperreal numbers. If ξ ≈a, then 0<|ξ−a |<γfor every real γ,so |f[ξ]−b|<θfor every real positive θ. In other words, f[ξ]≈ b, showing that (b) implies (a) and completing the proof of the theorem.

Example 3.1. Condition (a) Helps Prove a Limit

Plain Limits 33

Suppose we wish to prove completely rigorously that

lim ∆x→0

1 2(2+∆x)

=

1 4 The intuitive limit computation of just setting ∆x = 0 is one way to “see” the answer,

lim ∆x→0

1 2(2+∆x)

=

1 2(2+0

=

1 4 but this certainly does not demonstrate the “epsilon - delta” condition (b). Condition (a) is almost as easy to establish as the intuitive limit computation. We wish to show that when δx≈0 1 2(2+δx) ≈ 1 4 Subtract and do some algebra, 1 2(2+δx) − 1 4 = 2 4(2+δx) − (2+ δx) 4(2+δx) = −δx 4(2+δx) = δx· −1 4(2+δx) We completetheproofusing thecomputationrules ofTheorem1.12. Thefraction−1/(4(2+ δx)) is finite because 4(2+ δx)≈8 is not infinitesimal. The infinitesimal δx times a finite number is infinitesimal. 1 2(2+δx) − 1 4 ≈0 1 2(2+δx) ≈ 1 4 This is a complete rigorous proof of the limit. Theorem 3.2 shows that the “epsilon - delta” condition (b) holds.

Exercise set 3.1 1. Prove rigorously that the limit lim∆x→0 1 3(3+∆x) = 1 9. Use your choice of condition (a) or condition (b) from Theorem 3.2. 2. Prove rigorously that the limit lim∆x→0 1 √4+∆x+√4 = 1 4. Use your choice of condition (a) or condition (b) from Theorem 3.2. 3. The limit limx→0 Sin[x] x =1means that sine of a small value is nearly equal to the value, and near in a strong sense. Suppose the natural extension of a function f[x] satisfies f[ξ] ≈ 0 whenever ξ ≈ 0. Does this mean that limx→0 f[x] x exists? (HINT: What is limx→0√x? What is √ξ/ξ?) 4. Assume that the derivative of sine is cosine and use the increment approximation f[x+ δx]−f[x]=f0[x ]·δx+ε·δx

34 3. The Theory of Limits

with ε ≈0 when δx≈0, to prove the limitlimx→0 Sin[x] x =1. (It means essentially the same thing as the derivative of sine at zero is 1. HINT: Take x =0and δx = x in the increment approximation.)

3.2 Function Limits

Many limits in calculus are limits of functions. For example, the derivative is a limit and the derivative of x3 is the limit function 3x2. This section defines the function limits used in differentiation theory.

Example 3.2. A Function Limit

The derivative of x3 is 3x2, a function. When we compute the derivative we use the limit

lim ∆x→0

(x +∆x)3−x3 ∆x Again, the limiting expression is undefined at ∆x = 0. Algebra makes the limit intuitively clear, (x+∆x)3−x3 ∆x = (x3+3x2∆x+3x∆x2+∆x3)−x3 ∆x =3x2+3x∆x+∆x2 The terms with ∆x tend to zero as ∆x tends to zero.

lim ∆x→0

(x +∆x)3−x3 ∆x

= lim ∆x→0 (3x2 +3x∆x+∆x2)=3x2 This is clear without a lot of elaborate estimation, but there is an important point that might be missed if you don’t remember that you are taking the limit of a function. The graph of the approximating function approaches the graph of the derivative function. This more powerful approximation (than that just a particular value of x) makes much of the theory of calculus clearer and more intuitive than a fixed x approach. Intuitively, it is no harder than the fixed x approach and infinitesimals give us a way to establish the “uniform” tolerances with computations almost like the intuitive approach. Definition 3.3. Locally Uniform Function Limit Let f[x] and F[x,∆x] be real valued functions defined when x is in a real interval (a,b) and 0 < ∆x<∆with ∆ a fixed positive real number. We say lim ∆x→0 F[x,∆x]=f[x ] uniformly on compact subintervals of (a,b), or “locally uniformly” when one of the equivalent the conditions of Theorem 3.4 holds.

Function Limits 35

Theorem 3.4. Limit of a Real Function Let f[x] and F[x,∆x] be real valued functions defined when x is in a real interval (a,b) and 0 < ∆x<∆with ∆ a fixed positive real number. Then the following are equivalent: (a) Whenever the hyperreal numbers δx and x satisfy 0 < |δx|≈0 ,xis finite, and a<x<bwith neither x ≈ a nor x≈ b, the natural extension functions satisfy F[x,δx]≈ f[x] (b) For every accuracy tolerance θ and every real α and β in (a,b), there is a sufficiently small positive real number γ such that if the real number ∆x satisfies 0 <|∆x| <γand the real number x satisfies α ≤x ≤β, then | F[x,∆x]−f[x]| <θ

Proof: First, we prove not (b) implies not (a). If (b) fails, there are real α and β, a<α<β<b, and real positiveθsuch that for every real positive γ there are x and ∆x satisfying 0 < ∆x < γ, α≤x≤β, |F[x,∆x]−f[x]|≥θ Define real functions X[γ] andDX[γ] that select such values of x and ∆x, 0 < DX[γ]< γ, α≤X[γ]≤β, |F[X[γ],DX[γ]]−f[X[γ]]|≥θ Now apply the Function Extension Axiom 2.1 to the equivalent sets of inequalities, {γ>0}⇔{0< DX[γ]< γ, α≤X[γ]≤β, |F[X[γ],DX[γ]]−f[X[γ]]|≥θ} Choose an infinitesimal γ ≈0 and let x = X[γ] andδx = DX[γ]. Then 0 < δx < γ≈0,α≤ x ≤ β, |F[x,δx]−f[x]|≥θ so F[x,δx]−f[x] is not infinitesimal showing not (a) holds and proving (a) implies (b). Now we prove that (b) implies (a). Let δx be a non zero infinitesimal and let x satisfy the conditions of (a). We show that F[x,δx]≈ f[x] by showing that for any positive real θ, |F[x,δx]−f[x]| <θ. Fix any one such value of θ. Since x is finite and not infinitely near a nor b, there are real values α and β satisfying a<α<β<b. Apply condition (b) to these α and β together with θ fixed above. Then there is a positive real γ so that for every real ξ and ∆x satisfying 0 < |∆x| <γand α≤ ξ ≤ β, we have|F[ξ,∆x]−f[ξ]|<θ. In other words, the following implication holds in the real numbers, {0 < |∆x| < γ,α≤ξ≤β}⇒{|F[ξ,∆x]−f[ξ]|<θ} Apply the Function Extension Axiom 2.1 to see that the same implication holds in the hyperreals. Moreover, x = ξ and nonzero ∆x = δx ≈ 0 satisfy the left hand side of the implication, so the right side holds. Since θ was arbitrary, condition (a) is proved. Example 3.3. Computing Locally Uniform Limits

36 3. The Theory of Limits

The following limit is uniform on compact subintervals of (−∞,∞). lim ∆x→0 (x +∆x)3−x3 ∆x = lim ∆x→0 (3x2 +3x∆x+∆x2)=3x2 A complete rigorous proof based on condition (a) can be obtained with the computation rules of Theorem 1.12. The difference is infinitesimal (3x2 +3xδx+δx2)−3x2 = (3x+δx)δx when δxis infinitesimal. First, 3x+δxis finite becausea sum offinite numbers is finite. Second, infinitesimal times finite is infinitesimal. This is a complete proof and by Theorem 3.4 shows that both conditions (b) and (c) also hold.

Exercise set 3.2 1. Prove rigorously that the locally uniform function limit lim∆x→0 1 x(x+∆x) = 1 x2. Use your choice of condition (a) or condition (b) from Theorem 3.4. 2. Prove rigorously that the locally uniform function limit lim∆x→0 1 √x+∆x+√x = 1 2√x. Use your choice of condition (a), condition (b), or condition (c) from Theorem 3.4. 3. Prove the following: Theorem 3.5. Locally Uniform Derivatives Let f[x] and f0[x] be real valued functions defined when x is in a real interval (a,b). Then the following are equivalent: (a) Whenever the hyperreal numbers δx and x satisfy δx ≈ 0, x is finite, and a<x<bwith neither x ≈ a nor x ≈ b, the natural extension functions satisfy f[x+ δx]−f[x]=f0[x ]·δx+ε·δx for ε ≈0.(b) For every accuracy tolerance θ and every real α and β in (a,b), there is a sufficiently small positive real number γ such that if the real number ∆x satisfies 0 < |∆x| <γand the real number x satisfies α ≤ x ≤ β, then

 

 

 

 

 

 f[x+∆x]−f[x] ∆x −f0[x]

 

 

 

 
<θ(c) For every real c in (a,b), lim x→c,∆x→0 f[x+∆x]−f[x] ∆x =f0[c] That is, for every real c with a<c<band every real positive θ, there is a real positive γ such that if the real numbers x and ∆x satisfy 0 < ∆x<γand 0 < |x−c| <γ, then|f[x+∆x]−f[x] ∆x −f0[c]| <θ.

Computation of Limits 37 3.3 Computation of Limits

Limits can be computed in a way that rigorously establishes them as results by using the rules of Theorem 1.12.

Suppose we want to compute a limit like

lim d→0

(x + d)2 −x2 d First we observe that it does no good to substitute d = 0 into the expression, because we get 0/0. We do some algebra, (x+d)2 −x2 d = x2 +2xd+d2−x2 d = 2xd+d2 d =2x+d Now, lim d→0 (x + d)2 −x2 d = lim d→0 2x+ d =2x because making d smaller and smaller makes the expression closer and closer to 2x. The rules of small, medium and large numbers given in Theorem 1.12 just formalize the kinds of operations that work in such calculations. Theorem 3.2 and Theorem 3.4 show that these rules establish the limits as rigorously proven facts. Exercise 3.3.1 below contains a long list of drill questions that may be viewed as limit computations. For example, lim d→0 1 d1 b + d − 1 b=? is just asking what happens as d becomes small. Another way to ask the question is 1 δ1 b + δ − 1 b≈? whenδ ≈ 0 The latter approach is well suited to direct computations and can be solved with the rules of Theorem 1.12 that formalize our intuitive notions of small, medium and large numbers. Following are some sample calculations with parameters a,b,c,δ,ε,H,K from Exercise 3.3.1. Example 3.4. Infinitesimal, Finite and Infinite Computations We are told that a ≈ 2 andb≈5, so we may write a =2+ιand b =5+θwith ι ≈ 0and θ ≈ 0. Now we compute b−a =5+θ−2−ι=5−2+(θ−ι)=3+(θ−ι) by rules of algebra. The negative of an infinitesimal is infinitesimal and the sum of a positive and negative infinitesimal is infinitesimal, hence θ−ι ≈0. This makes b−a≈3

38 3. The Theory of Limits

Another correct way to do this computation is the following a ≈2 b ≈5 b−a ≈5−2=3 However, this is NOT use of ordinary rules of algebra, because ordinary rules of algebra do not refer to the infinitely close relation ≈. This form of the computation incorporates the fact that negatives of infinitesimals are infinitesimal and the fact that sums of infinitesimals are infinitesimal. Example 3.5. Small, Medium and Large as Limits The approximate computations can be re-phrased in terms of limits. We can replace the occurrences of δ ≈ 0 andε≈0 by variables approaching zero and so forth. Let’s just do this by change of alphabet, limd→0 for δ ≈0 and limα→2 for a≈2. The computation b−a ≈3 can be viewed as the limit lim α→2,β→5 β−α =3 The computation (2−δ)/a≈1 becomes lim d→0&α→2 2−d α =1 The computation √a+δ−√a δ ≈ 1 2√2 becomes

lim d→0&α→2

√α + d−√α d

=

1 2√2

Example 3.6. Hyperreal Roots When a ≈ 2 andc≈−7, the Function Extension Axiom guarantees in particular that√ a is defined and that √c is undefined, since a>0 is positive and c<0 is negative. The computation rules may be used to show that √a ≈√2, that is, we do not need any more rules to show this. First, √a is finite, because a<3 implies √a<√3<2 by the Function Extension Axiom. Next, √a−√2=(√a−√2)(√a +√2) √a +√2 = a−2 √a +√2 = ι· 1 √a +√2 an infinitesimal times a finite number, by approximation rule (4). Finally, approximation rule (3) shows √a−√2≈0 or√ a ≈ √ 2

Computation of Limits 39

Example 3.7. A Limit of Square Root The “epsilon - delta” proof (condition (b) of Theorem 3.2) of lim x→0p|x|=0 is somewhat difficult to prove. We establish the equivalent condition (a) as follows. Let 0 <ξ≈0 be a positive infinitesimal. Since 0≥ x implies √x is defined and positive, The Function Extension Axiom 2.1 guarantees that √ξ is defined and positive. Suppose√ξ is not infinitesimal. Then there is a positive real number 0 <awith a<√ξ. Squaring and using the Function Extension Axiom on the property 0 <b<cimplies 0 < √b<√c, we see that 0<a2<ξcontradicting the assumption that ξ ≈0 is infinitesimal. Example 3.8. Infinite Limits We know that c +76= 0 because we are given that c6=−7 andc≈−7 orc=−7+ιwithι ≈0, but ι6= 0. This means that c+7=ι6= 0 and so c+7≈0 This, together with what we know about reciprocals of infinitesimals tells us that 1 c+7 is infinite We do not know if it is positive or negative; we simply weren’t told whether c<7 orc>7, but only that c ≈7. In this example, the limit formulation has the result

lim γ→7

1 γ −7

does not exist or lim γ→7

 

 

 

 

 

 

1 γ −7

 

 

 

 
=+∞The precise meaning of these symbols is as follows. Definition 3.6. Infinite Limits Let f[x] be a real function defined for a neighborhood of a real number a, 0 < |x−a| < ∆. We say lim x→a f[x]=∞ provided that for every large positive real number B, there is a sufficiently small real tolerance, τ such that if 0 < |x−a| <τthen f[x] >B. The symbol ∞ means “gets bigger and bigger.” This is equivalent to the following hyperreal condition. Theorem 3.7. A Hyperreal Condition for Infinite Limits Let f[x] be a real function defined for a neighborhood of a real number a, 0 < |x−a| < ∆. The definition of limx→a f[x]=∞is equivalent to the following. For every hyperreal x infinitely close to a, but distinct from it, the natural extension satisfies, f[x] is a positive infinite hyperreal. Proof: Left as an exercise below. Example 3.9. ∞ is NOT Hyperreal

40 3. The Theory of Limits

The symbol ∞ cannot stand for a number because it does not obey the usual laws of algebra. Viewed as a numerical equation ∞·∞=∞says that we must have ∞ = 1 or ∞= 0, by ordinary rules of algebra. Since we want to retain the rules of algebra, ∞ in the sense of ‘very big’ can not be a hyperreal number. Example 3.10. An Infinite Limit with Roots The limit lim x→0p|x| |x| =+∞ Proof: Let 0 <ξ≈0. We know from the previous example that√ξ ≈0. We know from algebra and the Function Extension Axiom that √ξ ξ = 1 √ξ Using Theorem 1.12, we see that this expression is infinitely large. Example 3.11. Indeterminate Forms Even though arguments similar to the ones we have just done show that a+b+c≈0 we can not conclude that 1 a+b+c is defined. For example, we might have a =2−δ,b=5+3δand c =−7−2δ. Thena≈2, b ≈5 andc≈−7, but a + b + c = 0. (Notice that it is truethat a + b + c ≈ 0.) In this case 1 a+b+c is not defined. Other choices of the perturbations might make a + b + c 6= 0, so 1 a+b+c is defined (and positive or negative infinite) in some cases, but not in others. This means that the value of 1 a+ b+ c can not be determined knowing only that the sum is infinitesimal. In Webster’s unabridged dictionary, the term “indeterminate” has the following symbolic characters along with the verbal definition 0 0, ∞ ∞ , ∞·0, 1∞, 00, ∞0, ∞−∞ In the first place, Webster’s definition pre-dates the discovery of hyperreal numbers. The symbol ∞ does NOT represent a real or hyperreal number, because things like ∞·∞=∞ only denote ‘limit of big times limit of big is big.’ The limit forms above do not have a definite outcome. Each of the symbolic short-cuts above has a hyperreal number calculation with indeterminate outcomes in the sense that they may be infinitesimal, finite or infinite depending on the particular infinitesimal, finite or infinite numbers in the computation. In this sense, the older infinities are compatible with infinitely large hyperreal numbers. Example 3.12. The Indeterminate Form ∞−∞ Consider ∞−∞. The numbers H and L = H + δ are both infinite numbers, butH − L = −δ is infinitesimal. The numbers K and M = K + b are both infinite andK −M ≈−5. The numbers H and N = H2 are both infinite and H −N = (1−H)·His a negative infinite number.

Computation of Limits 41

We may view the symbolic expression “∞−∞is indeterminate” as a short-hand for the fact that the difference between two infinite hyperreal numbers can be positive or negative infinite, positive or negative and finite or even positive or negative infinitesimal. Of course, it can also be zero. Example 3.13. The Indeterminate Form 0·∞ The short-hand symbolic expression “0·∞is indeterminate” corresponds to the following kinds of choices. Suppose that H =1/δ. Thenδ·H= 1. An infinitesimal times an infinite number could be finite. Suppose K = H3, soKis infinite. Now δ · K = H is infinite. An infinitesimal times an infinite number could be infinite. Finally, suppose ε = δ5. Then ε·K=δ 5/δ2 = δ3 is infinitesimal. The following is just for practice at using the computation rules for infinitesimal, finite, and infinite numbers from Theorem 1.12 to compute limits rigorously. These computations prove that the “epsilon - delta” conditions hold.

Exercise set 3.3 1. Drill with Rules of Infinitesimal, Finite and Infinite Numbers In the following formulas, 0 <ε≈0 and 0 <δ≈0,H and K are infinite and positive. a ≈2,b≈ 5 ,c≈−7, but a6=2,b 6 =5,c 6 =− 7 Say whether each expression is infinitesimal, finite (and which real number it is near), infinite, or indeterminate (that is, could be in different categories depending on the particular values of the parameters above.) 1 y = ε×δ 2 y = ε−δ 3 y = ε/b 4 y = ε/δ 5 y = a+7ε b−4δ 6 y = b/ε 7 y = a+b−c 8 y = a+δ 9 y = c−ε 10 y = a−2 11 y = 1 a−2 12 y = 1 a−b 13 y = c a−b 14 y = 2−δ a 15 y = 5δ4−3δ2+2δ δ 16 y = 1 H 17 y = 2−δ a−K 18 y = 5δ4−3δ2+2δ 4δ+δ2 19 y = H2+3H H 20 y = H2+3H H2 21 y = 3δ2 δ+8δ2

22 y = H−K H 23 y = H−K HK 24 y = H−K H+K 25 y =√H 26 y =√δ 27 y = H+K H−K 28 y = √H H+a 29 y =√a + δ−√a 30 y = 1 b+δ − 1 b

42 3. The Theory of Limits

31 y = 3aδ2+δ3 δ 32 y =

√a+δ−√a δ 33 y = 1 δ1 b+δ − 1 b

2. Re-write the problems of the previous exercise as limits. 3. Prove Theorem 3.7.

CHAPTER 4 Continuous Functions A functionf[x ]is continuous if a small change in its input only produces a small change in its output. This chapter gives some fundamental consequences of this property.

Definition 4.1. Continuous Function Suppose a real function f[x] is defined in a neighborhood of a, |x−a| < ∆. We say f[x] is continuous at a if whenever x ≈a in the hyperreal numbers, the natural extension satisfies f[x]≈ f[a]. Notice that continuity assumes that f[a] is defined. The function

f[x]=

Sin[x] x is technically not continuous at x = 0, but since limx→0 f[x] = 1 we could extend the definition to include f[0] = 1 and then the function would be continuous. Theorem 4.2. Continuity as Limit Suppose a real function f[x] is defined in a neighborhood of a, |x−a| < ∆. Then f[x] is continuous at a if and only if limx→a f[x]=f[a]. Proof: Apply Theorem 3.2. We show in Section ?? that differentiable functions are continuous, so rules of calculus give us an easy way to verify that a function is continuous.

 

4.1 Uniform Continuity

A function is uniformly continuous if given an “epsilon,” the same “delta” works “uniformly” for all x.

The simplest intervals are the ones of finite length that include their endpoints, [a,b], for numbers a and b. These intervals are sometimes described as ‘closed and bounded,’ because they have the endpoints and have bounded length. A shorter name is ‘compact’ intervals.

43

44 4. Continuous Functions

Every hyperreal number satisfying a ≤ x≤ b is near a real number x≈ c with a ≤ c ≤b. First, the hyperreal x has a standard part since it is finite. Second, c must lie in the interval because real numbers r outside the interval are a noninfinitesimsl distance from the endpoints. We cannot have x ≈ r and r a noninfinitesimal distance from the interval. Thefact thateveryhyperrealpointof a setis neara standardpointof the setis equivalent to the “finite covering property” of general topologically compact spaces. The hyperreal condition is easy to apply directly. The following theorem illustrates this (although we do not need the theorem later in the course.) Theorem 4.3. Continuous on a Compact Interval Suppose that a real function f[x] is defined and continuous on the compact real interval [a,b]={x:a≤x≤b}. Then for every real positive θ there is a real positive γ such that if |x1 −x2| <γin [a,b], then|f[x 1]−f[x 2] |<θ. Proof: Since f[x] is continuous at every point of [a,b], if ξ ≈ c for a ≤ c ≤ b, thenf[ξ]≈f[c ]. Further, since the interval [a,b] includes its real endpoints, if a hyperreal number x satisfies a ≤x ≤b then its standard part from Theorem 1.11 c lies in the interval and x≈ c. Let x1 and x2 be any two points in [a,b] withx 1≈x 2. Both of these numbers have the same standard part (since the real standard parts have to be infinitely close and real, hence equal.) We have f[x1]≈f[c]≈ f[x2] so for any numbers x1 ≈ x2 in [a,b], f[x1]≈ f[x2]. Suppose the conclusion of the theorem is false. Then there is a real θ>0 such that for every γ>0 there exist x1 and x2 in [a,b] with|x 1−x 2|<γand |f[x1]−f[x2]|≥θ. Definereal functions X1[γ] andX2[γ] that select such values and give us the real logical statement {γ>0}⇒{a≤X1[γ]≤b,a ≤X2[γ]≤ b,|X1[γ]−X2[γ]|< γ,|f[X1[γ]]−f[X2[γ]]|≥θ} Now apply the Function Extension Axiom 2.1 to this implication and select a positive infinitesimal γ ≈ 0. Let x1 = X1[γ], x2 = X2[γ] and notice that they are in the interval, x1 ≈x2, but f[x1] is not infinitely close to f[x2]. This contradiction shows that the theorem is true. 4.2 The Extreme Value Theorem

Continuous functions attain their max and min on compact intervals.

Theorem 4.4. The Extreme Value Theorem If f[x] is a continuous real function on the real compact interval [a,b], thenf attains its maximum and minimum, that is, there are real numbers xm and xM such that a ≤ xm ≤ b, a ≤ xM ≤ b, and for allxwith a ≤ x ≤ b f[xm]≤ f[x]≤ f[xM] Intuitive Proof:

The Extreme Value Theorem 45

We will show how to locate the maximum, you can find the minimum. Partition the interval into steps of size ∆x, a<a+∆x<a+2∆x<···<b and define a real function M[∆x] = the x of the form x1 = a +k∆x so that f[M[∆x]=f[x1]=max[f[x]:x=a+h∆x, h =0,1,···,n] This function is the discrete maximum from among a finite number of possibilities, so that M[∆x] has two properties: (1) M[∆x] is one of the partition points and (2) all other partition points x = a+ h∆x satisfy f[x]≤f[M[∆x]]. Next, we partition the interval into infinitesimal steps, a<a+δx < a+2δx <···<b and consider the natural extension of the discrete maximizing function M[δx]. By the Function Extension Axiom 2.1 we know that (1) x1 = M[δx] is one of the points in the infinitesimal partition and (2) f[x]≤ f[x1] for all other partition points x. Since the hyperreal interval [a,b] only contains finite numbers, there is a real number xM ≈ x1 (standard part) and every other real number x2 in [a,b] is within δx of some partition point, x2 ≈x. Continuity of f means that f[x] ≈ f[x2] andf[xM]≈f[x 1]. The numbers x2 and xM are real, so f[x2] andf[xM] are also real and we have f[x2]≈ f[x]≤ f[x1]≈ f[xM] Thus, for any real x2, f[x2] ≤ f[xM], which says f attains its maximum at xM. This completes the proof. Partition Details of the Proof: Let a and b be real numbers and suppose a real function f[x] is defined for a ≤ x ≤ b. Let ∆x be a positive number smaller than b−a. There are finitely many numbers of the form a + k∆x between a and b; a = a +0∆x,a+∆x,a+2∆x,···,a+n∆x≤b. The corresponding function values, f[a], f[a +∆x], f[a+2∆x],···,f[a+n∆x] have a largest amongst them, say f[a + m∆x] ≥ f[a + k∆x] for all other k. We can express this with a function M[∆x]=a+m∆x“is the place amongst the points a<a+∆x<a+2∆x< ···<a+n∆x≤bsuch that f[M[∆x]] ≥ f[a + k∆x].” (There could be more than one, but M[∆x] chooses one of them.) A better way to formulate this logically is to say, ‘if x is of the form a + k∆x, then f[x ]≤f[M[∆x]].’ This can also be formulated with functions. Let I[x] be the ‘indicator function of integers,’ that is I[x] = 1 ifx=0,±1,±2,±3,···and I[x] = 0 otherwise. Then the maximizing property of M[∆x]=a+m∆xcan be summarized by a ≤ x≤ b and Ix−a ∆x =1 ⇒ f[x ]≤f[M[∆x]] The rigorous formulation of the Function Extension Axiom covers this case. We take S to be the set of formulas, ∆x>0, a ≤ x, x ≤ b and I[[x−a]/∆x] = 1 and takeTto be the inequality f[x]≤ f[M[∆x]]. The Function Extension Axiom shows that M[δx] is the place

46 4. Continuous Functions

where f[x] is largest among points of the form a + kδx, even whenδx≈0 is infinitesimal, but it says this as follows: a ≤ x≤ b and Ix−a δx =1 ⇒ f[x ]≤f[M[δx]] We interpret this as meaning that, ‘among the hyperreal numbers of the form a+kδx,f[x] is largest when x = M[δx],’ even when δx is a positive infinitesimal.

4.3 Bolzano’s Intermediate Value Theorem

The graphs of continuous functions have no “jumps.”

Theorem 4.5. Bolzano’s Intermediate Value Theorem If y = f[x] is continuous on the interval a ≤ x ≤ b, thenf [ x ]attains every value intermediate between the values f[a] and f[b]. In particular, iff[a ]<0and f[b] > 0, then there is an x0, a<x0<b, such thatf[x 0]=0.

Proof: The following idea makes a technically simple general proof. Suppose we want to hit a real value γ between the values of f[a]=αand f[b]=β. Divide the interval [a,b] up into small steps each ∆x long, a, a +∆x,a+2∆x,a+3∆x,···,b. Suppose α<γ<β. The function f[x] starts at x = a with f[a]=α<γ. At the next step, it may still be below γ, f[a+∆x]<γ, but there is a first step, a+k∆x where f[a+k∆x] >γand f[x] <γfor all x of the form x = a+h∆x with h<k.

[ a b ]

Figure 4.1: [a,b] in steps of size ∆x We need a general function for this. Let the function M[∆x] = Min[x : f[x] > γ, a < x < b, x=a+∆x,a +2∆x,a +3∆x,···] =a+k∆x give this minimal x as a function of the step size ∆x. The natural extension of this Min function has the property that even when we compute at an infinitesimal step size, ξ = M[δx] satisfies, f[ξ] >γ, andf[x ]<γfor x = a+hδx < ξ, in particular f[ξ−δx] <γ. Infinitesimals let continuity enter the picture. Continuity of f[x] means that if c ≈ x, thenf[c ]≈f[x]. We take c to be the standardreal number such that c ≈ ξ = M[δx]. We know f[c] ≈ f[ξ] >γand f[c] ≈ f[ξ−δx] <γ.Since f[c] must be a real value for a real function at a real input, and since we have just shown that f[c] ≈ γ, it follows that f[c]=γ, because ordinary reals can only be infinitely close if they are equal.

-4 -2 2 4

w

-4

-2

2

4

x

Part 3

1 Variable Differentiation

48

CHAPTER 5 The Theory of Derivatives This chapter shows how the traditional “epsilon-delta” theory, rigorous infinitesimal analysis, and the intuitive approximations of the main text are related.

The chapter shows that

lim ∆x→0

f[x +∆x]−f[x] ∆x

=f0[x] locally uniformly

⇔

f[x + δx]=f[x]+f0[x]δx+ε·δx with ε≈0 forδx≈0 with all the provisos needed to make both of these exactly formally correct. Then this approximation is used to prove some of the basic facts about derivatives.

5.1 The Fundamental Theorem: Part 1

We begin with an overview that illustrates the two main approximations of calculus, how they fit together, and how the fine details are added if you wish to make formal arguments based on an intuitive approximation.

We re-write the traditional limit for a derivative as an approximation for the differential. Then we plug this differential approximation into an approximation of an integral to see why the Fundamental Theorem of Integral Calculus is true. The two main approximations interact to let us compute integrals by finding antiderivatives. For now, we simply treat the symbol “wiggly equals,” ≈, as an intuitive “approximately equals.” Subsections below justify the use both in terms of hyperreal infinitesimals and in terms of uniform limits. The point of the section is that the intuitive arguments are correct because we can fill in all the details if we wish. The Intuitive Derivative Approximation

49

50 5. The Theory of Derivatives

The traditional approach to derivatives is the approximation of secant lines approaching the slope of the tangent. Symbolically, this is

lim ∆x→0

f[x +∆x]−f[x] ∆x

=f0[x] The intuitive meaning of this formula is that (f[x+∆x]−f[x])/∆x is approximately equal to f0[x] when the difference, ∆x, is small, ∆x = δx ≈ 0. We write this with an explicit error, f[x+δx]−f[x] δx = f0[x]+ε where the error given by the Greek letter epsilon, ε, is small, provided that δx is small. We use lower case or small delta to indicate that the approximationis valid for a small difference in the value of x.[ δis lower case ∆. Both stand for “difference” because the difference in x-input is δx =(x+δx)−x.] This approximation can be rewritten using algebra and expressed in the form f[x+ δx]−f[x]=f0[x ]δx+ε·δx with ε ≈0 forδx≈0 where now the wiggly equals ≈ only means “approximately equals” in an intuitive sense. This expresses the change in a function f[x + δx]−f[x] in moving from x to x + δx as approximately given by a change f0[x]δx, linear in δx, with an error ε ·δx that is small compared to δx,(ε·δx)/δx = ε≈0. This is a powerful intuitive formulation of the approximation of derivatives. (It is often called ‘Landau’s small oh formula.’) This also has a direct geometric counterpart in terms of microscopes given in the main text, but here we use it symbolically. An Antiderivative Suppose that we begin with a function f[x] and know an antiderivative F[x], that is,

dF dx

[x]=f[x] fora ≤ x ≤ b The approximation above becomes F[x + δx]−F[x]=f[x ]δx+ε·δx with ε ≈0 forδx≈0 Flip this around to tell us f[x]δx =(F[x+δx]−F[x])−ε·δx with ε ≈0 forδx≈0 provided that a ≤ x ≤b. Integrals are Sums of Slices The mainidea of integralcalculus is that integralsare‘sums of slices.’ One wayto express this is Zb a f[x]dx = lim ∆x→0 (f[a]∆x + f[a+∆x]∆x + f[a+2∆x]∆x + f[a+3∆x]∆x +···+f[b−2∆x]∆x + f[b−∆x]∆x) where the sum is over values of f[x]∆x where x starts at a and goes in steps of size ∆x until we get to the slice ending at b.

The Fundamental Theorem: Part 1 51

The limiting quantity is approximately equal to the integral when the step size is “small enough.” Zb a f[x]dx ≈ f[a]δx+f[a+δx]δx+···+f[b−δx]δx for δx≈0 where ≈ temporarily only means the intuitive “approximately equals.” Now we incorporate the differential approximation above at each of the x points, x = a, x = a+δx, ···,x=b−δx, in our sum approximation, obtaining Z b a f[x]dx ≈ ([F(a + δx)−F[a])+(F[a+2δx]−F[a+δx])+···+(F[b]−F[b−δx]) −(ε[a,δx]δx+ε[a+δx,δx])δx+···+ε[b,δx]δx) The first sum ‘telescopes,’ that is, positive leading terms in one summand cancel negative second terms in the next, all except for the first and last terms, (F[a + δx]−F[a])+(F[a +2δx]−F[a+δx])+···+(F[b]−F[b−δx]) =−F[a]+F[b] The second sum and can be estimated as follows, |ε[a,δx]δx+ε[a+δx,δx]δx+···+ε[b−δx,δx]δx| ≤|ε[a,δx]|δx+|ε[a+δx,δx]|δx+···+|ε[b−δx,δx]|δx ≤|εMax|(δx+δx+···+δx) ≤|εMax|(b−a) where |εMax|is the largest of the small errors, ε[x,δx], coming from the differential approximation. The sum of δx enough times to move from a to b is b−a, the distance moved. As long as we make the largest error small enough, the summed error less than|εMax|(b−a ) will also be small, so Z b a f[x]dx ≈ F[b]−F[a] But these are both fixed and do not depend on how small we take δx, hence Zb a f[x]dx = F[b]−F[a] This intuitive estimation illustrates the Fundamental Theorem of Integral Calculus. Theorem 5.1. Fundamental Theorem of Integral Calculus: Part 1 Suppose the real function f[x] has an antiderivative, that is, a real function F[x] so that the derivative of F[x] satisfies dF dx [x]=f[x ] for all x with a ≤ x ≤ b Then Z b a f[x]dx = F[b]−F[a]

52 5. The Theory of Derivatives

The above is not a formal proof, because we have not kept track of the “approximately equal” errors. This can be completed either from the “ε−δ” theory of limits or by using hyperreal infinitesimals. Both justifications follow in separate subsections.

5.1.1 Rigorous Infinitesimal Justification First, we take as our definition of derivative, dF dx = f[x], condition (a) of Theorem 3.5: for every hyperreal x with a ≤ x ≤ b and every δx infinitesimal, there is an infinitesimal ε so that the extended functions satisfy F[x + δx]−F[x]=f[x ]δx+ε·δx The only thing we need to know from the theory of infinitesimals is that

εMax

exists in the sense that |ε[a,δx]δx+ε[a+δx,δx]δx+···+ε[b−δx,δx]δx| ≤|ε[a,δx]|δx+|ε[a+δx,δx]|δx+···+|ε[b−δx,δx]|δx ≤|εMax|(δx+δx+···+δx) ≤|εMax|(b−a) still holds when δx is infinitesimal. This follows from the Function Extension Axiom. Let ε[x,∆x] be the real function of the real variables x and ∆x, ε[x,∆x]=F[x+∆x]−F[x] ∆x −f[x] For each ordinary real ∆x, there is an x of the form xm = a + m∆x (m =1 ,2 ,3 ,···)so that the inequalities above hold with εMax = ε[xm,∆x]. This is just a finite maximum being attained at one of the values. Define a real function m[∆x]=xm. Define a real function S[∆x]=|ε[a,∆x]∆x+ε[a+∆x,∆x]∆x+···+ε[ε[b−∆x,∆x]∆x| The inequalities above say that for real ∆x S[∆x]≤|ε[m[∆x],∆x]|(b−a) The Function Extension Axiom says S[δx]≤|ε[m[δx],δx]|(b−a) and the definition of derivativesays that ε[m[δx],δx] is infinitesimal, provided δx is infinitesimal. Since an infinitesimal times the finite number (b−a) is also infinitesimal, we have shown that the difference between the real integral and the real answer S[δx]=Zb a f[x ]dx−(F[b]−F[a]) is infinitesimal. This means that they must be equal, since ordinary numbers can not differ by an infinitesimal unless they are equal.

Derivatives, Epsilons and Deltas 53

5.1.2 Rigorous Limit Justification We need our total error to be small. This total error, ErrorIntegral, is the difference between the quantity F[b]−F[a] and the integral, so by the calculation above, ErrorIntegral = ε[a,∆x]∆x+ε[a+∆x,∆x]∆x+···+ε[b−∆x,∆x]∆x We know from the calculation above that |ErrorIntegral|≤| ε Max|(b−a). If we choosean arbitrary error tolerance of θ, then it is sufficient to have |εMax|≤θ/(b−a), becausethen we will have |ErrorIntegral|≤θ. This means that we must show that the differentialapproximation f[x]∆x =(F[x+∆x]−F[x])−ε·∆x holds with |ε|≤θ/(b−a) for every x in [a,b]. Using the algebra above in reverse, this is the same as showing that F[x +∆x]−F[x] ∆x −f[x]=ε[x,∆x] is never more than θ/(b−a), provided that ∆x is small enough. The traditional way to say this is lim ∆x→0 F[x +∆x]−F[x] ∆x =f[x] uniformly for a ≤ x≤ b The rigorous definition of the limit in question is: for every tolerance η and every x in [a,b], there exists a µ such that if |∆x| <µ, then | F[x+∆x]−F[x] ∆x −f[x]|<η This is the formal definition of derivative condition (b) of Theorem 3.5. Our proof of the Fundamental Theorem is complete (letting η = θ/(b−a)). The hypothesis says that if we can find a function f[x] so that

lim ∆x→0

F[x +∆x]−F[x] ∆x

=f[x] uniformly for a ≤ x≤ b

then the conclusion is

Z b a

f[x]dx = F[b]−F[a] (Notice that the existence of the limit defining the integral is part of our proof. The function f[x] is continuous, because of our strong definition of derivative.)

5.2 Derivatives, Epsilons and Deltas

The fundamental approximation defining the derivative of a real valued function can be formulated with or without infinitesimals as follows.

54 5. The Theory of Derivatives

Definition 5.2. The Rigorous Derivative In Theorem 3.5 we saw that the following are equivalent definitions of, “The real function f[x] is smooth with derivative f0[x] on the interval (a,b).” (a) Whenever a hyperreal x satisfies a<x<band x is not infinitely near a or b, then an infinitesimal increment of the naturally extended dependent variable is approximately linear, that is, whenever δx≈0 f[x+δx]−f[x]=f0[x ]δx+ε·δx for some ε ≈0.(b) For every compact subinterval [α,β]⊂(a,b), lim ∆x→0 f[x +∆x]−f[x] ∆x =f0[x] uniformly for α ≤ x ≤ β in other words, for every accuracy tolerance θ and every real α and β in (a,b), there is a sufficiently small positive real number γ such that if the real number ∆x satisfies 0 < |∆x|<γand the real number x satisfies α≤ x≤ β, then

 

 

 

 

 

 f[x+∆x]−f[x] ∆x −f0[x]

 

 

 

 
<θ(c) For every real c in (a,b), lim x→c,∆x→0 f[x+∆x]−f[x] ∆x =f0[c] That is, for every real c with a<c<band every real positive θ, there is a real positive γ such that if the real numbers x and ∆x satisfy 0 < ∆x<γ and 0 < |x−c| <γ, then|f[x+∆x]−f[x] ∆x −f0[c]| <θ. All derivatives computed by rules satisfythis strong approximationprovided the formulas are valid on the interval. This is proved in Theorem 5.5 below.

5.3 Smoothness ⇒ Continuity of Function and Derivative

This section shows that differentiability in the sense of Definition 5.2implies that the function and derivative are continuous.

One difficult thing about learning new material is putting new facts together. Bolzano’s Theorem and Darboux’s Theorem have hypotheses that certain functions are continuous. This means you must show that the function you are working with is continuous. How do you tell if a function is continuous? You can’t ‘look’ at a graph if you haven’t drawn one and are using calculus to do so. What does continuity mean? Intuitively, it just means that small changes in the independent variable produce only small changes in the dependent variable.

Smoothness ⇒ Continuity of Function and Derivative 55

In Theorem ?? we showed that the following are equivalent definitions Definition 5.3. Continuity of f[x] Suppose a real function f[x] is defined for at least a small neighborhood of a real number a, |x−a|< ∆, for some positive real ∆. (a) f[x] is continuous at a if whenever a hyperreal x satisfies x ≈a, the natural extension satisfies f[x]≈ f[a].(b) f[x] is continuous at a if limx→a f[x]=f[a]. Intuitively, this just means that f[x] is close tof[a] whenxis close to a, for every x ≈ a, f[x] is defined and f[x]≈ f[a] The rules of calculus (together with Theorem 5.5) make it easy to verify that functions given by formulas are continuous: Simply calculate the derivative. Theorem 5.4. Continuity of f[x] and f0[x] Suppose the real function f[x] is smooth on the real interval (a,b) (see Definition 5.2). Then both f[x] and f0[x] are continuous at every real point c in (a,b).

Proof for f[x]: Proof of continuity of f is easy algebraically but is obvious geometrically: A graph that is indistinguishable from linear in a microscope clearly only moves a small amount in a small x-step. Draw the picture on a small scale. Algebraically, we want to show that if x1 ≈ x2 then f[x1] ≈ f[x2], condition (a) above.Let c be any real point in (a,b) andx=x 1=c. Let δx = x2 −x1 be any infinitesimal anduse the approximation f[x2]=f[x+δx]=f[x1]+f0[x1]δx+εδx. The quantity [f 0[x 1]+ε]δx is medium times small = small, so f[x1]≈f[x2], by Theorem 1.12 (c). That is the algebraic proof. Proof for f0[x]: Proof of continuity of f0[x] requires us to view the increment from both ends. First take any real c in (a,b), x = x1 = c, andδx = x2 −x1 any nonzero infinitesimal. Use the approximation f[x2]=f[x+∆x]=f[x1]+f0[x1]δx+ε1δx. Next let x = x2, δx = x1 −x2 and use the approximation f[x1]=f[x+∆x]=f[x2]+f0[x2]δx+ε2δx. The different x-increments are negatives, so we have f[x1]−f[x2]=f0[x2](x1 −x2)+ε2(x1−x2) and f[x2]−f[x1]=f0[x1](x2 −x1)+ε1(x2−x1) Adding, we obtain 0 = ((f0[x2]−f0[x1])+(ε2 −ε1))(x1 −x2) Dividing by the non-zero (x1 −x2), we see that f0[x2]=f0[x1]+(ε1−ε2), so f0[x2]≈ f0[x1] Note:

56 5. The Theory of Derivatives

The derivative defined in many calculus books is a weaker pointwise notion than the notion of smoothness we have defined. The weak derivativefunction need not be continuous. (The same approximation does not apply at both ends with the weak definition.) This is explained in Chapter 6 on Pointwise Approximations.

Exercise set 5.3 1. (a) Consider the real function f[x]=1/x, which is undefined at x =0. We could extend the definition by simply assigning f[0] = 0. Show that this function is not continuous at x =0but is continuous at every other real x. (b) Give an intuitive graphical description of the definition of continuity in terms of powerful microscopes and explain why it follows that smooth functions must be continuous. (c) The function f[x]=√xis defined for x ≥ 0; there is nothing wrong with f[0]. However, our increment computation for √x above was not valid at x =0because a microscopic view of the graph focused at x =0looks like a vertical ray (or halfline). Explain why this is so, but show that f[x] is still continuous “from the right;” that is, if 0 <x≈0, then√x≈0but √x x is very large.

5.4 Rules ⇒ Smoothness This section shows that when we can compute a derivative by rules, then the smoothness Definition 5.2 is automatically satisfied on intervals where both formulas are defined.

Theorem 5.5. Rules ⇒ Smooth Suppose a function y = f[x] is given by a formula to which we can apply the rules of Chapter 6 of the main text, obtaining a formula for f0[x]. If bothf [ x ]and f0[x] are defined on the real interval (a,b), then then satisfy Definition 5.2 and, by Theorem 5.4, are continuous on (a,b).

Proof: This is a special case of Theorem 10.1.

Exercise set 5.4 1. What is the simple way to tell if a function is continuous? 2. Suppose that y = f[x] is given by a formula that you can differentiate by the rules of calculus from Chapter 6 of the main text. As you know, you can differentiate many

The Increment and Increasing 57

many formulas. What property does the function dy dx = f0[x] have to have so that you can conclude f[x] is continuous at all points of the interval a ≤ x ≤ b? (What about f[x]=1/x at x =0?) Give examples of: (a) One y = f[x] which you can differentiate by rules and an interval [a,b] where f0[x] is defined and f[x] is continuous on the whole interval. (b) Another y = f[x] which you can differentiate by rules, but where f0[x] fails to be defined on all of [a,b] and where f[x] is not continuous at some point a ≤ c ≤ b. (Hint: Read Theorem 5.4. What about y =1/x?) What properties does the function dy dx = f0[x] have to have so that you can conclude f0[x] is continuous at all points of the interval a ≤ x ≤ b? Give Examples of: (a) One y = f[x] which you can differentiate by rules and an interval [a,b] where f0[x] is defined and f0[x] is continuous on the whole interval. (b) Another y = f[x] which you can differentiate by rules, but where f0[x] fails to be defined on all of [a,b] and where f0[x] is not continuous at some point a ≤ c ≤ b. (Note: If f0[x] is undefined at x = c, it cannot be considered continuous at c. Well, there is a sticky point here. Perhaps f0[x] could be extended at an undefined point so that it would become continuous with the extension. It is actually fairly easy to rule this out with one of the functions you have worked with in previous homework problems.)

5.5 The Increment and Increasing

A positive derivative means a function is increasing near the point. We prove this algebraically in this section.

It is ‘clear’ that if we view a graph in an infinitesimal microscope at a point x0 and see the graph as indistinguishable from an upward sloping line, then the function must be ‘increasing’ near x0. Certainly, the graph need not be increasing everywhere - draw y = x2 and consider the point x0 = 1 withf0[1] = 2. Exactly how should we formulate this? Even if you don’t care about the symbolic proof of the algebraic formulation, the formulation itself may be useful in cases where you don’t have graphs. One way to say f[x] increases near x0 would be to say that if x1 <x 0<x 2(and these points are not ‘too far’ from x0), then f[x1] <f[x 0]<f[x 2]. Another way to formulate the problem is to say that if x1 <x 2(and these points are not ‘too far’ from x0), then f[x1] <f[x2]. Surprisingly, the second formulation is more difficult to prove (and even fails for pointwise differentiable functions). The second formulation essentially requires that we can move the microscope from x0 to x1 and continue to see an upward sloping line. We know from Theorem 5.4 that if x1 ≈ x0 the slope of the microscopic line only changes an small amount, so we actually see the same straight line.

58 5. The Theory of Derivatives

Theorem 5.6. Local Monotony Suppose the function f[x] is differentiable on the real interval a<x<band x0 is a real point in this interval. (a) If f0[x0] > 0, then there is a real interval [α,β], a<α<x 0<β<b, such that f[x] is increasing on [α,β], that is, α ≤ x1 <x2≤β ⇒ f[x1]<f[x2] (b) If f0[x0] < 0, then there is a real interval [α,β], a<α<x 0<β<b, such that f[x] is decreasing on [α,β], that is, α ≤ x1 <x2≤β ⇒ f[x1]>f[x2]

Proof We will only prove case (a), since the second case is proved similarly. First we verify that f[x] is increasing on a microscopic scale. The idea is simple: Compute the change in f[x] using the positive slope straight line and keep track of the error. Take x1 and x2 so that x0 ≈x1 <x2≈x0. Since f0[x1]≈ f0[x0] by Theorem 5.4 we maywrite f0[x1]=m+ε1where m = f0[x0] andε 1≈0. Let δx = x2 −x1 so f[x2]=f[x1+δx]=f[x1]+f0[x1]·δx+ε2·δx = f[x1]+m·δx+(ε1+ε2)·δx The number m is a real positive number, so m+ ε1 + ε2 > 0 and , since δx > 0, (m+ε1 + ε2)·δx > 0. This means f[x2]−f[x1] > 0 andf[x 2]>f[x 1]. This proves that for any infinitesimal interval [α,β] withα<x0<β, the function satisfies α ≤ x1 <x2≤β ⇒ f[x1]<f[x2] The Function Extension Axiom guarantees that real numbers α and β exist satisfying the inequalities above, since if the equation fails for all real α and β, it would fail for infinitely close ones. That completes the proof. Example 5.1. A Non-Increasing Function with Pointwise Derivative 1 The function f[x]=(0, if x =0 x+x2Sin[π x], if x6=0 has a pointwise derivative at every point and Dxf[0] = 1 (but is not differentiable in the usual sense of Definition 5.2). This function is not increasing in any neighborhood of zero (so it shows why the pointwise derivative is not strong enough to capture the intuitive idea of the microscope). See Example 6.3.1 for more details.

5.6 Inverse Functions and Derivatives

If a function has a nonzero derivative at a point, then it has a local inverse. The project on Inverse Functions expands this section with more details.

Inverse Functions and Derivatives 59

The inverse of a function y = f[x] is the function x = g[y] whose ‘rule un-does what the rule for f does.’ g[f[x]] = x If we have a formula for f[x], we can try to solve the equation y = f[x] forx. If we are successful, this gives us a formula for g[y], y = f[x] ⇔ x = g[y] Example 5.2. y = x2 and the Partial Inverse x =√y

For example, if y = f[x]=x 2, thenx=g [ y]=√y, at least when x ≥ 0. These twofunctions have the same graph if we plot g with its independent variable where the y axis normally goes, rather than plotting the input variable of g[y] on the horizontal scale.

x

  y=f[x],  x>0

x=g[y]

y

Figure 5.1: y = f[x] and its inverse x = g[y] The graph of x = g[y] operationally gives the function g by choosing a y value on the y axis, moving horizontally to the graph, and then moving vertically to the x output on the x axis. This makes it clear graphically that the rule for g ‘un-does’ what the rule for f does. If we first compute f[x] and then substitute that answer into g[y], we end up with the original x. Example 5.3. y = f[x]=x9+x7+x5+x3+xits Inverse

The graph of the function y = f[x]=x9+x7+x5+x3+xis always increasing because f0[x]=9x8+7x6+5x4+3x2+1 > 0ispositiveforallx. Sinceweknowlimx→−∞f[x]=−∞ and limx→+∞f[x]=+∞, Bolzano’s Intermediate Value Theorem 4.5 says that f[x] attains every real value y. By Theorem 5.6, f[x] can attain each value only once. This means that for every real y, there is an x = g[y] so thatf[x]=y. In other words, we see abstractly that f[x] has an inverse without actually solving the equation y = x9 +x7 +x5 +x3 +x for x as a function of y. Example 5.4. ArcTan[y]

The function y = Tan[x] has derivative dy dx = 1 (Cos[x])2. When−π/2 < x < π/2, cosine is not zero and therefore the tangent is increasing for −π/2 < x < π/2. How do we solve for

60 5. The Theory of Derivatives

x in the equation y = Tan[x]?

y = Tan[x ] ArcTan[y] = ArcTan[Tan[x]] = x x = ArcTan[y] But what is the arctangent? By definition, the inverse of tangent on (−π/2,π/2). So how would we compute it? The Inverse Function project answers this question. Example 5.5. A Non-Elementary Inverse Some functions are do not have classical expressions for their inverses. Let y = f[x]=xx This may be written using x = eLog[x], soxx=(eLog[x])x = ex Log[x], andf[x] has derivative

dy dx

= d(ex Log[x]) dx = (Log[x]+x

1 x )exLog[x] = (1+Log[x])xx It is clear graphically that y = f[x] has an inverse on either the interval (0,1/e) or (1/e,∞). We find where the derivative is positive, negative and zero as follows. First, xx = ex Log[x] is always positive, never zero, so 0 = (1+Log[x])xx 0 = (1+Log[x]) −1 = Log[x] e−1 = eLog[x] 1 e = x If x<1/e, sayx=1/e2, thendy dx = (1 +Log[e−2])(+) = (1 − 2)(+) = (−) < 0. If x = e>1/e, dy dx = (1 + Log[e])(+) = (2)(+) = (+) > 0. So dy dx < 0 for 0<x<1/e and dy dx > 0 for 1/e < x < ∞. (Note our use of Darboux’s Theorem 7.2.) This means that f[x]=xx has an inverse for x>1/e. It turns out that the inverse function x = g[y] can not be expressed in terms of any of the classical functions. In other words, there is no formula for g[y]. (This is similar to the nonelementary integrals in the Mathematica NoteBook SymbolicIntegr. Computer algebra systems have a non-elementary function ω[x] which can be used to express the inverse.) The Inverse Function project has you compute the inverse approximately. Example 5.6. A Microscopic View of the Graph We view the graph x = g[y] for the inverse as the graph y = f[x] with the roles of the horizontal and vertical axes reversed. In other words, both functions have the same graph, but we view y as the input to the inverse function g[y]. A microscopic view of the graph can likewise be viewed as that of either y = f[x] orx=g[y ].

Inverse Functions and Derivatives 61

dy

dx

y

y0

x0 x1

dy

dx

Figure 5.2: Small View of x = g[y] andy=f[x] at (x 0,y0)

The ratio of a change in g-output dx to a change in g-input dy for the linear graph is the reciprocal of the change in f-output dy to the change in f-input dx for the function. In other words, if the inverse function really exists and is differentiable, we see from the microscopic view of the graph that we should have

dy dx

= f0[x0] anddx dy

=

1 dy/dx

= g0[y0]

The picture is right, of course, and the Inverse Function Theorem 5.7 justifies the needed steps algebraically (in case you don’t trust the picture.)

Example 5.7. The Symbolic Inverse Function Derivative

Assume for the moment that f[x] andg[y] are smooth inversefunctions. Apply the Chain Rule (in function notation) as follows.

x = g[f[x]] dx dx = g0[f[x]]·f0[x] 1=g 0[f[x ]]·f0[x] g0[f[x]] = 1 f0[x]

At a point (x0,y0) on the graph of both functions, we have

g0[y0]=

1 f0[x0]

62 5. The Theory of Derivatives

In differential notation, this reads like ordinary fractions,

x = g[y] ⇔ y = f[x] dx dy = g0[y] ⇔ dy dx = f0[x] dx dy =1/dy dx

A concrete example may help at this point. Example 5.8. Derivative of ArcTangent

x = ArcTan[y] ⇔ y= Tan[x ] dx dy = ArcTan 0[y] ⇔ dy dx = 1 (Cos[x])2 ArcTan0[y]=dx dy =1/dy dx = (Cos[x])2

Correct, but not in the form of a function of the same input variable y. We know that Tan2[x]=Sin2[x] Cos2[x] = y2 and Sin2[x]+Cos2[x] = 1, so we can express Cos2[x] in terms ofy,

Sin2[x]+Cos2[x]=1

1+

Sin2[x] Cos2[x]

=

1 Cos2[x]

1+y2=

1 Cos2[x] Cos2[x]= 1 1+y2

So we can write

ArcTan0[y] = (Cos[x])2 =

1 1+y2

Thepointofthis concreteexampleis thatwecancomputethederivativeofthe arctangent even though we don’t have a way (yet) to compute the arctangent. In general, the derivative of an inverse function at a point is the reciprocal of the derivative of the function. In this case a trig trick lets us find a general expression in terms of y as well. Example 5.9. Another Inverse Derivative

It is sometimes easier to compute the derivative of the inverse function and invert for the derivative of the function itself – even if it is possible to differentiate the inverse function. For example, if y = x2 +1andx=√y−1 wheny≥1, then dy dx =2x. The inverse function

Inverse Functions and Derivatives 63

rule says

dx dy

=

1/dy dx

=

1 2x

=

1 2√y−1

The point of the last two examples is that computing derivatives by reciprocals is sometimes helpful. The next result justifies the method. Theorem 5.7. The Inverse Function Theorem Suppose y = f[x] is a real function that is smooth on an interval containing a real point x0 where f0[x0]6=0. Then (a) There is a smooth real function g[y] defined when |y − y0| < ∆, for some real ∆ > 0. (b) There is a real ε>0such that if |x − x0| <ε, then|f[x ]−y 0|<∆andg [f[x]] = x. g[y] is a “local” inverse for f[x].

Proof: Suppose we have a function y = f[x] and know thatf0[x] exists on an interval around a point x = x0 and we know the values y0 = f[x0] andm=f 0[x 0]6= 0. In a microscope wewould see the graph

dy

dx

y

y0

x0 x1

dy

dx

Figure 5.3: Small View of y = f[x] at (x 0,y0)

The point (dx,dy) = (0 ,0) in local coordinates is at (x0,y0) in regular coordinates. Suppose we are given y near y0, y ≈ y0. In the microscope, this appears on the dy axis at the local coordinate dy = y − y0. The corresponding dx value is easily computed by

64 5. The Theory of Derivatives

inverting the linear approximation

dy = mdx mdx=dy dx = dy/m

The value x1 that corresponds to dx = dy/m is dx = x1 −x0. Solve forx 1,

x 1=x 0+dx = x0 + dy/m = x0 +(y−y0)/m

Does this value of x = x1 satisfy y = f[x1] for the value of y we started with? We wouldn’t think it would be exact because we computed linearly and we only know the approximation

f[x0 + dx]=f[x0]+f0[x0]dx + ε·dx f[x1]=f[x0]+m·(x1−x0)+ε·(x1−x0) We know that the error ε≈0 is small whendx ≈0 is small, so we would have to move the microscope to see the error. Moving along the tangent line until we are centered over x1, we might see

?x

?y

Figure 5.4: Small View at (x1,y)

The graph of y = f[x] appears to be the parallel line above the tangent, because we have only moved x a small amount and f0[x] is continuous by Theorem 5.4. We don’t know how to compute x = g[y] necessarily, but we do know how to compute y1 = f[x1]. Suppose we have computed this and focus our microscope at (x1,y1) seeing

Inverse Functions and Derivatives 65

dy

dx

y

y1

x1 x2

dy

dx

Figure 5.5: Small View at (x1,y1) We still have the original y ≈ y1 and thus y still appears on the new view at dy = y−y1. The corresponding dx value is easily computed by inverting the linear approximation dy = mdx mdx=dy dx = dy/m The x value, x2, that corresponds to dx = dy/m is dx = x2−x1 with x2 unknown. Solve for the unknown, x2 = x1 + dx = x1 + dy/m = x1 +(y−y1)/m This is the same computation we did above to go from x0 to x1, in fact, this gives a discrete dynamical system of successive approximations x0 = given xn+1 = xn +(y−f[xn])/m xn+1 = G[xn], with G[x]=x+(y−f[x])/m The sequence of approximations is given by the general function iteration explained in Chapter 20 of the main text,

x 1=G [ x 0],x 2=G [ x 1 ]=G [ G [ x0]],x 3=G [ G [ G [ x 0 ]]],··· and we want to know how far we move, lim n→∞ xn =? The iteration function G[x] is smooth, in fact, G0[x]=1−f0[x ] /m

66 5. The Theory of Derivatives

and in particular, G0[x0]=1−f0[x0]/m =1−m/m = 0. By Theorem 5.4, whenever x ≈x0, G0[x]≈0. Differentiability of G[x] means that if xi ≈ xj ≈ x0, G[xi]−G[xj]=G 0[x j]·(x i−x j)+ι·(x i−xj) |G[x i]−G[xj]|=|G 0[xj]+ι|·|x i−xj| |G[x i]−G[xj]|≤r·|x i−xj| for any real positive r, since G0[xj]≈G0[x0] = 0 and someι≈0. If y ≈ y0, thenx 1=x 0−(y−y 0)/m ≈ x0. Similarly, if xn ≈ x0, thenf[x n]≈yandx n+1 ≈x0, so|G [ xn+1]−G[xn]|≤r·|xn+1 −xn|. Hence, |x2 −x1|=|G[x1]−G[x0]|≤r|x1−x0| |x3−x2|=|G [x2]−G [x1]|≤r|x2−x1|≤r(r|x1−x0|)=r2|x1−x0| |x4−x3|=|G [x3]−G [x2]|≤r|x3−x2|≤r(r2|x1−x0|)=r3|x1−x0| and in general |xn+1 −xn|≤rn|x1−x0| The total distance moved in x is estimated as follows. |xn+1 −x0|=|(xn+1 −xn)+(xn−xn−1)+(xn−1−xn−2)+···+(x1−x0)| ≤|xn+1 −xn|+|xn −xn−1|+|xn−1 −xn−2|+···+|x1−x0| ≤rn|x1−x0|+rn−1|x1−x0|+...+|x1−x0| ≤(rn+rn−1+...+r+1)|x1−x0| ≤ 1−rn+1 1−r |x1 −x0| The sum 1+r +r2 +r3 +···+rn = 1−rn+1 1−r is a geometric series as studied in the main text Chapter 25. Since limn→∞rn = 0 if|r |<1, 1 + r + r2 + r3 +···+rn → 1 1−r for |r| < 1. Thus, for any y ≈ y0 and any real r with 0 <r<1, |xn −x0|≤ 1 1−r|x1−x0| for all n =1,2,3,.... Similar reasoning shows that when y ≈ y0 |xk+n −xk|≤rk 1 1−r|x1−x0| because |xk+1 −xk|≤rk|x1−x0| |xk+2 −xk+1|=|G[xk+1]−G[xk]|≤r|xk+1 −xk|≤r(rk|x1−x0|) |xk+3 −xk+2|=|G[xk+2]−G[xk+1]|≤r|xk+2 −xk+1|≤r2(rk|x1−x0|) . . .

Inverse Functions and Derivatives 67 Take the particular case r = 2. We have shown in particular that whenever 0 <δ≈0and |y−y0|<δ, then for all k and n, |xn −x0|≤2|x1−x0| and |xk+n −xk|≤ 2 2k|x1−x0| and f[x] is defined and |f0[x]− f0[x0]| < |m|/2 for|x−x 0|<3δ/|m|. By the FunctionExtension Axiom 2.1, there must be a positive real ∆ such that if |y−y0|< ∆, then for allk and n, |xn −x0|≤2|x1−x0| and |xk+n −xk|≤ 2 2k|x1−x0| and f[x] is defined and |f0[x]−f0[x0]| < |m|/2 for|x−x 0|<3∆/|m|. Fix this positive real∆. Also, if |x − x0| <E≈0, then |f[x] − y0| < ∆ with ∆ as above. By the Function Extension Axiom 2.1, there is a real positive ε so that if |x − x0| <ε<3∆/|m|, then | f[x ]−y 0|<∆ and|f0[x ]−f0[x 0] |<|m | /2. Now, take any real y with |y−y0| < ∆ and consider the sequence x1 = G[x0],x 2=G [ x 1 ]=G [ G [ x0]],x 3=G [ G [ G [ x 0 ]]],··· This convergesbecause once we havegotten to the approximation xk, we never movebeyond that approximation by more than

|xk+n −xk|≤

1111111111111111111111111111111111111111111111111111112 2k|x1−x0| In other words, if we want an approximation good to one one millionth, we need only take k large enough so that 2 2k|x1 −x0|≤10−6 (1−k)Log[2]+Log[|x1 −x0|]≤−6log[10] k ≥ Log[2]+Log[|x1 −x0|]+6Log[10] Log[2] This shows by an explicit error formula that the sequence xn converges. We define a real function g[y] = limn→∞ xn whenever |y −y0| < ∆. We can approximate g[y]=x∞using the recursive sequence. (This is a variant of “Newton’s