0x003 Sequence
- 1. Sequence
- 2. Function Sequence
- 3. Reference
1. Sequence
1.1. Sequence
This subsection is about sequence in \(\mathbb{R}\)
1.1.1. Convergence
Definition (Convergence) A sequence \((a_n)^{\infty}_{n=N}\) is convergent to \(L\) iff the sequence is \(L\) is eventually \(\epsilon\)-close to \(L\) for every \(\epsilon > 0\)
Theorem (algebraic operations) If \(\lim a_n\) and \(\lim b_n\) exists, then
Theorem (Order operations)
The followings are some criterions useful to decide sequence convergence.
1.1.2. Convergence Criterions
The next two criterions are important because they do not require explicit limit to prove convergence. (although it implicitly requires a proposed limit to prove them)
Criterion (Cauchy) A sequence converges iff it is a Cauchy sequence
Criterion (Monotone bounded convergence) Let \((a_n)_{n=m}^{\infty}\) be a sequence of real numbers which has some finite upper bound \(M \in R\) and which is also increasing. Then \((a_n)_{n=m}^{\infty}\) is convergent and
proof of monotone bounded convergence
because \(a_n\) is bounded, we can proposed a limit \(s\) for \(a_n\)
because it is sup, for arbitrary \(\epsilon\), there exists an \(N\) such that \(|s - a_N| < \epsilon\), because \(a_n\) is monotonic, we know all \(n > N\) satisfy this, therefore \(s\) is the limit.
Criterion (Cauchy) A sequence converges iff it is a Cauchy sequence
proof of Cauchy criterion
it is trivial to see convergence implies Cauchy. To show the opposite, first notice that any Cauchy sequence is bounded. Then with BW-theorem, we can find a convergent subsequence, then show that the limit of this subsequence is the limit of the original sequence.
Criterion (squeeze test)
Criterion (Cesaro means) If \(x_n\) converges, then the mean sequence \(y_n\) also converges to the same limit
1.1.3. Limit Point
limit points over a sequence is a more general concept of limit. limit, limsup and liminf are instances of limit points. Normal Limit requires all points to be close enough to its convergence eventually, whereas limit points only ask for infinite points to be close enough.
Definition (limit points, adherent points over sequence) \(x\) is a limit points (adherent point) of \((a_n)_{n=m}^{\infty}\) when
Intuitively, the sequence visits \(\epsilon\) neighborhood of a limit point infinitely often (In contrast with the eventual visit of limits)
limsup and liminf are also special cases of limit points. Unlike limits, limsup and liminf exist for every sequence. I think they are useful to estimate the boundaries of sequences when they are oscillating.
Definition (limsup/liminf) limsup is the infimum over supremum of all elements. liminf is the superior over inferiors. (Both are defined over the extended reals to handle the divergence case)
Note limsup and liminf always exist for any sequence (it might be infity though)
Proposition (properties of limsup/liminf) Let \(L^+\) be limsup and \(L^-\) be liminf of sequence \((a_n)_{n=m}^{\infty}\). Then
- \(\forall x > L^+\), the sequence \((a_n)_{n=m}^{\infty}\) are eventually less than \(x\)
- \(\forall x < L^+\), the sequence \((a_n)_{n=m}^{\infty}\) exceeds \(x\) infinitely often
- \(\inf(a_n)_{n=m}^{\infty} \leq L^- \leq L^+ \leq \sup(a_n)_{n=m}^{\infty}\)
- If \(c\) is any limit points, then \(L^- \leq c \leq L^+\)
- If \(L^+\) is finite, then it is a limit point
- \(\lim_{n \to \infty} a_n \Longleftrightarrow c L^+ = L^- = c\)
As the order limit law, the sup and limsup also preserves the order
Lemma (sup/limsup preserves order) if \(a_n \leq b_n\), then
Proof: by contradition. Suppose \(\sup (a_n)_{n=m}^{\infty} > \sup (b_n)_{n=m}^{\infty}\), then take an \(\epsilon\) smaller than two sup diff, there exists one \(a_n\) be \(epsilon\) closer enough to its sup. This \(a_n\) is larger than b's sup, so its larger than every b.
1.1.4. Subsequences
Definition (subsequence) \((b_n)\) is a subsequence of \((a_n)\) iff there exists a function \(f: \mathbb{N} \to \mathbb{N}\) that is strictly increasing such that \(b_n = a_{f(n)}\)
Propositions (limits of subsequences) Following statements are equivalent
- sequence \((a_n)\) converges to \(L\)
- every subsequence of \((a_n)\) converges to \(L\)
Proposition (limit points of subsequences) Following statements are equivalent
- \(L\) is a limit point of sequence \((a_n)\)
- There exists a subsequence of \((a_n)\) which converges to \(L\)
Theorem (Bolzano-Weirstrass) A bounded sequence has at least one convergent subsequence
Proof: sequence bounded -> limsup is bounded -> limsup is a limit point -> subsequence exists
1.2. Series
This subsection is about taking sum of countable-size sequence
There are two types of convergence in series convergence: absolute convergence and conditional convergence. When a series converges absolutely, then it allows many manipulations such as rearrangement.
The series convergence is defined with respect to the convergence of the partial sum.
Definition (series convergence) The convergence of series \(\sum_{k=0}^{\infty} a_k\) is defined in terms of the convergence of partial sum \(s_n = \sum_{k=1}^n a_k\)
Example (Basel problem)
The following series is a convergent series, firstly solved by Euler.
uncountable sum
it is also possible to define sum over uncountable set \(X\) as with \(f: X \to [0, \infty]\)k
Theorem (Algebraic limit theorem for series) If \(\sum_{k=0}^{\infty} a_k = A\) and \(\sum_{k=0}^{\infty} b_k = B\), then
Proposition (associativity, commutativity of a series)
- Associativity holds if the series is convergent.
- Commutativity holds if the series is absolutely convergent.
Proof For associativity, consider the following regrouping
The regrouping creates a subsequence of the original series, which converges to the same limit when convergent. The associativity, however, does not work when the original series is not convergent (e.g: consider alternating -1, 1 series)
1.2.1. Conditional Convergence
Definition (conditional convergence) If \(\sum_{k=0}^{\infty} |a_k|\) not converge but \(\sum_{k=0}^{\infty} a_k\) converges, then \(\sum_{k=0}^{\infty} a_k\) is called conditional convergence
Alternating Harmonic Series
The alternating harmonic series is an example of conditional convergence, it does not converge absolutely.
1.2.2. Absolute Convergence
Absolute convergence is a stronger version of convergence over series. (It does not make much in standard sequence though). It has many good properties such that allow rearrangement of sequence.
Definition (absolute convergence) If the \(\sum_{k=0}^{\infty} |a_k|\) converges, then \(\sum_{k=0}^{\infty} a_k\) will converge and is called absolute convergence.
complete metric space
In a more general form, absolute convergence implies convergence in complete normed space (e.g. Banach space) where absolute convergence means \(\sum_i \|a_i\|\)
Theorem (rearangement on absolute convergence) If a series converges absolutely, then any rearangement of this series converges to the same limit
Theorem (rearangement over double sum) if \(\sum_{i}^{\infty} \sum_{j}^{\infty} | a_{ij} |\) converges, then rearangement is allowed and gives the same value
Fubini's theorem for infinite series
This theorem is the series version of the Fubini's theorem: double integral can be done iteratively if integral of absolute value of intergrand yield a finite answer
If
then
1.2.3. Convergence Criterion
Generally, speaking, it is much easier to determine whether a sequence converges or not rather than to compute the actual sum. For example, the series
can be easily confirmed that converges to something less than 2, but is much harder to compute its sum \(\frac{\pi^2}{6}\), which by the way discovered by Euler. Followings are some useful criterions to determine whether an series converges or not
Criterion (Cauchy) The series \(\sum_{k=0}^{\infty} a_k\) converges iff
Criterion (Condensation test) Suppose \(a_n\) is decreasing and satis \(a_n \geq 0\), then the series \(\sum_n b_n\) converges iff the following series converges
Harmonic Series
The harmonic series is a special case of the previous test
This series does not converge.
p-series
P-series is one generalization of the harmonic series, it converges when \(p>1\)
Proof: Condensation test says the series converges when the following series converges
this is the geometric series with \(2^(1-p) < 1\) when \(p>1\)
Criterion (comparison test) Assume \(a_k\) and \(b_k\) are sequences satisfying \((\forall k \in N)0 \leq a_k \leq b_k\)
if \(\sum_{k=1}^{\infty} b_k\) converges, then \(\sum_{k=1}^{\infty} a_k\) converges
if \(\sum_{k=1}^{\infty} b_k\) diverges, then \(\sum_{k=1}^{\infty} a_k\) diverges
Criterion (absolute convergence test) If series absolutely converges, then it converges as well
Criterion (Leibniz, alternating series test) Let \((a_n)\) be a sequence that \(a_1 > a_2 > a_3 ...\) and \(a_n \to 0\). Then the alternating series \(\sum_{n=1}^{\infty} (-1)^{n+1} a_n\) converges.
Proof Consider the tail of the series \(a_k - a_{k+1} + a_{k+2} ... + a_{k+l}\), it is easy to show that this tail is bounded by \([0, a_k]\), then obvious.
Criterion (ratio test) Given a series \(\sum_{n=1}^{\infty} a_n\), it will converges absolutely if
Criterion (Dirichlet's test) If the partial sum of \(\sum_{k=1}^{\infty} x_k\) are bounded, and \(y_k\) are monotone decreasing to 0, then \(\sum_{k=1}^{\infty} x_k y_k\) converges
2. Function Sequence
Infinite series representations of functions are both useful and elegant. Manipulations such as differentiation and anti-derivative can lead to remarkable conclusions when handled properly, however, they are not always justified.
2.1. Function Sequence
This subsection is about
2.1.1. Pointwise Convergence
Definition (pointwise convergence) The sequence \((f_n)\) of functions converges pointwise on \(A\) to a function \(f\) if for all \(x \in A\), the sequence of real numbers \(f_n(x)\) converges to \(f(x)\)
The point here is that the convergence speed depends on the point \(x\), it is possible for different values to require different, larger \(N\) to bound diff into \(\epsilon\)
The problem of pointwise convergence is that it might not preserve good properties such as continuity and differentiability. To preserve those good properties, we need to propose a more restricted class of convergence.
pointwise convergence does not preserve continuity
Let \(f_n(x)=x^n\) on the set \(x \in [0,1]\) where \(f_n(x)\) is continous function. when \(n \to \infty\), it converges to 0 for \(0 < x < 1\), 1 for \(x=1\), which is not a continuous function
pointwise convergence does not preserve differentiability
Let \(f_n(x)=x^{1+\frac{1}{2n-1}}\) on the set \([-1,1]\). This function converges to \(f(x)=|x|\), which is not differentiable.
pointwise convergence does not preserve Riemann integration
Let \(r_1, r_2, ...\) be a sequence that includes each rational number in \([0,1]\) exactly once, let \(f_k: [0, 1] \to R\) takes 1 when \(x \in \{ r_1, ..., r_k \}\) otherwise \(0\). Then each \(f_k\) is Riemann integrable and \(\int_0^1 f_k = 0\).
It is easy to see \(\lim_k f_k = f\) where \(f\) takes 1 when rational otherwise 0. However, \(f\) is not Riemann integrable (because set of discontinuity has measure of 1)
2.1.2. Uniform Convergence
Pointwise convergence does not imply uniform convergence. Consider the example \(f_n(x)=x^n\) where \(x \in [0, 1]\). It is pointwisely converging to \(f(x)=0\) for \(x \in [0, 1)\) and \(f(1)=1\), which is not an uniform convergence
Pointwise convergence is a local property, while the uniform convergence is a global property. Uniform convergence have a couple of good properties as follows. It preserves continuity, differentiability and integrability.
Definition (uniform convergence) The sequence \((f_n)\) converges uniformly on \(A\) to a limit function \(f\) defined on \(A\) if for every \(\epsilon > 0\), there exists an \(N\) such that
Criterion (Cauchy) \(f_n\) converges uniformly to \(f\) iff for every \(\epsilon > 0\), there exists a \(N\) such that
2.1.3. Uniform Convergence and Continuity
Theorem (continuity) Let \(f_n\) be a sequence of functions defined on \(A \subset R\) that converges uniformly on \(A\) to a function \(f\). If \(f_n\) is continuous at \(c \in A\), then \(f\) is continuous at \(c\)
Proof idea is to evaluate \(|f(x)-f(y)|\) by 3 pieces
where each piece can be bounded uniformly.
Corrolary (uniform continuity) uniform continuity is also preserved by uniform convergence by a similar proof
Dini's theorem
previous theorem shows that uniform convergence indicates continuity. Dini's theorem shows that continuity of converging function indicates uniform convergence under some conditions.
Assume \(f_n \to f\) pointwise on a compact set \(K\) and assume that for each \(x \in K\), the sequence \(f_n(x)\) is increasing (wrt \(n\)). If \(f_n, f\) are continuous on \(K\), then \(f_n \to f\) uniformly.
2.1.4. Uniform Convergence and Differentiability
Both pointwise convergence and uniform convergence of \(f_n\) implies nothing about the sequence of \(f'_n\), neither uniform convergence or pointwise convergence of \(f'_n\) is guaranteed.
Theorem (differentiability) Let \(f_n\) be a sequence of differetiable functions defined on \([a,b]\) and \(f'_{n}\) converges uniformly to a function \(g\) on \([a,b]\). If there exists a point \(x_0 \in [a,b]\) for which \(f_n(x_0)\) is convergent, then \(f_n\) converges uniformly. Moreover, the limit function \(f=\lim f_n\) is differentiable and satisfies \(f' = g\)
Proof: idea is to evaluate \(|\frac{f(x)-f(c)}{x-c} - g(c)|\) by 3 pieces
where the first term can be bound using mean value theorem
Note again that \(f_n\)'s uniform convergence does not imply \(f'_n\)'s convergence, but \(f'_n\)'s uniform convergence almost implies \(f_n\)'s uniform convergence
\(f_n\) uniform convergence does not imply \(f'_n\)'s uniform convergence
Consider the function
This function converges to 0 uniformly on \([0,1]\), however, \(f'_n(x)\) is not converge uniformly on \([0,1]\)
\(f_n\) uniform convergence does not even imply \(f'_n\)'s pointwise convergence
Consider the function
This function converges to 0 uniformly on \(R\) but \(f'_n\) diverges for every \(x\)
2.1.5. Uniform Convergence and Integrability
Theorem (integrability) Assume that \(f_n \to f\) uniformly on \([a, b]\), and each \(f_n\) is integrable. Then \(f\) is integrable and
\(\lim\) and \(\int\) might not interchange when \(f_n\) is pointwise convergence
Consider the function \(f(x)\) with domain \([0,1]\)
This converges pointwise to \(f(x) = 0\) on \([0,1]\), however,
Again, the interchange is valid when the convergence is uniform convergence
2.2. Function Series
This subsection is about
2.2.1. Convergence
Definition (function series convergence) For each \(n\), let \(f_n\) and \(f\) be functions defined on a set \(A\). The infinite series
converges pointwise on A to \(f(x)\) iff the sequence \(s_k(x)\) of partial sums defined by
converges pointwise to \(f(x)\). The series converges uniformly on \(A\) to \(f\) if the sequence \(s_k(x)\) converges uniformly on \(A\) to \(f(x)\).
By applying uniform convergence properties to series, we get following conclusions.
Theorem (Term-by-Term Continuity Theorem) Let \(f_n\) be continuous functions defined on a set \(A\), and \(\sum_{n=1}^{\infty} f_n\) converges uniformly on \(A\) to a function \(f\), then \(f\) is continuous on \(A\).
Theorem (Term-by-Term Differentiability Theorem) Let \(f_n\) be differentiable functions defined on an interval \(A\), and \(\sum_{n=1}^{\infty} f'_n(x)\) converges uniformly to a limit \(g(x)\) on \(A\). If there exists a point \(x_0 \in [a,b]\) where \(\sum_{n=1}^{\infty} f_n(x_0)\) converges, then the series \(\sum_{n=1}^{\infty} f_n(x)\) converges uniformly to a differentiable function \(f(x)\) satisfying \(f'(x)=g(x)\), which means
Criterion (Cauchy) A series \(\sum_{i} f_n\) converges uniformly on \(A \subseteq R\) iff
Criterion (Weierstrass M-test) For each \(n \in N\), let \(f_n\) be a function defined on a set \(A \subset R\) and let \(M_n > 0\) be a real number satisfying
where \(\sum_{n=1}^{\infty} M_n\) converges, then \(\sum_{n=1}^{\infty} f_n\) converges uniformly on \(A\)
M-test's opposite is not true
Even if \(f_n\) converges uniformly, an upper bound convergent series \(M_n\) may not exist. Think about the constant function of the alternating harmonic series
2.3. Power Series
Definition (power series) Power series are functions of the form:
2.3.1. Radius of Convergence
If a power series converges at some point \(x_0 \in \mathbf{R}\), then it converges absolutely for all \(|x| < |x_0|\).
The main implication here is that the set of convergence has to be \({0}, \mathbf{R}\) or a bounded interval: \((-R, R), [-R, R), (-R, R], [-R, R]\) where \(R\) is called the radius of convergence.
The series:
- absolutely converges when \(|x|<R\)
- diverges when \(|x|>R\)
- inconclusive when \(|x|=R\), it can one of three cases: absolutely convergence, conditional convergence or divergence.
radius is 0
Consider the series
This has the radius of convergece \(R=0\), it converges only at 0, and diverges all other points
radius is r
When radius=r, there are 4 possibility of convergence set
1) convergence happens at \((-r, r)\)
2) convergence happens at \([-r, r)\)
3) convergence happens at \((-r, r]\)
4) convergence happens at \([-r, r]\)
radius is infinity
Consider the series
This is \(e^x\) and has radius of convergence \(R=\infty\), it converges at every point
There are typically two approaches to compute radius:
Theorem (root test, Cauchy–Hadamard) Let \(\sum_{n=0}^{\infty} c_n (x-a)^n\) be a formal power series. we can compute the radius of convergence \(R\) of this series to be
Theorem (ratio test, d’Alembert) Another approach to find ratio sufficiently is
2.3.2. Continuity and Differentiation
On its interval of convergence, a power series can be manipulated more or less as if they are polynomials:
- they are continuous and infinitely differentiable.
- Successive derivatives and antiderivatives can be computed by performing the desired operation on each individual term in the series.
Theorem (Abel) Let \(g(x) = \sum_{n=0}^{\infty} a_n x^n\) be a power series that converge at \(x=R > 0\), then it converges uniformly on the interval \([0, R]\).
Theorem (uniform convergence of power series) If a power series converges pointwise on the set \(A \subseteq R\), then it converges uniformly on any compact set \(K \subseteq A\)
Theorem (continuity and differentiation of power series) Assume \(f(x)=\sum_{n=0}^{\infty} a_n x^n\) converges on an interval \(A \subseteq R\). The function \(f\) is continuous on \(A\) and differentiable on any open interval \((-R,R) \subseteq A\). The derivative is given by \(f'(x) = \sum_{n=1}^{\infty} na_n x^{n-1}\).
2.4. Taylor Series
Theorem (Taylor series) Let \(f(x)\) has a power series representation. i.e.:
be defined on some nontrivial interval centered at zero, then
Therefore, the Talyor series (Maclaurin series) is
Note this is saying if \(f(x)\) has a power series representation, then it has to be the Taylor series.
It does not guarantee Taylor series converges to \(f(x)\) everywhere.
Talyor series might not be convergent on R
Consider the following representation
This series is only convergent in (-1,1)
2.4.1. Convergence
Note there are two different concepts of convergence about Taylor series here:
- Taylor series converges within its radius of convergence
- Talyor series converges to its target function
Even Taylor series converges, it might not converges to the target function.
smooth function maynot be analytic
Consider the function
The Taylor series of this function is \(0\) everywhere, therefore it does not converges to the original \(f(x)\)
In this section, we want to think about the condition when Taylor series converges to its target function.
To measure the diff between the Talyor series and the actual value, we have a powerful tool.
Theorem (Lagrange's Remainder Theorem) Let \(f\) be differentiable \(N+1\) times on \((-R, R)\), define \(a_n = f^{(n)}(0)/n!\), Given \(x \neq 0\) in \((-R,R)\), there exists a point \(c\) such that \(|c| < |x|\) , then the error function \(E_N(x) = f(x) - S_N(x)\) is
when \(E_N(x) \to 0\) on some intervals, then we know it converges to the target function on this interval
Maclaurin series' convergence to target function
3. Reference
[1] Tao, Terence. Analysis. Vol. 1. Hindustan Book Agency, 2006.
[2] Tao, Terence. Analysis. Vol. 2. Hindustan Book Agency, 2006.
[3] Abbott, Stephen. Understanding analysis. Vol. 2. New York: Springer, 2001.
[4] Lax, Peter D., and Maria Shea Terrell. Multivariable Calculus with Applications. Springer, 2017.
[5] 杉浦光夫. "解析入門 I." 東京大学出版会
[6] 杉浦光夫. "解析入門 II." 東京大学出版会