Prerequisites

The syllabus lists the assumed knowledge for Math Camp (Chapters 1-4 of Moore and Siegel). Below, you will find additional review material, examples, and sample exercises.

Operators

Addition (+), Subtraction (-), multiplication and division are basic operations of arithmetic – combining numbers. In statistics and calculus, we want to add a sequence of numbers that can be expressed as a pattern without needing to write down all its components. For example, how would we express the sum of all numbers from 1 to 100 without writing a hundred numbers?

For this we use the summation operator \sum and the product operator \prod.

Summation

\sum\limits_{i=1}^{100} x_i = x_1+x_2+x_3+\cdots+x_{100}

The bottom of the \sum symbol indicates an index (here, i), and its start value 1. At the top is where the index ends. The notion of “addition” is part of the \sum symbol. The content to the right of the summation is the meat of what we add. While you can pick your favorite index, start, and end values, the content must also have the index.

Corollary 1

\sum\limits_{i=1}^n c x_i = c \sum\limits_{i=1}^n x_i
\sum\limits_{i=1}^n (x_i + y_i) = \sum\limits_{i=1}^n x_i + \sum\limits_{i=1}^n y_i
\sum\limits_{i=1}^n c = n c

Product

\prod\limits_{i=1}^n x_i = x_1 x_2 x_3 \cdots x_n

Corollary 2

\prod\limits_{i=1}^n c x_i = c^n \prod\limits_{i=1}^n x_i
\prod\limits_{i=k}^n c x_i = c^{n-k+1} \prod\limits_{i=k}^n x_i
\prod\limits_{i=1}^n (x_i + y_i) = a total mess
\prod\limits_{i=1}^n c = c^n

Factorials

Definition 1 (Factorials)

x! = x\cdot (x-1) \cdot (x-2) \cdots (1)

Modulo

Tells you the remainder when you divide the first number by the second.

17 \mod 3 = 2
100 \ \% \ 30 = 10

Exercise 1 (Operators) Let x_1 = 4, x_2 = 3, x_3 = 7, x_4 = 11, x_5 = 2. Find the following:

\sum\limits_{i=1}^{5} i
\prod\limits_{i=1}^{5} i
14 \mod 4
4!
\sum\limits_{i=1}^{3} (7)x_i
\sum\limits_{i=1}^{5} 2
\prod\limits_{i=3}^{5} (2)x_i

Functions

A function is a mapping that relates members of one set to members of another set.

For instance, if you have two sets: set A and set B, a function f from A to B maps every value a in set A such that f(a) \in B.

The set A is called the domain of function f
The set B is called the range of function f

Functions can be many-to-one, where many values from set A produce a single output in set B, or they can be one-to-one, where each value in set A corresponds to a single value in set B.

A function by definition has a single function value for each element of its domain. This means, there cannot be “one-to-many” mapping.

Dimensionality

\mathbb{R}^1 is the set of all real numbers extending from -\infty to +\infty — i.e., the real number line. \mathbb{R}^n is an n-dimensional space, where each of the n axes extends from -\infty to +\infty.

\mathbb{R}^1 is a one dimensional line.
\mathbb{R}^2 is a two dimensional plane.
\mathbb{R}^3 is a three dimensional space.

Points in \mathbb{R}^n are ordered n-tuples (just means an combination of n elements where order matters), where each element of the n-tuple represents the coordinate along that dimension.

For example:

\mathbb{R}^1: (3)
\mathbb{R}^2: (-15, 5)
\mathbb{R}^3: (86, 4, 0)

Notation (Functions)

Function of one variable:

f:\mathbb{R}^1\to\mathbb{R}^1

Example: f(x)=x+1
- For each x in \mathbb{R}^1, f(x) assigns the number x+1

Function of two variables:

f: \mathbb{R}^2\to\mathbb{R}^1

Example: f(x,y)=x^2+y^2
- For each ordered pair (x,y) in \mathbb{R}^2, f(x,y) assigns the number x^2+y^2

We often use variable x as input and another y as output, e.g. y=x+1

Exercise 2 (Functions) For each of the following, state whether they are one-to-one or many-to-one functions.

For x \in [0,\infty], f : x \rightarrow x^2 (this could also be written as f(x) = x^2).
For x \in [-\infty, \infty], f: x \rightarrow x^2.
For x \in [-3, \infty], f: x \rightarrow x^2.
For x \in [0, \infty], f: x \rightarrow \sqrt{x}

Some functions are defined only on proper subsets of \mathbb{R}^n.

Domain: the set of numbers in X at which f(x) is defined.
Range: elements of Y assigned by f(x) to elements of X, or we can use the notation f(X) to denote the range, where f(X)=\{ y : y=f(x), x\in X\}

Types of Functions

Monomials: f(x)=a x^k

a is the coefficient. k is the degree.

Examples: y=x^2, y=-\frac{1}{2}x^3

Polynomials: sum of monomials.

Examples: y=-\frac{1}{2}x^3+x^2, y=3x+5

The degree of a polynomial is the highest degree of its monomial terms. Also, it’s often a good idea to write polynomials with terms in decreasing degree.

Exponential Functions: Example: y=2^x

Logs and Exponents

Relationship of logarithmic and exponential functions:

y=\log_a(x) \iff a^y=x

The log function can be thought of as an inverse for exponential functions. a is referred to as the “base” of the logarithm.

Common Bases: The two most common logarithms are base 10 and base e.

Base 10: \quad y=\log_{10}(x) \iff 10^y=x. The base 10 logarithm is often simply written as “\log(x)” with no base denoted.
Base e: \quad y=\log_e(x) \iff e^y=x. The base e logarithm is referred to as the “natural” logarithm and is written as ``\ln(x)“.

Properties of exponential functions:

a^x a^y = a^{x+y}
a^{-x} = 1/a^x
a^x/a^y = a^{x-y}
(a^x)^y = a^{x y}
a^0 = 1

Properties of logarithmic functions (any base):

Generally, when statisticians or social scientists write \log(x) they mean \log_e(x). In other words: \log_e(x) \equiv \ln(x) \equiv \log(x)

\log_a(a^x)=x

and

a^{\log_a(x)}=x

\log(x y)=\log(x)+\log(y)
\log(x^y)=y\log(x)
\log(1/x)=\log(x^{-1})=-\log(x)
\log(x/y)=\log(x\cdot y^{-1})=\log(x)+\log(y^{-1})=\log(x)-\log(y)
\log(1)=\log(e^0)=0

Change of Base Formula: Use the change of base formula to switch bases as necessary:

\log_b(x) = \frac{\log_a(x)}{\log_a(b)}

Example:

\log_{10}(x) = \frac{\ln(x)}{\ln(10)}

You can use logs to go between sum and product notation. This will be particularly important when you’re learning maximum likelihood estimation.

\begin{aligned} \log \bigg(\prod\limits_{i=1}^n x_i \bigg) &= \log(x_1 \cdot x_2 \cdot x_3 \cdots \cdot x_n) \\ &= \log(x_1) + \log(x_2) + \log(x_3) + \cdots + \log(x_n) \\ &= \sum\limits_{i=1}^n \log (x_i) \end{aligned}

Therefore, you can see that the log of a product is equal to the sum of the logs. We can write this more generally by adding in a constant, c:

\begin{aligned} \log \bigg(\prod\limits_{i=1}^n c x_i\bigg) &= \log(cx_1 \cdot cx_2 \cdots cx_n) \\ &= \log(c^n \cdot x_1 \cdot x_2 \cdots x_n) \\ &= \log(c^n) + \log(x_1) + \log(x_2) + \cdots + \log(x_n) \\ &= n \log(c) + \sum\limits_{i=1}^n \log (x_i) \end{aligned}

Exercise 3 (Log) Evaluate each of the following logarithms

\log_4(16)
\log_2(16)
\log_\frac{3}{2}(\frac{27}{8})

Simplify the following logarithm. By “simplify”, we actually really mean - use as many of the logarithmic properties as you can.

\log_4(x^3y^5)
\log(\frac{x^9y^5}{z^3})
\ln{\sqrt{xy}}

Graphing Functions

What can a graph tell you about a function?

Is the function increasing or decreasing? Over what part of the domain?
How ``fast” does it increase or decrease?
Are there global or local maxima and minima? Where?
Are there inflection points?
Is the function continuous?
Is the function differentiable?
Does the function tend to some limit?
Other questions related to the substance of the problem at hand.

Solving for Variables

Sometimes we’re given a function y=f(x) and we want to find how x varies as a function of y. Use algebra to move x to the left hand side (LHS) of the equation and so that the right hand side (RHS) is only a function of y.

Example 1 (Solving for Variables) Solve for x:

y=3x+2
y=e^x

Solving for variables is especially important when we want to find the roots of an equation: those values of variables that cause an equation to equal zero. Especially important in finding equilibria and in doing maximum likelihood estimation.

Procedure: Given y=f(x), set f(x)=0. Solve for x.

Multiple Roots:

f(x)=x^2 - 9 \quad\Longrightarrow\quad 0=x^2 - 9 \quad\Longrightarrow\quad 9=x^2 \quad\Longrightarrow\quad \pm \sqrt{9}=\sqrt{x^2} \quad\Longrightarrow\quad \pm 3=x

Quadratic Formula: For quadratic equations ax^2+bx+c=0, use the quadratic formula:

x=\frac{-b\pm\sqrt{b^2-4ac}}{2a}

Exercise 4 (Solving for Variables) Solve for x:

f(x)=3x+2 = 0
f(x)=x^2+3x-4=0
f(x)=e^{-x}-10 = 0

Answers to Examples and Exercises

Answer to Exercise 1:

1 + 2 + 3 + 4 + 5 = 15
1 * 2 * 3 * 4 * 5 = 120
2
4 * 3 * 2 * 1 = 24
7(4 + 3 + 7) = 98
2 + 2 + 2 + 2 + 2 = 10
2^3(7)(11)(2) = 1232

Answer to Exercise 2:

one-to-one
many-to-one
many-to-one
one-to-one

Answer to Exercise 3:

2
4
3
3\log_4(x) + 5\log_4(y)
9\log(x) + 5\log(y) - 3\log(z)
\frac{1}{2}(\ln{x} + \ln{y})

Answer to Example 1:

y=3x+2 \quad\Longrightarrow\quad -3x=2-y \quad\Longrightarrow\quad 3x=y-2 \quad\Longrightarrow\quad x=\frac{1}{3}(y-2)
x = \ln{y}

Answer to Exercise 4:

\frac{-2}{3}
x = {1, -4}
x = - \ln10

# Prerequisites {-} The syllabus lists the assumed knowledge for Math Camp (Chapters 1-4 of Moore and Siegel). Below, you will find additional review material, examples, and sample exercises. ## Operators Addition ($+$), Subtraction ($-$), multiplication and division are basic operations of arithmetic -- combining numbers. In statistics and calculus, we want to add a _sequence_ of numbers that can be expressed as a pattern without needing to write down all its components. For example, how would we express the sum of all numbers from 1 to 100 without writing a hundred numbers? For this we use the summation operator $\sum$ and the product operator $\prod$. ### Summation $$ \sum\limits_{i=1}^{100} x_i = x_1+x_2+x_3+\cdots+x_{100} $$ The bottom of the $\sum$ symbol indicates an index (here, $i$), and its start value $1$. At the top is where the index ends. The notion of "addition" is part of the $\sum$ symbol. The content to the right of the summation is the meat of what we add. While you can pick your favorite index, start, and end values, the content must also have the index. ::: {#cor-summation} - $\sum\limits_{i=1}^n c x_i = c \sum\limits_{i=1}^n x_i$ - $\sum\limits_{i=1}^n (x_i + y_i) = \sum\limits_{i=1}^n x_i + \sum\limits_{i=1}^n y_i$ - $\sum\limits_{i=1}^n c = n c$ ::: ### Product $$ \prod\limits_{i=1}^n x_i = x_1 x_2 x_3 \cdots x_n $$ ::: {#cor-product} - $\prod\limits_{i=1}^n c x_i = c^n \prod\limits_{i=1}^n x_i$ - $\prod\limits_{i=k}^n c x_i = c^{n-k+1} \prod\limits_{i=k}^n x_i$ - $\prod\limits_{i=1}^n (x_i + y_i) =$ a total mess - $\prod\limits_{i=1}^n c = c^n$ ::: ### Factorials :::{#def-summation} #### Factorials $$ x! = x\cdot (x-1) \cdot (x-2) \cdots (1) $$ ::: ### Modulo Tells you the remainder when you divide the first number by the second. - $17 \mod 3 = 2$ - $100 \ \% \ 30 = 10$ :::{#exr-operators} ### Operators Let $x_1 = 4, x_2 = 3, x_3 = 7, x_4 = 11, x_5 = 2$. Find the following: 1. $\sum\limits_{i=1}^{5} i$ 2. $\prod\limits_{i=1}^{5} i$ 3. $14 \mod 4$ 4. $4!$ 5. $\sum\limits_{i=1}^{3} (7)x_i$ 6. $\sum\limits_{i=1}^{5} 2$ 7. $\prod\limits_{i=3}^{5} (2)x_i$ ::: ## Functions A **function** is a mapping that relates members of one set to members of another set. For instance, if you have two sets: set $A$ and set $B$, a function $f$ from $A$ to $B$ maps every value $a$ in set $A$ such that $f(a) \in B$. - The set $A$ is called the **domain** of function $f$ - The set $B$ is called the **range** of function $f$ Functions can be **many-to-one**, where many values from set $A$ produce a single output in set $B$, or they can be **one-to-one**, where each value in set $A$ corresponds to a single value in set $B$. A function by definition has a single function value for each element of its domain. This means, there cannot be "one-to-many" mapping. ### Dimensionality $\mathbb{R}^1$ is the set of all real numbers extending from $-\infty$ to $+\infty$ --- i.e., the real number line. $\mathbb{R}^n$ is an $n$-dimensional space, where each of the $n$ axes extends from $-\infty$ to $+\infty$. * $\mathbb{R}^1$ is a one dimensional line. * $\mathbb{R}^2$ is a two dimensional plane. * $\mathbb{R}^3$ is a three dimensional space. Points in $\mathbb{R}^n$ are ordered $n$-tuples (just means an combination of $n$ elements where order matters), where each element of the $n$-tuple represents the coordinate along that dimension. For example: * $\mathbb{R}^1$: $(3)$ * $\mathbb{R}^2$: $(-15, 5)$ * $\mathbb{R}^3$: $(86, 4, 0)$ ::: {.callout-important} ### Notation (Functions) **Function of one variable**: $$ f:\mathbb{R}^1\to\mathbb{R}^1 $$ - Example: $f(x)=x+1$ - For each $x$ in $\mathbb{R}^1$, $f(x)$ assigns the number $x+1$ **Function of two variables**: $$ f: \mathbb{R}^2\to\mathbb{R}^1 $$ - Example: $f(x,y)=x^2+y^2$ - For each ordered pair $(x,y)$ in $\mathbb{R}^2$, $f(x,y)$ assigns the number $x^2+y^2$ ::: We often use variable $x$ as input and another $y$ as output, e.g. $y=x+1$ ::: {#exr-functions} ### Functions For each of the following, state whether they are one-to-one or many-to-one functions. 1. For $x \in [0,\infty]$, $f : x \rightarrow x^2$ (this could also be written as $f(x) = x^2$). 2. For $x \in [-\infty, \infty]$, $f: x \rightarrow x^2$. 3. For $x \in [-3, \infty]$, $f: x \rightarrow x^2$. 4. For $x \in [0, \infty]$, $f: x \rightarrow \sqrt{x}$ ::: Some functions are defined only on proper subsets of $\mathbb{R}^n$. * **Domain**: the set of numbers in $X$ at which $f(x)$ is defined. * **Range**: elements of $Y$ assigned by $f(x)$ to elements of $X$, or we can use the notation $f(X)$ to denote the range, where $$f(X)=\{ y : y=f(x), x\in X\}$$ ### Types of Functions __Monomials__: $f(x)=a x^k$ $a$ is the coefficient. $k$ is the degree. Examples: $y=x^2$, $y=-\frac{1}{2}x^3$ __Polynomials__: sum of monomials. Examples: $y=-\frac{1}{2}x^3+x^2$, $y=3x+5$ The degree of a polynomial is the highest degree of its monomial terms. Also, it's often a good idea to write polynomials with terms in decreasing degree. __Exponential Functions__: Example: $y=2^x$ ## Logs and Exponents {#logexponents} __Relationship of logarithmic and exponential functions__: $$ y=\log_a(x) \iff a^y=x $$ The log function can be thought of as an inverse for exponential functions. $a$ is referred to as the "base" of the logarithm. __Common Bases__: The two most common logarithms are base 10 and base $e$. 1. Base 10: $\quad y=\log_{10}(x) \iff 10^y=x$. The base 10 logarithm is often simply written as "$\log(x)$" with no base denoted. 2. Base $e$: $\quad y=\log_e(x) \iff e^y=x$. The base $e$ logarithm is referred to as the "natural" logarithm and is written as ``$\ln(x)$". \begin{comment} {\includegraphics[width=1in, angle = 270]{ln.eps}} \, {\includegraphics[width=1in, angle = 270]{exp.eps}} \end{comment} __Properties of exponential functions:__ * $a^x a^y = a^{x+y}$ * $a^{-x} = 1/a^x$ * $a^x/a^y = a^{x-y}$ * $(a^x)^y = a^{x y}$ * $a^0 = 1$ __Properties of logarithmic functions__ (any base): Generally, when statisticians or social scientists write $\log(x)$ they mean $\log_e(x)$. In other words: $\log_e(x) \equiv \ln(x) \equiv \log(x)$ $$ \log_a(a^x)=x $$ and $$ a^{\log_a(x)}=x $$ * $\log(x y)=\log(x)+\log(y)$ * $\log(x^y)=y\log(x)$ * $\log(1/x)=\log(x^{-1})=-\log(x)$ * $\log(x/y)=\log(x\cdot y^{-1})=\log(x)+\log(y^{-1})=\log(x)-\log(y)$ * $\log(1)=\log(e^0)=0$ __Change of Base Formula__: Use the change of base formula to switch bases as necessary: $$ \log_b(x) = \frac{\log_a(x)}{\log_a(b)} $$ Example: $$ \log_{10}(x) = \frac{\ln(x)}{\ln(10)} $$ You can use logs to go between sum and product notation. This will be particularly important when you're learning maximum likelihood estimation. $$ \begin{aligned} \log \bigg(\prod\limits_{i=1}^n x_i \bigg) &= \log(x_1 \cdot x_2 \cdot x_3 \cdots \cdot x_n) \\ &= \log(x_1) + \log(x_2) + \log(x_3) + \cdots + \log(x_n) \\ &= \sum\limits_{i=1}^n \log (x_i) \end{aligned} $$ Therefore, you can see that the log of a product is equal to the sum of the logs. We can write this more generally by adding in a constant, $c$: $$ \begin{aligned} \log \bigg(\prod\limits_{i=1}^n c x_i\bigg) &= \log(cx_1 \cdot cx_2 \cdots cx_n) \\ &= \log(c^n \cdot x_1 \cdot x_2 \cdots x_n) \\ &= \log(c^n) + \log(x_1) + \log(x_2) + \cdots + \log(x_n) \\ &= n \log(c) + \sum\limits_{i=1}^n \log (x_i) \end{aligned} $$ ::: {#exr-log} ### Log Evaluate each of the following logarithms 1. $\log_4(16)$ 2. $\log_2(16)$ 3. $\log_\frac{3}{2}(\frac{27}{8})$ Simplify the following logarithm. By "simplify", we actually really mean - use as many of the logarithmic properties as you can. 4. $\log_4(x^3y^5)$ 5. $\log(\frac{x^9y^5}{z^3})$ 6. $\ln{\sqrt{xy}}$ ::: ## Graphing Functions What can a graph tell you about a function? * Is the function increasing or decreasing? Over what part of the domain? * How ``fast" does it increase or decrease? * Are there global or local maxima and minima? Where? * Are there inflection points? * Is the function continuous? * Is the function differentiable? * Does the function tend to some limit? * Other questions related to the substance of the problem at hand. ## Solving for Variables Sometimes we're given a function $y=f(x)$ and we want to find how $x$ varies as a function of $y$. Use algebra to move $x$ to the left hand side (LHS) of the equation and so that the right hand side (RHS) is only a function of $y$. ::: {#exm-solvevar} ### Solving for Variables Solve for x: 1. $y=3x+2$ 2. $y=e^x$ ::: Solving for variables is especially important when we want to find the __roots__ of an equation: those values of variables that cause an equation to equal zero. Especially important in finding equilibria and in doing maximum likelihood estimation. Procedure: Given $y=f(x)$, set $f(x)=0$. Solve for $x$. Multiple Roots: $$ f(x)=x^2 - 9 \quad\Longrightarrow\quad 0=x^2 - 9 \quad\Longrightarrow\quad 9=x^2 \quad\Longrightarrow\quad \pm \sqrt{9}=\sqrt{x^2} \quad\Longrightarrow\quad \pm 3=x $$ __Quadratic Formula:__ For quadratic equations $ax^2+bx+c=0$, use the quadratic formula: $$ x=\frac{-b\pm\sqrt{b^2-4ac}}{2a} $$ ::: {#exr-solvevar} ### Solving for Variables Solve for x: 1. $f(x)=3x+2 = 0$ 2. $f(x)=x^2+3x-4=0$ 3. $f(x)=e^{-x}-10 = 0$ ::: ## Answers to Examples and Exercises {-} Answer to @exr-operators: 1. $1 + 2 + 3 + 4 + 5 = 15$ 2. $1 * 2 * 3 * 4 * 5 = 120$ 3. $2$ 4. $4 * 3 * 2 * 1 = 24$ 1. $7(4 + 3 + 7) = 98$ 2. $2 + 2 + 2 + 2 + 2 = 10$ 3. $2^3(7)(11)(2) = 1232$ Answer to @exr-functions: 1. one-to-one 2. many-to-one 3. many-to-one 4. one-to-one Answer to @exr-log: 1. 2 2. 4 3. 3 4. $3\log_4(x) + 5\log_4(y)$ 5. $9\log(x) + 5\log(y) - 3\log(z)$ 6. $\frac{1}{2}(\ln{x} + \ln{y})$ Answer to @exm-solvevar: 1. $y=3x+2 \quad\Longrightarrow\quad -3x=2-y \quad\Longrightarrow\quad 3x=y-2 \quad\Longrightarrow\quad x=\frac{1}{3}(y-2)$ 2. $x = \ln{y}$ Answer to @exr-solvevar: 1. $\frac{-2}{3}$ 2. x = {1, -4} 3. x = - $\ln10$