Calculus I

1 Calculus

Calculus may be a foreign subject compared to what we have covered so far. Many of us have made career choices to deliberately avoid having to deal with calculus. Still, calculus is important because for computational social science because, at its core, it is about dealing with continuity in a consistent manner.

More often than not we will deal with variables and distributions that are (at least assumed) continuous. Moreover, many single-number quantities of interest in data analysis, like p-values or standard errors, are calculated from continuous distributions (more about this in POLI SCI 403).

Dealing with continuity also means introducing new operations beyond the usual addition, subtraction, multiplication, and division, hence the difficulty. We will take it slow and focus on one at a time. Today is all about derivatives.

2 Derivatives

Derivatives are about (instantaneous) rate of change.

A helpful way to think about a derivatives is as a slope. Let’s consider a linear function of the form \(y = 2 x\):

library(tidyverse) # could also just do library(ggplot2)
ggplot() +
  stat_function(fun = function(x){2 * x}, 
                xlim = c(-10, 10))

We can imagine any political variables in the x- and y-axes. What is the rate of change? In other words, what is the derivative? Remember that we can calculate the slope with

\[ m = \frac{f(x_2) - f(x_1)}{x_2-x_1} \]

which you can remember with the mnemonic “rise over run.”

Now consider another slightly more complicated function, a quadratic one, \(y = x^2\):

ggplot() +
  stat_function(fun = function(x){x ^ 2}, 
                xlim = c(-10, 10))

What happens when we apply our slope function?

Answer

Here the derivative depends on the value of \(x\). It is actually \(2x\).

Differential calculus is about finding these derivatives in a more straightforward manner. We can generalize our slope formula as

\[ m = \frac{f(x_1+ \Delta x) - f(x_1)}{\Delta x} \]

The point is that when \(\Delta x\) is arbitrarily small, we’ll get our rate of change. Formalizing this:

\[ \lim_{\Delta x\to0} \frac{f(x_1+ \Delta x) - f(x_1)}{\Delta x} = \frac{d}{dx} f(x) = \frac{dy}{dx} = f'(x) \]

A few points on notation:

\(\frac{d}{dx} f(x)\) is read “The derivative of \(f\) of \(x\) with respect to \(x\).” One would be tempted to read \(\frac{d}{dx}\) as a fraction, but it is just here for notation.

The variable with respect to which we’re differentiating is the one that appears in the bottom (in the case above, this is \(x\)).

\(f'(x)\) (\(f\) prime \(x\)) is the derivative of \(f(x)\). This is a more compact form to refer to derivatives when you have defined \(f(x)\) elsewhere.

2.1 Rules of differentiation

How to compute derivatives? Sometimes you can try a bunch of numbers and get at the answer. Sometimes you can use the limit-based formula above, if you know a few properties of limits. But in most cases you will either use software (more on this later) or the rules of differentiation, which we will cover now.

Constant rule: \((c)' = 0\).

There is no change in a constant:

ggplot() +
  stat_function(fun = function(x){2}, xlim = c(-10, 10))

Coefficient rule: \((c \cdot f(x))' = c \cdot f'(x)\).

ggplot() +
  stat_function(fun = function(x){2 * x}, xlim = c(-10, 10), aes(color = "y = 2x")) +
  stat_function(fun = function(x){4 * x}, xlim = c(-10, 10), aes(color = "y = 4x")) +
  scale_color_manual("Function", values = c("red", "blue"))

Sum/difference rule: \((f(x) \pm g(x))' = f'(x) \pm g'(x)\).

The two rules above give us that the derivative is a linear operator.

Power rule: \((x^n)'=nx^{(n-1)}\)

Remember when we wanted to calculate the derivative of \(y=x^2\) above? We can use the power rule, with \(n=2\)

\[ nx^{(n-1)} = 2x^{(2-1)}=2x \]

\(\;nx^{(n-1)} = 2x^{(2-1)}=2x\).

Let’s try out \(\frac{d}{dx}4x^3\) and \(\frac{d}{dx}(x^2 + 2x)\) on the board.

Here are a few functions to practice the differentiation rules:

Exercise

Use the differentiation rules we have covered so far to calculate the derivatives of \(y\) with respect to \(x\) of the following functions:

1. \(y= 4x^3\)
2. \(y= x^2 + 2x\)
3. \(y = 2x^2 + 10\)
4. \(y = 5x^4 - \frac{2}{3}x^3\)
5. \(y = 9 \sqrt x\)
6. \(y = \frac{4}{x^2}\)
7. \(y = ax^3 + b\), where \(a\) and \(b\) are constants.
8. \(y = \frac{2w}{5}\)

Exponent and logarithm rules:

\[ \begin{aligned} (c^x)' &= c^x \cdot ln(c), & \forall x>0 \\ \\ (log_a(x))' &= \frac{1}{x \cdot ln(a)}, & \forall x>0 \\ \end{aligned} \]

Euler’s number (\(e\)) has interesting properties when it comes to derivatives.

\[ \begin{aligned} (e^x)' &= e^x \\ \\ (ln(x))' &= \frac{1}{x}, & \forall x>0 \end{aligned} \]

Now we’ll get to a couple of more advanced (and powerful) rules.

• Product rule: \((f(x)g(x))'=f'(x)g(x) + g'(x)f(x)\)

• Quotient rule: \(\displaystyle(\frac{f(x)}{g(x)})' = \frac{f'(x)g(x) + g'(x)f(x)}{[g(x)]^2}\)

• Chain rule: \((f(g(x))' = f'(g(x)) \cdot g'(x)\)

2.2 Higher-order derivatives

We saw call \(f'(x)\) the first derivative, but we can also calculate higher-order derivatives. These are not common in social science data analysis, but they appear in game theory, complex systems, and agent-based modeling.

For example, the second derivative tells us whether the slope of a function is increasing, decreasing, or staying the same at any point \(x\) on the function’s domain. For example, when driving a car:

• \(f(x)\) = distance traveled at time \(x\)
• \(f'(x)\) = speed at time \(x\)
• \(f''(x)\) = acceleration at time \(x\)

To calculate higher-order derivatives, you just calculate the derivative using the usual differentiation rules, then calculate the derivative of the output, and so on for however many orders you have. In a college-level calculus course, these can get very crazy during your midterm. In practice, you will rarely lead anything beyond a second derivative.

For example, we can calculate the following second derivative:

\[f''(x) = \frac{d^2(x^4)}{dx^2}\]

First, we take the first derivative \[f'(x)=4x^3\]

Then we use that output to take the second derivative \[f''(x)=f'(4x^3)=12x^2\]

We can keep going… for example, the third derivative is \[f'''(x) = f'(12x^2) = 24x\]

2.3 Partial derivatives

So far, we have worked with derivatives of functions with only one variable. What happens when we have more than one variable that is changing?

Suppose we have a function \(f\) of two (or more) variables and we want to determine the rate of change relative to one of the variables. For a function \(f(x,z)\), we call this a partial derivative

\[ \frac{\partial}{\partial_x}f(x,z) = \frac{\partial_y}{\partial_x} = \partial_x f \]

To obtain a partial derivative, we treat all other variables as constants and take the derivative with respect to the variable of interest (here \(x\)). For example:

\[ \begin{aligned} y = f(x,z) &= xz \\ \frac{\partial_y}{\partial_x} &= z \end{aligned} \]

You can try to do the same for \(\displaystyle\frac{\partial_y}{\partial_z}?\) on your own

Practice

Take the partial derivative with respect to \(x\) and with respect to \(z\) of the following functions. What would the notation for each look like?

1. \(y = 3xz - x\)
2. \(x^3+z^3+x^4z^4\)
3. \(e^{xz}\)

2.4 Differentiability of functions

Not all functions are differentiable at every point of their domains!

An important concept here is whether functions are continuous at a point:

• Informally: A function is continuous at a point if its graph has no holes or breaks at that point

• Formally: A function is continuous at a point \(a\) if: \(\lim_{x \to a} f(x)=f(a)\)

When is a function differentiable at a point?

• If a function is differentiable at a point, it is also continuous at that point.

• If a function is continuous at a point, it is not necessarily differentiable at that point.

  • Impossible to calculate derivative at sharp turns, cusps, or vertical tangents.

ggplot() +
  stat_function(fun = function(x){abs(x) + 2}, xlim = c(-4, 4), 
                aes(color = "y = |x| + 2")) +
  stat_function(fun = function(x){sqrt(abs(x)) + 1}, xlim = c(-4, 4), 
                aes(color = "y = √(|x|) + 1")) +
  stat_function(fun = function(x){sign(x) * abs(x)^(1 / 3)}, xlim = c(-4, 4), 
                aes(color = "y = x^(1/3)")) +
  scale_colour_manual("Function", values = c("red", "blue", "black")) +
  labs(title = "Examples of functions that are not differentiable at x=0")

Informally, functions need to be continuous and reasonably smooth to be differentiable.