Calculus in a Nutshell: Functions and their Derivatives

When I was about thirteen, the library was going to get ‘Calculus for the Practical Man.’ By this time I knew, from reading the encyclopedia, that calculus was an important and interesting subject, and I ought to learn it.  

Richard P. Feynman, from What Do You Care What Other People Think?


Calculus is an important branch of mathematics that deals with the methods for calculating derivatives and integrals of functions and using this information to study the properties of functions. It was independently invented by I. Newton and W. Leibniz in the 18{^{th}} century and it was further developed by other great mathematicians in the centuries that follows (see Figure below).

Figure 1: Some of the great mathematician that invented the Calculus.

It comprises two areas:

  • Differential calculus {\rightarrow} It concerns the study of the rate of variation of functions.
  • Integral calculus {\rightarrow} It concern the study of the area under functions.

Depending on the nature of the functions involved in the calculations, we can further distinguish between the single- and multi-variable calculus. In this chapter, the main concepts and methods of the single-variable calculus are summarised.


Definition 1: A single variable function {f} from a set {\mathcal{A}} to a set {\mathcal{B}}{y \rightarrow f(x)} is a rule that assigns a unique (single) element {f(x) \in \mathcal{B}} to each element {x \in \mathcal{A}} (see Fig. 1). An infinite number of single variable functions can be generated from the combinations of arithmetic, exponential, logarithmic and trigonometric expressions. In the following paragraphs, we are going to review functions that are relevant to the solution of differential equations.

Sinusoidal function. Functions containing trigonometric expressions (also called sinusoidal function, oscillation, or signal) are particularly useful functions. They are usually defined in the time domain as {f(t) = A \cos(\omega t - \phi)}. where the parameters ({A, \omega, \phi}) in the equations have the following meanings:

  • {A >0} is the amplitude and it gives how high the graph {f(t)} rises above the t-axis at its maximum values.
  • {\phi} is the phase lag and it indicates the position of the maximum of the function.
  • {\omega} is the angular frequency and it gives the number of complete oscillation of {f(t)} in a time interval of length {2\pi}.

The following useful quantities are associated with the previous parameters:

  • {\nu=\omega/2 \pi} is the frequency of {f(t)} and it represent the number of complete oscillation of {f(t)} in a time interval of length 1.
  • {\tau=\phi/\omega} is the time delay or time shift and it indicates how far the graph of the {\cos(\omega t)} has been shifted along the t-axis.
  • {\mathcal{P}=2\pi/\omega=1/\nu} is the period and indicates the time required for one complete oscillation.

An example of sinusoidal functions is shown in Fig. 2.

Figure 2: Exmples of sinusoidal functions. The perpendicular segments are the tangent ( magenta) and normal (cyan) to the sine curve at the point x=2/3\pi.

In Fig. 3, two periodic functions having the same set of parameters but with a phase shift of {\Delta \phi=\pi/6} are shown.

Figure 3: Example of phase shifted sinusoidal functions.

Exponential and natural logarithm functions

The exponential and natural logarithm} functions are also very useful since they solutions of many differential equations. In Fig. 4, the plots of the two functions are shown.

Figure 4: Example of exponential and logarithm functions.

The transcendental number (the transcendental numbers are real or complex numbers that are not algebraic that is, they are not a root of a non-zero polynomial equation with integer (or, equivalently, rational) coefficients) {e} is defined as:

\displaystyle e=\lim_{n\rightarrow \infty}\left( {1+\frac{1}{n}} \right)^n \qquad (1)

The exponential function {\exp(x) = e^x} and the natural logarithm {\ln(x)} are inverse functions therefore we can write

\begin{aligned} e^{\ln(x)} &= x \\ \ln(e^x) &= x \end{aligned}

Some of the properties of the exponential functions are summarised in the following list

  • {e^0 = 1}.
  • {e^{at+c} = e^ce^{at}}.
  • {e^{at}} is never 0.
  • If {a>0} then {lim_{t\rightarrow \infty} e^{at} =\infty} and {lim_{t \rightarrow -\infty}e^{at} =0}.
  • If {a<0} then {lim_{t \rightarrow \infty} e^{at} =0} and {lim_{t \rightarrow -\infty}e^{at} =\infty}.

For any positive {a}, {e^{at}} grows much faster than any polynomial.


{lim_{ t \rightarrow \infty} {e^{t}}/{t^3} = \infty};

{lim_{t \rightarrow \infty} te^{-t} = 0}.

For the exponential function, the usual rules of exponents apply

\begin{aligned} e^x/e^y &= e^{x-y} \\e^{x/y} &= (e^{x})^{1/y}(e^x)^p &=& e^{px} \end{aligned}

with the corresponding properties for the logarithmic function

\begin{aligned} \ln(x/y) &= \ln x-\ln y, \\ \ln(xy) &=& \ln x+ \ln y  \\ \ln x^p &= p\ln x.  \end{aligned}

Numerical approximation of functions

Complex functions can be approximated using a polynomial series called Taylor series. In this way, the function can be calculates using basic mathematical operations. To know more give a look to my blog.

The derivative of a function

Definition of the limit of a function

Suppose {f(x)} is defined on an open interval about {x_0}, except possibly at {x_0} itself. If {f(x)} is arbitrarily close to {\mathcal{L}} (as close to {\mathcal{L}} as we like) for all {x} sufficiently close to {x_0}, we say that {f} approaches the limit {L} as {x} approaches {x_0}, and write:

\lim_{x \rightarrow x_0} f(x) = \mathcal{L} \ \ \ \ \ (11)

The limit of a function provides us with the mathematical tool necessary for introducing the concept of the derivative of a function. The derivative of a function is defined as follows:

\displaystyle {\lim_{h \rightarrow 0} \frac{f(a+h)-f(a)}{h}=f'(x) } \ \ \ \ \ (12)

The graphical representation of the formula is reported in the Fig. 5. The derivative in the point {A} of the function {f(x)} is the slope of the tangent line to the curve passing for this point and it that approximate the function in the point {M} with the value at the point {M'}.

Fig. 5: The graphical representation of the derivation formula.

Differentiation rules

Using the definition of derivative as the limit of the incremental ratio of the ratio, it is possible also to define the properties of the differentiation operations fiven as follow in term of rules.

The sum or difference rule

The derivative of the sum of {f(x)} and {g(x)}

\displaystyle { (f + g)' = f' + g'} \ \ \ \ \ (13)

Similarly, the derivative of the difference is

\displaystyle { (f - g)' = f' - g'} \ \ \ \ \ (14)

The product rule

The derivative of the product of {f(x)} and {g(x)} is

\displaystyle {\frac{d(f(x)g(x))}{dx}=f'g+fg' } \ \ \ \ \ (15)

The quotient rule

The derivative of the quotient of {f(x)} and {g(x)} is

\displaystyle {\left(\frac{f(x)}{g(x)}\right)'= \frac{f(x)'g(x) - f(x)g(x)'}{(g(x))^2}} \ \ \ \ \ (16)

The chain rule

The derivative of the composition of {f(x)} and {g(x)} is

\displaystyle { \frac{df(g(x))}{dx} =f'(g(x))g'(x)} \ \ \ \ \ (17)

The power rule

The derivative of a power of {x} is given by

\displaystyle {\frac{d}{d x}x^p=px^{p-1}} \ \ \ \ \ (18)

Examples of derivatives of elementary functions

(\sin x)' = \cos x

(\cos x)' = -\sin x

(e^x)' = e^x

(\ln x)' = \frac{1}{x}

Numerical approximation of functions

The trigonometric, transcendent or radical functions cannot be directly evaluated using the basic mathematics operations, for this reason for their numerical calculation is necessary to calculate as an approximate polynomial series. This can be easily accomplished using the Taylor series that are described in another blog.

The numerical differentiation

Calculation of derivative is a common task normally performed in the calculation of physical model when the evaluation of, for example, molecular forces (see my blog on the molecular dynamics methods) is required. Although. in this case, the calculation of derivatives is preferable to perform analytically from the corresponding potential energy functions, the numerical derivation is an easier and faster algorithm. The drawback is the accuracy and the pitfalls due to numerical approximations as for its definition it is necessary to subtract two numbers from each other that can differ only by a small amount.

The derivative in the point x of a function is defined as:

\frac{df(x)}{dx}=\lim_{h\rightarrow0}\frac{f(x+h)-f(x)}{h} (19)

To calculate the numerical derivative as in the equation (19), we need first to discretize the derivative as the incremental ratio of single variable function  on a grid of points equally spaced at the interval :

\frac{df(x)}{dx}\approx \frac{f(x_{x+1})-f(x_i)}{x_{i+1}-x_i} (20)

The function f(x) is known on a equally spaced lattice of x values (see Figure 6) f(x_n) with $latex x_n=n h$ and n=0,\pm 1,\pm 2, \dots and h=x_{i+1}-x_i is the step size.

Figure 6: Values of the f(x)on an equally-spaced lattice. Dashed lines show the linear interpolation.

Depending the order and the point on the grid that we use for the approximate formula, we can use three simple different formulas to perform numerical derivation:

  • Forward finite divided difference
  • backward finite divided difference
  • Center finite divided difference

In the equation (20) correspond to the forward formula. The error in the estimation of the exact (analytical) derivative is proportional to the step size (h). In particular, in the forward (as in the backward difference method) the error is on the square of h. The backward formula is given by

\frac{df(x)}{dx}\approx \frac{f(x_{x-1})-f(x_i)}{x_{i-1}-x_i} (21)

Centered Difference Approximation of the First Derivative

A more accurate formula for the derivative is given by the so-called centered or symmetric difference approximation:

\frac{df(x)}{dx}\approx \frac{f(x_{x-1})-f(x_i)}{2(x_{i-1}-x_i)} (22)

Programming the finite difference approximations of the First Derivative of a function

The formulas (20), (21) and (22) are easily implemented in the programming language as a function. For example, an implementation is awk language is the following.

[code language="cpp"]

#* diff.awk
#* Copyright (C) 2018 daniloroccatano 

function f(x)
# Return the value of the equation in x 
    return (sin(x))

function derf(x)
# Return the value of the analytical derivative of the equation in f(x)
    return (cos(x))

function fp_forw(h,x)
# Return the value of the numerical derivative of f(x) calculated using 
# the forward formula
    return (f(x+h)-f(x))/h

function fp_back(h,x)
# Return the value of the numerical derivative of f(x) calculated using 
# the backward formula
    return (f(x)-f(x-h))/h

function fp_sym(h,x)
# Return the value of the numerical derivative of f(x) calculated using 
# the symmetric formula
    return (f(x+h)-f(x-h))/(2*h)


#  Calculate and print the table with the difference between the numerical 
#  and analytical value of the derivative 
    print "---------------------------------------------------------"
    print "      h        Symmetric        Forward         Backward " 
    print "               3-points         2-point         2-point  " 
    print "---------------------------------------------------------" 

#   calculate the differences 
    diff= (fp_sym(h,x)- derf(x))
    diff<0?diff1= (-diff):diff1=diff
    diff= (fp_forw(h,x)- derf(x))
    diff<0?diff2= (-diff):diff2=diff
    diff= (fp_back(h,x)- derf(x))
    diff<0?diff3= (-diff):diff3=diff
    printf "%10.5f   %10.5f      %10.5f      %10.5f\n", h, diff1, diff2, diff3

Higher derivatives

A better estimation of first derivative can be obtained by considering more points. In this way, the derivative can be calculated from the polynomial approximation of the function among the different point. 4-and 5- points approximation can be derived considering the following Taylor expansions: 

Higher derivative formulas can be constructed by taking the appropriate combination of the Taylor expansion for h and 2h intervals. In following Table, 4- and 5-point formulas for derivatives up to the 4thorder are shown.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.