Which of the following shows the number of units the market will buy in a given time period

Poisson Distribution

The Poisson distribution is a discrete distribution that measures the probability of a given number of events happening in a specified time period. In finance, the Poisson distribution could be used to model the arrival of new buy or sell orders entered into the market or the expected arrival of orders at specified trading venues or dark pools. In these cases, the Poisson distribution is used to provide expectations surrounding confidence bounds around the expected order arrival rates. Poisson distributions are very useful for smart order routers and algorithmic trading.

Poisson Distribution Statistics1

Poisson Distribution Graph

D.2 Poisson distribution

Poisson distribution represents the distribution of Poisson processes and is in fact a limiting case of the binomial distribution. By Poisson processes, we mean processes that are discrete, independent, and mutually exclusive.

The p.d.f. of a Poisson distribution is defined as

(9.3.31)f(x;μ)= μxe−μx!,

where x=0, 1, … represents the discrete random variable, such as ADC counts recorded by a detection system, and μ>0 is the mean. Figure 9.3.3 depicts this distribution for different values of μ. It is apparent that the width of the distribution increases with μ, which indicates that the uncertainty in measurement increases with an increase in the value of x.

Let us now apply the maximum likelihood method to determine the best estimate of the mean of a set of n measurements, assuming that the underlying process is Poisson in nature. The best way to do this is to use the maximum likelihood method we outlined earlier and applied in the previous section when discussing the binomial distribution. Since Poisson distribution is a discrete probability distribution, its likelihood function for a set of n measurements can be written as

(9.3.32)L(μ)=∏ i=1nf(xi,μ)=∏i=1n[μxi e−μxi!]=μ∑xie−nμ x1!x2!…xn!.

The log-likelihood function of L(μ) is

(9.3.33)l≡ln(L)=(∑i=1nxi)ln(μ)−nμ−ln(x1!x2!…xn!).

Following the maximum likelihood method (∂l/∂μ=0), we get

(9.3.34)∂∂μ[(∑ i=1nxi)ln(μ)−nμ−ln (x1!x2!…xn!)]=01μ*∑i=1 nxi−n=0μ*=1n∑i=1nxi.

This shows that the simple mean is the most probable value of a Poisson distributed variable. To determine the error in μ, we first take the second derivative of the log-likelihood function and then substitute it in Eq. (9.3.22):

(9.3.35)∂ 2l∂μ2=−1μ∑i=1n xiΔμ=[−∂2l∂μ2]−1/2=[−μ*2∑ i=1nxi]−1/2 =1n[∑i=1nxi]−1/2.

This is one of the most useful results of the Poisson distribution. It implies that if we make one measurement, the statistical error we should expect would simply be the square root of the measured quantity. For example, if we count the number of γ-ray photons coming from a radioactive source using a GM tube and get a number N, the statistical error we should expect will simply be N. Fortunately, most of the processes we encounter in the field of radiation detection and measurement, such as the activity of a radioisotope, photoelectric effect, and electron multiplication in a PMT tube, can all be very well described by Poisson statistics.

5.2.1 Poisson Probabilities

The Poisson distribution arises from either of two models. In one model — quantities, for example — bacteria are assumed to be distributed at random in some medium with a uniform density of λ(lambda) per unit area. The number of bacteria colonies found in a sample area of size A follows the Poisson distribution with a parameter μ equal to the product of λ and A.

In terms of the model over time, we assume that the probability of one event in a short interval of length t1 is proportional to t1 — that is, Pr{exactly one event} is approximately λt1. Another assumption is that t1 is so short that the probability of more than one event during this interval is almost zero. We also assume that what happens in one time interval is independent of the happenings in another interval. Finally, we assume that λ is constant over time. Given these assumptions, the number of occurrences of the event in a time interval of length t follows the Poisson distribution with parameter μ, where μ is the product of λ and t.

The Poisson probability mass function is

Pr(X-x)=e-μμxx!forx=0,1,2……

where e is a constant approximately equal to 2.71828 and μ is the parameter of the Poisson distribution. Usually μ is unknown and we must estimate it from the sample data. Before considering an example, we shall demonstrate in Table 5.3 the use of the probability mass function for the Poisson distribution to calculate the probabilities when μ = 1 and μ = 2. These probabilities are not difficult to calculate, particularly when μ is an integer. There is also a recursive relation between the probability that X = x + 1 and the probability that X = x that simplifies the calculations:

Table 5.3. Calculation of poisson probabilities, Pr{X = x} = e−μ μx/x!, for μ = 1 and 2.

Pr{X=x+1}=(μx+1 )Pr(X=x)

for x beginning at a value of 0. For example, for μ = 2,

Pr{X=3} =(2/3)Pr(X=2)=(2/3) 0.2707=0.1804

which is the value shown in Table 5.3.

These probabilities are also found in Appendix Table B3, which gives the Poisson probabilities for values of μ beginning at 0.2 and increasing in increments of 0.2 up to 2.0, then in increments of 0.5 up to 7, and in increments of 1 up to 17. Computer software can provide the Poisson probabilities for other values of μ (see Program Note 5.1 on the website). Note that the Poisson distribution is totally determined by specifying the value of its one parameter, μ. The plots in Figure 5.4 show the shape of the Poisson probability mass and cumulative distribution functions with μ = 2.

The shape of the Poisson probability mass function with μ equal to 2 (the top plot in Figure 5.4) is similar to the binomial mass function for a sample of size 10 and π equal to 0.2 just shown. The cdf (the bottom plot in Figure 5.4) has the same general shape as that shown in the preceding binomial example, but the shape is easier to see here, since there are more values for the X variable shown on the horizontal axis.

The Poisson Distribution

The Poisson distribution, like the binomial, is a counted number of times something happens. The difference is that there is no specified number n of possible tries. Here is one way that it can arise. If an event happens independently and randomly over time and the mean rate of occurrence is constant over time, then the number of occurrences in a fixed amount of time will follow the Poisson distribution.13 The Poisson is a discrete distribution (because you can list the possibilities as 0, 1, 2, 3, …) and depends only on the mean number of occurrences expected.

Here are some random variables that might follow a Poisson distribution:

The number of orders your firm receives tomorrow.

The number of people who apply for a job tomorrow to your human resources division.

The number of defects in a finished product.

The number of calls your firm receives next week for help concerning an “easy-to-assemble” toy.

A binomial number X when n is large and π is small.

The following figures show what the Poisson probabilities look like for a system expecting a mean of 0.5 occurrence (Fig. 7.5.1), two occurrences (Fig. 7.5.2), and 20 occurrences (Fig. 7.5.3). Note from the bell shape of Fig. 7.5.3 that the Poisson distribution is approximately normal when many occurrences are expected.

There are three important facts about a Poisson distribution. These facts, taken together, tell you how to find probabilities for a Poisson distribution when you know only its mean.

For a Poisson Distribution

The standard deviation is always equal to the square root of the mean: σ=μ.

The exact probability that a Poisson random variable X with mean μ is equal to a is given by the formula

P(X=a) =μaa!e−μ

where e = 2.71828… is a special number.14

If the mean is large, then the Poisson distribution is approximately normal.

Example

How Many Warranty Returns?

Because your firm’s quality is so high, you expect only 1.3 of your products to be returned, on average, each day for warranty repairs. What are the chances that no products will be returned tomorrow? That one will be returned? How about two? How about three?

Because the mean (1.3) is so small, exact calculations are needed. Here are the details:

P(X=0)=1.3 00!e−1.3=11×0.27253=0.27253P(X=1)=1.311!e−1.3 =1.31×0.27253=0.35429P(X=2) =1.322!e−1.3=1.692×0.27253=0.23029P(X=3)=1.333! e−1.3=2.1976×0.27253=0.09979

From these basic probabilities, you could add up the appropriate probabilities for 0, 1, and 2 to also find the probability that two items or fewer will be returned. The probability is, then, 0.27253 + 0.35429 + 0.23029 = 0.857, or 85.7%.

To use Excel to compute these probabilities, you could use the function “=POISSON (value, mean, FALSE)” to find the probability that a Poisson random variable is exactly equal to some value, and you could use “=POISSON (value, mean, TRUE)” to find the probability that a Poisson random variable is less than or equal to the value. Here are the results:

Example

2 Poisson regression modeling

The Poisson distribution is commonly used to model rate of random events that occur (arrive) in some fixed time interval. If we assume that λ denotes the rate of arrivals, then the distribution of the random variable Y, which denotes the number of arrivals in a fixed time interval, follows the Poisson distribution with probability mass function

(1)P[Y=y]=exp(−λ)λyy!,y=0,1,2 ,…

It is an elementary exercise to show that the mean and variance of Y are both equal to λ; E[Y] = Var[Y] = λ. In fact, this property characterizes the Poisson distribution. A related property is that the cumulant generating function of a Poisson random variable is given by KY(t)≡logMY(t)=λ( exp(t)−1), where MY (t) is the moment generating function of Y. This can be proved by simple calculations, but for this presentation, it is instructive to consider the Poisson distribution as a member of the natural exponential family of distributions.

Let f (x; θ) denote the density function of the natural exponential family with parameter θ, i.e.,

(2)f(x;θ)=h( x)exp(θx−b(θ)),x∈A,

where h(⋅), b(⋅) are known functions and A is a subset of R. Then, it is straightforward to show that the Poisson distribution is expressed as in (2) with θ = log λ, b(θ) = exp(θ), and h(x) = 1/x!. Using the fact that the cumulant generating function of (2) is equal to b(t + θ) − b(θ), the claim follows.

In most of the applications, count data are usually observed with some covariate information. For example, see the works of McCullagh and Nelder (1989, Section 6.3.2) where the authors study the relation between the type of ship, its year of construction, and its service period to the expected number of damage incidents using the logarithm of the aggregate months of service as an offset. (An offset is a continuous regression variable with corresponding known regression coefficient equal to 1.) In general, assume that X1, …, Xp are p regression variables observed jointly with a count response variable Y that follows the Poisson distribution. A possible regression model for association between the regressors and the expected value of Y given X1, …, Xp is

(3)λ=β0+∑i=1pβiXi.

This is an ordinary linear model with unknown regression coefficients βi, i = 0, …, p to be estimated. Model (3) poses several difficulties for fitting, because the parameter λ has to be positive. Nevertheless, in the context of time series and when the correlation among successive observations is positive, models such as (3) are quite useful; recall Fig. 1. A more natural choice for the regression modeling of count data is the so called log-linear model which is specified by

(4)logλ= β0+∑i=1pβiX i,

where the notation is as in (3). Regardless of the chosen model, a fact that remains true is that both (3) and (4) belong to the class of generalized linear models as introduced by Nelder and Wedderburn (1972) and elaborated further by McCullagh and Nelder (1989). Recall, that a generalized linear model consists of three components; the random component that belongs to the exponential family of distributions (2) with E[X] = μ, the systematic component η, and the link function g(⋅). The link function is a monotone twice differentiable function that is chosen by the user (or can be estimated). This function associates the random and systematic component via g(μ) = η. For the Poisson distribution, it is clear that both (3) and (4) introduce a generalized linear model with η=β0+∑i=1pβi Xi and g(λ) = λ (for (3)) and g(λ) = log λ (for (4)). Estimation and inference are based on the maximum likelihood theory – this topic has been described in several texts; see McCullagh and Nelder (1989) and Agresti (2002), for example. In the next section, we explore these ideas in the context of count time series.

Poisson distribution

The Poisson distribution for a variable λ is:

[23]P(k)=λke−λk!

for k = 0, 1, 2, 3, etc. The mean of this distribution is λ and the standard deviation is √λ. When the number n of trials is very large and the probability p small, e.g. n > 25 and p < 0.1, binomial probabilities are often approximated by the Poisson distribution. Thus, since the mean of the binomial distribution is np (equation [21]) and the standard deviation (equation [22]) approximates to √np when p is small, we can consider λ to represent np. Thus λ can be considered to represent the average number of successes per unit time or unit length or some other parameter.

Key point

Approximating the binomial distribution to the Poisson distribution

If p is the possibility of an event occurring and q the possibility that it does not occur, then q + p = 1 and so if we consider n samples, we have (q + p)n = 1 and the binomial expression gives:

(q +p)n=qn+nqn−1p+n(n−1)qn−2p22! +n(n−1)(n−2)qn−3 p33!+…

If p is small then q = 1 − p × 1 and with n large the first few terms have n − 1 approximating to n, n − 2 to n, etc. The binomial expression can thus be approximated to:

1n=1n+n1n−1p+nn1n−2p22!+nnn1n−3p3 3!+…1n=1+np+n2 p22!+n3p33!+…

If we let np = λ then:

1n=1+λ+λ22!+λ3 3!+…

But this is the series for e1 (see Table 7.1) and so 1n = e1. We can thus write the binomial expression as:

(q+p)n=e−λ[1+λ+λ22!+λ33!+…]

and so the terms in the expression are given by equation [23].

POISSON EVENTS DESCRIBED

The Poisson distribution arises from situations in which there is a large number of opportunities for the event under scrutiny to occur but a small chance that it will occur on any one trial. The number of cases of bubonic plague would follow Poisson: A large number of patients can be found with chills, fever, tender enlarged lymph nodes, and restless confusion, but the chance of the syndrome being plague is extremely small for any randomly chosen patient. This distribution is named for Siméon Denis Poisson, who published the theory in 1837. The classic use of Poisson was in predicting the number of deaths of Prussian army officers from horse kicks from 1875 to 1894; there was a large number of kicks, but the chance of death from a randomly chosen kick was small.

2.2.3 The Poisson Distribution

The Poisson distribution is another common discrete distribution that is defined as follows. The Poisson probability distribution for x occurrences of a phenomenon when the average number of occurrences equals λ is

(2.5)p(x;λ)=e−λλxx!forx=0,1,2,…and someλ>0=0forothervalues ofx

The Poisson distribution occurs in several important situations. It is the limit of the binomial distribution b(x;n, p) when one lets n→∞ and p→0 in such a way that np remains fixed at the value λ>0. In practice, the Poisson distribution is an adequate approximation to the binomial distribution if n≥100 and p≤0.05.

The Poisson distribution also occurs in a more fundamental context, as a distribution associated with a “Poisson process,” which is a process satisfying the following conditions:

The number of occurrences in an interval of time [t,t+Δt] is independent of the number of occurrences in any other interval that does not overlap with this interval.

For small Δt, the probability of success in the time interval is approximately proportional to Δt.

If the time interval Δt, is small enough, the probability of more than one occurrence in the interval is small compared to Δt.

For a Poisson process, the mean number of occurrences in an interval Δt can thus be expressed as γΔt, and 1/γ is the mean time between occurrences. (See Chapter 3 for more information on the Poisson process.)

An example of a Poisson process is the decay of radioactive nuclei in a sample containing a large number of these nuclei. Over 100 radioactive nuclei occur naturally (such as carbon-14, potassium-40, and uranium-238), plus many others are produced in laboratories. It has been observed, to a very high degree of accuracy, that every radioactive nucleus has a fixed probability of decaying in any particular time interval, and that this probability is independent of how long the nucleus has already been in existence. As an example, the probability that a uranium-238 nucleus will decay in 1 second is 4.87 × 10−18;; for one mole =6.022x1023 nuclei of uranium238, the mean number decaying in 1 second is thus 6.022 × 1023 × 4.87 × 10−18 = 2.93 × 106 nuclei.

The Poisson distribution has a particularly simple mean, E(X)=λ, and variance, V(X)=λ.

The Poisson cumulative distribution function does not have a simple form, though it can be easily calculated using a computer; it is

(2.6)P(X≤x)=∑v=0 rp(y;λ)forx=0,1,2…,n

Figure 5(a) shows the Poisson distributions for two values of λ, while Figure 5(b) shows the cumulative Poisson distributions for three values of λ.

21.13 The Poisson distribution

The Poisson distribution is used to model processes where the distribution of the number of incidents occurring in any interval depends only on the length of that interval. Examples of such systems are:

incoming telephone calls to an exchange during a busy period;

customers arriving at a checkout counter in a supermarket from 4 to 6p.m.;

accidents on a busy stretch of the M1;

number of misprints in a book.

When modelling situations in a Poisson process we use four assumptions:

If A is the event of n incidents in an interval and B the event of m incidents in another non-overlapping interval then A and B are independent, that is, p(A∩B) = p(A)p(B).

If A is the event of n incidents in an interval then P(A) depends only on the length of the interval– not on the starting point of the interval.

The probability of exactly one incident in a small interval is approximately proportional to the length of that interval, that is, P1(t)≈λtfor small t.

The probability of more than one incident in a small interval is negligible. Thus, for small t, P2 (t) ≈0 and we can also say that

It follows that P0(t) + P1(t) ≈ 1 and as by assumption (3), P1(t) ≈ λt we get P0(t) ≈ 1 – λt. We now think about the number of incidents in an interval of time of any given length, (0, t), where t is no longer small. We can divide the interval into pieces of length h, where h is small, and use the assumptions above. We can see that in each small interval of length h there is either no event or a single event. Therefore, each small interval is approximately behaving like a Bernoulli trial. This means that we can approximate the events in the interval (0, t) by using the Binomial distribution for the number of successes r in n trials. The probability of r incidents in n intervals, where the probability of an incident in any one interval is λh, is given by

Pr(t)=limh→0(nr)(λh)r(1−λh)n−r.

Substituting h = t / n gives

Pr(t)=limn→∞n!(n−r)!r! λrtrnr(1−λtn) n−r.

We can reorganize this expression, by taking out of the limit terms not involving n

Pr(t)= (λt)rr!limn→∞n!(n−r)!nr(1−λt n)n(1−λtn)−r.

We can rewrite the first term inside the limit to give

Pr(t)=(λt)r r!limn→∞nnn−1n⋯n−rn(1−λtn)n(1−λtn)−r.

Now we notice that the first term inside the limit is made up of the product of r fractional expressions, which each have a term in n on the top and bottom lines. These will all tend to 1 as n tends to ∞. The last term is similar to the limit that we saw in Chapter 7 when calculating the value of e. There, we showed that

limn→∞(1+1n)n=e

and by a similar argument we could show that

limn→∞(1+xn)n=ex.

It, therefore follows that

limn→∞(1+λtn)n=e −λt.

The last expression involves a negative power of (1− λt/n), which will tend to 1 as n tends to ∞.

This gives the Poisson distribution as

Pr(t)=(λt)rr!e−λt.

This is an expression in both r and t where r is the number of events and t is the length of the time interval being considered. We usually consider the probability of r events in an interval of unit time, which gives the Poisson distribution as

Pr=λrr!e−λ

where λ is the expected number of incidents in unit time.

13.3 Poisson Regression

The Poisson distribution is widely used as a model for count data. As discussed in Section 2.3, it is frequently appropriate when the counts are of events in specific regions of time or space. Dependent variables that might be modeled using the Poisson regression would include the

number of fatal auto accidents during a year at intersections as a function of lane width,

number of service interruptions during a month for a network server as a function of usage, and

number of fire ant colonies in an acre of land as a function of tree density.

There is no fixed upper limit on the possible number of events. Recalling the properties of the Poisson, there is a single parameter, μ , which is the expected number of events. It is essential that μ be positive, and the regression function must enforce this.

Case Study 13.1

Warner (2007) studied the proportions of robberies where a gun was used as a function of the characteristics of the neighborhoods where the robberies occurred. Using logistic regression, the author related the probability the robbery would involve a gun to several independent variables:

Disadvantaged, a blend of poverty level, % female-headed households, and other economic variables, with higher scores indicating poorer neighborhoods

Percent population that are young black males

Faith in police, a score obtained by surveying residents of the neighborhood, with high scores indicating a greater trust of police

Perceived oppositional values, a score obtained by surveying residents of the neighborhood, with high scores indicating more opposition to mainstream attitudes toward drugs and crime

Table 13.5 shows the results of the bivariate (dependent variable against a single independent variable) logistic regressions. Also shown are the means (M) and standard deviations (SD) for the independent variables.

Table 13.5. Results of logistic regressions for gun use in robberies.

Disadvantaged shows a significant positive association; that is, the more disadvantaged a neighborhood, the greater the probability a robbery will involve a gun. Faith in police shows a significant negative association; that is, the greater the neighborhood’s faith in police, the less the probability the robbery will involve a gun. Neither of the other independent variables showed a significant association.

The information regarding the standard deviations of the independent variable is important in helping understand each variable’s impact. Disadvantaged has SD=1.0, so that a moderately low score might be 1 point below the mean and a high score might be 1 point above the mean. Moving from a moderately low score to a moderately high score on Disadvantaged would shift the odds ratio by exp(2×1×0.33)=1.93. By contrast, Faith in police, which has a much larger coefficient, only has SD= 0.17. Moving from a moderately high score to a moderately low score on Faith in police would shift the odds by exp(2×.17×3.37)=3.14. Thus, the difference in impact of the two variables is not as large as we would think if we only examined the coefficients. Note that we have described a shift from high to low for Faith in police, which changed the sign on the coefficient. This made the comparison of the two independent variables easier.

Poisson regression assumes each yi follows a Poisson distribution with mean μi, where

ln(μi)=β0+β1x1+β2x2+⋯+βmxm.

The linear expression may take on either positive or negative values, but μi=exp(β0+β1x1+β2x2+⋯+βmx m) will always be positive. Note that the link function is the logarithmic function. The proper method of fitting this model is via ML. Likelihood ratio tests replace F tests, and Wald χ2 tests will replace t tests.

Example 13.3

Hypoglycemic Incidents

A hospital tracks the number of hypoglycemic incidents among diabetic patients recovering from cardiovascular surgery. The dependent variable is the number of incidents experienced by a patient during the first 72 hours post-surgery. The research question is whether a patient’s age can be related to the frequency of incidents. An artificial data set illustrating this situation is given in Table 13.6.

Table 13.6. Number of incidents of hypoglycemia.

Sometimes each observation is a count from a region that varies greatly in size. For example, we might have y=number of flaws in a Mylar sheet, but some sheets are quite large and others are small. In this situation, size is an important part of the expected count. The independent variables are assumed to influence the rate per unit of size, denoted λ. The rate must be positive. Given a set of independent variables x1,x2,…,xm , we model

ln(λ)=β0+β1x1+β2x2+⋯+βmx m.

If observation yi comes from an observational unit with size si, then y i has the Poisson distribution with expected value μi=λisi and

ln(μi)=β0+ β1x1+β2x2+⋯+βmxm+ln(si).

At first glance, the term ln(si) may seem like just another independent variable in the Poisson regression. However, its coefficient is identically 1, so that no parameter need be estimated for it. This is called an offset variable, and all Poisson regression software will allow you to indicate such a size marker. Sometimes size is only specified up to a constant of proportionality. That is, we might not know exactly the size of units i and i′, but we know that unit i is twice the size of unit i′. This suffices, as the unknown proportionality constant will become an additive constant once logarithms are computed, and be combined with the intercept β0.

Example 13.4

Occupational Fatality Rates

Bailer et al. (1997) published an article showing how Poisson regression could be an important tool in safety research. Table 13.8 shows their counts of fatalities in the agriculture, forestry, and fishing industries and estimates of the number of workers in those industries. Figure 13.4 graphs the rates per 1,000 workers (number of fatalities×1,000/number of workers). We would like to see that fatality rates are declining, but is there any evidence that this is so?

Table 13.8. Fatalities and number of workers.