statistics — Mathematical statistics functions¶
This module provides functions for calculating mathematical statistics of numeric ( Real -valued) data.
The module is not intended to be a competitor to third-party libraries such as NumPy, SciPy, or proprietary full-featured statistics packages aimed at professional statisticians such as Minitab, SAS and Matlab. It is aimed at the level of graphing and scientific calculators.
Unless explicitly noted, these functions support int , float , Decimal and Fraction . Behaviour with other types (whether in the numeric tower or not) is currently unsupported. Collections with a mix of types are also undefined and implementation-dependent. If your input data consists of mixed types, you may be able to use map() to ensure a consistent result, for example: map(float, input_data) .
Some datasets use NaN (not a number) values to represent missing data. Since NaNs have unusual comparison semantics, they cause surprising or undefined behaviors in the statistics functions that sort data or that count occurrences. The functions affected are median() , median_low() , median_high() , median_grouped() , mode() , multimode() , and quantiles() . The NaN values should be stripped before calling these functions:
Averages and measures of central location¶
These functions calculate an average or typical value from a population or sample.
Arithmetic mean (“average”) of data.
Fast, floating point arithmetic mean, with optional weighting.
Geometric mean of data.
Harmonic mean of data.
Median (middle value) of data.
Low median of data.
High median of data.
Median, or 50th percentile, of grouped data.
Single mode (most common value) of discrete or nominal data.
List of modes (most common values) of discrete or nominal data.
Divide data into intervals with equal probability.
Measures of spread¶
These functions calculate a measure of how much the population or sample tends to deviate from the typical or average values.
Population standard deviation of data.
Population variance of data.
Sample standard deviation of data.
Sample variance of data.
Statistics for relations between two inputs¶
These functions calculate statistics regarding relations between two inputs.
Sample covariance for two variables.
Pearson’s correlation coefficient for two variables.
Slope and intercept for simple linear regression.
Function details¶
Note: The functions do not require the data given to them to be sorted. However, for reading convenience, most of the examples show sorted sequences.
statistics. mean ( data ) ¶
Return the sample arithmetic mean of data which can be a sequence or iterable.
The arithmetic mean is the sum of the data divided by the number of data points. It is commonly called “the average”, although it is only one of many different mathematical averages. It is a measure of the central location of the data.
If data is empty, StatisticsError will be raised.
Some examples of use:
The mean is strongly affected by outliers and is not necessarily a typical example of the data points. For a more robust, although less efficient, measure of central tendency, see median() .
The sample mean gives an unbiased estimate of the true population mean, so that when taken on average over all the possible samples, mean(sample) converges on the true mean of the entire population. If data represents the entire population rather than a sample, then mean(data) is equivalent to calculating the true population mean μ.
Convert data to floats and compute the arithmetic mean.
This runs faster than the mean() function and it always returns a float . The data may be a sequence or iterable. If the input dataset is empty, raises a StatisticsError .
Optional weighting is supported. For example, a professor assigns a grade for a course by weighting quizzes at 20%, homework at 20%, a midterm exam at 30%, and a final exam at 30%:
If weights is supplied, it must be the same length as the data or a ValueError will be raised.
New in version 3.8.
Changed in version 3.11: Added support for weights.
Convert data to floats and compute the geometric mean.
The geometric mean indicates the central tendency or typical value of the data using the product of the values (as opposed to the arithmetic mean which uses their sum).
Raises a StatisticsError if the input dataset is empty, if it contains a zero, or if it contains a negative value. The data may be a sequence or iterable.
No special efforts are made to achieve exact results. (However, this may change in the future.)
New in version 3.8.
Return the harmonic mean of data, a sequence or iterable of real-valued numbers. If weights is omitted or None, then equal weighting is assumed.
The harmonic mean is the reciprocal of the arithmetic mean() of the reciprocals of the data. For example, the harmonic mean of three values a, b and c will be equivalent to 3/(1/a + 1/b + 1/c) . If one of the values is zero, the result will be zero.
The harmonic mean is a type of average, a measure of the central location of the data. It is often appropriate when averaging ratios or rates, for example speeds.
Suppose a car travels 10 km at 40 km/hr, then another 10 km at 60 km/hr. What is the average speed?
Suppose a car travels 40 km/hr for 5 km, and when traffic clears, speeds-up to 60 km/hr for the remaining 30 km of the journey. What is the average speed?
StatisticsError is raised if data is empty, any element is less than zero, or if the weighted sum isn’t positive.
The current algorithm has an early-out when it encounters a zero in the input. This means that the subsequent inputs are not tested for validity. (This behavior may change in the future.)
New in version 3.6.
Changed in version 3.10: Added support for weights.
Return the median (middle value) of numeric data, using the common “mean of middle two” method. If data is empty, StatisticsError is raised. data can be a sequence or iterable.
The median is a robust measure of central location and is less affected by the presence of outliers. When the number of data points is odd, the middle data point is returned:
When the number of data points is even, the median is interpolated by taking the average of the two middle values:
This is suited for when your data is discrete, and you don’t mind that the median may not be an actual data point.
If the data is ordinal (supports order operations) but not numeric (doesn’t support addition), consider using median_low() or median_high() instead.
statistics. median_low ( data ) ¶
Return the low median of numeric data. If data is empty, StatisticsError is raised. data can be a sequence or iterable.
The low median is always a member of the data set. When the number of data points is odd, the middle value is returned. When it is even, the smaller of the two middle values is returned.
Use the low median when your data are discrete and you prefer the median to be an actual data point rather than interpolated.
statistics. median_high ( data ) ¶
Return the high median of data. If data is empty, StatisticsError is raised. data can be a sequence or iterable.
The high median is always a member of the data set. When the number of data points is odd, the middle value is returned. When it is even, the larger of the two middle values is returned.
Use the high median when your data are discrete and you prefer the median to be an actual data point rather than interpolated.
statistics. median_grouped ( data , interval = 1 ) ¶
Return the median of grouped continuous data, calculated as the 50th percentile, using interpolation. If data is empty, StatisticsError is raised. data can be a sequence or iterable.
In the following example, the data are rounded, so that each value represents the midpoint of data classes, e.g. 1 is the midpoint of the class 0.5–1.5, 2 is the midpoint of 1.5–2.5, 3 is the midpoint of 2.5–3.5, etc. With the data given, the middle value falls somewhere in the class 3.5–4.5, and interpolation is used to estimate it:
Optional argument interval represents the class interval, and defaults to 1. Changing the class interval naturally will change the interpolation:
This function does not check whether the data points are at least interval apart.
CPython implementation detail: Under some circumstances, median_grouped() may coerce data points to floats. This behaviour is likely to change in the future.
“Statistics for the Behavioral Sciences”, Frederick J Gravetter and Larry B Wallnau (8th Edition).
The SSMEDIAN function in the Gnome Gnumeric spreadsheet, including this discussion.
Return the single most common data point from discrete or nominal data. The mode (when it exists) is the most typical value and serves as a measure of central location.
If there are multiple modes with the same frequency, returns the first one encountered in the data. If the smallest or largest of those is desired instead, use min(multimode(data)) or max(multimode(data)) . If the input data is empty, StatisticsError is raised.
mode assumes discrete data and returns a single value. This is the standard treatment of the mode as commonly taught in schools:
The mode is unique in that it is the only statistic in this package that also applies to nominal (non-numeric) data:
Changed in version 3.8: Now handles multimodal datasets by returning the first mode encountered. Formerly, it raised StatisticsError when more than one mode was found.
Return a list of the most frequently occurring values in the order they were first encountered in the data. Will return more than one result if there are multiple modes or an empty list if the data is empty:
New in version 3.8.
Return the population standard deviation (the square root of the population variance). See pvariance() for arguments and other details.
Return the population variance of data, a non-empty sequence or iterable of real-valued numbers. Variance, or second moment about the mean, is a measure of the variability (spread or dispersion) of data. A large variance indicates that the data is spread out; a small variance indicates it is clustered closely around the mean.
If the optional second argument mu is given, it is typically the mean of the data. It can also be used to compute the second moment around a point that is not the mean. If it is missing or None (the default), the arithmetic mean is automatically calculated.
Use this function to calculate the variance from the entire population. To estimate the variance from a sample, the variance() function is usually a better choice.
If you have already calculated the mean of your data, you can pass it as the optional second argument mu to avoid recalculation:
Decimals and Fractions are supported:
When called with the entire population, this gives the population variance σ². When called on a sample instead, this is the biased sample variance s², also known as variance with N degrees of freedom.
If you somehow know the true population mean μ, you may use this function to calculate the variance of a sample, giving the known population mean as the second argument. Provided the data points are a random sample of the population, the result will be an unbiased estimate of the population variance.
Return the sample standard deviation (the square root of the sample variance). See variance() for arguments and other details.
Return the sample variance of data, an iterable of at least two real-valued numbers. Variance, or second moment about the mean, is a measure of the variability (spread or dispersion) of data. A large variance indicates that the data is spread out; a small variance indicates it is clustered closely around the mean.
If the optional second argument xbar is given, it should be the mean of data. If it is missing or None (the default), the mean is automatically calculated.
Use this function when your data is a sample from a population. To calculate the variance from the entire population, see pvariance() .
Raises StatisticsError if data has fewer than two values.
If you have already calculated the mean of your data, you can pass it as the optional second argument xbar to avoid recalculation:
This function does not attempt to verify that you have passed the actual mean as xbar. Using arbitrary values for xbar can lead to invalid or impossible results.
Decimal and Fraction values are supported:
This is the sample variance s² with Bessel’s correction, also known as variance with N-1 degrees of freedom. Provided that the data points are representative (e.g. independent and identically distributed), the result should be an unbiased estimate of the true population variance.
If you somehow know the actual population mean μ you should pass it to the pvariance() function as the mu parameter to get the variance of a sample.
Divide data into n continuous intervals with equal probability. Returns a list of n — 1 cut points separating the intervals.
Set n to 4 for quartiles (the default). Set n to 10 for deciles. Set n to 100 for percentiles which gives the 99 cuts points that separate data into 100 equal sized groups. Raises StatisticsError if n is not least 1.
The data can be any iterable containing sample data. For meaningful results, the number of data points in data should be larger than n. Raises StatisticsError if there are not at least two data points.
The cut points are linearly interpolated from the two nearest data points. For example, if a cut point falls one-third of the distance between two sample values, 100 and 112 , the cut-point will evaluate to 104 .
The method for computing quantiles can be varied depending on whether the data includes or excludes the lowest and highest possible values from the population.
The default method is “exclusive” and is used for data sampled from a population that can have more extreme values than found in the samples. The portion of the population falling below the i-th of m sorted data points is computed as i / (m + 1) . Given nine sample values, the method sorts them and assigns the following percentiles: 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, 90%.
Setting the method to “inclusive” is used for describing population data or for samples that are known to include the most extreme values from the population. The minimum value in data is treated as the 0th percentile and the maximum value is treated as the 100th percentile. The portion of the population falling below the i-th of m sorted data points is computed as (i — 1) / (m — 1) . Given 11 sample values, the method sorts them and assigns the following percentiles: 0%, 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, 90%, 100%.
New in version 3.8.
Return the sample covariance of two inputs x and y. Covariance is a measure of the joint variability of two inputs.
Both inputs must be of the same length (no less than two), otherwise StatisticsError is raised.
New in version 3.10.
Return the Pearson’s correlation coefficient for two inputs. Pearson’s correlation coefficient r takes values between -1 and +1. It measures the strength and direction of the linear relationship, where +1 means very strong, positive linear relationship, -1 very strong, negative linear relationship, and 0 no linear relationship.
Both inputs must be of the same length (no less than two), and need not to be constant, otherwise StatisticsError is raised.
New in version 3.10.
Return the slope and intercept of simple linear regression parameters estimated using ordinary least squares. Simple linear regression describes the relationship between an independent variable x and a dependent variable y in terms of this linear function:
y = slope * x + intercept + noise
where slope and intercept are the regression parameters that are estimated, and noise represents the variability of the data that was not explained by the linear regression (it is equal to the difference between predicted and actual values of the dependent variable).
Both inputs must be of the same length (no less than two), and the independent variable x cannot be constant; otherwise a StatisticsError is raised.
For example, we can use the release dates of the Monty Python films to predict the cumulative number of Monty Python films that would have been produced by 2019 assuming that they had kept the pace.
If proportional is true, the independent variable x and the dependent variable y are assumed to be directly proportional. The data is fit to a line passing through the origin. Since the intercept will always be 0.0, the underlying linear function simplifies to:
y = slope * x + noise
New in version 3.10.
Changed in version 3.11: Added support for proportional.
Exceptions¶
A single exception is defined:
exception statistics. StatisticsError ¶
Subclass of ValueError for statistics-related exceptions.
NormalDist objects¶
NormalDist is a tool for creating and manipulating normal distributions of a random variable. It is a class that treats the mean and standard deviation of data measurements as a single entity.
Normal distributions arise from the Central Limit Theorem and have a wide range of applications in statistics.
class statistics. NormalDist ( mu = 0.0 , sigma = 1.0 ) ¶
Returns a new NormalDist object where mu represents the arithmetic mean and sigma represents the standard deviation.
A read-only property for the arithmetic mean of a normal distribution.
A read-only property for the median of a normal distribution.
A read-only property for the mode of a normal distribution.
A read-only property for the standard deviation of a normal distribution.
A read-only property for the variance of a normal distribution. Equal to the square of the standard deviation.
classmethod from_samples ( data ) ¶
Makes a normal distribution instance with mu and sigma parameters estimated from the data using fmean() and stdev() .
The data can be any iterable and should consist of values that can be converted to type float . If data does not contain at least two elements, raises StatisticsError because it takes at least one point to estimate a central value and at least two points to estimate dispersion.
Generates n random samples for a given mean and standard deviation. Returns a list of float values.
If seed is given, creates a new instance of the underlying random number generator. This is useful for creating reproducible results, even in a multi-threading context.
Using a probability density function (pdf), compute the relative likelihood that a random variable X will be near the given value x. Mathematically, it is the limit of the ratio P(x <= X < x+dx) / dx as dx approaches zero.
The relative likelihood is computed as the probability of a sample occurring in a narrow range divided by the width of the range (hence the word “density”). Since the likelihood is relative to other points, its value can be greater than 1.0 .
Using a cumulative distribution function (cdf), compute the probability that a random variable X will be less than or equal to x. Mathematically, it is written P(X <= x) .
Compute the inverse cumulative distribution function, also known as the quantile function or the percent-point function. Mathematically, it is written x : P(X <= x) = p .
Finds the value x of the random variable X such that the probability of the variable being less than or equal to that value equals the given probability p.
Measures the agreement between two normal probability distributions. Returns a value between 0.0 and 1.0 giving the overlapping area for the two probability density functions.
Divide the normal distribution into n continuous intervals with equal probability. Returns a list of (n — 1) cut points separating the intervals.
Set n to 4 for quartiles (the default). Set n to 10 for deciles. Set n to 100 for percentiles which gives the 99 cuts points that separate the normal distribution into 100 equal sized groups.
Compute the Standard Score describing x in terms of the number of standard deviations above or below the mean of the normal distribution: (x — mean) / stdev .
New in version 3.9.
Instances of NormalDist support addition, subtraction, multiplication and division by a constant. These operations are used for translation and scaling. For example:
Dividing a constant by an instance of NormalDist is not supported because the result wouldn’t be normally distributed.
Since normal distributions arise from additive effects of independent variables, it is possible to add and subtract two independent normally distributed random variables represented as instances of NormalDist . For example:
New in version 3.8.
NormalDist Examples and Recipes¶
NormalDist readily solves classic probability problems.
For example, given historical data for SAT exams showing that scores are normally distributed with a mean of 1060 and a standard deviation of 195, determine the percentage of students with test scores between 1100 and 1200, after rounding to the nearest whole number:
Find the quartiles and deciles for the SAT scores:
To estimate the distribution for a model than isn’t easy to solve analytically, NormalDist can generate input samples for a Monte Carlo simulation:
Normal distributions can be used to approximate Binomial distributions when the sample size is large and when the probability of a successful trial is near 50%.
For example, an open source conference has 750 attendees and two rooms with a 500 person capacity. There is a talk about Python and another about Ruby. In previous conferences, 65% of the attendees preferred to listen to Python talks. Assuming the population preferences haven’t changed, what is the probability that the Python room will stay within its capacity limits?
Normal distributions commonly arise in machine learning problems.
Wikipedia has a nice example of a Naive Bayesian Classifier. The challenge is to predict a person’s gender from measurements of normally distributed features including height, weight, and foot size.
We’re given a training dataset with measurements for eight people. The measurements are assumed to be normally distributed, so we summarize the data with NormalDist :
Next, we encounter a new person whose feature measurements are known but whose gender is unknown:
Starting with a 50% prior probability of being male or female, we compute the posterior as the prior times the product of likelihoods for the feature measurements given the gender:
The final prediction goes to the largest posterior. This is known as the maximum a posteriori or MAP:
Rukovodstvo
статьи и идеи для разработчиков программного обеспечения и веб-разработчиков.
Вычисление среднего, медианы и моды в Python
Введение Когда мы пытаемся описать и обобщить выборку данных, мы, вероятно, начинаем с нахождения среднего [https://en.wikipedia.org/wiki/Mean] (или среднего), медианы [https: // en .wikipedia.org / wiki / Median] и режим [https://en.wikipedia.org/wiki/Mode_(statistics)] данных. Это центральная тенденция [https://en.wikipedia.org/wiki/Central_tendency] меры и часто первый взгляд на набор данных. В этом руководстве мы узнаем, как найти или вычислить среднее значение, медиану,
Время чтения: 9 мин.
Вступление
Когда мы пытаемся описать и обобщить выборку данных, мы, вероятно, начинаем с нахождения среднего (или среднего), медианы и режима данных. Это основные меры тенденций, которые часто являются нашим первым взглядом на набор данных.
В этом руководстве мы узнаем, как найти или вычислить среднее значение, медиану и режим в Python. Сначала мы закодируем функцию Python для каждой меры, а затем воспользуемся statistics Python для выполнения той же задачи.
Обладая этими знаниями, мы сможем быстро взглянуть на наши наборы данных и получить представление об общей тенденции данных.
Оглавление
Расчет среднего значения выборки
Если у нас есть выборка числовых значений, то ее среднее или среднее
- это общая сумма значений (или наблюдений), деленная на количество значений.
Допустим, у нас есть образец [4, 8, 6, 5, 3, 2, 8, 9, 2, 5] . Мы можем вычислить его среднее значение, выполнив операцию:
(4 + 8 + 6 + 5 + 3 + 2 + 8 + 9 + 2 + 5) / 10 = 5,2
Среднее арифметическое — это общее описание наших данных. Предположим, вы купили 10 фунтов помидоров. Если пересчитать дома помидоры, получится 25 помидоров. В этом случае вы можете сказать, что средний вес помидора составляет 0,4 фунта. Это было бы хорошее описание ваших помидоров.
Среднее также может быть плохим описанием выборки данных. Допустим, вы анализируете группу собак. Если вы возьмете совокупный вес всех собак и разделите его на количество собак, то это, вероятно, будет плохим описанием веса отдельной собаки, поскольку разные породы собак могут иметь очень разные размеры и вес.
Насколько хорошо или плохо среднее значение описывает выборку, зависит от того, насколько разбросаны данные. В случае помидоров, они почти одинакового веса, и среднее значение является хорошим их описанием. В случае с собаками нет актуальных собак. Они могут варьироваться от крошечного чихуахуа до гигантского немецкого мастифа. Итак, среднее само по себе в данном случае не очень хорошее описание.
Теперь пора приступить к делу и узнать, как вычислить среднее значение с помощью Python.
Расчет среднего с помощью Python
Чтобы вычислить среднее значение выборки числовых данных, мы будем использовать две встроенные функции Python. Один для вычисления общей суммы значений, а другой для вычисления длины выборки.
Первая функция — это sum() . Эта встроенная функция принимает итерацию числовых значений и возвращает их общую сумму.
Вторая функция — len() . Эта встроенная функция возвращает длину объекта. len() может принимать в качестве аргумента последовательности (строка, байты, кортеж, список или диапазон) или коллекции (словарь, набор или замороженный набор).
Вот как мы можем вычислить среднее значение:
Сначала мы суммируем значения в sample используя sum() . Затем мы делим эту сумму на длину sample , которая является результирующим значением len(sample) .
Использование Python mean ()
Поскольку вычисление среднего — это обычная операция, Python включает эту функцию в модуль statistics Он предоставляет некоторые функции для расчета базовой статистики по наборам данных. Функция statistics.mean() берет образец числовых данных (любых итерируемых) и возвращает их среднее значение.
Вот как работает функция mean() Python:
Нам просто нужно импортировать statistics а затем вызвать mean() с нашим образцом в качестве аргумента. Это вернет среднее значение выборки. Это быстрый способ найти среднее значение с помощью Python.
Нахождение медианы выборки
Медиана выборки числовых данных — это значение, которое находится посередине при сортировке данных. Данные могут быть отсортированы по возрастанию или убыванию, медиана остается прежней.
Чтобы найти медиану, нам необходимо:
-
образец
- Найдите значение в середине отсортированного образца
При нахождении числа в центре отсортированной выборки мы можем столкнуться с двумя типами ситуаций:
- Если в выборке есть нечетное количество наблюдений , то среднее значение в отсортированной выборке — это медиана.
- Если в выборке есть четное количество наблюдений , нам нужно вычислить среднее из двух средних значений в отсортированной выборке.
Если у нас есть выборка [3, 5, 1, 4, 2] и мы хотим найти ее медиану, то сначала мы сортируем выборку по [1, 2, 3, 4, 5] . Медиана будет равна 3 поскольку это значение посередине.
С другой стороны, если у нас есть выборка [1, 2, 3, 4, 5, 6] , то ее медиана будет (3 + 4) / 2 = 3.5 .
Давайте посмотрим, как мы можем использовать Python для вычисления медианы.
Поиск медианы с помощью Python
Чтобы найти медиану, нам сначала нужно отсортировать значения в нашей выборке . Этого можно добиться с помощью встроенной функции sorted() sorted() принимает итерацию и возвращает отсортированный list содержащий те же значения, что и исходная итерация.
Второй шаг — найти значение, которое находится в середине отсортированной выборки. Чтобы найти это значение в выборке с нечетным количеством наблюдений, мы можем разделить количество наблюдений на 2. Результатом будет индекс значения в середине отсортированной выборки.
Поскольку оператор деления ( / ) возвращает число с плавающей запятой, нам нужно использовать оператор деления этажа ( // ), чтобы получить целое число. Итак, мы можем использовать его как индекс в операции индексации ( [] ).
Если в выборке есть четное количество наблюдений, нам нужно найти два средних значения. Скажем, у нас есть образец [1, 2, 3, 4, 5, 6] . Если мы разделим его длину ( 6 ) на 2 с помощью деления пола, то получим 3 . Это индекс нашего верхнего среднего значения ( 4 ). Чтобы найти индекс нашего нижнего среднего значения ( 3 ), мы можем уменьшить индекс верхнего среднего значения на 1 .
Давайте объединим все это в функцию, которая вычисляет медиану выборки. Вот возможная реализация:
Эта функция берет образец числовых значений и возвращает их медиану. Сначала мы находим длину образца n . Затем мы вычисляем индекс среднего значения (или верхнего среднего значения) путем деления n на 2 .
Оператор if проверяет, есть ли в имеющейся выборке нечетное количество наблюдений. Если да, то медиана — это значение index .
Окончательный return выполняется, если в выборке есть четное количество наблюдений. В этом случае мы находим медиану, вычисляя среднее из двух средних значений.
Обратите внимание, что операция нарезки [index — 1:index + 1] получает два значения. Значение в index — 1 и значение в index поскольку операции нарезки исключают значение в конечном индексе ( index + 1 ).
Использование медианы Python ()
Функция Python statistics.median() берет выборку данных и возвращает ее медиану. Вот как работает метод:
Обратите внимание, что median() автоматически обрабатывает вычисление медианы для выборок с нечетным или четным числом наблюдений.
Поиск режима образца
Режим — это наиболее частое наблюдение (или наблюдения) в выборке. Если у нас есть образец [4, 1, 2, 2, 3, 5] , то его режим равен 2 потому что 2 появляется в образце два раза, тогда как другие элементы появляются только один раз.
Режим не обязательно должен быть уникальным. Некоторые образцы имеют более одного режима. Скажем, у нас есть образец [4, 1, 2, 2, 3, 5, 4] . В этом примере есть два режима — 2 и 4 потому что эти значения появляются чаще и оба появляются одинаковое количество раз.
Этот режим обычно используется для категориальных данных. Распространенными категориальными типами данных являются:
- логическое значение — может принимать только два значения, например true или false , male или female
- номинальный — может принимать более двух значений, например, American — European — Asian — African
- порядковый — может принимать более двух значений, но значения имеют логический порядок, например, few — some — many
Когда мы анализируем набор категориальных данных, мы можем использовать этот режим, чтобы узнать, какая категория является наиболее распространенной в наших данных.
Мы можем найти образцы, у которых нет режима. Если все наблюдения уникальны (нет повторяющихся наблюдений), то в вашей выборке не будет режима.
Теперь, когда мы знаем основы режима, давайте посмотрим, как его найти с помощью Python.
Поиск режима с помощью Python
Чтобы найти режим с помощью Python, мы начнем с подсчета количества вхождений каждого значения в рассматриваемом примере. Затем мы получим значения с большим количеством вхождений.
Поскольку подсчет объектов — обычная операция, Python предоставляет класс collections.Counter Этот класс специально разработан для подсчета предметов.
Класс Counter предоставляет метод, определенный как .most_common([n]) . Этот метод возвращает list кортежей из двух элементов с n более общими элементами и их соответствующими счетчиками. Если n опущено или None , то .most_common() возвращает все элементы.
Давайте воспользуемся Counter и .most_common() чтобы закодировать функцию, которая берет образец данных и возвращает свой режим.
Вот возможная реализация:
Сначала мы подсчитываем наблюдения в sample с помощью объекта Counter c ). Затем мы используем составление списка, чтобы создать list содержащий наблюдения, которые встречаются в выборке одинаковое количество раз.
Поскольку .most_common(1) возвращает list с одним tuple формы (observation, count) , нам нужно получить наблюдение с индексом 0 в list а затем элемент с индексом 1 во вложенном tuple . Это можно сделать с помощью выражения c.most_common(1)[0][1] . Это значение является первым режимом нашего образца.
Обратите внимание, что условие понимания сравнивает счетчик каждого наблюдения ( v ) со счетчиком наиболее распространенного наблюдения ( c.most_common(1)[0][1] ). Это позволит нам получить несколько наблюдений ( k ) с одним и тем же подсчетом в случае многомодовой выборки.
Использование режима Python ()
Python statistics.mode() принимает некоторые data и возвращает свой (первый) режим. Посмотрим, как это можно использовать:
В одномодовом примере функция Python mode() возвращает наиболее распространенное значение 2 . Однако в следующих двух примерах он вернул 4 и few . В этих образцах были другие элементы, встречающиеся такое же количество раз, но они не были включены.
Начиная с Python 3.8 мы также можем использовать statistics.multimode() который принимает итерацию и возвращает list режимов.
Вот пример использования multimode() :
Примечание . Функция всегда возвращает list , даже если вы передаете одномодовый образец.
Заключение
Среднее (или среднее), медиана и мода обычно являются нашим первым взглядом на выборку данных, когда мы пытаемся понять центральную тенденцию данных.
В этом руководстве мы узнали, как найти или вычислить среднее значение, медиану и режим с помощью Python. Сначала мы пошагово рассмотрели, как создавать наши собственные функции для их вычисления, а затем как использовать statistics Python как быстрый способ найти эти показатели.
What is Data?
It is a collection of facts such as numbers, words, experimental outcomes, sometimes a description of things, millions of images/videos on the internet, thousands of books, etc. Data is everywhere around us and to study, analyze and make decisions with this data we need statistics.
What is Statistics?
It is a branch of applied mathematics that deals with collecting and analyzing large numerical datasets to get some qualitative or quantitative information to make conclusions regarding the datasets.
Statistics is mainly of two types: Descriptive statistics and Inferential statistics
Descriptive statistics:
It describes and summarizes the data in a meaningful way. It gives the central tendency of the data. The most common descriptive statistics which we use to measure the central tendency of data are: mean, median, and mode.
Inferential statistics:
It allows us to make inferences (“predictions”) from the data sample and we can generalize those predictions over the population. Hypothesis testings, model designing, etc. come under inferential statistics.
This post is regarding mean, median, and mode, so let’s dive into these topics
Mean: the average value of data
It is calculated by summing all the data points and then dividing them by the total number of data points, scenarios where we calculate the mean: what is the average number of hours an employee works in the office in a month, what is the average weight of a group of people of a society/community, etc.
Median: the center value of data
It is the middle number of sorted data, so to calculate the median of data we need to first sort the data into either increasing or decreasing order, after sorting the data the middle data point will be the median.
Note: If the number of data points is even then the median will be the average of the two middle numbers.
Mode: the highest occurring value of data
The data point with the highest occurrence frequency will be the mode of the given data. We can have multiple modes of a given data sample, scenarios where mode is required: what is the exam score value most students got, by what time most employees leave the office, etc.
Formula connecting mean, mode, and median: mode = 3 * median — 2 * mean
Relation among mean, median, and mode w.r.t. Frequency Distribution Curve (FDC)
- Symmetrical FDC: mean = mode = median
- Positive/Right skewed FDC: mean > median > mode
- Negative/Left skewed FDC: mean < median < mode
Python code
In python, a lot of packages are available for mathematical calculations, and we don’t need to go long way to calculate the mean, median, and mode:
I hope you learned something exciting today, will meet you guys in the next blog until then enjoy the journey of life and keep learning.
Calculating Mean, Median, and Mode in Python
When we're trying to describe and summarize a sample of data, we probably start by finding the mean (or average), the median, and the mode of the data. These are central tendency measures and are often our first look at a dataset.
In this tutorial, we'll learn how to find or compute the mean, the median, and the mode in Python. We'll first code a Python function for each measure followed by using Python's statistics module to accomplish the same task.
With this knowledge, we'll be able to take a quick look at our datasets and get an idea of the general tendency of data.
Calculating the Mean of a Sample
If we have a sample of numeric values, then its mean or the average is the total sum of the values (or observations) divided by the number of values.
Say we have the sample [4, 8, 6, 5, 3, 2, 8, 9, 2, 5] . We can calculate its mean by performing the operation:
(4 + 8 + 6 + 5 + 3 + 2 + 8 + 9 + 2 + 5) / 10 = 5.2
The mean (arithmetic mean) is a general description of our data. Suppose you buy 10 pounds of tomatoes. When you count the tomatoes at home, you get 25 tomatoes. In this case, you can say that the average weight of a tomato is 0.4 pounds. That would be a good description of your tomatoes.
The mean can also be a poor description of a sample of data. Say you're analyzing a group of dogs. If you take the cumulated weight of all dogs and divide it by the number of dogs, then that would probably be a poor description of the weight of an individual dog as different breeds of dogs can have vastly different sizes and weights.
How good or how bad the mean describes a sample depends on how spread the data is. In the case of tomatoes, they're almost the same weight each and the mean is a good description of them. In the case of dogs, there is no topical dog. They can range from a tiny Chihuahua to a giant German Mastiff. So, the mean by itself isn't a good description in this case.
Now it's time to get into action and learn how we can calculate the mean using Python.
Calculating the Mean With Python
To calculate the mean of a sample of numeric data, we'll use two of Python's built-in functions. One to calculate the total sum of the values and another to calculate the length of the sample.
The first function is sum() . This built-in function takes an iterable of numeric values and returns their total sum.
The second function is len() . This built-in function returns the length of an object. len() can take sequences (string, bytes, tuple, list, or range) or collections (dictionary, set, or frozen set) as an argument.
Here's how we can calculate the mean:
We first sum the values in sample using sum() . Then, we divide that sum by the length of sample , which is the resulting value of len(sample) .
Using Python's mean()
Since calculating the mean is a common operation, Python includes this functionality in the statistics module. It provides some functions for calculating basic statistics on sets of data. The statistics.mean() function takes a sample of numeric data (any iterable) and returns its mean.
Here's how Python's mean() works:
We just need to import the statistics module and then call mean() with our sample as an argument. That will return the mean of the sample. This is a quick way of finding the mean using Python.
Finding the Median of a Sample
The median of a sample of numeric data is the value that lies in the middle when we sort the data. The data may be sorted in ascending or descending order, the median remains the same.
To find the median, we need to:
-
the sample
- Locate the value in the middle of the sorted sample
When locating the number in the middle of a sorted sample, we can face two kinds of situations:
- If the sample has an odd number of observations, then the middle value in the sorted sample is the median
- If the sample has an even number of observations, then we'll need to calculate the mean of the two middle values in the sorted sample
If we have the sample [3, 5, 1, 4, 2] and want to find its median, then we first sort the sample to [1, 2, 3, 4, 5] . The median would be 3 since that's the value in the middle.
On the other hand, if we have the sample [1, 2, 3, 4, 5, 6] , then its median will be (3 + 4) / 2 = 3.5 .
Let's take a look at how we can use Python to calculate the median.
Finding the Median With Python
To find the median, we first need to sort the values in our sample. We can achieve that using the built-in sorted() function. sorted() takes an iterable and returns a sorted list containing the same values of the original iterable.
The second step is to locate the value that lies in the middle of the sorted sample. To locate that value in a sample with an odd number of observations, we can divide the number of observations by 2. The result will be the index of the value in the middle of the sorted sample.
Since a division operator ( / ) returns a float number, we'll need to use a floor division operator, ( // ) to get an integer. So, we can use it as an index in an indexing operation ( [] ).
If the sample has an even number of observations, then we need to locate the two middle values. Say we have the sample [1, 2, 3, 4, 5, 6] . If we divide its length ( 6 ) by 2 using a floor division, then we get 3 . That's the index of our upper-middle value ( 4 ). To find the index of our lower-middle value ( 3 ), we can decrement the index of the upper-middle value by 1 .
Free eBook: Git Essentials
Check out our hands-on, practical guide to learning Git, with best-practices, industry-accepted standards, and included cheat sheet. Stop Googling Git commands and actually learn it!
Let's put all these together in function that calculates the median of a sample. Here's a possible implementation:
This function takes a sample of numeric values and returns its median. We first find the length of the sample, n . Then, we calculate the index of the middle value (or upper-middle value) by dividing n by 2 .
The if statement checks if the sample at hand has an odd number of observations. If so, then the median is the value at index .
The final return runs if the sample has an even number of observations. In that case, we find the median by calculating the mean of the two middle values.
Note that the slicing operation [index — 1:index + 1] gets two values. The value at index — 1 and the value at index because slicing operations exclude the value at the final index ( index + 1 ).
Using Python's median()
Python's statistics.median() takes a sample of data and returns its median. Here's how the method works:
Note that median() automatically handles the calculation of the median for samples with either an odd or an even number of observations.
Finding the Mode of a Sample
The mode is the most frequent observation (or observations) in a sample. If we have the sample [4, 1, 2, 2, 3, 5] , then its mode is 2 because 2 appears two times in the sample whereas the other elements only appear once.
The mode doesn't have to be unique. Some samples have more than one mode. Say we have the sample [4, 1, 2, 2, 3, 5, 4] . This sample has two modes — 2 and 4 because they're the values that appear more often and both appear the same number of times.
The mode is commonly used for categorical data. Common categorical data types are:
- boolean — Can take only two values like in true or false , male or female
- nominal — Can take more than two values like in American — European — Asian — African
- ordinal — Can take more than two values but the values have a logical order like in few — some — many
When we're analyzing a dataset of categorical data, we can use the mode to know which category is the most common in our data.
We can find samples that don't have a mode. If all the observations are unique (there aren't repeated observations), then your sample won't have a mode.
Now that we know the basics about mode, let's take a look at how we can find it using Python.
Finding the Mode with Python
To find the mode with Python, we'll start by counting the number of occurrences of each value in the sample at hand. Then, we'll get the value(s) with a higher number of occurrences.
Since counting objects is a common operation, Python provides the collections.Counter class. This class is specially designed for counting objects.
The Counter class provides a method defined as .most_common([n]) . This method returns a list of two-items tuples with the n more common elements and their respective counts. If n is omitted or None , then .most_common() returns all of the elements.
Let's use Counter and .most_common() to code a function that takes a sample of data and returns its mode.
Here's a possible implementation:
We first count the observations in the sample using a Counter object ( c ). Then, we use a list comprehension to create a list containing the observations that appear the same number of times in the sample.
Since .most_common(1) returns a list with one tuple of the form (observation, count) , we need to get the observation at index 0 in the list and then the item at index 1 in the nested tuple . This can be done with the expression c.most_common(1)[0][1] . That value is the first mode of our sample.
Note that the comprehension's condition compares the count of each observation ( v ) with the count of the most common observation ( c.most_common(1)[0][1] ). This will allow us to get multiple observations ( k ) with the same count in the case of a multi-mode sample.
Using Python's mode()
Python's statistics.mode() takes some data and returns its (first) mode. Let's see how we can use it:
With a single-mode sample, Python's mode() returns the most common value, 2 . However, in the proceeding two examples, it returned 4 and few . These samples had other elements occurring the same number of times, but they weren't included.
Since Python 3.8 we can also use statistics.multimode() which accepts an iterable and returns a list of modes.
Here's an example of how to use multimode() :
Note: The function always returns a list , even if you pass a single-mode sample.
Conclusion
The mean (or average), the median, and the mode are commonly our first looks at a sample of data when we're trying to understand the central tendency of the data.
In this tutorial, we've learned how to find or compute the mean, the median, and the mode using Python. We first covered, step-by-step, how to create our own functions to compute them, and then how to use Python's statistics module as a quick way to find these measures.