Sunday, February 28, 2021
  • Setup menu at Appearance » Menus and assign menu to Top Bar Navigation
Advertisement
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News
No Result
View All Result
NikolaNews
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News
No Result
View All Result
NikolaNews
No Result
View All Result
Home Data Science

A Strange Family of Statistical Distributions

August 24, 2019
in Data Science
A Strange Family of Statistical Distributions
586
SHARES
3.3k
VIEWS
Share on FacebookShare on Twitter

I introduce here a family of very peculiar statistical distributions governed by two parameters: p, a real number in [0, 1], and b, an integer > 1. These distributions were discovered by solving the following functional equation, corresponding to b = 2. 

You might also like

The Education Industrial Complex: The Hammer We Have

Increasing Adoption of Informatics will Promote Growth of Data Analytics Outsourcing Market

The Ethereum Virtual Machine (EVM)

Here f(x) is the density attached to that distribution. The support domain for x is also [0, 1]. This type of distribution appears in the following context. 

Let Z be an irrational number in [0, 1] (called seed) and consider the sequence x(n) = {b^n Z}. Here the brackets represent the fractional part function. In particular, INT(b x(n)) is the n-th digit of Z in base b. The values x(n) are distributed in a certain way due to the ergodicity of the underlying process. The density associated with this distribution is the function f, and for the immense majority of seeds Z, that density is uniform on [0, 1]. Seeds producing the uniform density are sometimes called normal numbers; their digit distribution is also uniform.

However, the functional equation 2f(x) = f(x/2) + f((1+x)/2) may have plenty of other solutions. Such solutions are called non-standard solutions. The set of seeds producing non-standard solutions is known to have Lebesgue measure zero, but there are infinitely many such seeds. All rational seeds are, but they produce a discrete distribution. Thus their density is of the discrete type. We are interested here in a non-discrete solution. 

1. Example with p = 0.75 and b = 2

The uniform distribution corresponds to p = 0.5. Below is a non-standard density satisfying the requirements. Actually, the plot below represents its percentile distribution. It was produced with a seed Z in [0,1] built as follows: the n-th binary digit of Z is 1 if Rand(n)  <  p, and 0 otherwise, using a pseudo random number generator. Here p = 0.75. Note that P.25 = 0.5 and corresponds to a dip in the chart below (P.25 denotes the 25-th percentile.) Dips are everywhere, only the big ones are visible. By contrast, the percentile distribution for the uniform (standard) case p = 0.5 is a straight line, with no dips.

2. General solution

The functional equation is a bit more complicated if b is not equal to 2. It becomes

Using the construction mechanism outlined in the previous section to generate a non-standard seed Z (sometimes called a non-normal number or bad seed), it is clear that x(n) is a random variable. We also havewhere b is the base and d(n+k) is the (n+k)-th digit of the seed Z in base b.  This formula is very useful for computations. Note that Z = x(0). Furthermore, by construction, these digits are identically and independently distributed with a Bernouilli distribution of parameter p. Thus, using the convolution theorem, the characteristic function for the seed Z is

Take the derivative of the inverse Fourier transform (see section inverse formula here) and you obtain

If p = 0.5 and b = 2 we are back to the uniform case. Otherwise the solution is quite special: the density f is nowhere continuous it seems. See picture below for p = 0.55 and b = 2.

Now we should prove that this case is ergodic, for the functional equation to apply. I also tried to check with some sampled values of x to see whether 2f(x) = f(x/2) + f((1+x)/2), but the function being discontinuous everywhere, and since I got its value approximated probably to no more than two decimals, it is not easy.

3. Applications, properties and data

The distribution attached to this type of density has the following moments:

  • Expectation: p / (b – 1).
  • Variance: p(1 – p) / (b^2 – 1).

Why does f(x) must satisfy the functional equation discussed above? This a consequence of the fact that the underlying distribution is the equilibrium distribution for the sequence x(n) = {b x(n-1) } = {b^n Z}. In particular, the equilibrium distribution is solution to some stochastic integral equation P(X < x) = P({b X}  <  x).  For details, see my book Applied Stochastic Processes, Chaos Modeling, and Probabilistic Properties of Numeration Systems available here, see pages 65-66.

Potential applications are found in cryptography, Fintech (stock market modeling), Bitcoin, number theory, random number generation, benchmarking statistical tests (see here) and even gaming (see here.) However, the most interesting application is probably to gain insights about how non-normal numbers look like, especially their chaotic nature. It is a fundamental tool to help solve one of the most intriguing mathematical conjectures of all times (yet unsolved): are the digits of standard constants such as Pi or SQRT(2) uniformly distributed or not?

The charts featured here, as well as the underlying computations, were all produced in Excel. You can download the spreadsheet here. In particular, a very efficient algorithm is used to produce (say) one million digits of Z, and to compute one million successive values of x(n) each with a precision of 14 decimals. You can play interactively with the parameters b and p in the spreadsheet, and even try non-integer values of b (I suggest you try b = 1.5 and p = 0.5). If b  < 2 is not an integer, the functional equation is more complicated: it is found in section 2.1 in this article. 


Credit: Data Science Central By: Vincent Granville

Previous Post

Crypto Exchange OKEx Pounds the Table on 34% Bitcoin Price Surge

Next Post

Hacker Ordered to Pay Back Nearly £1 Million to Phishing Victims

Related Posts

The Education Industrial Complex: The Hammer We Have
Data Science

The Education Industrial Complex: The Hammer We Have

February 27, 2021
Increasing Adoption of Informatics will Promote Growth of Data Analytics Outsourcing Market
Data Science

Increasing Adoption of Informatics will Promote Growth of Data Analytics Outsourcing Market

February 27, 2021
The Ethereum Virtual Machine (EVM)
Data Science

The Ethereum Virtual Machine (EVM)

February 27, 2021
Levels of Measurement (Nominal, Ordinal, Interval, Ratio) in Statistics
Data Science

Levels of Measurement (Nominal, Ordinal, Interval, Ratio) in Statistics

February 27, 2021
Give Your Business Users Simple Augmented Analytics
Data Science

Give Your Business Users Simple Augmented Analytics

February 26, 2021
Next Post
Hacker Ordered to Pay Back Nearly £1 Million to Phishing Victims

Hacker Ordered to Pay Back Nearly £1 Million to Phishing Victims

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Recommended

Plasticity in Deep Learning: Dynamic Adaptations for AI Self-Driving Cars

Plasticity in Deep Learning: Dynamic Adaptations for AI Self-Driving Cars

January 6, 2019
Microsoft, Google Use Artificial Intelligence to Fight Hackers

Microsoft, Google Use Artificial Intelligence to Fight Hackers

January 6, 2019

Categories

  • Artificial Intelligence
  • Big Data
  • Blockchain
  • Crypto News
  • Data Science
  • Digital Marketing
  • Internet Privacy
  • Internet Security
  • Learn to Code
  • Machine Learning
  • Marketing Technology
  • Neural Networks
  • Technology Companies

Don't miss it

New AI Machine Learning Reduces Mental Health Misdiagnosis
Machine Learning

Machine Learning May Reduce Mental Health Misdiagnosis

February 28, 2021
Why would you ever trust Amazon’s Alexa after this?
Internet Security

Why would you ever trust Amazon’s Alexa after this?

February 28, 2021
AI & ML Are Not Same. Here's Why – Analytics India Magazine
Machine Learning

AI & ML Are Not Same. Here's Why – Analytics India Magazine

February 27, 2021
Microsoft: We’ve open-sourced this tool we used to hunt for code by SolarWinds hackers
Internet Security

Microsoft: We’ve open-sourced this tool we used to hunt for code by SolarWinds hackers

February 27, 2021
Is Wattpad and its machine learning tool the future of TV? — Quartz
Machine Learning

Is Wattpad and its machine learning tool the future of TV? — Quartz

February 27, 2021
Oxford University lab with COVID-19 research links targeted by hackers
Internet Security

Oxford University lab with COVID-19 research links targeted by hackers

February 27, 2021
NikolaNews

NikolaNews.com is an online News Portal which aims to share news about blockchain, AI, Big Data, and Data Privacy and more!

What’s New Here?

  • Machine Learning May Reduce Mental Health Misdiagnosis February 28, 2021
  • Why would you ever trust Amazon’s Alexa after this? February 28, 2021
  • AI & ML Are Not Same. Here's Why – Analytics India Magazine February 27, 2021
  • Microsoft: We’ve open-sourced this tool we used to hunt for code by SolarWinds hackers February 27, 2021

Subscribe to get more!

© 2019 NikolaNews.com - Global Tech Updates

No Result
View All Result
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News

© 2019 NikolaNews.com - Global Tech Updates