kth quantiles of an $n$-element set

Problem statement

Exercise 9.3-8

The $k$th quantiles of an $n$-element set are the $k$ - 1 order statistics that divide the sorted set into $k$ equal sized sets (to within 1). Give an $O(n \cdot lg k)$-time algorithm to list the $k$th quantiles of a set. [1]


Definitions

In statistics and probability, quantiles are cut points dividing the range of a probability distribution into continuous intervals with equal probabilities, or dividing the observations in a sample in the same way. There is one fewer quantile than the number of groups created. [2]



Approach

Let's write a divide and conquer algorithm using selection to solve this problem. What if we find the $l = \lfloor\frac{n \cdot \lfloor\frac{k}{2}\rfloor}{k}\rfloor$ th smallest number  and partition the array around it. Then, we can find the first $\lfloor\frac{k}{2}\rfloor$ cut points in the first half $l$ and the remaining $\lceil\frac{k}{2}\rceil$ cut points in the second half $n - l$. At each step, we keep on dividing $k$ by two until it reaches $k$ = 1 which is the trivial case of our recursion. We first solve the left subproblem recursively, and then print the current solution before solving the right subproblem recursively. What follows is our pseudocode for finding the $k$th quantiles of an $n$-element set. A complete treatment of the procedure RANDOMIZED-SELECT used in our algorithm and proof of it's running time is beyond the scope of this article, but the interested reader is directed to the CLRS book [1].


$FIND-QUANTILES(s, k)$
$\phantom{{}++{}}$ let $n \gets s.length$
$\phantom{{}++{}}$ $FIND-QUANTILES-AUX(s, 1, n, k)$


$FIND-QUANTILES-AUX(s, i, j, k)$
$\phantom{{}++{}}$ // trivial case of recursion
$\phantom{{}++{}}$ `if` $k == 1$
$\phantom{{}++{}}$ $\phantom{{}++{}}$  `return`

$\phantom{{}++{}}$ let $n \gets j - i + 1$
$\phantom{{}++{}}$ let $k_1 \gets \lfloor\frac{k}{2}\rfloor$
$\phantom{{}++{}}$ let $l \gets \lfloor n \cdot \frac{k_1}{k} \rfloor$
$\phantom{{}++{}}$ let $p \gets RANDOMIZED-SELECT(s, i, j, l)$
$\phantom{{}++{}}$ $FIND-QUANTILES-AUX(s, i, i + l - 1, k_1)$
$\phantom{{}++{}}$ `print` $p$
$\phantom{{}++{}}$ $FIND-QUANTILES-AUX(s, i + l, j, k - k_1)$


Sample Implementation

Here's a sample Java implementation of the algorithms discussed above along with a driver program. 


Informal Proof for running time

We have $log_2 k + 1$ levels in our tree, each level except the leaf costs us $O(n)$ time for partitioning all the subarrays at that level. At the leaf level, no partitioning is performed and we merely return when k = 1, which is the base case of our recursion, doing some constant amount of work for each subproblem of size $\lfloor\frac{n}{k}\rfloor$ or $\lceil\frac{n}{k}\rceil$. Thus, the total running time of this approach is $n \cdot log_2 k + k \approx n \cdot log_2 k$.


Formal Proof for running time

Let's write the recurrence first. 

$T(n, k) = T(n_1, \lfloor\frac{k}{2}\rfloor) + T(n_2, \lceil\frac{k}{2}\rceil) + \Theta(n)$

We let $T(n, k) \le cn \cdot log_2 k$ for some constant $c \gt 0$

Let's solve the recurrence using the substitution method.

$T(n, k) \le cn_1 \cdot log_2 (\lfloor\frac{k}{2}\rfloor) + cn_2 \cdot log_2(\lceil\frac{k}{2}\rceil) + \Theta(n)$

$T(n, k) \le cn_1 \cdot log_2 (\frac{k}{2}) + cn_2 \cdot log_2(\frac{k}{2}) + \Theta(n)$

$T(n, k) \le c \cdot log_2 (\frac{k}{2}) \cdot (n_1 + n_2) + \Theta(n)$

$T(n, k) = cn \cdot log_2 (\frac{k}{2}) + \Theta(n)$

$T(n, k) = cn \cdot log_2 k - cn + \Theta(n)$

$T(n, k) \le cn \cdot log_2 k$

where the last step holds when we have some constant $c \gt 0$ for which the term $cn$ dominates the anonymous function hidden behind the term $\Theta(n).$ Thus, the running time is given by

$T(n, k) = O(n \cdot log_2 k)$

and that concludes the proof.


References

[1] https://www.amazon.com/Introduction-Algorithms-fourth-Thomas-Cormen/dp/026204630X

[2] https://en.wikipedia.org/wiki/Quantile

Comments

Post a Comment

Popular posts from this blog

Introducing Java Reactive Extentions in to a SpringBoot Micro Service

Image
Introduction Today we are going to take a look at Reactive Programming using Java8 and RxJava, which is a library for composing asynchronous and event-based programs by using observable sequences. RxJava stands for Reactive Extensions for the JVM. It extends the observer pattern [Gamma95] to support sequences of data/events and adds operators that allow you to compose sequences together declaratively while abstracting away concerns about things like low-level threading, synchronization, thread-safety and concurrent data structures [1]. Pre-Requisites : Having a good knowledge about Java8, Functional Programming and spring boot microservices. What is Reactive programming? In essence, Reactive programming is an asynchronous programming paradigm concerned with data streams. The system time, a list of entries from the DB/API, user clicks, everything can be a stream of data. Data from different streams can easily be combined and transformed, and in the end processed/observed by the
Read more

Optimal binary search trees

Image
Suppose that we are designing a program to translate text from English to French. For each occurrence of each English word in the text, we need to look up its French equivalent. We could perform these lookup operations by building a binary search tree with $n$ English words as keys and their French equivalents as satellite data. Because we will search the tree for each individual word in the text, we want the total time spent searching to be as low as possible. We could ensure an $Θ(log_2 \;n)$ search time per occurrence by using a red-black tree or any other balanced binary search tree. Words appear with different frequencies, however, and a frequently used word such as the may appear far from the root while a rarely used word appears near the root. Such an organization would slow down the translation, since the number of nodes visited when searching for a key in a binary search tree equals one plus the depth of the node containing the key. We want words that occur frequently in the t
Read more

Edit distance

Image
Biological applications often need to compare the DNA of two (or more) different organisms. A strand of DNA consists of a string of molecules called bases, where the possible bases are adenine, guanine, cytosine, and thymine. Representing each of these bases by its initial letter, we can express a strand of DNA as a string over the finite set $\{A, C, G, T\}$. For example, the DNA of one organism may be $S1 = ACCGGTCGAGTGCGCGGAAGCCGGCCGAA$, and the DNA of another organism may be $S2 = GTCGTTCGGAATGCCGTTGCTCTGTAAA$. One reason to compare two strands of DNA is to determine how “similar” the two strands are, as some measure of how closely related the two organisms are. We can, and do, define similarity in many different ways. For example, we can say that two strands are similar if the number of changes needed to turn one into the other is small. [1] We formalize this notion of similarity in our Edit distance problem. 15-5 Edit distance - Problem Statement In order to transform one sour
Read more