Establishing a lower bound for searching

Today, we are going to establish a lower bound for searching and prove that binary search is optimal, and no comparison based search in this universe would be better than that by more than a constant factor. Let’s define our problem more formally first.

Theorem

Let us assume that we have $n$ pre-processed items in the array. Finding a given item among them in a comparison model requires $\Omega(log_2 n)$ in the worst case.

Computation Model

Let’s try to model the situation using a decision tree. In this decision tree, an internal node represents a decision that our algorithm makes and a leaf represents an answer. Here’s our representation of the problem using a decision tree computation model for $n = 3$ element array for the sake of simplicity.



I mentioned here that the Items are preprocessed to mean you could do whatever you want with items ahead of time. For instance, I can sort them, build them into an AVL tree et.al.

Proof

Let $n$ be the number of items in the input array. Then, we have at least $n + 1$ leaves in our binary tree. Since a binary tree of height $h$ has no more than $2^h$ leaves, we have

$n + 1 \le 2^h$


By taking logarithms on both sides we then obtain:

$log_2 (n + 1) \le h = \;$worst case running time


$\Omega(log_2 n) = h = \;$worst case running time

Corollary

Binary search is asymptotically optimal comparison search.

References

[1] https://www.amazon.com/Introduction-Algorithms-fourth-Thomas-Cormen/dp/026204630X

Comments

Post a Comment

Popular posts from this blog

Introducing Java Reactive Extentions in to a SpringBoot Micro Service

Image
Introduction Today we are going to take a look at Reactive Programming using Java8 and RxJava, which is a library for composing asynchronous and event-based programs by using observable sequences. RxJava stands for Reactive Extensions for the JVM. It extends the observer pattern [Gamma95] to support sequences of data/events and adds operators that allow you to compose sequences together declaratively while abstracting away concerns about things like low-level threading, synchronization, thread-safety and concurrent data structures [1]. Pre-Requisites : Having a good knowledge about Java8, Functional Programming and spring boot microservices. What is Reactive programming? In essence, Reactive programming is an asynchronous programming paradigm concerned with data streams. The system time, a list of entries from the DB/API, user clicks, everything can be a stream of data. Data from different streams can easily be combined and transformed, and in the end processed/observed by the
Read more

Optimal binary search trees

Image
Suppose that we are designing a program to translate text from English to French. For each occurrence of each English word in the text, we need to look up its French equivalent. We could perform these lookup operations by building a binary search tree with $n$ English words as keys and their French equivalents as satellite data. Because we will search the tree for each individual word in the text, we want the total time spent searching to be as low as possible. We could ensure an $Θ(log_2 \;n)$ search time per occurrence by using a red-black tree or any other balanced binary search tree. Words appear with different frequencies, however, and a frequently used word such as the may appear far from the root while a rarely used word appears near the root. Such an organization would slow down the translation, since the number of nodes visited when searching for a key in a binary search tree equals one plus the depth of the node containing the key. We want words that occur frequently in the t
Read more

Edit distance

Image
Biological applications often need to compare the DNA of two (or more) different organisms. A strand of DNA consists of a string of molecules called bases, where the possible bases are adenine, guanine, cytosine, and thymine. Representing each of these bases by its initial letter, we can express a strand of DNA as a string over the finite set $\{A, C, G, T\}$. For example, the DNA of one organism may be $S1 = ACCGGTCGAGTGCGCGGAAGCCGGCCGAA$, and the DNA of another organism may be $S2 = GTCGTTCGGAATGCCGTTGCTCTGTAAA$. One reason to compare two strands of DNA is to determine how “similar” the two strands are, as some measure of how closely related the two organisms are. We can, and do, define similarity in many different ways. For example, we can say that two strands are similar if the number of changes needed to turn one into the other is small. [1] We formalize this notion of similarity in our Edit distance problem. 15-5 Edit distance - Problem Statement In order to transform one sour
Read more