Finished 2.2

1. added 2.2-1
2. part of 2.2-2(pseudocode and code)

.gitignore

@@ -0,0 +1,9 @@
+#mac-os
+.DS_Store
+
+#tex
+.texpadtmp/
+
+#py
+__pycache__
+*.pyc

Foundations/overview/P1-1.tex

@@ -0,0 +1,20 @@
+\documentclass{article}
+\usepackage{array}
+\usepackage[margin=0.5in]{geometry}
+\begin{document}
+	\title{Problems 1-1}
+	\author{pezy}
+	\maketitle
+	\begin{tabular}{ c | c | c | c | c | c | c | c | }
+		  & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\
+		  & second & minute & hour & day & month & year & century \\ \hline
+		$\lg n$ & $2^{10^6}$ & $2^{6\cdot10^7}$ & $2^{36\cdot10^8}$ & $2^{864\cdot10^8}$ & $2^{2592\cdot10^9}$ & $2^{31536\cdot10^9}$ & $2^{31536\cdot10^{11}}$ \\ \hline
+		$\sqrt{n}$ & $10^{12}$ & $3.6\cdot10^{15}$ & $1.296\cdot10^{19}$ & $7.46496\cdot10^{21}$ & $6.718464\cdot10^{24}$ & $9.945192960\cdot10^{26}$ & $9.945192960\cdot10^{30}$ \\ \hline
+		$n$ & $10^6$ & $6\cdot10^7$ & $3.6\cdot10^9$ & $8.64\cdot10^{10}$ & $2.592\cdot10^{12}$ & $3.1536\cdot10^{13}$ & $3.1536\cdot10^{15}$ \\ \hline
+		$n \lg n$ & $62746$ & $2801418$ & $133378059$ & $2755147513$ & $71870856404$ & $7.9763389\cdot10^{11}$ & $6.86109568\cdot10^{13}$ \\ \hline
+		$n^2$ & $1000$ & $7745$ & $60000$ & $293938$ & $1609968$ & $5615692$ & $56156922$ \\ \hline
+		$n^3$ & $100$ & $391$ & $1532$ & $4420$ & $13736$ & $31593$ & $146645$ \\ \hline
+		$2^n$ & $19$ & $25$ & $31$ & $36$ & $41$ & $44$ & $51$ \\ \hline
+		$n!$ & $9$ & $11$ & $12$ & $13$ & $15$ & $16$ & $17$ \\ \hline
+	\end{tabular}
+\end{document}

Foundations/overview/README.md

@@ -1,17 +1,17 @@
 ##Solutions of Exercises
 
-> 1.1-1
+### 1.1-1
-Give a real-world example that requires sorting or a real-world example that requires computing a convex hull.
+> Give a real-world example that requires sorting or a real-world example that requires computing a convex hull.
 
 such as [the problem](convex-hulls) that computes the convex hull of any given set of n points in the plane efficiently. we need sort the sort the points by x-coordinate, and then computes the convex hull's set.
 
-> 1.1-2
+### 1.1-2
-Other than speed, what other measures of efficiency might one use in a real-world setting?
+> Other than speed, what other measures of efficiency might one use in a real-world setting?
 
 space size.
 
-> 1.1-3
+### 1.1-3
-Select a data structure that you have seen previously, and discuss its strengths and limitations.
+> Select a data structure that you have seen previously, and discuss its strengths and limitations.
 
 Array
 
@@ -24,13 +24,136 @@ Array
 	- One block allocation
 	- Complex position-based insertion
 
-> 1.1-4
+### 1.1-4
-How are the shortest-path and traveling-salesman problems given above similar? How are they different?
+> How are the shortest-path and traveling-salesman problems given above similar? How are they different?
 
 - similar: Both of them are seeking the lowest overall distance.
 - different: shortest-path is the problem of finding a path between two vertices(or nodes); traveling-salesman is the problem of finding the shortest possible route that visits each city exactly once.(and TSP is NP-hard)
 
-> 1.1-5
+### 1.1-5
-Come up with a real-world problem in which only the best solution will do. Then come up with one in which a solution that is "approximately" the best is good enough.
+> Come up with a real-world problem in which only the best solution will do. Then come up with one in which a solution that is "approximately" the best is good enough.
 
 Given a set of integers, finding the maximum one is only the best solution will do, but finding any non-empty subset of them add up to zero is "approximately" the best is good enough.
+
+### 1.2-1
+> Give an example of an application that requires algorithmic content at the application level, and discuss the function of the algorithms involved.
+
+Routing in network, finding next node, would request Dijkstra algorithm. It could find the shortest paths between nodes.
+
+### 1.2-2
+> Suppose we are comparing implementations of insertion sort and merge sort on the same machine. For inputs of size n, insertion sort runs in 8n2 steps, while merge sort runs in 64n lg n steps. For which values of n does insertion sort beat merge sort?
+
+from `2` to `43`.
+
+```py
+import math
+
+for n in range(2, 100):
+	if 8 * n * n >= 64 * n * math.log(n, 2):
+		print(n)
+		break
+```
+
+### 1.2-3
+> What is the smallest value of n such that an algorithm whose running time is 100*n^2 runs faster than an algorithm whose running time is 2^n on the same machine?
+
+n == 15
+
+```py
+import math
+
+for n in range(1, 100):
+	if 100 * n * n < math.pow(2, n):
+		print n
+		break
+```
+
+### Problems 1-1
+> For each function `f(n)` and time `t` in the following table, determine the largest size `n` of a problem that can be solved in time `t`, assuming that the algorithm to solve the problem takes `f(n)` microseconds.
+
+check and run [calc.py](calc.py)
+see [P1-1.pdf](P1-1.pdf)
+
+### 2.1-1
+> Using Figure 2.2 as a model, illustrate the operation of `INSERTION-SORT` on the array `A = <31, 41, 59, 26, 41, 58>.`
+
+```
+31 [41] 59 26 41 58
+31 41 [59] 26 41 58
+31 41 59 [26] 41 58
+26 31 41 59 [41] 58
+26 31 41 41 59 [58]
+26 31 41 41 58 59
+```
+
+### 2.1-2
+> Rewrite the `INSERTION-SORT` procedure to sort into nonincreasing instead of non- decreasing order.
+
+[procedure](nonincreasing-insertion-sort.pdf) |
+[code](insertion_sort.py) |
+[unittest](insertion_sort_unittest.py)
+
+### 2.1-3
+> Consider the searching problem:
+
+> **Input**: A sequence of n numbers `A = <a1, a2,...,an>` and a value v􏰁.
+
+> **Output**: An index i such that `v = A[i]` or the special value NIL if 􏰁v does not
+appear in A.
+
+> Write pseudocode for linear search, which scans through the sequence, looking for 􏰁v. Using a loop invariant, prove that your algorithm is correct. Make sure that your loop invariant fulfills the three necessary properties.
+
+**loop invariant**: At the start of each iteration `for` loop, the `v` does not appear in the subarray `A[1,...,j-1]`.
+
+**Initialization**: We start by showing that the loop invariant holds before the first loop iteration, when `j` == 1. The subarray is empty. therefore, we can't find `v` in an empty array, which shows that the loop invariant holds prior to the first iteration of the loop.
+
+**Maintenance**: Next, we tackle the second property: showing that each iteration maintains the loop invariant. Informally, the body of the `for` loop works by comparing `A[1]` with `v`, if they are equal, we return index `j`, which is the result we want. Otherwise, we move to next element of the array, then preserves the loop invariant.
+
+**Termination**: Finally, we examine what happens when the loop terminates. The loop terminates when `j` equal to the `A.length`, we have that the subarray A[1..n] consists of the elements originally in A[1..n], but `v` does not appear. Observing that the subarray A[1..n] is the entire array, we conclude that `v` does not appear in A. Hence, the algorithm is correct.
+
+[pseudocode](linear-search.pdf) | [code](insertion_sort.py) |
+[unittest](insertion_sort_unittest.py)
+
+### 2.1-4
+> Consider the problem of adding two n-bit binary integers, stored in two n-element arrays A and B. The sum of the two integers should be stored in binary form in an (n+1)-element array `C`. State the problem formally and write pseudocode for adding the two integers.
+
+**Input**: Two n-bit binary integers, `A=<a1,...,an>` and `B=<b1,...,bn>`.
+**Output**: An (n+1)-element array, `C=<c1,...,cn>` which is the sum of the two input integers.
+
+[pseudocode](add-binary.pdf) | [code](insertion_sort.py) |
+[unittest](insertion_sort_unittest.py)
+
+### 2.2-1
+> Express the function `n^3/1000 - 100n^2 + 100n + 3` in terms of O-notation.
+
+`O(n^3)`
+
+### 2.2-2
+> Consider sorting `n` numbers stored in array `A` by first finding the smallest element of `A` and exchanging it with the element in `A[1]`. Then find the second smallest element of `A`, and exchange it with `A[2]`􏰃. Continue in this manner for the first `n-1` elements of `A`. Write pseudocode for this algorithm, which is known as **selection sort**. What loop invariant does this algorithm maintain? Why does it need to run for only the first `n-1` elements, rather than for all `n` elements? Give the best-case and worst-case running times of selection sort in O-notation.
+
+**loop invariant**: At the start of each iteration `for` loop, the subarray `A[1..j-1]` consists of the smallest elements of the array, and in sorted order.
+
+Because we always find the smallest element of `A`, when the first `n-1` elements in sorted order. The last element must bigger than before. Thus, we don't need to run for all `n` elements.
+
+[O-notation and pseudocode](selection-sort.pdf) | [code](insertion_sort.py) |
+[unittest](insertion_sort_unittest.py)
+
+Selection sort is always a `O(n^2)` algorithm.
+
+### 2.2-3
+> Consider linear search again (see Exercise 2.1-3). How many elements of the input sequence need to be checked on the average, assuming that the element being searched for is equally likely to be any element in the array? How about in the worst case? What are the average-case and worst-case running times of linear search in ‚O-notation? Justify your answers.
+
+- half of the elements need to be checked on the average.
+- worst case: all of them need to be checked.
+- average: n/2 -> O(n)
+- worst: n -> O(n)
+
+Thus, `O(n)` in O-notation.
+
+### 2.2-4
+> How can we modify almost any algorithm to have a good best-case running time?
+
+Hardcode the solution for some particular input, and check whether the input
+is that one. If so, simply output the hardcoded solution. One could also check whether the problem is already solved (when sorting, for instance, check first if input is already sorted).
+
+see more answer in [StackOverflow](http://stackoverflow.com/questions/18103762/how-can-we-modify-any-algorithm-to-have-a-best-case-running-time)

Foundations/overview/add-binary.tex

@@ -0,0 +1,20 @@
+\documentclass{article}
+\usepackage{clrscode3e}
+\begin{document}
+	\title{2.1-4}
+	\author{pezy}
+	\maketitle
+	\begin{codebox}
+	\Procname{$\proc{Add-Binary}(A, B)$}
+	\li $C \gets \proc{Array}(\attrib{A}{length}+1)$	
+	\li $carry \gets 0$
+	\li \For $j = \attrib{A}{length}$ \Downto $1$
+	\li		\Do
+				$C[j+1] \gets (A[j] + B[j] + carry) \% 2$
+	\li			$carry \gets (A[j] + B[j] + carry) / 2$
+			\End
+	\li	$C[1] \gets carry$
+	\li \Return $C$
+	\end{codebox}
+
+\end{document}

Foundations/overview/calc.py

@@ -0,0 +1,111 @@
+import math
+from decimal import *
+
+sec = 1000000
+minute = 60 * sec
+hour = 60 * minute
+day = 24 * hour
+month = 30 * day
+year = 365 * day
+century = 100 * year
+
+# n * lg(n) == c
+# ==> 2 ** (lg(n)) * lg (n) == c
+# ==> lg(n) == W(c)
+# ==> n == exp(W(c))
+# see http://en.wikipedia.org/wiki/Lambert_W_function#Example_2
+
+
+def n_lgn(c):
+    lower = 0.0
+    upper = 10e13
+    while True:
+        middle = (lower + upper) / 2
+        if lower == middle or middle == upper:
+            return middle
+        if middle * math.log(middle, 2) > c:
+            upper = middle
+        else:
+            lower = middle
+
+
+# n!
+def de_factorial(c):
+    n = 0
+    while True:
+        if math.factorial(n) >= c:
+            return n - 1
+        else:
+            n += 1
+
+# lg(n)
+print("2 ^ 10 ^ 6")
+print("2 ^ ( %d * 10 ^ 6 )" % (minute / sec))
+print("2 ^ ( %d * 10 ^ 6 )" % (hour / sec))
+print("2 ^ ( %d * 10 ^ 6 )" % (day / sec))
+print("2 ^ ( %d * 10 ^ 6 )" % (month / sec))
+print("2 ^ ( %d * 10 ^ 6 )" % (year / sec))
+print("2 ^ ( %d * 10 ^ 6 )" % (century / sec))
+print("")
+# sqrt(n)
+print("{:.1e}".format(Decimal(sec ** 2)))
+print("{:.1e}".format(Decimal(minute ** 2)))
+print("{:.3e}".format(Decimal(hour ** 2)))
+print("{:.5e}".format(Decimal(day ** 2)))
+print("{:.6e}".format(Decimal(month ** 2)))
+print("{:.9e}".format(Decimal(year ** 2)))
+print("{:.9e}".format(Decimal(century ** 2)))
+print("")
+# n
+print("{:.1e}".format(Decimal(sec)))
+print("{:.1e}".format(Decimal(minute)))
+print("{:.1e}".format(Decimal(hour)))
+print("{:.2e}".format(Decimal(day)))
+print("{:.3e}".format(Decimal(month)))
+print("{:.4e}".format(Decimal(year)))
+print("{:.4e}".format(Decimal(century)))
+print("")
+# n lg(n)
+print(n_lgn(sec))
+print(n_lgn(minute))
+print(n_lgn(hour))
+print(n_lgn(day))
+print(n_lgn(month))
+print(n_lgn(year))
+print(n_lgn(century))
+print("")
+# n^2
+print(math.sqrt(sec))
+print(math.sqrt(minute))
+print(math.sqrt(hour))
+print(math.sqrt(day))
+print(math.sqrt(month))
+print(math.sqrt(year))
+print(math.sqrt(century))
+print("")
+# n^3
+print(sec ** (1 / 3.0))
+print(minute ** (1 / 3.0))
+print(hour ** (1 / 3.0))
+print(day ** (1 / 3.0))
+print(month ** (1 / 3.0))
+print(year ** (1 / 3.0))
+print(century ** (1 / 3.0))
+print("")
+# 2^n
+print(math.log(sec, 2))
+print(math.log(minute, 2))
+print(math.log(hour, 2))
+print(math.log(day, 2))
+print(math.log(month, 2))
+print(math.log(year, 2))
+print(math.log(century, 2))
+print("")
+# n!
+print(de_factorial(sec))
+print(de_factorial(minute))
+print(de_factorial(hour))
+print(de_factorial(day))
+print(de_factorial(month))
+print(de_factorial(year))
+print(de_factorial(century))

Foundations/overview/clrscode3e.sty

@@ -0,0 +1,343 @@
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+% clrscode3e.sty
+
+% See the document "Using the clrscode3e Package in LaTeX 2e" for
+% examples.
+
+% Package for producing pseudocode in the style of Cormen, Leiserson,
+% Rivest, and Stein, Introduction to Algorithms, Third edition.
+
+% LIMITATION: This package works only if each procedure has at most 99
+% numbered lines of code.
+
+% Each pseudocode procedure is typeset within a codebox environment,
+%   \begin{codebox}...\end{codebox}.
+
+% Normally, the first line within the codebox environment is a \Procname
+% command.  The argument of the \Procname command is a math-mode
+% expression consisting of the procedure name and its parameters.  The
+% name of the procedure itself goes within a \proc command.  Example:
+%   \Procname{$\proc{Matrix-Multiply}(A,B)$}
+% The \Procname command is optional.
+
+% To typeset the name of a procedure (e.g., Matrix-Multiply) in small
+% caps, use the \proc command:
+%   \proc{Matrix-Multiply}
+
+% To typeset the name of a constant (e.g., nil) in small caps, use the
+% \const command:
+%   \const{nil}
+
+% To typeset the name of an identifier (e.g., rank) in regular italics,
+% use the \id command:
+%   \id{rank}
+
+% To typeset the name of a fixed function (e.g., sin) in roman, use the
+% \func command:
+%   \func{sin}
+% (Note that several fixed functions, like sin, are already built into
+% LaTeX.)
+
+% The \proc, \const, \id, and \func commands not only use the correct
+% font, they also perform the important service of interpreting a dash
+% as a hyphen, rather than as a minus sign.  These commands may be used
+% either in or out of math mode.
+
+% For attributes, use the various forms of the \attrib commands.
+
+% Other than the \Procname line, all lines begin with either \li (for a
+% numbered line) or \zi (for an unnumbered line).  The following
+% commands are provided for typesetting keywords and handling automatic
+% indentation:
+
+% Loops: \For, \To, \By, \Downto, \Do, \While, \Repeat, \Until
+% Selection: \If, \Then, \Else, \ElseIf, \ElseNoIf
+% Jumps: \Return, \Error, \Goto
+% Multithreading: \Spawn, \Sync, \Parfor
+% Comments: \Comment, \RComment, \CommentSymbol
+% Indentation: \Indentmore, \Startalign, \Stopalign
+
+% \label commands appearing in or after the first numbered line in a
+% codebox resolve to the number of the most recent numbered line.
+
+% \twodots produces the ".." notation used for subarrays.
+
+% Written for general distribution by Thomas H. Cormen, March 2009.
+
+% The author grants permission for anyone to use this macro package and
+% to distribute it unchanged without further restriction.  If you choose
+% to modify this package, you must indicate that you have modified it
+% prior to your distributing it.  I don't want to get bug reports about
+% changes that *you* have made!
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\ProvidesPackage{clrscode3e}
+\RequirePackage{graphics}	% needed for \scalebox command
+
+% The \useregularv command is because we used a different lowercase v
+% in math mode in the book.  Most users of the clrscode3e package will
+% use the standard lowercase v in math mode, and so the \useregularv
+% command is a no-op for them.
+\ifdefined\useregularv\else\newcommand{\useregularv}{}\fi
+
+% Commands for typesetting constants, procedure names, identifiers, and
+% fixed functions.
+\newcommand{\const}[1]{\textnormal{\scshape#1}}
+\newcommand{\proc}[1]{\textnormal{\scshape#1}}
+\newcommand{\text@hyphens}{\mathcode`\-=`\-\relax}
+\newcommand{\func}[1]{%
+  \ensuremath{{\useregularv\mathop{\text@hyphens\operator@font#1}\nolimits}}}
+\newcommand{\id}[1]{%
+  \ensuremath{\mathit{\text@hyphens#1}}}
+
+% Commands for typesetting object attributes.  All of these commands
+% may be used either in or out of math mode.
+
+% Definitions:
+% An i-string is a string that you would use in an \id command,
+%   typically one or more non-Greek letters, numerals, or dashes.
+% An x-string is a string that you would not use in an \id command,
+%   typically because it has a subscript or one or more Greek letters.
+% A single non-Greek letter can be either an i-string or an x-string.
+
+% Most of the time, we use the \attrib command, which assumes that its
+% first argument, the object name, is an x-string and its second
+% argument, the attribute name, is an i-string.  Examples of the
+% \attrib command: \attrib{A}{length}, \attrib{x}{next},
+% \attrib{\rho}{next-item}.  \attrib is just a direct call to the
+% \attribxi macro.
+
+% The four macros \attribxi, \attribxx, \attribix, and \attribii each
+% take two arguments.  The two letters at the end of the macro name
+% determine, in order, how each argument is treated.  x indicates that
+% the argument should be an x-string, and i indicates that the
+% argument should be an i-string.
+
+% Because we use a single letter to indicate most objects and a string
+% of one or more letters to indicate most attribute names, \attribxi
+% is the most common macro we use.  That's why \attrib is just a
+% direct call to \attribxi.
+
+% Use \attribxx when the object name is an x-string and the object
+% name is also an x-string, for example, when the attribute name has a
+% subscript: \attribxx{x}{c_i} or \attrib{x}{\id{key}_i}.  Another
+% time you would use \attribxx is when the attribute name is a Greek
+% letter: \attrib{v}{\pi}.
+
+% When the object name has more than one letter, it is usually an
+% i-string.  In this case, use \attribii or \attribix, depending on
+% the attribute name: \attribii{item}{key}, \attribii{prev-item}{np},
+% \attribix{item}{\pi}.
+
+% The \attribb, \attribbb, \attribbbb, and \attribbxxi macros are for
+% cascading attributes.  They just call the appropriate \attribxi and
+% \attribxx commands.  For one level of cascading, use \attribb:
+% \attribb{x}{left}{size}.  For two levels, use \attribbb:
+% \attribbb{y}{p}{left}{size}.  For three levels, use \attribbbb (but
+% no examples in the book use this macro).  The \attribbxxi macro is
+% for one level of cascading where the first attribute given is an
+% x-string: \attribbxxi{x}{c_i}{n}.
+
+% When the object is an edge of a graph, specified by two vertices,
+% use \attribe or \attribex: \attribe{u}{v}{f}, \attribex{u}{v}{c'}.
+
+\newcommand{\attribxi}[2]{\ensuremath{#1.\hspace*{1pt}\id{#2}}}
+\newcommand{\attribxx}[2]{\ensuremath{#1.\hspace*{1pt}#2}}
+\newcommand{\attribix}[2]{\ensuremath{\id{#1}\hspace*{1pt}.#2}}
+\newcommand{\attribii}[2]{\ensuremath{\id{#1}\hspace*{1pt}.\hspace*{1pt}\id{#2}}}
+\newcommand{\attrib}[2]{\attribxi{#1}{#2}}
+\newcommand{\attribe}[3]{\attribxi{(#1,#2)}{#3}}
+\newcommand{\attribex}[3]{\attribxx{(#1,#2)}{#3}}
+\newcommand{\attribb}[3]{\attribxi{\attribxi{#1}{#2}}{#3}}
+\newcommand{\attribbb}[4]{\attribxi{\attribb{#1}{#2}{#3}}{#4}}
+\newcommand{\attribbbb}[5]{\attribxi{\attribbb{#1}{#2}{#3}{#4}}{#5}}
+\newcommand{\attribbxxi}[3]{\attribxi{\attribxx{#1}{#2}}{#3}}
+
+% Command for typesetting subarray ranges.
+\newcommand{\twodots}{\mathinner{\ldotp\ldotp}}
+
+% The codelinenumber counter counts the current line number.
+\newcounter{codelinenumber}
+
+% The indent counter keeps track of the current indentation level.
+\newcounter{indent}
+
+% The \iffirstcodeline command tells us whether we are about to
+% produce the first line other than the procedure declaration.
+\newif\iffirstcodeline\firstcodelinetrue
+
+% The \zeroli command makes it so that we're about to produce the
+% first line other than the procedure declaration.
+\newcommand{\zeroli}{\setcounter{codelinenumber}{0}%
+\setcounter{indent}{0}%
+\firstcodelinetrue}
+
+% \digitwidth gives the width of a single digit.  All digits are the
+% same width.  We'll need this amount to do the right thing for line
+% numbers.
+\newlength{\digitwidth}
+\settowidth{\digitwidth}{0}
+
+% The \li command bumps the counter, outputs it, and skips some space
+% A \label cmd for a given numbered line is allowed to appear after the
+% \\, as in 
+%	\li $x\gets y$  \label{li:assign-x}
+% But if \li merely set \@currentlabel in the usual way via
+% \refstepcounter, the value of \@currentlabel does not persist outside
+% the current cell.  Solution: use an additional, global variable
+% \@lilabel.
+
+% THC: This next command is magic to me.  I didn't write it.
+\def\@startline{\global\@curtabmar\@nxttabmar\relax
+   \global\@curtab\@curtabmar\setbox\@curline\hbox
+    {}\@startfield\strut}
+
+% \code@init is run at the beginning of a codebox environment.
+\def\code@init{%
+  \zeroli%						  producing the first line
+  \setlength{\tabbingsep}{1em}%   distance between numbers and code
+  % Initialize \@lilabel to allow a pageref \label cmd at the beginning
+  % of the codebox
+  \global\let\@lilabel\@currentlabel
+  \def\@currentlabel{\@lilabel}%
+}
+
+% When we make a codebox, we save the code part into a box before
+% printing it.  We do not actually print the code until we know how many
+% line numbers there are.
+\newsavebox{\savecode}
+
+% The \ifprocname command tells us whether this procedure has been
+% given a name yet.
+\newif\ifprocname\procnamefalse
+
+% Assume that the width of the codebox is the width of the text, minus
+% the width of 2 digits.  We'll correct for that later.
+\newlength{\codeboxwidth}
+\setlength{\codeboxwidth}{\linewidth} % Thanks, David Etherington!
+\addtolength{\codeboxwidth}{-2\digitwidth}
+
+% The "codebox" environment produces an unbreakable section of code
+\newenvironment{codebox}{%
+\global\procnamefalse%            this proc hasn't been given a name yet
+\code@init%                       set up for first line
+\begin{lrbox}{\savecode}%         save the code into a box
+\begin{minipage}[t]{\codeboxwidth}%  it'll be a minipage
+% Set up the tab stops
+\def\codeindent{\textbf{else} }%
+\begin{tabbing}%
+99\=\codeindent\=\codeindent\=\codeindent\=\codeindent\=\codeindent\=\codeindent\=\codeindent\=codeindent\=codeindent\=\+\kill%
+}{%
+% Here's what's run at the end of a codebox environment.  Start by
+% making sure that we have ended at indent level 0.  Otherwise, print a
+% warning.
+\ifnum\value{indent}=0\else\typeout{Warning: Indentation ends at level \theindent\space in codebox on page \thepage.}\fi%
+\end{tabbing}\end{minipage}\end{lrbox}%
+\addtolength{\topsep}{0.5ex}%     for the following trivlist
+\begin{trivlist}\item\parindent=0pt%
+% If there was a procedure name given, print it now but with a little
+% space below, and disallow a page break after the procedure name.
+\@nobreaktrue%
+\ifprocname\saveprocname\rule[-1.25ex]{0pt}{0pt}\\ \fi%
+% Put in the right amount of space, depending on whether we reached
+% double digits in the line numbers.
+\ifnum\value{codelinenumber}>9\hspace*{2\digitwidth}\else\hspace*{1\digitwidth}\fi%
+% Now print the code
+\usebox{\savecode}\end{trivlist}%
+\addtolength{\topsep}{-0.5ex}\global\procnamefalse}
+
+% Use the \Procname macro to give the name of the procedure.
+\newcommand{\Procname}[1]{\global\def\saveprocname{#1}\global\procnametrue}
+
+\newcounter{thisindent}			% counter for recursive indenting code
+\newcommand{\Indent}{\setcounter{thisindent}{\value{indent}}\putindents}
+% \putindents is a recursive macro that indents a number of times given
+% by the counter thisindent.
+\newcommand{\putindents}{\ifnum\value{thisindent}>0\>\addtocounter{thisindent}{-1}\putindents\fi}
+
+% For typesetting any keyword in the main text.
+\newcommand{\kw}[1]{\textbf{#1}}
+
+% Override the 'gets' symbol.
+\def\gets{\mathrel{\hspace{1pt}=\hspace{1pt}}}
+\newcommand{\isequal}{\mathrel{\scalebox{0.8}[1]{=}\hspace*{1pt}\scalebox{0.8}[1]{=}}}
+
+% All of our favorite keywords.
+\newcommand{\For}{\textbf{for} }
+\newcommand{\To}{\ifmmode\ \textrm{\textbf{to}}\ \else\textbf{to}\ \fi}
+\newcommand{\By}{\ifmmode\ \textrm{\textbf{by}}\ \else\textbf{by}\ \fi}
+\newcommand{\Downto}{\ifmmode\ \textrm{\textbf{downto}}\ \else\textbf{downto}\ \fi}
+\newcommand{\While}{\textbf{while} }
+\newcommand{\Repeat}{\textbf{repeat}\>\addtocounter{indent}{1}}
+\newcommand{\Until}{\kill\addtocounter{indent}{-1}\liprint\textbf{until} }
+\newcommand{\If}{\textbf{if} }
+\newcommand{\Then}{\>\addtocounter{indent}{1}}
+\newcommand{\Else}{\kill\addtocounter{indent}{-1}\liprint\textbf{else}\>\addtocounter{indent}{1}}
+\newcommand{\End}{\addtocounter{indent}{-1}}
+\newcommand{\ElseIf}{\kill\addtocounter{indent}{-1}\liprint\textbf{elseif} }
+\newcommand{\ElseNoIf}{\kill\addtocounter{indent}{-1}\liprint\textbf{else} \addtocounter{indent}{1}}
+\newcommand{\Do}{\>\addtocounter{indent}{1}}
+\newcommand{\Return}{\textbf{return} }
+\newcommand{\CommentSymbol}{\texttt{\textbf{/\hspace*{-0.3em}/}}}
+\newcommand{\Comment}{\CommentSymbol\ }
+\newcommand{\RComment}{\`\CommentSymbol\ }
+\newcommand{\Goto}{\textbf{goto} }
+\newcommand{\Error}{\textbf{error} } % optionally followed by string argument
+\newcommand{\EndTest}{\textbf{:}}
+\newcommand{\Spawn}{\ifmmode\textbf{spawn}\ \else\textbf{spawn} \fi}
+\newcommand{\Sync}{\textbf{sync}}
+\newcommand{\Parfor}{\textbf{parallel for} }
+
+% Indent the next line one level more
+\newcommand{\Indentmore}{\addtocounter{indent}{1}}
+
+\newif\ifnumberedline
+\numberedlinetrue
+
+% The \li command starts a new numbered line.
+\newcommand{\li}{\global\numberedlinetrue%
+\iffirstcodeline\global\firstcodelinefalse\else\\ \fi
+\stepcounter{codelinenumber}%
+\liprint}
+
+% The \lispace command starts a new numbered line with a little extra
+% space above, given by the argument.
+\newcommand{\lispace}[1]{\iffirstcodeline\global\firstcodelinefalse\else\\[#1] \fi
+\stepcounter{codelinenumber}%
+\liprint}
+
+% \liprint actually prints the line number and sets up the indentation.
+\newcommand{\liprint}{\protected@xdef\@lilabel{\thecodelinenumber}%
+\ifnumberedline\thecodelinenumber\fi\'\Indent%
+}
+
+\providecommand{\numref}[1]{%
+  \@ifundefined{r@#1}{000}{%
+    \expandafter\expandafter\expandafter\@firstoftwo
+    \csname r@#1\endcsname
+  }%
+}
+
+% \setlinenumber sets the line number to its argument
+\newcommand{\setlinenumber}[1]{\setcounter{codelinenumber}{\numref{#1}}%
+\addtocounter{codelinenumber}{-1}}
+% \setlinenumberplus sets the line number to its first argument plus its
+% second argument.
+\newcommand{\setlinenumberplus}[2]{\setcounter{codelinenumber}{\numref{#1}}%
+\addtocounter{codelinenumber}{-1}\addtocounter{codelinenumber}{#2}}
+
+% The \zi command starts a new unnumbered line.
+\newcommand{\zi}{\global\numberedlinefalse%
+\iffirstcodeline\global\firstcodelinefalse\else\\ \fi
+\liprint}
+
+% Temporarily make all lines indented so that they start at the end of
+% a given text.
+\newcommand{\Startalign}[1]{\\ \pushtabs\FakeIndent#1\=\kill}
+\newcommand{\Stopalign}{\poptabs}
+\newcommand{\FakeIndent}{\setcounter{thisindent}{\value{indent}}\putfakeindents}
+\newcommand{\putfakeindents}{\ifnum\value{thisindent}>0\textbf{else }\addtocounter{thisindent}{-1}\putfakeindents\fi}
+
+\endinput

Foundations/overview/insertion_sort.py

@@ -0,0 +1,50 @@
+__author__ = 'pezy'
+
+
+def insertion_sort(lst):
+    for j in range(1, len(lst)):
+        key = lst[j]
+        i = j - 1
+        while i >= 0 and lst[i] > key:
+            lst[i + 1] = lst[i]
+            i -= 1
+        lst[i + 1] = key
+    return lst
+
+
+def insertion_sort_non_increasing(lst):
+    for j in range(1, len(lst)):
+        key = lst[j]
+        i = j - 1
+        while i >= 0 and lst[i] < key:
+            lst[i + 1] = lst[i]
+            i -= 1
+        lst[i + 1] = key
+    return lst
+
+
+def linear_search(lst, v):
+    for j in range(0, len(lst)):
+        if v == lst[j]:
+            return j
+    return None
+
+
+def add_binary(a_lst, b_lst):
+    c_lst = [None]*(len(a_lst)+1)
+    carry = 0
+    for j in reversed(range(len(a_lst))):
+        c_lst[j+1] = (a_lst[j] + b_lst[j] + carry) % 2
+        carry = (a_lst[j] + b_lst[j] + carry) // 2
+    c_lst[0] = carry
+    return c_lst
+
+
+def selection_sort(lst):
+    for j in range(0, len(lst)-1):
+        smallest = j
+        for i in range(j+1, len(lst)):
+            if lst[i] < lst[smallest]:
+                smallest = i
+        lst[j], lst[smallest] = lst[smallest], lst[j]
+    return lst

Foundations/overview/insertion_sort_unittest.py

@@ -0,0 +1,34 @@
+__author__ = 'pezy'
+
+import unittest
+from insertion_sort import *
+
+
+# case from @Mooophy
+class InsertionSortTest(unittest.TestCase):
+    def test_insertion_sort(self):
+        self.assertEqual(insertion_sort([5, 2, 4, 6, 1, 3]), [1, 2, 3, 4, 5, 6])
+        self.assertEqual(insertion_sort([5]), [5])
+        self.assertEqual(insertion_sort([]), [])
+        self.assertEqual(insertion_sort([1, 5, 2, 1, 6]), [1, 1, 2, 5, 6])
+
+    def test_insertion_sort_non_increasing(self):
+        self.assertEqual(insertion_sort_non_increasing([1, 5, 2, 1, 6]), [6, 5, 2, 1, 1])
+        self.assertEqual(insertion_sort_non_increasing([]), [])
+
+    def test_linear_search(self):
+        self.assertEqual(linear_search([1, 2, 3, 4, 5], 3), 2)
+        self.assertEqual(linear_search([1, 2, 3, 4, 5], 6), None)
+
+    def test_add_binary(self):
+        self.assertEqual(add_binary([0, 0, 1], [1, 1, 1]), [1, 0, 0, 0])
+        self.assertEqual(add_binary([1, 1, 1], [1, 1, 1]), [1, 1, 1, 0])
+
+    def test_selection_sort(self):
+        self.assertEqual(selection_sort([5, 2, 4, 6, 1, 3]), [1, 2, 3, 4, 5, 6])
+        self.assertEqual(selection_sort([5]), [5])
+        self.assertEqual(selection_sort([]), [])
+        self.assertEqual(selection_sort([1, 5, 2, 1, 6]), [1, 1, 2, 5, 6])
+
+if __name__ == '__main__':
+    unittest.main()

Foundations/overview/linear-search.tex

@@ -0,0 +1,17 @@
+\documentclass{article}
+\usepackage{clrscode3e}
+\begin{document}
+	\title{2.1-3}
+	\author{pezy}
+	\maketitle
+	\begin{codebox}
+		\Procname{$\proc{Linear-Search}(A, v)$}
+		\li \For $j \gets 1$ \To $\attrib{A}{length}$
+		\li		\Do
+					\If $A[j] \isequal v$
+		\li				\Then \Return $j$
+		\li			\Else
+						\Return \const{nil}
+				\End
+	\end{codebox}
+\end{document}

Foundations/overview/nonincreasing-insertion-sort.tex

@@ -0,0 +1,24 @@
+\documentclass{article}
+\usepackage{clrscode3e}
+\begin{document}
+	\title{2.1-2}
+	\author{pezy}
+	\maketitle
+	\begin{codebox}
+	\Procname{$\proc{Nonincreasing-Insertion-Sort}(A)$}
+	\li \For $j \gets 2$ \To $\attrib{A}{length}$
+	\li		\Do	
+				$\id{key} \gets A[j]$
+	\li			\Comment Insert $A[j]$ into the sorted sequence $A[1 \twodots j-1]$.
+	\li			$i \gets j-1$
+	\li			\While $i > 0$ and $A[i] < \id{key}$
+	\li				\Do
+						$A[i+1] \gets A[i]$
+	\li					$i \gets i-1$
+					\End
+	\li			$A[i+1] \gets \id{key}$
+			\End				
+	\end{codebox}
+\end{document}
+	
+

Foundations/overview/selection-sort.tex

@@ -0,0 +1,131 @@
+\documentclass[twoside]{article}
+\setlength{\oddsidemargin}{0.25 in}
+\setlength{\evensidemargin}{-0.25 in}
+\setlength{\topmargin}{-0.6 in}
+\setlength{\textwidth}{6.5 in}
+\setlength{\textheight}{8.5 in}
+\setlength{\headsep}{0.75 in}
+\setlength{\parindent}{0 in}
+\setlength{\parskip}{0.1 in}
+
+\usepackage{clrscode3e}
+\usepackage{multicol}
+\usepackage{amsmath}
+
+\begin{document}
+	\title{2.2-2}
+	\author{pezy}
+	\maketitle
+	\begin{multicols}{3}
+		\begin{codebox}
+			\Procname{$\proc{Selection-Sort}(A)$}
+			\li \For $j \gets 1$ \To $\attrib{A}{length} - 1$
+			\li \Do	$min \gets j$
+			\li		\For $i \gets j + 1$ \To $\attrib{A}{length}$
+			\li			\Do \If $A[i] < A[min]$	
+			\li					\Do $min \gets i$ 
+			\End
+			\End
+			\li			\func{swap}($A[min], A[j]$)
+			\End
+		\end{codebox}
+		
+		\columnbreak
+		
+		\begin{center}	% column 2 holds cost of each statement
+			\textbf{\large Cost}
+			\\*$c_{1}$
+			\\*$c_{2}$
+			\\*$c_{3}$
+			\\*$c_{4}$
+			\\*$c_{5}$
+			\\*$c_{6}$
+		\end{center}
+		
+		\columnbreak
+		
+		\begin{flushleft} % displays run time of each statement
+			\textbf{\large Time}
+			\\* $n$
+			\\* $n-1$
+			\\* $\sum _{ j=1 }^{ n-1 } t_j $
+			\\* $\sum _{ j=1 }^{ n-1 } (t_j - 1) $
+			\\* $0 \To \sum _{ j=1 }^{ n-1 } (t_j - 1)$
+			\\* $n-1$
+		\end{flushleft}
+	\end{multicols}
+	
+	We will now calculate the running time, $T(n)$,  of \proc{Selection-Sort}:
+	
+	\begin{align}
+	\notag
+	T(n) &= c_1 n +  c_2 (n-1) + c_3 \sum _{j=1}^{n-1}  t_j + c_4 \sum _{j=1}^{n-1} ( t_j -1 ) + k c_5 \sum _{j=1}^{n-1}  (t_j -1)  + c_6 (n-1),\\
+	\notag
+	&= c_1 n+ (c_2 + c_6)(n-1) + c_3 \sum _{j=1}^{n-1}  t_j + (c_4 + k c_5) \sum _{j=1}^{n-1}  (t_j -1),\\
+	\notag
+	&= c_1 n+ (c_2 + c_6)(n-1) + (c_3 + c_4 + k c_5 ) \sum _{j=1}^{n-1}  t_j - (c_4 + k c_5) (n-1),\\
+	\label{eq:Tn}
+	&= c_1 n +(c_2 + c_6 - c_4 - k c_5) (n-1) + (c_3 + c_4 + k c_5 ) \sum _{j=1}^{n-1}  t_j.
+	\end{align}
+	
+	\section{Best case running time}
+	In the  {\bf BEST CASE} running time, the list of input will already be sorted. Thus, the body of \If is never step in, and $k = 0$. we obtain that $t_j = j+1$, for every choice of j. Thus,
+	
+	\[
+	\sum_{j=1}^{n-1} t_j = \frac{1}{2} n(n+1) - 1 = (\frac{n}{2}+1)(n-1)
+	\]
+	
+	Substituting this into the last term of Eqn.~(\ref{eq:Tn}) yields,
+	\begin{align}
+	T(n) &= c_1 n +(c_2 + c_6 - c_4)(n-1) + (c_3 + c_4)(\frac{n}{2}+1)(n-1) \\
+	&= \frac{c_3+c_4}{2}n^2 + (c_1+c_2+\frac{c_3}{2}-\frac{c_4}{2}+c_6)n - (c_2+c_3+c_6)
+	\end{align}
+	
+	which can be simplified to the linear equation $T(n) = An^2+Bn+C$ where
+	\begin{align*}
+	A &= \frac{c_3+c_4}{2} > 0,\\
+	B &= c_1+c_2+\frac{c_3}{2}-\frac{c_4}{2}+c_6, \quad\text{and,}\\
+	C &= -c_2-c_3-c_6 < 0.
+	\end{align*}
+	
+	Therefore, the {\bf BEST CASE} running time of the \proc{Selection-Sort} Algorithm equals:
+	\begin{center}
+		$\boldsymbol{T(n) = An^2+Bn+C}$
+	\end{center}
+	
+	%
+	%Begin section for Worst Case Running Time
+	%
+	%
+	\section{Worst case running time}
+	We will now look at the {\bf WORST CASE} for \proc{Selection-Sort}:
+	\begin{itemize}
+		\item In the worst case, the \If statement is invoked on every occasion.
+		\item This means $ k = 1$
+	\end{itemize}
+	
+	Substituting $t_{ j }$ with $j$ into the last summation in  Eqn.~(\ref{eq:Tn}) yields,
+	
+	\[
+	\sum_{j=1}^{n-1} t_j = \frac{1}{2} n(n+1) - 1 = (\frac{n}{2}+1)(n-1)
+	\]
+	
+	Thus, Eqn.~(\ref{eq:Tn}) becomes,
+	\begin{align*}
+	T(n)   &=   c_1 n +(c_2+c_6-c_4-c_5)(n-1) + (c_3+c_4+c_5)(\frac{n}{2}+1)(n-1),\\
+	&= \frac{c_3+c_4+c_5}{2} n^2 + (c_1+c_2+\frac{c_3}{2}-\frac{c_4}{2}-\frac{c_5}{2}+c_6) n - (c_2+c_3+c_6)
+	\end{align*}
+	a \emph{quadratic function} of $n$, the input sequence length, where,
+	\begin{align*}
+	A' &= \frac{c_3+c_4+c_5}{2} > 0,\\
+	B' &= c_1+c_2+\frac{c_3}{2}-\frac{c_4}{2}-\frac{c_5}{2}+c_6, \quad\text{and,}\\
+	C' &= -c_{2}-c_{3}-c_{6} < 0.
+	\end{align*}
+	
+	
+	Therefore, the {\bf WORST CASE} running time of the \proc{Selection-Sort} Algorithm also equals:
+	\begin{center}
+		$\boldsymbol{T(n) = An^2+Bn+C}$
+	\end{center}
+	
+\end{document}