Cardinality and Uncomputability

Emina Torlak and Kevin Zatloukal

- Course evaluation
- Is open; please tell us what you think!
- Languages and representations
- How powerful are general-purpose programming languages?
- Cardinality and countability
- What does it mean for two sets to have the same size?
- Uncomputability
- Are there problems computers can’t solve?

Is open; please tell us what you think!

How powerful are general-purpose programming languages?

- We can think of languages as functions from strings to booleans.
- Such a function returns
`true`

iff a string is in the language. - DFAs and CFGs can represent some functions but not others.
- E.g., no DFA for $ \{ 0^n1^n : n \geq 0\} $.
- General-purpose programs can represent even more functions.
- E.g., no CFG for $ \{ 0^n1^n2^n : n \geq 0\} $.
- Are there some functions no program can represent?
- That’s what we’ll study in these last two lectures :)

What does it mean for two sets to have the same size?

- What does it mean for two sets to have the same size?
- We can establish a
*one-to-one correspondence*between their elements.

- One-to-one (injective) functions
- A function $f : A \to B$ is
*one-to-one*(1-1) if every output corresponds to at most one input: $f(x) = f(x’) \Rightarrow x = x’$ for all $x,x’\in A$.

- Onto (surjective) functions
- A function $f : A \to B$ is
*onto*if there is at least one input for every output: for each $y\in B$, there is an $x\in A$ such that $f(x) = y$.

- One-to-one correspondences (bijections)
- A function $f : A \to B$ is a
*one-to-one correspondence*if it is both one-to-one and onto.

- Cardinality of two sets
- Sets $A$ and $B$ have the same
*cardinality*if there is a one-to-one correspondence between them, i.e., there is a bijection $f : A \to B$.

This definition also works for infinite sets!

- Example: do $\N$ and even natural numbers have the same cardinality?
- Yes! The 1-1 correspondence is $f(n) = 2n$.

0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | $\ldots$ |

0 | 2 | 4 | 6 | 8 | 10 | 12 | 14 | 16 | 18 | 20 | $\ldots$ |

- Countable set
- A set is
*countable*iff it has the same cardinality as some subset of $\N$.

- Equivalently, we can say that …
- A set $S$ is countable iff there is an
*onto*function $g: \N \to S$.

- And we can also say that …
- A set $S$ is countable iff we can order its elements: $S = \{x_0, x_1, x_2, \ldots\}$.

- Example: is the set $\Z$ of all integers countable?

$\N$ | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | $\ldots$ |

$\Z$ | 0 | 1 | -1 | 2 | -2 | 3 | -3 | 4 | -4 | 5 | -5 | $\ldots$ |

There are infinitely many rationals between any two rational numbers.

➚1 / 1 | ➚1 / 2 | ➚1 / 3 | ➚1 / 4 | ➚1 / 5 | ➚1 / 6 | ➚1 / 7 | ➚1 / 8 | $\ldots$ |

➚2 / 1 | ➚2 / 2 | ➚2 / 3 | ➚2 / 4 | ➚2 / 5 | ➚2 / 6 | ➚2 / 7 | ➚2 / 8 | $\ldots$ |

➚3 / 1 | ➚3 / 2 | ➚3 / 3 | ➚3 / 4 | ➚3 / 5 | ➚3 / 6 | ➚3 / 7 | ➚3 / 8 | $\ldots$ |

➚4 / 1 | ➚4 / 2 | ➚4 / 3 | ➚4 / 4 | ➚4 / 5 | ➚4 / 6 | ➚4 / 7 | ➚4 / 8 | $\ldots$ |

➚5 / 1 | ➚5 / 2 | ➚5 / 3 | ➚5 / 4 | ➚5 / 5 | ➚5 / 6 | ➚5 / 7 | ➚5 / 8 | $\ldots$ |

➚6 / 1 | ➚6 / 2 | ➚6 / 3 | ➚6 / 4 | ➚6 / 5 | ➚6 / 6 | ➚6 / 7 | ➚6 / 8 | $\ldots$ |

➚7 / 1 | ➚7 / 2 | ➚7 / 3 | ➚7 / 4 | ➚7 / 5 | ➚7 / 6 | ➚7 / 7 | ➚7 / 8 | $\ldots$ |

$\vdots$ | $\vdots$ | $\vdots$ | $\vdots$ | $\vdots$ | $\vdots$ | $\vdots$ | $\vdots$ | $\ldots$ |

➚1 / 1 | ➚1 / 2 | ➚1 / 3 | ➚1 / 4 | ➚1 / 5 | ➚1 / 6 | ➚1 / 7 | ➚1 / 8 | $\ldots$ |

➚2 / 1 | ➚2 / 2 | ➚2 / 3 | ➚2 / 4 | ➚2 / 5 | ➚2 / 6 | ➚2 / 7 | ➚2 / 8 | $\ldots$ |

➚3 / 1 | ➚3 / 2 | ➚3 / 3 | ➚3 / 4 | ➚3 / 5 | ➚3 / 6 | ➚3 / 7 | ➚3 / 8 | $\ldots$ |

➚4 / 1 | ➚4 / 2 | ➚4 / 3 | ➚4 / 4 | ➚4 / 5 | ➚4 / 6 | ➚4 / 7 | ➚4 / 8 | $\ldots$ |

➚5 / 1 | ➚5 / 2 | ➚5 / 3 | ➚5 / 4 | ➚5 / 5 | ➚5 / 6 | ➚5 / 7 | ➚5 / 8 | $\ldots$ |

➚6 / 1 | ➚6 / 2 | ➚6 / 3 | ➚6 / 4 | ➚6 / 5 | ➚6 / 6 | ➚6 / 7 | ➚6 / 8 | $\ldots$ |

➚7 / 1 | ➚7 / 2 | ➚7 / 3 | ➚7 / 4 | ➚7 / 5 | ➚7 / 6 | ➚7 / 7 | ➚7 / 8 | $\ldots$ |

$\vdots$ | $\vdots$ | $\vdots$ | $\vdots$ | $\vdots$ | $\vdots$ | $\vdots$ | $\vdots$ | $\ldots$ |

The set of all positive rational numbers is countable.

List elements in the order of the sum of the numerator and denominator, breaking ties according to the denominator.

Only $k$ pairs of positive numbers add up to $k+1$, so every positive rational number comes up some point.

This technique is called *dovetailing*.

- How would we show this?
- Alphabetical / lexicographic order doesn’t work (infinitely many A’s):
- A, AA, AAA, AAAA, AAAAA, $\ldots$

- Use dovetailing again!
- List strings in the order of length, breaking ties lexicographically.
- There are only $\vert\Sigma\vert^k$ strings on length $k$.

- For example, $\{0,1\}^* $ is countable:
- $\{\varepsilon, 0, 1, 00, 01, 10, 11, 000, 001, 010, 011, 100, 101, 110, 111, \ldots \}$

Java programs are just strings in $\Sigma^* $ where $\Sigma$ is the alphabet of ASCII characters. Since $\Sigma^* $ is countable, so is the set of all Java programs.

This is true for other programming languages too: C, C++, Python, JavaScript, etc.

So, is everything countable?

- Theorem [due to Cantor]
- The set of real numbers between 0 and 1, $[0,1)$, is not countable.

Proof will be by contradiction.
Using a method called *diagonalization*.

First, note that every number in $[0,1)$ has an infinite decimal expansion:

This representation is unique except for the cases where the decimal expansion ends in all 0’s or all 9’s. We will use the all 0’s representation.

Suppose for contradiction that there is a list $\{r_0, r_1, r_2, \ldots \}$ of all real numbers in $[0,1)$.

Consider the digits $x_0, x_{1}, x_{2}, x_{3}, \ldots$ on the diagonal of this list, i.e., the $n$-th digit of $r_n$ for $n\in\N$.

For each such digit $x_{i}$, construct the digit $\widehat{x}_i$ as follows:

- If $x_{i}=1$ then $\widehat{x}_i = 0$.
- If $x_{i}\neq 1$ then $\widehat{x}_i = 1$.

Now, consider the number $\widehat{r} = 0.\widehat{x} _ 0 \widehat{x} _ 1 \widehat{x} _ 2 \widehat{x} _ 3 \ldots$

Note that $r_n \neq \widehat{r}$ for any $n\in\N$ because they differ on the $n$-th digit.

So the list doesn’t include $\widehat{r}$, which is a contradiction. Thus the set $[0, 1)$ is uncountable.

$r_0$ | $0.$$5$$00000000000\ldots$ |

$r_1$ | $0.3$$3$$3333333333\ldots$ |

$r_2$ | $0.14$$2$$857142857\ldots$ |

$r_3$ | $0.141$$5$$92653589\ldots$ |

$r_4$ | $0.2000$$0$$0000000\ldots$ |

$\vdots$ |

Suppose for contradiction that there is a list $\{f_1, f_2, f_3, \ldots \}$ of functions from $\N$ to $\{0,1\}$.

Consider the outputs $x_0, x_1, x_2, x_3, \ldots$ on the diagonal of this list, i.e., $f_n(n)$ for $n\in\N$.

For each such output $x_{i}$, construct $\widehat{x}_i$ as follows:

- If $x_{i}=1$ then $\widehat{x}_i = 0$.
- If $x_{i}\neq 1$ then $\widehat{x}_i = 1$.

Now, consider the function $\widehat{f}(n) = \widehat{x} _ {n}$.

Note that $f_n \neq \widehat{f}$ for any $n\in\N$ because the functions differ on the $n$-th output.

So the list doesn’t include $\widehat{f}$, which is a contradiction. Thus the set $\{f \,|\, f:\N\to\{0,1\}\}$ is uncountable.

$f_0$ | $0$$\,0\,0\,0\,0\,0\,0\,0\,0\ldots$ |

$f_1$ | $1\,$$1$$\,1\,1\,1\,1\,1\,1\,1\ldots$ |

$f_2$ | $0\,1\,$$0$$\,1\,0\,1\,0\,1\,0\ldots$ |

$f_3$ | $0\,1\,1\,$$1$$\,0\,1\,1\,1\,0\ldots$ |

$f_4$ | $1\,1\,0\,0\,$$0$$\,1\,1\,0\,0\ldots$ |

$\vdots$ |

Are there problems computers can’t solve?

- We have seen that …
- The set of all (Java) programs is countable.
- The set of all functions $f : \N \to \{0,1\}$ is uncountable.

So there must be some function $f : \N \to \{0,1\}$ that is not computable by any program! We’ll study one such function next time.

- Cardinality and countability.
- Two sets have the same cardinality if there is a bijection between them.
- A set is countable iff it has the same cardinality as some subset of $\N$.
- Use dovetailing to show that a set is countable and diagonalization to show that it’s uncountable.
- Computability.
- Countability of programs and uncountability of functions $f : \N \to \{0,1\}$ tells us that there is some function that can’t be computed by any program!