CSE 525: Randomized Algorithms Spring 2026 Lecture 19: Chaining for Norms Lecturer: Shayan Oveis Gharan 06/04/25

Disclaimer: These notes have not been subjected to the usual scrutiny reserved for formal publications.

The content of these notes are based on https://homes.cs.washington.edu/~jrl/cse599wi23/notes/lec4.html.

Entropy-number convention.

For a metric space $(S,\rho )$, write $e_h(S,\rho )$ for the smallest radius $r$ such that $S$ is coverable by at most $2^{2^h}$ balls of radius $r$ in metric $\rho$.

19.1 Norms and the main estimate

Definition 19.1 (Norms and seminorms).

A map $N:{ℝ}^n\to {ℝ}_{+}$ is a norm when, for all $x,y\in {ℝ}^n$ and $\lambda \in ℝ$,

1. $N(\lambda x)=\left|\lambda \right|N(x)$,

2. $N(x+y)\le N(x)+N(y)$,

3. $N(x)=0$ if and only if $x=0$.

When only the first two properties are used, $N$ is a seminorm. The arguments below use the word “norm” in this broad sense.

Let $N_1,\mathrm{\dots },N_m$ be norms on ${ℝ}^n$, and let

$$T\subseteq B_2^n,B_2^n≔\{x\in {ℝ}^n:{\parallel x\parallel }_2\le 1\}.$$

For a standard Gaussian $g\sim N(0,I_n)$, define

$$\kappa ≔𝔼\underset{k=1,\mathrm{\dots },m}{\max }{N}_k(g).$$

Theorem 19.2.

If ${ϵ}_1,\mathrm{\dots },{ϵ}_m$ are independent random signs, then

$$𝔼\underset{x\in T}{\max }\sum _{k=1}^m{ϵ}_k{N}_k{(x)}^2\lesssim \kappa \log (n)\underset{x\in T}{\max }\sqrt{\sum _{k=1}^m{N}_k{(x)}^2}.$$ (19.1)

19.1.1 Example: sums of random matrices

Let

$$A=\sum _{k=1}^m{ϵ}_k{A}_k^T{A}_k,$$

with each $A_k^T{A}_k$ positive semidefinite. Then

${\parallel A\parallel }_{\mathrm{op}}$ $=\underset{{\parallel x\parallel }_2\le 1}{\max }⟨x,Ax⟩$

$=\underset{{\parallel x\parallel }_2\le 1}{\max }{\displaystyle \sum _{k=1}^m}{ϵ}_k⟨x,A_k^T{A}_{k}x⟩$

$=\underset{{\parallel x\parallel }_2\le 1}{\max }{\displaystyle \sum _{k=1}^m}{ϵ}_k{\parallel A_{k}x\parallel }_2^2.$

This is the preceding setting with $T=B_2^n$ and $N_k(x)={\parallel A_{k}x\parallel }_2$.

Translator note.

The source text appears to phrase the final identification in squared form. The normalization above is the one for which ${\sum }_k{ϵ}_k{N}_k{(x)}^2={\sum }_k{ϵ}_k{\parallel A_{k}x\parallel }_2^2$.

19.2 Dudley’s inequality and metric reduction

The process

$$\{\sum _{k=1}^m{ϵ}_k{N}_k{(x)}^2:x\in T\}$$

is subgaussian with respect to

$$d(x,y)≔{\left(\sum _{k=1}^m{\left(N_k{(x)}^2-N_k{(y)}^2\right)}^2\right)}^{1/2}.$$

Dudley’s entropy inequality therefore gives

$$𝔼\underset{x\in T}{\max }\sum _{k=1}^m{ϵ}_k{N}_k{(x)}^2\lesssim \sum _{h\ge 0}{2}^{h/2}{e}_h(T,d).$$ (19.2)

Both sides of (19.1) are homogeneous of degree two in the family ${(N_k)}_{k=1}^m$. Thus one may rescale and assume

$$\underset{x\in T}{\max }\sqrt{\sum _{k=1}^m{N}_k{(x)}^2}=1.$$ (19.3)

Define

$${\parallel x\parallel }_N≔\underset{k=1,\mathrm{\dots },m}{\max }{N}_k(x).$$

For $x,y\in T$, use $a^2-b^2=(a-b)(a+b)$ to obtain

$d(x,y)$ $={\left({\displaystyle \sum _{k=1}^m}{\left(N_k(x)-N_k(y)\right)}^2{\left(N_k(x)+N_k(y)\right)}^2\right)}^{1/2}$

$\le {\left({\displaystyle \sum _{k=1}^m}{(N_k(x-y))}^2{(N_k(x)+N_k(y))}^2\right)}^{1/2}$

$\le {\left({\displaystyle \sum _{k=1}^m}\underset{k}{\max }{N}_k{(x-y)}^2\cdot {(N_k(x)+N_k(y))}^2\right)}^{1/2}$

$={\parallel x-y\parallel }_N{\left({\displaystyle \sum _{k=1}^m}{\left(N_k(x)+N_k(y)\right)}^2\right)}^{1/2}$

$\le {\parallel x-y\parallel }_N\cdot {\left({\displaystyle \sum _{k=1}^m}2N_k{(x)}^2+2N_k{(y)}^2\right)}^{1/2}$

$=2{\parallel x-y\parallel }_N.$

The first inequality uses $\left|N_k(x)-N_k(y)\right|\le N_k(x-y)$, and the last equality follows from (19.3). Consequently, $e_h(T,d)\le 2e_h(T,{\parallel \cdot \parallel }_N)$, and (19.2) implies

$$𝔼\underset{x\in T}{\max }\sum _{k=1}^m{ϵ}_k{N}_k{(x)}^2\lesssim \sum _{h\ge 0}{2}^{h/2}{e}_h(T,{\parallel \cdot \parallel }_N).$$ (19.4)

We now split the right-hand side into the ranges $h\le 4\log n$ and $h>4\log n$.

19.3 The large-entropy tail

Let

$$B_N≔\{x\in {ℝ}^n:{\parallel x\parallel }_N\le 1\}.$$

For $x\in T$, the normalization (19.3) gives

$${\parallel x\parallel }_N\le \sqrt{\sum _{k=1}^m{N}_k{(x)}^2}\le 1,$$

so $T\subseteq B_N$, hence $e_h(T,{\parallel \cdot \parallel }_N)\le e_h(B_N,{\parallel \cdot \parallel }_N)$.

Claim 19.3.

For any norm on ${ℝ}^n$, and any $h\ge 1$,

$$e_h(B_N,{\parallel \cdot \parallel }_N)\le 4\cdot 2^{-2^h/n}.$$

Proof.

Fix $\delta \in (0,1)$, and choose a maximal collection $x_1,\mathrm{\dots },x_s\in B_N$ with pairwise distances at least $2\delta$ in ${\parallel \cdot \parallel }_N$. Maximality gives the cover

$$B_N\subseteq \bigcup _{j=1}^s(x_j+2\delta B_N).$$

The sets $x_j+\delta B_N$ are pairwise disjoint and contained in $2B_N$, so

$${\mathrm{vol}}_n(2B_N)\ge s{\mathrm{vol}}_n(\delta B_N)=s{(\delta /2)}^n{\mathrm{vol}}_n(2B_N).$$

Therefore $s\le {(2/\delta )}^n$. Taking $\delta =2\cdot 2^{-2^h/n}$ yields $s\le 2^{2^h}$ and gives a cover of $B_N$ by at most $2^{2^h}$ balls of radius $2\delta =4\cdot 2^{-2^h/n}$. ∎

Using 19.3, the large-$h$ part of Equation 19.4 obeys

$$\sum _{h>4\log n}{2}^{h/2}{e}_h(T,{\parallel \cdot \parallel }_N)\le 4\sum _{h>4\log n}{2}^{h/2}{2}^{-2^h/n}\le O(1).$$

Thus

$$𝔼\underset{x\in T}{\max }\sum _{k=1}^m{ϵ}_k{N}_k{(x)}^2\lesssim O(1)+\sum _{0\le h\le 4\log n}{2}^{h/2}{e}_h(T,{\parallel \cdot \parallel }_N).$$

19.4 The relevant entropy range and dual Sudakov

Since $T\subseteq B_2^n$,

$$e_h(T,{\parallel \cdot \parallel }_N)\le e_h(B_2^n,{\parallel \cdot \parallel }_N).$$

The required ingredient is the following dual Sudakov bound.

Lemma 19.4 (Dual Sudakov).

For any norm $\parallel \cdot \parallel$ on ${ℝ}^n$ and every $h\ge 0$,

$$e_h(B_2^n,\parallel \cdot \parallel )\lesssim 2^{-h/2}𝔼\parallel g\parallel ,$$

where $g\sim N(0,I_n)$.

Applying 19.4 with $\parallel \cdot \parallel ={\parallel \cdot \parallel }_N$ gives

$$𝔼{\parallel g\parallel }_N=𝔼\underset{k=1,\mathrm{\dots },m}{\max }{N}_k(g)=\kappa .$$

Therefore

$$\sum _{0\le h\le 4\log n}{2}^{h/2}{e}_h(T,{\parallel \cdot \parallel }_N)\lesssim \kappa \log n.$$

Under the normalization (19.3), this proves (19.1); undoing the rescaling gives the stated form.

19.5 Gaussian shift lemma

Lemma 19.5 (Gaussian shift).

Let $K\subseteq {ℝ}^n$ be symmetric and convex, and let ${\gamma }_n$ denote standard Gaussian measure on ${ℝ}^n$. For every $x\in {ℝ}^n$,

$${\gamma }_n(K+x)\ge \exp \left(-\frac{{\parallel x\parallel }_2^2}{2}\right){\gamma }_n(K).$$

Proof.

Using symmetry of $K$ and writing $\sigma$ for a uniform random sign,

${\gamma }_n(K+x)$ $={(2\pi )}^{-n/2}{\displaystyle {\int }_K}\exp \left(-{\displaystyle \frac{{\parallel x+z\parallel }_2^2}{2}}\right)𝑑z$

$={(2\pi )}^{-n/2}{\displaystyle {\int }_K}{𝔼}_{\sigma \in \{-1,1\}}\exp \left(-{\displaystyle \frac{{\parallel \sigma x+z\parallel }_2^2}{2}}\right)𝑑z.$

Since

$${𝔼}_{\sigma }{\parallel \sigma x+z\parallel }_2^2={\parallel x\parallel }_2^2+{\parallel z\parallel }_2^2,$$

Jensen’s inequality yields

${\gamma }_n(K+x)$ $\ge {(2\pi )}^{-n/2}{\displaystyle {\int }_K}\exp \left(-{\displaystyle \frac{{𝔼}_{\sigma }{\parallel \sigma x+z\parallel }_2^2}{2}}\right)𝑑z$

$={(2\pi )}^{-n/2}{\displaystyle {\int }_K}\exp \left(-{\displaystyle \frac{{\parallel x\parallel }_2^2+{\parallel z\parallel }_2^2}{2}}\right)𝑑z$

$=\exp \left(-{\displaystyle \frac{{\parallel x\parallel }_2^2}{2}}\right){\gamma }_n(K).$

∎

Translator note.

The displayed conclusion above matches the Gaussian-shift bound used in the proof of dual Sudakov; constants are immaterial for the subsequent $\lesssim$ estimate.

19.6 Proof of the dual Sudakov lemma

Let

$$ℬ≔\{x\in {ℝ}^n:\parallel x\parallel \le 1\}$$

be the unit ball of the norm $\parallel \cdot \parallel$. Choose $x_1,\mathrm{\dots },x_s\in B_2^n$ maximally so that the translated sets $x_j+\delta ℬ$ are pairwise disjoint. Then

$$B_2^n\subseteq \bigcup _{j=1}^s(x_j+2\delta ℬ),$$ (19.5)

so $B_2^n$ is covered by $s$ balls of radius $2\delta$ in the norm $\parallel \cdot \parallel$.

For any $\lambda >0$, the scaled sets $\lambda (x_j+\delta ℬ)$ are pairwise disjoint. Therefore

$1$ $\ge {\gamma }_n\left({\displaystyle \bigcup _{j=1}^s}\lambda (x_j+\delta ℬ)\right)$

$={\displaystyle \sum _{j=1}^s}{\gamma }_n\left(\lambda x_j+\lambda \delta ℬ\right)$

$\ge {\displaystyle \sum _{j=1}^s}\exp \left(-{\displaystyle \frac{{\lambda }^2{\parallel x_j\parallel }_2^2}{2}}\right){\gamma }_n(\lambda \delta ℬ)$

$\ge s\exp \left(-{\displaystyle \frac{{\lambda }^2}{2}}\right){\gamma }_n(\lambda \delta ℬ),$

where 19.5 is used in the third line and $x_j\in B_2^n$ in the final line.

Choose

$$\lambda ≔\frac{2}{\delta }𝔼\parallel g\parallel .$$

Then, by Markov’s inequality,

$${\gamma }_n(\lambda \delta ℬ)=\mathbb{P}\left(\parallel g\parallel \le \lambda \delta \right)=\mathbb{P}\left(\parallel g\parallel \le 2𝔼\parallel g\parallel \right)\ge \frac{1}{2}.$$

Combining the previous inequalities gives

$$1\ge \frac{s}{2}\exp \left(-\frac{1}{2}{\left(\frac{2𝔼\parallel g\parallel }{\delta }\right)}^2\right).$$

Equivalently, up to universal constants,

$$\delta \lesssim \frac{𝔼\parallel g\parallel }{\sqrt{\log (s/2)}}.$$

With $s=2^{2^h}$, the cover in (19.5) has radius $2\delta \lesssim 2^{-h/2}𝔼\parallel g\parallel$. Hence

$$e_h(B_2^n,\parallel \cdot \parallel )\lesssim 2^{-h/2}𝔼\parallel g\parallel ,$$

which proves 19.4.