CSE 525: Randomized Algorithms Spring 2025 Lecture 7: Negative Correlation and Applications Lecturer: Shayan Oveis Gharan 04/29/2025

Disclaimer: These notes have not been subjected to the usual scrutiny reserved for formal publications.

7.1 Positive Association

Theorem 7.1 (The Four Functions Theorem).

Let $\alpha ,\beta ,\gamma ,\delta :2^{[n]}\to {ℝ}_{\ge 0}$ be non-negative functions defined on subsets of $[n]$. If for any two subsets $A,B\subseteq [n]$ we have

$$\alpha (A)\beta (B)\le \gamma (A\cup B)\delta (A\cap B),$$

then, for every two families of subsets $𝒜,ℬ\subseteq 2^{[n]}$, we have

$$\alpha (𝒜)\beta (ℬ)\le \gamma (𝒜\cup ℬ)\delta (𝒜\cap ℬ),$$

where $\alpha (𝒜)={\sum }_{A\in 𝒜}\alpha (A)$.

The following theorem, known as the FKG inequality, is a direct consequence of the above theorem:

Definition 7.2 (Log-supermodular probability distributions).

We say a probability distribution $\mu :2^{[n]}\to {ℝ}_{\ge 0}$ is log-supermodular if for any $A,B\subseteq [n]$, we have

$$\mu (A)\mu (B)\le \mu (A\cup B)\mu (A\cap B).$$

This property is also known as Positive Lattice Condition.

For a concrete example, consider the family of Erdos-Reyni $G(n,p)$ random graphs. In such a case, for any set $F\subseteq \left(\genfrac{}{}{0pt}{}{n}{2}\right)$ we have

$$\mu (F)=p^{|F|}{(1-p)}^{\left(\genfrac{}{}{0pt}{}{n}{2}\right)-|F|}={(1-p)}^{\left(\genfrac{}{}{0pt}{}{n}{2}\right)}{(\frac{p}{1-p})}^{|F|}.$$

We claim that this distribution is log-supermodular. Cancelling out the normalizing constant ${(1-p)}^{\left(\genfrac{}{}{0pt}{}{n}{2}\right)}$, we need to check for any two sets $A,B\subseteq E$, and $q=\frac{p}{1-p}$

$$q^{|A|}{q}^{|B|}\le q^{|A\cup B|}{q}^{|A\cap B|}$$

But this holds simply because $|A|+|B|=|A\cap B|+|A\cup B|$.

Definition 7.3 (Increasing functions).

We say a function $f:2^{[n]}\to {ℝ}_{geq0}$ is increasing if for any $A,B\subseteq [n]$ such that $A\subseteq B$ we have

$$f(A)\le f(B).$$

We say $f$ is a decreasing function if the above inequality holds in the reverse direction.

For a concrete example, notice for any $i\in [n]$, $f(A)=𝕀\left[i\in A\right]$ is increasing and $f(A)=𝕀\left[i\notin A\right]$ is decreasing.

But, more generally, consider the domain $\left(\genfrac{}{}{0pt}{}{n}{2}\right)$ of the set of all possible edges in a graph with $n$ vertices. Then, for $A\subseteq \left(\genfrac{}{}{0pt}{}{n}{2}\right)$,

$$f(A)=𝕀\left[A\text{ is connected}\right],f(A)=𝕀\left[A\text{ has a Hamiltonian cycle}\right]$$

are increasing, but,

$$f(A)=𝕀\left[G(V,A)\text{ is 3-colorable}\right]$$

is decreasing.

Theorem 7.4 (FKG Inequality).

Let $\mu :2^{[n]}\to {ℝ}_{\ge 0}$ be a log-supermodular probability distribution. Then, for any two increasing functions $f,g:2^{[n]}\to {ℝ}_{\ge 0}$ we have

$$\mathbb{P}\left[f\right]\mathbb{P}\left[g\right]\le \mathbb{P}\left[fg\right],$$

i.e., $\mu$ is positively associated.

Proof.

We use the Four functions theorem with

$$\alpha =\mu \cdot f,\beta =\mu \cdot g,\gamma =\mu \cdot f\cdot g,\text{ and }\delta =\mu .$$

We claim that these four functions satisfy the assumption of the Four functions theorem. In particular, for any $A,B\subseteq [n]$ by log-supermodularity of $\mu$ we have

$\alpha (A)\beta (B)$ $=\mu (A)f(A)\mu (B)g(B)\underset{\text{log-supermodularity}}{\le }\mu (A\cup B)f(A)g(B)\mu (A\cap B)$

$\underset{f,g\text{are increasing}}{\le }\mu (A\cup B)f(A\cup B)g(A\cup B)\mu (A\cap B)=\gamma (A\cup B)\delta (A\cap B).$

Therefore, letting $𝒜,ℬ=[n]$, we conclude

$$\alpha ([n])\beta ([n])=\left(\sum _{A\subset [n]}\mu (A)f(A)\right)\cdot \left(\sum _{B\subseteq [n]}\mu (B)g(B)\right)=𝔼\left[f\right]𝔼\left[g\right]$$

On the other hand,

$$\gamma ([n])\delta ([n])=\left(\sum _{A\subseteq [n]}\mu (A)f(A)g(A)\right)\left(\sum _{B\subseteq [n]}\mu (B)\right)=𝔼\left[fg\right]\cdot 1$$

Putting them together proves the theorem. ∎

Note that the above inequality also holds if both $f,g$ are decreasing functions. In case that $f$ is increasing and $g$ is decreasing, the inequality holds just in the opposite direction.

Consequently, FKG theorem implies that any pair of elements $i,j$ are positively correlated in a log-supermodular probability distribution,

$$\mathbb{P}\left[i\right]\mathbb{P}\left[j\right]\le \mathbb{P}[i,j]\iff \mathbb{P}\left[i|j\right]\ge \mathbb{P}\left[i\right]$$

More interestingly, we can use it to prove the following fact about $G(n,p)$ graphs:

Fact 7.5.

For any $0\le p\le 1$, let $G$ be a random Erdos-Reyni graph with parameter $p$.

$$\mathbb{P}\left[G\text{ has a Hamiltonian cycle}|G\text{ is 3-colorable}\right]\le \mathbb{P}\left[G\text{ has a Hamiltonian cycle}\right].$$

7.2 Negatively Correlated Random Variables

We say that a collection $\{X_1,\mathrm{\dots },X_n\}$ of random variables are negatively correlated if it holds that for any subset $S\subseteq [n]$:

$$𝔼\left[\prod _{i\in S}{X}_i\right]\le \prod _{i\in S}𝔼\left[X_i\right].$$

Note that if $\{X_1,\mathrm{\dots },X_n\}$ are independent, then this holds with equality.

Furthermore, we say $X_1,\mathrm{\dots },X_n$ are pairwise negatively correlated if for all ,

$$𝔼\left[X_i{X}_j\right]\le 𝔼\left[X_i\right]𝔼\left[X_j\right].$$

Theorem 7.6 (Chernoff for negatively correlated random variables).

. Suppose $X_1,\mathrm{\dots },X_n$ are negatively correlated Bernoulli random variables (instead of independent), then the conclusion of the multiplicative Chernoff bound still holds.

Proof.

To see this, note that the one place we used independence in the proof of the Chernoff bound is in the calculation: When $X=X_1+\mathrm{\cdots }+X_n$,

$$𝔼\left[e^{tX}\right]=𝔼\left[e^{t{\sum }_i{X}_i}\right]=𝔼\left[\prod _{i=1}^n{e}^{t{X}_i}\right]=\prod _{i=1}^n𝔼\left[e^{t{X}_i}\right].$$

The main observation is that the above statement still holds except the last identity will be an inequality. So the rest of the proof of the Chernoff bound follows. In particular, when $X_1,\mathrm{\dots },X_n$ are negatively correlated we show

$$𝔼\left[e^{tX}\right]\le \prod _{i=1}^n𝔼\left[e^{t{X}_i}\right].$$

Let $\{{\tilde{X}}_1,\mathrm{\dots },{\tilde{X}}_n\}$ be independent Bernoulli random variables with $𝔼\left[{\tilde{X}}_i\right]=𝔼\left[X_i\right]$ for each $i\in \{1,\mathrm{\dots },n\}$ and define $\tilde{X}:={\tilde{X}}_1+\mathrm{\cdots }+{\tilde{X}}_n$. For any nonnegative integer $k$,

$𝔼\left[X^k\right]$ $={\displaystyle \sum _{\alpha }}𝔼\left[X_1^{{\alpha }_1}{X}_2^{{\alpha }_2}\mathrm{\dots }{X}_n^{{\alpha }_n}\right]$

$\underset{X_i\in \{0,1\}}{=}{\displaystyle \sum _{\alpha }}𝔼\left[{\displaystyle \prod _{i=1}^n}{X}_i^{𝕀\left[{\alpha }_i\equiv 1mod2\right]}\right]$

$\underset{\text{Negative Correlation}}{\le }{\displaystyle \sum _{\alpha }}{\displaystyle \prod _{i=1}^n}𝔼\left[X_i^{𝕀\left[{\alpha }_i\equiv 1mod2\right]}\right]$

$={\displaystyle \sum _{\alpha }}{\displaystyle \prod _{i=1}^n}𝔼\left[X_i^{{\alpha }_i}\right]\underset{𝔼\left[X_i\right]=𝔼\left[{\tilde{X}}_i\right]}{=}{\displaystyle \sum _{\alpha }}{\displaystyle \prod _{i=1}^n}𝔼\left[{\tilde{X}}_i^{{\alpha }_i}\right]$

where the sum is over all non-negative integer vectors $\alpha$ such that ${\sum }_i{\alpha }_i=k$.

On the other hand, since ${\tilde{X}}_1,\mathrm{\dots },{\tilde{X}}_n$ are independent,

$$\sum _{\alpha }\prod _{i=1}^n𝔼\left[{\tilde{X}}_i^{{\alpha }_i}\right]=\sum _{\alpha }𝔼\left[{\tilde{X}}_1^{{\alpha }_1}\mathrm{\dots }{\tilde{X}}_n^{{\alpha }_n}\right]=𝔼\left[\tilde{X}\right].$$

Putting these together we obtain, for every $k\ge 0$,

$$𝔼\left[X^k\right]\le 𝔼\left[{\tilde{X}}^k\right]$$ (7.1)

Lastly, using the Taylor expansion

$$e^{tX}=1+tX+\frac{t^2{X}^2}{2}+\frac{t^3{X}^3}{6}+\mathrm{\dots }$$

Applying (7.1) to every monomial above, we get

$$𝔼\left[e^{tX}\right]\le 𝔼\left[e^{t\tilde{X}}\right]\underset{\text{independence}}{=}\prod _{i=1}^n𝔼\left[e^{t{\tilde{X}}_i}\right]=\prod _{i=1}^n𝔼\left[e^{t{X}_i}\right]$$

as desired. ∎

Definition 7.7 (Generating Polynomial).

It is natural to express a probability distribution $\mu$ over subsets of $[n]$ by its generating polynomial. To do that we consider $n$ variables, $z_1,\mathrm{\dots },z_n$ and write

$$g_{\mu }(z_1,\mathrm{\dots },z_n)=\sum _{S\subseteq [n]}\mu (S)z^S,$$

where $z^S={\prod }_{i\in S}{z}_i$.

For a concrete example, let $B_1,\mathrm{\dots },B_n$ be $n$ independent Bernoulli random variables where $B_i$ has success probability $p_i$. Then, we can write the corresponding generating polynomial as follows:

$$(p_1{z}_1+1-p_1)(p_2{z}_2+1-p_2)\mathrm{\dots }(p_n{z}_n+1-p_n).$$

The following facts about the generating polynomial are straightforward:

Fact 7.8.

Let $\mu$ be a probability distribution over $[n]$ with generating polynomial $g_{\mu }$, then

• $g_{\mu }(\mathrm{𝟏})=1$, i.e., sum of the coefficients of $g_{\mu }$ is 1.

• ${\partial }_i{g}_{\mu }(\mathrm{𝟏})={\mathbb{P}}_{\mu }\left[i\right]$, i.e., the marginals can be deduced by take partial derivatives.

• $i,j$ are negatively correlated if

  $$g(\mathrm{𝟏}){\partial }_i{\partial }_j{g}_{\mu }(\mathrm{𝟏})=\mathbb{P}[i,j]\ge \mathbb{P}\left[i\right]\mathbb{P}\left[j\right]={\partial }_i{g}_{\mu }(\mathrm{𝟏}){\partial }_j{g}_{\mu }(\mathrm{𝟏}).$$

• Say we have two probability distributions ${\mu }_1,{\mu }_2$ over disjoint sets, then the product distribution is the probability distribution with generating polynomial ${\mu }_1\cdot {\mu }_2=g_{{\mu }_1}{g}_{{\mu }_2}$.

• If ${\mu }_1,{\mu }_2$ are pairwise negatively correlated then so is ${\mu }_1\cdot {\mu }_2$.

Next, we explain a few examples of negatively correlated random variables:

Example 1: Observe that any probability distribuition over subsets of size (exactly) one among $n$ objects is negatively correlated, namely

$$p_1{z}_1+p_2{z}_2+\mathrm{\cdots }+p_n{z}_n$$

where ${\sum }_i{p}_i=1$. Following the above fact, product of these distributions are also negatively correlated.

As an application, recall that in lecture 4, we introduced a probability distribution over paths $P\in {𝒫}_i$ connecting the $i$-th terminal pairs $s_i,t_i$ where we chose one path with its probability $y_P$ and we independently run the procedure for every $i$. It follows that the resulting probability distribution over the random variables $Y_P=𝕀\left[P\text{ is chosen}\right]$ is negatively correlated. So, we could have directly apply the Chernoff bound instead of defining a new family of random variables $X_{e,i}=𝕀\left[\text{a path of }{𝒫}_i\text{ going through }e\text{ is chosen}\right]$.

Example 2: Edges of a uniform spanning tree One of the most interesting family of negative correlated probability distributions is the distribution of the set of edges of a uniform spanning tree. Namely, let $G=(V,E)$ be a connected undirected graph; assign a variable $z_e$ to every edge $e\in E$, then $\mu$ is the distribution with the following generating polynomial,

$$g_{\mu }({\{z_e\}}_{e\in E})=\sum _{T\text{ spanning tree}}{z}^T.$$

We will discuss ideas to prove this fact in the next lecture.

7.3 Towards a theory of Negative dependence

One of the ongoing research directions in probability theory is to study under what conditions one can expect negative correlation and and negative association.

Following the above discussion, a natural choice is the reverse of the positive lattice condition, namely negative lattice condition:

$$\mu (A)\mu (B)\ge \mu (A\cup B)\mu (A\cap B),\forall A,B\subseteq [n]$$

Unfortunately, it can be seen that this property does not even imply a pairwise negative correlation property:

Example 7.9.

Consider the distribution $\mu$ over $[4]$ with the following generating polynomial,

$$\frac{1}{2}(z_1{z}_2+z_3{z}_4).$$

This distribution satisfies the NLC but it not negative correlated as $\mathbb{P}[1,2]=0.5>0.25=\mathbb{P}\left[1\right]\mathbb{P}\left[2\right]$.

In the next lecture we will introduce strongly Rayleigh distribution as a generic method to study negative dependence.