Announcements

		Guest lecture today:  Aseem Agarwala
		Final project out today
			you and your partner must submit a proposal by this Friday
	Today’s Reading
		Alexei A. Efros and Thomas K. Leung, “Texture Synthesis by Non-parametric Sampling,” Proc. International Conference on Computer Vision (ICCV), 1999.
			http://www.cs.berkeley.edu/~efros/research/NPS/efros-iccv99.pdf

Modeling Texture

	What is texture?
	How can we model it?

Markov Chains

	Markov Chain
		a sequence of random variables
		is the state of the model at time t
		Markov assumption:  each state is dependent only on the previous one
			dependency given by a conditional probability:
		The above is actually a first-order Markov chain
		An N’th-order Markov chain:

Markov Chain Example:  Text

	“A dog is a man’s best friend. It’s a dog eat dog world out there.”

Text synthesis

	Create plausible looking poetry, love letters, term papers, etc.
	Most basic algorithm
		Build probability histogram
			find all blocks of N consecutive words/letters in training documents
			compute probability of occurance
		Given words
			compute          by sampling from
	Example on board...

[Scientific American, June 1989, Dewdney]

	“I Spent an Interesting Evening Recently with a Grain of Salt”
	- Mark V. Shaney
	(computer-generated contributor to UseNet News group called net.singles)

Modeling Texture

	What is texture?
		An image obeying some statistical properties
		Similar structures repeated over and over again
		Often has some degree of randomness

Markov Random Field

Texture Synthesis [Efros & Leung, ICCV 99]

	Can apply 2D version of text synthesis

Synthesizing One Pixel

		What is                                                                           ?
		Find all the windows in the image that match the neighborhood
			consider only pixels in the neighborhood that are already filled in
		To synthesize x
			pick one matching window at random
			assign x to be the center pixel of that window

Really Synthesizing One Pixel

Growing Texture

Window Size Controls Regularity

More Synthesis Results

More Results

Failure Cases

Image-Based Text Synthesis

Speed

Slide 19

Slide 20

Slide 21

Slide 22

Texture Transfer

Slide 24

Slide 25

Slide 26

Issues

	Imposes artificial grid of overlapping blocks on synthesized image, and greedily chooses blocks in left-right, top-bottom order
	Dynamic programming limits applicability to related problems.
	Solution: use graph cuts instead
	Let’s explore two examples, first.

Combining two images

Graph cut setup

Spatio-temporal texture synthesis

Graphcut textures (Kwatra ’03)

Progressive Refinement

Comparison

Image Analogies (Hertzmann ’01)

Slide 35

Artistic Filters

Texture-by-numbers

Other applications of Image Analogies

	Texture synthesis
	Super-resolution
	Texture transfer
	Image colorization
	Simple filters (blur, emboss)
	More details
		http://mrl.nyu.edu/projects/image-analogies/

Applications of Texture Modeling

	Super-resolution
		Freeman & Pasztor, 1999
		Baker & Kanade, 2000
	Image/video compression
	Texture recognition,
	segmentation
		DeBonet
	Restoration
		removing scratches, holes, filtering
		Zhu et al.
	Art/entertainment