Texture Synthesis

Texture

	Today’s Reading
		Alexei A. Efros and Thomas K. Leung, “Texture Synthesis by Non-parametric Sampling,” Proc. International Conference on Computer Vision (ICCV), 1999.
			http://www.cs.berkeley.edu/~efros/research/NPS/efros-iccv99.pdf

Modeling Texture

	What is texture?
	How can we model it?

Markov Chains

	Markov Chain
		a sequence of random variables
		is the state of the model at time t
		Markov assumption:  each state is dependent only on the previous one
			dependency given by a conditional probability:
		The above is actually a first-order Markov chain
		An N’th-order Markov chain:

Markov Chain Example:  Text

	“A dog is a man’s best friend. It’s a dog eat dog world out there.”

Text synthesis

	Create plausible looking poetry, love letters, term papers, etc.
	Most basic algorithm
		Build probability histogram
			find all blocks of N consecutive words/letters in training documents
			compute probability of occurance
		Given words
			compute          by sampling from
	Example on board...

[Scientific American, June 1989, Dewdney]

	“I Spent an Interesting Evening Recently with a Grain of Salt”
	- Mark V. Shaney
	(computer-generated contributor to UseNet News group called net.singles)

Modeling Texture

	What is texture?
		An image obeying some statistical properties
		Similar structures repeated over and over again
		Often has some degree of randomness

Markov Random Field

Texture Synthesis [Efros & Leung, ICCV 99]

	Can apply 2D version of text synthesis

Synthesizing One Pixel

		What is                                                                           ?
		Find all the windows in the image that match the neighborhood
			consider only pixels in the neighborhood that are already filled in
		To synthesize x
			pick one matching window at random
			assign x to be the center pixel of that window

Really Synthesizing One Pixel

Growing Texture

Window Size Controls Regularity

More Synthesis Results

More Results

Failure Cases

Image-Based Text Synthesis

Speed

	Given:  image of k2 pixels
	Output:  image of n2 pixels
	how many window comparisons does this algorithm require?

Slide 20

Slide 21

Slide 22

Texture Transfer

Slide 24

Slide 25

Slide 26

Combining two images

Slide 28

Graph cut setup

Graph cut texture synthesis:  Video

Image Analogies (Hertzmann ’01)

Artistic Filters

Texture-by-numbers

Other applications of Image Analogies

	Texture synthesis
	Super-resolution
	Texture transfer
	Image colorization
	Simple filters (blur, emboss)
	More details: Hertzmann et al., SIGGRAPH 2001
		http://mrl.nyu.edu/projects/image-analogies/

Applications of Texture Modeling

	Super-resolution
		Freeman & Pasztor, 1999
		Baker & Kanade, 2000
	Image/video compression
	Texture recognition,
	segmentation
		DeBonet
	Restoration
		removing scratches, holes, filtering
		Zhu et al.
	Art/entertainment