Announcements

	Project 4 questions?
	Review thisThursday
		Bring your questions!
	Final Exam
		10:30-12:20pm,  Thursday, Mar. 20
	Evaluations today at the end of class

Texture Synthesis

Texture

	Today’s Reading
		Alexei A. Efros and Thomas K. Leung, “Texture Synthesis by Non-parametric Sampling,” Proc. International Conference on Computer Vision (ICCV), 1999.
			http://www.cs.berkeley.edu/~efros/research/NPS/efros-iccv99.pdf

Modeling Texture

	What is texture?
	How can we model it?

Markov Chains

	Markov Chain
		a sequence of random variables
		is the state of the model at time t
		Markov assumption:  each state is dependent only on the previous one
			dependency given by a conditional probability:
		The above is actually a first-order Markov chain
		An N’th-order Markov chain:

Markov Chain Example:  Text

	“A dog is a man’s best friend. It’s a dog eat dog world out there.”

Text synthesis

	Create plausible looking poetry, love letters, term papers, etc.
	Most basic algorithm
		Build probability histogram
			find all blocks of N consecutive words/letters in training documents
			compute probability of occurance
		Given words
			compute          by sampling from
	Example on board...

[Scientific American, June 1989, Dewdney]

	“I Spent an Interesting Evening Recently with a Grain of Salt”
	- Mark V. Shaney
	(computer-generated contributor to UseNet News group called net.singles)
	You can try it online here: http://www.yisongyue.com/shaney/

Modeling Texture

	What is texture?
		An image obeying some statistical properties
		Similar structures repeated over and over again
		Often has some degree of randomness

Markov Random Field

Texture Synthesis [Efros & Leung, ICCV 99]

	Can apply 2D version of text synthesis

Synthesizing One Pixel

		What is                                                                           ?
		Find all the windows in the image that match the neighborhood
			consider only pixels in the neighborhood that are already filled in
		To synthesize x
			pick one matching window at random
			assign x to be the center pixel of that window

Really Synthesizing One Pixel

Growing Texture

Window Size Controls Regularity

More Synthesis Results

More Results

Failure Cases

Image-Based Text Synthesis

Extrapolation

Speed

	Given:  image of k2 pixels
	Output:  image of n2 pixels
	how many window comparisons does this algorithm require?

Slide 22

Slide 23

Slide 24

Fill Order

	In what order should we fill the pixels?

Fill Order

	In what order should we fill the pixels?
		choose pixels that have more neighbors filled
		choose pixels that are continuations of lines/curves/edges

Exemplar-based Inpainting demo

More on Image Inpainting

	Can also be formulated as image diffusion
	Idea of propagating along lines comes from
		Bertalmío, Sapiro, Caselles, and Ballester, “Image Inpainting,” Proc. SIGGRAPH 2000.

Image Inpainting

Image Inpainting

Image Inpainting

Texture Transfer  [Efros & Freeman 2001]

Slide 33

Slide 34

Slide 35

Combining two images

Slide 37

Graph cut setup

Graph cut texture synthesis:  Video

Image Analogies (Hertzmann ’01)

Artistic Filters

Texture-by-numbers

Colorization

References

		Efros and Leung, “Texture Synthesis by Non-parametric Sampling,” Proc. ICCV, 1999.
		Efros and Freeman, “Image Quilting for Texture Synthesis and Transfer,” Proc. SIGGRAPH 2001.
		Bertalmío, Sapiro, Caselles, and Ballester, “Image Inpainting,” Proc. SIGGRAPH 2000.
		Criminisi, Perez, and Toyama. “Object Removal by Exemplar-based Inpainting,” Proc. CVPR, 2003.
		Kwatra, Schödl, Essa, Turk, and Bobick, “Graphcut Textures: Image and Video Synthesis Using Graph Cuts,” Proc. SIGGRAPH 2003.
		Hertzmann, Jacobs, Oliver, Curless, and Salesin, “Image Analogies,” Proc. SIGGRAPH 2001.
		Bhat, Seitz, Hodgins, Khosla, “Flow-Based Video Synthesis and Editing,” Proc. SIGGRAPH 2004.