Photometric Stereo
Project Overview:
For Project 3 we were required to add code to a
program to create a 3D reconstruction of an object from a series of images taken
with different lighting sources. This required several subtasks:
Determine lighting directions -- Given images of a chrome sphere we had to determine the location of our twelve different lighting locations (same for all the pictures).
Determine normals for an object -- Given images of an object with the same lighting locations as #1, determine the objects surface normals for all points within a mask.
Determine albedo of an object -- Given the calculated normals from #2 and the color values from the original series, determine the objects albedo for all points within a mask.
Determine depth (i.e. 3D reconstruction) of an object -- Given the calculated normals from #2, determine the z depth of all points within a mask.
The UI and a lot of the matrix math was implemented for us and we were required to implement the underlying math for #1-#4.
Project Requirements:
For three (3) of the sample images we had to capture snapshots of the follow:
RGB-encoded normals (#2 above)
Needle map (#2 above)
Albedo map (#3 above)
Two or more views of the reconstructed surface without albedos (#4 above)
Two or more views of the reconstructed surface with albedos (#4 above)
These images are presented in the table below. A column represents the series of transformations for one object. Each row is one of the bullets above.
Project Discussion:
Overall the project was much easier than the first
two. I found that we didn't really have to figure out any algorithms or
determine any fast or clever ways of performing calculations since all of that
code was already provided. The actual coding ended up being very short and
just a matter of either a) checking boundary conditions, or b) doing some simple
math as described on the project page. The most challenging part was that
the code seemed to run slowly within Visual Studio (i.e. hitting F5) but fast
without and I couldn't determine why this was. Aside from that I had some
interesting bugs:
In some of my matrix operations I was using += for assignment instead of = with the thought process that I'd sum up any information previously stored there without realizing the matrixes were not initialized. This surprisingly worked within VS as it evidentially zeros out the memory the app uses before running it. However when executing outside of VS the memory was not zeroed and therefore the += picked up undetermined values from memory. Going through the code I realized that for all the matrix positions I only ever set them once (and they're not initialized) so a simple = was the correct answer. The bug was in my code, but the way it showed itself outside of VS but not inside was interesting.
I realized that I had to increment my constraint row variable (n for me) the same in both the makeMmatrix and makeVvector functions. However at first I was doing this by ignoring all pixels who were -1 or whose neighbors were -1. This lead to some situations where a pixel had one neighbor (i.e. right) but not the other (i.e. down) and I would set no constraints. This produced some artifacts on the edges. To fix it I incremented the constraint row variable (n) whenever I had a valid pixel (!= -1) and checked within that if for each direction independently, setting neither, either, or both depending on whether the neighbors existed. This eliminated the artifacts on the right and bottom edges that was different from the sample solution. Note: both my solution and the sample have some artifacts on the left and top which could be fixed by using a symmetrical formula for the edge detection, instead of always left and down. I didn't implement this but the modifications to the constraints equations should be fairly straight forward. The most direct path would be to add in 2 more constraints per pixels for the other two directions.
Project Artifacts:
| Picture | budha | horse | owl |
| RGB Normals |
|
|
|
| Needle Map |
|
|
|
| Albedo Map |
|
|
|
| 3D View #1 |
|
|
|
| 3D View #1 (Albedo) |
|
|
|
| 3D View #2 |
|
|
|
| 3D View #2 (Albedo) |
|
|
|