DAVID structured light scanners

[mm_alex]

hmm. the forum ate my 8k zip file! maybe I'll try again. sorry for double post. hopefully this has the code.

[johnrpm]

Well I think you have done a wonderful job, to go from a standing start and get results in such a short time is remarkable, and although I am as thick as two planks I do know people who are working in this area, I could ask for papers if you let me know what to ask for.

[azkuene]

Wow, great job!!
How are you going to synchronize the projector and camera??

[Bongobat]

Hi again mm.alex

Your quick work has turned me from a skeptic into a believer. I was hoping that perhaps you could do me a favor and briefly explain the whole process you used in laymen's terms. I read the paper's available and even read up on DLP technology and still cannot figure out how exactly 3d data is being derrived from the fringe patterns. Does phase refer to the phase of a light wave? What exactly is a fringe pattern? and etc. Basically I need you to avoid using too much math lol. Thanks again for your terrific work and I hope you continue the development.

[mm_alex]

hello bongobat. I've implemented less than you imagine! not having any access to a DLP projector, I haven't made any sync or mods. That stuff is required to make a projector capable of projecting over 180fps greyscale images. the fundamental scanning technique takes 3 still images and turns them into a 3d model; if you mod a DLP projector and sync a video camera you can therefore capture 3d models at 180/3=60 frames per second. this is also made possible by the fact that the algorithm to reconstruct the model from the images is simple enough to run in 1/60th of a second. this is also the clue: if it's simple enough to run in 1/60th, it's simple enough to write in a few hours
so long as you can create 3 images of your object without object or camera moving, and where you have projected 3 images of grayscale horizontal lines (or vertical) you can reconstruct the mesh. the idea is similar to david laser scanning; you are looking for the vertical deformation of a known straight line (the laser line). however instead of projecting/measuring one line at a time, you project a whole image of lines. further, rather than looking for a bright line on a dark ground, you look for the vertical shift of a known pattern (in this case, horizontal stripes that you know are in a sine-wave pattern, also known as fringes due to their ability to be constructed using interference patterns; but in this case I just use a projector). in theory you can do this with a single image, and see the deformation of the lines up and down (you can see it in the attached images from my last post). however if you project 3 images with the sinewaves shifted ('scrolled') by 1/3 of a stripe each time, then you can use some simple maths such that each pixel can be computed independently. this maths tells you where on the wave (crest, or high part, or low part, whever) the pixel lies. this is the phase! once you know where you lie 'within' a particular stripe, you have to be able to tell one stripe from another (ie, all stripes look the same, so how do I know if a given pixel is from the first one, or an offset version of the 2nd? etc). this is the unwrapping process. my code does this unwrapping step by walking pixel by pixel around the image, measuring the 'confidence' it has in whether one pixel relates to the same stripe as its neighbour. generally, it can be confident enough to fill in all the pixels in relation to the neighbour, at least for objects without too much self occlusion.
all of this relies on reliably measuring where you are in the stripe, it accurately measuring the greyscale image and relating it to the original sinewave pattern that was sent to the projector; any distortion of brightness (not shape! just brightness) leads to ridges.

there are many other more simple structured light approaches, but they either require more complex code, or more input images to generate a scan. the phase-based stuff seemed like a sweet spot between complexity of setup, and complexity of code. that said, until I write some calibration code to remove the ridges successfully, it's not worth much. a simpler approach requiring less calibration could (for example) extract a 256x256 mesh by using a sequence of 16 projected binary images.

[azkuene]

Thanks for the explanation
If you don't want to make real-rime scan, you can project the three fringes in just one RGB image and only take one picture. Then extract each projected fringe from each channel, avoiding movements between pictures.

[mm0zct]

The disadvantage to the 3 channel split is where 3 images of white bands would work to a degree on object with coloured patches, the rgb split would not, as, for example, the blue channel would just be completely black on a red part of the object.

As for the bands in the scans (nice work by the way) it seems to me that you're code thinks the darker stripes are further/nearer than the lighter stripes, which even when averaged out over the 3 images could still leave some residual effect. If i get the chance i might look into this this holiday too

(sorry i haven't looked at the code yet but does your triangulation algorithm definitely allow for the darker bands being futher than the lighter ones or have you accidentally used the value of the brightest bit for that entire fringe? this could result in the artifacts produced)

[mm_alex]

azkuene: it's as mm0zct says, and even worse! the phase shifting relies on comparing the relative brightnesses of *each pixel* between the 3 images. so it relies on the surface reflectance of the surface at each point to be the same for all 3 fringe images. thus it will only work as you describe if the object is perfectly white and you've calibrated your camera's whitepoint perfectly.
this sensitivity to exact brightnesses is what makes the phase technique tricky but powerful: the depth value isn't just extracted from the highs (or lows) of the 32 (or whatever) projected stripes, but from *every pixel* of the image. in theory you could project one gigantic wide stripe and still get the same mesh resolution! (would look like a gradient). however the stripe width trades off brightness-resolution for uncertainty in the unwrapping process....
mm0zct, you're roughly right about the dark areas appearing 'further away'. I'm incorrectly assuming the projector and camera are linear (ie brightness sinewave into projector -> surface -> camera -> pc I still get a sinewave. I don't!) I need to implement something like chapter 6 of zhang's phd; however I prefer something simpler where you scan a known flat surface and then build a lookup table of the errors for later scans.
however I won't have time to look at this for a few weeks now oh well.

in the meantime, I'm going to blog progress on my company's blog which just went live. that way I can also avoid filling this forum too much more with my ramblings. see:

www.mediamolecule.com/blog

[Bongobat]

"You do not really understand something unless you can explain it to your
grandmother." - Albert Einstein

Thanks mm_alex! You helped me tremendously to get my brain around this.

[azkuene]

ok, thanks for the explanations!
I'll be watching for your progress

[Salah]

Has anyone tested the freeware : FlexScan3D Lite ?
The address : http://www.reversemodeling.com/try.html
PS : Now I am not a lazy reader only :wink:

[sematik]

I am very interested in this aproach. Anyone knows of an opensource or free/very cheap software able to project the light patterns and obtain a point cloud from it?

Regarding the patterns the Gühring algorithm seems amazing.

[sematik]

BUMP

No new tests??

[hal]

Hi to all... but where are all?

This topic began with so much passion, but ended in nothing...

Where is mm_alex? Where are you gone? A lot of good ideas and then PUFF! All is going down.

I'm sad for this...

bye, Mattia

[Lachlann]

Has anyone tested the freeware : FlexScan3D Lite ?
The address : http://www.reversemodeling.com/try.html
PS : Now I am not a lazy reader only :wink:

Salah, do you have a valid link for that download, or a copy of the file? I don't see the link up on that page anymore.

I'd love to see something like this for us to use. I've been interested in the David scanner for a while, but mm_alex seems to be close to turning out something like FlexScan or Digiscan 2000, which is more suited for what I would like to do (scan people) but are both expensive.