Structure light modeling is a new method to get 3D model using simple tools: camera and projector.
The projector emits various kinds of light stripes or patterns so that the reflection of light in objects are captured and the depth of objects can be calculated.
We want to get a complete 3D model, which means we need to capture for several times in each direction so that the pieces could be merged together.
Thanks for Brown Univ mesh group, we set up a similar experiment environment to get 3D model.
Tools we use :
Laptop, camera(Logitech C920,resolution 2304*1536), projector(Asus S1,resolution 854*480), toy models, chessboard(for calibration), rotating plate(controlled by Arduino using serial port)
Win7+QT4.8.4+VS 2010+OpenCV2.4.9 for compiling QT user inteface
Win10+OpenCV1.0+VS 2010 for compiling capture and calibrate program
Win10+OpenCV2.4.9+PCL 1.7.2+VS 2013 for compiling merge and visualization program
Win 10+Arduino 1.6.5 for compiling and downloading turing program in Arduino
Win 10+VS 2010 for compiling serial port communication to control rotating plate
The first step is to calibrate for the projector and camera.
The image captured and the screen projected is not in ideal flat and square size, especially the margin of the view. So the calibration is essential for the high accuracy.
The projector projects a set of patterns to the chessboard for at least three times, each time the board need to change a position.
The placement of these tools(captured in our common room under dorm)
The second step is to capture objects in structure light. The light pattern is the same.
The stripe is twisted on the object, so the depth info can be calculated
The stripe is horizontal then vertical, and become denser and thiner each time until the lines cannot be identified.
With a set of 42 pictures captured for each light pattern, the point cloud is generated.
Sample point cloud file test.rar
The result shown in MeshLab, a software to view and edit point cloud files
The third step is to capture the object in all rotation, so that the complete 3D model can be acquired.
A part of common areas should be used as reference, so we cannot just capture front and back, just two positions of the same object. In our test, we use 30 degree as a rotation, so 12 positions are captured in the way similar to step 2.
But before the merge, preprosessing is needed. The background board should be removed and noise points should be filtered. The passthrough filter is used to remove points whose depth is larger than a threshold, and the statistical filter is used to remove points whose closest neighbours is larger than another threshold.
After that, ICP algorithm is used to calculate the relations between two position and merge them together.
Now we get a pretty nice point cloud, but in order to make the surface smooth or generate an 'std' file for 3D printing, Greedy Triangulation is used for smooth visualization.
Sample point cloud merged together: RegisteredFiltered.rar
After mergence(download attachment above to see all it in all directions)
Based on the open source code given by Brown group, we tailored our need to re-design and implement the software again.
The project consists of one user interface and several command line executable programs, using buttons to connect together.
Mainly based on QT and use QProcess to link the user interface and the command line program.
All the source:
Noted that in bin folder, many exe is not executable other than my environment and should be recompiled.
How to use
First click 'Calibrate' for initial setup(do not need again unless the position of camera and projector is changed)
Click 'capture' to start capturing. Adjust camera and projector gain then press any key in the window.
Spin the number to adjust the rotating plate and click 'turn'.
Capture again until rotated all the degree.
Click merge after several merge.
Pay much attention to xml files in the bin folder. Understand and adjust them correctly before you click any button. 'config.xml' for calibrate and capture, 'config2.xml' for merging.
NOTE for Windows user
The complilation of the software uses QT, and there are many version of QT available.
In my attempt, I tried many of the QT versions, and the only workable one is like what said in INSTALL.txt in the source code package.
---- Microsot Windows 7 - Visual Studio 2010 ---- 1) Install Qt (from binaries) Download http://releases.qt-project.org/qt4/source/qt-win-opensource-4.8.4-vs2010.exe Install to C:\Qt\4.8.4 Add C:\Qt\4.8.4\bin to PATH 2) Install OpenCV 2.4.3 (from binaries) Download http://sourceforge.net/projects/opencvlibrary/files/opencv-win/2.4.3/OpenCV-2.4.3.exe Extract/Install to C:\opencv\opencv-2.4.3 Add C:\opencv\opencv-2.4.3\opencv\build\x86\vc10\bin to PATH 3) Build scanning software cd scan3d-capture/build qmake nmake release
Windows 10 can complile but the program went wrong when capturing and opening new directory.
So I personaly use VMWare and install exactly the same software and platform as instructed.
Opencv 2.4.3 is not online now but 2.4.9 is also workable(I have tested)
Use Qt 4.8.4 command prompt instead of QT creator in higher version(I cannot compile successfully).
Use the same version of libs in all components, in this case, I use Visual Studio 2010, so Qt and Opencv should also be this 2010 version
When installing PCL package, I highly recommend using all in one installer. The official version of all in one installer is only in 1.6, a website for higher version:http://unanancyowen.com/?p=1255&lang=en
I tried to compile the source code of 1.7.1 by Visual Studio 2010, I tried to compiled and installed the PCL with success, but when using such PCL in project, I failed to compile my project using PCL and Opencv at the same time.
Build projects of PCL with official instruction http://pointclouds.org/documentation/tutorials/using_pcl_pcl_config.php#using-pcl-pcl-config , but still I need to add many libs in Visual Studio project. The complete list of libs I use in compiling PCL projects of merging multiple faces together(Some libs already existed when building the projects, some of them are not needed).
kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib D:\Program Files (x86)\PCL 1.7.2\3rdParty\Boost\lib\libboost_system-vc120-mt-gd-1_55.lib D:\Program Files (x86)\PCL 1.7.2\3rdParty\Boost\lib\libboost_filesystem-vc120-mt-gd-1_55.lib D:\Program Files (x86)\PCL 1.7.2\3rdParty\Boost\lib\libboost_thread-vc120-mt-gd-1_55.lib D:\Program Files (x86)\PCL 1.7.2\3rdParty\Boost\lib\libboost_date_time-vc120-mt-gd-1_55.lib D:\Program Files (x86)\PCL 1.7.2\3rdParty\Boost\lib\libboost_iostreams-vc120-mt-gd-1_55.lib D:\Program Files (x86)\PCL 1.7.2\3rdParty\Boost\lib\libboost_mpi-vc120-mt-gd-1_55.lib D:\Program Files (x86)\PCL 1.7.2\3rdParty\Boost\lib\libboost_serialization-vc120-mt-gd-1_55.lib D:\Program Files (x86)\PCL 1.7.2\3rdParty\Boost\lib\libboost_chrono-vc120-mt-gd-1_55.lib D:\Program Files (x86)\PCL 1.7.2\lib\pcl_common_debug.lib D:\Program Files (x86)\PCL 1.7.2\lib\pcl_features_debug.lib D:\Program Files (x86)\PCL 1.7.2\lib\pcl_io_debug.lib D:\Program Files (x86)\PCL 1.7.2\lib\pcl_io_ply_debug.lib D:\Program Files (x86)\PCL 1.7.2\lib\pcl_kdtree_debug.lib D:\Program Files (x86)\PCL 1.7.2\lib\pcl_keypoints_debug.lib D:\Program Files (x86)\PCL 1.7.2\lib\pcl_ml_debug.lib D:\Program Files (x86)\PCL 1.7.2\lib\pcl_outofcore_debug.lib D:\Program Files (x86)\PCL 1.7.2\lib\pcl_people_debug.lib D:\Program Files (x86)\PCL 1.7.2\lib\pcl_recognition_debug.lib D:\Program Files (x86)\PCL 1.7.2\lib\pcl_registration_debug.lib D:\Program Files (x86)\PCL 1.7.2\lib\pcl_sample_consensus_debug.lib D:\Program Files (x86)\PCL 1.7.2\lib\pcl_segmentation_debug.lib D:\Program Files (x86)\PCL 1.7.2\lib\pcl_stereo_debug.lib D:\Program Files (x86)\PCL 1.7.2\lib\pcl_surface_debug.lib D:\Program Files (x86)\PCL 1.7.2\lib\pcl_tracking_debug.lib D:\Program Files (x86)\PCL 1.7.2\lib\pcl_filters_debug.lib D:\Program Files (x86)\PCL 1.7.2\lib\pcl_search_debug.lib D:\Program Files (x86)\PCL 1.7.2\lib\pcl_io_debug.lib D:\Program Files (x86)\PCL 1.7.2\lib\pcl_octree_debug.lib D:\Program Files (x86)\PCL 1.7.2\lib\pcl_visualization_debug.lib D:\Program Files (x86)\PCL 1.7.2\3rdParty\VTK\lib\vtk-5.10\vtkCharts-gd.lib D:\Program Files (x86)\PCL 1.7.2\3rdParty\VTK\lib\vtk-5.10\vtkViews-gd.lib D:\Program Files (x86)\PCL 1.7.2\3rdParty\VTK\lib\vtk-5.10\vtkInfovis-gd.lib D:\Program Files (x86)\PCL 1.7.2\3rdParty\VTK\lib\vtk-5.10\vtkWidgets-gd.lib D:\Program Files (x86)\PCL 1.7.2\3rdParty\VTK\lib\vtk-5.10\vtkHybrid-gd.lib D:\Program Files (x86)\PCL 1.7.2\3rdParty\VTK\lib\vtk-5.10\vtkVolumeRendering-gd.lib D:\Program Files (x86)\PCL 1.7.2\3rdParty\VTK\lib\vtk-5.10\vtkRendering-gd.lib D:\Program Files (x86)\PCL 1.7.2\3rdParty\VTK\lib\vtk-5.10\vtkGraphics-gd.lib D:\Program Files (x86)\PCL 1.7.2\3rdParty\VTK\lib\vtk-5.10\vtkImaging-gd.lib D:\Program Files (x86)\PCL 1.7.2\3rdParty\VTK\lib\vtk-5.10\vtkIO-gd.lib D:\Program Files (x86)\PCL 1.7.2\3rdParty\VTK\lib\vtk-5.10\vtkFiltering-gd.lib D:\Program Files (x86)\PCL 1.7.2\3rdParty\VTK\lib\vtk-5.10\vtkCommon-gd.lib D:\Program Files (x86)\PCL 1.7.2\3rdParty\VTK\lib\vtk-5.10\vtksys-gd.lib ws2_32.lib opencv_ml249d.lib opencv_calib3d249d.lib opencv_contrib249d.lib opencv_core249d.lib opencv_features2d249d.lib opencv_flann249d.lib opencv_gpu249d.lib opencv_highgui249d.lib opencv_imgproc249d.lib opencv_legacy249d.lib opencv_objdetect249d.lib opencv_ts249d.lib opencv_video249d.lib opencv_nonfree249d.lib opencv_ocl249d.lib opencv_photo249d.lib opencv_stitching249d.lib opencv_superres249d.lib opencv_videostab249d.lib opencv_objdetect249.lib opencv_ts249.lib opencv_video249.lib opencv_nonfree249.lib opencv_ocl249.lib opencv_photo249.lib opencv_stitching249.lib opencv_superres249.lib opencv_videostab249.lib opencv_calib3d249.lib opencv_contrib249.lib opencv_core249.lib opencv_features2d249.lib opencv_flann249.lib opencv_gpu249.lib opencv_highgui249.lib opencv_imgproc249.lib opencv_legacy249.lib opencv_ml249.lib
In all, compiling code in Windows is very tough, so it is recommeded using Linux platform where using QT is with much convenience.