Object Modeling

SimTrack uses textured meshes for pose tracking and provides a simple tool to supplement these meshes with SIFT keypoints for pose detection.

The geometry and appearance of richly textured objects can be obtained with relative ease using Autodesk 123D Catch (preferably the Windows application). Autodesk provides extensive tips for a successful shoot. In summary you’ll want to:

  • disable the camera’s auto-exposure and flash
  • place the object on a newspaper (as below) to facilitate camera pose estimation
  • take about 40 pictures in total, moving around the object in small steps in one high and one low loop (see the recovered camera positions below)

Upload the pictures through the Windows application and wait for them to be processed. If all goes well, it should return an initial model similar to this:

01_123dcatch_initial_model

Click on the pictures one by one to ensure that they correctly map onto the reconstructed model. Either remove or manually stitch grossly misaligned images. Although typically not necessary, this step is critical to avoid blurry textures.

The initial model contains too much of the background. Use the Lasso Selection tool to select the object itself.

02_123dcatch_select_area_of_intrest

Re-process the mesh at maximum quality.

03_123dcatch_increase_mesh_resolution

The next step consists of introducing an absolute distance measure to the scene. This requires identifying two reference points on the model for which the real-world distance can be established easily. Use F4 to create a new point.

04_123dcatch_create_reference_point_1

This point needs be marked in a number of pictures (e.g. 3) in order to triangulate its position on the model.

05_123dcatch_create_reference_point_2

Repeat this procedure for a second reference point and select Define Reference Distance.

06_123dcatch_define_reference_distance_1

Connect both points and enter the absolute distance in centimeter. 123D Catch only allows one decimal so we’ll convert this to meter later on.

07_123dcatch_define_reference_distance_2

Export the scene as Wavefront OBJ-file. This concludes the 123D part.

08_123dcatch_export_wavefront_obj

Next, we’ll remove the supporting surface from the object. This is straightforward with Meshlab. Rotate the object as below, and select the bottom region using the Select Faces in a rectangular region tool.

09_meshlab_cut_surface_1

Press delete to remove the faces. Rotate and repeat until only the object remains.

10_meshlab_cut_surface_2

Export the mesh as Wavefront OBJ.

11_meshlab_export

Next, we’ll use Blender to reposition the object. This step is not critical but it simplifies the interpretation of the estimated pose. For example, the object can be translated and rotated until the front view looks like this:

12_blender_reposition_reorient

SimTrack and ROS use meter as unit of length so we’ll need to rescale the dimensions.

13_blender_scale_to_meter

The object is now ready for use, but pose reliability evaluation can be improved by adding a surface to close the bottom of the (now bottomless) object. Add a plane and position it to intersect the bottom.

14_blender_add_surface_1

Add a boolean modifier to the object and intersect with the plane.

15_blender_add_surface_2

After removing the plane, the object has a bottom. The texture of this bottom will be somewhat random but this will help to signal the tracker that tracking has failed.

16_blender_add_surface_3

Export the model as Wavefront OBJ.

17_blender_export_1

Make sure you Include Normals and Triangulate Faces!

18_blender_export_1

Open the generated .mtl-file and remove the path from the map_Kd options.

map_Kd campbells_tex_0.jpg

Store all three files (.obj, .mtl, .jpg) in a folder with the same name as the .mtl and .obj files.

campbells/
   campbells.obj
   campbells.mtl
   campbells_tex_0.jpg

Perform a final check by opening the .obj-file in MeshLab and verify that the texture is visible.

Next, generate the SIFT-model using SimTrack:

rosrun interface cmd_line_generate_sift_model <path_to_obj_file>

Your object is now ready for pose detection and tracking!

19_simtrack