COMP61342 : Computer Vision

MSc Module part of Taught Postgraduate Programmes


Matlab version: /opt/matlab-r2009a/
Tutorial directory: /opt/info/courses/COMP61342/matlab/utvisToolbox/tutorials/

Tutorial 1: Manual Calibration and Volumetric Reconstruction

(Calibration Toolbox Webpage: http://www.vision.caltech.edu/bouguetj/calib_doc/)

This tutorial demonstrates simple volumetric reconstruction (from two views) using manually calibrated cameras. The student will be given a set of images to use to calibrate the intrinsic and extrinsic parameters of the cameras. A set of "bad" calibration images are also provided, student can compare the differences in calibration results and volumetric reconstructions.

The script first performs background segmentation in HSV space using simple thresholding. It then samples 3D points at regular intervals and projecting them onto the image planes of the cameras. A 3D point is accepted as part of the object if it is classified as foreground in every views. A simple colour matching algorithm is then used to determine the points that are on the object's surface and also their colours.

WHAT TO DO:

1. add /opt/info/courses/COMP61342/matlab/utvisToolbox/tutorials/SIFTtutorial/TOOLBOX_calib to matlab path

2. If you have the necessary space, copy the calibration images in the following directory to your home directory (this just makes it easier, you don't have to). If you don't have enough space in your home directory, the copy the images to the harddrive (.e.g /tmp) and then work from in there. You will need to add write permission to your copy of these directories.

/opt/info/courses/COMP61342/matlab/utvisToolbox/tutorials/SIFTtutorial/calibration_images/good_intrinsic_images
/opt/info/courses/COMP61342/matlab/utvisToolbox/tutorials/SIFTtutorial/calibration_images/bad_intrinsic_images
/opt/info/courses/COMP61342/matlab/utvisToolbox/tutorials/SIFTtutorial/calibration_images/extrinsic_images

the first two directories contains images for calibrating the intrinsic camera parameters fc (focal lengths), cc (principal point) and kc (distortion).
the last directory contains images for calibrating the extrinsic camera parameters Tc_ext (translation) and Rc_ext (rotation)

3. perform manual calibration (with the good_intrinsic_images and extrinsic images) using the toolbox provided in /opt/info/courses/COMP61342/matlab/utvisToolbox/tutorials/SIFTtutorial/TOOLBOX_calib. You may wish to assume that the focal length in x and y are the same and that the principal point is at the centre of the image space, by setting the variables est_aspect_ratio and center_optim to 0 before optimising the calibration.

for each camera, first calibrate intrinsic parameters (extract corners clockwise, grid size is 29mm), then calibrate extrinsic parameters, save the calibration parameters for each camera as a matlab struct, e.g.

cam0.fc = fc;
cam0.cc = cc;
...
save cam0Params.mat cam0

4. edit tutoria1.m to point to your saved calibration parameters

5. run tutorial1.m, which saves two text files object_points.mat and objects_colours.mat which defines the reconstructed surface of the object.

6. Visualise the results using the show_mug program in Matlab.
The show_mug program can be found in /opt/info/courses/COMP61342/matlab/utvisToolbox/tutorials/SIFTtutorial/files_to_copy .

7. repeat from step 3 to step 6 using the bad_intrinsic_images and compare the reconstruction results

Tutorial 2 & 3: SIFT

SIFT tutorial pdf: /opt/info/courses/COMP61342/matlab/utvisToolbox/tutorials/SIFTtutorial/SIFTtutorial.pdf
or click here

1. copy tutorial2.m and tutorial3.m from /opt/info/courses/COMP61342/matlab/utvisToolbox/tutorials/SIFTtutorial/files_to_copy to your home directory.

2. chmod +w tutorial2.m tutorial3.m

3. start matlab and add /opt/info/courses/COMP61342/matlab/utvisToolbox/tutorials/SIFTtutorial/files_to_copy to your path

4. run tutorial2.m

5. run tutorial3.m

Tutorial 5: Eigen Eyes

This tutorial is located at /opt/info/courses/COMP61342/matlab/utvisToolbox/tutorials/eigenTut/tutorial5.m

1. The aim of this tutorial is to detect and extract a pair of eyes from the input image, using a learnt eigen eye model. At the start of the tutorial script, the model is loaded as the following variables:

dcIm, a constant intensity image
meanIm, the normalised mean eye image
basisIm, the set of eigen eye images (column-wise)
varIm, variance over training images

(NOTE, each eye image is originally of size 25x20, later reshaped into a 500x1 vector for convenience, you can use the reshape function to convert a 25x20 image to a 500x1 vector and vice versa)

2. The task is to locate both eyes in the test image using the learnt eigen eye model. This can be done by scanning a 25x20 window over all the pixels in the test image, and computing its mean coefficient and out-of-subspace statistic (OOS), using the first N (e.g. 5) eigen images, this can be computed as follows:

a0 = dcIm'*test_image_patch
meancoeff = meanIm'*test_image_patch, // mean coefficient

a2 = basisIm(:,1:N)'*test_image_patch
e = test_image_patch - dcIm*a0 - meanIm*meancoeff - basisIm(:, 1:N)*a2;
OOS^2 = sum(e.^2./varIm)/500; // OOS squared

3. Given meancoeff and OOS, whether the test image patch contains an eye can be determined by applying the following threshold:

(meancoeff > tol1) and (atan2(OOS, meancoeff) < tol2)
300 and 0.05 are reasonable guesses for tol1 and tol2, respectively.

Finally, produce a binary mask which can be used to segment the eyes from the test images.

4. You can further enhance the result by testing for a pair of eyes; for each candidate eye pixel, this can be tested by checking that there is another candidate eye pixel roughly 40 pixels to the left or the right of it. Further improvement and speedup can be obtained by limiting the eye detection to areas in and around skin coloured regions.