CAMERA LOCALIZATION

USING INCOMPLETE CHESSBOARD PATTERN

Marek Solony, Pavel Zak, Vitezslav Beran and Michal Spanel

Faculty of Information Technology, Brno University of Technology, Bozetechova 2, 612 66 Brno, Czech Republic

Keywords:

Chessboard pattern, Camera localization, Kalman ﬁlter, Tracking, Real-time processing.

Abstract:

This paper introduces the approach for the real-time camera localization by capturing the plane of chessboard

pattern. This task has been already solved by several different approaches, but we present the novel method of

the chessboard reconstruction from its incomplete image, that enables successful camera localization even if

the captured chessboard plane is partially covered by an unknown object. The camera position and orientation

is during the processing of the videosequence tracked with the Kalman ﬁlter that enables correct localization

also in the closeup views on the pattern.

1 INTRODUCTION

The process of automatic camera localization is

a non-trivial task. Usually some auxiliary marks or

patterns featuring easy detectability are used in the

scene to enable the successful localization process.

One of the most common pattern is the chessboard

plane due to its geometric simplicity and good con-

trast visibility.

In this paper we propose the novel approach for

the camera localization using the incomplete chess-

board pattern. This means that there is no need to cap-

ture the whole chessboard plane to reach the proper

localization. The algorithm was designed to deal with

the situation when an unknown object or obstacle is

occluding the chessboard pattern.

2 RELATED WORK

Over the past years numerous different aproaches for

the camera localization and auxiliary marks deﬁni-

tion have been presented. These works include the

description of the marks in the form of the plain

squares (Kato and Billinghurst, 1999), labelled self-

identifying squares (Fiala and Shu, 2008), circular el-

ements ((Ahn et al., 2001), (Forbes et al., 2002)) and

others.

Camera localization using the chessboard pat-

tern have been examined in number of works that

include methods employing Delaunay triangulation

(Shu et al., 2003), characteristics of local intensity

and the grid line architecture of the pattern (Wang

et al., 2010) and mainly the symmetric characteristics

of chessboard corner areas - (Weixing et al., 2009),

(Bevilacqua et al., 2008) and (Ha, 2007).

Unlike other approaches, the method proposed in

(de la Escalera and Armingol, 2010) uses a chess-

board pattern when no information regarding the

number of rows or columns is supplied and the ap-

proach presented in (Bevilacqua et al., 2008) aims for

the ability to detect the chessboard from the incom-

plete set of the inner chessboard corners.

3 SYSTEM DESIGN

This section introduces our approach for the camera

localization using the incomplete chessboard pattern.

The scenario is that there is a planar pad with the

chessboard pattern that is observed by the moving

camera. In each frame of so obtained videosequence,

it is desired to retrieve the position and orientation of

the camera relative to the observed chessboard plane.

The chessboard plane can be partially covered by un-

known object so the crucial part of this process is the

chessboard area detection, reconstruction and follow-

up tracking.

The video processing begins with the initialization

phase, when the chessboard plane is detected and re-

constructed, and the initial camera position and orien-

tation is determined. After the initialization, the track-

415

Solony M., Zak P., Beran V. and Spanel M..

CAMERA LOCALIZATION USING INCOMPLETE CHESSBOARD PATTERN.

DOI: 10.5220/0003329004150418

In Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP-2011), pages 415-418

ISBN: 978-989-8425-47-8

 2011 SCITEPRESS (Science and Technology Publications, Lda.)

ing phase continues with the frame-by-frame updat-

ing of the camera position and orientation, this time

with less restrictions on the view itself. This means

that during the tracking phase the camera can move

close to the chessboard plane and still the proper lo-

calization is possible.

4 THE ALGORITHM

4.1 Image Analysis

The main visual marker, inner corner of the chess-

board pattern, features strong gradient changes and

the neighbourhood that is according to the color (or

intensity) values separable into two distinct groups

arranged into four perspectively distorted rectangles.

This perspective distortion nevertheless does not cur-

vate the lines forming the borders between the chess-

board ﬁelds which is the key feature used in the pro-

cesing.

The detector is based on mentioned properties of

a corner neighbourhood and with slight implemen-

tation changes and enhancements originally comes

from the work (Bevilacqua et al., 2008).

The chessboard reconstruction algorithm (see

Section 4.2) takes as its input the set of chessboard

pattern line segments, which are the lines connecting

two adjacent corner points.

To check whether two corner points lie on the

same line segment, their join is iteratively divided in

halves. At every dividing point, the difference of two

side pixels intensities is evaluated. The side pixels

are found on the normal to the corner join in speciﬁed

distance. The difference of all side pixel intensities

should be in the case of proper line segment alongside

the joint merely the same, as can be seen on Figure 1.

Figure 1: Properties of chessboard corner neighbourhood

(left). Sampling of the line segment (right).

4.2 Chessboard Assembly

In order to compute the position and rotation of cam-

era, the known planar structure of calibration chess-

board and its image in camera’s projection plane can

be exploited. During the extraction, the mutual posi-

tions of the corners are not preserved, so the structure

of detected corners is unknown. We propose an al-

gorithm that focuses on ﬁnding the chessboard grid -

inner vertical and horizontal lines.

Figure 2: The detected line segments form points in Hough

space.

To extract lines of chessboard grid, we exploit the

fact that every two adjacent corners form a line that

passes through other corners of the same row/column.

If at least one such line from each row and column is

detected, the whole chessboard grid can be assembled

in three steps:

1. All detected line segments are transformed into

2D Hough space. This transformation maps a line

into a point such that the lines with same angle

and distance are mapped to the same point (Fig-

ure 2), thus deﬁning the same row or column of

chessboard pattern.

2. The parallel lines lie in the Hough space on

the same line. The chessboard is composed of

two sets of parallel lines - horizontal and verti-

cal, so two sets of collinear points should be in

Hough space, which can be found using RANSAC

(Forsyth and Ponce, 2002) algorithm.

3. Points from each set are sorted, so numbered lists

of vertical and horizontal lines are created. These

lines intersect in points, where the chessboard cor-

ners should lie in image from camera, so the small

areas around intersections are searched for best

corner candidates.

Knowing the structure of chessboard corners, the 2D

to 3D correspondences can be easily found.

4.3 Camera Localization and Tracking

The localization of the camera is a process of ﬁnding

camera’s exact position and rotation in arbitrary world

coordinate frame. The real camera can be approx-

imated with pinhole camera model which describes

relationship between the coordinates of 3D point and

its image in camera’s projection plane.

The mapping between 3D scene points and their

images is deﬁned by equation:

x = A[R| − T ]X (1)

VISAPP 2011 - International Conference on Computer Vision Theory and Applications

416

where vector x represents the image coordinates of

3D point X, matrix A contains intrinsic camera pa-

rameters and joint matrix [R| − T ] represents external

camera parameters - rotation matrix R and translation

vector T , which relate the camera orientation and po-

sition to a world coordinate system.

With known intrinsic camera parameters, the po-

sition and rotation of camera can be estimated using

at least 4 coplanar 3D points and their corresponding

images in camera projection plane. Algorithm such as

Levenberg-Marquadt (Forsyth and Ponce, 2002) can

be used to minimize projection error, which is com-

puted as distance between corners extracted from im-

age, and their projections using estimates of unknown

parameters.

The initialization step is applied only once, next

the camera movement is tracked using Kalman ﬁlter

(Forsyth and Ponce, 2002), because it can cope with

unstable motion of hand-held camera.

Because any 4 non-collinear corners have to be

found to compute new camera positions, the cam-

era motion is not restricted by detecting whole chess-

board, so various angles and camera close-ups are

possible without losing track of camera position.

5 EXPERIMENTS AND RESULTS

The system has been tested on both real and synthetic

data. The test scenes have been captured in the reso-

lution of 640x480 pixels by several cameras featuring

different capture properties and also rendered by 3D

software.

5.1 Stability

The test data were captured to contain intrusive fac-

tors that can signiﬁcantly worsen the image quality

and therefore affect the reliability of the proposed al-

gorithms. These factors include shaky or fast camera

movements, poor scene lighting (insufﬁcient contrast,

strong shadows), image noise, complex background

or an object occluding the chessboard area.

The experiments with different kinds of occlu-

sions proved the ability of correct initialization also in

the cases, when there are false positive detections of

chessboard corners in the object or background area.

Fast or shaky camera movements lead to blurred

videosequences which causes losing of limited num-

ber of detected chessboard corners. Such failures

within short time (several successive frames at most)

are suppresed due to the usage of Kalman ﬁlter.

Very fast camera movements can lead to the unde-

tectable failure when the located chessboard is shifted

and placed on incorrect chessboard lines. This alias-

ing happens when the movement of the chessboard

image between adjacent frames is close or greater

than the size of the pattern squares.

Figure 3: Examples of the chessboard detection in distant

and closeup view. The correctness of the localization is

demonstrated by drawn world coordinate axis and virtual

cube.

Figure 4: Examples of the chessboard detection during the

initialization (left) and tracking phase (right).

5.2 Accuracy

The evaluation of localization accuracy has been

based on synthetic data. In the case of a general cam-

era movement, the information about rotation of the

camera turned out to be more precise than the cam-

era position. The average deviation of the precise and

computed camera position was about the tenth of the

size of chessboard square side, the average deviation

in the rotational space did not exceeded 0.01 radians.

The accuracy of the output is directly affected by

the distance of the camera from the chessboard plane.

The smaller the distance is the more precise is the

computed camera position (see Figure 5). The relative

disparity, that is an average of the measured value di-

vided by the distance of the camera to the chessboard

plane, was smaller than 0.006.

5.3 Performance

The speed of the initialization part depends primarily

on the amount of the inner chessboard corners within

the pattern, the distance of the camera to the chess-

board plane, image resolution setting and background

complexity.

In the case of the chessboard pad with 11x8 inner

corners that was used during the tests, the process-

ing time of single initialization step did not exceeded

CAMERA LOCALIZATION USING INCOMPLETE CHESSBOARD PATTERN

417

Figure 5: The dependance of the computation accuracy to

the distance between camera and chessboard pad (1 unit

equals the size of the chessboard square side).

80ms. The average time of the processing during the

tracking phase was 16 miliseconds which corresponds

to more than 60 frames per second. This period in-

cludes all steps of frame processing, including the

querying for the frame image, camera prediction and

localization.

Processing times were measured on the notebook

having Core 2 Duo processor and 2 GB of RAM.

6 CONCLUSIONS

The approach for the automatic camera localization

and tracking using the chessboard pattern has been

presented. The described approach has been designed

to perform in real-time so it can be used in various

tasks such as augmented reality, user interfaces (the

camera or a small chessboard plane can be used as

a 6 DOF device), 3D scene reconstruction or in com-

bination with object scanner.

The chessboard reconstruction algorithm is rather

original and performs well with the occluded or in-

complete chessboard plane. Nevertheless, the pos-

sibilty of the line skip alias in the tracking phase is

the weak part of the process and could be improved

in future work. This skip is usually caused by quick

camera movements, so increasing the framerate of the

camera could reduce the risk of failure.

ACKNOWLEDGEMENTS

This work has been supported by Security-Oriented

Research in Informational Technology, Czech Min-

istry of Education, Youth and Sports, CEZMSMT

MSM0021630528, Recognition and presentation of

multimedia data, Faculty of Information Technology,

Brno University of Technology, Czech Republic, FIT-

S-10-2, EU project FP7-ARTEMIS R3-COP, grant

no. 100233 and the company 3Dim Laboratory Ltd.

REFERENCES

Ahn, S. J., Rauh, W., and Kim, S. I. (2001). Circular coded

target for automation of optical 3d-measurement and

camera calibration. IJPRAI, 15(6).

Bevilacqua, A., Gherardi, A., and Carozza, L. (2008). Au-

tomatic perspective camera calibration based on an in-

complete set of chessboard markers. In Sixth Indian

Conference on Computer Vision, Graphics Image Pro-

cessing.

de la Escalera, A. and Armingol, J. M. (2010). Automatic

chessboard detection for intrinsic and extrinsic camera

parameter calibration. Sensors, 10(3):2027–2044.

Fiala, M. and Shu, C. (2008). Self-identifying patterns for

plane-based camera calibration. Machine Vision Ap-

plications, 19(4).

Forbes, K., Voigt, A., and Bodika, N. (2002). An inex-

pensive, automatic and accurate camera calibration

method. In In Proceedings of the Thirteenth Annual

South African Workshop on Pattern Recognition.

Forsyth, D. A. and Ponce, J. (2002). Computer Vision:

A Modern Approach. Prentice Hall.

Ha, J.-E. (2007). Automatic detection of calibration mark-

ers on a chessboard. Optical Engineering, 46(10).

Kato, H. and Billinghurst, M. (1999). Marker tracking and

hmd calibration for a video-based augmented reality

conferencing system. In Proceedings of the 2nd IEEE

and ACM International Workshop on Augmented Re-

ality, Washington, USA. IEEE Computer Society.

Shu, C., Brunton, A., Fiala, M., Shu, C., Brunton, A., and

Fiala, M. (2003). Automatic grid ﬁnding in calibra-

tion patterns using delaunay triangulation. Technical

report.

Wang, Z., Wang, Z., and Wu, Y. (2010). Recognition of

corners of planar checkboard calibration pattern im-

age. In Control and Decision Conference (CCDC).

Weixing, Z., Changhua, M., Libing, X., and Xincheng, L.

(2009). A fast and accurate algorithm for chessboard

corner detection. In Image and Signal Processing,

2009. CISP ’09. 2nd International Congress on.

VISAPP 2011 - International Conference on Computer Vision Theory and Applications

418