Stereo SLAM - Politecnico di...

Stereo SLAM

Davide Migliore, PhD [email protected]

Department of Electronics and Information, Politecnico di Milano, Italy

Monday, 15 June 2009

mailto:[email protected]


Slide n°

Davide Migliore

What is a Stereo Camera? ‣Do you remember the pin-hole camera?

2


Slide n°

Davide Migliore

What is a Stereo Camera? ‣Two cameras that perceive the world

- Each camera has a P matrix

3


Slide n°

Davide Migliore


4


Slide n°

Davide Migliore


5


Slide n°

Davide Migliore


6


Slide n°

Davide Migliore

What is a Stereo Camera? ‣Error modeling problem

7


Slide n°

Davide Migliore

Stereo SLAM (Paz et al. 2008)

‣The idea - Use the Unified Inverse Depth parametrization (Montiel et al.

2006)

- Rectify images and initialize the point using

8


Slide n°

Davide Migliore


‣Measurement Equations

9


Slide n°

Davide Migliore


‣Measurement Equations

10


Slide n°

Davide Migliore

Stereo SLAM (Paz et al. 2008)11


Slide n°

PhD Davide Migliore - [email protected]

Classic EKF SLAM 12




Slide n°


Classic EKF SLAM

‣Extended Kalman Filter

12

Video Frame




Slide n°


Classic EKF SLAM


12

Video Frame

Feature

Detection

FD

Feature Initialization

Prediction

Update

SLAM Filter




Slide n°


Classic EKF SLAM


12

Video Frame

Feature

Detection

FD

Data Association

DA

Feature Initialization

Prediction

Update

SLAM Filter




Slide n°

Davide Migliore


‣Data Association Trouble

13


Slide n°

Davide Migliore



14


Slide n°

Davide Migliore



15


Slide n°

Davide Migliore

Compatibility 16


Slide n°

Davide Migliore

NN Data Association 17


Slide n°

Davide Migliore

NN Data Association 18


Slide n°

Davide Migliore

Joint Compatibility 19


Slide n°

Davide Migliore

JCBB 20


Slide n°

Davide Migliore

JCBB 21


Slide n°

Davide Migliore

Demo Time‣Switch on Matlab

22


Slide n°

Davide Migliore


‣Joint Compatibility Branch & Bound Results

23


Slide n°

Davide Migliore


‣Results

24


Slide n°

Davide Migliore

Scaling problem 25


Slide n°

Davide Migliore

Scaling problem 26

O(n2)


Slide n°

Davide Migliore

Solution: local maps 27

‣Switch to matlab again


Slide n°

Davide Migliore


‣Results

28


Slide n°

Davide Migliore


‣Results

29


Slide n°

Davide Migliore

Stereo SLAM (Tomono 2009)

‣Results

30


Slide n°

Davide Migliore


‣Results

31


Slide n°

Davide Migliore


‣Results

32


Slide n°

Davide Migliore

Inverse Scaling?‣ Is it possible to use the inverse scaling?‣Yes

‣Results? Coming soon!!

33


Slide n°

Davide Migliore PhD - [email protected]

Thanks for your attention34




Slide n°



Questions




Omnidirectional SLAM

Davide Migliore, PhD [email protected]

Department of Electronics and Information, Politecnico di Milano, Italy




Slide n°

Davide Migliore

What is an Omni Camera? Omnidirectional sensors come in many varieties, but

by definition must have a wide field-of-view.

~180º FOV

wide FOV dioptric cameras (e.g. fisheye)

~360º FOV

polydioptric cameras (e.g. multiple overlapping cameras)

>180º FOV

catadioptric cameras (e.g. cameras and mirror systems)


Slide n°

Davide Migliore

(Poly-)Dioptric solutions

Pros: - High resolution

per viewing angle

Cons:- Bandwidth

- Multiple cameras

One to two fish-eye cameras or many synchornized cameras


http://www.fxpal.xerox.com/smartspaces/flycam/flycam_home.htm


http://www.cfar.umd.edu/~larson/EyesFromEyes.html


http://www.immersivemedia.com/


http://www.nearfield.com/~dan/Photo/sphere/sphere.htm


http://www.ptgrey.com/products/pands.html


http://www.viewplus.co.jp/products/sos/sos_english/sos_main_english.html


http://www.cs.wustl.edu/~pless/camera.html


Slide n°

Davide Migliore

(Poly-)Dioptric solutionsOne to two fish-eye cameras or many synchornized

cameras

Homebrewed polydioptric cameras are cheaper, but require calibrating and synchronizing; commercial designs tend to be expensive
















Slide n°

Davide Migliore

Catadioptric solutionsUsually single camera combined with convex mirror

Cons:- Blind spots

- Low resolution

Pros: - Single image


http://www.pluto.dti.ne.jp/~accowle1/

http://www.pluto.dti.ne.jp/~accowle1/

http://www.versacorp.com/

http://www.versacorp.com/

http://www.fpvideo.com/

http://www.fpvideo.com/

http://www.remotereality.com/

http://www.remotereality.com/

http://www.eyesee360.com/

http://www.eyesee360.com/

http://www.laas.fr/~simon/eden/rover/perception/pano.php

http://www.laas.fr/~simon/eden/rover/perception/pano.php

http://cmp.felk.cvut.cz/demos/OmnidirectionalVision.html

http://cmp.felk.cvut.cz/demos/OmnidirectionalVision.html

http://w3.sys.es.osaka-u.ac.jp/projects/robot/index-e.html

http://w3.sys.es.osaka-u.ac.jp/projects/robot/index-e.html

Slide n°

Davide Migliore

Camera ModelsPerspective camera

Single effective viewpoint

Image plane (CCD)

40


Slide n°

Davide Migliore

Catadioptric cameras

Camera Models


Slide n°

Davide Migliore

Catadioptric cameras• mirror

Camera Models


Slide n°

Davide Migliore

Catadioptric cameras• mirror• perspective camera

Camera Models


Slide n°

Davide Migliore

Central catadioptric cameras

• mirror

• camera

Camera Models


Slide n°

Davide Migliore


• mirror

• camera

• single effective viewpoint

Camera Models


Slide n°

Davide Migliore


• mirror

• camera

• single effective viewpoint

(surface of revolution of a conic)

Camera Models


Slide n°

Davide Migliore

F1

F2

Types of central catadioptric cameras 43


Slide n°

Davide Migliore

• hyperbola + perspective camera

F1

F2



Slide n°

Davide Migliore

• hyperbola + perspective camera• parabola + orthographic lens

F1

F2

F1



Slide n°

Davide Migliore

• hyperbola + perspective camera• parabola + orthographic lens

• ...F1

F2

F1



Slide n°

Davide Migliore

Other types of central cameras 44


Slide n°

Davide Migliore

u

v

X

Y

Z

p =

Why do we need calibration? 45


Slide n°

Davide Migliore

u

v

X

Y

Z

p =

Calibration gives the relation between 2D & 3D

For each pixel → 3D vector emanating from the

single viewpoint

Why do we need calibration? 45


Slide n°

Davide Migliore

u

v

X Y

Z

What?


Slide n°

Davide Migliore

u

v

X Y

Z

• Center of the omnidirectional image

What?


Slide n°

Davide Migliore

u

v

X Y

Z

• Center of the omnidirectional image • Camera focal length

Focal length

What?


Slide n°

Davide Migliore

u

v

X Y

Z

• Center of the omnidirectional image • Camera focal length• Orientation and position between camera & mirror

Focal length

R, T

What?


Slide n°

Davide Migliore

u

v

X Y

Z

• Center of the omnidirectional image • Camera focal length• Orientation and position between camera & mirror• Mirror shape

Focal length

R, T

What?


Slide n°

Davide Migliore

u

v

Focal length

R, T

X Y

Z

Assumptions


Slide n°

Davide Migliore

1. Mirror and camera axes are aligned =>

u

v

Focal length

R, T

X Y

Z

Assumptions


Slide n°

Davide Migliore

1. Mirror and camera axes are aligned =>

2. x-y mirror axes coincide with u-v camera axes =>

u

v

Focal length

R, T

X Y

Z

Assumptions


Slide n°

Davide Migliore

Reflected rays do not intersect in a point but are tangent to a “caustic”

And how about non-central cameras?


Stereo SLAM - Politecnico di...

Documents

Transcript of Stereo SLAM - Politecnico di...