Perceptual Hysteresis Thresholding:Towards Driver Visibility Descriptors

Nicolas Hautière, Jean-philippe Tarel, Roland BrémondLaboratoire Central des Ponts et Chaussées, Paris, France

Presentation overview

1. Introduction

2. Angular resolution of a camera

3. Human vision system modeling

4. Discrete Cosine Transform

5. Design of a visibility criteria

6. Perceptual hysteresis thresholding

7. Towards road visibility descriptors

8. Conclusion

Introduction

The major part of all the information used in driving is visual. Reduced visibility thus leads to accidents.

Reductions in visibility may have a variety of causes (geometry, obstacles, adverse weather/lighting conditions).

Different proposals exist in the literature to mitigate the dangerousness of each of these situations using in-vehicle cameras.

One objective is to inform the driver if he is driving too quickly according to the measured visibility conditions.

Detecting the visible edges in the image is a critical step to assess the driver visibility.

We propose such a technique based on the Contrast Sensitivity Function (CSF) of the Human Visual System (HVS).

Angular resolution of a camera

♦ Let’s express the angular resolution of a camera in cpd. ♦ With the notations of Fig. 1, the length d for a visual field = 1° is expressed by:

♦To have the maximum angular resolution of the camera in cpd, we divide d by the size of two pixels (black and white alternation) of the CCD array:

)2(2

tan*

pixcpd t

fr

C

f

d

CCDarray

C

f

d

CCDarray

Cycle per degree (cpd): This unit is used to measure how well details of an object can be seen separately without being blurry. This is the number of lines that can be distinguished in a degree of a visual field.

)1(2

tan2

fd

Fig. 1

♦ Our ability to discern low contrast patterns varies with the size of the pattern, i.e. its spatial frequency f (cpd).

♦The CTF is a measure of the minimum contrast needed for an object (a sinusoidal grating) to become visible.

♦This CTF is defined as 1/CSF, where CSF is a Contrast Sensitivity Function (see Fig. 2). In this paper, we use Mannos CSF, plotted in Fig. 3 and expressed by:

Human vision system modeling

1.1114.0114.00192.06.2)( feffCSF 0

0,2

0,4

0,6

0,8

1

0 5 10 15 20 25 30 35 40 45Spatial frequency (cpd)

CSF

0

5

10

15

20

25

30

35

CTF

Fig. 2

Fig. 3

Discrete Cosine Transform

A={aij} a block of the original image

B={bij} the corresponding block in the transformed image

where c0=1/sqrt(2), ci=1 for i=1...n-1.

The maximum frequency of the DCT is obtained for the maximum resolution of the sensor, i.e. r*

cpd:

To express the bij in cpd, we use the following scale factor obtained by computing the ratio between (1) and (4):

)3(2

)12(cos

2

)12(cos

2 1

0

1

0kl

n

k

n

ljiij a

n

jl

n

ikcc

nb

)4(12

4max

n

nf

2tan

4

)12(

pixt

f

n

n

Design of a visibility criteriaDCT vs CTF

We can now plot the DCT coefficients with respect to the CTF curve:

(a) (b)

(c) (d)

0

0,1

0,2

0,3

0,4

0,5

0,6

0,7

0,8

0,9

1

0,56 1,68 2,79 3,91 5,03 6,14 7,26 8,38 8,94

Spatial frequency (cpd)

Re

co

ns

tru

cte

d C

SF

0

1

2

3

4

5

6

Re

co

ns

tru

cte

d C

TF

Fig. 5: plot of the DCT in the marked blocks with respect to the CTF

Fig. 4: Curves of the CSF (__) and of the CTF(---) for the sensor used to grab the images

tpix=8.3μm, f =8.5mm.

Design of a visibility criteria Visibility Level Definition

Visibility can be related to the contrast C, defined by:

For suprathreshold contrasts, the Visibility Level (VL) of a target can be quantified by the ratio:

As Lb is the same for both conditions, then this equation reduces to:

ΔLthreshold depends on many parameters and can be estimated using Adrian’s empirical target visibility model (Adrian, 1989).

b

bt

b L

LL

L

LC

thresholdbactualbthreshold

actual LLLLCC

VL

thresholdactual LLVL tL

bL

Design of a visibility criteria Visibility Level for Periodic Targets

We propose a new definition of the VL, denoted VLp, valid for periodic targets, i.e. sinusoidal gratings.

We first consider the ratio rij between a DCT coefficient of the block and the corresponding coefficient of the CTF:

Based on the CSF definition CSF, rij ≥1 means that the block contains visible edges.

To define VLp, we choose the greatest rij:

ij

ijij CTF

br

ijji

p rVL)0,0(),(

max

Fig. 6: Map of VLp ≥1

Perceptual hysteresis thresholding: Edges Detection by Segmentation

The proposed approach may be used with different edge detectors (Canny-Deriche, zero-crossing approach, Sobel)

We propose an alternative method which consists in finding the border F which maximizes the contrast C(s0) between two parts of a block, without adding a threshold on this contrast value. The edges are the pixels on this border:

This approach is based on Köhler’s binarization method and is detailed in [16].

[16] N. Hautière, D. Aubert, and M. Jourlin. Measurement of local contrast in images, application to the measurement of visibility distance through use of an onboard camera. Traitement du Signal, 23(2):145–158, Septembre 2006.

f(x1)

f(x)

Cx,x1(s)s

x x1

F(s)

)(),(

1[255,0[

0 )(,)(min)(#

1max)(

sFyxs

xIsxIssF

sC

Perceptual hysteresis thresholding: Hysteresis Thresholding on the VLp

In the usual hysteresis thresholding, a high threshold and a low threshold of gradient magnitude are set.

We propose to replace these thresholds by thresholds on the VLp (cf. Fig. 7)

Thus, the algorithm is as following: 1. All possible edges are extracted,

2. The edges are selected thanks to its VLp value using low tL and high tH thresholds.

tH

tL

Rejected Detected

VL

i, j

Fig. 7: Principle behind thresholding by hysteresis:

Perceptual hysteresis thresholding: Results Samples

tL=1 ; tH=10

tL=1 ; tH=20

No noisy features are detected whatever are the lighting conditions whereas thresholds are fixed. The method is thus clearly adaptive.

Perceptual hysteresis thresholding: Contrast Detection Threshold of the Human Eye

The value of tL is easy to choose, because it can be related to the HVS.

Setting tL=1 should be appropriate for most applications. The hysteresis thresholding has now only one parameter !

The value of tH depends on the application. For lighting engineering, the CIE published some

guidelines to set the VL according to the visual task complexity.

VL=7 is a adequate value for night-time driving task. We can set tL=7 as a starting point. However, a

psychophysical validation is necessary.

Towards road visibility descriptors

Once visible edges have been extracted, they can be used in the context of an onboard camera to derive driver visibility descriptors: visibility distance estimation…

There are three steps to complete and validate the algorithm from a psychophysical point of view: An extension to color images may be necessary, The CSF is valid for a given adaptation level of the HVS. It is

interesting to automatically select the properly CSF. To compare our results with the set of edges which are

manually extracted by different people.

Conclusion

We present a visible edges selector and use it for in-vehicle applications.

It proposes an alternative to the traditional hysteresis filtering.

We propose to replace the thresholds on the gradient magnitude by visibility levels.

The low threshold can be fixed at 1 in general. Some guidelines to set the high threshold are proposed. This algorithm may be used to develop sophisticated

driver visibility descriptors. Thereafter, it can be fused with other visibility descriptors

to develop driving assistance systems which takes into account all the visibility conditions.

Thank you for your attention !

This work is partly founded by the French ANR project DIVAS (2007-2010) dealing with vehicle-infrastructure

cooperative systems.