Learning Spatial Filters for Multispectral Image Segmentation

Learning Spatial Filters forMultispectral Image Segmentation

Devis Tuia∗, Gustavo Camps-Valls∗,Remi Flamary∗∗, Alain Rakotomamonjy∗∗

∗Image Processing Laboratory (IPL)Universitat de Valencia, Spain

∗∗LITIS EA 4108, Universite de Rouen76800 Saint Etienne du Rouvray, France

August 24, 2010

Devis Tuia et al (IPL, LITIS) Learning Spatial Filters August 24, 2010 1 / 14

Introduction Multispectral Image segmentation

Multispectral Image segmentation

I In multispectral images we have highspatial variability of the spectralsignature

I VHR images allows us betterrecognition, but noisy maps

I Strong intraclass variance, higher thaninterclass

I Including spatial, not only spectralinfo, is mandatory!

⇒ Spatial regularization

image ground thruth

0 100 200 300 400 500 600 700 800 900 1000200

0

200

400

600

800

1000

1200

1400

1600

red bandN

IR b

and

Class “Buildings’ (red)against the others (blue)’


Introduction Spatial Filtering

Spatial FilteringSpatial filtering solves such problems:

I Mathematical morphology [Benediktsson et al., 2005, Tuia et al., 2010]

I Geometrical features [Inglada, 2007]

I Composite kernels with spatial filters [Camps-Valls et al., 2006]

SVMK(xiw, xj

w) K(xir, xj

r) K(xic, xj

c) K(xis, xj

s)

K(xi, xj)

But remain the problem of defining

I what kind of features,

I at what scale, ...

In this paper we propose tolearn the spatial filter that maximizes separability of the classes

in a SVM framework.


Large Margin Filtering Definitions

Definitions

I X ∈ Rr1×r2×d is an image containing r1 × r2 pixels ∈ Rd .

I Xi,j,k = Xp,k ais the kth component of pixel p = (i , j).

I 2D convolution filter band-by-band:

Xp,k =

f ,f∑u=1,v=1

Fu,v,k Xp+(u,v)−f0,k

where f0 = f /2 and F ∈ Rf×f×d .

I RBF kernel between filtered pixels:

Kp,q = k(Xp,., Xq,.) = exp

(−||Xp,. − Xq,.||2

2σ2

), (1)

where σ is the kernel width or bandwidth.


Large Margin Filtering Optimization Problem

Optimization Problem

Large Margin Filtering [Flamary et al., 2010]

ming,F

1

2‖g‖2 +

C

n

∑p∈Sl

H(Yp, g(Xp,.))

︸︷︷︸SVM objective function

+ λΩ(F)︸︷︷︸Filter regularization

(2)

where:

I H(Yp, g(Xp,.) = max(0, 1− Yp · g(Xp,.)) is the SVM hinge loss.

I C and λ are the regularization parameters.

I Ω(·) is a 3D Frobenius Norm: Ω(F) =∑f ,f ,d

u,v,k F2u,v,k

I g(·) is the SVM decision function:

g(Xp,.) =∑q∈Sl

αqYqk(Xq,., Xp,.) + b, (3)

where αp are the dual variables of problem.


Large Margin Filtering Optimization Problem

Solving the problem

Approach

I Convex problem for a fixed F.

I We can always find the optimal decision function g∗ for a fixed F.

I Do gradient descent on F and stay in the optimal g∗ space[Bonnans and Shapiro, 1998]:

minF

J(F) = minF

J ′(F) + λΩ(F) (4)

with:

J ′(F) = ming

1

2‖g‖2 +

C

n

∑p∈Sl

H(Yp, g(Xp,.))

(5)

Algorithm [Flamary et al., 2010]

I Conjugate Gradient descent on F + linesearch.

I Solve a SVM at each cost calculation in the linesearch.


Results Dataset

Dataset and experimental setup

Dataset

I VHR QuickBird image of the city of Zurich, Switzerland.

I 7 classes, difficult to discriminate ’buildings’ classes (’residential’ vs ’commercial’).If merged, difficulty to discriminate ’buildings’ and ’roads’

Compared Models

1. SVM pixel classifier.

2. AvgSVM, averaged pixel classifier.

3. WinSVM, classification of a window of pixels.

4. KF-SVM, Large margin filtering.


Results Binary classification

Binary Classification

Method Class Training #Class AUC KappaPixels Pixels

SVMResidential ∼ 5000 2000

0.904 0.638AvgSVM 0.916 0.689WinSVM 0.947 0.730KF-SVM 0.938 0.742

SVMBuildings∗ ∼ 4000 1000


∗ Pixels from classes ‘Residential’ and ‘Commercial’.

Results

I The estimated Area Under the ROC Curve (AUC) and Kappa coefficient arecomputed.

I Improving over the SVM classification and average Filtering.

I Similar results between KF-SVM and WinSVM (slightly better Kappa).


Results Multiclass classification

Multiclass classification

Method Classes Filtersize

TrainingPixels

[%]OA Kappa

SVM7 9 ∼ 5000


SVM6∗ 9 ∼ 5000

83.04 0.772AvgSVM 89.48 0.860WinSVM 91.71 0.889KF-SVM 91.45 0.885∗ Pixels from classes ‘Residential’ and ‘Commercial’.

Results

I WinSVM and KF-SVM give similar results and both outperform SVM and AvgSVM.

I But with KF-SVM, only pixels are classified.

I Optimal preprocessing done by filtering.


Results Visualization

Segmentation Visualization

Image SVM KF-SVM

Results

I Smoothed borders thanks to the filtering.

I ’Commercial Buildings’ (red class) detectedover ’Buildings’ (orange class).

I What about the filters ? ⇒ Fourier transform



Filter Visualization (1)

Class: Houses, Residential buildingsMagnitude of the FT for different components

Blue filter for Houses

Inverse frequency (m)

Inve

rse

fre

qu

en

cy (

m)

50 10 5 2

50

10

5

2

0

2

4

6

Green filter for Houses


Inve

rse

fre

qu

en

cy (

m)

50 10 5 2

50

10

5

2

0

2

4

6

Red filter for Houses


Inve

rse

fre

qu

en

cy (

m)

50 10 5 2

50

10

5

2

0

2

4

6

Near Infrared filter for Houses


Inve

rse

fre

qu

en

cy (

m)

50 10 5 2

50

10

5

2

0

2

4

6

I Low pass but larger band (houses are small).

I Green, Red and InfraRed are selected.

(Orange)



Filter Visualization (2)

Class: Commercial buildingsMagnitude of the FT for different components

Blue filter for Commercial


Inve

rse

fre

qu

en

cy (

m)

50 10 5 2

50

10

5

2

0

10

20

Green filter for Commercial


Inve

rse

fre

qu

en

cy (

m)

50 10 5 2

50

10

5

2

0

10

20

Red filter for Commercial


Inve

rse

fre

qu

en

cy (

m)

50 10 5 2

50

10

5

2

0

10

20

Near Infrared filter for Commercial


Inve

rse

fre

qu

en

cy (

m)

50 10 5 2

50

10

5

2

0

10

20

I Low pass but small band to detect largebuildings.

I Red is the most discriminant feature.

(Red)


Conclusion

Conclusion

Large Margin Filtering

I Method to learn jointly a SVM pixel classifier and a spatial filtering.

I Large margin spatial filtering/Preprocessing.

I Possibility to use other classifier after filtering, e.g. GMM.

I Visualization for the filtering, no black box approach.

Future Work

I Propose other regularization terms.

I Going local, a global filter is limited.

I Test the method in hyperspectral images, where stacking approaches fail.


References

References

[Benediktsson et al., 2005] Benediktsson, J., Palmason, J. A., and Sveinsson, J. R. (2005).

Classification of hyperspectral data from urban areas based on extended morphological profiles.

IEEE Trans. Geosci. Remote Sens., 43(3):480–490.

[Bonnans and Shapiro, 1998] Bonnans, J. and Shapiro, A. (1998).

Optimization problems with pertubation : A guided tour.

SIAM Review, 40(2):202–227.

[Camps-Valls et al., 2006] Camps-Valls, G., Gomez-Chova, L., Munoz-Marı, J., Vila-Frances, J., and Calpe-Maravilla, J. (2006).

Composite kernels for hyperspectral image classification.

IEEE Geosci. Remote Sens. Lett., 3(1):93–97.

[Flamary et al., 2010] Flamary, R., Labbe, B., and Rakotomamonjy, A. (2010).

Large margin filtering for signal sequence labeling.

In IEEE Conference on Acoustic, Speech and Signal Processing ICASSP, Austin, Texas, USA.

[Inglada, 2007] Inglada, J. (2007).

Automatic recognition of man-made objects in high resolution optical remote sensing images by SVM classification of geometric image features.

ISPRS Journal of Photogrammetry Rem. Sens., 62:236–248.

[Tuia et al., 2010] Tuia, D., Ratle, F., Pozdnoukhov, A., and Camps-Valls, G. (2010).

Multi-source composite kernels for urban image classification.

IEEE Geosci. Remote Sens. Lett., Special Issue ESA EUSC, 7(1):88–92.


Learning Spatial Filters for Multispectral Image Segmentation

Documents

Transcript of Learning Spatial Filters for Multispectral Image Segmentation