Ppt manqing

21
Robust Unsupervised Feature Selection on Networked Data Reading Group: Manqing Paper resource: SDM 2016

Transcript of Ppt manqing

Page 1: Ppt manqing

Robust Unsupervised Feature Selection on

Networked DataReading Group: Manqing

Paper resource: SDM 2016

Page 2: Ppt manqing

IntroductionNetworked data: encodes pairwise relations among instances in a network.

Features:

1. Structural interactions

2. High-dimensional features

Challenges:

3. The curse of dimensionality.

4. Memory storage requirements and computational costs for data analytics.

5. The existence of irrelevant, redundant and noisy features.

Page 3: Ppt manqing

IntroductionWhy feature selection?

Feature selection, as a data preprocessing step has shown to be effective in preparing high-dimensional data for many data mining tasks such as sentiment analysis and node classification.

How feature selection?

According to the availability of labels, feature selection methods consist of supervised methods and unsupervised methods.

Page 4: Ppt manqing

Introduction1. Supervised

Minimum redundancy feature selection from microarray gene expression data. (C. Ding et al. 2005)

Efficient and robust feature selectionvia joint 2,1-norm minimization. (F. Nie et al. 2010)

Regression shrinkage and selection via the lasso. (R. Tibshirani 1996)

2. Unsupervised

Unsupervised feature selection for multi-cluster data. (D. Cai et al. 2010)

Laplacian score for feature selection. (X. He et al. 2005)

Unsupervised feature selection using nonnegative spectral analysis. (Z. Li et al. 2012)

Spectral feature selection for supervised and unsupervised learning. (Z. Zhao et al. 2007)

Page 5: Ppt manqing

IntroductionAs it is easy to amass substantial amounts of unlabeled data while label information is costly to obtain, unsupervised feature selection has received increasingly attention in the past few years.

And it exploit different criteria to define the relevance of features such as: data similarity, local discriminative information, and data reconstruction error.

Challenges for using current model to analyze network data:

1. In networks, data instances are not independent and identically distributed but inherently interconnected with each other. Meanwhile, they are often associated with some content features.

2. In addition to noisy features in the content space, link information is prevalent and also contains a lot of noise.

Page 6: Ppt manqing

Problem Statement A robust unsupervised feature selection framework NetFS:

First, to capture the inherent interactions among networked instances, introduce the concept of latent representations to uncover some hidden attributes encoded in the network structure.

Second, reduce the negative effects from noisy links: embedding the latent representation learning into the feature selection phase. Specifically, content information can help mitigate noisy links.

Page 7: Ppt manqing

Problem Statement denote a set of n linked instances in the network.

represent the network structure of

For undirected network,

For directed network,

Feature set:

Content information:

Page 8: Ppt manqing

Robust Unsupervised Feature Selection for Networked Data - NetFS

Modeling link information with latent representation.

Embedding latent representation learning into feature selection.

Optimization solution.

Convergence analysis.

Page 9: Ppt manqing

1.Modeling link information with latent representation

By a symmetric nonnegative matrix factorization model (SymNMF).

Mathematically, it decomposes the adjacency matrix A into a product of a nonnegative matrix U and its transpose U′ in a low-dimensional latent space:

Page 10: Ppt manqing

2. Embedding latent Representation Learning into Feature selectionAs latent factors encode some hidden attributes of instances, they should be related to some features (or attributes) of networked instances.

Therefore, we take U as a constraint to model the content information through a multivariate linear regression model:

Page 11: Ppt manqing

2. Embedding latent Representation Learning into Feature selection

Page 12: Ppt manqing

3. Optimization solutionEquation (3.4) is not convex in both U and W.

Besides, is not smooth.

And based on paper: Efficient and robust feature selection via joint 2,1-norms minimization, it adopted alternating optimization scheme.

Thus, when U is fixed, the objective funciton is convex.

Therefore, we take the derivative of with respect to W to be zero, then we have:

Page 13: Ppt manqing

3. Optimization solution

Page 14: Ppt manqing

4. Convergence AnalysisGoal: to prove

Page 15: Ppt manqing

ExperimentsDatasets:BlogCatalog Flickr Epinions

Page 16: Ppt manqing

ExperimentsDatasets

Experimental settings

Quality of Selected Features by Netfs

Effect of Parameters

Page 17: Ppt manqing

Experimental SettingsUsing NMI (normalized mutual information) and ACC (accuracy) to see the clustering performance.

[Notes: Let C and C’ denote the clustering results from ground truth class labels and the predicted cluster labels, respectively.

The definition:

The mutual information between two clusters C and C’ is:

NMI:

ACC:

So the higher the ACC and NMI values are, the better the feature selection performance is.

Page 18: Ppt manqing

Experimental settings

Page 19: Ppt manqing

Effect of parametersalpha controls the sparsity of the model while beta balances the latent representation learning and feature selection phase.

Page 20: Ppt manqing

Future WorkFirst, in this work, the authors use social media as a test bed to evaluate the proposed NetFS framework, we also would like to validate the proposed framework on other kinds of networks such as gene networks, citation networks.

Second, real-world networks are usually not static but evolve over time such that both the network structure and the content information may change. Future works can include how to perform feature selection on dynamic networks in the future.

Page 21: Ppt manqing

Thank you