Robust Unsupervised Feature Selection on

Networked DataReading Group: Manqing

Paper resource: SDM 2016

IntroductionNetworked data: encodes pairwise relations among instances in a network.

Features:

1. Structural interactions

2. High-dimensional features

Challenges:

3. The curse of dimensionality.

4. Memory storage requirements and computational costs for data analytics.

5. The existence of irrelevant, redundant and noisy features.

IntroductionWhy feature selection?

Feature selection, as a data preprocessing step has shown to be effective in preparing high-dimensional data for many data mining tasks such as sentiment analysis and node classification.

How feature selection?

According to the availability of labels, feature selection methods consist of supervised methods and unsupervised methods.

Introduction1. Supervised

Minimum redundancy feature selection from microarray gene expression data. (C. Ding et al. 2005)

Efficient and robust feature selectionvia joint 2,1-norm minimization. (F. Nie et al. 2010)

Regression shrinkage and selection via the lasso. (R. Tibshirani 1996)

2. Unsupervised

Unsupervised feature selection for multi-cluster data. (D. Cai et al. 2010)

Laplacian score for feature selection. (X. He et al. 2005)

Unsupervised feature selection using nonnegative spectral analysis. (Z. Li et al. 2012)

Spectral feature selection for supervised and unsupervised learning. (Z. Zhao et al. 2007)

IntroductionAs it is easy to amass substantial amounts of unlabeled data while label information is costly to obtain, unsupervised feature selection has received increasingly attention in the past few years.

And it exploit different criteria to define the relevance of features such as: data similarity, local discriminative information, and data reconstruction error.

Challenges for using current model to analyze network data:

1. In networks, data instances are not independent and identically distributed but inherently interconnected with each other. Meanwhile, they are often associated with some content features.

2. In addition to noisy features in the content space, link information is prevalent and also contains a lot of noise.

Problem Statement A robust unsupervised feature selection framework NetFS:

First, to capture the inherent interactions among networked instances, introduce the concept of latent representations to uncover some hidden attributes encoded in the network structure.

Second, reduce the negative effects from noisy links: embedding the latent representation learning into the feature selection phase. Specifically, content information can help mitigate noisy links.

Problem Statement denote a set of n linked instances in the network.

represent the network structure of

For undirected network,

For directed network,

Feature set:

Content information:

Robust Unsupervised Feature Selection for Networked Data - NetFS

Modeling link information with latent representation.

Embedding latent representation learning into feature selection.

Optimization solution.

Convergence analysis.

1.Modeling link information with latent representation

By a symmetric nonnegative matrix factorization model (SymNMF).

Mathematically, it decomposes the adjacency matrix A into a product of a nonnegative matrix U and its transpose U′ in a low-dimensional latent space:

2. Embedding latent Representation Learning into Feature selectionAs latent factors encode some hidden attributes of instances, they should be related to some features (or attributes) of networked instances.

Therefore, we take U as a constraint to model the content information through a multivariate linear regression model:

2. Embedding latent Representation Learning into Feature selection

3. Optimization solutionEquation (3.4) is not convex in both U and W.

Besides, is not smooth.

And based on paper: Efficient and robust feature selection via joint 2,1-norms minimization, it adopted alternating optimization scheme.

Thus, when U is fixed, the objective funciton is convex.

Therefore, we take the derivative of with respect to W to be zero, then we have:

3. Optimization solution

4. Convergence AnalysisGoal: to prove

ExperimentsDatasets:BlogCatalog Flickr Epinions

ExperimentsDatasets

Experimental settings

Quality of Selected Features by Netfs

Effect of Parameters

Experimental SettingsUsing NMI (normalized mutual information) and ACC (accuracy) to see the clustering performance.

[Notes: Let C and C’ denote the clustering results from ground truth class labels and the predicted cluster labels, respectively.

The definition:

The mutual information between two clusters C and C’ is:

NMI:

ACC:

So the higher the ACC and NMI values are, the better the feature selection performance is.

Experimental settings

Effect of parametersalpha controls the sparsity of the model while beta balances the latent representation learning and feature selection phase.

Future WorkFirst, in this work, the authors use social media as a test bed to evaluate the proposed NetFS framework, we also would like to validate the proposed framework on other kinds of networks such as gene networks, citation networks.

Second, real-world networks are usually not static but evolve over time such that both the network structure and the content information may change. Future works can include how to perform feature selection on dynamic networks in the future.

Thank you