Sécurité des données Enjeux liés à l'internet des objets et à l'intelligence artificielle

Thierry MatusiakArchitecte Division Sécurité IBMMembre actif du CLUSIFthierry_matusiak@fr.ibm.com

LinkedIn : https://fr.linkedin.com/in/thierrymatusiak

Sécurité des données

Travaux du CLUSIF

Internet des objets

Intelligence artificielle

Clu

b d

e la s

écu

rité

de

l’i

nfo

rmati

on

fra

nçais

Présentation du CLUSIF

➢ Association de professionnels de la sécurité de l’informationLieu d’échange pour ses 700 membres, permettant de mettre en commun expertises et réflexions au service d’une SSI efficace

➢ Les activités de l’association• des groupes et Espaces de Travail• des publications• des conférences thématiques• des ateliers fournisseurs sur le grill• un exercice de Cyber-Crise (ECRANS)

Pour plus d’information : clusif@clusif.fr

Clu

b d

e la s

écu

rité

de

l’i

nfo

rmati

on

fra

nçais

A propos des GTs

➢ GT GDPR

➢ GT IoT

➢ GT Sécurité des systèmes industriels

➢ Pas de GT "sécurité de l'IA"➢Sujet d'innovation

➢Lancement de "l'invité du CLUSIF" pour aborder ces sujets prospectifs

➢ Pas de GT "sécurité des données"➢Probablement parce que le sujet est très (trop) vaste

➢Mais c'est un sujet récurrent, par exemple sur l'anonymisation des données

Clu

b d

e la s

écu

rité

de

l’i

nfo

rmati

on

fra

nçais

Zoom sur le GT GDPR

➢ "RSSI & DPO"

➢ Des profils très différents : juristes, CIL, RSSI, fournisseurs de solutions

➢ Infographie publiée début 2018 (Fr : 50000+ vues, En : 7000+ vues)

➢ Augmentation des fuites de données communiquées➢Probablement parce que ca devient une obligation légale

➢Risque de banalisation. Qui a entendu parler de la fuite Adidas par exemple ?

➢ Les amendes de la CNIL restent modérées

➢ Priorités de la CNIL ➢La première règle est que vous devriez être conformes avec les règles de 1978

➢3 secteurs sont dans le viseur en 2018: logement, emploi et stationnement

Sécurité des données

Travaux du CLUSIF

Internet des objets

Intelligence artificielle

Clu

b d

e la s

écu

rité

de

l’i

nfo

rmati

on

fra

nçais

Groupe de travail IoT• Le premier problème est de définir de quoi on parle

• IoT grand public • IIoT / SCADA (qui a son propre GT)

• L'IoT introduit des nouvelles classes de risques • Vies humaines mises en danger • Risque systémique

• Toutes les entreprises sont clientes, au moins à travers :• La gestion de leurs bâtiments • Le shadowIoT importé par leurs employés

• Données IoT• Big Data• Enjeux de privacy forts car ce sont souvent des données personnelles ou sensibles

• Besoin de réglementation pour réguler le secteur - sujet d'actualité

Clu

b d

e la s

écu

rité

de

l’i

nfo

rmati

on

fra

nçais

Cas concrets

Exposition de données : Strava / Polarhttps://www.theguardian.com/world/2018/jan/28/fitness-tracking-app-gives-away-location-of-secret-us-army-bases

https://www.bellingcat.com/resources/articles/2018/07/08/strava-polar-revealing-homes-soldiers-spies/

Détournement et surveillance : Enceintes connectées https://www.theguardian.com/technology/2018/feb/14/amazon-alexa-ad-avoids-ban-after-viewer-complaint-ordered-cat-food

Mauvaise sécurité générale : Jouetshttps://www.cnil.fr/fr/jouets-connectes-mise-en-demeure-publique-pour-atteinte-grave-la-vie-privee-en-raison-dun-defaut-de

Sécurité des données >> sécurité de l'objet (exemple : vol de voiture)https://www.wired.com/story/hackers-steal-tesla-model-s-seconds-key-fob/

Clu

b d

e la s

écu

rité

de

l’i

nfo

rmati

on

fra

nçais

Quelques constats et recommandations

1. Les objets vont probablement fonctionner dans un environnement hostile

2. Leur niveau de sécurité diminue avec le temps

3. Un secret initial ou partagé entre objets n'est pas un vrai secret

4. Si la configuration initiale est faible, elle le restera

5. Le risque de fuite grossit avec le volume des données qui s'accumulent

https://www-01.ibm.com/common/ssi/cgi-bin/ssialias?htmlfid=SEF03018USEN

Sécurité des données

Travaux du CLUSIF

Internet des objets

Intelligence artificielle

IoT + Cognitive = Actionable AI

AI security extends the perimeter – where no one has gone before

• Political Security

Mass surveillance

Data & people manipulation

• Physical Security

SCADA systems get exposed to the Internet

IoT & AI bring threats into the real world (e.g. connected vehicles)

• Digital Security remains crucial

https://www.theatlantic.com/technology/archive/2014/07/makeup/374929

https://www.theguardian.com/technology/2018/may/29/tesla-crash-autopilot-california-police-car

https://www.businessinsider.de/cambridge-analytica-could-rebrand-emerdata-2018-5?r=US&IR=T

/

https://www.wired.com/story/hackers-steal-tesla-model-s-seconds-key-fob/

AI Digital Security – 2018 Landscape

• AI security includes the same "usual suspects"

Identity & Access Management

Network

Application

Data

Infrastructure & Cloud

• But AI also introduces new risks

It can be misused

It can be abused

AI may also be difficult to audit

AI & Security : The Good, the Bad and the Ugly

• The Good : AI improves security

• The Bad : AI becomes a weapon for attackers

• The Ugly : AI applications can be attacked

Using Artificial Intelligence to address growing security needs

• Approach: Model behaviors and

identify emerging and past

threats and risks

• Applications: Network, user,

endpoint, app and data, cloud

Data Analytics

• Approach: Curation of

intelligence and contextual

reasoning

• Applications: Structured and

unstructured (NLP) data sources

Intelligence Consolidation

• Approach: Assist admins &

users

• Applications: Cognitive SOC

analyst, orchestration,

automation and digital guardian

Trusted Advisors & Response

Data analytics

What to predict… Inputs Output

Insider ThreatsSecurity logs and events

Peer grouping, time-series, anomalyRisk score of users

Malicious Traffic Network data Risk score of flows

Botnet Domains DNS data, registrar info Domain risk score and reputation

Vulnerable Code Benchmark set of applicationsNew vulnerability rules

Reduced false positives

Database AttacksSql queries, errors, file access activity

Anomalies & clustersAbnormal activity, risk scores

Risky User AccessIAM data, logs and UBA alerts

Outlier detection with peer groupRisk score of users, apps

Fraudulent Users Behavioral Biometrics

Keystrokes, app, mouse usageRisk score of users

Phishing Websites URLs and website content Risk score of suspected sites

Malware infection Endpoint activity Alerts

Trusted advisors

What to do… Inputs Output

Automatic offense investigations Events Root cause analysis, augmented context

Virtual cybersecurity analystVoice, unstructured content, threat

content

Contextual security information, spoken

content

Administrator advisor Unstructured content, threat alerts, etc. Personalized recommendations

User self-service assistantUser commands, calendar and email

contents, support knowledge base

Coordinates calendar and email activities;

provides real-time end-user support

Crisis management

"Pull" Assistants help you to answer a question

"Push" Assistants provide relevant information / alert to the user

In the future, AI will also learn how to proactively adapt

Intelligence consolidation

What to do… Inputs Output

Security intelligence consolidation Unstructured content, web content Cybersecurity contextual knowledge base

Data Lake Custom Implementation Unstructured + structured data Neutral / agnostic knowledge base

Watson for Cyber Security has ingested over 2 billion documents in the corpus and is adding thousands more every day. It’s reduced the time to analyze an incident from hours to minutes, greatly accelerating mitigation and reducing the impact to the organization.

A SIEM ingests and analyzes structured data

Artificial Intelligence adds a major dimension : unstructured data

AI & Security : The Good, the Bad and the Ugly

• The Good : AI improves security

• The Bad : AI becomes a weapon for attackers

• The Ugly : AI applications can be attacked

AI For Bad Guys ?

• Should we be scared by AI in the hands of cybercriminals ?

• Mass Surveillance & Mass Influence

• Attacks automation

• Massive change of scale

• New types of attack

• Generated text, sound, images & videos

… even though AI can be used to identify forged videos…

http://fortune.com/2018/02/21/artificial-intelligence-oxford-cambridge-report/

https://youtu.be/AmUC4m6w1wo

TRUE FALSE

https://www.blackhat.com/docs/us-16/materials/us-16-Seymour-Tully-Weaponizing-

Data-Science-For-Social-Engineering-Automated-E2E-Spear-Phishing-On-Twitter.pdf

http://deepangel.media.mit.edu/

• Targeted phishing attacks rely on Twitter data

• Neural networks produce a better password cracker

• Generative Adversarial Networks learn novel steganographic channels

• XEvil breaks 1000's of existing catchas

• Remember that your webcam may be watching you

AI-powered attacks

https://arxiv.org/pdf/1709.00440.pdf

To learn more about GANshttps://securityintelligence.com/generative-adversarial-networks-and-cybersecurity-part-1/

https://securityintelligence.com/generative-adversarial-networks-and-cybersecurity-part-2/

AI & Security : The Good, the Bad and the Ugly

• The Good : AI improves security

• The Bad : AI becomes a weapon for attackers

• The Ugly : AI applications can be attacked

AI Can Be Attacked

Direct Attack Theft (IC or data)

https://qz.com/823820/carnegie-mellon-made-a-special-pair-of-glasses-that-lets-you-steal-a-digital-identity/

AI Can Be Fooled

• Model poisoning

• Noise Introduction

• Reinforcement

http://images.complex.com/complex/image/upload/t_in_content_image/tay-hitler_o4kq62.jpg

https://www.darpa.mil/attachments/AIFull.pdf

Darpa

- noise -

Microsoft

- reinforcement -

TAY

https://iotsecurity.eecs.umich.edu/#roadsigns

https://arxiv.org/abs/1707.08945

Proof of Concept

AI Can Also Be Poorly Implemented

• Poor Categories

• Overfitting

• Convergence of views

https://www.theverge.com/2018/1/12/16882408/google-racist-gorillas-photo-recognition-algorithm-ai

Google

- categories -

https://hackernoon.com/memorizing-is-not-learning-6-tricks-to-prevent-overfitting-in-machine-learning-820b091dc42

Underfitting

vs overfitting

AI risks go beyond "traditional" security

• Privacy

GDPR - Data Protection & Right To Be Forgotten

Welcome to 1984 !

• Transparency

Am I currently scrutinized by an AI ?

Can you explain why your bot made this decision ?

• Accountability

An autonomous car accident occurs : Who is responsible ?

What about a robot purchasing illegal items on the darkweb ?https://www.independent.co.uk/arts-entertainment/art/news/swiss-artists-programme-laptop-to-make-random-purchases-from-the-dark-web-a6761891.html

• Ethics

Ethics & Responsability

You are "driving" your autonomous car.

Suddenly in a curve…

No escape …

Ethics & Responsability

Who would you hurt?

- The black car?

- The white car?

- Yourself?

Ethics & Responsability

The audience usually chooses the white car.

After all, it should not be driving this side of the road.

Ethics & Responsability

What if the black one has a single driver, while the white one conveys 2 babies + their mom ?

Ethics & Responsability

You do not know it, but your car's thermal camera does…

Ethics & Responsability

http://moralmachine.mit.edu/

Who would you hurt?

- The black car?

- The white car?

- Yourself?

Moral can even be evaluated online…

AI & Security : The Good, the Bad and the Ugly

• The Good : AI improves security

• The Bad : AI becomes a weapon for attackers

• The Ugly : AI applications can be attacked

In the end, the Good wins !

Attacks against AI: Countermeasures

Data Security Training data & privacy

Model SecurityRobust and resilient models

Operations SecurityDetect and eliminate adversarial inputs

Training

Data

Trusted SME

Training

Robust Models

Non-trusted

Actor

Dete

ction

1. Data Security 2. Model Security 3. Operations Security

Zoom Into Model Security

https://securityintelligence.com/adversarial-ai-as-new-attack-vector-opens-researchers-aim-to-defend-against-it/

eXplainable Artificial Intelligence (XAI)

3 Guiding Principles For Cybersecurity

XXXX By DesignThink about security since the inception phase

Include privacy, transparency, ethics in the design process

XXXX By DefaultDeny everything to everyone by default

Grant access when it is required - on a Need-To-Know basis

XXXX Impact AssessmentGDPR introduced the PIA : Privacy Impact Assessment

Reuse an extend the concept

Apply these 3 principles to AI