Post on 08-Jul-2020
Sécurité des données Enjeux liés à l'internet des objets et à l'intelligence artificielle
Thierry MatusiakArchitecte Division Sécurité IBMMembre actif du CLUSIFthierry_matusiak@fr.ibm.com
LinkedIn : https://fr.linkedin.com/in/thierrymatusiak
Sécurité des données
Travaux du CLUSIF
Internet des objets
Intelligence artificielle
Clu
b d
e la s
écu
rité
de
l’i
nfo
rmati
on
fra
nçais
Présentation du CLUSIF
➢ Association de professionnels de la sécurité de l’informationLieu d’échange pour ses 700 membres, permettant de mettre en commun expertises et réflexions au service d’une SSI efficace
➢ Les activités de l’association• des groupes et Espaces de Travail• des publications• des conférences thématiques• des ateliers fournisseurs sur le grill• un exercice de Cyber-Crise (ECRANS)
Pour plus d’information : clusif@clusif.fr
Clu
b d
e la s
écu
rité
de
l’i
nfo
rmati
on
fra
nçais
A propos des GTs
➢ GT GDPR
➢ GT IoT
➢ GT Sécurité des systèmes industriels
➢ Pas de GT "sécurité de l'IA"➢Sujet d'innovation
➢Lancement de "l'invité du CLUSIF" pour aborder ces sujets prospectifs
➢ Pas de GT "sécurité des données"➢Probablement parce que le sujet est très (trop) vaste
➢Mais c'est un sujet récurrent, par exemple sur l'anonymisation des données
Clu
b d
e la s
écu
rité
de
l’i
nfo
rmati
on
fra
nçais
Zoom sur le GT GDPR
➢ "RSSI & DPO"
➢ Des profils très différents : juristes, CIL, RSSI, fournisseurs de solutions
➢ Infographie publiée début 2018 (Fr : 50000+ vues, En : 7000+ vues)
➢ Augmentation des fuites de données communiquées➢Probablement parce que ca devient une obligation légale
➢Risque de banalisation. Qui a entendu parler de la fuite Adidas par exemple ?
➢ Les amendes de la CNIL restent modérées
➢ Priorités de la CNIL ➢La première règle est que vous devriez être conformes avec les règles de 1978
➢3 secteurs sont dans le viseur en 2018: logement, emploi et stationnement
Sécurité des données
Travaux du CLUSIF
Internet des objets
Intelligence artificielle
Clu
b d
e la s
écu
rité
de
l’i
nfo
rmati
on
fra
nçais
Groupe de travail IoT• Le premier problème est de définir de quoi on parle
• IoT grand public • IIoT / SCADA (qui a son propre GT)
• L'IoT introduit des nouvelles classes de risques • Vies humaines mises en danger • Risque systémique
• Toutes les entreprises sont clientes, au moins à travers :• La gestion de leurs bâtiments • Le shadowIoT importé par leurs employés
• Données IoT• Big Data• Enjeux de privacy forts car ce sont souvent des données personnelles ou sensibles
• Besoin de réglementation pour réguler le secteur - sujet d'actualité
Clu
b d
e la s
écu
rité
de
l’i
nfo
rmati
on
fra
nçais
Cas concrets
Exposition de données : Strava / Polarhttps://www.theguardian.com/world/2018/jan/28/fitness-tracking-app-gives-away-location-of-secret-us-army-bases
https://www.bellingcat.com/resources/articles/2018/07/08/strava-polar-revealing-homes-soldiers-spies/
Détournement et surveillance : Enceintes connectées https://www.theguardian.com/technology/2018/feb/14/amazon-alexa-ad-avoids-ban-after-viewer-complaint-ordered-cat-food
Mauvaise sécurité générale : Jouetshttps://www.cnil.fr/fr/jouets-connectes-mise-en-demeure-publique-pour-atteinte-grave-la-vie-privee-en-raison-dun-defaut-de
Sécurité des données >> sécurité de l'objet (exemple : vol de voiture)https://www.wired.com/story/hackers-steal-tesla-model-s-seconds-key-fob/
Clu
b d
e la s
écu
rité
de
l’i
nfo
rmati
on
fra
nçais
Quelques constats et recommandations
1. Les objets vont probablement fonctionner dans un environnement hostile
2. Leur niveau de sécurité diminue avec le temps
3. Un secret initial ou partagé entre objets n'est pas un vrai secret
4. Si la configuration initiale est faible, elle le restera
5. Le risque de fuite grossit avec le volume des données qui s'accumulent
https://www-01.ibm.com/common/ssi/cgi-bin/ssialias?htmlfid=SEF03018USEN
Sécurité des données
Travaux du CLUSIF
Internet des objets
Intelligence artificielle
IoT + Cognitive = Actionable AI
AI security extends the perimeter – where no one has gone before
• Political Security
Mass surveillance
Data & people manipulation
• Physical Security
SCADA systems get exposed to the Internet
IoT & AI bring threats into the real world (e.g. connected vehicles)
• Digital Security remains crucial
https://www.theatlantic.com/technology/archive/2014/07/makeup/374929
https://www.theguardian.com/technology/2018/may/29/tesla-crash-autopilot-california-police-car
https://www.businessinsider.de/cambridge-analytica-could-rebrand-emerdata-2018-5?r=US&IR=T
/
https://www.wired.com/story/hackers-steal-tesla-model-s-seconds-key-fob/
AI Digital Security – 2018 Landscape
• AI security includes the same "usual suspects"
Identity & Access Management
Network
Application
Data
Infrastructure & Cloud
• But AI also introduces new risks
It can be misused
It can be abused
AI may also be difficult to audit
AI & Security : The Good, the Bad and the Ugly
• The Good : AI improves security
• The Bad : AI becomes a weapon for attackers
• The Ugly : AI applications can be attacked
Using Artificial Intelligence to address growing security needs
• Approach: Model behaviors and
identify emerging and past
threats and risks
• Applications: Network, user,
endpoint, app and data, cloud
Data Analytics
• Approach: Curation of
intelligence and contextual
reasoning
• Applications: Structured and
unstructured (NLP) data sources
Intelligence Consolidation
• Approach: Assist admins &
users
• Applications: Cognitive SOC
analyst, orchestration,
automation and digital guardian
Trusted Advisors & Response
Data analytics
What to predict… Inputs Output
Insider ThreatsSecurity logs and events
Peer grouping, time-series, anomalyRisk score of users
Malicious Traffic Network data Risk score of flows
Botnet Domains DNS data, registrar info Domain risk score and reputation
Vulnerable Code Benchmark set of applicationsNew vulnerability rules
Reduced false positives
Database AttacksSql queries, errors, file access activity
Anomalies & clustersAbnormal activity, risk scores
Risky User AccessIAM data, logs and UBA alerts
Outlier detection with peer groupRisk score of users, apps
Fraudulent Users Behavioral Biometrics
Keystrokes, app, mouse usageRisk score of users
Phishing Websites URLs and website content Risk score of suspected sites
Malware infection Endpoint activity Alerts
Trusted advisors
What to do… Inputs Output
Automatic offense investigations Events Root cause analysis, augmented context
Virtual cybersecurity analystVoice, unstructured content, threat
content
Contextual security information, spoken
content
Administrator advisor Unstructured content, threat alerts, etc. Personalized recommendations
User self-service assistantUser commands, calendar and email
contents, support knowledge base
Coordinates calendar and email activities;
provides real-time end-user support
Crisis management
"Pull" Assistants help you to answer a question
"Push" Assistants provide relevant information / alert to the user
In the future, AI will also learn how to proactively adapt
Intelligence consolidation
What to do… Inputs Output
Security intelligence consolidation Unstructured content, web content Cybersecurity contextual knowledge base
Data Lake Custom Implementation Unstructured + structured data Neutral / agnostic knowledge base
Watson for Cyber Security has ingested over 2 billion documents in the corpus and is adding thousands more every day. It’s reduced the time to analyze an incident from hours to minutes, greatly accelerating mitigation and reducing the impact to the organization.
A SIEM ingests and analyzes structured data
Artificial Intelligence adds a major dimension : unstructured data
AI & Security : The Good, the Bad and the Ugly
• The Good : AI improves security
• The Bad : AI becomes a weapon for attackers
• The Ugly : AI applications can be attacked
AI For Bad Guys ?
• Should we be scared by AI in the hands of cybercriminals ?
• Mass Surveillance & Mass Influence
• Attacks automation
• Massive change of scale
• New types of attack
• Generated text, sound, images & videos
… even though AI can be used to identify forged videos…
http://fortune.com/2018/02/21/artificial-intelligence-oxford-cambridge-report/
https://youtu.be/AmUC4m6w1wo
TRUE FALSE
https://www.blackhat.com/docs/us-16/materials/us-16-Seymour-Tully-Weaponizing-
Data-Science-For-Social-Engineering-Automated-E2E-Spear-Phishing-On-Twitter.pdf
http://deepangel.media.mit.edu/
• Targeted phishing attacks rely on Twitter data
• Neural networks produce a better password cracker
• Generative Adversarial Networks learn novel steganographic channels
• XEvil breaks 1000's of existing catchas
• Remember that your webcam may be watching you
AI-powered attacks
https://arxiv.org/pdf/1709.00440.pdf
To learn more about GANshttps://securityintelligence.com/generative-adversarial-networks-and-cybersecurity-part-1/
https://securityintelligence.com/generative-adversarial-networks-and-cybersecurity-part-2/
AI & Security : The Good, the Bad and the Ugly
• The Good : AI improves security
• The Bad : AI becomes a weapon for attackers
• The Ugly : AI applications can be attacked
AI Can Be Attacked
Direct Attack Theft (IC or data)
https://qz.com/823820/carnegie-mellon-made-a-special-pair-of-glasses-that-lets-you-steal-a-digital-identity/
AI Can Be Fooled
• Model poisoning
• Noise Introduction
• Reinforcement
http://images.complex.com/complex/image/upload/t_in_content_image/tay-hitler_o4kq62.jpg
https://www.darpa.mil/attachments/AIFull.pdf
Darpa
- noise -
Microsoft
- reinforcement -
TAY
https://iotsecurity.eecs.umich.edu/#roadsigns
https://arxiv.org/abs/1707.08945
Proof of Concept
AI Can Also Be Poorly Implemented
• Poor Categories
• Overfitting
• Convergence of views
https://www.theverge.com/2018/1/12/16882408/google-racist-gorillas-photo-recognition-algorithm-ai
- categories -
https://hackernoon.com/memorizing-is-not-learning-6-tricks-to-prevent-overfitting-in-machine-learning-820b091dc42
Underfitting
vs overfitting
AI risks go beyond "traditional" security
• Privacy
GDPR - Data Protection & Right To Be Forgotten
Welcome to 1984 !
• Transparency
Am I currently scrutinized by an AI ?
Can you explain why your bot made this decision ?
• Accountability
An autonomous car accident occurs : Who is responsible ?
What about a robot purchasing illegal items on the darkweb ?https://www.independent.co.uk/arts-entertainment/art/news/swiss-artists-programme-laptop-to-make-random-purchases-from-the-dark-web-a6761891.html
• Ethics
Ethics & Responsability
You are "driving" your autonomous car.
Suddenly in a curve…
No escape …
Ethics & Responsability
Who would you hurt?
- The black car?
- The white car?
- Yourself?
Ethics & Responsability
The audience usually chooses the white car.
After all, it should not be driving this side of the road.
Ethics & Responsability
What if the black one has a single driver, while the white one conveys 2 babies + their mom ?
Ethics & Responsability
You do not know it, but your car's thermal camera does…
Ethics & Responsability
http://moralmachine.mit.edu/
Who would you hurt?
- The black car?
- The white car?
- Yourself?
Moral can even be evaluated online…
AI & Security : The Good, the Bad and the Ugly
• The Good : AI improves security
• The Bad : AI becomes a weapon for attackers
• The Ugly : AI applications can be attacked
In the end, the Good wins !
Attacks against AI: Countermeasures
Data Security Training data & privacy
Model SecurityRobust and resilient models
Operations SecurityDetect and eliminate adversarial inputs
Training
Data
Trusted SME
Training
Robust Models
Non-trusted
Actor
Dete
ction
1. Data Security 2. Model Security 3. Operations Security
Zoom Into Model Security
https://securityintelligence.com/adversarial-ai-as-new-attack-vector-opens-researchers-aim-to-defend-against-it/
eXplainable Artificial Intelligence (XAI)
3 Guiding Principles For Cybersecurity
XXXX By DesignThink about security since the inception phase
Include privacy, transparency, ethics in the design process
XXXX By DefaultDeny everything to everyone by default
Grant access when it is required - on a Need-To-Know basis
XXXX Impact AssessmentGDPR introduced the PIA : Privacy Impact Assessment
Reuse an extend the concept
Apply these 3 principles to AI