Architectural Solutions for Next Generation Software Systemschristoph/seschool/... · Faheem Ullah...

Architectural Solutions for

Next Generation Software Systems

Presenter: Faheem Ullah & Nguyen K. TranPhD Students

Supervisor: M. Ali Babar

CREST – The Centre for Research on Engineering Software Technologies The University of Adelaide, Australia

[email protected], [email protected]

mailto:[email protected]


Architectural Solutions for Next Generation Software SystemsFaheem Ullah & Nguyen Tran CREST

Prof. M. Ali Babar

Centre for Research on

Engineering Software Technologies

“Helping industries, governments, and societies to build human- as well as technological-based competencies in software systems engineering”

http://www.crest-centre.net/

Who we are

2

Faheem UllahPh.D. Student – The University of Adelaide

“Engineering big data security analytics solutions”

Nguyen TranPh.D. Student – The University of Adelaide

“Engineering search engine systems for Internet of Things”

ASWEC 2018 | 26 Nov. – 30 Nov. 2018 | Adelaide, Australia

http://www.aswec2018.com/@ASWEC2018

http://www.aswec2018.com/


Outline

• Architecting for Big Data Cybersecurity Analytics– How software architecture enables the achievement of the quality of

service delivered by Big Data Cybersecurity Analytics Systems

• An Architectural Solution for Internet of Things Search Engines– How software architecture enables Search Engine Systems for the Future

Internet and research on these systems.

3

ARCHITECTING FOR BIG DATA CYBERSECURITY ANALYTICS

How software architecture enables the achievement of the quality of service delivered by Big Data Cybersecurity Analytics Systems

4


Outline

▪ Introduction

▪ Architectural Tactics for Big Data Cybersecurity Analytics: A Systematic Literature Review

▪ Towards Evidence-Based Understanding of Architectural Tactics for Cybersecurity Analytics

▪ Architecture-Driven Self-Adaptation for Cybersecurity Analytics

5


Big Data Cybersecurity Analytics

Intrusion Detection System (IDS)

Security Information and

Event Management (SIEM)

Big Data Analytics for Cyber Security

6

Big Data Working Group, “Big Data Analytics for Security Intelligence”, Sept 2013

A research domain that leverage big data technologies for analysing security events data to protect organizational networks, computers, and data from cyber attacks


Architectural Tactics for Big Data Cybersecurity Analytics: An SLR

7

Research Questions

RQ1: Which are the most important quality attributes for security analytic systems?

RQ2: What are the architectural tactics for addressing quality concerns in security analytic systems?


Quality Attributes

8

0 20 40 60 80

Performance

Accuracy

Scalability

Reliability

Usability

Interoperability

Adaptability

Modifiability

Generality

Privacy assurance

Security

Stealthiness

Number of papers

Qu

alit

y at

trib

ute

Performance

Accuracy

Scalability

Reliability

Usability

Interoperability

Adaptivity

Modifiability

Privacy

Realtime response required to attacks

Size and speed of security event data hinders real-time response

Catastrophic consequences of letting attack go undetected

Only detect attacks and not shield legitimate access

Challenging to estimate the speed and size of security data

Long period attacks such as Advanced Persistent Threats

High speed security data input can crash the resources

Reliable data collection for ensuring attack detection

Unfriendly system can lead to delay in response to attack

Large number of alerts generated by the system

Collaborate with other security systems i.e., security orchestration

Data collection from a variety of sources

Adapt to comply with the required Quality of Service

Changing operating environments e.g., network topology

Updating attacks signatures to detect new attacks

Incorporating new tools and technologies

Comply with the privacy laws while analysing the data

Qu

alit

y A

ttri

bu

tes

Avoid processing content of a packet


Architectural Tactics

9


Future Research Areas

10

Under addressed Quality Attributes

Tactics Evaluation

Quality Trade-offs among tactics

Dependencies among tactics

Modelling the tactics

▪ Several quality attributes such as interoperability, adaptivity, modifiability, generality, and stealthines requires further investigation from the architectural perspective

▪ The codified tactics should be evaluated both qualitatively and quantitatively to investigate their impact on various quality attributes

▪ The quality trade-offs among the tactics should be established to help a software architect select the required set of tactics

▪ Considering that the tactics cannot be applied in isolation, it is important to explore the possible dependencies and collaborations among the codified set of tactics

▪ To facilitate the software architect, the codified tactics need to be modelling using a standard modelling language such as UML

Futu

re R

esea

rch

Are

as


Validation of Tactics

11

Data

Sources

Applications

Data

Collection

Removal of

DuplicatesFeature

SelectionFeature

Extraction

Model

Training

Data

Collection

Feature

Selection

Feature

Extraction

Attack

Detection

Model Alerts

Alert

Ranking

Visualization

Network

Databases

UserTraining phase

Testing phase

Removal of

Duplicates Tactic

Feature Selection and

Extraction Tactic

Alert Ranking

Tactic

collects security

event data removes duplicated

records

selects specific

features

extracts the

selected features

trains the

model

collects security event data for

testing the trained model

selects specific

features from the data

extracts the

selected features

tests the

model

generates the

alertsranks the

generated alerts

visualizes

the alerts

user responds

to alerts

Legend

Tactic

Phase

Model

Component

Data

Sources

1 2 3 4 5

6 7 8

910 11

12

13

Motivation

• Establishing quality trade-offs among the tactics• Developing evidence-based design space• Quantification of the contribution of tactics to

the intended quality attributes

Research Questions

What is the impact of the

RQ1: Removal of Duplicates TacticRQ2: Feature Selection and Extraction TacticRQ3: Alert Ranking Tactic

on the accuracy and response time of a security analytics system

An illustration tactics applied in the system


Empirical Findings

12

Without the Tactic With the Tactic

Tactic Quality

Attribute

Matric Mean Median Mean Median Wilcoxon’s P

Removal of

Duplicates

Accuracy DR 91.532 91.533 92.625 92.627 -2.611

FPR 29.632 29.621 7.372 7.373 -2.611

Response

Time

TrT 2090.2 2087 1101.6 1100 -3.521

PrT 19.6 19.0 19.5 19.6 0.522

Feature

Selection and

Extraction

Accuracy DR 91.403 91.404 78.54 78.54 0.000137

FPR 6.706 6.706 13.863 13.861 0.000086

Response

Time

TrT 632 625 538.2 540 0.000173

PrT 21 21 20.6 20.8 0.000135

Alert Ranking Response

Time

TrT 632 625 630.2 625 0.5655

PrT 21 21 41.6 42 0.000055

Removal of Duplicates Tactic improves DR by 1.11%, reduces FPR by 22.26%, and improves TrT by 89.74%

Feature Selection and Extraction Tactic reduces DR by 12.86%, increases FPR by 7.15%, and improves TrT and

PrT by 17.43% and 1.93%.

Alert Ranking Tactic improves usability, which leads to enhanced accuracy, but increases PrT by 98.11%.DR – Detection Rate

FPR – False Positive RateTrT – Training TimePrT – Prediction Time

RQ1

RQ2

RQ3


Architecture-Driven Self-Adaptation for Security Analytics

13

Operating Environment

Visualization

Removal of Duplicates

Data CutOff

Static Feature Selection

Dynamic Feature Selection

ML Algorithm Selection

Hadoop Booster

MapReduce Job Adjustment

MinMaxNormalization

ML Algorithm Application

Signature-based Detection

Alert Correlation

False Positive Reduction

Alert Ranking

Result Polling

Adaptation Analysis

Adaptation Model

Adaptation Application

Data Source

s

Security Analytics

Self-Adaptation

Network

Database

Application

Dashboard

Report

Email Notification

Visualization

Motivation

• Accuracy and response time are the most significant quality attributes

• Increasing accuracy reduces response time and vice versa


Architecture-Driven Self-Adaptation for Security Analytics

14

C# Component Name Contributes to Contr. Score

C1 Duplicates Removal AccuracyResponse time

A=+0.1R=+0.6

C2 Data CutOff Response time A=-0.2R=+0.3

C3 Dynamic Feature Selection Accuracy A=+0.2R=-0.2

C4 Static Feature Selection Response time A=-0.3R=+0.2

C5 Result Polling Response time A=0R=+0.02

C6 ML Algorithm Application AccuracyResponse time

A=+0.5R=+0.5

C7 ML Algorithm Selector Accuracy A=+0.2R=+0.1

C8 Signature-based Detection Accuracy A=+0.2R=-0.8

C9 Alert Correlation Accuracy A=+0.2R=-0.6

C10 False Positive Reduction Accuracy A=+0.2R=-0.3

C11 Alert Ranking Accuracy A=+0.2R=-0.8

C12 Hadoop Booster Response time A=0R=+0.1

C13 Adjusting MapReduce Jobs Response time A=0R=+0.05

C14 MinMax Normalization Accuracy A=+0.1R=0

C1 C6

C1 C6 C8

C1 C4 C6 C8

C1 C2 C3 C6 C9

WF-1

WF-2

WF-3

WF-N

𝐹 (𝑤𝑓) =𝑘=1

𝐾

൯𝑤𝑘 ∗ V k + D(wf

𝐹 (𝑤𝑓 − 1)

𝐹 (𝑤𝑓 − 2)

𝐹 (𝑤𝑓 − 3)

𝐹 (𝑤𝑓 − N)


Summary

15

Systematic Literature Review

12 Quality Attributes

17 Architectural Tactics

5 Future Research Areas

Empirical Validation of Architectural Tactics

Removal of Duplicates

Feature Selection and Extraction

Alert Ranking

Accu

racy

Resp

on

se Time

Architecture-Driven Self-Adaptation

Runtime architecture adaptation

Utility function and workflow for optimizing QoS

Qo

S

AN ARCHITECTURAL SOLUTION FOR INTERNET OF THINGS SEARCH ENGINES

How software architecture enables Search Engine Systems for the Future Internet and research on these systems.

16


Background – Internet of Things Search Engines

Find “meeting room which reporting abnormal energy consumption”

Find “Available parking bay nearest to the Uni”

On-campus IoT Infra.

Smart-city IoT Infra.

Internet of Things Search Engine

Internet of Things Search Engine

Room A1 and B2

Try bay 6 on Grenfell

str.

Detect Things

Detect Things

Collect metadata

Collect readings

Collect readings

Collect metadata

Collect location

Metadata query (is this meeting room?)

Process (Abnormality detection)

Return

Return

Reading query (is there abnormality?)

Metadata query (is this a parking bay?)

Reading query (is this available?)

Location query (is this a close to Uni?)

17


Cloud 2Cloud 1Edge Node

Motivating Scenario

Internet of Things Search Engine 1 Internet of Things Search Engine 2

Detector 1 Storage 1 Index 1 Query 1 Detector 2 Storage 2 Index 2 Query 2

Rep

osi

tory Detector 1 Storage 1 Index 1 Query 1

Detector 2 Storage 2 Index 2 Query 2

Software Infrastructure

IoTSE Composition and Deployment Pattern

Internet of Things Search Engine 3

Which components?Which architecture patterns?Architecting for

reuse and composition?

Software infrastructure to support reuse and composition?

18


Research Problem

“How to enable the construction of Internet of Things Search Enginefrom

independently developed and reusable components?”

19


Research Questions

“How to enable the construction of Internet of Things Search Engine from independently developed and reusable components?”

RQ1: What components and patterns constitute IoTSE?

Reference Architecturefor IoTSE

RQ2: How to architect IoTSE to be supportive to composition and reuse?

Service-oriented Architecture (SOA)

for IoTSE

RQ3: How to support the application of the architectural solution?

Software Platform for Development, Composition and Deployment of IoTSE20


RQ2

ISO/IEC18384

Arch. Design

IoTSE SOA Sol.

RQ3

Scenario-based Req. Elicitation

Op. Requirements

Dev. RequirementsDesign & Impl.IoTSE

Platform

Research Design

Guiding Principles:P1: Independence of component developers is the top priority.

P2: Workflow logic is separated from the logic of IoTSE component.

P3: Deployment is separated from the logic of IoTSE component.

RQ1

IoTSE Literature

SLR Arch. Design

Requirements

Components

Patterns

IoTSE Ref. Arch.

Eva

lua

tio

n

Case StudyTechnical Action

Research

PrototypePrototypeStudy on

Arch. Impact

21


Results

IoTSE Reference Architecture Platform for Composition and Deployment of IoTSE

IoTS

E C

om

po

nen

ts a

s C

on

tain

eris

ed W

eb S

ervi

ces

Published: CSURWIP & Submitted: CACM, TSE

WIP & Submitted: WISE 2018

Published: WISE 2017WIP & Submitted: ASE 2018 22


Results – Proof of ConceptFinding “sensors measuring apparent temperature in Celsius, whose

readings are less than 25 degrees”

Unit of measurement: Celsius

Observed Property: Apparent Temperature

Sensor reading: 24.1044 degrees

23


Results – Impact of architecture

Finding “sensors measuring apparent temperature in Celsius, whose readings are less than 25 degrees”

Query Metadata

Query Real-time Readings

ID_P

S_C

Detect IoT sensors

Collect & Index Metadata

Collect & Index Readings

Query Metadata

Query Real-time Readings

Return results

ID_S

S_C Detect IoT sensors

Collect & Index Metadata

Collect & Index Readings

Query MetadataQuery Real-time

Readings

Return results

Parallelism actually INCREASES THE

RESPONSE TIME!

24


Results

Finding available parking bay around a certain radius in Adelaide

(Simulated data generated from real sensor data)

25


Summary

• Architectural solution provides the glue to bring independent components together for next-gen software systems

• Potential for new research


Thank You!

Questions and Comments!

Faheem Ullah, Nguyen Tran

CREST – Centre for Research on Engineering Software Technologies

The University of Adelaide, Australia

[email protected], [email protected]

27

ASWEC 2018 | 26 Nov. – 30 Nov. 2018 | Adelaide, Australia

http://www.aswec2018.com/@ASWEC2018



http://www.aswec2018.com/

Architectural Solutions for Next Generation Software Systemschristoph/seschool/... · Faheem Ullah...

Documents

Transcript of Architectural Solutions for Next Generation Software Systemschristoph/seschool/... · Faheem Ullah...