Towards Testing Model Transformation Chains Using...

32
Towards Testing Model Transformation Chains Using Precondition Construction in Algebraic Graph Transformation Elie Richa Télécom ParisTech & AdaCore Etienne Borde Télécom ParisTech Laurent Pautet Télécom ParisTech Matteo Bordin AdaCore José F. Ruiz AdaCore September 29, 2014

Transcript of Towards Testing Model Transformation Chains Using...

Page 1: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Towards Testing Model Transformation ChainsUsing Precondition Construction in

Algebraic Graph Transformation

Elie Richa Télécom ParisTech & AdaCore Etienne Borde Télécom ParisTech Laurent Pautet Télécom ParisTech Matteo Bordin AdaCore José F. Ruiz AdaCore

September 29, 2014

Page 2: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Development of Critical Software• Critical : Software failure can have catastrophic consequences

!

!

• Certification standards are mandatory and costly to apply

• Model-based development enables Automatic Code Generation (ACG)

• ACGs must be Qualified. e.g. SCADE Suite KCG

• Qualification of an ACG is very costly

• Extensive testing of the ACG is required

2Future WorkRelated WorkDetailed ApproachContributionProblemContextContext

Source Code

ACG

Model-based Specification

DO178C EN50128 ISO26262

Page 3: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Unit Testing v/s Integration Testing in ACGs

• An ACG is typically a Model Transformation Chain (MTC)

• Unit testing (Unit = = intermediate transformation) – Consider intermediate transformations in isolation – Develop test models in intermediate representations

• Integration testing – Consider whole chain – Develop test models in the input language

3Future WorkRelated WorkDetailed ApproachContributionProblemContext

Mi�1 MiTi�1 Ti . . .. . . Ti�2M0

Context

User-visible Language Intermediate Representations

Ti

Page 4: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Reality Check

• Feedback from developers of ACGs and the GCC compilation chain – 10 ~ 20 intermediate transformations

• Unit testing of intermediate transformation is rarely performed

• Intermediate test models are difficult to produce and maintain – Intermediate models increase in size along the chain – Internal languages have no dedicated editors – Intermediate languages and transformations evolve during the lifecycle

!General Problem

Perform only integration tests but cover all unit testing needs

4Future WorkRelated WorkDetailed ApproachContributionProblemProblemContext

Mi�1 MiTi�1 Ti . . .. . . Ti�2M0

Page 5: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Unit Testing of Model Transformations

• Coverage – Testing all individual cases is unfeasible – An analysis identifies sets of equivalent cases

!• Test objective/requirement

Constraint over the input language of a transformation – Describes a set of input models exhibiting a common

property

!

!• One test model is sufficient to cover a test objective

5Future WorkRelated WorkDetailed ApproachContributionProblemProblemContext

Mi�1 MiTi�1 Ti . . .. . . Ti�2M0

Coverage Criteria

Analysis

trj

Page 6: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Unit Testing of Model Transformations

• Coverage – Testing all individual cases is unfeasible – An analysis identifies sets of equivalent cases

!• Test objective/requirement

Constraint over the input language of a transformation – Describes a set of input models exhibiting a common

property

!

!• One test model is sufficient to cover a test objective

5Future WorkRelated WorkDetailed ApproachContributionProblemProblemContext

Mi�1 MiTi�1 Ti . . .. . . Ti�2M0

Coverage Criteria

Analysis

trj

tr1 : 9 p : Param | p.value = 0

tr2 : 9 p : Param | 0 < p.value <= 5

tr3 : 9 p : Param | 5 < p.value

Page 7: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Covering Non-Satisfied Test Requirements

• Constraint Satisfaction Problem (CSP)

• A CSP Solver can theoretically generate a satisfying instance

• Encoding all transformations in the CSP is not scalable

• The result is an instance and not a constraint which prevents an iterative analysis

6Future WorkRelated WorkDetailed ApproachContributionProblemProblemContext

Mi�1 MiTi�1 Ti . . .. . . Ti�2M0

tri,j

Page 8: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Covering Non-Satisfied Test Requirements

• Constraint Satisfaction Problem (CSP)

• A CSP Solver can theoretically generate a satisfying instance

• Encoding all transformations in the CSP is not scalable

• The result is an instance and not a constraint which prevents an iterative analysis

6Future WorkRelated WorkDetailed ApproachContributionProblemProblemContext

Mi�1 MiTi�1 Ti . . .. . . Ti�2M0

Solver

tri,j

Page 9: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Covering Non-Satisfied Test Requirements

• Constraint Satisfaction Problem (CSP)

• A CSP Solver can theoretically generate a satisfying instance

• Encoding all transformations in the CSP is not scalable

• The result is an instance and not a constraint which prevents an iterative analysis

6Future WorkRelated WorkDetailed ApproachContributionProblemProblemContext

Mi�1 MiTi�1 Ti . . .. . . Ti�2M0

SolverSolver

tri,j

Page 10: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Covering Non-Satisfied Test Requirements

• Constraint Satisfaction Problem (CSP)

• A CSP Solver can theoretically generate a satisfying instance

• Encoding all transformations in the CSP is not scalable

• The result is an instance and not a constraint which prevents an iterative analysis

6Future WorkRelated WorkDetailed ApproachContributionProblemProblemContext

Mi�1 MiTi�1 Ti . . .. . . Ti�2M0

SolverSolver

tri,j

Page 11: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Satisfying Intermediate Test Requirements

ProblemHow to produce a new model in the input language to satisfy a

given intermediate test requirement

• It is a challenging problem because – We have to consider an arbitrary number of preceding transformations – Tester has to manually “inverse” transformations – Transformations are non-injective and non-surjective – We have to reason on constraints and not on instances

7Future WorkRelated WorkDetailed ApproachContributionProblemProblem

Mi�1 MiTi�1 Ti . . .. . .M0 Ti�2

tri,j?

Context

Page 12: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Step-by-step Advancement of Test Requirements

Contribution

• Step-by-step automatic advancement of test requirementsup to input language – Test requirement as a postcondition of previous step – Transform a postcondition into a sufficient precondition – Iterate process up to the input language

8Future WorkRelated WorkDetailed ApproachContributionContributionProblem

tri,jtri�1,jPost2Pre

Mi�1 MiTi�1 Ti . . .. . .M0 Ti�2

Post2Pre

Context

Page 13: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Algebraic Graph Transformation

• Algebraic Graph Transformation (AGT) – Formal framework based on Category Theory

• Construction of Weakest Precondition – Constructs a precondition ensuring the satisfaction of the postcondition

9Future WorkRelated WorkDetailed ApproachContributionContributionProblem

wp

Mi�1 MiTi�1

PostPre

tri,jtri�1,jPost2Pre

Graph Transformation

wp

AGT Theory

Context

Page 14: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Using AGT to Advance Test Requirements

• Transpose our problem into the AGT Theory

• The ACG is specified in industry standard languages – ATL and OCL

10Future WorkRelated WorkDetailed ApproachContributionContributionProblem

tri,jtri�1,j

Mi�1 MiTi�1

PostPre

Post2Pre

Graph Transformation

AGT Theory

Context

Page 15: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Using AGT to Advance Test Requirements

• 3 main components – Translation of transformations ATL2AGT – Translation of test requirements OCL2NGC (NGC2OCL) – Advancement of constraints Post2Pre

11Future WorkRelated WorkDetailed ApproachContributionContributionProblemContext

AGT Theory

tri,jtri�1,j

ATL2AGT

Mi�1 MiTi�1

PostPre

Post2Pre

OCL2NGCNGC2OCL

ATL

OCL

NGC

Graph Transformation

Nested Graph Constraints

Page 16: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Using AGT to Advance Test Requirements

• 3 main components – Translation of transformations ATL2AGT – Translation of test requirements OCL2NGC (NGC2OCL) – Advancement of constraints Post2Pre

12Future WorkRelated WorkDetailed ApproachContributionContributionProblem

AGT Theory

tri,jtri�1,j

ATL2AGT

Mi�1 MiTi�1

PostPre

Post2Pre

OCL2NGCNGC2OCL

ATL

OCL

NGC

Graph Transformation

Nested Graph Constraints

Context

Page 17: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Translation of Transformations – ATL2AGTChallenge

• Semantic gap between Declarative Model Transformation and AGT

• ATL semantics – create a new output model – Rules executed simultaneously – A rule can resolve the output of other rules using implicit input to output tracing

• AGT semantics – in-place graph rewriting – Rules applied sequentially and atomically – No resolve mechanism

Contribution

• Model an ATL transformation as a 2-phase rewriting of the input graph – Instantiation phase —> Resolving phase – Explicit Trace nodes

• Prototype implementation using Henshin framework (Eclipse-based) – Limited to structural aspects

13Future WorkRelated WorkDetailed ApproachDetailed ApproachContributionProblemContext

Page 18: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Example — ATL2AGT

14Future WorkRelated WorkDetailed ApproachDetailed ApproachContributionProblemContext

rule S2VExp { from sig : SMM!Signal to varExp : CMM!VarExp (variable <- sig.srcPort) } -- Resolve !rule O2Var { from oport : SMM!Outport to var : CMM!Variable }

Instantiation

Resolving

:Signal

:Outport

srcPort

Initial GraphS2VExp_Inst

O2Var_Inst

S2VExp_Res

Page 19: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Example — ATL2AGT

14Future WorkRelated WorkDetailed ApproachDetailed ApproachContributionProblemContext

rule S2VExp { from sig : SMM!Signal to varExp : CMM!VarExp (variable <- sig.srcPort) } -- Resolve !rule O2Var { from oport : SMM!Outport to var : CMM!Variable }

Instantiation

Resolving

:Signal

:Outport

srcPort

:VarExp:S2VExp_Trace

Initial GraphS2VExp_Inst

O2Var_Inst

S2VExp_Res

Page 20: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Example — ATL2AGT

14Future WorkRelated WorkDetailed ApproachDetailed ApproachContributionProblemContext

rule S2VExp { from sig : SMM!Signal to varExp : CMM!VarExp (variable <- sig.srcPort) } -- Resolve !rule O2Var { from oport : SMM!Outport to var : CMM!Variable }

Instantiation

Resolving

:Signal

:Outport

srcPort

:VarExp

:Variable

:S2VExp_Trace

:O2Var_Trace

Initial GraphS2VExp_Inst

O2Var_Inst

S2VExp_Res

Page 21: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Example — ATL2AGT

14Future WorkRelated WorkDetailed ApproachDetailed ApproachContributionProblemContext

rule S2VExp { from sig : SMM!Signal to varExp : CMM!VarExp (variable <- sig.srcPort) } -- Resolve !rule O2Var { from oport : SMM!Outport to var : CMM!Variable }

Instantiation

Resolving

:Signal

:Outport

srcPort

:VarExp

:Variable

:S2VExp_Trace

:O2Var_Trace

variable

Initial GraphS2VExp_Inst

O2Var_Inst

S2VExp_Res

Page 22: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Example — ATL2AGT

14Future WorkRelated WorkDetailed ApproachDetailed ApproachContributionProblemContext

rule S2VExp { from sig : SMM!Signal to varExp : CMM!VarExp (variable <- sig.srcPort) } -- Resolve !rule O2Var { from oport : SMM!Outport to var : CMM!Variable }

Instantiation

Resolving

:Signal

:Outport

srcPort

:VarExp

:Variable

:S2VExp_Trace

:O2Var_Trace

variable

Initial Graph Output ModelS2VExp_Inst

O2Var_Inst

S2VExp_Res

Page 23: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Translation of Transformations – ATL2AGTChallenge

• Semantic gap between Declarative Model Transformation and AGT

• ATL semantics – create a new output model – Rules executed simultaneously – A rule can resolve the output of other rules using implicit input to output tracing

• AGT semantics – in-place graph rewriting – Rules applied sequentially and atomically – No resolve mechanism

Contribution

• Model an ATL transformation as a 2-phase rewriting of the input graph – Instantiation phase —> Resolving phase – Explicit Trace nodes

• Prototype implementation using Henshin framework (Eclipse-based) – Limited to structural aspects

15Future WorkRelated WorkDetailed ApproachDetailed ApproachContributionProblemContext

Page 24: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Advancement of Constraints – Post2PreChallenge

• The theoretical construction in AGT consists in 1. Enumerate all overlaps of a rule with the postcondition 2. Unroll the effects of applying the rule (reverse application)

16Future WorkRelated WorkDetailed ApproachDetailed ApproachContributionProblemContext

PostconditionPreconditionPost2Pre

:VarExp

:Variable

variable

:AssignmentStatement

lhs

Page 25: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Advancement of Constraints – Post2PreChallenge

• The theoretical construction in AGT consists in 1. Enumerate all overlaps of a rule with the postcondition 2. Unroll the effects of applying the rule (reverse application)

16Future WorkRelated WorkDetailed ApproachDetailed ApproachContributionProblemContext

PostconditionPreconditionPost2Pre

:VarExp

:Variable

variable

:AssignmentStatement

lhsS2VExp_Res

Page 26: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Advancement of Constraints – Post2PreChallenge

• The theoretical construction in AGT consists in 1. Enumerate all overlaps of a rule with the postcondition 2. Unroll the effects of applying the rule (reverse application)

16Future WorkRelated WorkDetailed ApproachDetailed ApproachContributionProblemContext

PostconditionPreconditionPost2Pre

:VarExp

:Variable

variable

:AssignmentStatement

lhs

Overlap

S2VExp_Res

Page 27: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Advancement of Constraints – Post2PreChallenge

• The theoretical construction in AGT consists in 1. Enumerate all overlaps of a rule with the postcondition 2. Unroll the effects of applying the rule (reverse application)

16Future WorkRelated WorkDetailed ApproachDetailed ApproachContributionProblemContext

PostconditionPreconditionPost2Pre

∃∃

:VarExp

:Variable

variable

:AssignmentStatement

lhs

:VarExp

:Variable

:AssignmentStatement

lhs

Overlap

S2VExp_Res

Page 28: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Advancement of Constraints – Post2PreChallenge

• The theoretical construction in AGT consists in 1. Enumerate all overlaps of a rule with the postcondition 2. Unroll the effects of applying the rule (reverse application)

16Future WorkRelated WorkDetailed ApproachDetailed ApproachContributionProblemContext

PostconditionPreconditionPost2Pre

∃∃

:VarExp

:Variable

variable

:AssignmentStatement

lhs

:VarExp

:Variable

:AssignmentStatement

lhs

:Signal

:Outport

:S2VExp_Trace

:Trace

Overlap

S2VExp_Res

Page 29: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Advancement of Constraints – Post2PreChallenge

• The theoretical construction in AGT consists in 1. Enumerate all overlaps of a rule with the postcondition 2. Unroll the effects of applying the rule (reverse application)

and add the application condition

• Theoretical weakest precondition is infinite because of infinite rule iteration Contribution

• Bound the number of iteration of rule iterations – Obtain a sufficient precondition instead of the weakest – Similar to size bounds in CSP-based approaches

• Eliminate overlaps based on knowledge of ATL semantics

• Prototype implementation in AGG framework for basic constraints ∃(graph)

• Validation of Post2Pre and ATL2AGTon a simplified code generation step : 3 ATL rules

one-step advancement of simple test requirements

17Future WorkRelated WorkDetailed ApproachDetailed ApproachContributionProblemContext

Page 30: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Related Work• Test suite quality for model transformation chains

E. Bauer, J. Küster, and G. Engels Cover unit test requirements with integration tests Detect unsatisfied unit test requirements No support for producing new test models

!

• Synthesis of OCL pre-conditions for graph transformation rules J. Cabot, R. Clariso, E. Guerra, and J. de Lara

Construct OCL preconditions from OCL postconditions No formal proof of completeness and correctness

18Future WorkRelated WorkRelated WorkDetailed ApproachContributionProblemContext

Page 31: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Future Work• OCL2NGC — Translation of Test Requirements

– Very active topic in the community • ICGT, July 2014 T. Arendt et al. , “From Core OCL Invariants to Nested Graph Constraints” • MODELS, September 2014 G. Bergmann, “Translating OCL to Graph Patterns”

• ATL2AGT – Translate OCL embedded in ATL to AGT – Realistic ATL transformations

• Post2Pre – Handle complete Nested Graph Constraints – Investigate performance of overlapping algorithm

19Future WorkFuture WorkRelated WorkDetailed ApproachContributionProblemContext

Page 32: Towards Testing Model Transformation Chains Using …msdl.cs.mcgill.ca/conferences/AMT/2014/files/richa_amt.pdf · Unit Testing of Model Transformations • Coverage – Testing all

Thank you!

Credits Slide 5: Alert by Juergen Bauer from The Noun Project