RECIPE : Converting Concurrent DRAM Indexes to Persistent...

70
RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon Lee, Jayashree Mohan, Sanidhya Kashyap, Taesoo Kim, Vijay Chidambaram 1

Transcript of RECIPE : Converting Concurrent DRAM Indexes to Persistent...

Page 1: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes

Se Kwon Lee, Jayashree Mohan, Sanidhya Kashyap, Taesoo Kim, Vijay Chidambaram

1

Page 2: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

Persistent Memory (PM)

2

Intel Optane DC Persistent Memory

• New storage class memory technology

• Performance similar to DRAM

• Non-volatile & high-capacity

• Up-to 6TB on a single machine

Page 3: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

3

• PM has high capacity and low latency• 6TB on a single machine à 100 billion 64-byte key-value pairs

• Indexing data on PM is crucial for efficient data access

Indexing on PM

Page 4: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

PM Indexes need to achieve three goals simultaneously

PM Indexes

4

PM

Crash ConsistencyConcurrencyCache

Efficiency

Page 5: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

PM

• Cache Efficiency• Persistent memory is attached to the memory bus• 3x higher latency than DRAM à More cache-sensitive

PM Indexes

5

Crash ConsistencyConcurrencyCache

Efficiency

Page 6: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

PM

• Concurrency• High concurrency is necessary for scalability on any modern

multicore platform

PM Indexes

6

Crash ConsistencyConcurrencyCache

Efficiency

Page 7: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

PM

• Crash Consistency• CPU cache is still volatile• Arbitrarily-evicted cache lines à Persistence reordering

PM Indexes

7

Crash ConsistencyConcurrencyCache

Efficiency

Page 8: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

logcommit

Core

Cache

• Crash Consistency• CPU cache is still volatile• Arbitrarily-evicted cache lines à Persistence reordering

PM Indexes

8

Volatile

Persistent

Program orderwrite (log);write (commit);

① ②

commitlog

Page 9: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

commit

Core

Cache

• Crash Consistency• CPU cache is still volatile• Arbitrarily-evicted cache lines à Persistence reordering

PM Indexes

9

Volatile

Persistent

Persistence reorderingwrite (log);write (commit);log Reordered

commit

Page 10: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

commit

commit

Core

Cache

• Crash Consistency• CPU cache is still volatile• Arbitrarily-evicted cache lines à Persistence reordering

PM Indexes

10

Volatile

Persistent

Persistence reorderingwrite (log);write (commit);log

commit InconsistencyCrash

Page 11: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

Core

Cache

• Crash Consistency• CPU cache is still volatile• Arbitrarily-evicted cache lines à Persistence reordering

• Flush: persist writes to PM• Fence: ensure one write prior another to be persisted first

PM Indexes

11

Volatile

Persistent commitlog

Consistent persistence orderingwrite (log)flush (log)fence ()write (commit)flush (commit)fence ()

Page 12: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

Challenge in building PM indexes

12

Correct

Concurrency

CorrectCrash

Consistency

ConsistentData

Conflict Crash

Correctness condition: return previouslyinserted data without data loss or corruption

Page 13: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

Bug

Bug

Challenge in building PM indexes

13

Concurrency and crash consistency interact witheach other, a bug in either can lead to data loss

Bug

Concurrency

BugCrash

Consistency

DataLoss

Conflict Crash

Page 14: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

• We found bugs in FAST&FAIR [FAST’18] and CCEH [FAST’19]• FAST&FAIR: Concurrent PM-based B+Tree

• One bug in concurrency mechanism• Two bugs in recovery mechanism

• CCEH: Concurrent PM-based dynamic hash table• One bug in concurrency mechanism• One bug in recovery mechanism

Bug in Concurrent PM Index

14

Page 15: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

15

How can we reduce the effort involved in building concurrent, crash-consistent PM

indexes?

Answer: We can convert concurrent DRAM indexes to PM indexes with low effort

Insight: Isolation and Crash Consistency are similar

Page 16: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

16

How can we reduce the effort involved in building concurrent, crash-consistent PM

indexes?

Approach: Convert concurrent DRAM indexes to PM indexes with low effort

Insight: Isolation and Crash Consistency are similar

Page 17: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

• Already designed for cache efficiency and concurrency

DRAM Index

17

Concurrency

CacheEfficiency

T-Tree

CSS-Tree CSB+

Tree

BD-Tree

FAST

Bw-Tree

ART

HOT

Masstree

1986

CLHT

2010 2019

Cac

he E

ffici

ency

Con

curre

ncy

Time

Page 18: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

18

Crash

DRAM Index

Volatile

Concurrency

Cache Efficiency

DRAM Index

Concurrency

Cache Efficiency

Crash Vulnerable

DRAM Index

Page 19: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

Challenge in Conversion

19

• Require minimal changes to DRAM index• Without modifying the original design principles of DRAM index

DRAM Index

Volatile

Concurrency

Cache Efficiency

PM Index

Concurrency

Cache Efficiency

Crash Consistency

Conversion

Page 20: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

201. Steven Pelley et al., Memory Persistency, ISCA’14

• Similar semantics between isolation and consistency1

• Isolation• Return consistent data while multiple active threads are running

• Crash consistency• Return consistent data even after a crash happens at any point

Insight for Conversion

Page 21: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

• Similar semantics between isolation and consistency1

• Isolation• Return consistent data while multiple active threads are running

• Crash consistency• Return consistent data even after a crash happens at any point

211. Steven Pelley et al., Memory Persistency, ISCA’14

Insight for Conversion

Approach: reuse mechanisms for isolation in DRAM indexes to obtain crash consistency

Page 22: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

RECIPE

• Principled approach to convert DRAM indexes into PM indexes

• Case study of changing five popular DRAM indexes

• Conversion involves different data structures such as Hash Tables, B+ Trees, and Radix Trees

• Conversion required modifying <= 200 LOC

• Up-to 5.2x better performance in multi-threaded evaluation

22

RECIPE

https://github.com/utsaslab/RECIPE

Page 23: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

• Overall Intuition• Conversion Conditions• Conversion Example: Masstree• Assumptions & Limitations• Evaluation

Outline

23

Page 24: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

• Overall Intuition• Conversion Conditions• Conversion Example: Masstree• Assumptions & Limitations• Evaluation

Outline

24

Page 25: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

Overall Intuition for Conversion

25

• Blocking algorithms• Use explicit locks to prevent the conflicts of threads to shared

data• Non-blocking algorithms

• Use well-defined invariants and ordering constraints without locks

• Employed by most high-performance DRAM indexes

Page 26: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

Overall Intuition for Conversion

26

• Non-blocking algorithms• Readers Detect and Tolerate inconsistencies

• E.g., Ignore duplicated keys

InconsistencyReader

Tolerate

Detect

Page 27: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

Overall Intuition for Conversion

27

• Non-blocking algorithms• Readers Detect and Tolerate inconsistencies

• E.g., Ignore duplicated keys• Writers also Detect, but Fix inconsistencies

• E.g., Eliminate duplicated keys

WriterDetect Fix

ConsistentInconsistency

Page 28: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

Overall Intuition for Conversion

28

• Non-blocking algorithms• Readers Detect and Tolerate inconsistencies• Writers also Detect, but Fix inconsistencies• Helping mechanism1 ≈ Crash Recovery2

• Such indexes are *inherently* crash consistent

WriterDetect Fix

ConsistentInconsistency

1. Keren Censor-Hillel et al., Help!, PODC’152. Ryan Berryhill et al., Robust shared objects for non-volatile main memory, OPODIS’15

Page 29: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

• Not all DRAM indexes can be converted with low effort

• Exploit inherent crash recovery in the index

• Provide specific conditions that must hold for a DRAM index to be converted

• Provide a matching conversion actions for each condition

29

Page 30: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

• Overall Intuition• Conversion Conditions• Conversion Example: Masstree• Assumptions & Limitations• Evaluation

Outline

30

Page 31: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

• Condition 1: Updates via Single Atomic Store• Condition 2: Writers fix inconsistencies• Condition 3: Writers don’t fix inconsistencies• Conditions are not exhaustive!

Three Conversion Conditions

31

Page 32: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

• Condition 1: Updates via Single Atomic Store• Condition 2: Writers fix inconsistencies• Condition 3: Writers don’t fix inconsistencies

Three Conversion Conditions

32

Page 33: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

Condition 1: Updates via Single Atomic Store

33

• Non-blocking readers, (Non-blocking or Blocking) writers• Updates become visible to other threads via single

atomic commit store

Atomic Store

CrashStep 1 Step 2 Step N… Step 1 Step 2 Step N…

Invisible Visible

Page 34: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

Condition 1: Updates via Single Atomic Store

34

• Updates become visible to other threads via single atomic commit store

• Conversion: Add flushes after each store and bind final atomic store using fences

Atomic Store

Step 1 Step 2 Step N… Step 1 Step 2 Step N…

Flushes

Flush

FenceFence

Page 35: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

• Condition 1: Updates via Single Atomic Store• Condition 2: Writers fix inconsistencies• Condition 3: Writers don’t fix inconsistencies

Three Conversion Conditions

35

Page 36: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

Condition 2: Writers fix inconsistencies

36

• Non-blocking readers and writers (don’t hold locks)• Readers & Writers à Detect ( ), Tolerate ( ), Fix ( )

Step 1 Step 2 Step 3

Update

Writer 1

A sequence of ordered deterministic steps

CommitStep

Page 37: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

Step 3

Condition 2: Writers fix inconsistencies

37

• Non-blocking readers and writers (don’t hold locks)• Readers & Writers à Detect ( ), Tolerate ( ), Fix ( )

Step 1 Step 2Writer 1Reader

Detect

CommitStep

Tolerate

Page 38: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

Condition 2: Writers fix inconsistencies

38

• Non-blocking readers and writers (don’t hold locks)• Readers & Writers à Detect ( ), Tolerate ( ), Fix ( )

Step 1 Step 2Writer 1Writer 2

Step 3

Fix

Detect

CommitStep

Page 39: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

PM

Condition 2: Writers fix inconsistencies

39

• Readers & Writers à Detect ( ), Tolerate ( ), Fix ( )• Inherently crash recoverable

Step 1 Step 2 Step 3Writer 1 CommitStep

Page 40: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

PM

Condition 2: Writers fix inconsistencies

40

• Readers & Writers à Detect ( ), Tolerate ( ), Fix ( )• Inherently crash recoverable

Step 1 Step 2 Step 3Writer 1

CrashCommit

Step

Page 41: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

PM

Condition 2: Writers fix inconsistencies

41

• Readers & Writers à Detect ( ), Tolerate ( ), Fix ( )• Inherently crash recoverable

Step 1 Step 2Thread Step 3

Recover

CommitStep

Page 42: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

PM

Condition 2: Writers fix inconsistencies

42

• Readers & Writers à Detect ( ), Tolerate ( ), Fix ( )• Inherently crash recoverable• Conversion: Adding flushes and fences after each store and

specific loads

Step 2Thread Step 3

Flushes Flushes Flushes Flushes

Fence Fence Fence

Step 1 CommitStep

Page 43: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

• Condition 1: Updates via Single Atomic Store• Condition 2: Writers fix inconsistencies• Condition 3: Writers don’t fix inconsistencies

Three Conversion Conditions

43

Page 44: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

• Non-blocking readers, Blocking writers (hold locks)• Readers & Writers à Detect ( ), Tolerate ( ), Fix (X)

Condition 3: Writers don’t fix inconsistencies

44

Step 1 Step 2 Step 3Writer 1

Step 1 Step 2 Step 3Writer 2

Update

A sequence of ordered deterministic steps

UpdateCommit

Step

CommitStep

A sequence of ordered deterministic steps

Page 45: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

• Non-blocking readers, Blocking writers (hold locks)• Readers & Writers à Detect ( ), Tolerate ( ), Fix (X)

Condition 3: Writers don’t fix inconsistencies

45

Step 1 Step 2 Step 3Writer 1

Step 1 Step 2 Step 3Writer 2

CommitStep

CommitStep

Page 46: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

• Non-blocking readers, Blocking writers (hold locks)• Readers & Writers à Detect ( ), Tolerate ( ), Fix (X)

Condition 3: Writers don’t fix inconsistencies

46

Step 1 Step 2Writer 1

Step 1 Step 2 Step 3Writer 2

Crash

CommitStep

Failure

Page 47: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

Writer 2

• Readers & Writers à Detect ( ), Tolerate ( ), Fix ( )• Conversion: Add helping mechanism

• Reuse existing algorithm handling each step

Condition 3: Writers don’t fix inconsistencies

47

Step 1 Step 2 Step 3Thread

Step 1 Step 2

Detect

Fix CommitStep

Page 48: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

• Overall Intuition• Conversion Conditions• Conversion Example: Masstree• Assumptions & Limitations• Evaluation

Outline

48

Page 49: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

• Example: B-link Tree (Masstree)

Conversion of Masstree

49

10

1 10 15 25

30

30

High Key

High Key

Page 50: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

• Example: B-link Tree (Masstree)

Conversion of Masstree

50

Insert 261510

1 10

30

30

15 251. installing new sibling

2. insert middle key to parent

Page 51: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

• Example: B-link Tree (Masstree)

Conversion of Masstree

51

Insert 2610

1 10

30

30

15 25Intermediate state

Page 52: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

• Example: B-link Tree (Masstree)

Conversion of Masstree

52

Insert 2610

1 10

30

30

Lookup 25

15 25

Page 53: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

• Example: B-link Tree (Masstree)

Conversion of Masstree

53

Insert 2610

1 10

30

30

Lookup 25

15 25

Detect(25 > 15)

>>>>Tolerate

Page 54: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

• Example: B-link Tree (Masstree)

Conversion of Masstree

54

10

1 10

30

30

15 25

Crash Permanent Inconsistency

Page 55: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

• Example: B-link Tree (Masstree)• Add helping mechanism to resume split

Conversion of Masstree

55

10

1 10

30

30

15 25

Insert 30

Detect

15Resume split (recovery)

Page 56: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

Conversion Results of Five DRAM Indexes

56

DRAM Index DS Type

CLHT (Cache-Line Hash Table) [ASPLOS’15] Hash tableHOT (Height Optimized Trie) [SIGMOD’18] TrieBwTree [ICDE’13] B+TreeART (Adaptive Radix Tree) [ICDE’13] Radix TreeMasstree [Eurosys’12] Hybrid (B+Tree & Trie)

Page 57: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

Conversion Results of Five DRAM Indexes

57

DRAM Index PM Index ConditionCLHT P-CLHT #1HOT P-HOT #1

BwTree P-BwTree #1, #2ART P-ART #1, #3

Masstree P-Masstree #1, #3

• We produce the P-* family of PM indexes

Page 58: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

• Overall Intuition• Conversion Conditions• Conversion Example: Masstree• Assumptions & Limitations• Evaluation

Outline

58

Page 59: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

• Assume garbage collection in memory allocator• Assume locks are volatile or re-initialized after a crash• Provide low level of isolation: Read Uncommitted• RECIPE applies only to individual data structures

Assumptions & Limitations

59

Page 60: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

• Overall Intuition• Conversion Conditions• Conversion Example: Masstree• Assumptions & Limitations• Evaluation

Outline

60

Page 61: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

Evaluation

61

• How much effort is involved in converting indexes?• What is the performance of converted indexes?• Are the converted indexes crash consistent?

Page 62: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

Evaluation

62

• How much effort is involved in converting indexes?• What is the performance of converted indexes?• Are the converted indexes crash consistent?

Page 63: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

Evaluation

63

• How much effort is involved in converting indexes?• What is the performance of converted indexes?

Page 64: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

Modified Lines of Code

64

• Conversion for all indexes à <= 200 LoC changes

RECIPE-converted IndexesLines of Code

Index Core ModifiedP-CLHT 2.8K 30 (1%)P-HOT 2K 38 (2%)

P-BwTree 5.2K 85 (1.6%)P-ART 1.5K 52 (3.4%)

P-Masstree 2.2K 200 (9%)

Page 65: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

Modified Lines of Code

65

• Conversion for all indexes à <= 200 LoC changes

RECIPE-converted IndexesLines of Code

Index Core ModifiedP-CLHT 2.8K 30 (1%)P-HOT 2K 38 (2%)

P-BwTree 5.2K 85 (1.6%)P-ART 1.5K 52 (3.4%)

P-Masstree 2.2K 200 (9%)

Conversion for all indexes: <= 200 LoC changes<= 9% from core code base

Page 66: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

Evaluation

66

• How much effort is involved in converting indexes?• What is the performance of converted indexes?

Page 67: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

• 2-socket 96-core machine with 32MB LLC• 768 GB Intel Optane DC PMM, 378 GB DRAM• YCSB with 16 threads• Ordered/Unordered indexes, Integer/String keys

Performance Evaluation

67

Load Workload A Workload B Workload C Workload E

Insertion 100% Insertion 50%Point Lookup 50%

Insertion 5%Point Lookup 95%

Point Lookup 100%

Insertion 5%Range Scan 95%

Page 68: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

Ordered Index

68

• Support both point and range operations• P-HOT

• Persistent Height-Optimized Trie converted by RECIPE• FAST & FAIR [FAST’18]

• Hand-crafted PM-based concurrent B+Tree

Page 69: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

Ordered Index

69

• P-HOT produced by RECIPE conversion• P-HOT performs up-to 5.2x better in point operations• Cache-efficient designs of P-HOT à Low cache misses

0123456

Load A B C E

Nor

mal

ized

thro

ughp

ut

FAST&FAIR P-HOT

Page 70: RECIPE : Converting Concurrent DRAM Indexes to Persistent …nvmw.ucsd.edu/nvmw2020-program/unzip/current2/nvmw2020... · 2020-02-20 · Resume split (recovery) Conversion Results

RECIPE• Principled approach to convert concurrent DRAM

indexes into PM indexes• Case study of changing five DRAM indexes• Evaluations with YCSB show RECIPE indexes have

better performance than hand-crafted PM indexes• Try our indexes: https://github.com/utsaslab/RECIPE

70

RECIPE