Dégraissons le mammouth ou Darwin a encore frappé - La théorie de l'évolution appliquée au...

82
DEgraissons le Mammouth Je suis pas gros, j'ai une ossature lourde! Arnauld Loyer @aloyer 29ème semaine de Conférences Technologiques 2014

description

Dégraissons le mammouth ou Darwin a encore frappé La théorie de l'évolution appliquée au développement informatique - cas pratique de l'architecture du site PMU.fr Depuis 1980, Lehman nous avertit: un programme doit évoluer ou péricliter, mais alors qu'il devient de plus en plus gros, la complexité résultante tend à limiter son évolution. Comment remédier à cela? Quelle architecture adopter pour un site à fort trafic comme celui du PMU? Après avoir abordé les problématiques d'évolution et de maintenance d'une application monolithique, nous verrons pourquoi et surtout comment séparer les composants et les comportements de notre application. Du monolithe aux micro services, du distribué, des messages, du publish/subscribe, du REST, une approche polyglotte, ... au cours de cet exposé, nous verrons quelques uns des choix retenus pour garantir la survie et l'évolution de notre application. Nous verrons comment nous avons construit un socle solide permettant de répondre aux nouvelles manières de faire du Web, d'être adapté aux applications mobiles et aux télés connectées. Ce sera l'occasion d'aborder aussi bien les principes architecturaux et les principes organisationnels qui nous ont permis d'atteindre cet objectif.

Transcript of Dégraissons le mammouth ou Darwin a encore frappé - La théorie de l'évolution appliquée au...

  • 1.DEgraissons le Mammouth Je suis pas gros, j'ai une ossature lourde! Arnauld Loyer @aloyer 29me semaine de Confrences Technologiques 2014

2. Back To The PAST! 3. 2008 2010 4. 2010 2008-2009 5. PMU Turf Partner Paddy Power Paris Sportifs https://info.pmu.fr/entreprise/partenariats-et-sponsoring Paris Hippiques Poker Partner PartyGaming PMU Entreprise 6. ooo 7. ooo Portlet Portal: A Good Promise! The Next-Gen Architect An Enterprise Solution 8. ooo Portlet Portal: A Good Promise! Portlet App 2 Portlet App 1 Portlet App 3 P O R T A L Portlet Container Portlet API 9. Page JS Content Portal: A Good Promise! User Auth +1 Enterprise Portal Portlet CMS Content Management System Marketing +1 SSO 10. Page JS Content Business Logic User Auth Enterprise Portal PortletSSO 11. User Auth Page JS Content Business Logic Enterprise Portal PortletSSO SI Auth 12. How to Plug our own Authentication Search doc. Do It Yourself Google stack overflow MAN/API Sample Just kidding! obsolete Give UP ! & B X p D U g I o a n Hack Framework Does Not Work 13. Customization Enterprise Portal User Auth Page Content Business Logic SI Auth PortletSSO JS 14. Customization Enterprise Portal User Auth Page Content Business Logic SI Auth PortletSSO JS User Portal User SI 15. Customization Enterprise Portal User Auth Page Content Business Logic SI Auth PortletSSO JS User Portal User SI 16. Customization Enterprise Portal User Auth Page Content Business Logic SI Auth PortletSSO JS User Portal User SI 17. Customization Enterprise Portal User Auth Page Content Business Logic SI Auth PortletSSO JS User Portal User SI 18. Framework Library 1 Framework vs Library Library 2 Application Side N ote 19. Customization Enterprise Portal User Auth Page JS Content Business Logic API Mobile App.Portal: A Good Promise! euh no SI Auth 3rd Party Logic PortletSSO 20. LEGACY... 21. LEGACY... Its OLD Its Crapy Its complicated Its not understandable It does not reflect what it does Its not Tested Its not Testable ! Someone else wrote it! 22. Accidental Complexity 23. Cost of understanding 24. Cost of understanding Knowledge scattering 25. Cost of understanding Knowledge scattering Ripple Effect Big Ball of Mud http://www.laputan.org/mud/ http://akvo.org/blog/the-ball-of-mud-transition/ h 26. time Productivity / Cost of Ownership Understanding Onboarding 27. time Velocity Productivity Productivity / Cost of Ownership Understanding Onboarding 28. Scaling 29. Customization Enterprise Portal User Auth Page JS Conte nt Business Logic API SI Auth 3rd Party Logic PortletSSO Customization Enterprise Portal User Auth Page JS Conte nt Business Logic API SI Auth 3rd Party Logic PortletSSO User Manny 1.1 1.2 Load Balancer Session 1.3 30. Customization Enterprise Portal User Auth Page JS Conte nt Business Logic API SI Auth 3rd Party Logic PortletSSO Customization Enterprise Portal User Auth Page JS Conte nt Business Logic API SI Auth 3rd Party Logic PortletSSO User Manny Load Balancer Session 1.4 1.5 1.1 1.2 1.3 31. Customization Enterprise Portal User Auth Page JS Conte nt Business Logic API SI Auth 3rd Party Logic PortletSSO Customization Enterprise Portal User Auth Page JS Conte nt Business Logic API SI Auth 3rd Party Logic PortletSSO User Manny 2.1 Load Balancer ? 32. Customization Enterprise Portal User Auth Page JS Conte nt Business Logic API SI Auth 3rd Party Logic PortletSSO Customization Enterprise Portal User Auth Page JS Conte nt Business Logic API SI Auth 3rd Party Logic PortletSSO User Manny 2.1 Load Balancer ? Session Sticky Session 33. #procs Multiple servers Single server Scaling 34. Mammouth is Seducing but does not Scale well 35. 2014 Gartner, Inc. and/or its affiliates. All rights reserved. Monolithic design Over-provisioning and hoarding of resources Stateful No support of parallel or in-memory computing Not open for programmatic access Tightly-coupled Not modular Not designed for 24/7 use Discontinuous versioning Not horizontally-scalable Not instrumented for governance Single-channel front-end Complex Expensive Locked into platforms and vendors The Old Architecture Does Not Measure Up http://public.brighttalk.com/resource/core/36753/may_8_software_designed_arch_ynatis_54295.pdf may 8, 2014h 36. 2014 Gartner, Inc. and/or its affiliates. All rights reserved. Monolithic design Over-provisioning and hoarding of resources Stateful No support of parallel or in-memory computing Not open for programmatic access Tightly-coupled Not modular Not designed for 24/7 use Discontinuous versioning Not horizontally-scalable Not instrumented for governance Single-channel front-end Complex Expensive Locked into platforms and vendors The Old Architecture Does Not Measure Up http://public.brighttalk.com/resource/core/36753/may_8_software_designed_arch_ynatis_54295.pdf may 8, 2014 dEVOPS movement 37. It is not the strongest of the species that survives, nor the most intelligent, but the one most responsive to change. ! Charles Darwin 38. cockroaches may not look really sexy at first, but they survive 39. http://www.faqs.org/docs/artu/ch01s06.html Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface. Unix Philosophy 40. http://www.faqs.org/docs/artu/ch01s06.html Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface. Unix Philosophy Small is beautiful Make each program do one thing and one thing well Favor simplicity and portability over feature completeness Write programs to work together 41. Rule of Modularity: Write simple parts connected by clean interfaces. Rule of Clarity: Clarity is better than cleverness. Rule of Composition: Design programs to be connected to other programs. Rule of Separation: Separate policy from mechanism; separate interfaces from engines. Rule of Simplicity: Design for simplicity; add complexity only where you must. Rule of Parsimony: Write a big program only when it is clear by demonstration that nothing else will do. Rule of Transparency: Design for visibility to make inspection and debugging easier. Rule of Robustness: Robustness is the child of transparency and simplicity. Rule of Representation: Fold knowledge into data so program logic can be stupid and robust. Rule of Least Surprise: In interface design, always do the least surprising thing. Rule of Silence: When a program has nothing surprising to say, it should say nothing. Rule of Repair: When you must fail, fail noisily and as soon as possible. Rule of Economy: Programmer time is expensive; conserve it in preference to machine time. Rule of Generation: Avoid hand-hacking; write programs to write programs when you can. Rule of Optimization: Prototype before polishing. Get it working before you optimize it. Rule of Diversity: Distrust all claims for one true way. Rule of Extensibility: Design for the future, because it will be here sooner than you think. http://www.faqs.org/docs/artu/ch01s06.html Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface. Unix Philosophy 42. http://www.faqs.org/docs/artu/ch01s06.html Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface. Unix Philosophy Read a le of text, determine the n most frequently used words, and print out a sorted list of those words along with their frequencies. ! http://www.leancrew.com/all-this/2011/12/more-shell-less-egg/ http://franklinchen.com/blog/2011/12/08/revisiting-knuth-and-mcilroys-word-count-programs/h 43. EVOLUTION Learn From THE PAST Learn from the ELDERs 44. Components 45. Components 46. Account management Racing & Horse Information Distribution Horse RACE Betting Extra CONTENT MANAGEMENT Authentication Information Updates RACE Status WINNERS RUNNERS METEO Odds update Login/OUT Profile G Y N U2 H E u 47. Components Shared Libs 48. Components micro-services 10-100 LOC!!! Mammouth cockroach Tardigrade hhttp://vimeo.com/79866979 http://martinfowler.com/articles/microservices.html 49. Message Bus Communication 50. Communication 51. Communication Broker less approach e.g.: HTTP/REST, ZeroMQ, distributed actors/process, http://zeromq.org/whitepapers:brokerless http://www.openstack.org/assets/presentation-media/zmqslides2.pdf h 52. Front End 53. Account management Racing & Horse Information Distribution Horse RACE Betting Extra CONTENT MANAGEMENT Authentication Information Updates RACE Status WINNERS RUNNERS METEO Odds update Login/OUT Profile G Y N U2 H E u 54. Account management Racing & Horse Information Distribution AccountTurfInfoTurfPari Update Listener Horse RACE Betting Information Updates TurfInfo Lib Account Lib TurfPari Lib Update Lib Auth Lib DTO Lib 55. AccountTurfInfoTurfPari Update Listener TurfInfo Lib Accoun t Lib TurfPari Lib Update Lib Auth Lib DTO Lib Backend ooo REST HTTP/JSON Request -> Response RIA backbone jquery underscore less 56. BackEnd, fRONtEND AND CMS TEAMS RA A A A AC LLL LLLE AA Backend CMS FrontEnd 57. L Backend dev. Backend REWORK Frontend Starts using API BUT IT DOES NOT WORK AS EXPECTED API STILL NOT USED A 3 Months LaTER 58. RA A A A AC LLL LLLE AA Backend CMS FrontEnd TEAMS ReORGANIZATIOn 59. R A A A A AC LL L L LL E A A A A Team 1 Team 2 Team 3 A 60. Spotify and the Feature Teams http://blog.crisp.se/2012/11/14/henrikkniberg/scaling-agile-at-spotify http://blog.crisp.se/2012/11/14/henrikkniberg/scaling-agile-at-spotify 61. Firewall Traffic Manager mysql memcached nginx tomcat tomcat SI http https http://www.couchbase.com/memcached 62. ooo REST HTTP/JSON Request -> Response Backend 63. ooo ooo REST HTTP/JSON Request -> Response Backend ooo 64. Traffic Manager mysql nginx tomcat tomcat SI http https ooo GET /turf/programme/ GET /account/ POST /account/ Firewall 65. Traffic Manager mysql couchbase nginx tomcat tomcat SI http https ooo GET /turf/programme/ GET /account/ POST /account/ Stateless Firewall 66. Traffic Manager mysql couchbase nginx tomcat tomcat SI http https Scaling Stateless Firewall 67. Traffic Manager mysql couchbase nginx tomcat tomcat SI http https Resilience Firewall 68. Traffic Manager mysql couchbase nginx tomcat tomcat SI http https Scaling Firewall 69. Traffic Manager mysql couchbase apache/varnishnginx drupaltomcat tomcat SI http New Requirement https Firewall 70. ooo ooo REST HTTP/JSON Request -> Response Backend ooo 71. REST HTTP/JSON Request -> Response Backend ooo 72. ooo REST HTTP/JSON Request -> Response Backend 73. ooo REST HTTP/JSON Request -> Response WebSocket bidirectional Backend 74. Traffic Manager SI http https Firewall 75. Traffic Manager LVS stunnel haproxy Vert.x SI https/ssl http Firewall 76. Traffic Manager LVS stunnel haproxy Vert.x redis subscribepublish SI https/ssl http Firewall 77. ooo REST HTTP/JSON Request -> Response WebSocket bidirectional APN Backend 78. Be CAREFUL OF ARCHITECT BE CAREFUL OF FRAMEWORK AVOID MONOLITHIC APPLICATION LEGACY IS NEXT DOOR THINK SMALL PREFER STATELESS Some Heuristics 79. Be CAREFUL OF ARCHITECT BE CAREFUL OF FRAMEWORK AVOID MONOLITHIC APPLICATION LEGACY IS NEXT DOOR THINK SMALL PREFER STATELESS Some Heuristics 80. Questions? 81. Gartner Presentation May 8th, 2014 http://public.brighttalk.com/resource/core/36753/may_8_software_designed_arch_ynatis_54295.pdf Some Resources PMU website https://www.pmu.fr/ Vert.x http://vertx.io/ redis http://redis.io/ No Silver Bullet http://en.wikipedia.org/wiki/No_Silver_Bullet Lehman's laws of software evolution http://users.ece.utexas.edu/~perry/work/papers/feast1.pdf Microservices http://vimeo.com/79866979 http://martinfowler.com/articles/microservices.html liferay https://www.liferay.com/ The Big Ball of Mud http://www.laputan.org/mud/mud.html http://blog.codinghorror.com/the-big-ball-of-mud-and-other-architectural-disasters/ http://www.arolla.fr/ SockJS http://sockjs.org/