Microservices Architecture Enables DevOps An Experience

Microservices Architecture Enables Devops An Experience-PDF Download

  • Date:01 Jul 2020
  • Views:2
  • Downloads:0
  • Pages:13
  • Size:3.13 MB

Share Pdf : Microservices Architecture Enables Devops An Experience

Download and Preview : Microservices Architecture Enables Devops An Experience


Report CopyRight/DMCA Form For : Microservices Architecture Enables Devops An Experience


Transcription:

Figure 1 Google Trends report for DevOps and Microservices keywords. DevOps and Microservices SIDEBAR DevOps is a set of practices 1 which not only aims to de. crease the time between applying a change to a system and the change being transferred to the production. environment but also insists on keeping the software quality in terms of both code and the delivery mech. anism as one of the key elements in the development process Any techniques that enables the mentioned. goal is considered as a DevOps practice 1 5, Continuous Delivery CD 6 is a DevOps practice that enables on demand deployment of a software to. any environment through the automated machinery CD is an essential counterpart for microservices as the. number of deployable units increases Yet another critical DevOps practice is the Continuous Monitoring. CM 7 which not only provides developers with performance related feedbacks but also facilitates detecting. any operational anomalies 5, 2 The Architectural Concerns for Microservices Migration. 2 1 The Architecture of Backtory Before the Migration. Backtory is a commercial MBaaS platform that has been developed in PegahTech Co http www. pegahtech ir Backtory provides mobile developers who does not know any server side programming. languages with backend services The original functionality of Backtory was a RDBMS as a Service De. velopers could define their database schemas in the Backtory s developer dashboard and Backtory would. provide them an SDK for their desired target platform e g Android or iOS Afterwards the developers. can only code in their desired platforms using their domain objects and the objects would make some service. calls on their behalf in order to fulfill their requests Over time new services are being added to Backtory. like Chat as a Service Indexing as a Service NoSQL as a Service and so on. Backtory is written in Java using the Spring framework The underlying RDBMS is an Oracle 11g Maven. is used for fetching dependencies and building the project All of the services were in a Git repository and. the modules feature of Maven was used to build different services The deployment of services to development. machines was done using the Maven s Jetty plugin However the deployment to the production machine. was a manual task The architecture of Backtory before the migration to microservices is illustrated in. Figure 2 a In this figure solid arrows and dashed arrows respectively illustrate the direction of service calls. and library dependencies Figure 2 a also demonstrates that Backtory consisted of five major components. see Appendix A, 2 2 Why Did We Plan to Migrate Towards the Microservices in the First Place. What motivated us to perform a migration to a microservices architecture was an issue raised with a require. ment for a Chat as a Service To implement this requirement we chose ejabberd due to its known built in. scalability and its ability to run on clusters To this end we wrote a python script that enabled ejabberd. to perform authentications using our Backtory platform The big issue in our service was the on demand. capability This issue made us to perform a series of actions which resulted in raising new motivations for. migration to microservices, The need for reusability To address the on demand capability we started to automate the process of. setting up a chat service One of these steps was to spin off a database for each user In our system. there was a pool of servers each of which had an instance of the Oracle DBMS installed and an instance. of DeveloperServices running During the RDBMS instantiation a server was selected randomly and. related users and table spaces were created in the Oracle server This had several issues since it was. just designed to fulfill the RDBMS service needs and it was tightly coupled to the Oracle server. Consequently we needed a database reservation system that both of our services could use. The need for decentralized data governance Another issue was that every time anyone intended to. add some metadata about different services they were added to DeveloperData This was not a good. practice since services are independent units that only share their contracts with other parts of the. The need for automated deployment As the number of services was growing another problem was to. automate the deployment process and to decouple the build life cycle of each service from others. The need for built in scalability The vision of Backtory is to serve millions of users By increasing the. number of services we needed a new approach for handling such kind of scalability because scaling. services individually requires major efforts and can be error prone if not handled properly Therefore. our solution was to locate service instances dynamically through the Service Discovery 4 component. and balancing the load among them using an internal Load Balancer component. 2 3 The Target Architecture of Backtory After the Migration. To realize microservices architecture and to satisfy our new requirements we transformed the Backtory s core. architecture to a target architecture by performing some refactorings These changes included introducing. microservices specific components and re architecting the system. In the state of the art about microservices 8 9 Domain Driven Design and Bounded Context are. introduced as common practices to transform the system s architecture into microservices 10 As we did not. have a complex domain we decided to re architect the system based on domain entities in the DeveloperData. The final architecture is depicted in Figure 2 g, The new technology stack for Backtory was the Spring Boot for embedded application server and fast.
service initialization the operating system s environment variables for configuration and the Spring Cloud s. Context and the Config Server for separating the configuration from the source code as recommended by. the CD practices Additionally we chose the Netflix OSS for providing some of the microservices specific. components i e Service Discovery 4 and the Spring Cloud Netflix that integrates the Spring framework. with the Netflix OSS project We also chose Eureka for Service Discovery 4 Ribbon as Load Balancer. Hystrix as Circuit Breaker 11 and Zuul as Edge Server 4 that all are parts of the Netflix OSS project. We specifically chose Ribbon among other load balancers e g HAProxy because of its integration with the. Spring framework and other Netflix OSS projects in particular Eureka. Figure 2 a The architecture of Backtory before the migration b Transforming DeveloperData to a. Service c Introducing Configuration Server d Introducing Edge Server e Introducing Dynamic Service. Collaboration f Introducing ResourceManager g Target architecture of Backtory after the migration. h The final delivery pipeline i The monitoring and performance feedback infrastructure j DevOps. team formation,3 The Migration Process, The changes we made throughout the migration process included i architectural refactorings and ii some. necessary changes to enable DevOps,3 1 Architectural Refactoring. Migrating the system towards the target architecture was not a one step procedure and we performed it. incrementally without affecting the end users We treated the migration steps as architectural changes. adding or removing components that consists of two states i before the migration and ii after the. 3 1 1 Preparing the Continuous Integration Pipeline. Continuous Integration CI is the first step towards CD It allows developers to integrate their work with. the others early and on a regular basis and helps to prevent future conflicts 6 To this end a CI server an. as a service or self hosted code repository and an artifact repository are needed We chose Jenkins as the CI. server self hosted Gitlab as the code repository and Artifactory as the artifact repository cf Figure 2 h. Since services can have a number of instances running deploying microservices using virtualization is. not cost effective and introduces a heavy computational overhead Furthermore we needed to use the. Configuration Management systems in order to create the production and test environments. By utilizing containers we could deploy service instances with lower overheads than the virtualization. and with a better isolation Another major benefit is the portability since we could deploy anywhere that. supports containerization without any changes to our source codes or container images Docker is a tool for. the containerization of applications 12 As we were going to use Docker we needed the Docker Registry to. be in our pipeline as well, To summarize in this step we integrated the Gitlab Jenkins Artifactory and Docker Registry as a CI. pipeline As can be seen in Figure 2 h the fundamental difference between this delivery pipeline with a. monolithic one is in the existence of independent pipeline delivery for each service thereby each of them can. be deployed independently Previously we were using integration tests which needed running the whole set of. tests in case of a change in just one service We replaced integration tests with consumer driven contracts 13. and tests which leaded to independent testing of each service using expectations of their consumers This. change minimizes the inter team coordination which despite of a more complex testing strategy enables. forming smaller teams as a DevOps practice,3 1 2 Transforming DeveloperData to a Service. In this step we changed the DeveloperData to use Spring Boot because of its advantages see Section 2 3. Furthermore as shown in Figure 2 b we changed it to expose its functionalities as a RESTful API In. this way its dependent services would not be affected when the internal structure of DeveloperData changes. Since they have service level dependency the governance of DeveloperData entities will be done by a single. service and DeveloperData would not act as an Integration Database 14 for its dependent services anymore. Accordingly we adapted DeveloperServices and ContentServices to use DeveloperData as a service and not as. a Maven dependency,3 1 3 Introducing Continuous Delivery.
One of the best practices in the CD is to separate the source code the configuration and the environment. specification so that they can evolve independently 6 In this way we can change the configuration without. redeploying the source code By leveraging Docker we removed the need for specifying environments since. the Docker images produce the same behavior in different environments In order to separate the source code. and the configuration we ported every service to Spring Boot and changed them to use the Spring Cloud. Configuration Server and the Spring Cloud Context for resolving their configuration values cf Figure 2 c. In this step we also separated services code repositories to have a clearer change history and to separate. the build life cycle of each service We also created the Dockerfile for each service that is a configuration for. creating Docker images for that service We then created a CI job per service and ran them to populate our. repositories Having the Docker image of each service in our private Docker registry we were able to run. the whole system with Docker Compose using only one configuration file Starting from this step we had. an automated deployment on a single server,3 1 4 Introducing Edge Server. As we were going to re architect the system and it was supposed to change the internal service architecture. we introduced Edge Server 4 to the system to minimize the impact of internal changes on end users as. shown in Figure 2 d Accordingly we adapted the DeveloperWebsite. 3 1 5 Introducing Dynamic Service Collaboration, In this step we introduced the Service Discovery 4 Load Balancer and Circuit Breaker 11 to the system. as shown in Figure 2 e Dependent services should locate each other via the Service Discovery 4 and. Load Balancer and the Circuit Breaker 11 will make our system more resilient during the service calls. By introducing these components to the system we made our developers more comfortable with these new. concepts and it accelerated the rest of the migration process. 3 1 6 Introducing ResourceManager, In this step we introduced the ResourceManager by factoring out the entities that were related to servers. i e the AvailableServer from the DeveloperData and introducing some new features i e MySQL database. reservation for satisfying our chat service requirements cf Figure 2 f Accordingly we adapted the. DeveloperServices to use this service for database reservations. 3 1 7 Introducing ChatServices and DeveloperInfoServices. As the final step in the architectural refactoring we introduced the following services see the target archi. tecture in Figure 2 g, The DeveloperInfoServices by factoring out developer related entities e g the Developer from the. DeveloperData, The ChatServices for persisting chat service instances metadata and handling chat service instance.
3 1 8 Clusterization, In this step we set up a cluster of CoreOS instances with Kubernetes agents installed on them We then. deployed our services on this cluster instead of a single server As can be seen in Figure 2 h independent. testing of services using consumer driven tests enabled us to also deploy each service independently Hence. the occurrence of a change in a service would not result in the redeployment of the whole system anymore. 3 2 Crosscutting Changes, 3 2 1 Filling the Gap Between the Dev and Ops via Continuous Monitoring. Microservices Architecture Enables DevOps An Experience Report on Migration to a Cloud Native Architecture Armin Balalaie Abbas Heydarnoori Pooyan Jamshidi Abstract The microservices architecture is one of the rst service based architectural styles that has been introduced applied in practice and become popular when the DevOps practices gained momentum in the software industry Migrating

Related Books