Microservices with Spring Boot, Netflix OSS and Maven - Monolithic Build

In the overview post I mentioned that it's possible to pack all microservice into one application. This application is fully functionally, without the need of any change, without Eureka and without remote calls. In the demo the module all-in-one showcases this monolithic application. In this short blog entry I will describe how it works.

Actually it's very simple. It's done with the dependency injection of Spring. Nevertheless is a quite cool setup.

Microservice Setup and Configuration

Let's look into the UserController of the frontend-service module and take a look what's normally happens if you start the FrontendApplication.

• the UserController requires the UserHystrixClient 
• the UserHystrixClient requires the UserClient 
• the UserClient has the @FeignClient annotation 
• thereby a Spring service is created, given that the FrontendApplication has this configuration: @EnableFeignClients({"at.rseiler.concept.microservice.client"}) 
• this Spring service is then injected into the UserHystrixClient 
• => that's it. It's this simple.

Monolithic Setup and Configuration

Now the explanation what happens if the MonolithApplication is started.

• the UserController requires the UserHystrixClient (no difference)
• the UserHystrixClient requires the UserClient (no difference)
• the UserClient has the @FeignClient annotation, but no service is created. Because MonolithApplication doesn't has the @EnableFeignClients annotation.
• the UserService of the user-service implements the UserClient and its annotated with @RestController. 
• thereby a Spring service is created 
• this Spring service is then injected into the UserHystrixClient 
• => that's it. It's this simple.

In the end the "magic" is done with the Spring configuration. Because we don't need the Feign clients in the monolithic application we don't need to create them. However the REST endpoints are created, because of the @RestController annotation. So it's if two microservices defines the same endpoint then you get an exception at the startup.

Improved Setup

It's also possible to improve the setup so that the REST endpoints aren't created in the monolithic application. To do that you need to improve the configuration. Each @RestController annotated class needs additional a @Service annotation. In the monolithic application only the @Service are defined as includeFilters for the at.rseiler.concept.microservice.*.rest package and the @RestController annotation will be ignored. In a microservice configuration the configuration is reversed. Only the @RestController is defined as includeFilters.

Conclusion

By the expressive Spring dependency injection configuration it's possible to do very cool things. It's only necessary to think outside of the box and use the power of the configuration options. The only downside could be that it's probably confusing for others if an class has a @RestController annotation and a @Service annotation. So you either good documentation to explain this or keep it simple and don't do both. I chose the second approach for the microservice example.

Microservices with Spring Boot, Netflix OSS and Maven - Overview

In my last blog posts (Microservices: a few thoughts and The Importance of structuring Microservices) I wrote about microservices. Now it's time to go from theory to practice and write some code. Based on Spring Boot, Netflix OSS (Eureka, Feign, Hystrix and Ribbon) and Maven I created a setup for a microservice architecture.

I have put my example microservice project on GitHub. 

What the microservice example does

• It's a microservice system consists out of 3 microservices and one frontend microservice.
• product service: which just returns the data for a product (CPU, keyboard or mouse).
• user service: which just returns the data for a user.
• shopping cart service: which stores for the users the products in their shopping carts.
• frontend service: communicates with all microservices, aggregates the data and generates out of the data a simple HTML page.
• The microservices are grouped by domains.
• Eureka is used to discover the microservices. Multiple instances can be started and new instances will automatically registered into the system and will be used from the existing microservices.
• Ribbon is used transparently to do the load balancing on the client. So there is no need to have static IPs and a manually maintained load balancer.
• The REST clients for the microservices only consists out of interfaces. No cooding needed.
• Hystrix REST clients are used to ensure resilience. The Hystrix REST clients are very effortless to write.
• The REST endpoints are done with Spring`s @RestController and implements the interface of the REST Client to ensure that the REST client and the REST endpoint matches each other.
• Enhanced log output to make you life easier. It's done transparently with a Spring Interceptor and a Feign Interceptor.
• The frontend microservice generates a UUID for each request. This UUID is put into the MDC (Mapped Diagnostic Context) and will be printed out for each log entry. The UUID is passed as a header to the every called microservice - even if the called microservice calls another microservice the UUID is passed along too. Therefor it's possible to know which user request caused which log entry in any microservice.
• Additional a calling stack is generated and logged. The calling stack looks like this: ServiceB <- ServiceA <- /product/1 <- 127.0.0.1
• /product/1 <- 127.0.0.1 is generated by the frontend microservice and shows the IP address of the request and the called endpoint
• ServiceA: the frontend microservice called the ServiceA
• ServiceB: the ServiceA called the ServiceB
• => Therefor you always know who called the microservice. Which isn't always a easy to answer question in a big microservice system. You could improve it furthermore if you would include zipkin.
• It's possible to pack all the microservices together into one monolithic application and this application works without the need to change anything. Therefor you have great flexibility. For example in development you probably don't want to start the complete microservice system on your machine. It's handy if you can just start up everything you need, which could be only a part of the system, packed into one application

Overview: Main Frameworks
I give you an overview over the main frameworks/libraries and what they do.

Spring Boot: It saves a lot of time for the setup. The auto configuration and the spring-cloud-starter-* packages are awesome. In addition the Dependency Injection and the Spring Feign abstraction are great, too.

Eureka: It's a service registry. Each service instance registers itself to Eureka. Afterwards all instances of the services can be requested just with the name of the service. So there is no need for IPs or ports. If another service instance is started then it will be added into the corresponding service group or if a service instance is shutdown then the instance will be removed from Eureka.

Ribbon: It's a client side load balancer working together with Eureka. Spring Boot transparently integrates Ribbon. So you have to do nothing to use it. Ribbon uses the round robin scheduling algorithms.

Hystrix: It's a latency and fault tolerance library designed to isolate points of access to remote systems. I am using annotations to configure Hystrix. It's very easy to use Hystrix and does only require a very little amount of effort. By all means the programmatically side is very easy. The hard part is to react correctly to errors.

Feign: It's a REST client based on the Apache HttpClient which has a great level of abstraction. With Spring the only thing you need to do is to write an interface for you REST endpoint and annotate it with @FeignClient. You can use the Spring MVC annotations instead of the Feign annotations. This has the advantage that you reduce the complexity and the @RestController can implement the interface to ensure that everything is compatible.

The top level modules of the project

• all-in-one: this module includes all microservices and it is fully functionally without any change. Eureka isn't needed anymore and all remote calls will be done with standard Java method calls.
• common: this module contains some common code and following microservices: Eureka, Hystrix Dashboard and Turbine Dashboard.
• frontend-service: this module generates the HTML and calls the other services to get the data.
• product-domain: this module represents the product domain and contains all product microservices.
• user-domain: this module represents the user domain and contains all user microservices.

One repository to rule them all

I deliberately put all microservices into a big project. The reason is that it makes many things easier:

• The need to clone only one repository. Especially for this example no one would like to clone several repositories.
• Refactoring, searching, code navigation and so on works very well if you have everything in one project. Especially if you run the all-in-one monolithic application then it's very handy for debugging and coding.
• Changes can be done globally or locally. If you want to upgrade one library because of a security issue then it's very nice if only one pom.xml file needs to be changed. For example you could change the Spring version for all microservices. Otherwise you would need to clone and to open several projects and fix them all. Still Maven allows you to change the version of a dependency only for a specific module. You have the flexibility. Each module/microservice can be configured individual or mostly everything can be configured globally.
• Only one build is needed. Depending on the size you could build always everything. Or individual domain groups. Or individual microservices. Just execute mvn install wherever you need it.

I would extract the common module into an own repository. This module shouldn't change that often or is interesting to debug. Therefore it shouldn't be painful if it's separated.

With the current structure there is one problem: you can't build the project directly after you have cloned it. It's necessary to build the common/common-configuration module first. Because it creates an artifact which is needed and sadly Maven can't resolve this dependency correctly.

If the project begins to grow then possible it makes sense to split the project further more. For example you could create an own repository for each domain-module. But it's up to you.

Microservice Structure

Each microservice is located in a domain-module with the exception of the frontend-service. Because the frontend-service aggregates all domains. In order to have a consistent design and usability. In the frontend-service should only be very little logic. It should only requests and aggregates data from the endpoints and prepares the data to be displayed. But it shouldn't have any business logic.

A microservice consists of two parts:

• microservice-client: contains the Feign interface for the REST endpoint and the model/POJO classes. Additional there is a Hystrix-REST-client which just wraps the Feign-client. Actual everyone should use the Hystrix implementation instead of the Feign client.
• microservice-service: is the microservice itself with the business logic and the REST endpoints. It has a dependency to the microservice-client. Since the model/POJO classes are needed and it implements the REST interface. Thereby fewer mistakes can happen.

Maven

I am using my parent-pom for this project. This POM file enables several useful features like static code analysis. 

 In common-configuration the logback.xml is defined. The file will be packed into an Maven artifact and then extracted into each service-module. Thereby it prevents to have several copies of that file. Probably you could also move the application.yml and the bootstrap.yml into this module. Otherwise there is nothing special to the Maven setup.

Scripts and Execution

To make it easier to build and run the project I provided some scripts. They are located in the scripts folder and are very easy. So I think I don't need to explain them.

The only thing what is important to know is that it can take a little while until the microservices are registered correctly and all microservices received the instances of the other microservices. In the case of an error just try to reload the page a little bit later.

Conclusion

I think it's a quite sophisticated setup and that it is very well suited to be the basis of production microservices. One amazing thing is that it's possible to pack all the microservices together - as long as the dependencies are compatible. This makes supposedly the development easier. Instead of the need to start many microservices you just need to start one application. In addition it can be a quick fix for performance issues. If there are two very talkative microservices just glue them together until you fixed the performance issue.

I hope you like my setup and my blog entry. I probably will write another blog post to explain some details of the setup. This article is already so long that I didn't want to include more details.

I am always happy about comments and suggestions for improvements! :-)

Tutorial: How to push / upload artifacts into the Maven Central Repository

I have several GitHub repositories and that's great. It's so easy to share the projects. But for some projects it's not enough. I created a POM, which I'm using as parent POM for all my projects. So if you want to checkout a project which depends on the POM then it's necessary to checkout the pom-project first and do a $ mvn install. That's quite annoying. The second example is my spBee library. I think most users aren't interested to checkout that project and to contribute to it - they just want to use it. Therefore it's necessary in such cases to put the artifacts on the Maven Central Repository. Then the user can add the library to the Maven dependencies and everything works. So I faced the question: how to push my artifacts to the Maven Central Repository?

A short overview over all steps

1. install PGP - that's necessary to sign the artifacts - and upload your public key 
2. prepare the POM file to satisfy all requirements
3. create an Sonatype account
4. create a New Project ticket
5. wait for a comment.
6. perform a stating release and comment the ticket that you have successfully done the stating release and the artifacts are ready to be released.
7. wait until the sync to the Maven Central Repository will be activated and now you can do the releases yourself.
1. do another staging release
2. release your artifacts on this site https://oss.sonatype.org/
3. or drop your staging release
8. enjoy your artifacts on the Maven Central Repository

It seems to be quite complicated. But actually it's not that bad. Hopefully it's very easy with the help of this post!

1. Install PGP and upload your public key
 On this site everything is explained. In short:

• download and install GPG
• $ gpg --version
• $ gpg --gen-key
• enter the required information
• $ gpg --list-keys
• in the output you will see the keyid of the public certificate
• pub   1024D/C6EED57A 2010-01-13
• the C6EED57A string is the keyid
• distribute the public key so that the signed files can be verified
• $ gpg --keyserver hkp://pool.sks-keyservers.net --send-keys C6EED57A

2. Prepare the POM file to satisfy all requirements
On this site and that site everything is explained. In short:

• the groupId
• if the groupId is at.rseiler.spbee then you need to own rseiler.at.
• if you don't have an own domain, but use GitHub then the groupId must be: github.com/rseiler => com.github.rseiler
• the javadoc.jar must be generated
• the sources.jar must be generated
• all files must be signed with PGP
• following meta data must be provided
• project name, description and URL
• license information
• developer information 
• scm (the repository URL)
• the nexus-staging-maven-plugin must be setup
• the distributionManagement (snapshotRepository and
  repository) must be setup

Take a look at these both small POM files, which satisfies all requirements: spBee POM pom-project POM
The  release profile will create the javadoc.jar sources.jar and signs the artifacts. So if you do a release you need to activate the profile with: $ mvn clean deploy -P release

To upload the artifacts you need to setup the settings.xml (.m2/settings.xml).

<settings>
  <servers>
    <server>
      <id>ossrh</id>
      <username>your-jira-id</username>
      <password>your-jira-pwd</password>
    </server>
  </servers>
</settings>

3. Create an Sonatype account
Go to this site and create an account.

4. Create a New Project ticket

• go to this site and create a ticket - see mine as an example
• enter the root groupId
• at.rseiler - even if your first artifact uses at.rseiler as groupId
• com.github.rseiler - if you don't have an own domain
• fill out the rest of the fields

6. Preform a stating release

• check if everything is setup correctly
• I recommend to set <autoReleaseAfterClose>true</autoReleaseAfterClose> to false so you can check the output first
• notice that there aren't allowed any JavaDoc errors. If there are errors than the javadoc.jar file won't be created and then the requirements are missed.
• $ mvn clean deploy -P release
• check on this site under Build Promation => Repositories => Content your uploaded artifacts
• if everything is fine then use: $ mvn nexus-staging:release to do a stating release
• otherwise $ mvn nexus-staging:drop to drop the stating release
• both commands can be executed on the website, too
• comment the ticket

7. Release yourself
After the sync is activated you can release your stating-releases yourself to the Maven Central Repository.

8. Enjoy your artifacts on the Maven Central Repository 
You have done it! Congratulation! :)


I hope that my blog post helped you and gave you a good overview over all required steps.

Publish Maven Site Documentation automatically to the GitHub Pages

Maven site is a great tool as well as GitHub. The coolest thing about those tools is that they play together very well. Only a little Maven configuration is needed. But let me explain shortly what both tools do.

Maven Site: generates automatically a nice documentation website for your Maven project. Many Apache projects uses this generated site as their main website. So probably you have already seen such generated websites. An nice example is the my (parent) POM project or the maven-site-plugin. Mainly the documentation consists of three parts:

• Project information like dependencies, license, source repository and so on. 
• Project reports like unit tests reports, static code analysis reports, JavaDocs and so on. 
• The documentation written by the developers. With an additional Maven Plugin it's possible to write the documentation with Markdown.

GitHub Pages: GitHub doesn't only provide a GIT repository but also can be used as webhost for static website. Which is ideal for a project website. The GitHub Pages concept is very cool. It's just a branch, called gh-pages, in you project repository. All the files that are pushed into this branch will be served from GitHub`s webserver. An nice example is the documentation for my (parent) POM project.
Before you start with the Maven configuration you should read Creating Project Pages manually for a better understanding.

Before we can start with the GitHub Pages integration into Maven we firstly need to correctly build the website. For multi module projects it's necessary to deploy (locally is sufficient) the website first. Otherwise the references between modules won't work. To do so you just need to configure the distribution management like this:

<distributionManagement>
<site>
<id>site-docs</id>
<url>file://${env.HOME}/sitedocs/pom-project</url>
</site>
</distributionManagement>

How to fully configure the generation of the Maven Site is too much for this blog post. Look at the POM for some good basic configuration or just us it as parent POM for your project. The minimum of configuration is:

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-site-plugin</artifactId>
            <version>3.4</version>
            <executions>
                <!-- used for multiproject builds -->
                <execution>
                    <id>attach-descriptor</id>
                    <goals>
                        <goal>attach-descriptor</goal>
                    </goals>
                </execution>
            </executions>
            <dependencies>
                <!-- To use the Markdown format -->
                <dependency>
                    <groupId>org.apache.maven.doxia</groupId>
                    <artifactId>doxia-module-markdown</artifactId>
                    <version>1.6</version>
                </dependency>
            </dependencies>
        </plugin>
    </plugins>
</build>

<reporting>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-project-info-reports-plugin</artifactId>
            <version>2.7</version>
            <configuration>
                <dependencyLocationsEnabled>false</dependencyLocationsEnabled>
            </configuration>
            <reportSets>
                <reportSet>
                    <reports>
                        <report>index</report>
                        <!--<report>cim</report>-->
                        <report>dependencies</report>
                        <!--<report>dependency-convergence</report>-->
                        <report>dependency-info</report>
                        <report>dependency-management</report>
                        <!--<report>distribution-management</report>-->
                        <!--<report>issue-tracking</report>-->
                        <report>license</report>
                        <!--<report>mailing-list</report>-->
                        <report>modules</report>
                        <report>plugin-management</report>
                        <report>project-team</report>
                        <report>scm</report>
                        <report>summary</report>
                    </reports>
                </reportSet>
            </reportSets>
        </plugin>
    </plugins>
</reporting>

To build the Maven Site just use: mvn clean site site:deploy

The integration into the GitHub pages is very easy and looks like this:

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-scm-publish-plugin</artifactId>
            <version>1.1</version>
            <inherited>true</inherited>
            <configuration>
                <checkoutDirectory>${project.basedir}/github.com</checkoutDirectory>
                <checkinComment>publishing site documentation</checkinComment>
                <content>${env.HOME}/sitedocs/pom-project</content>
                <pubScmUrl>scm:git:https://github.com/rseiler/pom-project.git</pubScmUrl>
                <scmBranch>gh-pages</scmBranch>
            </configuration>
        </plugin>
    </plugins>
</build>

Additional you need to configure your username and password in the .m2/settings.xml. It should look like this:

<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
                      http://maven.apache.org/xsd/settings-1.0.0.xsd">
  <servers>
    <server>
      <id>github.com</id>
      <username>github-username</username>
      <password>github-password</password>
    </server>
  </servers>
</settings>

If everything is done just type: 

mvn clean site site:deploy scm-publish:publish-scm -Dscmpublish.dryRun=true

The scmpublish.dryRun flag prevents the plugin to commit anything and just outputs what changes it would commit. Check it and if everything works than rerun the command without this flag: 

mvn clean site site:deploy scm-publish:publish-scm

That's it! It's Very simple :)