Skip to main content

Microservices - Config management using Spring cloud bus and Rabbit MQ

In this tutorial we will learn how to add Rabbit MQ capabilities to our Spring cloud config server so any changes to configurations can be pushed to all connected applications during runtime. We need such kind of behaviour when we need to refresh the properties without restarting our application. Below is the overall architecture for this complete setup.
spring cloud bus

Spring Cloud Config Server

Spring cloud config server is used to setup the distributed configuration using GIT or local file system where we can keep our configuration files and serve as them from Spring cloud config server. Client application just has to connect with config server by providing their application and profile name for specific configuration. Please refer below link where I have explained more about cloud config and how to code it.
https://www.thetechnojournals.com/2019/10/spring-cloud-config.html

Install Rabbit MQ

Please refer below link on how to install rabbit-mq and virtual host verification.
https://www.thetechnojournals.com/2019/11/installing-rabbit-mq.html
Note: If there is any problem running the virtual host then your Spring config server may not be able to connect to rabbitmq and you will see similar error in Spring boot console.
c.r.c.impl.ForgivingExceptionHandler     : An unexpected connection driver error occured (Exception message: Socket closed)

Spring Cloud Bus

When we have changes to our configuration in GIT source repository then it may notify the config server for the changes and these changes can be pushed to all connected clients so that they can refresh the properties with latest available changes. It can be achieved through the webhook which is supported by many providers like github to publish the event in which users are interested. Then config server push the event to all clients through rabbitmq and all connected clients refresh the changes at runtime. Below changes are required to enable the spring cloud bus in spring cloud config server application which we have seen in first section "Spring Cloud Config Server".

Maven dependencies

We need to add below dependencies where monitor starter stream dependency is added to communicate with rabbit mq and config monitor is added to receive notifications from git repository.
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-stream-rabbit</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-config-monitor</artifactId>
        </dependency>

Configuration changes

Add below properties in your application.properties or application.yml to enable the Spring cloud bus with RabbitMQ.
spring:
  cloud:
    bus:
      enabled: true
  rabbitmq:
    host: localhost
    port: 5672
    virtual-host: /
    username: guest
    password: guest

Webhook configuration using Github

A webhook need to be configured using Github so it can notify our Spring cloud config server for any changes committed to the configuration repository. We configure the webhook in Github by registering our /monitor endpoint of config server application and due to this we should have public IP instead of localhost. See in next section how to refresh for localhost manually.
Refer the below screenshot for webhook configuration.
git webhook

Manual refresh using /monitor endpoint

 In case we don't have a public IP for our host where cloud config server application is running, we can auto refresh all the connected applications by hitting only one URL manually on cloud config server application.

Monitoring URL: http://localhost:8888/monitor
Please note that to access above URL you have to make a POST request with below headers and request body.
Headers:
 Content-Type: application/json
 X-Event-Key: repo:push
 X-Hook-UUID: webhook-uuid
Request body:
 {"push": {"changes": []}}
While executing above request, you may face similar error as given below due to the default enablement of csrf in spring.
{
    "timestamp": "2019-11-23T16:37:08.098+0000",
    "status": 403,
    "error": "Forbidden",
    "message": "Forbidden",
    "path": "/monitor"
}
To fix this issue we need to disable the csrf by creating below class.
@Configuration
@EnableWebSecurity
public class SecurityConfiguration extends WebSecurityConfigurerAdapter {
    @Override
    protected void configure(HttpSecurity http) throws Exception {
        http.csrf().disable();
    }
}

Config Test Service (Config client)

Now we will create a client application which will access the properties using cloud config server and refresh them without restarting it.

Maven Dependencies

    <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-starter-bus-amqp</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-starter-config</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-actuator</artifactId>
    </dependency>

bootstrap.yml

Add below properties to bootstrap properties or yml file.
spring:
  profiles:
    active: DEV
  cloud:
    config:
      name: microservices-config
      uri: http://localhost:8888/
      username: config_user
      password: config_user
  rabbitmq:
    host: localhost
    port: 5672
    virtual-host: /
    username: guest
    password: guest

PropertiesController.java

This class uses property key "message" which we will test with refresh event. We have used RefreshScope annotation on this class so it can be refreshed with all the message properties in this class.
@RefreshScope
@RestController
public class PropertiesController {
    @Value("${message}")
    private String message;

    @RequestMapping(value = "/test", method = RequestMethod.GET)
    public String testMessage(){
        return message;
    }
}

Testing the application

Below are the steps to test the application.

Other posts you may like

Microservice development using Spring boot
Spring cloud API gateway tutorial
Registry/ discovery server using Spring boot and Eureka

Labels:

Comments

Post a Comment

Popular Posts

Setting up kerberos in Mac OS X

Kerberos in MAC OS X Kerberos authentication allows the computers in same domain network to authenticate certain services with prompting the user for credentials. MAC OS X comes with Heimdal Kerberos which is an alternate implementation of the kerberos and uses LDAP as identity management database. Here we are going to learn how to setup a kerberos on MAC OS X which we will configure latter in our application. Installing Kerberos In MAC we can use Homebrew for installing any software package. Homebrew makes it very easy to install the kerberos by just executing a simple command as given below. brew install krb5 Once installation is complete, we need to set the below export commands in user's profile which will make the kerberos utility commands and compiler available to execute from anywhere. Open user's bash profile: vi ~/.bash_profile Add below lines: export PATH=/usr/local/opt/krb5/bin:$PATH export PATH=/usr/local/opt/krb5/sbin:$PATH export LDFLAGS=&
89 comments
Read more

Why HashMap key should be immutable in java

HashMap is used to store the data in key, value pair where key is unique and value can be store or retrieve using the key. Any class can be a candidate for the map key if it follows below rules. 1. Overrides hashcode() and equals() method.   Map stores the data using hashcode() and equals() method from key. To store a value against a given key, map first calls key's hashcode() and then uses it to calculate the index position in backed array by applying some hashing function. For each index position it has a bucket which is a LinkedList and changed to Node from java 8. Then it will iterate through all the element and will check the equality with key by calling it's equals() method if a match is found, it will update the value with the new value otherwise it will add the new entry with given key and value. In the same way it check for the existing key when get() is called. If it finds a match for given key in the bucket with given hashcode(), it will return the value other
1 comment
Read more

Entity to DTO conversion in Java using Jackson

It's very common to have the DTO class for a given entity in any application. When persisting data, we use entity objects and when we need to provide the data to end user/application we use DTO class. Due to this we may need to have similar properties on DTO class as we have in our Entity class and to share the data we populate DTO objects using entity objects. To do this we may need to call getter on entity and then setter on DTO for the same data which increases number of code line. Also if number of DTOs are high then we need to write lot of code to just get and set the values or vice-versa. To overcome this problem we are going to use Jackson API and will see how to do it with minimal code only. Maven dependency <dependency> <groupId>com.fasterxml.jackson.core</groupId> <artifactId>jackson-databind</artifactId> <version>2.9.9</version> </dependency> Entity class Below is
Post a Comment
Read more

Multiple data source with Spring boot, batch and cloud task

Here we will see how we can configure different datasource for application and batch. By default, Spring batch stores the job details and execution details in database. If separate data source is not configured for spring batch then it will use the available data source in your application if configured and create batch related tables there. Which may be the unwanted burden on application database and we would like to configure separate database for spring batch. To overcome this situation we will configure the different datasource for spring batch using in-memory database, since we don't want to store batch job details permanently. Other thing is the configuration of  spring cloud task in case of multiple datasource and it must point to the same data source which is pointed by spring batch. In below sections, we will se how to configure application, batch and cloud task related data sources. Application Data Source Define the data source in application properties or yml con
52 comments
Read more