RabbitMQ and ZE Service Usage and Requirement
Solution Architecture
Deployment Options
Infrastructure Requirements
There were various tests performed with several variations of environment and deployments (single queue, multiple queues, cluster, etc.).
The setup has been tested by publishing more than 6 million test events (without JIRA), and we were able to achieve a publish rate of approx. 500 events/second.
During the tests, the max publish rate observed from Jira was around 1/second, even when more than 10 issues/second were getting updated on Jira.
The new webhook solution gives optimum response considering the peak loads of approx. 10k events per day observed historically, and the consumer was able to process the events and update ZE at a rate of approx. 1+ event/second.
It is also evident that with this processing capacity, the setup shall accommodate a 5x growth in event volume (up to 50k per day).
The proposed infrastructure required to deploy the new webhook solution is as follows:
Model | vCPU per Node | Memory per Node | Storage per Node | Remark |
|---|---|---|---|---|
3 Node Cluster of RabbitMQ - Single Queue | 4 core | 16GB | 100GB | Volume >= 10k events per day |
3 Node Cluster of RabbitMQ - Multi Queue (Max 5 Queues) | 8 core | 32GB | 200GB | Volume >= 10k events per day |
Single Node of RabbitMQ - Single Queue | 4 core | 16GB | 100GB | Volume <= 10k events per day |
(ZE Shared) RabbitMQ- Single Node | 4 cores | 16GB | 100GB | Volume <= 10k events p er day |
Prerequisites
Windows and Linux (Ubuntu 20.04 and RedHat-9)
RabbitMQ 3.12.10
Java 17.0.10
Tomcat 10. x (if self-host not allowed)/ Docker/Jar deployment
Source | Destination | Ports | Comments |
|---|---|---|---|
Jira Server/Cloud | Webhook Server | 80/443 | For Jira to push events to the Webhook service |
Webhook Server | RabbitMQ Server | 5672/15672 | Connectivity to all nodes in case of cluster |
RabbitMQ Server (s) | RabbitMQ Server (s) | 5672/15672 | Applicable in case of Cluster for inter-node communication |
Consumer Server | RabbitMQ Server | 5672/15672 | Connectivity to all nodes in case of cluster |
Consumer Server | Jira Server/Cloud | 80/443 | For Consumers to communicate with JIRA |
Consumer Server | ZE Database Server | Port on which DB service is running | For Consumers to update requirements on ZE |
Consumer Server | ZE Application Server | 80/443 | For Consumers to handshake with ZE APIs |