ju
/
openinfraday
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Autocommit action=MODIFY on file=Follow the RabbitMQ - Plan.md detected

This commit is contained in:
Flatnotes 2024-05-05 20:32:11 +00:00
parent ab0f6574fd
commit 4c3f3195cd
1 changed files with 5 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
Plan.md → Follow the RabbitMQ - Plan.md
View File

@ -1,4 +1,3 @@
> What kind of issues we faced with rabbit > What kind of issues we faced with rabbit
> Is it a RabbitMQ setup issue or an Openstack issue ? > Is it a RabbitMQ setup issue or an Openstack issue ?
@ -9,7 +8,6 @@
* pause-minority helped crash the cluster * pause-minority helped crash the cluster
* reset cluster was ... the solution * reset cluster was ... the solution
> Which methods did we use to troubleshoot those issues > Which methods did we use to troubleshoot those issues
> Observability, tools > Observability, tools
@ -28,14 +26,13 @@
* fanout mechanism 1 message published, duplicated to N queues * fanout mechanism 1 message published, duplicated to N queues
* Nova rpc usage is clearly != neutron * Nova rpc usage is clearly != neutron
> Before going further, let's take some time to understand how oslo.messaging work > Before going further, let's take some time to understand how oslo.messaging work
> How RPC is implemented in Openstack > How RPC is implemented in Openstack
* Under the hood ? * Under the hood ?
* pub/sub mechanism * pub/sub mechanism
* subscriber: RPC server topic=name * subscriber: RPC server topic=name
* setup class endpoints * setup class endpoints
* create queues / setup consumer thread * create queues / setup consumer thread
* publish with rpc provided methods * publish with rpc provided methods
* call - reply (topic / transient for reply) * call - reply (topic / transient for reply)
@ -43,10 +40,9 @@
* cast / fanout=true (fanout queue) * cast / fanout=true (fanout queue)
* notifications for external use: kafka * notifications for external use: kafka
> What we did to put rabbits back to their holes > What we did to put rabbits back to their holes
* Journey to get a stable infra. * Journey to get a stable infra.
* Infra * Infra
* split rabbit-neutron / rabbit-\* * split rabbit-neutron / rabbit-\*
* scale problematic clusters to 5 node * scale problematic clusters to 5 node
@ -58,8 +54,8 @@
* heartbeat in pthread fix * heartbeat in pthread fix
* move from HA queue > Quorum queues * move from HA queue > Quorum queues
* fix to autodelete broken quorum queues * fix to autodelete broken quorum queues
* replace 'fanout' queues by stream queues * replace 'fanout' queues by stream queues
* reduce queue nb a lot * reduce queue nb a lot
* patch to avoid tcp reconnection when a queue is deleted (kombu/oslo) * patch to avoid tcp reconnection when a queue is deleted (kombu/oslo)
* reduce queues declared by a RPC server (3 queues by default to only 1) * reduce queues declared by a RPC server (3 queues by default to only 1)
* use same connection for mutiple topics * use same connection for mutiple topics