{"id":265,"date":"2018-04-25T12:16:43","date_gmt":"2018-04-25T11:16:43","guid":{"rendered":"https:\/\/www.cogin.com\/blog\/?p=265"},"modified":"2018-04-26T10:42:17","modified_gmt":"2018-04-26T09:42:17","slug":"distributed-brokers-rabbitmq-for-msmq-users-part-5","status":"publish","type":"post","link":"https:\/\/www.cogin.com\/blog\/rabbitmq\/distributed-brokers-rabbitmq-for-msmq-users-part-5\/","title":{"rendered":"Distributed brokers – RabbitMQ for MSMQ users, part 5"},"content":{"rendered":"

When working with MSMQ, we usually don’t think about “distributed brokers” as a separate topic. It’s core MSMQ functionality. From user’s point of view, you only change destination address (i.e. format or path), and message magically goes to remote MSMQ server. Behind the scene, it’s supported by outgoing queues where messages are temporarily stored until they’re delivered, they’re retried if destination or network is down, etc. In fact, MSMQ doesn’t even offer us to talk directly with some remote server – all communication always goes through local service.<\/p>\n

With RabbitMQ it’s not as simple. It’s quite usual that you’ll set up just one\u00a0broker and use it remotely from multiple hosts. However, it does have some kind of support for connecting multiple brokers – Federation and Showel. They are both implemented as RabbitMQ plugins, included in installation but not enabled by default. We’ll also talk about clustering.<\/p>\n

Federation<\/h2>\n

Federation is a way to connect queues and exchanges, which can reside on different brokers. It\u2019s a one-way type of connection – we define an “upstream”, and that upstream defines where messages come from. Federation is set up on “destination” broker, nothing is configured on\u00a0the source.<\/p>\n

Federated exchanges<\/h3>\n

If two exchanges are federated, copies of messages are forwarded from source to destination. It\u2019s very similar to binding, except that destination exchange can be on a\u00a0different broker. Behind the scene, this is supported by “upstream queue” on source broker – it’s similar concept to MSMQ outgoing queue, since it keeps messages which are not delivered yet. You’ll recognize it by “federation:” prefix, and its name tells us for which destination it is.<\/p>\n

\"\"<\/p>\n

We can use QueueExplorer<\/a> to see or modify messages which are waiting\u00a0for delivery. RabbitMQ automatically creates “federated” exchange, also on source broker:<\/p>\n

\"\"<\/p>\n

Federated exchanges are fairly complex to set up. First, you have to define upstream, and then you have to create a policy which will apply federation. You can see more details about federated exchanges here:<\/p>\n

https:\/\/www.rabbitmq.com\/federation.html<\/a>
\nhttps:\/\/www.rabbitmq.com\/federated-exchanges.html<\/a><\/p>\n

Federated queues<\/h3>\n

Federated queues are a bit different – destination queue acts as a consumer for source queue. When a\u00a0message is consumed on a\u00a0destination machine, it gets removed from source queue and no other consumer will see it. Therefore each message is processed only once, either by a consumer on a\u00a0source machine, or by some of federated destination queues (there can be more that one), whichever receives it first. That is different from Federated exchanges, where messages are duplicated and all receivers receive all messages.<\/p>\n

Effectively, all federated queues act as a single logical queue, with multiple receivers on multiple machines. So federated queues can be used for load balancing.<\/p>\n

http:\/\/www.rabbitmq.com\/federated-queues.html<\/a><\/p>\n

Shovel<\/h2>\n

Shovel is a plugin which moves messages from one place to another. It’s like a client which we could write (but don’t have to), which receives messages from a queue and sends them to an exchange,\u00a0including handling failures in that process. Queue and exchange can be on same or different virtual host, or on another broker. It can be set up on independent node, and configured however we like. Problem with shovel was that we had to restart a broker where it runs in order to reconfigure it, but dynamic shovels removed that limitation.<\/p>\n

https:\/\/www.rabbitmq.com\/shovel.html<\/a><\/p>\n

Clustering<\/h2>\n

RabbitMQ clustering is different from MSMQ. They both consist of multiple nodes which show up as single logical broker. Let\u2019s recall first how clustering works on MSMQ. Only one node is kept active, other will take over only if the\u00a0first one fails, aka active\/passive configuration. There are no performance gains from additional machine(s), but there are also no problems with consistency between nodes, if one fails. Queues and messages are kept on shared storage.<\/p>\n

On RabbitMQ cluster is geared toward increasing scalability, not high availability. All nodes in a cluster work all the time, and\u00a0load is distributed, which improves performance. Definitions (e.g. queue\/exchange names\/parameters, users, policies, etc) are distributed on all nodes. However, by default, queue messages reside\u00a0on just one of the nodes. Such queue is not fault tolerant. If node fails, queues from that node won\u2019t be available anymore.<\/p>\n

In order to get high availability, RabbitMQ has a\u00a0feature called queue mirroring, which can be turned on after cluster is created. With mirroring turned on, messages will be synced to other nodes. However, they\u2019ll still be consumed only from their primary location (i.e. master queue) and other nodes will just maintain their own copy of queue. When node fails, one of mirrored nodes takes over.<\/p>\n

Problem is if there are communication issues between nodes. They may end up thinking that other node(s) are down, and continue separately. If messages are not produced\/consumed in the exactly same way on disconnected nodes, they\u2019ll end up with conflicting data. That state is known as a\u00a0network partition<\/strong>. In these cases RabbitMQ cluster requires manual intervention after failures, although there are few auto modes where cluster would try to fix problems itself.<\/p>\n

https:\/\/www.rabbitmq.com\/clustering.html<\/a><\/p>\n

In the last part<\/a> of this series we’ll talk about RabbitMQ programming basics.<\/p>\n

Links to all 6 parts<\/a> of this series.<\/p>\n","protected":false},"excerpt":{"rendered":"

When working with MSMQ, we usually don’t think about “distributed brokers” as a separate topic. It’s core MSMQ functionality. From user’s point of view, you only change destination address (i.e. format or path), and message magically goes to remote MSMQ server. Behind the scene, it’s supported by outgoing queues where messages are temporarily stored until […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2,12],"tags":[5,13],"class_list":["post-265","post","type-post","status-publish","format-standard","hentry","category-msmq","category-rabbitmq","tag-msmq","tag-rabbitmq"],"yoast_head":"\nDistributed brokers - RabbitMQ for MSMQ users, part 5<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.cogin.com\/blog\/rabbitmq\/distributed-brokers-rabbitmq-for-msmq-users-part-5\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Distributed brokers - RabbitMQ for MSMQ users, part 5\" \/>\n<meta property=\"og:description\" content=\"When working with MSMQ, we usually don’t think about “distributed brokers” as a separate topic. It’s core MSMQ functionality. From user’s point of view, you only change destination address (i.e. format or path), and message magically goes to remote MSMQ server. Behind the scene, it’s supported by outgoing queues where messages are temporarily stored until […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.cogin.com\/blog\/rabbitmq\/distributed-brokers-rabbitmq-for-msmq-users-part-5\/\" \/>\n<meta property=\"og:site_name\" content=\"Cogin blog\" \/>\n<meta property=\"article:published_time\" content=\"2018-04-25T11:16:43+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2018-04-26T09:42:17+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.cogin.com\/blog\/wp-content\/uploads\/2018\/04\/Federated-queue.png\" \/>\n<meta name=\"author\" content=\"Dejan Grujic\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Dejan Grujic\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.cogin.com\\\/blog\\\/rabbitmq\\\/distributed-brokers-rabbitmq-for-msmq-users-part-5\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.cogin.com\\\/blog\\\/rabbitmq\\\/distributed-brokers-rabbitmq-for-msmq-users-part-5\\\/\"},\"author\":{\"name\":\"Dejan Grujic\",\"@id\":\"https:\\\/\\\/www.cogin.com\\\/blog\\\/#\\\/schema\\\/person\\\/e0c4fd92043868daa052d5978d69ed22\"},\"headline\":\"Distributed brokers – RabbitMQ for MSMQ users, part 5\",\"datePublished\":\"2018-04-25T11:16:43+00:00\",\"dateModified\":\"2018-04-26T09:42:17+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.cogin.com\\\/blog\\\/rabbitmq\\\/distributed-brokers-rabbitmq-for-msmq-users-part-5\\\/\"},\"wordCount\":910,\"image\":{\"@id\":\"https:\\\/\\\/www.cogin.com\\\/blog\\\/rabbitmq\\\/distributed-brokers-rabbitmq-for-msmq-users-part-5\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.cogin.com\\\/blog\\\/wp-content\\\/uploads\\\/2018\\\/04\\\/Federated-queue.png\",\"keywords\":[\"MSMQ\",\"RabbitMQ\"],\"articleSection\":[\"MSMQ\",\"RabbitMQ\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.cogin.com\\\/blog\\\/rabbitmq\\\/distributed-brokers-rabbitmq-for-msmq-users-part-5\\\/\",\"url\":\"https:\\\/\\\/www.cogin.com\\\/blog\\\/rabbitmq\\\/distributed-brokers-rabbitmq-for-msmq-users-part-5\\\/\",\"name\":\"Distributed brokers - RabbitMQ for MSMQ users, part 5\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.cogin.com\\\/blog\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.cogin.com\\\/blog\\\/rabbitmq\\\/distributed-brokers-rabbitmq-for-msmq-users-part-5\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.cogin.com\\\/blog\\\/rabbitmq\\\/distributed-brokers-rabbitmq-for-msmq-users-part-5\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.cogin.com\\\/blog\\\/wp-content\\\/uploads\\\/2018\\\/04\\\/Federated-queue.png\",\"datePublished\":\"2018-04-25T11:16:43+00:00\",\"dateModified\":\"2018-04-26T09:42:17+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/www.cogin.com\\\/blog\\\/#\\\/schema\\\/person\\\/e0c4fd92043868daa052d5978d69ed22\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.cogin.com\\\/blog\\\/rabbitmq\\\/distributed-brokers-rabbitmq-for-msmq-users-part-5\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.cogin.com\\\/blog\\\/rabbitmq\\\/distributed-brokers-rabbitmq-for-msmq-users-part-5\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.cogin.com\\\/blog\\\/rabbitmq\\\/distributed-brokers-rabbitmq-for-msmq-users-part-5\\\/#primaryimage\",\"url\":\"https:\\\/\\\/www.cogin.com\\\/blog\\\/wp-content\\\/uploads\\\/2018\\\/04\\\/Federated-queue.png\",\"contentUrl\":\"https:\\\/\\\/www.cogin.com\\\/blog\\\/wp-content\\\/uploads\\\/2018\\\/04\\\/Federated-queue.png\",\"width\":494,\"height\":226},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.cogin.com\\\/blog\\\/rabbitmq\\\/distributed-brokers-rabbitmq-for-msmq-users-part-5\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.cogin.com\\\/blog\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Distributed brokers – RabbitMQ for MSMQ users, part 5\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.cogin.com\\\/blog\\\/#website\",\"url\":\"https:\\\/\\\/www.cogin.com\\\/blog\\\/\",\"name\":\"Cogin blog\",\"description\":\"Queuing tools and articles\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.cogin.com\\\/blog\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.cogin.com\\\/blog\\\/#\\\/schema\\\/person\\\/e0c4fd92043868daa052d5978d69ed22\",\"name\":\"Dejan Grujic\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/7edfd9071a535694242755af2736da7972b73f5d026be6563586a9b0ff7829d0?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/7edfd9071a535694242755af2736da7972b73f5d026be6563586a9b0ff7829d0?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/7edfd9071a535694242755af2736da7972b73f5d026be6563586a9b0ff7829d0?s=96&d=mm&r=g\",\"caption\":\"Dejan Grujic\"},\"sameAs\":[\"http:\\\/\\\/www.cogin.com\"],\"url\":\"https:\\\/\\\/www.cogin.com\\\/blog\\\/author\\\/gruja\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Distributed brokers - RabbitMQ for MSMQ users, part 5","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.cogin.com\/blog\/rabbitmq\/distributed-brokers-rabbitmq-for-msmq-users-part-5\/","og_locale":"en_US","og_type":"article","og_title":"Distributed brokers - RabbitMQ for MSMQ users, part 5","og_description":"When working with MSMQ, we usually don’t think about “distributed brokers” as a separate topic. It’s core MSMQ functionality. From user’s point of view, you only change destination address (i.e. format or path), and message magically goes to remote MSMQ server. Behind the scene, it’s supported by outgoing queues where messages are temporarily stored until […]","og_url":"https:\/\/www.cogin.com\/blog\/rabbitmq\/distributed-brokers-rabbitmq-for-msmq-users-part-5\/","og_site_name":"Cogin blog","article_published_time":"2018-04-25T11:16:43+00:00","article_modified_time":"2018-04-26T09:42:17+00:00","og_image":[{"url":"https:\/\/www.cogin.com\/blog\/wp-content\/uploads\/2018\/04\/Federated-queue.png","type":"","width":"","height":""}],"author":"Dejan Grujic","twitter_misc":{"Written by":"Dejan Grujic","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.cogin.com\/blog\/rabbitmq\/distributed-brokers-rabbitmq-for-msmq-users-part-5\/#article","isPartOf":{"@id":"https:\/\/www.cogin.com\/blog\/rabbitmq\/distributed-brokers-rabbitmq-for-msmq-users-part-5\/"},"author":{"name":"Dejan Grujic","@id":"https:\/\/www.cogin.com\/blog\/#\/schema\/person\/e0c4fd92043868daa052d5978d69ed22"},"headline":"Distributed brokers – RabbitMQ for MSMQ users, part 5","datePublished":"2018-04-25T11:16:43+00:00","dateModified":"2018-04-26T09:42:17+00:00","mainEntityOfPage":{"@id":"https:\/\/www.cogin.com\/blog\/rabbitmq\/distributed-brokers-rabbitmq-for-msmq-users-part-5\/"},"wordCount":910,"image":{"@id":"https:\/\/www.cogin.com\/blog\/rabbitmq\/distributed-brokers-rabbitmq-for-msmq-users-part-5\/#primaryimage"},"thumbnailUrl":"https:\/\/www.cogin.com\/blog\/wp-content\/uploads\/2018\/04\/Federated-queue.png","keywords":["MSMQ","RabbitMQ"],"articleSection":["MSMQ","RabbitMQ"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.cogin.com\/blog\/rabbitmq\/distributed-brokers-rabbitmq-for-msmq-users-part-5\/","url":"https:\/\/www.cogin.com\/blog\/rabbitmq\/distributed-brokers-rabbitmq-for-msmq-users-part-5\/","name":"Distributed brokers - RabbitMQ for MSMQ users, part 5","isPartOf":{"@id":"https:\/\/www.cogin.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.cogin.com\/blog\/rabbitmq\/distributed-brokers-rabbitmq-for-msmq-users-part-5\/#primaryimage"},"image":{"@id":"https:\/\/www.cogin.com\/blog\/rabbitmq\/distributed-brokers-rabbitmq-for-msmq-users-part-5\/#primaryimage"},"thumbnailUrl":"https:\/\/www.cogin.com\/blog\/wp-content\/uploads\/2018\/04\/Federated-queue.png","datePublished":"2018-04-25T11:16:43+00:00","dateModified":"2018-04-26T09:42:17+00:00","author":{"@id":"https:\/\/www.cogin.com\/blog\/#\/schema\/person\/e0c4fd92043868daa052d5978d69ed22"},"breadcrumb":{"@id":"https:\/\/www.cogin.com\/blog\/rabbitmq\/distributed-brokers-rabbitmq-for-msmq-users-part-5\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.cogin.com\/blog\/rabbitmq\/distributed-brokers-rabbitmq-for-msmq-users-part-5\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.cogin.com\/blog\/rabbitmq\/distributed-brokers-rabbitmq-for-msmq-users-part-5\/#primaryimage","url":"https:\/\/www.cogin.com\/blog\/wp-content\/uploads\/2018\/04\/Federated-queue.png","contentUrl":"https:\/\/www.cogin.com\/blog\/wp-content\/uploads\/2018\/04\/Federated-queue.png","width":494,"height":226},{"@type":"BreadcrumbList","@id":"https:\/\/www.cogin.com\/blog\/rabbitmq\/distributed-brokers-rabbitmq-for-msmq-users-part-5\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.cogin.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Distributed brokers – RabbitMQ for MSMQ users, part 5"}]},{"@type":"WebSite","@id":"https:\/\/www.cogin.com\/blog\/#website","url":"https:\/\/www.cogin.com\/blog\/","name":"Cogin blog","description":"Queuing tools and articles","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.cogin.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.cogin.com\/blog\/#\/schema\/person\/e0c4fd92043868daa052d5978d69ed22","name":"Dejan Grujic","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/7edfd9071a535694242755af2736da7972b73f5d026be6563586a9b0ff7829d0?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/7edfd9071a535694242755af2736da7972b73f5d026be6563586a9b0ff7829d0?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/7edfd9071a535694242755af2736da7972b73f5d026be6563586a9b0ff7829d0?s=96&d=mm&r=g","caption":"Dejan Grujic"},"sameAs":["http:\/\/www.cogin.com"],"url":"https:\/\/www.cogin.com\/blog\/author\/gruja\/"}]}},"_links":{"self":[{"href":"https:\/\/www.cogin.com\/blog\/wp-json\/wp\/v2\/posts\/265","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.cogin.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.cogin.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.cogin.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.cogin.com\/blog\/wp-json\/wp\/v2\/comments?post=265"}],"version-history":[{"count":16,"href":"https:\/\/www.cogin.com\/blog\/wp-json\/wp\/v2\/posts\/265\/revisions"}],"predecessor-version":[{"id":351,"href":"https:\/\/www.cogin.com\/blog\/wp-json\/wp\/v2\/posts\/265\/revisions\/351"}],"wp:attachment":[{"href":"https:\/\/www.cogin.com\/blog\/wp-json\/wp\/v2\/media?parent=265"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.cogin.com\/blog\/wp-json\/wp\/v2\/categories?post=265"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.cogin.com\/blog\/wp-json\/wp\/v2\/tags?post=265"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}