I think about queues all the time.

Here’s something to waste your time and bore your brains out.

There are three nodes, say Nodes A, B and C. Node A and C are application servers. Node B is a messenger for nodes A and C. Although Node B is not needed but for reasons I’m not allowed to discuss and would rather not discuss, it has to be there and is the root cause why we have a problem.

Let’s start with an example series of events. A new transaction begins from Node C which sends data to Node B. Node B stores the data in a queue. Node A fetches x items from the queue every y seconds.

After Node A processes the transaction, it begins a new transaction. Node A sends data to Node B only this time, it is no longer stored in a queue. Node B immediately forwards the data to Node C.

On some occasions, Node C starts a new transaction as an acknowledgment from the data that originated from Node A. This transaction is treated as a transaction similar to the first one only that Node A knows that it’s an acknowledgment.

Originally, Node A and Node C communicated to each other directly. The queue didn’t exist. The queue came in to fix a couple of problems but it introduced new ones. Recently, the script in Node A that polls data from Node B died and now it’s catching up with backlog. The script processed data one by one. This showed how slow the main application in Node A is.

I already have a solution in mind and I enjoy writing boring stuff but I’d like to hear other views.

close Reblog this comment
blog comments powered by Disqus
Locations of visitors to this page
De La Salle Canlubang Top Sites top blogs Best blogs on the Web: all about WWW