Adventures using Rhino ServiceBus

Recently, with the team I work, we tried Rhino ServiceBus on a project. The fact that it has it's own queuing subsystem Rhino Queues (which requires no installation or administration) and it is integrated with the IoC container (we had already chosen for that project), Castle Windsor, made it a good candidate for spending some weeks trying it.

While we were doing some sort of stress testing (about 6000 requests), we noticed that the memory was increasing (like it does in a memory leak situation). 

Since the behavior was unexpected, we spent some time on ways to workaround it, incl. reading the source code, building with the latest and greatest. My colleague, G. Nikolaropoulos, started also a thread on the Rhino Tools Dev discussion group. We got feedback very fast: the memory is held inside Rhino Queues, to avoid replying with the same message more than once in case of network loss. Every 3 minutes, Rhino Queues cleans the list up and memory is freed.

We built a very basic sample application but we also tested the article code that was published on MSDN (there is also a second part). The behavior was the same even after 3 minutes wait (not to say worse).

We thought it should be fair to investigate even further and ask in person via E-mail. We got a reply fast, instructing that we should set the Extensible Storage Engine (ESE) CacheSizeMax parameter to 1024. Rhino Queues use ESE internally for message storage.

In the sample application which can be found on here, I set the CacheSizeMax to 512.

But what exactly is the CacheSizeMax parameter?

Well, Rhino Queues uses  the ManagedEsent library which provides managed access to ESE. I opened the library with Reflector and searched for the CacheSizeMax which is declared  inside the JET_param enum.

Microsoft.Isam.Esent.Interop.JET_param, CacheSizeMax This parameter configures the maximum size of the database page cache. The size is in database pages. If this parameter is left to its default value, then the maximum size of the cache will be set to the size of physical memory when JetInit is called. 

Setting the Microsoft.Isam.Esent.Interop.SystemParameters.CacheSizeMax to 1024 or 512 seems to solve the problem with the increasing memory usage. The question is, if this call sets a global variable then it may have a negative impact to the rest applications using the ESE (ex. Microsoft Exchange).

I decided to contact Laurion Burchall, a Software Engineer on the ESE/ESENTteam, with my question. The answer: The setting is per-process so it won’t cause problems with other applications that use ESENT. By default ESENT will automatically manage the cache size so you shouldn’t need to worry. We look at the amount of free memory on the system, the amount of paging activity and the amount of database I/O when deciding on the cache size. Although the cache is growing you should find that the system isn’t paging and that if you start using memory in other applications ESENT will actually shrink its cache.

It is very easy to use and deploy with Rhino ServiceBus. If you are planning to use it, I hope you find this information useful.