Tuxedo process IPC queue monitoring
779785Oct 1 2010 — edited Dec 6 2010Hi
This question is aimed mostly at Oracle techies who have experience supporting Tuxedo sites.
We've been developing on Tuxedo for a couple of years now and are about to go live with a Tuxedo based transactional system.
Throughout the development and testing phase it has become pretty apparent to us that we need to be able to keep an eye on IPC request queues for processes running under Tuxedo.
For example, we have 10 instances of server "A" running within Tuxedo. As long as these ten instances can cope with the load, we are getting good end-to-end response times. When we overload the system whereby our requests start being queued up on IPC queues for these processes, quite predictably our end-to-end response times go up. We would like to get an early warning when situations like this occur.
Sometimes there are obvious reasons for this behaviour (i.e. we're sending more requests per seconds than our application can cope with) and at other times, it is a degradation in performance of a downstream process (e.g. database query taking longer and longer to return as time goes by) that is causing our requests to start backlogging. It is this latter scenario that we would like to be alerted to before our clients ring us up to tell us that our service is not performing as well as it should.
However, there seems to be a distinct lack of importance placed on server request queue monitoring within Tuxedo infrastructure and API set. Perhaps we are missing something obvious?
We would like to be able to have a more proactive monitoring and reporting facility in place to help us observe queuing behaviour. It seems that there are two ways to obtain any queuing diagnostics:
1) Use a shell script to call tmadmin/pq from time to time to take snapshots or use OS commands to monitor IPC queues
2) Use Tuxedo MIB and poll the queue OIDs to get queue depth snapshots.
We would like to be able to get a lot more than that. For example:
* to be able to plot a graph of queue depths over a period of time
* get alerted when queue depths cross a certain limit (trap vs poll)
* work out how long an item spent on the queue before it got processed
So, are we placing too much emphasis on something that is not important to other Tuxedo users or is there a suite of tools one can use to obtain this sort of information that we are not aware of?
How important is request queuing to a typical Tuxedo site.
Regards,
Mike