IPC on the Linux kernel
Not that a relevant question as far as the usual questions go in this forum - but perhaps there are one or two kernel type hackers that can comment on the following.
What is the fastest and most scalable IPC method on the Linux kernel? (realise it is easier asked then answered)
Currently I'm using message queues. The basic processing model is as follows:
The main process, open n number of message queues. It reads 180+ network packets/sec and then in a round robin fasion, send a packet down a message queue for processing.
The queue processes. One or more process per message queue. Reads a message from the queue, and processes the message buffer (which is basically a network packet payload).
Obviously, the latency/performance of processing the buffer, determines how quick the queue process can read the next message from the queue. That aside though, is IPC message queues suited as a low latency transport mechanism for this type of parallel processing? Are there specific kernel options to look at? (e.g. on a 2.4 kernel, setting the message queue buffer via msgctl() causes subsequent msgsnd() failures)
Is using shmem a better option in this regard - despite the increase in complexity in having to use semaphores to coordinate access by multiple worker processes for finding "dirty" buffers in shared mem to process?
Googling, I've found a fair bit of man pages and sample code on these topics - but not real suggestions and recommendations as to what type of method suits what type of parallel processing. Or providing basic performance comparisons between methods.