Java HotSpot Virtual Machine

Hi All,
We have 24 managed instance server in a cluster, each one having 4GB/4GB (min/max) memory settings.
We have setup the ClusterAddress and Listen Address settings properly.

Still when we run a certrain job, the Linux server itself goes unresponsive and no one can even log in to that server.

We constantly get this error message in the logs:
<Free memory in the server is x,xxx,xxx bytes. There is danger of OutOfMemoryError>

What could be the reason and where to start to identify the root cause?

Thanks,