troubleshooting
How to prevent out-of-memory issues on a compute node with limited resources?
The team is facing freezing issues on a compute node with 60 GB RAM and a 1000 GB disk while ingesting a large table via Kafka. Materialized views cannot be declared during processing, and queries creating views are timing out. Rick Otten suggested setting a memory limit on the compute node to prevent OOM errors. Victor Müller mentioned that the issue is causing the node to become unresponsive, not necessarily running out of memory. Eric recommended checking the Troubleshooting and Troubleshoot out-of-memory issues documentation for guidance.
Vi
Victor Müller
Asked on Feb 16, 2024
- Set a memory limit on the compute node to prevent out-of-memory errors.
- Consider setting the memory limit to about 75% of the total memory available.
- Example:
--total-memory-bytes: 8589934592for 8GB memory limit. - Monitor the memory usage to ensure the node does not exceed the set limit.
- Check the Troubleshooting and Troubleshoot out-of-memory issues documentation for additional guidance.
Feb 19, 2024Edited by