Hardware Sizing Guidelines

Different workload characteristics affect how services should be scaled and how resource limits are applied. The main factors that drive resource requirements are message throughput and message size.

When processing large volumes of small messages, scale Connect horizontally to increase throughput through concurrency. When handling large payloads, scale Connect vertically to ensure sufficient memory to process incoming data.

Note that increased integration workloads impact not only the integration service, but also the persistence provider, log indexer, and other GridOS Foundation services.

For any questions on specific use cases or integration scenarios, contact Connect Support.

To meet the different processing needs for Connect, three standard GridOS cluster sizes are defined. Each size specifies CPU and memory resources per node, with separate storage recommendations for Elasticsearch and PostgreSQL.

These guidelines are designed to support the different levels of expected message throughput handled by Connect. As the number of messages processed by Connect increases, additional compute resources might be required to support and maintain stability, reliability, and performance across the integration service.

The following table outlines the recommended hardware sizing for small, medium, and large deployments, based on anticipated hourly message volumes. All compute resource values are specified per node. It is recommended that each cluster include at least four nodes, three running control plane components and all running worker components.

When deploying Elasticsearch and PostgreSQL, it is important to consider their specific resource requirements. For best performance, use storage designed for critical, Input/Output Operations Per Second (IOPS)-intensive, and throughput-intensive workloads that require low latency, such as Amazon EBS Provisioned IOPS SSD volumes.