Hardware sizing guide

Acceldata hardware sizing guide

  • Cluster size
  • Data retention
  • Concurrent users
  • Single node or multi-node?
  • Hardware sizing for a single node
  • Types of nodes
  • Hardware sizing for multi node setups
  • How much CPU and MEM
  • General storage guidelines
  • Examples
  • General recommendations

Cluster size

Your cluster may be small(S), Medium(M) Large(L) or Very Large(VL)

ClusterNumber of nodes
Small<10
Medium10-30
Large30-100
Very Large100+

Data retention

Acceldata can retain your data for

RetentionNumber of days
low1 week
Mediumupto 3 months
Highupto 1 year

For most installations, We recommend using a medium retention policy to keep storage costs low

Concurrent users

As the number of Acceldata users increase, Higher memory and storage capacities are required to serve the queries

ConcurrencyNumber of users
low<10
Medium10-50
High50-100
Very High100+

Single node or multi-node?

  • Acceldata (without log-search module) can be run on a single node for small(S) or medium(M) clusters with low concurrent users (POC and demo systems)
  • Enabling log-search for any sized cluster will require a multi node setup, Log search indexes have high storage and memory requirements
  • Users with high or very high concurrency should use a multi-node setup (recommended for medium - better Ui experience)

Hardware sizing for a single node

For small(S) clusters we recommend

  • CPU 4-8 cores
  • MEM 16-32 GB
  • Disk ~32GB SSD storage (low retention), 500 GB SSD for medium retention 1 TB+ for high retention

For medium(M) clusters we recommend

  • CPU 8-16 cores
  • MEM 32-64 GB
  • Disk 128GB SSD storage for low retention, 500 GB+ SSD for medium retention 1 TB+ SSD for high retention

Large/Very large clusters should use a multi-node setup

Types of nodes

  • Storage nodes - Where Acceldata stores your data
  • Log-search nodes - Where the log-search indexes are stored
  • Swarm nodes - Application nodes

Hardware sizing for a Multi node setup

Based on your load/data retention policy our advisors will recommend you the number of nodes. For up to 100 nodes with low/medium retention we have observed satisfactory performance with 3-5 nodes. More log-search nodes are required when the volume or logs to be indexed increases, replication must also be considered when planning for capacity.

Storage and log-search nodes

  • CPU 4/8+ cores
  • MEM 32+ GB (more is better)
  • 500 GB SSD for medium retention 1 TB+ for high retention

Swarm nodes

Acceldata are deployed on a docker Swarm

  • CPU 4+ cores
  • MEM 16+ GB (32 GB recommended)
  • ~64GB SSD

How much CPU and MEM

When it comes to RAM, the more the better. The above guidelines are indicative, according to the size of your cluster and the number of logs enabled for search , our advisors will recommend the amount of RAM needed on your nodes.

General storage guidelines

  • Use SSD's - Our storage engines are designed and tested on SSD's and other types of devices may have lower performance.
  • Multiple storage devices - Have multiple storage devices on different mounts Acceldata will assign different devices to different storage engines for better performance

Examples

  • Medium cluster(M) with medium concurrency and medium (3 months data retention) with no log-search

    • 3 nodes with services layout as follows

      NodeConfigServices
      18-16 cores,64GB+ MEM, 500GB SSDAcceldata Database
      28-16 cores,64GB+ MEM, 500GB SSDAcceldata Time series database, Acceldata services
      38-16 cores,32+ MEM, 64GB SSDAcceldata services
  • Large cluster(L) with High concurrency and medium (3 months data retention) with no log-search

    • 4 nodes with services layout as follows

      NodeConfigServices
      18-16 cores,128GB+ MEM, 500GB SSDAcceldata Database
      28-16 cores,64GB+ MEM, 500GB SSDAcceldata Time series database
      38-16 cores,32+ MEM, 64GB SSDAcceldata services(HA)
      48-16 cores,32+ MEM, 64GB SSDAcceldata services(HA)
  • Large cluster with medium data and log retention and high concurrency and log search enabled

    • 7 nodes with services layout as follows

      NodeConfigServices
      18-16 cores,128GB+ MEM, 500GB SSDAcceldata Database
      28-16 cores,64GB+ MEM, 500GB SSDAcceldata Time series database
      38-16 cores,32GB+ MEM, 64GB SSDAcceldata services(HA)
      48-16 cores,32GB+ MEM, 64GB SSDAcceldata services(HA)
      58-16 cores,64GB+ MEM, 500GB SSDLog search
      68-16 cores,64GB+ MEM, 500GB SSDLog search
      78-16 cores,64GB+ MEM, 500GB SSDLog search