Difference between revisions of "Old Legacy Server Sizing"

From Observer GigaFlow Support | VIAVI Solutions Inc.
Jump to: navigation, search
m (Kevin Wilkie moved page Server Sizing to Old Legacy Server Sizing)
 
(40 intermediate revisions by 3 users not shown)
Line 1: Line 1:
Anuview Flow has been tested and certified to support up to 1000 concurrent devices or up to 40k flows/s from < 20 devices.
+
[[Category:Database]]
 +
Since September 2020, the [http://gigaflowsupport.viavisolutions.com/manual/server_specification_calculator.html old Server Specification Calculator] has been deprecated. Our new server sizing calculator will be posted soon.
  
 +
This article is out of date.
  
Below we can see as more devices are added to the system, the supported flow rate decreases in proportion.
+
== Flows and Server Sizing ==
 +
 
 +
GigaFlow has been tested and certified to support up to 1,000 concurrent devices or up to 40,000 flows per second (flow/s) from less than 20 devices.
 +
 
 +
The flow rate changes with the number of connected devices as follows:
  
 
  Flows/s Devices
 
  Flows/s Devices
  50000 10
+
  50,000 10
  40000 20
+
  40,000 20
  20000 40
+
  20,000 40
  10000 80
+
  10,000 80
  5000 160  
+
  5,000 160
  2500 300
+
  2,500 300
  1250 600
+
  1,250 600
  1000 1000
+
  1,000 1,000
  
 +
== Disk Throughput for Flow Writing to Database ==
  
'''Disk Throughput For Flow Writing To DB'''
+
Allow for at least 600 bytes per flow record per second for I/O throughput, i.e.
Allow for at least 600 bytes per flow record per seconds for IO throughput e.g.
+
100 flows/s = 1000*600 = 60000=0.6MB/sec of sustained write performance required.
+
2000 flows/s = 2000*600 = 1200000 = 1.2MB/s of sustained write performance required.
+
10000 flows/s = 10000*600 = 6000000 = 6MB/s of sustained write performance required.
+
40000 flows/s = 40000*600 = 24000000 = 24MB/s of sustained write performance required.
+
  
'''Disk IO For Flow Writing To DB'''
+
{| class="wikitable" border="1"
The formula 20+(flowPerSecond/500)+(devices/5)=IOPs Write sustained
+
|-
 +
! Flow/s
 +
! Bytes/s (Sustained Write Performance)
 +
! MB/s (Sustained Write Performance)
 +
|-
 +
| 100
 +
| 60,000
 +
| 0.06
 +
|-
 +
| 2,000
 +
| 1,200,000
 +
| 1.20
 +
|-
 +
| 10,000
 +
| 6,000,000
 +
| 6.00
 +
|-
 +
| 40,000
 +
| 24,000,000
 +
| 24.00
 +
|}
  
1) Allow for a base of 20 IOPs.
+
== Disk I/O for Flow Writing To Database ==
  
2) Add an additional 1 IOP/s for every 500 flows/s.
+
With
 +
f = flow/s  
 +
d = number of devices
 +
I = Input/Output performance measurement (IOP), nominally sustained sequential writing.
  
3) Add an additional 1 IOP/s for every 5 devices
+
I = 20 + (f / 500) + (d / 5)
1k flows/s, 1000 devices = (20 +(1000/500)+(1000/5)) = 20+2+200= 222 IOPs
+
5k flows/s, 1000 devices = (20 +(5000/500)+(1000/5)) = 20+10+200= 230 IOPs
+
10k flows/s, 100 devices = (20 +(10000/500)+(100/5)) = 20+20+20= 60 IOPs
+
40k flows/s, 10 devices = (20 +(40000/500)+(10/5)) = 20+80+2= 102 IOPs
+
  
'''Disk IO For Flow Reading From DB'''
+
i.e. allow for a base of 20 IOPs, add an additional 1 IOP/s for every 500 flow/s and another 1 IOP/s for every 5 devices.
Allow for 100 IOPs read to deliver a suitable reporting experience
+
100,000 rows of flow data take approx 1 second to report on at a 100MB/s read rate.
+
  
 +
{| class="wikitable" border="1"
 +
|-
 +
!Flow/s
 +
! Number of Devices
 +
! IOPs
 +
|-
 +
| 1,000
 +
| 1,000
 +
| 222
 +
|-
 +
| 5,000
 +
| 1,000
 +
| 230
 +
|-
 +
| 10,000
 +
| 100
 +
| 60
 +
|-
 +
| 40,000
 +
| 10
 +
| 102
 +
|}
 +
 +
== Disk I/O for Flow Reading from Database ==
 +
 +
Allow for at least 100 IOPs read.
 +
 +
== Disk Sizing ==
  
'''Disk Sizing'''
+
The server must support at least 300 MB/s sustained read and write to handle the peak device or flow count. Anything less than this will result in dropped flows. For Linux, we recommend EXT4 or XFS file systems as well a dedicated RAID partition for the database. Adding a hardware RAID controller that supports RAID 10, or at least RAID 5, will improve performance and provide hardware redundancy. The amount of storage required is directly related to the flow rate and features enabled.
The server must support at least 300MB/s sustained for both read and write to support the peak device or flow count. Failing to do so will result in dropped flow details.
+
For Linux, the recommendation would be to use EXT4 or XFS file systems and have the database on it's own raid partition.
+
The addition of a hardware raid controller supporting Raid 10 or at least Raid 5 will also improve performance and provide hardware redundancy.
+
The amount of storage required is directly related to the flow rate and what features are enabled.
+
  
 
{| class="wikitable" border="1"
 
{| class="wikitable" border="1"
 
|-
 
|-
 
! Data Type
 
! Data Type
! Min Space Per Record
+
! Minimum Space per Record (Bytes)
 
|-
 
|-
 
| Forensics Flow
 
| Forensics Flow
| 250bytes
+
| 250
 
|-
 
|-
 
| Event Record
 
| Event Record
| 900Bytes
+
| 900
 
|}
 
|}
e.g
 
500 flow/s of forensics = 450MB per hour = 11GB of disk space per day.
 
  
 +
500 flow/s of forensics == 450 MB per hour == 11 GB disk space per day.
 +
 +
== RAM Sizing ==
 +
 +
A basic installation should have 4 GB RAM available for the OS and additional 50 MB per device to monitor. More RAM will always improve performance:
  
'''RAM Sizing'''
 
A base installation should have 4GB or RAM available for the OS and and additional 50MB per device to monitor. More ram will always improve performance
 
e.g
 
 
{| class="wikitable" border="1"
 
{| class="wikitable" border="1"
 
|-
 
|-
! Number Of Devices
+
! Number of Devices
! Min Ram
+
! Min RAM (GB)
 
|-
 
|-
 
| 10
 
| 10
| 4.5GB
+
| 4.5
 
|-
 
|-
 
| 100
 
| 100
| 9GB
+
| 9.0
 
|-
 
|-
 
| 500
 
| 500
| 29GB
+
| 29.0
 
|-
 
|-
| 1000
+
| 1,000
| 54GB
+
| 54.0
 
|}
 
|}
  
'''CPU sizing'''
+
== CPU Sizing ==
As Anuview Flow is based on the Postgres database, overall performance is also dependent on CPU performance.  
+
 
 +
CPU sizing in GigaFlow is based on the Postgre SQL database. Overall performance is also dependent on CPU performance.
 +
 
 +
While there is little to gain by going beyond 8 cores if you have more than 20 devices the load can be spread across additional cores, more powerful CPUs will provide a better experience. Intel's Xeon X5680 3GHz or Core i7-3770S 3GHz are recommended as a minimum required specification.
  
There is little gain going beyond 8 cores however more powerful CPUs would provide a better experience. Intel XeonX5680 3GHz or Core i7-3770S 3GHz would be recommended as a minimum.
+
[http://gigaflowsupport.viavisolutions.com/manual/server_specification_calculator.html See here for a useful server size calculator.]
  
[http://wiki.anuviewsoftware.com/calculator.html Calculator]
+
See Manual FAQs for more, including sample calculations.

Latest revision as of 15:05, 8 March 2022

Since September 2020, the old Server Specification Calculator has been deprecated. Our new server sizing calculator will be posted soon.

This article is out of date.

Contents

Flows and Server Sizing

GigaFlow has been tested and certified to support up to 1,000 concurrent devices or up to 40,000 flows per second (flow/s) from less than 20 devices.

The flow rate changes with the number of connected devices as follows:

Flows/s	Devices
50,000	10
40,000	20
20,000	40
10,000	80
5,000	160
2,500	300
1,250	600
1,000	1,000

Disk Throughput for Flow Writing to Database

Allow for at least 600 bytes per flow record per second for I/O throughput, i.e.

Flow/s Bytes/s (Sustained Write Performance) MB/s (Sustained Write Performance)
100 60,000 0.06
2,000 1,200,000 1.20
10,000 6,000,000 6.00
40,000 24,000,000 24.00

Disk I/O for Flow Writing To Database

With f = flow/s d = number of devices I = Input/Output performance measurement (IOP), nominally sustained sequential writing.

I = 20 + (f / 500) + (d / 5)

i.e. allow for a base of 20 IOPs, add an additional 1 IOP/s for every 500 flow/s and another 1 IOP/s for every 5 devices.

Flow/s Number of Devices IOPs
1,000 1,000 222
5,000 1,000 230
10,000 100 60
40,000 10 102

Disk I/O for Flow Reading from Database

Allow for at least 100 IOPs read.

Disk Sizing

The server must support at least 300 MB/s sustained read and write to handle the peak device or flow count. Anything less than this will result in dropped flows. For Linux, we recommend EXT4 or XFS file systems as well a dedicated RAID partition for the database. Adding a hardware RAID controller that supports RAID 10, or at least RAID 5, will improve performance and provide hardware redundancy. The amount of storage required is directly related to the flow rate and features enabled.

Data Type Minimum Space per Record (Bytes)
Forensics Flow 250
Event Record 900

500 flow/s of forensics == 450 MB per hour == 11 GB disk space per day.

RAM Sizing

A basic installation should have 4 GB RAM available for the OS and additional 50 MB per device to monitor. More RAM will always improve performance:

Number of Devices Min RAM (GB)
10 4.5
100 9.0
500 29.0
1,000 54.0

CPU Sizing

CPU sizing in GigaFlow is based on the Postgre SQL database. Overall performance is also dependent on CPU performance.

While there is little to gain by going beyond 8 cores if you have more than 20 devices the load can be spread across additional cores, more powerful CPUs will provide a better experience. Intel's Xeon X5680 3GHz or Core i7-3770S 3GHz are recommended as a minimum required specification.

See here for a useful server size calculator.

See Manual FAQs for more, including sample calculations.