Observer GigaFlow

Documentation

Table of Contents

Observer GigaFlow Documentation

Documentation > Reference Manual for GigaFlow > System > Global

Global

New icons along top.

Located at System > Global.

You can find most of the system-wide settings here.

Along the top of the page are quick links to the different settings options. These are:

  • General.
  • LDAP.
  • SSL.
  • Import/Export.
  • Remote Services.
  • SNMP V2.
  • SNMP V3.
  • Log.
  • Proxy.
  • MAC Vendors.
  • Mail.
  • Storage.
  • Integrations.

General

In the General settings box, you can change:

  • Server name. This is the name associated with the GigaFlow server and does not have to be the same as the hostname. This is the name used when when the server calls the GigaFlow home servers. It aids identification.
  • Theme: light or dark.
  • Display IP address on login? Yes or No.
  • Max number of First Packet Response Clients. This is the maximum number of First Packet Response clients supported by the system and any one point in time. 400 is the default.
  • Max memory IPs, 300,000 by default. This is the maximum number of IP addresses that can be cached in RAM before writing to disk. It functions as a memory table lookup for GigaFlow Search. In general, there is no need for you to change this. The aim is to reduce disk read and writes.
  • TCP Testing Ports, e.g. 22, 23, 80, 443, 3389, 1143, 21.
  • TCP Testing Timeout, e.g. 200 ms.
  • TCP Port Threads, e.g. 4.
  • TCP Device Threads, e.g. 4.
  • TCP Syn Port Threshold, e.g 5.
  • TCP Syn Address Threshold, e.g 10.
  • Duplicate Cache Size, e.g 90. This is the number of flows that each devices should cache in order to try and detect duplicates. It will use the start/end time, src/dst port and IP address as well as the byte and packet count to match.
  • Process Duplicate Flows: Yes or No.
  • Ignore Duplicates On Summaries: Yes or No.
  • SNMP Timout: 2000 milliseconds.
  • Server Discovery Limit: default is 100 servers.
Server changes require a system restart.
  • Server Auto Discovery Enabled: Yes or No.
  • Web Session Timeout: default is 30 minutes.

LDAP

Before you can authenticate users using LDAP or AD, you must set the LDAP (Lightweight Directory Access Protocol) server.

Examples for an LDAP server integration :

  • Server address, e.g. ldap://172.21.40.189:389.
  • LDAP Group DN, the branch that should be searched to return a list of groups from e.g. ou=observer,dc=viavi,dc=solutions
  • LDAP Group Field, the dn of the field to return e.g. entryDN
  • LDAP Group Search,the filter to use when searching for groups e.g. (&(objectClass=groupOfNames))
  • LDAP Group Search Filtered, the filter to use in the users page when filterling the list of available groups e.g. (&(objectClass=groupOfNames)(cn=$FILTER))
  • LDAP User DN Base, the branch from which to search users e.g. ou=users,dc=viavi,dc=solutions
  • LDAP User DN Field, the DN for the users e.g. entryDN
  • LDAP Users Group Field, the filed representing the users group membership e.g. memberOf
  • LDAP User Filter,the filter to apply when searching for users e.g. (&(objectClass=inetOrgPerson)(uid=$USERID))
  • Username, the dn of the user to bind when searching the server e.g. cn=admin,dc=viavi,dc=solutions
  • Domain Name, not required for NON AD servers e.g. LEAVE BLANK
  • Password, e.g. XXXXXXXXXX
  • Status, i.e. if the connection is good and how many LDAP groups have been retrieved.

Examples for Active Directory server integration :

  • Server address, e.g. ldap://172.21.40.189:389.
  • LDAP Group DN, e.g. dc=anuview,dc=net
  • LDAP Group Field, e.g. distinguishedName
  • LDAP Group Search, e.g. (&(objectClass=group))
  • LDAP Group Search Filtered, e.g. (&(objectClass=group)(cn=$FILTER))
  • LDAP User DN Base, e.g. dc=anuview,dc=net
  • LDAP User DN Field, e.g. cn
  • LDAP Users Group Field, e.g. memberOf
  • LDAP User Filter, e.g. (&(objectClass=user)(sAMAccountName=$USERID))
  • Username, e.g. Administrator
  • Domain Name, e.g. anuview.net
  • Password, e.g. XXXXXXXXXX
  • Status, i.e. if the connection is good and how many LDAP groups have been retrieved.

SSL

In the SSL settings box, you can view and select or change:

  • Existing certificate aliases.
  • Keystore location on the network or server. Leave this blank for default setting.
  • Keystore password. Leave this blank for default setting.
  • Click Save Keystore Settings.

Saving will restart the HTTPS service which may take up to one minute.

Import

The Import and Export windows allow you to easily import or export all of your GigaFlow configuration and system settings.

To import GigaFlow configuration and system settings:

  • Click Import
  • Copy the text version of the new configuration and system settings.
  • Paste this text into input window.
  • Click Save new configuration and system settings to enter and save the new configuration and system settings.

Export

To export your GigaFlow configuration and system settings:

  • Use the first icon from the left, above the main tabs, to Export configuration and system settings.
  • Copy and paste this text into a text editor, e.g. Notepad, and save it.

Remote Services

ZeroMQ
  • Status: default, Not Configured.
  • Listener: default, '*.5555.
  • Cache Period (milliseconds): default, 60000.
  • Cache Size: default, 100000.
  • Max Flows Transmitted Per Message: default, 100000.
  • Max Seconds Per Message: default, 60000.
  • Message Store:
Restart required.
  • Message Store Max Size MB: default, 50000.

Save ZeroMQ Settings.

Observer GigaStor
  • URL.
  • Display.
  • User Name.
  • Password.
  • Status.
  • Actions.
Kafka
  • Status: No status for now.
  • Broker: enter the Kafka broker address.
  • SSL Trust Store: path to trust store.
  • Trust Store Pass: password.
  • Topic: default, gigaflowraw.
  • Group ID: group name.
  • Cache Period (milliseconds): default, 300000.
  • Cache Size: 5000.

Save Kafka Settings.

Syslog Forensics
  • Status: for example, has sent w packets , x dropped, y largest, z, longest t (ms).
  • Format: Text or JSON.
  • Storage Buffer: 10000.
  • Server: IP address and port in the format xxx.xxx.xxx.xxx:xx
A change requires a restart to affect all devices.
  • Cache Period (milliseconds): 300000.
A change requires a restart to affect all devices.
  • Cache Size: 5000.
A change requires a restart to affect all devices.

Save Syslog Settings.

SNMP V2 Settings

Click the second icon from the left, above the main tabs, to add an SNMP v2 Settings Commuity.

  • Enter the community name in the text box and click Save.

SNMP allows GigaFlow to poll sending devices. GigaFlow supports multiple concurrent pollers. Each device is serviced every 30 minutes. SNMP polling is used to retrieve:

  • Device information, i.e. location, sysOID, interface names, descriptions, aliases and IP addresses.
  • ARP table for IP address to MAC address to interface mapping.
  • CAM table for MAC to switch port mapping.

In the SNMP V2 settings box, you can add or delete new SNMP V2 community strings. To add or delete a community string:

  • Delete existing communities by clicking the delete icon.

SNMP V3 Settings

Click the third icon from the left, above the main tabs, to add an SNMP v3 Settings Commuity.

  • Enter the new SNMP V3 community name.
  • Enter a user name.
  • Authentication Type, i.e. MD5 or SHA secure hash functions.
  • Enter an authentication type password.
  • Privacy Type, i.e. AES-128, AES -198, AES-256, DES or 3DES.
  • Enter a privacy password.
  • Enter a context. SNMP contexts provide VPN users with a secure way of accessing MIB data. See Cisco and the Glossary for more.
  • Submit by clicking Save.

SNMP allows GigaFlow to poll sending devices. In the SNMP V3 settings box, you can view and delete SNMP v3 community strings.

Log Setting

In the Log settings box, you can view the log file and change the logging level:

  • Click on Log File to view the log file entries.
  • Select a logging level from the drop-down menu, i.e. DEBUG, INFO, ERROR, FATAL, WARN.
  • Click Save Save icon. to save changes to the logging level.

Proxy Setting

These are the proxy settings used to access GigaFlow's homeworld server. The information is required for several reasons: (i) to allow GigaFlow to call home and register itself; (ii) to let us know that your installation is healthy and working; (iii) to update blacklists. All calls home are in cleartext. See System > Licences for more.

  • Server ID. This is the unique server ID; it is sent in calls home.
  • Proxy address. Leave blank to disable proxy use.
  • Proxy port, e.g. 80.
  • Proxy user.
  • Proxy password.
  • Click Test Proxy Settings to test the proxy server.
  • Click Save Proxy Settings to save the settings.

MAC Vendors

Click the fourth icon from the left, above the main tabs, to add a MAC Vendor. To add a new vendor prefix:

  • Enter a MAC address.
  • Enter the prefix length in octets, i.e. 12 for the full address, 10 leftmost octets, 8 leftmost octets, 6 leftmost octets or 4 leftmost octets.
  • Enter the vendor name, e.g. Dell.
  • Click Save to save changes.

Existing vendor prefixes are displayed in the table below alongside the date created and available actions, i.e. modify or delete.

Mail Settings

If you are using email alerts, you can configure email here. To set up email:

  • Change Enabled from No to Yes.
  • Enter the email address that GigaFlow will send email from.
  • Enter the email address of the user receiving GigaFlow alerts.
  • TLS (transport layer security)? Yes or No. See Glossary for more about TLS.
  • Enter the mail server username.
  • Enter the mail server password.
  • Enter the mail server address.
  • Enter the mail server port number, e.g. 25.
  • Click Test Mail Settings to test the email server.
  • Click 'Save Mail Settings to save changes.

Storage

Storage Settings

In the Storage settings box, you can make changes to the following settings:

  • Monitor Drive Space: enable or disable drive space monitoring, i.e. Yes or No.
  • Drive To Monitor: set the drive to monitor, e.g. C:/.
  • Min Free Space (GB): set the minimum free space allowed (GB).
  • Default Device Storage Space (GB): set the default device storage space (GB).
  • Min Forensics Storage (Days): set the minimum forensics storage (Days), e.g. 21. After this time, flowsec records will be deleted.
Requires 720 MB or 15 per hour.
See Disk Usage by Device', below.
  • Max Forensics Storage (Days):
Cannot determine how many tables may be generated as value =0.
  • IP Search Storage (Days): set the IP search storage (Days), e.g. 21. After this time, IP address history will be deleted.
  • Forensics Storage Threads:
  • Forensics Table Cache Size: set the forensics table cache size, e.g. 10,000. This is the number of entries cached before writing to disk.
  • Forensics Table Cache Age (ms): set the forensic table cache age (milliseconds), e.g. 10,000. After 10 seconds, the forensic data is written to disk.
  • Forensics Cache Storage: set the forensic cache storage size, e.g. 40,000.
  • Forensics Indexes: enter forensics indexes. This is a comma-delimited list of forensics table field names, e.g. "srcadd,dstadd,appid". See Reports > Forensics in the Reference Manual for more.
  • Enter forensics indexes
  • Forensics Rollup Age (Days): set the forensic rollup age (Days), e.g. 4 days, period after which data should be rolled into daily tables.
  • Forensics Rollup Location: set the forensics rollup location.
  • Event Storage Period (Days): set the event storage period (Days), e.g. 100 days, how long events should be recorded for.
  • ARP Storage Period (Days): set the ARP storage period (Days), e.g. 100 days, how long ARP entries should be recorded for.
  • CAM Storage Period (Days): set the CAM storage period (Days), e.g. 100 days, how long ARP entries should be recorded for.
  • Event Summary Storage Period (Days): set the event summary storage period (Days), e.g. 200 days.
  • Interface Summary Storage Period (Days): set the interface summary storage period (Days), e.g. 200 days.
  • DB Fetch Size:
  • Allow Unlogged flow tables:
  • Auto-tune PostgreSQL: enable or disable Auto Tune of the Postgres database. Yes or No.
  • Archive Folder Location:
  • Enter Archive Folder Location
  • Archiving Enabled :
  • TCP Syslog Threads:*
  • TCP Syslog Buffer Size*

To commit changes, click Save Storage Settings.

Disk Usage by Device

The following information is given for each flow device:

Netflow

  • Device: device name.
  • Largest Time: date and hour.
  • File Window: hours.
  • Total MB.
  •  % of All.
  • Average File MB.
  • Average Per Day MB.
  • Largest Size.

Data Retention and Rollup

The minimum Forensics Storage is 21 days. Forensics data is stored in tables for up to four hours to speed up search and reporting. These tables are rolled into one-day tables after the Forensics Rollup Age period; this is four days by default. (see System > Global.) This is a minimum period.

It is important to set aside space for drive monitoring. GigaFlow will fill the disk drives until the pre-defined minimum amount of free space is left. GigaFlow caps the storage that any particular device is using for forensics data; this is 50 GB by default. This cap can be changed globally and on a per device basis which in turn sets an overall cap on the amount of space used by GigaFlow.

Data Retention and Rollup
TypeResolutionTable Duration (default)RetentionSetting Involved
Raw Flowsmillisecond1 hour-1 day21 daysMin Free Space, Default Device Storage Space, Min Forensics Storage, Forensics Rollup Age.
IP Searchmillisecond1 day21 daysIP Search Duration.
Eventsmillisecond4 hour100 daysEvent Storage Period, Event Summary Storage Period.
ARPmillisecond1 day100 daysARP Storage Period.
CAMmillisecond1 day100 daysCAM Storage Period.
Interface SummariesMinute2 day200 daysInterface Summary Storage Period.
Traffic SummariesHour7 day200 daysInterface Summary Storage Period.

Integrations

An integration allows you to call defined external web pages or scripts directly from the Device Interface Overview page. To add an integration, navigate to System > Global. In the Integrations settings box, you can add or change an integration.

There are two types of integration:

  • Device: where you can pass the device IP address.
  • Interface: where you can pass the device IP address and the ifIndex of the interface being used.

To add an integration:

  • Click on the Add Integration icon, fifth from left at the top of the page.
  • Enter a new name, e.g. "Interface Action 1".
  • Select the type of target, i.e. a device or an interface. These map to the IP address of the device and the ifIndex of the interface.
  • Select the type of action, i.e. will the integration work with a web page or a script.
  • Enter the web page URL or select a script.
  • Enter inputs to be passed via the website or script.
  • Click Save Save icon. to save changes.

Example 1

In the code below, the populated field tells the software that you want to populate the device field with the IP address and the ifindex field with the ifindex. These fields (device and ifindex) will be passed to the target. We also have required fields, which are fields which the user is required to populate.

{'populated':{'device':'flow_device','ifindex':'flow_ifindex'}, 'required':[{'name':'user','display':,'type':'text','value':},{'name':'password','display':'Password','type':'password'},{'name':'macro','display':'What Macro?','type':'select','data':['macro1','macro2','macro3','macro4']}

]}

Example 2:

var ProcessBuilder = Java.type('java.lang.ProcessBuilder');var BufferedReader = Java.type('java.io.BufferedReader');var InputStreamReader= Java.type('java.io.InputStreamReader');output.append(data);output.append("Device IP:"+data.get("device")+"");output.append("IFIndex:"+data.get("ifindex")+"");  try {    // Use a ProcessBuilder    //var pb = new ProcessBuilder("ls","-lrt","/"); //linux    var pb = new ProcessBuilder("cmd.exe", "/C", "dir"); //windows    output.append("Command Run");    var p = pb.start();    var is = p.getInputStream();    var br = new BufferedReader(new InputStreamReader(is));    var line = null;    while ((line = br.readLine()) != null) {    output.append(line+"");   }   var r = p.waitFor(); // Let the process finish.   if (r == 0) { // No error      // run cmd2.   }   output.append("All Done"); } catch ( e) {   output.append(e.printStackTrace()); } log.warn("end")

Add Observer GigaStor

Click the Add Observer GigaStor icon; this is the last icon at the top of the page. Then enter:

  • Display Name:
  • URL:
  • User:
  • Password:

And click Save.