Difference between revisions of "Event Record Fields"

From Observer GigaFlow Support | VIAVI Solutions Inc.
Jump to: navigation, search
 
(30 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{| class="wikitable" style="text-align: center; color: green;"
+
[[Category:Database]]
|customerid  
+
[[Category:Scripts]]
 +
See also [http://gigaflowsupport.viavisolutions.com/index.php/Event_Records Event Records].
 +
 
 +
In the '''myipfix''' database associated with the GigaFlow installation, the Events tables contain all the Event records. The Event record fields are:
 +
 
 +
= Field Descriptions =
 +
 
 +
{| class="wikitable" style="text-align: left;"
 +
|'''Record Field'''
 +
|'''Type''' 
 +
|'''Description'''
 +
|-
 +
|id
 +
|integer
 +
|Event identifier.
 +
|-
 +
|customerid
 
|integer  
 
|integer  
|Used to store the traffic group source identifier
+
|The traffic group source identifier
 
|-
 
|-
 
|device
 
|device
 
|numeric(39,0)
 
|numeric(39,0)
|Store the numeric IPV6 address of the device sending us the flowsyslog records
+
|The numeric IPV6 address of the device sending us the flowsyslog records
 
|-
 
|-
| engineid
+
|firstseen
| integer
+
| bigint
| Used to store the traffic group destination identifier
+
| Millisecond timestamp of when this flow started
 
|-
 
|-
| srcadd
+
|eventtype
| numeric(39,0)
+
|integer
| Store the numeric IPV6 address of the source for the traffic in this record
+
|Types of threat identified, e.g. -11 means attempted access by blacklisted resources. See [http://gigaflowsupport.viavisolutions.com/index.php/Event_Records Event Records] for more.
 
|-
 
|-
| dstadd
+
|eventsrctype
| numeric(39,0)
+
|integer
| Store the numeric IPV6 address of the destination for the traffic in this record
+
|Flag indicating type of event source.
 
|-
 
|-
| nexthop
+
|eventsrc
| numeric(39,0)
+
|text
| Store the numeric IPV6 address of the nexthop for the traffic in this record
+
|IP address that triggered the event. If many sources contributed to the event, this will read "many".
 
|-
 
|-
| inif
+
|message
| integer
+
|text
| SNMP ifindex of the input interface that seen the traffic for this flow
+
|Message from system indicating reason for the event, e.g. "Syn Dst Unreachable Server xxx.xxx.xxx.xxx->xxx.xxx.xxx.xxx"
 
|-
 
|-
| outif
+
|datatype
| integer
+
|integer
| SNMP ifindex of the output interface that seen the traffic for this flow
+
|Flag indicating data type. "1" indicates text.
 
|-
 
|-
| pkts
+
|data
| bigint
+
|text
| Number of packets transmitted in this flow
+
|Data output from source, e.g. blacklist.
 
|-
 
|-
| bytes
+
|datasource
| bigint
+
|text
| Number of octetsbytes transmitted in this flow
+
|URL of matched blacklist or other source, e.g. internal script.
 
|-
 
|-
| firstseen
+
|confidence
| bigint
+
|integer
| Millisecond timestamp of when this flow started
+
|Confidence in the completeness and accuracy of the matched blacklist.
 
|-
 
|-
| duration
+
|severity
| bigint
+
|integer
| Millisecond duration of this flow
+
|The importance (or severity) of the blacklist to your organisation.
 
|-
 
|-
| srcport
+
|category
| integer
+
|text
| Source port number for traffic in this flow record
+
|Threat category, e.g. "Botnet Strong". This can be defined in GigaFlow. See [http://gigaflowsupport.viavisolutions.com/manual/watchlists.html Watchlists in the Reference Manual].
 
|-
 
|-
| dstport
+
|target
| integer
+
|text
| Destination port number for traffic in this flow record
+
|Event target host; an IP address, e.g. "xxx.xxx.xxx.xxx"
 
|-
 
|-
| flags
+
|country_src
| integer
+
|text
| TCP Flags as an Integer value
+
|Source country code, e.g. "IE" for Ireland.
 
|-
 
|-
| proto
+
|division_src
| integer
+
|text
| IP Protocol number for this flow record
+
|Source state/county/division, e.g. "Sligo".
 
|-
 
|-
| tos
+
|latit_src
| integer
+
|numeric
| IP TOSCOS value for this flow record
+
| Source IP address latitude.
 
|-
 
|-
| appid
+
|longd_src
| integer
+
|numeric
| Flowsec assigned application id, out of this box this would be the lowest of srcdst port number
+
| Source IP address longitude.
 
|-
 
|-
| srcas
+
|country_dst
| integer
+
|text
| Source AS number used for this flow
+
|Destination country.
 
|-
 
|-
| dstas
+
|division_dst
| integer
+
|text
| Destination AS number used for this flow
+
|Destination state/county/division.
 
|-
 
|-
| userid
+
|latit_dst
| text
+
|numeric
| COLLATE pg_catalog."default" User ID for this flow, may be as sent or inferred from other sources
+
| Destination IP address latitude.
 
|-
 
|-
| userdomain
+
|longd_dst
| text
+
|numeric
| COLLATE pg_catalog."default" User Domain for this flow, may be as sent or inferred from other sources
+
| Destination IP address longitude.
 +
|}
 +
 
 +
= Sample Records =
 +
 
 +
Typical records from an Event table with obfuscated IP addresses and locations:
 +
 
 +
'''postureBlackListSrc (-11)'''
 +
 
 +
Cut and pasted directly from an Events table:
 +
 
 +
<code>
 +
805729  0  "-1407899398"  "1541583542242"            -11          8              "xxx.xxx.208.28"                "Black List Src Hit(Botnet Strong) xxx.xxx.208.28->xxx.xxx.166.159"            1 {"Application":"SSH TCP/22","Black List":"https://lists.blocklist.de/lists/strongips.txt","Black List Type":"Source","Eventer":"xxx.xxx.208.28","appid":393238,"bytes":44,"device":"xxx.xxx.40.250","domain":"","dstadd":"xx.xxx.166.159","dstport":22,"duration":0,"eventname":"Black List Src","flags":2,"fwevent":0,"fwextcode":0,"inif":14,"macdst":"00:00:00:00:00:00","macsrc":"00:00:00:00:00:00","outif":13,"packets":1,"proto":6,"srcadd":"xxx.xxx.208.28","srcport":28148,"time":1541583574022,"timeH":"7-Nov-2018 09:39:34.22","tos":0,"user":""}"            "https://lists.blocklist.de/lists/strongips.txt"      75          80                "Botnet Strong" "xxx.xxx.166.159"                "CN"      "Gansu"              "0.00000000000"            "0.00000000000"                "IE"        "County Sligo"  "0.00000000000"            "0.00000000000"
 +
</code>
 +
 
 +
Arranged more neatly in a table:
 +
 
 +
{| class="wikitable" style="text-align: left;"
 +
|'''Record Field'''
 +
|'''Value'''
 
|-
 
|-
| srcmac
+
|id
| bigint
+
|805729 
| Source MAC address (java long value), either as supplied or inferred from other sources
+
 
|-
 
|-
| dstmac
+
|customerid
| bigint
+
|0
| Destination MAC address (java long value), either as supplied or inferred from other sources
+
|-
 +
|device
 +
|"-1407899398"
 +
|-
 +
|firstseen
 +
|"1541583542242"
 +
|-
 +
|eventtype
 +
| -11
 +
|-
 +
|eventsrctype
 +
|8
 +
|-
 +
|eventsrc
 +
|"xxx.xxx.208.28"
 +
|-
 +
|message
 +
|"Black List Src Hit(Botnet Strong) xxx.xxx.208.28->xxx.xxx.166.159"
 +
|-
 +
|datatype
 +
|1
 +
|-
 +
|data
 +
|{"Application":"SSH TCP/22","Black List":"https://lists.blocklist.de/lists/strongips.txt","Black List Type":"Source","Eventer":"xxx.xxx.208.28","appid":393238,"bytes":44,"device":"xxx.xxx.40.250","domain":"","dstadd":"xx.xxx.166.159","dstport":22,"duration":0,"eventname":"Black List Src","flags":2,"fwevent":0,"fwextcode":0,"inif":14,"macdst":"00:00:00:00:00:00","macsrc":"00:00:00:00:00:00","outif":13,"packets":1,"proto":6,"srcadd":"xxx.xxx.208.28","srcport":28148,"time":1541583574022,"timeH":"7-Nov-2018 09:39:34.22","tos":0,"user":""}"
 +
|-
 +
|datasource
 +
|"https://lists.blocklist.de/lists/strongips.txt"
 +
|-
 +
|confidence
 +
|75
 +
|-
 +
|severity
 +
|80
 +
|-
 +
|category
 +
|"Botnet Strong"
 +
|-
 +
|target
 +
|"xxx.xxx.166.159"
 +
|-
 +
|country_src
 +
|"CN"
 +
|-
 +
|division_src
 +
|"Gansu"
 +
|-
 +
|latit_src
 +
|"0.00000000000"
 
|-
 
|-
| postureid
+
|longd_src
| integer
+
|"0.00000000000"
| Marking to indicate this flow is of interest (due to blacklist or profiling problems)
+
 
|-
 
|-
| spare
+
|country_dst
| integer
+
|"IE"
| Used to store the first packet response value. -1=unset, -2=no response in scope
+
 
|-
 
|-
| url
+
|division_dst
| text
+
|"County Sligo"
| COLLATE pg_catalog."default" Free for text field we use for things like applcation names (which will soon be moved to fwextcode) or URL data
+
 
|-
 
|-
| fwextcode
+
|latit_dst
| integer
+
|"0.00000000000"
| Additional field used to identify traffic (from Cisco NSEL)
+
 
|-
 
|-
| fwevent
+
|longd_dst
| integer
+
|"0.00000000000"
| Additional field used to identify events(from Cisco NSEL)
+
 
|}
 
|}
 +
 +
'''synSrc (-100)'''
 +
 +
<code>
 +
805752  0              "-1407899398"  "1541583572146"            -100      8              "xxx.xxx.86.55"    "Syn Src Port Sweep xxx.xxx.86.55->xxx.xxx.85.74"          1              "{"Application":"TCP/3392","Eventer":"xxx.xxx.86.55","Syn Type":"Source","appid":396608,"bytes":40,"device":"xxx.xxx.40.250","domain":"","dstadd":"xxx.xxx.85.74","dstport":3392,"duration":0,"eventname":"Syn Src Port Sweep","flags":2,"fwevent":0,"fwextcode":0,"inif":12,"macdst":"00:00:00:00:00:00","macsrc":"00:00:00:00:00:00","outif":30,"packets":1,"proto":6,"srcadd":"xxx.xxx.86.55","srcport":43017,"time":1541583603834,"timeH":"7-Nov-2018 09:40:03.834","tos":40,"user":""}"        "Syn Source"      100        100        "Syn Src Port Sweep"    "Many" "RU"                "St.-Petersburg"              "0.00000000000"            "0.00000000000"            "IE"        "County Sligo"                "0.00000000000"            "0.00000000000"
 +
</code>
 +
 +
'''synDst (-120)'''
 +
 +
<code>
 +
805738  0              "29885185"        "1541583568627"            -120      8              "Many" "Syn Dst Unreachable Server xxx.xxx.80.248->xxx.xxx..45.208"          1              "{"Application":"HTTP TCP/80","Eventer":"xxx.xxx.45.208","Syn Type":"Destination","appid":393296,"bytes":44,"device":"xxx.xxx.3.1","domain":"","dstadd":"xxx.xxx.45.208","dstport":80,"duration":0,"eventname":"Syn Dst Unreachable Server","flags":2,"fwevent":0,"fwextcode":0,"inif":8,"macdst":"00:00:00:00:00:00","macsrc":"00:00:00:00:00:00","outif":3,"packets":1,"proto":6,"srcadd":"xx.xxx..80.248","srcport":54028,"time":1541583584143,"timeH":"7-Nov-2018 09:39:44.143","tos":32,"user":""}"        "Syn Destination"            100        100        "Syn Dst Unreachable Server"                "xxx.xxx.45.208"                "CN"      "Guangdong"    "0.00000000000"            "0.00000000000"          "IE"        "null"                "0.00000000000"            "-0.00000000000"
 +
</code>
 +
 +
'''postureBlackListDst (-12)'''
 +
 +
<code>
 +
805801  0              "-1407899398"  "1541583616150"            -12          8              "xxx.xxx.195.7"  "Black List Dst Hit(Apache(WWW) Scan/Brute) xxx.xxx.86.213->xxx.xxx.195.7"    1              "{"Application":"HTTPS TCP/443","Black List":"https://lists.blocklist.de/lists/apache.txt","Black List Type":"Destination","Eventer":"xxx.xxx.195.7","appid":393659,"bytes":104,"device":"xxx.xxx.40.250","domain":"","dstadd":"xxx.xxx.195.7","dstport":443,"duration":0,"eventname":"Black List Dst","flags":16,"fwevent":0,"fwextcode":0,"inif":30,"macdst":"00:00:00:00:00:00","macsrc":"00:00:00:00:00:00","outif":12,"packets":2,"proto":6,"srcadd":"xxx.xxx.86.213","srcport":51330,"time":1541583646846,"timeH":"7-Nov-2018 09:40:46.846","tos":0,"user":""}"          "https://lists.blocklist.de/lists/apache.txt"            75          25                "Apache(WWW) Scan/Brute"    "xxx.xxx.86.213"                "IE"        "County Sligo"  "0.00000000000"            "0.00000000000"                "US"      "null"    "0.00000000000"            "0.00000000000"
 +
</code>

Latest revision as of 14:52, 29 November 2018

See also Event Records.

In the myipfix database associated with the GigaFlow installation, the Events tables contain all the Event records. The Event record fields are:

Field Descriptions

Record Field Type Description
id integer Event identifier.
customerid integer The traffic group source identifier
device numeric(39,0) The numeric IPV6 address of the device sending us the flowsyslog records
firstseen bigint Millisecond timestamp of when this flow started
eventtype integer Types of threat identified, e.g. -11 means attempted access by blacklisted resources. See Event Records for more.
eventsrctype integer Flag indicating type of event source.
eventsrc text IP address that triggered the event. If many sources contributed to the event, this will read "many".
message text Message from system indicating reason for the event, e.g. "Syn Dst Unreachable Server xxx.xxx.xxx.xxx->xxx.xxx.xxx.xxx"
datatype integer Flag indicating data type. "1" indicates text.
data text Data output from source, e.g. blacklist.
datasource text URL of matched blacklist or other source, e.g. internal script.
confidence integer Confidence in the completeness and accuracy of the matched blacklist.
severity integer The importance (or severity) of the blacklist to your organisation.
category text Threat category, e.g. "Botnet Strong". This can be defined in GigaFlow. See Watchlists in the Reference Manual.
target text Event target host; an IP address, e.g. "xxx.xxx.xxx.xxx"
country_src text Source country code, e.g. "IE" for Ireland.
division_src text Source state/county/division, e.g. "Sligo".
latit_src numeric Source IP address latitude.
longd_src numeric Source IP address longitude.
country_dst text Destination country.
division_dst text Destination state/county/division.
latit_dst numeric Destination IP address latitude.
longd_dst numeric Destination IP address longitude.

Sample Records

Typical records from an Event table with obfuscated IP addresses and locations:

postureBlackListSrc (-11)

Cut and pasted directly from an Events table:

805729 0 "-1407899398" "1541583542242" -11 8 "xxx.xxx.208.28" "Black List Src Hit(Botnet Strong) xxx.xxx.208.28->xxx.xxx.166.159" 1 {"Application":"SSH TCP/22","Black List":"https://lists.blocklist.de/lists/strongips.txt","Black List Type":"Source","Eventer":"xxx.xxx.208.28","appid":393238,"bytes":44,"device":"xxx.xxx.40.250","domain":"","dstadd":"xx.xxx.166.159","dstport":22,"duration":0,"eventname":"Black List Src","flags":2,"fwevent":0,"fwextcode":0,"inif":14,"macdst":"00:00:00:00:00:00","macsrc":"00:00:00:00:00:00","outif":13,"packets":1,"proto":6,"srcadd":"xxx.xxx.208.28","srcport":28148,"time":1541583574022,"timeH":"7-Nov-2018 09:39:34.22","tos":0,"user":""}" "https://lists.blocklist.de/lists/strongips.txt" 75 80 "Botnet Strong" "xxx.xxx.166.159" "CN" "Gansu" "0.00000000000" "0.00000000000" "IE" "County Sligo" "0.00000000000" "0.00000000000"

Arranged more neatly in a table:

Record Field Value
id 805729
customerid 0
device "-1407899398"
firstseen "1541583542242"
eventtype -11
eventsrctype 8
eventsrc "xxx.xxx.208.28"
message "Black List Src Hit(Botnet Strong) xxx.xxx.208.28->xxx.xxx.166.159"
datatype 1
data {"Application":"SSH TCP/22","Black List":"https://lists.blocklist.de/lists/strongips.txt","Black List Type":"Source","Eventer":"xxx.xxx.208.28","appid":393238,"bytes":44,"device":"xxx.xxx.40.250","domain":"","dstadd":"xx.xxx.166.159","dstport":22,"duration":0,"eventname":"Black List Src","flags":2,"fwevent":0,"fwextcode":0,"inif":14,"macdst":"00:00:00:00:00:00","macsrc":"00:00:00:00:00:00","outif":13,"packets":1,"proto":6,"srcadd":"xxx.xxx.208.28","srcport":28148,"time":1541583574022,"timeH":"7-Nov-2018 09:39:34.22","tos":0,"user":""}"
datasource "https://lists.blocklist.de/lists/strongips.txt"
confidence 75
severity 80
category "Botnet Strong"
target "xxx.xxx.166.159"
country_src "CN"
division_src "Gansu"
latit_src "0.00000000000"
longd_src "0.00000000000"
country_dst "IE"
division_dst "County Sligo"
latit_dst "0.00000000000"
longd_dst "0.00000000000"

synSrc (-100)

805752 0 "-1407899398" "1541583572146" -100 8 "xxx.xxx.86.55" "Syn Src Port Sweep xxx.xxx.86.55->xxx.xxx.85.74" 1 "{"Application":"TCP/3392","Eventer":"xxx.xxx.86.55","Syn Type":"Source","appid":396608,"bytes":40,"device":"xxx.xxx.40.250","domain":"","dstadd":"xxx.xxx.85.74","dstport":3392,"duration":0,"eventname":"Syn Src Port Sweep","flags":2,"fwevent":0,"fwextcode":0,"inif":12,"macdst":"00:00:00:00:00:00","macsrc":"00:00:00:00:00:00","outif":30,"packets":1,"proto":6,"srcadd":"xxx.xxx.86.55","srcport":43017,"time":1541583603834,"timeH":"7-Nov-2018 09:40:03.834","tos":40,"user":""}" "Syn Source" 100 100 "Syn Src Port Sweep" "Many" "RU" "St.-Petersburg" "0.00000000000" "0.00000000000" "IE" "County Sligo" "0.00000000000" "0.00000000000"

synDst (-120)

805738 0 "29885185" "1541583568627" -120 8 "Many" "Syn Dst Unreachable Server xxx.xxx.80.248->xxx.xxx..45.208" 1 "{"Application":"HTTP TCP/80","Eventer":"xxx.xxx.45.208","Syn Type":"Destination","appid":393296,"bytes":44,"device":"xxx.xxx.3.1","domain":"","dstadd":"xxx.xxx.45.208","dstport":80,"duration":0,"eventname":"Syn Dst Unreachable Server","flags":2,"fwevent":0,"fwextcode":0,"inif":8,"macdst":"00:00:00:00:00:00","macsrc":"00:00:00:00:00:00","outif":3,"packets":1,"proto":6,"srcadd":"xx.xxx..80.248","srcport":54028,"time":1541583584143,"timeH":"7-Nov-2018 09:39:44.143","tos":32,"user":""}" "Syn Destination" 100 100 "Syn Dst Unreachable Server" "xxx.xxx.45.208" "CN" "Guangdong" "0.00000000000" "0.00000000000" "IE" "null" "0.00000000000" "-0.00000000000"

postureBlackListDst (-12)

805801 0 "-1407899398" "1541583616150" -12 8 "xxx.xxx.195.7" "Black List Dst Hit(Apache(WWW) Scan/Brute) xxx.xxx.86.213->xxx.xxx.195.7" 1 "{"Application":"HTTPS TCP/443","Black List":"https://lists.blocklist.de/lists/apache.txt","Black List Type":"Destination","Eventer":"xxx.xxx.195.7","appid":393659,"bytes":104,"device":"xxx.xxx.40.250","domain":"","dstadd":"xxx.xxx.195.7","dstport":443,"duration":0,"eventname":"Black List Dst","flags":16,"fwevent":0,"fwextcode":0,"inif":30,"macdst":"00:00:00:00:00:00","macsrc":"00:00:00:00:00:00","outif":12,"packets":2,"proto":6,"srcadd":"xxx.xxx.86.213","srcport":51330,"time":1541583646846,"timeH":"7-Nov-2018 09:40:46.846","tos":0,"user":""}" "https://lists.blocklist.de/lists/apache.txt" 75 25 "Apache(WWW) Scan/Brute" "xxx.xxx.86.213" "IE" "County Sligo" "0.00000000000" "0.00000000000" "US" "null" "0.00000000000" "0.00000000000"