Difference between revisions of "Event Records"

From Observer GigaFlow Support | VIAVI Solutions Inc.
Jump to: navigation, search
(Sample Records)
(Sample Records)
Line 111: Line 111:
 
Typical records from an Event table with obfuscated IP addresses and locations:
 
Typical records from an Event table with obfuscated IP addresses and locations:
  
  805729  0              "-1407899398"  "1541583542242"            -11          8              "xxx.xxx.208.28"                "Black List Src Hit(Botnet Strong) xxx.xxx.208.28->xxx.xxx.166.159"            1              "{"Application":"SSH TCP/22","Black List":"https://lists.blocklist.de/lists/strongips.txt","Black List Type":"Source","Eventer":"xxx.xxx.208.28","appid":393238,"bytes":44,"device":"xxx.xxx.40.250","domain":"","dstadd":"xx.xxx.166.159","dstport":22,"duration":0,"eventname":"Black List Src","flags":2,"fwevent":0,"fwextcode":0,"inif":14,"macdst":"00:00:00:00:00:00","macsrc":"00:00:00:00:00:00","outif":13,"packets":1,"proto":6,"srcadd":"xxx.xxx.208.28","srcport":28148,"time":1541583574022,"timeH":"7-Nov-2018 09:39:34.22","tos":0,"user":""}"            "https://lists.blocklist.de/lists/strongips.txt"      75          80                "Botnet Strong" "xxx.xxx.166.159"                "CN"      "Gansu"              "0.00000000000"            "0.00000000000"                "IE"        "County Sligo"  "0.00000000000"            "0.00000000000"
+
  805729  0              "-1407899398"  "1541583542242"            -11          8              "xxx.xxx.208.28"                "Black List Src Hit(Botnet Strong) xxx.xxx.208.28->xxx.xxx.166.159"            1              "
 +
 
 +
{"Application":"SSH TCP/22","Black List":"https://lists.blocklist.de/lists/strongips.txt","Black List Type":"Source","Eventer":"xxx.xxx.208.28","appid":393238,"bytes":44,"device":"xxx.xxx.40.250","domain":"","dstadd":"xx.xxx.166.159","dstport":22,"duration":0,"eventname":"Black List Src","flags":2,"fwevent":0,"fwextcode":0,"inif":14,"macdst":"00:00:00:00:00:00","macsrc":"00:00:00:00:00:00","outif":13,"packets":1,"proto":6,"srcadd":"xxx.xxx.208.28","srcport":28148,"time":1541583574022,"timeH":"7-Nov-2018 09:39:34.22","tos":0,"user":""}"            "https://lists.blocklist.de/lists/strongips.txt"      75          80                "Botnet Strong" "xxx.xxx.166.159"                "CN"      "Gansu"              "0.00000000000"            "0.00000000000"                "IE"        "County Sligo"  "0.00000000000"            "0.00000000000"
  
 
* 805738  0              "29885185"        "1541583568627"            -120      8              "Many" "Syn Dst Unreachable Server xxx.xxx.80.248->xxx.xxx..45.208"          1              "{"Application":"HTTP TCP/80","Eventer":"xxx.xxx.45.208","Syn Type":"Destination","appid":393296,"bytes":44,"device":"xxx.xxx.3.1","domain":"","dstadd":"xxx.xxx.45.208","dstport":80,"duration":0,"eventname":"Syn Dst Unreachable Server","flags":2,"fwevent":0,"fwextcode":0,"inif":8,"macdst":"00:00:00:00:00:00","macsrc":"00:00:00:00:00:00","outif":3,"packets":1,"proto":6,"srcadd":"xx.xxx..80.248","srcport":54028,"time":1541583584143,"timeH":"7-Nov-2018 09:39:44.143","tos":32,"user":""}"        "Syn Destination"            100        100        "Syn Dst Unreachable Server"                "xxx.xxx.45.208"                "CN"      "Guangdong"    "0.00000000000"            "0.00000000000"          "IE"        "null"                "0.00000000000"            "-0.00000000000"
 
* 805738  0              "29885185"        "1541583568627"            -120      8              "Many" "Syn Dst Unreachable Server xxx.xxx.80.248->xxx.xxx..45.208"          1              "{"Application":"HTTP TCP/80","Eventer":"xxx.xxx.45.208","Syn Type":"Destination","appid":393296,"bytes":44,"device":"xxx.xxx.3.1","domain":"","dstadd":"xxx.xxx.45.208","dstport":80,"duration":0,"eventname":"Syn Dst Unreachable Server","flags":2,"fwevent":0,"fwextcode":0,"inif":8,"macdst":"00:00:00:00:00:00","macsrc":"00:00:00:00:00:00","outif":3,"packets":1,"proto":6,"srcadd":"xx.xxx..80.248","srcport":54028,"time":1541583584143,"timeH":"7-Nov-2018 09:39:44.143","tos":32,"user":""}"        "Syn Destination"            100        100        "Syn Dst Unreachable Server"                "xxx.xxx.45.208"                "CN"      "Guangdong"    "0.00000000000"            "0.00000000000"          "IE"        "null"                "0.00000000000"            "-0.00000000000"

Revision as of 14:34, 22 November 2018

An Observer GigaFlow Event is a record that is created when a monitored flow pattern matches certain criteria.

Contents

What Generates an Event?

Some things that will trigger an event record include:

  • Attempts to access blacklisted resources.
  • Profile exceptions, i.e. behaviours deviating from norms.
  • A SYN flood event.
  • A lost neighbour.
  • A new connected device sending flows.
  • A connected device that stops sending flows.

See the Event definition in the Glossary and the main entry for Events in the Reference Manual

What is Recorded?

All GigaFlow flow records contain 29 fields or table columns. In the myipfix database associated with the GigaFlow installation, the netflow tables contain all flow records. See Flow Record Fields for the complete list.

Similarly, the Events tables contain all the Event records. See Event Record Fields for a complete list with descriptions. The Event record fields are:

id, customerid, device, firstseen, eventtype, eventsrctype, eventsrc, message, datatype, data, datasource, confidence, severity, category, target, country_src, division_src, latit_src, longd_src, country_dst, division_dst, latit_dst, 
longd_dst

Three events have both eventtype and posture set in the netflow and event tables in the database, i.e. these three event types can be determined immediately from Netflow records:

Record Field Type Code
postureBlackListSrc int -11
postureBlackListDst int -12
profilerException int -20

In addition, all of these event types have only eventtype set in the event tables, i.e. these event types are classified with additional information or information from many records and have no postureid set in the netflow tables:

Record Field Type Code
synSrc int -100
synSrcPortException int -101
synSrcIPException int -102
synSrcIPAndPortException int -103
synDst int -120
synDstPortException int -121
synDstIPException int -122
synDstIPAndPortException int -123
lldpNewNeighbour int -130
lldpLostNeighbour int -131
deviceRestartsSendingFlows int -150
deviceStopsSendingFlows int -151

How are the Records Accessed?

Data can be accessed:

  1. As a normal user, via the GigaFlow interface. See Events in the Reference Manual.
  2. As an automated portal user, via the GigaFlow API. See Forensic Data Using the API.
  3. From the database directly using a viewer such as pgAdmin or in code using a postgreSQL connector such as Psycopg2 for Python.

Sample Records

Typical records from an Event table with obfuscated IP addresses and locations:

805729  0              "-1407899398"   "1541583542242"             -11          8              "xxx.xxx.208.28"                 "Black List Src Hit(Botnet Strong) xxx.xxx.208.28->xxx.xxx.166.159"             1              "

{"Application":"SSH TCP/22","Black List":"https://lists.blocklist.de/lists/strongips.txt","Black List Type":"Source","Eventer":"xxx.xxx.208.28","appid":393238,"bytes":44,"device":"xxx.xxx.40.250","domain":"","dstadd":"xx.xxx.166.159","dstport":22,"duration":0,"eventname":"Black List Src","flags":2,"fwevent":0,"fwextcode":0,"inif":14,"macdst":"00:00:00:00:00:00","macsrc":"00:00:00:00:00:00","outif":13,"packets":1,"proto":6,"srcadd":"xxx.xxx.208.28","srcport":28148,"time":1541583574022,"timeH":"7-Nov-2018 09:39:34.22","tos":0,"user":""}" "https://lists.blocklist.de/lists/strongips.txt" 75 80 "Botnet Strong" "xxx.xxx.166.159" "CN" "Gansu" "0.00000000000" "0.00000000000" "IE" "County Sligo" "0.00000000000" "0.00000000000"

  • 805738 0 "29885185" "1541583568627" -120 8 "Many" "Syn Dst Unreachable Server xxx.xxx.80.248->xxx.xxx..45.208" 1 "{"Application":"HTTP TCP/80","Eventer":"xxx.xxx.45.208","Syn Type":"Destination","appid":393296,"bytes":44,"device":"xxx.xxx.3.1","domain":"","dstadd":"xxx.xxx.45.208","dstport":80,"duration":0,"eventname":"Syn Dst Unreachable Server","flags":2,"fwevent":0,"fwextcode":0,"inif":8,"macdst":"00:00:00:00:00:00","macsrc":"00:00:00:00:00:00","outif":3,"packets":1,"proto":6,"srcadd":"xx.xxx..80.248","srcport":54028,"time":1541583584143,"timeH":"7-Nov-2018 09:39:44.143","tos":32,"user":""}" "Syn Destination" 100 100 "Syn Dst Unreachable Server" "xxx.xxx.45.208" "CN" "Guangdong" "0.00000000000" "0.00000000000" "IE" "null" "0.00000000000" "-0.00000000000"
  • 805752 0 "-1407899398" "1541583572146" -100 8 "xxx.xxx.86.55" "Syn Src Port Sweep xxx.xxx.86.55->xxx.xxx.85.74" 1 "{"Application":"TCP/3392","Eventer":"xxx.xxx.86.55","Syn Type":"Source","appid":396608,"bytes":40,"device":"xxx.xxx.40.250","domain":"","dstadd":"xxx.xxx.85.74","dstport":3392,"duration":0,"eventname":"Syn Src Port Sweep","flags":2,"fwevent":0,"fwextcode":0,"inif":12,"macdst":"00:00:00:00:00:00","macsrc":"00:00:00:00:00:00","outif":30,"packets":1,"proto":6,"srcadd":"xxx.xxx.86.55","srcport":43017,"time":1541583603834,"timeH":"7-Nov-2018 09:40:03.834","tos":40,"user":""}" "Syn Source" 100 100 "Syn Src Port Sweep" "Many" "RU" "St.-Petersburg" "0.00000000000" "0.00000000000" "IE" "County Sligo" "0.00000000000" "0.00000000000"
  • 805801 0 "-1407899398" "1541583616150" -12 8 "xxx.xxx.195.7" "Black List Dst Hit(Apache(WWW) Scan/Brute) xxx.xxx.86.213->xxx.xxx.195.7" 1 "{"Application":"HTTPS TCP/443","Black List":"https://lists.blocklist.de/lists/apache.txt","Black List Type":"Destination","Eventer":"xxx.xxx.195.7","appid":393659,"bytes":104,"device":"xxx.xxx.40.250","domain":"","dstadd":"xxx.xxx.195.7","dstport":443,"duration":0,"eventname":"Black List Dst","flags":16,"fwevent":0,"fwextcode":0,"inif":30,"macdst":"00:00:00:00:00:00","macsrc":"00:00:00:00:00:00","outif":12,"packets":2,"proto":6,"srcadd":"xxx.xxx.86.213","srcport":51330,"time":1541583646846,"timeH":"7-Nov-2018 09:40:46.846","tos":0,"user":""}" "https://lists.blocklist.de/lists/apache.txt" 75 25 "Apache(WWW) Scan/Brute" "xxx.xxx.86.213" "IE" "County Sligo" "0.00000000000" "0.00000000000" "US" "null" "0.00000000000" "0.00000000000"