Difference between revisions of "Forensic Data Using the API"

From Observer GigaFlow Support | VIAVI Solutions Inc.
Jump to: navigation, search
(API Request)
(API Request)
Line 68: Line 68:
 
If you don't specify a '''myid''' field, one will automatically be populated in the format of "report_TIMESTAMP_UUID".
 
If you don't specify a '''myid''' field, one will automatically be populated in the format of "report_TIMESTAMP_UUID".
 
  {"action":"get","device":["172.21.40.199","172.21.21.254"],"myid":"report_1578915304695_4be2423e-d5cf-4c85-8da0-5e39c1393a8a","reportName":"Application Flows","sourcetype":"forensics","type":"Graph"}
 
  {"action":"get","device":["172.21.40.199","172.21.21.254"],"myid":"report_1578915304695_4be2423e-d5cf-4c85-8da0-5e39c1393a8a","reportName":"Application Flows","sourcetype":"forensics","type":"Graph"}
 +
 +
== API Response ==
 +
When you make a request for forensics, normally the request session is held open until the data is available and returned in the response to the request.
 +
If you want to fire of a request but not wait for the results, you can add the "async" parameter to the request.
 +
This will run the query but rather than wait and return the data, it will immediately just return the '''myid''' value for that request which can be used later to retrieve the data.
  
  

Revision as of 13:03, 13 January 2020

GigaFlows forensics API can be queried using a rest/json based HTTP/HTTPS request.

API Endpoint

Requests should be sent to the api endpoint using the following URL convention:

http(s)://SERVER_IP:SERVER_PORT/rest/


If you aren’t making requests from a logged in web session, you can either supply your user credentials in the URL or use the "portalUser" functionality This allows requests from a predefined IP address (or regexp) and userid.

Failure in supplying suitable user credentials will result in the following message being returned

{"errors":"notLoggedIn","returnCode":"error"} 

Request using user credentials

http://172.21.21.1:7902/rest/?userName=reportUser&userPass=reportuser254

Request using user Portal

http://172.21.21.1:7902/rest/?portalUser=admin

API Request

Gigaflow expects a post containing a single JSON object under the jsonRequest input form data field, e.g.

jsonRequest={"action":"get",
"type":"Graph",
"sourcetype":"forensics",
"device":["172.21.40.199","172.21.21.254"],
"starttime":1525086647000,
"endtime":1525093847000,
"reportName":"Application Flows"
}

or via URL params

http://172.21.21.1:7902/rest/?jsonRequest={"action":"get","type":"Graph","sourcetype":"forensics","starttime":1525086647000,"endtime":1525093847000,
"device":["172.21.40.199","172.21.21.254"],"reportName":"Application Flows"}

The various (case sensitive) fields in this example are:

Param Meaning
type graph or table depending on whether you want report results grouped by time or as a single table output.
sourcetype Should always be forensics for accessing the forensics results
reportName The name of the report you want to query, this will determine the fields and grouping of data returned. In this example, the “Application Flows” report is requested. A complete list of available reports and their configuration are available from within GigaFlow via Configuration->Reporting->Forensics Reports. From there you can select the report and see which fields are included and what grouping occurs. You can also clone existing reports to create new reports.
starttime This is the start time for the data to be included in the report. This can either be in exact UTC milliseconds or a period relative to now e.g. "4 hours"
endtime This is the end time for the data to be included in the report. This can either be in exact UTC milliseconds or a period relative to now e.g. "2 hours"
device This should be an array [] containing the IP address of the infrastructure devices you want to have data processed from and included in this report. In this example data from both “172.21.40.199” and “172.21.21.254” will be included in this report. Where possible you should always specify the minimum number of devices required for the report. Excluding the device param will run the report over all devices in the database.

When requesting a report, you can also supply a myid parameter, the will be attached to the query and the results so that they can be retrieved at a later date. It is highly recommended that if you specify this field, make sure that the ids are always unique

http://172.21.21.1:7902/rest/?jsonRequest={"action":"get","type":"Graph","sourcetype":"forensics","starttime":1525086647000,"endtime":1525093847000,
"device":["172.21.40.199","172.21.21.254"],"reportName":"Application Flows","myid":"TestMYID 123456"}

If you don't specify a myid field, one will automatically be populated in the format of "report_TIMESTAMP_UUID".

{"action":"get","device":["172.21.40.199","172.21.21.254"],"myid":"report_1578915304695_4be2423e-d5cf-4c85-8da0-5e39c1393a8a","reportName":"Application Flows","sourcetype":"forensics","type":"Graph"}

API Response

When you make a request for forensics, normally the request session is held open until the data is available and returned in the response to the request. If you want to fire of a request but not wait for the results, you can add the "async" parameter to the request. This will run the query but rather than wait and return the data, it will immediately just return the myid value for that request which can be used later to retrieve the data.


Simple reports available for forensics reporting:

"Address Pairs"
"Addresses By Dest"
"Addresses By Source"
"All Fields"
"Application Flows"
"Applications"
"ASs By Dest"
"ASs By Source"
"ASs Pairs"
"Class of Service"
"FW Event"
"FW Ext Code"
"Interface Pairs"
"Interfaces By Dest"
"Interfaces By Source"
"MAC Address Pairs"
"MAC Addresses By Dest"
"MAC Addresses By Source"
"Ports By Dest"
"Ports By Source"
"Posture"
"Protocols"
"Sessions Flows"
"Sessions With Ints"
"Sessions"
"Subnet Class A By Dest"
"Subnet Class A By Source"
"Subnet Class B By Dest"
"Subnet Class B By Source"
"Subnet Class C By Dest"
"Subnet Class C By Source"
"TCP Flags"
"URLs"
"Users"

Enhanced reports available:

"Interfaces By Source Pct" 
"Interfaces By Destination Pct"
"Applications With Flow Count" 
"Application Flows With User"
"All Fields Max FPR" 
"All Fields Avg FPR"
"Duration Max"
"Duration Avg" 
"Ports As Dest By Dest Add Popularity"
"Addresses As Sources And Dest Port By Dest Count"
"Ports As Dest By Src Add Popularity"
"Ports As Src By Dest Add Popularity"
"Ports As Src By Src Add Popularity"
"Servers As Src Address"
"Servers As Dst Address"
"Servers As Src With Ports"
"Servers As Dest With Ports" 

Key fields for each report:

"Application Flows"                                      srcadd, dstadd, appid
"Address Pairs"                                          srcadd, dstadd
"MAC Address Pairs"                                      srcmac, dstmac
"Interfaces By Source"                                   device, device||'_'||inif as difin
"Interfaces By Dest"                                     device, device||'_'||outif as difout
"Interface Pairs"                                        device, device||'_'||inif as difin, device||'_'||outif as difout
"Sessions"	       	                                  srcadd, srcport, dstadd, dstport, appid
"Sessions Flows"                                         firstseen, srcadd, srcport, dstadd, dstport, appid, proto
"Sessions With Ints"	                                  srcadd, srcport, inif, dstadd, dstport, outif, appid
"Protocols"	                                          proto
"Posture"	      	                                  postureid
"Applications"	   	                                  appid
"URLs"	             	                                  url
"Ports By Source"	                            	  srcport
"Ports By Dest"                                          dstport
"TCP Flags"	      	                                  flags
"FW Event"	                                          fwevent
"FW Ext Code"	                                          fwextcode
"Class of Service"	                                  tos
"ASs Pairs"	                                          srcas, dstas
"ASs By Source"                                          srcas
"ASs By Dest"	                                          dstas
"Users"                                                  userid
"MAC Addresses By Source"                                srcmac
"MAC Addresses By Dest"                      	          dstmac
"Addresses By Source"	                                  srcadd
"Subnet Class A By Dest"                      	          dstadd-modulus(dstadd,16777216) as dstsubneta
"Subnet Class B By Dest"                      	          dstadd-modulus(dstadd,65536) as dstsubnetb
"Subnet Class C By Dest"                      	          dstadd-modulus(dstadd,256) as dstsubnetc
"Subnet Class A By Source"                      	  srcadd-modulus(srcadd,16777216) as srcsubneta
"Subnet Class B By Source"                      	  srcadd-modulus(srcadd,65536) as srcsubnetb
"Subnet Class C By Source"	                          srcadd-modulus(srcadd,256) as srcsubnetc
"Addresses By Dest"	                                  dstadd
"Interfaces By Source Pct"                               {"difin"}, "pct_avg_in", "afirstseen"
"Interfaces By Destination                               {"difout"}, "pct_avg_out", "afirstseen"
"Applications With Flow Count"                           {"appid"}, "flowcount", "afirstseen"
"Application Flows With User"                            {"srcadd", "dstadd", "appid", "userid"}, "bits_avgsec", "afirstseen"
"All Fields"                                             {}, "bits_avgsec", "afirstseen"
"All ID Fields"                                          new int[]{reportDomainForensics}, ROS.forensicsMinDuration, reportTypeForensics
"All Fields Max FPR"                                     {}, "maxfpr", "afirstseen"
"All Fields Avg FPR"                                     {}, "maxfpr", "afirstseen"
"Duration Max"                                           {}, "maxduration", "afirstseen"
"Duration Avg"                                           {}, "avgduration", "afirstseen"
"Ports As Dest By Dest Add Popularity"                   {"dstport"}, "dstcount", "afirstseen"
"Addresses As Sources And Dest Port By Dest Count"       {"srcadd", "dstport"}, "dstcount", "afirstseen"
"Ports As Dest By Src Add Popularity"                    {"dstport"}, "srccount", "afirstseen"
"Ports As Src By Dest Add Popularity"                    {"srcport"}, "dstcount", "afirstseen"
"Ports As Src By Src Add Popularity"                     {"srcport"}, "srccount", "afirstseen"
"Servers As Src Address"                                 {"srcadd"}, "bits_avgsec", "afirstseen"
"Servers As Dst Address"                                 {"dstadd"}, "bits_avgsec", "afirstseen"
"Servers As Src With Ports"                              {"srcadd", "srcport"}, "dstsavgsec", "afirstseen"
"Servers As Dest With Ports"                             {"dstadd", "dstport"}, "srcsavgsec", "afirstseen"

Fields available:

Application ID as selected after profiling:appid
Device ID:device
Device And Input Interface ID:device||'_'||inif as difin
Device And Output Interface ID:device||'_'||outif as difout
Destination IP Address:dstadd
Destination Class A Subnet:dstadd-modulus(dstadd,16777216) as dstsubneta
Destination Class C Subnet:dstadd-modulus(dstadd,256) as dstsubnetc
Destination Class B Subnet:dstadd-modulus(dstadd,65536) as dstsubnetb
Destination AS Number:dstas
Destination MAC Address:dstmac
Destination IP Port Number:dstport
MS Timestamp Of When Flow Record Was First Seen:firstseen
TCP Flags:flags
Forwarding Event Code:fwevent
First Packet Response:fpr
Extended Event Code:fwextcode
Input Interface ID:inif
Output Interface ID:outif
Posutre ID Of Flow As Marked After Processing:postureid
IP Protocol Nunmber:proto
Source IP Address: srcadd
Source Class A Subnet:srcadd-modulus(srcadd,16777216) as srcsubneta
Source Class C Subnet:srcadd-modulus(srcadd,256) as srcsubnetc
Source Class B Subnet:srcadd-modulus(srcadd,65536) as srcsubnetb
Source AS number:srcas
Source MAC Address:srcmac
Source IP Port:srcport
TOS/QOS Marking:tos
Text Field:url
User ID:userid