Parser

Current version v0.4.0

The Parser gives structure to and divides a JSON, Key Value, CSV, Delimited Values or Fixed Length file into separate fields.

Find Parser in the Actions tab and drag it onto the canvas to use it.

To open the configuration, click the Action in the canvas and select Configuration.

In order to configure this action, you must first link it to a Listener. Go to Building a Pipeline to learn how to link.

First, choose the field to parse from the Listener by typing it in the search bar or selecting it from the list.

You can also use the arrow keys on your keyboard to navigate up and down the list.

The next steps are to make further specifications on this field to parse.

Configuration

Input

This is where you specify how to read the incoming data:

real_data - this is the data taken directly from the linked Listener.
paste - there may be times that you will receive a JSON with updated data for the Listener. If this is the case, you can paste it here.

In the Events drop-down, you can write how many events to show in this window.

Now you have specified where to source the data from, you need to determine how to process the events.

Parser

There are two ways to parse your data:

auto - automatically parses all key-value fields.
manual - manually split fields and rename them.

Here you can view the fields as a list, or as code. Each data type has a color. Learn more here.

The language and grammar used to parse is PCL (Parser Configuration Language). We have provided an extensive run-down of each command in this article.

Now we have decided which field, from where, and how to parse, we need to specify the interpretation of the output to the next action.

Output

Here you can see the raw message that has been generated.

Below, each individual field is color-coded according to the legend and separated into its type and name.

Here you can change the data type and edit the field name.

For fields containing subfields (field.subfield), changing the field name will change all of the prefixes.

When you are happy with your event, select Save.

Example

Learn how to use the Parser with this example.

We have opened the configuration of the Parser to our Pipeline, which is receiving data from the linked Listener.

First we must select the field to parse from the Listener to separate into more specific data. This is the field containing the raw data.

In the Input field, select Real data if you feel comfortable with how the Parser works.

Select Paste and paste this log to follow along with us for the various examples of log sources.

A CSV file using "," as a separator

Log to paste

2024-10-02T14:22:03Z,DESKTOP-1234,FileExecution,notepad.exe,4321,JohnDoe,192.168.1.2,a3b5c2d4e6f7,35

Select Auto to automatically parser this data into separate fields.

The parser will automatically parse the log, having recognized the separator. The default values will be fieldName1,2,3 etc.

Now we have decided which field, from where, and how to parse, we need to specify how it is output to the next action. In the output field, we can change the names of each field.

TIMESTAMP
HOSTNAME
EVENT TYPE
PROCESS NAME
PROCESS ID
USERNAME
IP ADDRESS
HASH
THREAT SCORE

A JSON file.

Log:

{"timestamp":"2024-10-02T14:45:15Z","client_ip":"203.0.113.45","http_method":"GET","uri":"/login","response_code":403,"action":"BLOCK","rule_id":"981176","rule_description":"SQL Injection Attempt","user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36","headers":{"host":"example.com","referer":"","x_forwarded_for":"203.0.113.45"},"request_body":"","threat_details":{"attack_type":"SQL Injection","payload":"' OR 1=1 --"}}

Select Auto to automatically parser this data into separate fields.

The parser will automatically parse the log.

Now we have decided which field, from where, and how to parse, we need to specify how it is output to the next action. In the output field, we can change the names of each field.

TIMESTAMP
CLIENT IP
HTTP METHOD
URI
RESPONSE CODE
ACTION
RULE ID
RULE DESCRIPTION
USER AGENT
HEADERS
REQUEST BODY
THREAT DETAILS

Use a key-value file with = to as a separator.

Log to paste

timestamp=2024-10-02T15:00:45Z src_ip=192.168.1.100 dst_ip=203.0.113.45 action=ALLOW protocol=TCP src_port=443 dst_port=54321 bytes_sent=1024 bytes_received=2048 rule_id=1002 threat_level=low

Select Auto to automatically parser this data into separate fields.

The parser will automatically parse the log, having recognized the separator.

Now we have decided which field, from where, and how to parse, we need to specify how it is output to the next action. In the output field, we can change the names of each field.