Read about how you can use one Pipeline (this case on Firewall Threat Reduction) to carry out multiple use cases and data solutions.
For the first use case, we will reduce the firewall.paloalto.threat data and send on the results.
We have an automated parser that uses machine learning to take a look at the firewall logs, automatically splitting it out into strings. Although the parser provides automated parsing, you can modify this to extract certain fields, split fields, change type etc.
Use the message builder to remove redundant information and take only certain logs from the parser.
Send the data over to a Syslog data hub, reducing significantly the data usage.
A second use case would be to enrich your data with IPs coming from an external database.
The Lookup action will pull enrichments uploaded to Onum containing IP addresses potentially exposed to threats.
This creates a new field, enriching the data with exposure database.
From here, you can narrow down the data using the is not null condition to discard null fields and only send on data when there is a match.
The Twilio sink can be used to send an SMS to Twilio if there is an exposure level on the given IP.
At this point we have reduced data to send on for storage and archiving and enriched in real-time using a conditional for matches.
You can use Onum to analyze your data in real-time on the platform by grouping and aggregating it.
Group by IPs every five minutes and count how many times it contains the message ID field.
This aggregated data can then be sent on to Splunk for further analysis.
Data reduction filters out the noise to leave and send on the relevant data, massively reducing your logs.
Enrichment uses a database with an added level of complexity by filtering out null values, alerting if exposure is detected by sending an SMS to signal the need for further inspection.
Real-time analysis involves grouping and sending data over to Splunk for quick processing there.
