1. Introduction
Splunk Search Processing Language (SPL) is a query language designed by Splunk which provides search commands with associated functions, arguments, and clauses to search, filter, modify, manipulate, insert, and delete data. The SPL syntax is similar to UNIX pipeline and SQL.
In this example, I will show how to use SPL to:
- Search raw events with search terms
- Transform search results with commands
2. Technologies Used
The example in this article was built and run using:
- Docker 19.03.8
- Splunk 8.1.1
- Google Chrome 87.0.4280.88
Click my other article to install Splunk. Click here to configure your SPL Editor preference.
3. Search Raw Events
We search raw events based on search terms. Search terms can be keywords, phrases, key-value pairs or any logic combinations of multiple search terms.
Please click my other article on how to filter raw events based on the time range picker. In this step, I will use the queries in the sample reports to explain the search language.
Under the Reports section, click the “Open in Search” button to view the SPL query.
3.1 Keyword Search
Click the “Open in Search” button next to the “Errors in the last 24 hours” report. There are several keywords and key-value pairs used in search terms.
keyword search
error OR failed OR severe OR ( sourcetype=access_* ( 404 OR 500 OR 503 ) )
It searches for any raw events which contain any of the keywords: error, failed, or severe-plus any raw events from sourcetype=access_* and contains any keywords of 404, 500, or 503.
Note:
- key=value pair – the key is case-sensitive and value is case-insensitive.
- The Boolean OR – is a logic OR operator.
- () – is used to group search terms.
- * – is for wildcard characters.
3.2 Phrase Search
Click the “Open in Search” button next to the “Splunk error in last 24 hours” report.
It searches any raw events from index=_internal and contain the “ error “ phrase and NOT contain the debug keyword within source=*splunkd.log*.
phrase search
index=_internal " error " NOT debug source=*splunkd.log*
Note: the search phrase is surrounded with quotes.
4. Transform Search
SPL provides commands to transform the search results. In this step, I will show a few common commands.
4.1 timechart
Click to view the search query in the “Messages by minute last 3 hours” report. It uses the timechart command to display the search results based on the sum of events.
timechart
index=_internal source="*metrics.log" eps "group=per_source_thruput" NOT filetracker | eval events=eps*kb/kbps | timechart fixedrange=t span=1m limit=5 sum(events) by series
The time chart’s X-axis is time and Y-axis is the sum of events.
4.2 stats
Click to view the SPL query used in the “License Usage Data Cube” report. It uses the stats command to calculate the sum of bytes for license usage data.
stats
index=_internal source=*license_usage.log* type="Usage" | eval h=if(len(h)=0 OR isnull(h),"(SQUASHED)",h) | eval s=if(len(s)=0 OR isnull(s),"(SQUASHED)",s) | eval idx=if(len(idx)=0 OR isnull(idx),"(UNKNOWN)",idx) | bin _time span=1d | stats sum(b) as b by _time, pool, s, st, h, idx
It calculates the sum of bytes.
4.3 transaction
The transaction command groups raw events with the optional start and finish conditions.
In this step, I will use the following SPL query to find the raw events which start with STAGE_1 and end with STAGE_5 and group by objid. It uses keepevicted option to find any raw events which did not complete successfully.
transaction
source="GCR_reschedule.txt" host="mary" sourcetype="log4j" "*GCR_NOTIFICATION_PROCESSING*" | transaction objid startswith="STAGE_1" endswith="STAGE_5" keepevicted=true| search closed_txn=0 | highlight objid notifyId
Here is the screenshot of the search results.
5. Summary
SPL has over 100 commands and functions. In this article, I demonstrated a few of them. Please visit the Splunk documentation for more details.
6. Download the Source Code
You can download the SPL source code of this example here: Splunk Search Language Example
