countof_regex

This page explains how to use the countof_regex function in APL.

The countof_regex function counts occurrences of a regular expression pattern within a string. Use this function when you need to count complex patterns or character classes in log messages, requiring more flexibility than simple substring matching.

For users of other query languages

If you come from other query languages, this section explains how to adjust your existing queries to achieve the same results in APL.

Splunk SPL users

In Splunk SPL, you use rex with max_match to count regex matches. APL’s countof_regex provides a more straightforward approach.

Splunk example

| rex field=message max_match=0 "error|warning"
| eval pattern_count=mvcount(rex)

['sample-http-logs']
| extend pattern_count = countof_regex('error|warning', uri)

ANSI SQL users

In ANSI SQL, counting regex matches typically requires database-specific functions. APL’s countof_regex provides a standard approach.

SQL example

SELECT REGEXP_COUNT(field, 'pattern') AS count FROM logs;

['sample-http-logs']
| extend count = countof_regex('pattern', field)

Usage

Syntax

countof_regex(regex, text)

Parameters

Name	Type	Required	Description
regex	string	Yes	The regular expression pattern to search for within the text.
text	string	Yes	The source string where pattern occurrences are counted.

Returns

Returns the number of times the regex pattern matches in the text.

Use case examples

Log analysis

Count numeric patterns in URIs to identify parameterized endpoint usage.

Query

['sample-http-logs']
| extend numeric_params = countof_regex('[0-9]+', uri)
| where numeric_params > 0
| summarize avg_params = avg(numeric_params), request_count = count() by method
| sort by request_count desc

Run in Playground

Output

method	avg_params	request_count
GET	1.8	3421
POST	1.2	1876
PUT	2.1	654
DELETE	1.5	234

This query counts numeric parameters in request URIs using regex, helping identify how frequently parameterized endpoints are accessed by different HTTP methods.

OpenTelemetry traces

Count specific character patterns in trace IDs to analyze ID generation patterns.

Query

['otel-demo-traces']
| extend hex_chars = countof_regex('[a-f]', trace_id)
| summarize avg_hex_chars = avg(hex_chars), trace_count = count() by ['service.name']
| sort by trace_count desc
| limit 10

Run in Playground

Output

service.name	avg_hex_chars	trace_count
frontend	8.3	2345
checkout	8.1	1987
cart	8.5	1654
product-catalog	7.9	1234

This query counts hexadecimal characters (a-f) in trace IDs to analyze the distribution of characters, which can help identify issues with trace ID generation.

Security logs

Identify requests with multiple special characters that might indicate injection attacks.

Query

['sample-http-logs']
| extend special_chars = countof_regex('[<>%;()&+]', uri)
| where special_chars >= 3
| project _time, uri, special_chars, id, status, method
| sort by special_chars desc
| limit 10

Run in Playground

Output

_time	uri	special_chars	id	status	method
2024-11-06T10:00:00Z	/search?q=	8	user123	403	GET
2024-11-06T10:01:00Z	/api?param=‘OR’1’=‘1	6	user456	403	POST

This query counts special characters commonly used in injection attacks, helping identify potentially malicious requests that warrant further investigation.

List of related functions

• countof: Counts plain substring occurrences. Use this when you need exact string matching without regex complexity.
• extract: Extracts the first substring matching a regex. Use this when you need to capture the matched text, not just count occurrences.
• extract_all: Extracts all substrings matching a regex. Use this when you need both the count and the actual matched values.
• replace_regex: Replaces all regex matches with another string. Use this when you need to modify matched patterns rather than count them.