Add new regex natives. by Drifter321 · Pull Request #767 · alliedmodders/sourcemod

Drifter321 · 2018-02-13T00:26:25Z

Not sure how long this has been broken but currently MatchRegex doesn't actually return the number of matches. It instead returns 1 + the number of capture group matches. This simply fixes it to work how the docs currently say it should work. However, it is possible to add capture group support at some point but would require some effort along with new natives.

Some test outputs.

Matches 5
Match 0 is t
Match 1 is t
Match 2 is t
Match 3 is t
Match 4 is t

Matches 1
Match 0 is _

Matches 3
Match 0 is .
Match 1 is _
Match 2 is +

Matches 1
Match 0 is test

Test plugin

#pragma newdecls required
 
#include <sourcemod>
#include <regex>
 
public void OnPluginStart()
{
	Regex regex = CompileRegex("t", PCRE_UTF8);
	int filter = MatchRegex(regex, "test.string_doesnt+match");
	
	PrintToServer("Matches %i", filter)
	
	for(int i = 0; i < filter; i++)
	{
		char reply[100];
		
		GetRegexSubString(regex, i, reply, sizeof(reply));
		
		PrintToServer("Match %i is %s", i, reply);
	}
	
	Regex regex2 = CompileRegex("_", PCRE_UTF8);
	filter = MatchRegex(regex2, "test.string_doesnt+match");
	
	PrintToServer("\nMatches %i", filter)
	
	for(int i = 0; i < filter; i++)
	{
		char reply[100];
		
		GetRegexSubString(regex2, i, reply, sizeof(reply));
		
		PrintToServer("Match %i is %s", i, reply);
	}
	
	Regex regex3 = CompileRegex("[\\\\\\W|^_]", PCRE_UTF8);
	filter = MatchRegex(regex3, "test.string_doesnt+match");
	
	PrintToServer("\nMatches %i", filter)
	
	for(int i = 0; i < filter; i++)
	{
		char reply[100];
		
		GetRegexSubString(regex3, i, reply, sizeof(reply));
		
		PrintToServer("Match %i is %s", i, reply);
	}
	
	Regex regex4 = CompileRegex("tes(t)", PCRE_UTF8);
	filter = MatchRegex(regex4, "test.string_doesnt+match");
	
	PrintToServer("\nMatches %i", filter)
	
	for(int i = 0; i < filter; i++)
	{
		char reply[100];
		
		GetRegexSubString(regex4, i, reply, sizeof(reply));
		
		PrintToServer("Match %i is %s", i, reply);
	}
}

I'm not exactly sure if this is the best approach. But I am open to all feedback.

KyleSanderson · 2018-02-13T00:34:17Z

This is tough to change because a for loop would have to be fixed at the plugin level. Fundamentally, even at a base level you're absolutely correct this is wrong. But people using < length would have to be fixed to <=. I'm sure there are other cases, but this to me is technical debt that can't be changed at this point.

asherkin · 2018-02-13T07:38:49Z

My understanding is still that this is currently working exactly as expected and documented, the documentation just confuses people more familiar with modern regex implementations than Perl.

Global matching behaviour should be behind a new native, maybe MatchAll?

…ffset to MatchRegex. Clarify documentaion.

…x.MatchOffset.

Drifter321 · 2018-02-13T18:29:34Z

After additional discussion it appears it was working correctly and the documentation was leading to lots of confusion and various features lacking.

MatchRegex has been reverted to how it originally was, but will now also accept an additional param for offset (defaulting to 0). GetRegexSubString now also takes an additional param to identify the match (defaulting to 0) to get substrings for.

MatchAll has been added. MatchCount, MatchOffset (Offset in the test plugin since i decided to rename it after), CaptureCount have also been added to provide info when using MatchAll, or when trying to manually loop using MatchRegex.

Test Plugin
https://paste.ee/p/CdFjB

And output
https://paste.ee/p/r0i44

I tried my best to clarify the documentation, but its probably still not great...

asherkin

Functionality seems good, see inlines.

asherkin · 2018-02-13T20:29:14Z

extensions/regex/CRegEx.cpp

-	rc = pcre_exec(re, NULL, subject, (int)strlen(subject), 0, 0, ovector, 30);
+	unsigned int len = strlen(subject);
+
+	rc = pcre_exec(re, NULL, subject, len, offset, 0, mMatches[0].mVector, sizeof(mMatches[0].mVector));


This sizeof is incorrect, it needs to be the element count, not the byte size.

The PRCE documentation points this out about 100 times 😛

asherkin · 2018-02-13T20:29:40Z

extensions/regex/CRegEx.cpp

+	unsigned int len = strlen(subject);
+	unsigned int matches = 0;
+
+	while (offset < len && (rc = pcre_exec(re, 0, subject, len, offset, 0, mMatches[matches].mVector, sizeof(mMatches[matches].mVector))) >= 0)


asherkin · 2018-02-13T20:35:50Z

extensions/regex/CRegEx.h

+struct RegexMatch
+{
+	int mSubStringCount;
+	int mVector[30];


Let's double this, 10 captures seems limiting, we have the RAM.

…er of matches and captures.

asherkin · 2018-02-14T09:29:52Z

plugins/include/regex.inc

 */
+native bool GetRegexSubString(Handle regex, int str_id, char[] buffer, int maxlen, int match = 0);
+
 stock int SimpleRegexMatch(const char[] str, const char[] pattern, int flags = 0, char[] error="", int maxLen = 0)


It looks like the documentation got deleted for SimpleRegexMatch?

Oops, good catch must have deleted it when i deleted the new natives from non-methodmap.

…natives.

Fix regex not getting all matches correctly.

31b4799

Drifter321 added 2 commits February 13, 2018 13:15

Add additional natives for regex extension. Add support for passing o…

ea8bf18

…ffset to MatchRegex. Clarify documentaion.

Remove non-methodmap natives. Change the name of Regex.Offset to Rege…

95007a7

…x.MatchOffset.

Drifter321 added 2 commits February 13, 2018 13:47

Actually remove non-methodmap natives. And fix Regex.MatchOffset name.

284bf49

Remove MatchRegexAll non-methodmap naitve.

37c78dc

asherkin requested changes Feb 13, 2018

View reviewed changes

Fix build and fix not passing the number of elements. Double the numb…

e3b5150

…er of matches and captures.

asherkin approved these changes Feb 13, 2018

View reviewed changes

Make matches a signed int.

0e52c11

asherkin reviewed Feb 14, 2018

View reviewed changes

Drifter321 added 2 commits February 14, 2018 09:02

Add SimpleRegexMatch documentation back.

d77222d

Clarify docuemtnation slightly. Remove new params from non-methodmap …

e769f66

…natives.

Drifter321 changed the title ~~Fix regex not getting all matches correctly.~~ Add new regex natives. Feb 15, 2018

Drifter321 merged commit 5ac3390 into master Feb 15, 2018

Drifter321 added a commit that referenced this pull request May 28, 2018

Fix/Update regex extension and docs (#775, #767)

cdb9851

Headline deleted the regex-fix branch July 31, 2018 21:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add new regex natives.#767

Add new regex natives.#767
Drifter321 merged 9 commits intomasterfrom
regex-fix

Drifter321 commented Feb 13, 2018

Uh oh!

KyleSanderson commented Feb 13, 2018

Uh oh!

asherkin commented Feb 13, 2018

Uh oh!

Drifter321 commented Feb 13, 2018

Uh oh!

asherkin left a comment

Uh oh!

asherkin Feb 13, 2018

Uh oh!

asherkin Feb 13, 2018

Uh oh!

asherkin Feb 13, 2018

Uh oh!

asherkin Feb 14, 2018

Uh oh!

Drifter321 Feb 14, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

Drifter321 commented Feb 13, 2018

Uh oh!

KyleSanderson commented Feb 13, 2018

Uh oh!

asherkin commented Feb 13, 2018

Uh oh!

Drifter321 commented Feb 13, 2018

Uh oh!

asherkin left a comment

Choose a reason for hiding this comment

Uh oh!

asherkin Feb 13, 2018

Choose a reason for hiding this comment

Uh oh!

asherkin Feb 13, 2018

Choose a reason for hiding this comment

Uh oh!

asherkin Feb 13, 2018

Choose a reason for hiding this comment

Uh oh!

asherkin Feb 14, 2018

Choose a reason for hiding this comment

Uh oh!

Drifter321 Feb 14, 2018

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants