Updated on

Source-generated RegEx moves the work of building a regular expression from run time to build time. We mark a partial method with the [GeneratedRegex] attribute, and the .NET SDK writes the matching code for us as plain C#.

In our benchmark on .NET 10, it validated a password 57 times faster than creating a new Regex on every call, and in about 20 percent less time than a cached RegexOptions.Compiled instance.

To download the source code for this article, you can visit our GitHub repository.

Let’s dive in!

How Does RegEx Work?

Regular expressions are vital to the programming world, but do we know how they work in .NET?

Let’s examine the example:

using System.Text.RegularExpressions;

public static class PasswordValidator
{
    public static bool ValidatePasswordWithRegularRegEx(string password)
    {
        var regex = new Regex(@"^(?=.*[a-z])(?=.*[A-Z])(?=.*\d)(?=.*[^\da-zA-Z]).{8,}$");

        return regex.IsMatch(password);
    }
}

Inside the ValidatePasswordWithRegularRegEx() method, we create a new instance of the Regex class, by passing a pattern to the constructor. We can use this pattern to validate if a password is at least 8 characters long. It also checks whether the password has at least one: lower-case letter, an upper-case letter, a digit, and a special character.  To check what each part of a pattern like this matches, the .NET regex cheat sheet lists lookaheads and character classes with the result .NET actually returns for every example.

A few things happen when we use either the constructor or one of the static methods on the Regex class. First, the Regex class parses the pattern we pass to ensure its validity. Then it turns the pattern into a node tree representation of that pattern. Next, the tree is transformed into a set of instructions that the internal RegexInterpreter engine can interpret. Finally, when we try to match something against that pattern, the internal regular expressions interpreter goes over those instructions and compares them to the input. Our method creates a new instance on every call, so it repeats all of this work for every password it checks.

If you want to know more about the basics of regular expressions, pay a visit to our article Introduction to Regular Expressions in C#.

How Does Compiled RegEx Work?

With .NET, we have the option to use a compiled RegEx as well:

public static bool ValidatePasswordWithCompiledRegEx(string password)
    => Regex.IsMatch(
        password,
        @"^(?=.*[a-z])(?=.*[A-Z])(?=.*\d)(?=.*[^\da-zA-Z]).{8,}$",
        RegexOptions.Compiled);

We create ValidatePasswordWithCompiledRegEx() method in our PasswordValidator class and use the static IsMatch() method to return a result. To the method, we pass the password we wish to validate, the pattern, and using the RegexOptions.Compiled enumeration value, we specified that our application must use compiled regular expressions.

When we do this, all of the steps up to the generation of instructions for the RegexInterpreter engine will be the same. But then, a compiler built on reflection emit will further process those instructions and write them as IL into several DynamicMethod instances. So, when we try to perform a match, the Regex class won’t use the interpreter but will execute those DynamicMethod instances. This makes matching input faster but construction more costly, as we need to perform additional operations, and the new IL still has to be JIT-compiled on its first use.

Our method hides most of that cost because it calls the static IsMatch() method. As Microsoft Learn notes about the Regex cache, “Only regular expressions used in static method calls are cached.” By default, that cache holds 15 instances, so the compiled Regex is built once and reused on every later call.

What Is GeneratedRegex in C#?

GeneratedRegex is an attribute that asks the .NET SDK to write a regular expression for us at build time. We put it on a partial method that returns Regex, and the source generator that ships with the SDK fills in the method body.

The method has to be parameterless and non-generic, and the class around it has to be partial as well. Since .NET 9 and C# 13, the attribute also works on a get-only partial property.

The attribute takes the pattern first. After it, we can pass RegexOptions, a match timeout in milliseconds and a culture name for case-insensitive matching. RegexOptions.Compiled is ignored, because the generated code is compiled together with the rest of our project.

What we get back is a class derived from Regex, written as plain C# for our exact pattern, and a cached instance of it. Every call returns that same instance, so we never pay for parsing or compiling the pattern at run time.

With .NET 7, we got a new way of using regular expressions:

public static partial class PasswordValidator
{   
    public static bool ValidatePasswordWithSourceGeneratedRegEx(string password)
        => PasswordRegEx().IsMatch(password);

    [GeneratedRegex(@"^(?=.*[a-z])(?=.*[A-Z])(?=.*\d)(?=.*[^\da-zA-Z]).{8,}$")]
    private static partial Regex PasswordRegEx();
}

We create the partial PasswordRegEx() method that returns a Regex instance.  Next, we decorate the method with the GeneratedRegex attribute. Then, we also mark our class as partial as well. Finally, we create the ValidatePasswordWithSourceGeneratedRegEx() method and return the result of the IsMatch() method when called on the result, that we get from the PasswordRegEx() method.

When we use the GeneratedRegex attribute on a partial method that returns a Regex instance, the internal source generator recognizes this and provides all the necessary logic behind the scenes.

Since .NET 9, we can put the same attribute on a partial property instead:

[GeneratedRegex(@"^(?=.*[a-z])(?=.*[A-Z])(?=.*\d)(?=.*[^\da-zA-Z]).{8,}$")]
private static partial Regex PasswordRegExProperty { get; }

Both forms produce the same code. Because our two patterns are identical, the generator even returns the same cached instance from both of them.

Microsoft Learn is direct about which form of compilation to reach for: “Where possible, use source-generated regular expressions instead of compiling regular expressions using the RegexOptions.Compiled option.”

If you want to know more about source generators and how they work you can check out our article Source Generators in C#.

How Does Source-Generated RegEx Improve Performance in .NET?

With source-generated code, we get the option to examine it:

/// <summary>Cached, thread-safe singleton instance.</summary>
internal static readonly PasswordRegEx_0 Instance = new();
    
/// <summary>Initializes the instance.</summary>
private PasswordRegEx_0()
{
    base.pattern = "^(?=.*[a-z])(?=.*[A-Z])(?=.*\\d)(?=.*[^\\da-zA-Z]).{8,}$";
    base.roptions = RegexOptions.None;
    ValidateMatchTimeout(Utilities.s_defaultTimeout);
    base.internalMatchTimeout = Utilities.s_defaultTimeout;
    base.factory = new RunnerFactory();
    base.capsize = 1;
}

The first thing that sticks out is the fact that we now get an internal thread-safe instance that is also cached. The source generator doesn’t just initialize a new Regex instance as well. It produces a code that is in many ways similar to the one that the compiler produces when we use RegexOptions.Compiled. We get all the benefits of a compiled regular expression as well as some start-up-related benefits as well. The generator also writes XML comments above our method that explain the pattern step by step, so hovering over PasswordRegEx() in the IDE tells us what it matches.

Moreover, the code is very similar to the DynamicMethod instances produced by the compiler. The compiler is then responsible for translating the generated code to an IL one, which can bring further optimizations and performance improvements. For example, if the generated code produces a switch statement, the compiler then has a myriad of ways to improve it in the produced IL code. This is something we don’t get when we instantiate or use the static Regex methods.

Is Source-Generated RegEx Faster Than Compiled RegEx?

Yes, by a small margin for matching, and by a huge one for construction. In our benchmark on .NET 10.0.10, a source-generated regex checked a password in 59.68 nanoseconds. A cached RegexOptions.Compiled regex took 74.31 nanoseconds, so the generator saved about 15 nanoseconds per match, or 20 percent.

Building a new compiled Regex took 3.2 milliseconds per call, because the pattern is turned into IL and then JIT-compiled at run time. The source generator moves the IL generation to build time, so our running application only has to JIT-compile that code, like the rest of our methods.

Creating a new interpreted Regex for every call took 3,422.60 nanoseconds, and most of that was construction. The static Regex.IsMatch() method, which caches its instance, ran the same interpreter in 478.97 nanoseconds.

So, let’s avoid building a regex on every call and keep the instance we build. When the pattern is known at compile time, we can leave the building to the generator.

Let’s start by installing the BenchmarkDotNet package:

dotnet add package BenchmarkDotNet

To see how much of the cost comes from building a Regex, let’s add two more methods to our PasswordValidator class. The second one builds a new compiled Regex on every call, which is the mistake the static cache usually hides from us, so it belongs in a benchmark and never in real code:

public static bool ValidatePasswordWithStaticRegEx(string password)
    => Regex.IsMatch(
        password,
        @"^(?=.*[a-z])(?=.*[A-Z])(?=.*\d)(?=.*[^\da-zA-Z]).{8,}$");

public static bool ValidatePasswordWithNewCompiledRegEx(string password)
{
    var regex = new Regex(
        @"^(?=.*[a-z])(?=.*[A-Z])(?=.*\d)(?=.*[^\da-zA-Z]).{8,}$",
        RegexOptions.Compiled);

    return regex.IsMatch(password);
}

The first one uses the same interpreter as our baseline, but gets its instance from the static cache. The second one pays the full cost of compiling the pattern every time we call it.

Then, let’s create our benchmarking class:

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Columns;
using BenchmarkDotNet.Configs;
using BenchmarkDotNet.Reports;

[MemoryDiagnoser(true)]
[Config(typeof(StyleConfig))]
public class RegexBenchmarks
{
    private const string Password = "c0d3-MaZ3-Pa55w0rd";

    [Benchmark(Baseline = true)]
    public bool RegularRegex()
        => PasswordValidator.ValidatePasswordWithRegularRegEx(Password);

    [Benchmark]
    public bool StaticRegex()
        => PasswordValidator.ValidatePasswordWithStaticRegEx(Password);

    [Benchmark]
    public bool CompiledRegex()
        => PasswordValidator.ValidatePasswordWithCompiledRegEx(Password);

    [Benchmark]
    public bool NewCompiledRegex()
        => PasswordValidator.ValidatePasswordWithNewCompiledRegEx(Password);

    [Benchmark]
    public bool SourceGeneratedRegex()
        => PasswordValidator.ValidatePasswordWithSourceGeneratedRegEx(Password);

    private class StyleConfig : ManualConfig
    {
        public StyleConfig()
            => SummaryStyle = SummaryStyle.Default.WithRatioStyle(RatioStyle.Trend);
    }
}

First, we create the RegexBenchmarks class. Then, we decorate it with the MemoryDiagnoser and Config attributes. With the former one we will measure memory allocation, and the latter one will write the ratio of improvement compared to the RegularRegex() use as “x faster”, thanks to the baseline styling we set in StyleConfig.

Every benchmark returns the result of IsMatch(). BenchmarkDotNet recommends this, because the JIT can drop a calculation whose result we throw away.

One final thing left to do:

using BenchmarkDotNet.Running;

BenchmarkRunner.Run<RegexBenchmarks>();

In our Program.cs file, we pass the benchmarking class to BenchmarkRunner, and we run the application in Release mode with dotnet run -c Release.

For a detailed look at using BenchmarkDotNet, check out our article Introduction to Benchmarking in C# and ASP.NET Core Projects.

Next, let’s run the benchmark:

| Method               | Mean            | Error         | StdDev        | Median          | Ratio          | RatioSD | Gen0   | Gen1   | Allocated | Alloc Ratio |
|--------------------- |----------------:|--------------:|--------------:|----------------:|---------------:|--------:|-------:|-------:|----------:|------------:|
| RegularRegex         |     3,422.60 ns |     67.372 ns |    108.794 ns |     3,410.43 ns |       baseline |         | 0.7515 | 0.0114 |    6304 B |             |
| StaticRegex          |       478.97 ns |      9.206 ns |     19.816 ns |       473.75 ns |   7.16x faster |   0.36x |      - |      - |         - |          NA |
| CompiledRegex        |        74.31 ns |      1.511 ns |      3.649 ns |        73.08 ns |  46.16x faster |   2.60x |      - |      - |         - |          NA |
| NewCompiledRegex     | 3,224,361.75 ns | 48,015.276 ns | 42,564.291 ns | 3,209,500.39 ns | 942.99x slower |  31.64x |      - |      - |   18816 B |  2.98x more |
| SourceGeneratedRegex |        59.68 ns |      0.549 ns |      0.487 ns |        59.56 ns |  57.35x faster |   1.85x |      - |      - |         - |          NA |

When we look at the results, we see that the method using a new Regex instance has an execution time of 3,422.60 nanoseconds, which is also our baseline. It allocates 6304 bytes of memory on every call.

The StaticRegex() method runs the same interpreter in 478.97 nanoseconds, without allocating anything. The only difference is that it gets its instance from the cache. So, most of our baseline’s time goes into building the Regex for every password.

Next, we have the compiled RegEx method with a run time of 74.31 nanoseconds, which is 46.16 times faster than our baseline method, with no memory allocation. Against the cached interpreter, compiling buys us a match that is about 6 times faster.

The NewCompiledRegex() method shows the price of that speed. Building a compiled Regex on every call takes 3.2 milliseconds, which makes it 943 times slower than our baseline.

Finally, we have the source-generated one, which is about 15 nanoseconds faster than the compiled one. With it, we see a RegEx performance increase 57 times over our baseline, and it never pays the construction cost that the compiled version pays at run time.

We ran the benchmark on an AMD Ryzen 5 3600 with Windows 10, .NET SDK 10.0.302 and .NET 10.0.10, using BenchmarkDotNet 0.15.8. On different hardware the absolute numbers will change, so the ratios are the part to compare.

When Should We Not Use Source-Generated RegEx?

The source generator needs the pattern at compile time. When the pattern comes from user input, a configuration file or a database, we’re back to the Regex constructor. The best we can do then is build the instance once and keep it, in a static field or through the static methods’ cache.

A few patterns don’t get custom code either. With RegexOptions.NonBacktracking, or with case-insensitive backreferences, the generator falls back to caching a regular Regex instance. We still get the caching and the XML comments, just not the faster matching code.

The generator also only supports C#, and every generated regex adds code to our assembly. For a regex we run once at startup, where throughput doesn’t matter, the plain interpreter is fine.

As with everything else, the context is everything. But for a pattern on a hot path that we know at compile time, source generation is the default choice.

Conclusion

In this article, we’ve learned how the Regex class builds a regular expression, what RegexOptions.Compiled adds on top of that, and how the GeneratedRegex attribute moves all of that work to build time.

Our benchmark showed that creating a new Regex for every call costs us mostly construction. The compiled RegEx is 46 times faster than that baseline, and the source-generated RegEx is 57 times faster, without the 3.2 milliseconds a compiled Regex costs to build.

So, when we know the pattern at compile time, let’s mark a partial method with GeneratedRegex and let the SDK do the rest.

Tested with .NET 10.0.10, .NET SDK 10.0.302 and BenchmarkDotNet 0.15.8.