Add no-allocation version of new StringInfo(string).LengthInTextElements

[Joy-less]

The new StringInfo(string).LengthInTextElements method is used to get the number of graphemes in a string. However, you have to allocate a StringInfo, which is not performant for a method that is an alternative to string.Length.

If you don't want to allocate, you have to write a method like this:

public static int GetLengthInTextElements(scoped ReadOnlySpan<char> Input) {
    int TextElementCount = 0;
    int Index = 0;
    while (true) {
        int Length = StringInfo.GetNextTextElementLength(Input[Index..]);
        if (Length <= 0) {
            break;
        }
        TextElementCount++;
        Index += Length;
    }
    return TextElementCount;
}

Could this method be added to the standard library?

EDIT: See my proposal: #123092

[huoyaoyuan]

However, you have to allocate a StringInfo, which is not performant for a method that is an alternative to string.Length.

StringInfo is a high cost API. It can never be a performance alternative to string.Length:

        [Benchmark]
        [Arguments("qwertyuiopasdfghjklzxcvbnm")]
        public int StringInfoLength(string str) => new StringInfo(str).LengthInTextElements;

        [Benchmark]
        [Arguments("qwertyuiopasdfghjklzxcvbnm")]
        public int LengthFromTextElements(string str)
        {
            int TextElementCount = 0;
            int Index = 0;
            while (true)
            {
                int Length = StringInfo.GetNextTextElementLength(str.AsSpan(Index));
                if (Length <= 0)
                {
                    break;
                }
                TextElementCount++;
                Index += Length;
            }
            return TextElementCount;
        }

Method	str	Mean	Error	StdDev	Gen0	Allocated
StringInfoLength	qwert(...)cvbnm [26]	397.5 ns	1.30 ns	1.15 ns	0.0067	128 B
LengthFromTextElements	qwert(...)cvbnm [26]	395.7 ns	0.73 ns	0.65 ns	-	-

The overhead of allocation is negligible comparing to the cost of the algorithm.

Moreover, your code has an issue that amplifies the allocation much more. Input[Index..] slices on the string and allocates.

[Joy-less]

I ran a benchmark with 100,000 iterations:

Method	str	Mean	Error	StdDev	Gen0	Allocated
StringInfoLength	qwert(...)cvbnm [26]	54.70 ms	0.134 ms	0.126 ms	2500.0000	16000000 B
LengthFromTextElements	qwert(...)cvbnm [26]	51.82 ms	0.087 ms	0.077 ms	-	-

Benchmark code

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using System.Globalization;

namespace LengthInTextElementsBenchmark;

public class Program {
    public static void Main() {
        BenchmarkRunner.Run<Benchmarks>();
    }
}

[MemoryDiagnoser]
public class Benchmarks {
    [Benchmark]
    [Arguments("qwertyuiopasdfghjklzxcvbnm")]
    public int StringInfoLength(string str) {
        int Counter = 0;

        for (int I = 0; I < 100_000; I++) {
            Counter += new StringInfo(str).LengthInTextElements;
        }

        return Counter;
    }

    [Benchmark]
    [Arguments("qwertyuiopasdfghjklzxcvbnm")]
    public int LengthFromTextElements(string str) {
        ReadOnlySpan<char> Span = str;

        int Counter = 0;

        for (int I = 0; I < 100_000; I++) {
            int TextElementCount = 0;
            int Index = 0;
            while (true) {
                int Length = StringInfo.GetNextTextElementLength(Span[Index..]);
                if (Length <= 0) {
                    break;
                }
                TextElementCount++;
                Index += Length;
            }
            Counter += TextElementCount;
        }

        return Counter;
    }
}

The allocating version is slightly slower, but more importantly the allocation is 16 MB for 100,000 iterations. The Error is also ~1.5x higher, which implies certain runs are slower, which implies garbage collection spikes are present.

Also, I don't know if the StringInfo.GetNextTextElementLength method is the most performant way to implement this method - there may be internal optimizations that can be done.

Moreover, your code has an issue that amplifies the allocation much more. Input[Index..] slices on the string and allocates.

I think you've misread the original example - it takes a span not a string.

[huoyaoyuan]

I ran a benchmark with 100,000 iterations

The Error is also ~1.5x higher, which implies certain runs are slower, which implies garbage collection spikes are present.

This is not the best practice, and will count the instability introduced by GC into the benchmark. But yes, it correctly shows the impact of GC.

However, benchmarking LengthInTextElements may not match real-world workloads. It's very likely to enumerate the elements after acquiring the length. In this case, StringInfo will be optimal because it won't recalculate the offsets.

Also, I don't know if the StringInfo.GetNextTextElementLength method is the most performant way to implement this method - there may be internal optimizations that can be done.

It's hardly. You can check the source code of StringInfo.

[Joy-less]

For what it's worth, I looked at the new StringInfo(string).LengthInTextElements implementation and optimised the original example, now I benchmarked it at ~8% faster.

public static int GetLengthInTextElements(scoped ReadOnlySpan<char> Input) {
    ReadOnlySpan<char> Remaining = Input;
    int TextElementCount = 0;
    while (!Remaining.IsEmpty) {
        TextElementCount++;
        Remaining = Remaining.Slice(StringInfo.GetNextTextElementLength(Remaining));
    }
    return TextElementCount;
}

After looking at the StringInfo implementation, it's complicated and not necessarily optimised for simple (ASCII only, single-rune characters only) examples. If anything, it may be good to add a StringInfo.GetLengthInTextElements method so that if it gets optimised in the future then code can take advantage of it.

[ufcpp]

I want not only LengthInTextElements but also a struct enumerator. Just like string.EnumerateRunes returns a StringRuneEnumerator struct.

[Joy-less]

You can already do StringInfo.GetTextElementEnumerator(string). Although that returns a TextElementEnumerator, which is sadly a class.