[API Proposal]: Allow Span<T> to implement IEnumerable<T> via Compiler Call Site Substitution / Specialization

[AlexRadch]

Background and motivation

Currently, the .NET ecosystem suffers from API dichotomy: developers are forced to create and maintain duplicate methods — one set for IEnumerable<T> and another for ReadOnlySpan<T>.

In C# 13 (.NET 9), ref struct types gained the ability to implement interfaces. However, due to fundamental limitations of ref struct (the inability to be boxed on the heap), Span<T> cannot support the IEnumerable<T> interface. The IEnumerable<T>.GetEnumerator() method returns IEnumerator<T> (a reference type). Since the Span enumerator (Span<T>.Enumerator) is itself a ref struct, it cannot be returned as an interface without boxing.

Because of this limitation, we cannot use Span<T> in a vast number of existing methods that accept IEnumerable<T>, ICollection<T>, and IList<T>, even though Span<T> logically supports these interfaces.

API Proposal

I propose introducing support for IEnumerable<T> in Span<T> (and ReadOnlySpan<T>) using a compiler "hint" mechanism (e.g., via an attribute).

The core of the proposal:

1. Interface Implementation: Allow Span<T> to explicitly implement IEnumerable<T>, but mark the implementation of IEnumerable<T>.GetEnumerator with a special attribute. This signals the compiler to substitute calls to this method with a call to the analogous method that returns the ref struct Span<T>.Enumerator.
2. Call Specialization (Lowering): Introduce a mechanism where the compiler, upon detecting a GetEnumerator() call for Span<T>, substitutes it with a direct call to the GetEnumerator() method of the structure itself.
3. Enumerator Handling: Most importantly, the returned result of GetEnumerator() (which is a ref struct) must not be cast to IEnumerator<T>. The compiler must continue to treat it as a struct, similar to how it currently handles struct enumerators in foreach loops.

Essentially, this proposes extending the "Compiler Call Site Substitution" pattern to attribute-marked methods by invoking the struct method directly without casting the result to the IEnumerator<T> interface.

Expected Result

Methods accepting IEnumerable<T>, ICollection<T>, and IList<T> will be able to transparently accept Span<T> without memory allocations or data copying. This will allow for the reuse of a massive amount of existing code with the Span<T> type without needing to rewrite libraries, while achieving significant gains in both code performance and memory efficiency.

Code for Span<T> that support IEnumerable<T> interface

    public readonly ref struct Span<T>: IEnumerable<T> // add IEnumerable<T> implementation
    // Also it can support ICollection<T>, IList<T> and other interfaces for generic collections
    {
         // Inform the compiler that Span<T>.GetEnumerator() implements IEnumerable<T>.GetEnumerator() via specialization
         [SpecializedImplementation(IEnumerable<T>.GetEnumerator())] 
         public Enumerator GetEnumerator() => new Enumerator(this);

         public ref struct Enumerator: IEnumerator<T> // add IEnumerator<T> implementation
         {
             // Explicit interface implementations required by contract
             object IEnumerator.Current => Current!;
             void IEnumerator.Reset() => throw new NotSupportedException();
             void IDisposable.Dispose() { }
         }
    }

Explanation of the proposal

I propose introducing compiler support for a [SpecializedImplementation] attribute (naming is tentative). This attribute signals to the compiler that the marked method or property serves as a specialized implementation of a specific interface member.

Key Requirement: The method's input arguments and return type must be signature-compatible with the interface member.

In this specific case:

1. The GetEnumerator method has no input arguments (matching the interface signature).
2. The return type Span<T>.Enumerator implements IEnumerator<T>, which satisfies the requirement of the IEnumerable<T> interface method.

This enables the compiler to perform a safe call site substitution: instead of emitting a standard interface dispatch (or boxing), it emits a direct call to the ref struct method. Crucially, the return value is not boxed into IEnumerator<T>; instead, the compiler continues to treat it as a ref struct which effectively satisfies the interface contract.

Note: I am open to discussing the specific syntax. Perhaps, instead of an attribute, a more elegant approach could be proposed, such as new keywords or an extension of the implements syntax.

API Usage

Using HashSet<T> as an example, we can see how adding the allows ref struct constraint, combined with the proposed specialization mechanism, allows existing generic methods to accept Span<T> natively.

// Changes to allow ref struct for generic methods relying on IEnumerable<T>
public class HashSet<T>
{
    // Note: Since instance constructors in C# cannot be generic, creation from Span 
    // would still require a specific overload or a static generic factory method.
   public HashSet(ReadOnlySpan<T>)
    
    // However, existing instance methods can be retrofitted simply by adding 'allows ref struct'
    // allowing them to consume Span<T> via the specialized generic path.

    public void ExceptWith<TEnumerable>(TEnumerable other) 
        where TEnumerable : IEnumerable<T>, allows ref struct;

    public void IntersectWith<TEnumerable>(TEnumerable other) 
        where TEnumerable : IEnumerable<T>, allows ref struct;

    public bool IsProperSubsetOf<TEnumerable>(TEnumerable other) 
        where TEnumerable : IEnumerable<T>, allows ref struct;

    public bool IsProperSupersetOf<TEnumerable>(TEnumerable other) 
        where TEnumerable : IEnumerable<T>, allows ref struct;

    public bool IsSubsetOf<TEnumerable>(TEnumerable other) 
        where TEnumerable : IEnumerable<T>, allows ref struct;

    public bool IsSupersetOf<TEnumerable>(TEnumerable other) 
        where TEnumerable : IEnumerable<T>, allows ref struct;

    public bool Overlaps<TEnumerable>(TEnumerable other) 
        where TEnumerable : IEnumerable<T>, allows ref struct;

    public bool SetEquals<TEnumerable>(TEnumerable other) 
        where TEnumerable : IEnumerable<T>, allows ref struct;

    public void SymmetricExceptWith<TEnumerable>(TEnumerable other) 
        where TEnumerable : IEnumerable<T>, allows ref struct;

    public void UnionWith<TEnumerable>(TEnumerable other) 
        where TEnumerable : IEnumerable<T>, allows ref struct;
}

// Usage Example:
Span<int> span1 = stackalloc int[] { 1, 2, 3 };
Span<int> span2 = stackalloc int[] { 4, 5, 6 };

// No memory allocation, no boxing, and no additional code needed in HashSet to support Span specifically
var hashSet = new HashSet<int>(span1); 

// The compiler specializes this call to use the struct enumerator of Span<T>
hashSet.UnionWith(span2);


### Alternative Designs

_No response_

### Risks

_No response_

[huoyaoyuan]

Discussed in dotnet/csharplang#9961. API proposals shouldn't be opened directly that requires language support.

[tannergooding]

This would have a large amount of hidden cost and I don't think is feasible.

It would require every callsite or even local/field that takes IEnumerable<T> (or any of the other interfaces listed) to implicitly expand and specialize to support both IEnumerable<T> and ROSpan<T>. This is "a lot" of code explosion and new specialization support that simply does not exist today.

Additionally, it would likely still not be used in most places in the BCL due to the differing semantics between working with a span or working with one of these interfaces and the different guarantees they provide, particularly as it pertains to large structs.

There isn't some magic zero-cost way to provide all the things you need. Sometimes, you write code twice if you need to support both of the specialized scenarios. That's just how programming is, across all languages and ecosystems.

I would propose we simply close this as inactionable and not something that .NET is looking at doing. -- CC. @jkotas

[jkotas]

Duplicate of dotnet/csharplang#9961 . It should be discussed there.

As @tannergooding said this looks very complicated for what it provides.

[AlexRadch]

@tannergooding

Thank you for the detailed feedback.

I respectfully disagree with the assessment that this would involve a large amount of hidden cost. Support for ref structs implementing interfaces is already in place. The only additional work required would be a compile-time check for the attribute and a substitution of the interface method call with a call to the corresponding method that returns a ref struct implementing the interface — because that method is marked with the special attribute.

         // Method that already exists and return ref struct that already implement IEnumerator<T> interface
         public Enumerator GetEnumerator() => new Enumerator(this);

         // Inform the compiler that this method is implemented by another method through specialization.
         [SpecializedImplementation(GetEnumerator)] 
         IEnumerator<T> IEnumerable<T>.GetEnumerator() => throw new InvalidOperationException();

The additional code in Span itself would be just these two lines. In the compiler, it would involve checking for the SpecializedImplementation attribute and replacing one method call with another.

After this, Span will implement the typed interfaces IEnumerable, ICollection, IList, IReadOnlyCollection. It may also be useful for other reference structures to implement a useful interfaces, which is not possible now.

[huoyaoyuan]

The additional code in Span itself would be just these two lines.

This kind of specialization requires much more design decisions than "two lines".

The only additional work required would be a compile-time check for the attribute and a substitution of the interface method call with a call to the corresponding method that returns a ref struct implementing the interface

It's not as simple as such. The ABI of returning and calling on boxed interface is entirely different from struct. Currently it requires getting escape analysis involved and get the boxing canceled. Such type of special treatment are avoided for cases without much value.

[AlexRadch]

@tannergooding @huoyaoyuan

The additional code in Span itself would be just these two lines.

This kind of specialization requires much more design decisions than "two lines".

I completely agree that more design decisions would be required. Enabling ref structs to implement interfaces already involved extensive discussions and a lot of work. What I am proposing is simply to continue that effort and extend the existing interface support to cover interfaces that currently cannot be implemented by ref structs. In other words, it would build directly on the work already done, in the same direction. Yes, it would require many additional design decisions, but I believe the benefits would be comparable to those provided by the support that has already been implemented.

The only additional work required would be a compile-time check for the attribute and a substitution of the interface method call with a call to the corresponding method that returns a ref struct implementing the interface

It's not as simple as such. The ABI of returning and calling on boxed interface is entirely different from struct. Currently it requires getting escape analysis involved and get the boxing canceled. Such type of special treatment are avoided for cases without much value.

I might be mistaken here, but I believe the compiler already performs such checks. The compiler already handles ABI-related and similar checks in generic methods that include the allows ref struct constraint. Therefore, the existing infrastructure for allows ref struct may already provide most of what is required, and my proposal would only be a modest extension of that.

[tannergooding]

I respectfully disagree with the assessment that this would involve a large amount of hidden cost.

It is not as simple as you're claiming. It comes down to thousands, if not more, lines of code to support those "two lines" and will still be quite a mess and not really work.

The only additional work required would be a compile-time check for the attribute and a substitution of the interface method call with a call to the corresponding method that returns a ref struct implementing the interface — because that method is marked with the special attribute.

This is a massive, ridiculously so, amount of work. It is likewise not something that can be handled by Roslyn at all, but rather would be required to be handled by the runtime.

For starters, Roslyn doesn't have access to the IL implementation of most methods (there literally is no such IL in the ref assembly) and so cannot do any kind of replacement here itself. That is, you cannot simply take the List<T>(IEnumerable<T>) constructor that exists in metadata and have Roslyn do something such that new List<T>(span) works. But beyond that, this kind of replacement by Roslyn wouldn't be safe because the implementation can change between its point of compilation and what exists at runtime (see also things like netstandard, customization that exists for RyuJIT vs Unity or other runtimes, etc).

Now the runtime theoretically could do some magic here. It would have to be the thing to recognize new List<T>((IEnumerable<T>)someSpan) (which is illegal today due to boxing), which forces it to specialize that method to now have a span variant (resulting in extensive implicit code duplication, on top of the existing generic specialization for structs). But additionally, it has to factor in various "impossibilities" that can exist in other code.

Some examples of these impossibilities include any kind of storage of that span field. For example, say you have void M(IEnumerable<T> items) and internally there is some List<IEnumerable<T>> or even just an IEnumerable<T> _items field. Roslyn can't know that such code exists and therefore casting or substituting the span is invalid, so the runtime has to recognize it instead. The runtime then has to rationalize that an attempt to persist the span in a way that forces a box to be realized will occur and so must fail the specialization attempt, all occurring at runtime.

Now you could say, well code needs to just introduce a new generic method M<TEnumerable>(TEnumerable items) where TEnumerable : IEnumerable<T>, allows ref struct. However, this has its own negative costs and isn't always feasible, particularly for already generic types where the code explosion (particularly for AOT, where specialization cannot happen incrementally as types are actually used) can become significant. This likewise doesn't really solve the problem of ROSpan the interfaces, since it can't actually be used by anything that implements the interfaces, only by these special methods. It doesn't solve the problem of needing to maintain two methods, particularly for the vast amount of code already out there and that will always persist. It doesn't make code simpler to consume, since many cases will now have 2 or more generic types involved (since many already involve one generic for the T today and will now also have a TEnumerable).

It is a lot of work for something that is trivially and often far more efficiently handled by simply making that other method take ROSpan<T> itself and handling it.

[timcassell]

However, this has its own negative costs and isn't always feasible, particularly for already generic types where the code explosion (particularly for AOT, where specialization cannot happen incrementally as types are actually used) can become significant.

Is that why we don't have interface IEnumerable<T, TEnumerator> where TEnumerator : IEnumerator<T>, allows ref struct?

[tannergooding]

That'd be one reason. Another is that its not very "usable" in practice, you end up having to both expose and know what the TEnumerator is, as well as have the user know to pass that down.

In most cases, inlining or other things are far more viable options to optimize typical scenarios. Everyone would be paying cost for something that is rarely if ever actually needed.

[timcassell]

its not very "usable" in practice, you end up having to both expose and know what the TEnumerator is, as well as have the user know to pass that down.

Well that at least should be solvable by improved type inference in C#.