Skip to content

Conversation

@gtong-nv
Copy link
Contributor

@gtong-nv gtong-nv commented Nov 12, 2025

Fix Enum Constructor Specialization in Generic Types

This PR fixes an issue where synthesized constructors for enums nested within generic types were failing to specialize correctly.

The Problem

When slang synthesizes a requirement witness (like a constructor) for an enum, it creates a new function declaration (synFunc) and adds it to the AST. The original code was creating a DirectDeclRef to this new function:

RequirementWitness(m_astBuilder->getDirectDeclRef(synFunc))

A DirectDeclRef represents a reference to a declaration without any context. It effectively treats the function as if it exists globally or independently.

This becomes a problem when the enum is nested within a generic type (e.g., struct Test<T> { enum Inner { ... } }). The constructor for Inner inherently depends on the generic parameters of its parent (e.g., T). When the compiler later attempts to specialize Test<int>, it encounters the DirectDeclRef. Because this reference has no base or parent pointer, the compiler cannot trace it back to the generic Test<T>, and thus has no way to substitute int for T. This results in an invalid reference to an unspecialized generic constructor within a specialized type context, leading to verification failures or incorrect IR.

The Fix

The fix is to construct the witness using a MemberDeclRef that is rooted at the parent declaration:

RequirementWitness(
    m_astBuilder->getMemberDeclRef(getDefaultDeclRef(context->parentDecl), synFunc))
  1. getDefaultDeclRef(context->parentDecl) retrieves the correct reference to the parent enum (which correctly links back to the generic struct).
  2. getMemberDeclRef creates a reference to synFunc relative to that parent.

This constructs a proper chain of references: Constructor -> Enum -> Generic Struct. When the compiler specializes Test<int>, it can now traverse this chain, identify that the constructor is part of a generic instantiation, and correctly apply substitutions to produce the specialized constructor for Test<int>.Inner.

This PR also adds a few nullptr checks and removes the duplicated findWitnessVal with findWitnessTableEntry.

Fixes: #8887

@gtong-nv gtong-nv added the pr: non-breaking PRs without breaking changes label Nov 21, 2025
@gtong-nv gtong-nv marked this pull request as ready for review November 21, 2025 01:05
@gtong-nv gtong-nv requested a review from a team as a code owner November 21, 2025 01:05
@gtong-nv gtong-nv changed the title Check nullptr before accssing the generic inner return val Fix Enum Constructor Specialization in Generic Types Nov 22, 2025
Copy link
Contributor

@tangent-vector tangent-vector left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great. Thank you again for digging down (well, up in terms of the compiler's dataflow) to the root cause of the problem.

I would like to hear your response to the two questions I raised about the code, but I don't know if either of those can/should turn into an actual code change.

}
for (auto decor : innerInst->getDecorations())

if (innerInst)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we think this conditional is still good/necessary?

Unless we have a reason to believe that innerInst could be null for valid input code, I would rather see a SLANG_ASSERT(innerInst) than an if(innerInst). Either way, this code seems unrelated to the actual fix.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like we still need to keep this conditional check.
As the innerInst = findInnerMostGenericReturnVal(genInst); could return nullptr for the generic that doesn't have return val ( enum in this case?)

The genInst causing the trouble has only a basic block contains a T Op_Param.
We would have to keep check, unless this inst should not be here in the first place..

Copy link
Contributor Author

@gtong-nv gtong-nv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the feedback! Sorry for the late response. I have addressed the comment and update the way we generate memberDeclRef.

}
for (auto decor : innerInst->getDecorations())

if (innerInst)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like we still need to keep this conditional check.
As the innerInst = findInnerMostGenericReturnVal(genInst); could return nullptr for the generic that doesn't have return val ( enum in this case?)

The genInst causing the trouble has only a basic block contains a T Op_Param.
We would have to keep check, unless this inst should not be here in the first place..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr: non-breaking PRs without breaking changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Declaring enum inside generic struct segfaults

3 participants