Need replacement for EngineVersion checking

[CharliePoole]

UPDATED 19 Oct 2025

Because NUnit Extensibility was designed for the engine, the ExtensionAttribute supports a property 'EngineVersion, which is checked by ExtensionManager`. However, the V4 update to the extensibility model envisions its use by other NUnit components than just the engine and even other applications... TestCentric for example. When hosted by other components or applications, the host version will not necessarily be the same as the NUnit version.

It would be possible, of course, to continue to use the NUnit engine version and require other hosts to pick some engine version with which they are compatible. This would require no changes to the NUnit.Extensibility package but would put a large burden on any other users.

The alternative proposed here would replace the existing EngineVersion with two new properties:

1. ExtensiibilityVersion, to represent the level of the extensibility model used by the extension.
2. HostVersion, to represent the level of functionality the extension requires from the host.

The host would need to check both these versions before installing the extension using something like the following pseudo-code:

if the version of extensibility I am using >= ExtensibilityVersion and my version >= HostVersion
    install the extension

For the engine as it is today both checks will require a minimum value of 4.0. My TestCentric runner would probably use 4.0 and 2.0 for the minimum extensibility and host values. Similarly if any other NUnit packages were to use this feature, two different values would be needed. (Thinking of NUnitLite here)

NOTES:

1. Although the 4.0 engine could continue to support EngineVersion when it is found, I'm not in favor of that. It makes the logic slightly more complicated and seems unnecessary since all extensions will need to be rewritten for 4.0 anyway.
2. The ExtensibilityVersion property would be supported on the ExtensionAttribute just as EngineVersion is now.
3. The HostVersion property may be more appropriate on the ExtensionPointAttribute. Some experimentation is needed.
4. Any host is free to assume a default value if either property is not found. The host would, of course, need to deal with any resulting failures.
5. Any extension that is designed to be used on multiple hosts should probably use ExtensionPointAttribute rather than TypeExtensionPointAttribute. The latter is a shortcut that works best for extensions that are embedded in the host itself.
6. This proposal should be considered as experimental at this point. As we work with it, we may want to consider modifications before the final 4.0 release.@nunit/engine-team Thoughts?

[manfred-brands]

@CharliePoole I got a deja-vu with this, did you reword it from the previous incarnation?

If we only need a single version, then there should be a Version property on the ExtensionAttribute
If besides the extension interface, also the engine version is relevant, then use two properties.

[CharliePoole]

@manfred-brands I duplicated it from TestCentric/TestCentric.Extensibility#71 using GitHub "Duplicate issue" and then edited it, which took me a while. You may not have seen the updated text, which mentions the duplication.

I originally used EngineVersion rather than Version because I thought that [Extension, Version=3.5] would confuse users, making it appear to be the version of the extension rather than of the hosting engine. I think this may still apply with the new property.

Regarding having one versus two properties, we could eliminate EngineVersion in V4 since V3 extensions will no longer work under the V4 engine anyway. Maybe that's the cleanest approach.

[CharliePoole]

The background that gave rise to this issue has changed, I think for the better, since TestCentric no longer maintains it's own TestCentric.Extensibility package but uses NUnit.Extensibility.

However, for the future, I think we still need to deal with two essentially different concepts:

1. The version of the extensibility model, which is logically equivalent to the version of the assembly that encapsulates it... i.e. nunit.engine.extensibility.dll. An example of this can be found in TeamCityEventListener, which uses [Extension(Enabled = false, EngineVersion = "3.4")] because the Enabled property was introduced with that release.
2. The version of the host, which is being extended, i.e. the engine up to now but possibly the console runner or even the framework in the future as well as the TestCentric Gui. This version would encapsulate any features required of the host in order for the extension to work.

Right now, a single EngineVersion works in both roles (1) the engine is the only host for which extensions exist and (2) the version of the engine and the version of the extensibility assembly are the same. Either of these factors could easily change in the future.

Therefore, as a hedge for future development I'm modifying the proposal in the initial description above to include two different "versions", ExtensibilityVersion and HostVersion. Both of these would be actual version strings rather than integers because it makes a bit less of a change and eliminates the need to coordinate a separate integer with the assembly versions.

@nunit/engine-team Please review the modified proposal and add your comments. This is a very significant change and I'd like to be sure everyone is on board before implementing it.

[CharliePoole]

Further thoughts...

Since I mentioned NUnitLite above, I started thinking about how that would work.

It seems to me that the proposal I have made works for any extension that targets a single host. However, let's say we wanted to replace the current duplicate code for TeamCityEventListener in NUnitLite with use of the actual extension. This would be of benefit, since it would eliminate changing the code in NUnitLite whenever JetBrains changes the extension.

In this case, we would have an extension that targets two different hosts. What would we use for the HostVersion?

For that reason, I think we may want to place HostVersion on the ExtensionPoint rather than the Extension. Extension points are present in each host.

@nunit/engine-team @nunit/team-city-team Keep this in mind as you review. Meanwhile, I'll try out the code using both approaches.

[manfred-brands]

@CharliePoole An Extension follows an interface specification. Why would the EngineVersion be relevant?

If all 3 users of the extension use the same specification, it shouldn't matter if the engines might be different.

[CharliePoole]

@manfred-brands We're dropping EngineVersion so I'm guessing you mean HostVersion.

It's a good question. I'd say for two reasons...

1. In the past, we have allowed for methods to be added to our extensibility and other API interfaces so long as the old methods remained. We could change that practice and always require a new interface to be defined in future, i.e. ISomething2 or ISomethingEx or some entirely new name. It's what we did back in COM days actually.

2. Our interface definitions are quite general and don't deal with the particular semantics of each extension. At the moment, however, I can't think of an example.

I'd be willing to hold off on HostVersion until there is a demonstrated need for it. We'd then rely on a combination of the interface and a particular extension point path associated with it by the host. That is, IFF the host defines a particular path for use with a particular interface, then the extension can be installed.

What do you think?

[manfred-brands]

@CharliePoole It first depends on who calls what. I didn't check.

Is the host is calling into the extension or is the extension calling into the host.

If the first, the host calls the methods as defined by the version supported by the extension.

If the second, an existing extension doesn't know about the new method and hence would not call the new methods, so that would work.

But if we want to make a distinction between compatible and non-compatible changes, you will have to use semantic versioning.

A version 1.1 can add new methods, an extension can call or not.
A version 2.0 means that the methods for version 1 might no longer be available.

[CharliePoole]

@manfred-brands Up to now, the host has only called the engine although I envisioned having the engine call the host for some services if it were needed. It could be done by setting an interface as the value of some well-known ExtensionProperty just as we now set the Enabled property. It's there when/if the need arises, but for now the calling is all one-way.

The issue with the host calling the methods supported by the extension is that the interfaces are not defined by the extension, but by the host. Let's take the an extension point /NUnit/Engine/TestEventListeners, as it is now defined. This may be used for different purposes. JetBrains uses it to provide the special output needed for TeamCity. Imagine that a few other CI providers were to do the same thing.

The TeamCity listener is currently at version 1.10. Vendor 2 comes along and creates a new extension starting at version 1.0. Some corporate user creates a private extension and starts it off at Version 4.0 to signal that it only works with version 4 of the engine. All of them are using the same version of the ITestEventListener interface, which has never changed since it was originally defined in the 3.0 engine.

The host can, of course, find out the version of the extension assembly. But that's not of much use unless the host is familiar with the requirements of each extension. For example, the current version of TeamCityEventListener only works correctly under version 3.4 of the engine or later because it relies on the engine setting the Enabled property to turn it on. The version under development will only work with version 4.x of NUnit.Extensibility, which we bundle with the 4.x engine. Those other two hypothetical extensions may be limited similarly, differently or not at all.

Semantic versioning can and does help. But it doesn't do the whole job since it applies to dependent packages, not assemblies embedded together in the same package. And it particularly doesn't apply to the engine API, because its AssemblyVersion is pegged at 3.0.0.0 currently.

All that said, what I'm describing is how we have done it for the lifetime of V3.x. If we want to change things, this is the opportune time to discuss it. Ideally, I'd like to see all the breaking changes in place for beta.2 and go on from there.

For me, a video chat would work best, provided you have bandwidth for it. I have an hour free now and then I'm tied up for the rest of the day. The remainder of the week is open except for a webinar 10-11am PDT on Wednesday. Would you be up for that? I could share some more information and maybe we could pick a course of action.