Scala.js implementation report

[sjrd]

Since I had quite a few hours in the train on the way back from Munich, I implemented custom descriptors in the Scala.js emitter. In total I think it took me 2 full working days to get everything done and polished.

The implementation PR is there: scala-js/scala-js#5255
We won't merge it before there is a Node.js 25.2.0, and also our second main maintainer is currently on vacation, so it will stay there for a few weeks at least.

That said, some interesting bits. I have no performance evaluation at the moment, only expressivity/semantics reports.

It does allow us to implement @JSExport

In Scala.js, one can annotate methods and properties of Scala classes with @JSExport. This is supposed to let JavaScript call those methods, assuming it holds an instance of said class. This was the only bit of Scala.js semantics that we could not implement in Wasm before.

I'm happy to report that Custom Descriptors does fill that gap. methodconfigs and parentidx prototype chaining give us what we need. We can now run the full Scala.js test suite. We don't need exceptions to skip the tests about @JSExport.

null value for the constructors param of configureAll?

We do not use constructorconfig. That's not a problem against the spec, obviously. We have other mechanisms for "top-level" exports in the JS interop layer that are appropriate for Scala.js semantics.

What was a bit surprising is that the 4th parameter of configureAll is therefore useless to us. Its type is externref, so I initially thought I could give it a ref.null. It turns out this yields an "Invalid cast" in V8. Is that expected? Should we be able to pass ref.null noextern if we have no constructorconfig?

Methods with ...rest parameters

I mentioned it before, but to be complete: it's a bummer for us that we can't define methodconfigs that have ...rest parameters. We can use configureAll for all other @JSExports. For @JSExport methods with a ...rest param, we have to fall back to JS glue code.

You can find here our call to configureAll, followed by the additional glue code to deal with the ...rest params:
https://github.com/scala-js/scala-js/pull/5255/files#diff-2755f7122e3b32c0e33f9b9ca3af52496f76ade5f938140654de61853cedd900R330
Note that we need a lot of JS glue code for many other parts of our compiler, so this is really not an issue. It's just a bit weird sitting there, so I'm pointing it out anyway.

Dynamic descriptor allocation

At run-time, Scala shares the peculiarities of Java arrays. In particular, arrays of object types retain their component type in their metadata. Moreover, because of APIs like jl.reflect.Array.newInstance, the set of distinct array types we will need is only dynamically learned at run-time. In Scala.js-on-Wasm, we do this by using a single struct type for ObjectArray, but we dynamically create distinct instances of its vtable type. So we end up with one vtable for jl.Object[], one vtable for jl.String[], one other for jl.String[][], etc.

I expected it, but I was still happy that this scheme worked out-of-the-box with descriptors. We can dynamically create distinct descriptors of ObjectArray for every array type we discover at run-time. It's very nice that we are actually able to do that!

java.lang.Throwable and JS' Error

In Scala.js-on-JS, we rewire our Throwable class so that it truly extends Error. This has several advantages. The most notable one is that debuggers understand that they should store stack trace information in them. When we throw an instance of jl.Throwable, it benefits from all the debugging features you would expect.

With custom descriptors, we are able to do part of that rewiring. We make the prototype we attach to jl.Throwable inherit from Error.prototype. We do this with configureAll, but it's a bit funny. We have to pass in Error.prototype as one of the prototypes we give to configureAll, only to get a parentidx for it. But we do not configure anything to it (constructorconfigs, methodconfigs are empty and parentidx is -1). I'm not sure if that has any adverse effect on Error.prototype on a global scale?

Anyway, doing that, Error.prototype appears on the prototype chain of our Throwables, which is nice for things like x instanceof Error.

What we're still missing is that our Throwables are still not true error objects. In spec terms, they don't have an [[ErrorData]] attribute. That means the debuggers still don't store anything in them for debugging purposes.

I feel like this is still a gap we should try to fill, but I'm not sure whether Custom Descriptors is the right place for that?

[tlively]

What was a bit surprising is that the 4th parameter of configureAll is therefore useless to us. Its type is externref, so I initially thought I could give it a ref.null. It turns out this yields an "Invalid cast" in V8. Is that expected?

No, that's not expected. I'll be writing configureAll spec tests soon that will catch this (among several other bugs, probably).

I mentioned it before, but to be complete: it's a bummer for us that we can't define methodconfigs that have ...rest parameters.

How many of these do you typically have compared to non-variadic methods? Since we have to draw the line for supported features and code patterns somewhere, I would definitely want to see hard data motivating special support for this.

I expected it, but I was still happy that this scheme worked out-of-the-box with descriptors. We can dynamically create distinct descriptors of ObjectArray for every array type we discover at run-time. It's very nice that we are actually able to do that!

That's great validation for the generality of the design 🎉 Do you use configureAll for these, or do you manually configure the prototypes for these descriptors?

We do this with configureAll, but it's a bit funny. We have to pass in Error.prototype as one of the prototypes we give to configureAll, only to get a parentidx for it. But we do not configure anything to it (constructorconfigs, methodconfigs are empty and parentidx is -1). I'm not sure if that has any adverse effect on Error.prototype on a global scale?

This is working as intended and should not have any global adverse effect that I'm aware of.

What we're still missing is that our Throwables are still not true error objects. In spec terms, they don't have an [[ErrorData]] attribute. That means the debuggers still don't store anything in them for debugging purposes.

Yeah, I would prefer to consider this out of scope for this proposal, but perhaps it's something we can consider in a follow-on proposal. I suspect this will be tricky for implementations because the spec machinery of having an [[ErrorData]] attribute is likely just a cute way of specifying "is an instance of a well-known Error type," where that well-known type is definitely not a Wasm type. All the places in the implementation that receive Error objects would have to be updated to handle a new kind of Wasm Error as well. (This is the same problem as in #50.)

[sjrd]

No, that's not expected. I'll be writing configureAll spec tests soon that will catch this (among several other bugs, probably).

Oh, good to know.

I mentioned it before, but to be complete: it's a bummer for us that we can't define methodconfigs that have ...rest parameters.

How many of these do you typically have compared to non-variadic methods? Since we have to draw the line for supported features and code patterns somewhere, I would definitely want to see hard data motivating special support for this.

Probably almost none. I do think the current line is fine. Even if we did expand to support ...rest params, various languages might not agree on how they should map to Wasm. For us we just want the JS array (as an externref/anyref). But I can see other languages preferring an (array externref) for example. So it would lead to even more bikeshedding or spec complexity. IMO it's not worth it.

That's great validation for the generality of the design 🎉 Do you use configureAll for these, or do you manually configure the prototypes for these descriptors?

Ah, neither 😅 Array classes never introduce @JSExports, so they always share the prototype object of jl.Object (their parent class). The latter has an export for toString(), and we configure it with configureAll. But we don't need to create any more prototypes at run-time. We only create more descriptors, which reuse existing prototypes.

This is working as intended and should not have any global adverse effect that I'm aware of.

Nice, thanks.

[jakobkummerow]

Thanks for the feedback! Nice to hear that the proposal mostly works well for you. A few comments from an engine implementation perspective:

null value for the constructors param of configureAll?

Makes sense that null should be allowed there. It's mostly an accident that so far it isn't; checking for JS object up front (before even knowing whether any constructors will be installed) is easier/cheaper than any alternative and was enough to validate the idea. I'll figure out how to relax that.

Throwable class so that it truly extends Error

The one thing I feel strongly about is that Wasm objects inheriting from JS objects isn't going to happen.
I'm open to ideas that achieve similar benefits in different ways; e.g. there could be some sort of Wasm-side annotation that certain objects/types should have the conceptual equivalent of an [[ErrorData]] attribute. I agree that that's probably out of scope for this proposal. (It could perhaps be a separate "exceptions with stack traces" proposal; perhaps combined with a "catchable traps" proposal: for mapping a Java-style NullPointerException to Wasm, ideally there'd be a way to catch the trap that struct.get and similar instructions produce, and also have a stack trace attached to it...)

[sjrd]

The one thing I feel strongly about is that Wasm objects inheriting from JS objects isn't going to happen.

Definitely agree about that. ;)