The way how code execution is implemented

[bingo-soft]

Hello, dear professionals! I try to reverse-engineer Enso library and can not understand one detail about the way how language server actually executes the source code (provided for example through AttachVizualisation command). I'm struggling with it, because I see that when truffle context is initialized, then RuntimeServerInstrument.onLanguageContextInitialized is called. And inside this method we enter, initialized ExecutionService and leave the context. So, I do understand (or at least it seems like that) that ExecutionService is responsible for executing source code, but I do not see in the source code, when we enter and leave truffle context again, it seems like truffle context is bound to the thread where ExecutionService is initialized, but how can we be certain that code execution later runs in the same thread as where ExecutionService was instantiated? What if we are runing in a multi-threading environment and call expressions evaluation in other threads? Thanks in advance!

[bingo-soft]

I finally figured it out. At least I see, that I can submit commands, that need to have access to context and language, to context thread manager. And in this case, I do not have to explicitely enter and leave language server truffle context. I checked it and see that it works like so, but if someone from core team could provide more details on that or give some clarification, it would be great. Thank you for your attention to this matter!

[JaroslavTulach]

Interesting question. What do you actually want to achieve? (except reverse engineer an open source code!)

[bingo-soft]

@JaroslavTulach Hello, Jaroslav! I'm actually porting some enso code to Kotlin stack. No Rust, no Scala, no Akka - just pure Kotlin with coroutines, channels and other stuff. I did a lot - there is a parser implemented with Antlr, I know its "cons" - slow speed and etc, but I need its expressiveness and simplicity to add new language constructs. I also implemented the foundation of language server and some commands are already working - like session initialization and context creation. Now I'm working on code execution and instrumentalization

[bingo-soft]

@JaroslavTulach I plan to opensource it in a couple of months =) The language syntax will significantly differ from enso - it will be simplified Kotlin. What is already implemented: expressions, classes, class inheritance, member declaration, methods and package loading. I do not plan to touch client side, but tend more to jupyter notebook style (not canvas DnD)

[Akirathan]

@bingo-soft Few notes/recommendations about your approach.

there is a parser implemented with Antlr

I remember that a few years ago, we held an internal meeting about the parser, and ANTLR was pitched up as a viable option. If I remember correctly, the reason why we created our own parser in Rust from scratch was (a) custom error recovery and (b) proper support for our custom indent based syntax. I have tried to port our Rust parser to ANTLR myself but failed to reproduce the indentation specifics. I have not spent a lot of time on it, though.

I plan to opensource it in a couple of months =)

That will be greatly appreciated. Especially the part with ANTLR grammar.

No Rust, no Scala, no Akka.
I also implemented the foundation of language server and some commands are already working

Note that there are plans to get rid of AKKA (#11159), and preferably also replace remaining Scala code with Java.

I'am actually porting some enso code to Kotlin stack.
Now I'm working on code execution and instrumentalization

I wouldn't personally recommend using Kotlin with Truffle:

• Truffle heavily relies on annotation processors. I am not sure if it is even possible to trigger Java annotation processors from Kotlin compiler? Even if that would be possible, it smells like some build tool hacks are needed for that.
• On "fast path", you wouldn't be able to use a lot of Kotlin fancy stuff along with its standard library, since I expect vast majority of it is block listed. Note that majority of Java's standard library is block listed as well and cannot be used. That is why a lot of "Truffle friendly" code is just a pure and very simple Java.
• I am not aware of any Truffle project using Kotlin. That doesn't mean there is none.

Still, if you manage to develop a working prototype, I will be more than interested.

[bingo-soft]

Hello, @Akirathan ! Thanks for comments. As for Antlr grammar, there is in fact nothing so special, because my syntax is simplified Kotlin (not indent based as enso language). So, I took already known grammars as a starting point. As for AKKA - it was the simplest part to port in pure Kotlin, because actor is easily modeled with a message channel and event loop, but I'm not so sure, that this architecture is as mature and battle-tested as Akka framework. As to your recomendation not to use Kotlin with Truffle because of annotation processor hurdles - that is absolutely valid. I use some dirty hack for that purpose - implement a template in Java, compile it with annotation processor and then port the resulting code to Kotlin.