Using BabylonJS for a uno-library control

[VRage]

Hey,
i try to get BabylonJS working in an Uno Platform library to build a multiplatform 3D viewer for WASM and WinUI3 (via WebView2)

So, I added an Uno library project to the solution, downloaded the BabylonJS file, and included it as an embedded resource over the csproj file (btw: if i added the file as an embeeded resource via the properties window in VS2022, i get following error):

An error has occurred while saving the edited properties listed below:
Build Action
One or more values are invalid. Cannot add 'Platforms\Webassembly\WasmScripts\babylon.max.js' to the project, because the path is explicitly excluded from the project (C:\Users\Jakob.nuget\packages\uno.sdk\6.2.29\targets\Uno.Sdk.After.targets (12,3))..

I used BrowserHTMLElement to execute the sample JS code from the BabylonJS homepage:

var js = @"const canvas = document.getElementById(""renderCanvas""); // Get the canvas element
        const engine = new BABYLON.Engine(canvas, true); // Generate the BABYLON 3D engine

        const createScene = function () {
            // This creates a basic Babylon Scene object (non-mesh)
            const scene = new BABYLON.Scene(engine);

            // This creates and positions a free camera (non-mesh)
            const camera = new BABYLON.FreeCamera(""camera1"", new BABYLON.Vector3(0, 5, -10), scene);

            // This targets the camera to scene origin
            camera.setTarget(BABYLON.Vector3.Zero());

            // This attaches the camera to the canvas
            camera.attachControl(canvas, true);

            // This creates a light, aiming 0,1,0 - to the sky (non-mesh)
            const light = new BABYLON.HemisphericLight(""light"", new BABYLON.Vector3(0, 1, 0), scene);

            // Default intensity is 1. Let's dim the light a small amount
            light.intensity = 0.7;

            // Our built-in 'sphere' shape.
            const sphere = BABYLON.MeshBuilder.CreateSphere(""sphere"", {diameter: 2, segments: 32}, scene);

            // Move the sphere upward 1/2 its height
            sphere.position.y = 1;

            // Our built-in 'ground' shape.
            const ground = BABYLON.MeshBuilder.CreateGround(""ground"", {width: 6, height: 6}, scene);

            return scene;
        };

        const scene = createScene(); //Call the createScene function

        // Register a render loop to repeatedly render the scene
        engine.runRenderLoop(function () {
                scene.render();
        });

        // Watch for browser/canvas resize events
        window.addEventListener(""resize"", function () {
                engine.resize();
        });

I was able to include this control into a single-page Uno project for WebAssembly and Windows. However, no matter what I tried, BABYLON was always undefined.

So I checked the Source tab in the developer tools, and it seems like the BabylonJS file was loaded:

Also, the JavaScript file was referenced on the page:

When I run everything locally in an HTML file, it works. I also tried loading BabylonJS via CDN, without success.

One might think it’s just not possible to do it—but I’ve already managed to display the BabylonViewer, just not with a library in between BJS Viewer

Does anyone have tips, hints, or even a solution for this?
Here’s the source:
BJSIntegration.zip

[jeromelaban]

Thanks for the report. Your repro does not have all the required files for us to help (it's missing a project).

Have checked this how-to as well: https://platform.uno/docs/articles/interop/wasm-javascript-1.html ?

[VRage]

Hi @jeromelaban That's right somehow VisualStudio added a project somewhere unexpected. Here is the whole solution:
BJSIntegration.zip

I've check all of the javascript integration docs, but it's not that easy if you try to integrate js as a uno library. For example it is not documentated which package is needed to get it run as a library project, also is it enough to declare the javascript files as embedded resources or is the folder WasmScripts mandatory?! Further more, i get a weird error trying the declare the javascript file as embedded resource on Build Action:

Which let me question if this javascript file is intergrated properly.

[jeromelaban]

Thanks for the update.

Yes, the WasmScripts folder location is mandatory at this time, yes, but you don't need to set it as EmbeddedResource in the latest versions of Uno (when the csproj uses Sdk="Uno.Sdk", it's done automatically. We'll update the docs to mention it.

[Jani-z]

Thanks for the update.

Yes, the WasmScripts folder location is mandatory at this time, yes, but you don't need to set it as EmbeddedResource in the latest versions of Uno (when the csproj uses Sdk="Uno.Sdk", it's done automatically. We'll update the docs to mention it.

With the Version "6.2.36" it is STILL mandatory to declare the Javafile as <EmbeddedResource> in the csproj by hand. Also when i try to add files to the assets folder. It is not copied to the outputfolder like mentions in . No matter if its in the Library project or the uno wasm project. If i place them in the wwwroot folder they are available.

[jeromelaban]

This discussion was the topic of this livestream: https://youtu.be/_uAy1CvSkYo?t=534.

EmbeddedResource is required for all additional .js files in the WasmScripts folder, and it's required for now to import Babylonjs using an explicit require call inside of an async function:

// Load the Babylon.js library asynchronously
await new Promise((resolve) => {
    require([`babylonjs`], c => { window.BABYLON = c; resolve() });
});

[VRage]

Thank you once again to take a look at this problem. It was very interesting how you continuously solved this problem, while streaming live.
One more thing about assets: as you mentioned, they are loaded lazily.
Is it possible to force the loading of assets?
When a scene in BabylonJS requires a GLB file, there is basically no way to put it into the library’s assets folder, it is available on startup.

[jeromelaban]

Thank you once again to take a look at this problem. It was very interesting how you continuously solved this problem, while streaming live.

Glad to know it's of help :)

One more thing about assets: as you mentioned, they are loaded lazily. Is it possible to force the loading of assets? When a scene in BabylonJS requires a GLB file, there is basically no way to put it into the library’s assets folder, it is available on startup.

There's no notion of "force loading" in the sense that to provide the file, you have to make it available to the javascript side of the app. You can either use the StorageFile.GetFileFromApplicationUriAsync in order to download the file as a blob, then pass it to javascript then to BablylonJS, or you can download it using fetch and await for the file to load in order to provide it to BabylonJS.

[VRage]

Something weird i've noticed, tho. If I use ContentControl (like in the docs) instead of ContentPresenter (like in the stream) nothing is shown by BabylonJS even tho it's getting initilized. "BJS - [10:44:38]: Babylon.js v8.23.1 - WebGL2 - Parallel shader compilation" in the console

[jeromelaban]

This is likely caused by the fact that ContentControl uses Vertical/HorizontalContentAlignment Top/Left. Try using stretch.