You can try these examples how it might improve user experience of WebGL applications.
There are online resources, which describe how to use OffscreenCanvas, even with React (@react-three/offscreen). But the technology is still new, and it is still uncommon to think of frontend applications as of multithreaded ones. Most of tutorials found online would assume that you have an isolated WebGL application, meaning that it is not exchanging information with the main thread.
From my experience, I can tell - this is a very unlikely situation. We have implemented multiple complex applications with React and WebGL at SABO. And every time both parts rendered their views from the same data!
So, inpractice, you will search for a way to synchronize the application state between multiple threads.
This article explores the more likely scenario, where both React DOM and WebGL parts of the application need read/write access to the application state.
Imagine a feature for inspecting and configuring a 3D object. It might look like this:
By movingWebGL to a worker we might achieve following:
Next, let us describe the capabilities:
It should be obvious by now that WebGL code should be notified when we change the attribute in the panel and the other way around.
In general, you cannot access memory of one thread from another thread. This is a design choice to ensure thread-safe operations. Workers are supposed to communicate via messages.
That is why the most straightforward approach would be to implement a message-based communication between two workers. But before we go to that, I would like to mention a more advanced approach, which would be to store application state in SharedArrayBuffer.
SharedArrayBuffer allows shared access and supports atomic operations from different contexts(main thread or worker) without copying it. But data structure itself is used to represent a generic raw binary data buffer. For most of the time, it is too low level. Nevertheless, this is the way how a Manifold multithread engine is doing it.
SharedArrayBuffer is the most memory and speed effective, but much less user-friendly way. Even with proxy objects like bitECS (that allow you work with data as JS objects) will get you nowhere near to beloved JSON structures.
Now, let's get back to message-based communication and oneof workable solutions.
The core idea is to have a shared application store. Each part will have its local store and we will write a thin synchronization layer between via Broadcast Channel API. In the end, neither of the parts should be worried about the other one, synchronization will be seamless.
First, I have chosen valtio to store application state and soon you will see why. If you are not familiar with it, it is a library offering mutable Proxy-based reactive state management.
This is basic store I used:
import { proxy } from 'valtio'
export const store = proxy({
name: 'Suzi',
color: '#FFA500'
})
I then connected it to my inspector panel in React DOM:
import { useSnapshot } from 'valtio'
import { store } from './store'
import { Layout, Inspector } from './dom'
import { Canvas } from './webgl'
const setName = (name: string) => (store.name = name)
const setColor = (color: string) => (store.color = color)
export const App = () => {
const snapshot = useSnapshot(store)
return (
<Layout>
<Canvas />
<Inspector
name={snapshot.name}
onNameChange={setName}
color={snapshot.color}
onColorChange={setColor}
/>
</Layout>
)
}
And as well to my 3D scene:
import { store } from '../store'
import { invertColor } from '../utils'
import { Stage, Suzi, Label } from './components'
// example write operation
const invertAttributes = () => {
store.color = invertColor(store.color)
store.name = `Inverted ${store.name}`
}
export const Scene = () => {
const snapshot = useSnapshot(store)
return (
<Stage>
<Suzi color={snapshot.color} onClick={invertAttributes} />
<Label>{snapshot.name}</Label>
</Stage>
)
}
I have chosen valtio because it provides nice serializable patches to the listeners that you feed to subscribe function.
Let us use it to notify other threads about changes that were made on our side.
import { subscribe } from 'valtio'
// const store = ...
export function syncStore() {
const channel = new BroadcastChannel('sync')
subscribe(store, (ops) => {
channel.postMessage({ ops })
})
}
Step by stepon example:
Valtio operation consists of:
There are more, but we do not need them now.
Now, let us handle messages with ops (operations) when received from the other thread.
channel.onmessage = (event) => {
event.data.ops.forEach(([type, path, newValue]) => {
switch (type) {
case 'set':
setByPath(store, path, newValue)
break
case 'delete':
deleteByPath(store, path)
break
}
})
}
To apply the newvalue to a sub-tree of our store we introduce utility functions that traversethe store object according to path.
function setByPath(obj: any, path: (string | symbol)[], value: any) {
const last = path.pop()
let current = obj
for (const p of path) {
current = current[p]
}
current[last!] = value
}
function deleteByPath(obj: any, path: (string | symbol)[]) {
const last = path.pop()
let current = obj
for (const p of path) {
current = current[p]
}
delete current[last!]
}
That’s about it! Oursynchronization logic is ready. Call the function on the main thread and inrender worker to start synchronizing.
Now, whenever anything changes in store on one thread, the other one will receive an array of operations and apply it to its local version of the store.
This solution sacrifices memory for easier experience and will not be suitable for every scenario, but it solves the problem elegantly and easily.
You can testyourself the deployed version here on Netlify or get the code at evstinik/multithreaded-react-webgl-example.
