How do I render text with GPU? Do I render text on texture with CPU and then send that texture to GPU? Do I vectorize it and draw millions of tiny triangles to approximate those vectors? It's interesting for me to explore WebGPU as a canvas for app development, but it's not obvious to me how to draw text and text is everywhere.
One of the most exciting features of WebGPU, especially over WebGL, is the ability to run compute shaders. That opens up an entire class of 2D vector graphics rendering techniques using compute shaders, including Vello. It's still in early stages, but the current code does include text rendering. There's a very rough demo at https://levien.com/vello-demo (the demo doesn't do variable fonts but the code can), and we'll be polishing that in time for Chrome M113 going stable.
There are no active plans to do this, but it's all open source and there are tools such as cbindgen, so if there is sufficient interest there's a good chance it will happen.
WebGL 2.0 is kind of PlayStation 3 like in capabilities, that we seldom see that in action besides shadertoy, Google Maps/Earth, and 360° views on ecommerce sites is another matter.
Interesting. Do you have any more context around this? It sounds uncharacteristic of Google to do so given their usual stance on browsers being all things to all people.
Most web games use the dom as a layer over top of the 3D canvas and then use 3D to 2D projections to align the elements. This allows you to use regular HTML and CSS for a lot of text elements.
Also, you can render and rasterize fonts + dynamic text to a texture and UV map it to a quad in 3D space if you need to. This is pretty inexpensive and easy to do as well.
Lastly, the most difficult but performant option is using a shader to compute the font rendering. This is basically moving the font raster from CPU to GPU and there are various shader examples for this in GLSL (popular shader format used by WebGL/OpenGL) that you can use.
From personal experience, if you're trying to make a game that cares about its frame rate, do not ever use the DOM for anything, ever. It routinely takes several frames to layout a single absolutely positioned element.
I recommend the second option. Rasterize your font with stb_truetype to a bitmap, and draw quads yourself
It wouldn't matter if the DOM layout engine ran on a separate thread, but it runs on the main thread, so it blocks your games update loop for a few frames every time you shoot, or whatever.
Now, that's not to say it has to be that way; real engines have a separate render thread that would sort of solve that problem. And they run the game update code in parallel, and all kinds of other stuff. But, by the time you have all that stuff, you can probably rasterize a font and draw quads ;)
But don't take my word for it. Maybe it's better now. Make a thing that moves some divs around in a requestAnimationFrame callback and take a look at the profile. I bet 2% of the time the layout engine takes 30ms+ to do basically nothing
The easiest is just to use a html canvas element and draw text to it and then upload that as a texture map. This can handle kerning, ligatures, emojis, different fonts and languages. Writing your own font layout tool is horribly difficult and only do it if you must.
If you have a couple of different sizes of texts you can render out each character into a sprite map and render a quad per character which is probably the fastest and works for most 2D UIs. You can prerender ahead of time or have some runtime code that renders the characters with a given font using the 2D canvas api. You will need also need to store the characters sizes somewhere unless it’s a monospace font
If you need to scale the text arbitrarily or render it in 3D space you can look at multi channel signed distance fields (MSDF) which also renders out each character but encodes some extra data which makes them scalable. It has pretty good results
It depends on the quality and performance requirements, but you are basically on your own (e.g. the browser's integrated text rendering won't be of much help unfortunately - at least if performance matters). You need to bring your own font data and 'pre-rasterize' glyphs either on the CPU or GPU into a font cache texture, or render directly on the GPU using an approach like Slug or Pathfinder.
Of course there's also always the option to overlay DOM elements over the WebGPU canvas if you just want to render some 2D UI on top of the 3D rendering.
Note that this approach is useless on its own if your app needs to render user-created text because nowadays everybody expects emojis to work and look roughly the same as on Apple’s platforms, which means detailed multi-colored layered vector shapes that SDF font renderers can’t handle.
The solution I've used for this, which was a bit tedious but not that hard to implement, was sprites for the icons/emojis and SDF for everything else. I'm sure there would be other solutions too, like layering. So it's not useless, you just need to get creative to overcome the limitations. Like everything else in 3D graphics.
Yes, I agree — I did write “useless on its own” to highlight that you need to extend the SDF rendering model with a different one to handle actual user-generated text.
> Do I vectorize it and draw millions of tiny triangles to approximate those vectors
That is becoming feasible, using something called mesh shaders. The millions of tiny triangles will only be created on the fly inside the GPU, you will just send the vector geometry.
This is true, but there is a lot more to the story. For one, WebGPU does not (yet) support mesh shaders, though it may later as an extension. For two, consider a glyph such as "o" that has two contours. Real triangulation generates a mesh that only generates triangles between the outer and inner contours, and mesh shaders aren't good at that. There are techniques compatible with mesh shaders (cover and stencil) that draw twice, incrementing and decrementing a winding number stored in the stencil buffer (see contrast renderer[1] for a clean modern implementation), but it does require nontrivial tracking on the CPU side, and can result in lots of draw calls to switch between the cover and stencil stages unless sophisticated batching is done. (The outer contour is drawn as a disc with winding number +1, and the inner contour is drawn as a smaller disc with winding number -1, so the inner part ends up as 0)
Compute shaders avoid all these problems and work on WebGPU 1.0 today.
> Real triangulation generates a mesh that only generates triangles between the outer and inner contours, and mesh shaders aren't good at that.
I'm a bit confused. Can't you send the shape of O as a low res rough band (a closed wide loop) and enhance it in the mesh shader? This is how previous generation tessellation shaders and sub-division surfaces worked.
You might be able to do something along those lines, but I know of no font / vector 2D renderer that does so. There is a lot of literature[1] on this, so I suspect if doing things similar to subdivision surfaces were viable for font rendering, it would have been explored by now.
You could convert the paths in a font file into a series of triangles using some vertices as a control points for quadratic bezier curves. Then some code for laying out the glyphs in a line. Much of this could be done on a shader I think.
