There is an oddity in the behavior of ReactDom.render() and ReactDom.hydrate(). Both have similar looking function signatures, so you’d expect them to handle errors in about the same way. And while they both will trigger error boundaries correctly, hydrate() will throw an exception if there was a problem rendering, while render() will not.
Hope this helps someone else.