Streaming LLM answers to the browser with Server-Sent Events
· 9 min read
A model writes its answer one token at a time. If your API waits for the last token before it returns anything, the user stares at a spinner for the whole generation, even though the first sentence was ready long before.
Streaming fixes that, and in ASP.NET Core it is now a few lines of code. The interesting parts are around those lines: which transport to use, what happens when the user closes the tab, what proxies do to a long response, and what "resume the stream" actually asks of your server.
