Skip to main content

One post tagged with "aspnetcore"

View All Tags

Streaming LLM answers to the browser with Server-Sent Events

· 9 min read

A model writes its answer one token at a time. If your API waits for the last token before it returns anything, the user stares at a spinner for the whole generation, even though the first sentence was ready long before.

Streaming fixes that, and in ASP.NET Core it is now a few lines of code. The interesting parts are around those lines: which transport to use, what happens when the user closes the tab, what proxies do to a long response, and what "resume the stream" actually asks of your server.