Posts tagged #Local inference
-
The rule, the watermark and the pen
From August 2026, text generated by Anthropic's models will carry a statistical watermark, in compliance with Article 50 of the EU AI Act. How you watermark a sequence of words (logits, softmax, entropy: the signal lives where the model is undecided), what that signal actually says and how well it holds up: a paraphrase dissolves it, a positive can be forged, open models run locally never apply it; with a test replicated locally, a watermark inserted, measured and removed on a half-billion-parameter model. And the question the watermark does not answer: a text is always someone's work of synthesis, and what makes it reliable is the process, the sources, the responsibility of whoever signs it, not the pen it was written with. Do the watermark, and the rule that requires it, add anything to the reliability of content, or do they risk creating misunderstandings? Where it speaks of editorial responsibility, the rule itself suggests the answer.
-
A journey inside the machine that answers: how the bytes move in inference
What happens, physically, when a language model answers, and what changes if the one answering is a single machine, at home? A path through the principles of how it works, from attention to the craft of inference, with its two phases, its organization of the data, the techniques that make it cost less, read along one thread: where the bytes are and how much it costs to move them. In the background, an idea: local inference, as a choice and as a possibility, already matters today and will matter more and more. Keeping it concrete, an engine written in C that brings a model of hundreds of billions of parameters to run locally: DwarfStar, with which antirez shows how far one can push, and that this road can be travelled.