← logs/

Foundation models: a field map

I work in three areas that most people treat as separate specialties: post-training, model editing, and inference. I treat them as one loop. You train behavior with verifiable rewards, you maintain knowledge with targeted edits, and you serve both under latency and cost constraints. Each stage has the same bottleneck: cheap, reliable signal. A verifier for training, an attribution probe for editing, observability for routing.

This post is the map I wanted before diving into any of the deep-dives below. For each of the three areas it covers what the field is trying to do, what is actually hard right now, and where my work sits. It ends with what I would watch and what I would build, then a linked index of the individual deep-dives.

The loop, not the disciplines

post-training          model editing          inference
verifiable reward      targeted updates      paged KV, routing
RLVR / GRPO family     weights vs context   test-time compute
        |                     |                     |
        v                     v                     v
     behavior             knowledge          cost per useful token

The interesting work happens at the seams. A verifier built for RLVR doubles as a routing signal at serving time. An editing failure mode (forgetting) is a training failure mode (catastrophic interference) wearing different clothes. Test-time compute changes what post-training has to optimize for. I keep returning to the same three seams in my own work.

Post-training and verifiable rewards

What the field is trying to do: take a base model and turn it into a system that reasons, follows tool protocols, and improves whenever its output can be checked by a program. The check is the load-bearing part. DeepSeek-R1 showed that reinforcement learning against verifiable outcomes can elicit reasoning without human-labeled reasoning trajectories, and that behaviors like self-reflection and verification emerge rather than being taught. DeepSeekMath supplied the algorithm that made this affordable: GRPO, a PPO variant that trades the learned critic for a group-relative baseline, cutting the memory overhead of policy-gradient training and changing the field's default cost curve.

The GRPO family is now a small ecosystem, and my deep-dive on the GRPO family tree walks through the variants. The RLVR side has its own cracks, cataloged in RLVR and its cracks. What is hard right now, in my reading:

  • Verifier engineering. The reward is only as good as the checker. Programmatic verifiers are cheap to run and immune to reward hacking, but writing one that gives partial credit without leaking shortcuts is a design discipline of its own. I write about this in verifier engineering.
  • Collapse. Large-scale RLVR runs develop entropy collapse and verbosity collapse. DAPO is the best open account of both: it open-sources a complete system that hits 50 points on AIME 2024 from a Qwen2.5-32B base, and the four techniques that make it work, including decoupled clipping to hold off entropy collapse and dynamic sampling so rollouts are spent only on prompts that still carry gradient signal.
  • Sample efficiency. RLVR looks data-hungry; the One-Shot RLVR result says it is not, at least on math. The short version is one of the two close reads below; the long version is in the RLVR deep-dive.
  • Distillation. The R1 report's second stage distilled long reasoning traces into smaller models, and everyone now treats trace distillation as a standard move. It deserves more scrutiny than it gets; see on-policy distillation.

Close read: GRPO's origin paper

DeepSeekMath's real legacy is not the data pipeline, it is the algorithm: GRPO computes advantages within a group of sampled completions instead of from a learned value function, which removed the critic along with its memory footprint and made RLVR cheap enough to run everywhere. My read: GRPO won because it is simple, not because it is principled — the group is a Monte Carlo baseline, and most of the RLVR literature since is a series of patches on that approximation. The variant-by-variant walk is in the GRPO family tree.

Close read: One-Shot RLVR

One-Shot RLVR asks the efficiency question directly: what if the dataset is a single example? The answer rewired how I think about data volume in RLVR, and I treat the result as a probe: if one example cannot move a model under your verifier, the reward is the constraint. The numbers, and the cracks that sit next to this result, are in RLVR and its cracks.

Model editing

What the field is trying to do: fix facts and behaviors in a deployed model without retraining. Two camps. Weights-side methods locate the relevant parameters and patch them: ROME, MEMIT, AlphaEdit and their descendants. Context-side methods leave weights alone and route through external memory or in-context retrieval. The real requirement, the one the field keeps missing, is sequential editing: models are edited continuously, thousands of times, while staying accurate on everything they already knew. I cover both sides in sequential knowledge editing and weights vs context memory.

What is hard right now:

  • Forgetting at scale. The next close read is the canonical demonstration.
  • Attribution. If you cannot prove the answer came from the edit rather than from retrieval or priors, you cannot ship edits with confidence. This is the same verifiability problem as RLVR, one level up.
  • Evaluation that lies. Benchmarks that measure one or few edits cannot see sequential degradation, and many reported "state of the art" results in editing do not hold past a few hundred edits.

Close read: the forgetting wall

Model Editing at Scale leads to Gradual and Catastrophic Forgetting (Gupta et al., 2024) is the paper I hand to anyone who thinks editing is solved. It evaluates ROME and MEMIT, the two canonical locate-then-edit methods, under sequential editing: hundreds of edits applied one after another to the same model. The result is that edited models continually forget previously edited facts and lose downstream task ability, in two phases: an initial gradual, progressive forgetting, then an abrupt catastrophic phase. The gradual phase makes editing less effective as the edit count grows; the catastrophic phase caps scalability entirely.

My read: this reframes the field's evaluation culture. Single-edit reliability, generalization, and locality are necessary but nowhere near sufficient; what matters is the forgetting curve over thousands of edits, plus what happens to unrelated knowledge. The paper's call to evaluate with scalability in mind has, slowly, become the default for serious work: AlphaEdit (ICLR 2025) projects each update into the null space of preserved knowledge precisely so sequential edits stop disrupting what came before, and MEMOIR (NeurIPS 2025) sparsifies edits into a residual memory so long edit streams stop overwriting each other. Both papers exist because the sequential curve is now the metric that counts. Everything I build in this area treats it as the headline metric.

Inference and serving

What the field is trying to do: serve models whose workloads no longer look like single-shot Q&A. Agent workloads generate long contexts, tool-calling loops, and reasoning traces of wildly variable length. The cost of a request is now a distribution, not a number.

The anchor result is vLLM: PagedAttention treats the KV cache like virtual memory, cutting fragmentation and allowing cache sharing, with 2-4x throughput over state-of-the-art systems at the time (FasterTransformer, Orca) at the same latency. Serving became a memory-management problem, and every serious serving stack since is a descendant. My notes on the current landscape are in inference systems for agents.

The second thread is test-time compute. Snell et al. showed that compute-optimal scaling of test-time search beats a best-of-N baseline by about 4x in efficiency, and that a small model with test-time compute can outperform a 14x larger one in FLOPs-matched comparisons on easier and intermediate prompts, if the budget is allocated per prompt rather than uniformly. s1 made the mechanics embarrassingly simple: 1,000 curated questions, SFT on reasoning traces, and budget forcing, where you terminate or extend the model's thinking mid-generation (appending "Wait" makes it re-check). s1-32B beats o1-preview by up to 27% on competition math. The boundary between what you train into the model and what you compute at request time is now a real engineering decision; I dig into where it breaks in test-time compute limits.

The third thread is the cost curve itself. Muon (a matrix-orthogonalization optimizer with weight decay and per-parameter update scaling) reports roughly 2x compute efficiency over AdamW under compute-optimal training, demonstrated on Moonlight, a 3B/16B MoE trained on 5.7T tokens. If optimizer gains hold at scale, they change the economics of both training and of retraining after editing. More in the muon optimizer revival. Architectures get their own map in architectures 2026, and the diffusion side in diffusion language models.

What is hard right now: batching for reasoning-heavy traffic (long generations wreck naive schedulers), routing between a frontier model and specialist small models, and the train-versus-test-time compute tradeoff, which has no agreed accounting. I know these three from one product in detail: at PiPlan.ai my routing layer sends routine agent steps to a self-hosted multi-GPU vLLM node and escalates complex planning to frontier APIs, and whether that split pays for itself is decided by exactly the batching, routing, and budget questions above.

My position

Three bets, stated plainly.

First, verifiable rewards are the highest-leverage input in the whole loop. R1 showed the behavior emerges; One-Shot RLVR showed the data can be tiny; DAPO showed the failure modes are fixable. The remaining bottleneck is verifier quality, so that is where I spend my time. A programmatic verifier that runs in milliseconds is worth more than a larger model.

Second, sequential editing is a serving problem before it is a research problem. The wall is not the math of the update, it is proving at scale that the update holds while everything else stays intact. That means attribution probes, checkpointed edit history, and regression suites on the forgetting curve. The field will converge on tooling before it converges on theory.

Third, routing is where post-training and serving merge. Once you have small verifier-trained specialists, the question is when to call them instead of the frontier model. That decision needs a signal, and the signal is the same verifier you trained with. I would build the router around the verifier and let latency heuristics break ties.

Concretely, the next artifacts: a training loop that treats the falsification control as a mandatory artifact rather than an optional extra, so every shipped checkpoint comes with proof that the gain was not formatting; an editing harness whose eval is the sequential forgetting curve; and a serving layer where per-prompt test-time budget is a first-class knob, set by verifier confidence.

Where my work sits

The ties, stated at the level I can defend.

  • Post-training: VerifierForge trains small models with GRPO against a programmatic verifier, with a falsification control. The demonstrated loop is an NL-to-SQL workload: 50 training examples, selection on a frozen 60-question held-out exam (pass@1 from 58.3% to 78.3%), and a deliberately imperfect random-reward reference whose training-pool monitor ended at 0.40 while the verifier-rewarded monitor ended at 0.80 — with the repo itself calling it a falsification check, explicitly short of a strict causal experiment. The verifier executes each generated SQL against the frozen schema and compares result sets, so the reward is an executable check, not a judge model.
  • Reproductions: I keep paper reproductions, GRPO and One-Shot RLVR among them, in my HoReN-paper-reproduction repo. I do not trust a training claim until I have run it on hardware I control.
  • Model editing: my first-author preprint on sequential model editing is under review, so I will not oversell it here; the write-up lives in sequential knowledge editing. The reproduction harness above also runs GRACE, the discrete key-value adaptor method for lifelong editing, through the same codebase, which is where my forgetting-curve tooling comes from.
  • Serving: at PiPlan.ai I run a self-hosted multi-GPU vLLM node with dynamic routing. The lesson I keep applying: when you control the serving layer, the verifier and the router can share one signal, and small specialists stop being a research topic and become an ops decision.

The map

The deep-dives slot in beneath this overview, one per node of the map:

Pick any node; the loop is the point: train behavior you can check, edit knowledge you can attribute, and serve both on a cost curve you understand.

Sources linked in this post were fetched and verified.