Field Notes / Research judgment

Research has to earn its place in the system.

By Joey Victorino · November 7, 2026

An interesting idea, a convincing experiment and a useful production change are three different achievements. I want a clear line between them.

John Langford’s The End of AI Frontiers argues for coordinated research, strong baselines, shared infrastructure and a route from discovery to use. He also acknowledges the business appeal of scaling: its returns can be more predictable. Those tensions are useful well beyond frontier-model training.

My question is practical: given a security objective, a limited budget and an existing system, which experiment could change what we decide to build or deploy? The answer has to survive scrutiny from both the researcher who sees a new possibility and the operator who has to carry the pager.

The strongest objections deserve an answer.

The Hacker News discussion sharpens the question. Here are the objections I take seriously, and the discipline each one should impose on the work.

1. Give the alternative its strongest case.

A new approach should compete with a credible alternative. For an AI workflow, I would include the current implementation, a stronger available model with competent configuration, and a simpler deterministic workflow where one is applicable. Compare them on the same workload, under declared constraints. Record tuning effort as well as runtime.

A weak baseline makes an idea look more valuable than it is. An unfairly large tuning budget for the candidate does the same. The decision is whether the change beats the best feasible use of the resources, including the option to keep what already works.

There are two useful comparisons. At a fixed budget, which system produces more acceptable results? At a fixed quality and security requirement, which system costs less? Reporting both avoids turning a tradeoff into a universal victory.

2. Separate a mechanism from a headline.

The Full-bandwidth transformer paper feeds a previous top-layer hidden state back into the next input alongside the token embedding. Its abstract reports experiments at the 1B-parameter scale, with gains relative to specified transformer baselines. ByteDance’s Recycler publication also investigates hidden-state feedback and reports parameter-efficiency gains.

These are reasons to investigate a mechanism. They do not establish a universal reduction in production cost. Parameter count, training tokens, arithmetic operations, accelerator utilization, memory traffic and wall-clock latency measure different things. A gain on one axis can move a bottleneck onto another.

Before transferring a result, I would ask: which tasks improved, what changed in the training recipe, how competitive was the baseline, what did the ablation isolate, and what did the hardware actually spend? Then I would test the workload I intend to operate. A small-model result becomes a hypothesis at a different scale until the evidence supports the transfer.

Hidden computation raises a separate security question. A fluent explanation is not an execution trace, and a hidden state is not a trustworthy audit record. I would preserve externally checkable evidence of inputs, permissions, tool calls, policy decisions and resulting state. A system can become more efficient while becoming harder to diagnose; those are separate outcomes to measure.

The discussion also raises separating reasoning from stored knowledge. Moving facts into retrieval could reduce what a model must carry, but it introduces dependencies: source quality, access controls, freshness, availability and hostile retrieved text. The useful test is whether the complete system improves under those conditions. A smaller model with more tools has shifted the security boundary.

3. Price the accepted result.

A lower cost per model call can be erased by more retries, more human review or fewer useful answers. I start with a unit that the business can use: an accepted result under a fixed rubric.

If an attempt costs c and the accepted fraction is q, the simple average cost per accepted result is c / q. That ratio is meaningful only when attempts and acceptance are counted consistently. For a change requiring research and integration, add those fixed costs and the incremental cost of keeping it working.

At 100,000 accepted results per month, the candidate saves about $3,243 per month after its added upkeep. It pays back the initial $30,000 in about 9.25 months. Over a 12-month horizon the net benefit is about $8,912. If demand falls, the candidate’s yield deteriorates, or adoption takes longer, that conclusion can reverse. The interactive calculation lets you change each assumption.

This is a screening model. It assumes stable yields, adequate capacity and linear costs. It leaves out financing, discounting, latency penalties and residual incident losses. Security and operating requirements remain gates; an attractive average does not waive them.

4. Design a test that can say no.

Suppose I am evaluating a change to an agent that reads a security ticket and proposes a bounded action. I would define the allowed action, the required evidence and the forbidden effects before testing. The agent’s own explanation would not decide whether it succeeded. A separate checker would inspect the output and the relevant tool records.

I would lock a representative evaluation set before tuning, keep development cases separate, and identify duplicates or common templates that could leak information between them. Each candidate gets the same task distribution and an explicit inference budget. Repeated runs characterize sampling variation. Paired comparisons on the same tasks help distinguish a real difference from a change in task difficulty.

Then I would run a separate adversarial set: misleading retrieved text, ambiguous authority, stale permissions, incomplete evidence and attempted cross-task contamination. Its purpose is to find failure mechanisms. A high attack success rate on a deliberately hostile set is not an estimate of the event rate in normal traffic. Both results matter, but they answer different questions.

The decision record should include accepted-task yield, prohibited actions, abstentions, reviewer workload, end-to-end cost and tail latency. Report important task slices separately. An average can hide a regression in the very workflow that makes the system consequential. Timeouts and missing traces belong in the denominator and the failure analysis.

Finally, remove or vary the proposed change in a controlled ablation. If the improvement survives without the supposedly important component, the explanation needs to change. Publish the unfavorable result too.

5. A clean test run still leaves uncertainty.

Zero observed failures is a count, not a proof of impossibility. Under independent, representative trials of a fixed system, with complete event detection, the probability of observing zero events at event probability p is (1 − p)ⁿ. Solving for a 5% tail probability gives a one-sided 95% upper confidence bound of 1 − 0.05^(1/n).

With a fixed sample size and zero events in 100 trials, that bound is about 2.95%. With zero in 3,000, it is about 0.0998%. The difference matters if someone is about to claim a rate below 0.1%.

Those numbers are conditional on the sampling model. Correlated cases, a changing system, selective reporting or incomplete detection invalidate a simple interpretation. The bound also concerns a specific event definition and distribution, not every possible harm. Repeatedly testing until the bound happens to pass requires a sequential method; the fixed-sample calculation does not justify that stopping rule. I would keep this calculation separate from adaptive red teaming and explain the remaining uncertainty to the decision owner.

6. Build a research option with a stopping rule.

A research program can create value even when its first idea fails. A trustworthy evaluation harness, a useful dataset or a measured constraint can make the next decision cheaper. That does not give a project an unlimited budget.

I would stage the commitment: reproduce the baseline, run the smallest discriminating experiment, test the candidate under operational constraints, and expand only when the result earns the next stage. At each stage, record what would stop the work. If a supported negative result arrives early, keeping the baseline is a completed decision.

The value of another experiment depends on whether it can change the action and how much that change is worth. If every plausible result leads to the same deployment decision, more measurement may have little immediate value. If a cheap test could prevent an expensive integration mistake, it can be worth doing even when the underlying idea is unlikely to win.

7. Make the handoff part of the design.

Shared code is valuable when it lets someone else reproduce, challenge and extend a result. I want a common experiment format: code revision, model and data versions, environment, configuration, seeds, raw outcomes, costs, failures and the decision attached to them. Access controls and data rights still apply. Shared infrastructure should reduce duplication without turning private evidence into public material.

The product owner and the research owner should agree on acceptance criteria before the experiment. A successful result needs a maintainer, a rollout boundary, a monitoring plan and a rollback trigger. Research that cannot reach a real system may still be scientifically useful; it should be funded and described with that purpose in view.

For a small practice, this also defines a sensible scope. I can make a bounded claim testable, build the evaluation around it, and expose the result to independent inspection. Larger training programs require the relevant compute, data, specialists and organizational support. The credibility comes from showing the work at its actual scale.

That is how I want to demonstrate capability: a useful question, a strong alternative, an experiment that can fail, and a decision another person can inspect.

Work through the example