<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>LoRA adapters &#8211; &gt; Jeff Blog</title>
	<atom:link href="https://blog.jefersonjeronimoai.online/tag/lora-adapters/feed/" rel="self" type="application/rss+xml" />
	<link>https://blog.jefersonjeronimoai.online</link>
	<description></description>
	<lastBuildDate>Sun, 15 Nov 2026 21:33:25 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	

<image>
	<url>https://blog.jefersonjeronimoai.online/wp-content/uploads/2026/01/cropped-icons8-technology-100-150x150.webp</url>
	<title>LoRA adapters &#8211; &gt; Jeff Blog</title>
	<link>https://blog.jefersonjeronimoai.online</link>
	<width>32</width>
	<height>32</height>
</image> 
	<item>
		<title>Model-Specific Fine-Tuning Strategy: Smaug Lessons</title>
		<link>https://blog.jefersonjeronimoai.online/2026/11/15/model-specific-fine-tuning-strategy-smaug/</link>
		
		<dc:creator><![CDATA[Tech News]]></dc:creator>
		<pubDate>Sun, 15 Nov 2026 21:33:25 +0000</pubDate>
				<category><![CDATA[AI Agents]]></category>
		<category><![CDATA[Agentic coding]]></category>
		<category><![CDATA[AI model evaluation]]></category>
		<category><![CDATA[LoRA adapters]]></category>
		<category><![CDATA[Model adaptation]]></category>
		<category><![CDATA[Model evaluation]]></category>
		<category><![CDATA[Multimodal reasoning]]></category>
		<guid isPermaLink="false">https://blog.jefersonjeronimoai.online/?p=6059</guid>

					<description><![CDATA[Intro: Why a Model-Specific Fine-Tuning Strategy Matters Fine-tuning is often described as if it were a universal recipe: select a capable base model, add domain data, train it on examples, and expect…]]></description>
										<content:encoded><![CDATA[<p><img height="150" sizes="(max-width: 500px) 100vw, 500px" srcset="https://blog.jefersonjeronimoai.online/wp-content/uploads/2026/01/banner_final0000.webp 1400w, https://blog.jefersonjeronimoai.online/wp-content/uploads/2026/01/banner_final0000-300x32.webp 300w, https://blog.jefersonjeronimoai.online/wp-content/uploads/2026/01/banner_final0000-1024x110.webp 1024w, https://blog.jefersonjeronimoai.online/wp-content/uploads/2026/01/banner_final0000-768x82.webp 768w" width="1400" decoding="async" 
  src="https://blog.jefersonjeronimoai.online/wp-content/uploads/2026/01/banner_final0000.webp"
  alt="Diagram comparing Smaug models and their specialized fine-tuning strategies for agentic AI workloads"
  style="width:500px; height:auto; display:block;"
></p>
<hr/>
<h2>Intro: Why a Model-Specific Fine-Tuning Strategy Matters</h2>
<p>Fine-tuning is often described as if it were a universal recipe: select a capable base model, add domain data, train it on examples, and expect better results. In practice, the outcome depends heavily on the model being adapted, the capability being targeted, and the operating environment in which the model will work.</p>
<p>That is the central lesson behind a <strong>model-specific fine-tuning strategy</strong>. A model that struggles with long-context tool use may need a different intervention from one that needs stronger instruction following or more reliable coding behavior. Applying the same training method to all of them can produce uneven gains, or even reduce performance in areas where the base model was already strong.</p>
<p>The Smaug AI model family illustrates this shift. Rather than treating fine-tuning as a generic upgrade, its creators adapted three different open-weight models for distinct agentic workloads. The results show why transfer learning for LLMs increasingly requires diagnosis, specialization, and production-oriented evaluation.</p>
<h2>Smaug AI Model Results Point to a New Adaptation Question</h2>
<p>The important question is no longer simply, “How much does fine-tuning improve the model?” It is: <strong>Which capability does fine-tuning improve, and why does that improvement appear in this model?</strong></p>
<p>Smaug Flash, Smaug Mini, and Smaug Agentic were trained using a shared philosophy involving human-curated agent traces and synthetic examples derived from difficult cases. Yet their gains were different. Smaug Flash showed especially strong improvements in agentic coding, automation, and repository-level tasks. Smaug Mini concentrated its gains in instruction following, multimodal reasoning, and job completion. Smaug Agentic emphasized endurance and stability during long-horizon coding workflows.</p>
<p>This is similar to tuning three vehicles for different terrain. Better tires may transform an off-road vehicle but provide little value to a racing car already optimized for a track. Likewise, a fine-tuning recipe should target the base model’s actual weaknesses rather than assume every model has the same capability gaps.</p>
<h2>What This Guide Explains About Transfer Learning for LLMs</h2>
<p>This guide explains how <strong>transfer learning for LLMs</strong> adapts a pretrained model, where parameter-efficient fine-tuning fits into the process, and why LoRA is useful for some model families.</p>
<p>It also examines the Smaug fine-tuning recipe comparison, including the differences between LoRA adaptation, reasoning-preserving supervised fine-tuning, and dense-model adaptation. Finally, it connects benchmark results to production concerns such as task completion, latency, cost, reproducibility, and behavioral stability.</p>
<p>The practical conclusion is straightforward: the best fine-tuning strategy is determined by the relationship between a model, a workload, and a measurable capability gap.</p>
<h2>Background: What Is a Model-Specific Fine-Tuning Strategy?</h2>
<p>A model-specific fine-tuning strategy is a deliberate process for adapting a particular pretrained model to a defined task or operating environment. It considers the model’s architecture, parameter scale, reasoning behavior, context handling, inference cost, and existing strengths.</p>
<p>Instead of asking, “What is the best fine-tuning method?” teams should ask:</p>
<ul>
<li>What does this base model already do well?</li>
<li>Where does it fail under realistic workloads?</li>
<li>Which training method can improve that weakness without damaging existing capabilities?</li>
<li>How will improvement be measured outside a static benchmark?</li>
</ul>
<p>This approach treats fine-tuning as targeted engineering rather than a generic optimization step.</p>
<h2>How Transfer Learning for LLMs Adapts a Pretrained Model</h2>
<p>Transfer learning begins with a model that has learned broad language, reasoning, coding, or multimodal patterns during pretraining. Fine-tuning then adjusts the model for a narrower objective using curated examples.</p>
<p>For agentic systems, those examples may include tool calls, error recovery, API interactions, code edits, terminal commands, document workflows, and multi-step decisions. The model is not merely learning to produce a better answer. It is learning how to maintain context, select actions, interpret tool outputs, and continue toward a goal.</p>
<p>A useful analogy is apprenticeship. Pretraining gives a model general education; fine-tuning provides supervised practice in a specific profession. An apprentice engineer and an apprentice customer-support specialist may have the same academic foundation, but their practice data must reflect different responsibilities.</p>
<h2>Full Fine-Tuning Versus Parameter-Efficient Fine-Tuning</h2>
<p>Full fine-tuning updates every parameter in a model. This can provide broad flexibility, but it requires substantial compute, memory, storage, and operational effort. Maintaining separate fully tuned copies for multiple customers or workloads can also become expensive.</p>
<p>Parameter-efficient fine-tuning, or PEFT, updates only a small portion of the model or adds trainable components while keeping most pretrained weights frozen. This reduces the resources needed to train and store specialized versions.</p>
<p>PEFT methods can be particularly valuable when:</p>
<ul>
<li>The base model is very large.</li>
<li>Teams need multiple task-specific variants.</li>
<li>Training hardware is limited.</li>
<li>Fast experimentation matters.</li>
<li>The original model’s general capabilities should remain intact.</li>
</ul>
<p>The tradeoff is that PEFT does not automatically guarantee the right adaptation. Efficient training still requires the correct data, target layers, rank settings, loss design, and evaluation plan.</p>
<h2>Where LoRA Fits Into the Fine-Tuning Process</h2>
<p>LoRA, or Low-Rank Adaptation, freezes the original model weights and inserts trainable low-rank matrices into selected Transformer layers. Instead of rewriting the entire model, LoRA learns a compact update representing the direction of adaptation.</p>
<p>This makes LoRA useful when teams want a lightweight specialization layer. It is similar to adding a software plugin rather than rebuilding an entire operating system. The original model remains available, while the adapter modifies behavior for a particular task.</p>
<p>LoRA can reduce trainable parameters and GPU memory requirements substantially. However, its effectiveness depends on the task and base model. A low-rank adapter may be ideal for targeted behavior changes, while a more extensive training approach may be necessary when the model must substantially alter its reasoning or tool-use patterns.</p>
<h2>Why AI Model Capability Specialization Changes Training Goals</h2>
<p>Traditional model evaluation often focuses on answer accuracy. Agentic systems require a wider definition of quality.</p>
<p>An agent must:</p>
<ul>
<li>Preserve the original objective across many steps.</li>
<li>Choose appropriate tools.</li>
<li>Recover from tool errors.</li>
<li>Avoid repetitive or runaway reasoning.</li>
<li>Maintain reliable behavior as context grows.</li>
<li>Complete the task within acceptable time and cost.</li>
</ul>
<p>This changes the training goal from “produce a correct response” to “complete a useful workflow reliably.” <strong>AI model capability specialization</strong> reflects this shift by aligning training with the actual job the model must perform.</p>
<h2>5 Benefits of a Model-Specific Fine-Tuning Strategy</h2>
<p>A model-specific fine-tuning strategy provides five major benefits:</p>
<ol>
<li><strong>More targeted gains:</strong> Training focuses on measurable weaknesses rather than broad, inefficient adaptation.</li>
<li><strong>Better capability preservation:</strong> Existing strengths are less likely to be overwritten.</li>
<li><strong>Lower training cost:</strong> PEFT methods can reduce memory, compute, and storage requirements.</li>
<li><strong>Improved deployment fit:</strong> The tuned model can be optimized for context length, latency, tools, or hardware.</li>
<li><strong>Clearer evaluation:</strong> Teams can connect training changes to workload-specific outcomes.</li>
</ol>
<h2>Trend: Fine-Tuning Is Moving Toward Specialized AI Models</h2>
<p>The next generation of AI systems will likely consist less of one universal model and more of coordinated specialized models. A company may use one model for long-context research, another for fast automation, and a third for complex coding.</p>
<p>This does not mean general-purpose models will disappear. Instead, they may become platforms from which specialized systems are derived. The strategic advantage will come from matching the right model to the right workflow.</p>
<h2>Why One Training Recipe No Longer Fits Every Base Model</h2>
<p>Different base models have different architectures, reasoning styles, context behaviors, and capability ceilings. Fine-tuning can expose those differences.</p>
<p>A model with strong native reasoning may benefit from loss masking that preserves its internal deliberation. A smaller dense model may require more direct supervised examples. A model prone to confusion during long tool interactions may benefit from targeted traces focused on context management and recovery.</p>
<p>The same recipe can therefore produce a major gain in one model, a modest gain in another, and regression in a third.</p>
<h2>Smaug AI Model Family and Agentic Workload Specialization</h2>
<p>The Smaug AI model family applies a common training philosophy to three distinct bases and workload priorities. The comparison is valuable because it demonstrates that shared data principles do not imply identical training procedures or results.</p>
<h2>Smaug Flash for Long-Context Tool Use</h2>
<p>Smaug Flash is designed for continuously running enterprise agents. Its fine-tuning emphasizes long-context tool use and sustained interaction, addressing problems such as spinning, confusion, and loss of direction during extended workflows.</p>
<p>Its reported gains include a 14.3-point improvement in agentic coding, from 46.8 to 61.1. It also improved by 13.7 points on AutomationBench and 19 points on NL2Repo-Bench. These results suggest that the training targeted weaknesses closely tied to practical tool use and software operations.</p>
<h2>Smaug Mini for Instruction Following and Automation</h2>
<p>Smaug Mini focuses on multimodal tasks, instruction following, tool use, and job-level automation. Its gains are concentrated in areas such as mathematics, reasoning, language, repository coding, instruction precision, and task completion.</p>
<p>Its results also demonstrate why benchmark interpretation matters. Smaug Mini’s agentic coding score was slightly lower than its base model, while it performed strongly on instruction-following and automation benchmarks. In two categories, the base model remained ahead: Terminal-Bench 2.1 and MMMU-Pro.</p>
<p>This is not necessarily a failure. It may indicate that the training deliberately prioritized different capabilities.</p>
<h2>Smaug Agentic for Long-Horizon Coding Stability</h2>
<p>Smaug Agentic is aimed at long-horizon, frontier-scale coding tasks. Its headline benchmark changes are smaller, but its operational behavior is the main point.</p>
<p>Across 113 agentic coding tasks and more than seven hours of continuous operation, it completed workflows using a median of 78 agent steps per task without infrastructure errors or timeouts. This highlights a capability that ordinary benchmarks may miss: endurance.</p>
<p>For production agents, avoiding failure after dozens of steps can be more valuable than gaining a few points on a short test.</p>
<h2>The Rise of Operational Metrics Beyond Benchmark Scores</h2>
<p>Operational metrics are becoming essential to AI model evaluation. Relevant measures include:</p>
<ul>
<li>Task completion ratio.</li>
<li>Reproducibility across repeated runs.</li>
<li>Cost per agent step.</li>
<li>Latency at long context lengths.</li>
<li>Tool-call accuracy.</li>
<li>Recovery from errors.</li>
<li>Timeout and infrastructure failure rates.</li>
</ul>
<p>A benchmark is like a driving test on a controlled course. Production is a cross-country journey involving traffic, weather, detours, and limited fuel. Both are useful, but they measure different realities.</p>
<h2>Insight: What the Smaug Fine-Tuning Recipe Comparison Reveals</h2>
<p>The Smaug fine-tuning recipe comparison shows that the shared methodology was adapted to each base model rather than applied mechanically. This is one of the clearest signals that future LLM training will be increasingly model-specific.</p>
<h2>Fine-Tuning Recipe Comparison Across Three Smaug Models</h2>
<p>The approaches can be summarized as follows:</p>
<figure class="wp-block-table">
<table>
<thead>
<tr>
<th>Model</th>
<th>Adaptation approach</th>
<th>Primary focus</th>
<th>Key signal</th>
</tr>
</thead>
<tbody>
<tr>
<td>Smaug Flash</td>
<td>LoRA adapters</td>
<td>Long-context tool use</td>
<td>Large benchmark gains</td>
</tr>
<tr>
<td>Smaug Agentic</td>
<td>Reasoning-preserving SFT</td>
<td>Long-horizon coding</td>
<td>Operational stability</td>
</tr>
<tr>
<td>Smaug Mini</td>
<td>Dense-model adaptation</td>
<td>Instructions and automation</td>
<td>Selective specialization</td>
</tr>
</tbody>
</table>
</figure>
<p>The takeaway is that training efficiency and training depth should be chosen according to the base model’s structure and target workload.</p>
<h2>LoRA Adaptation for Smaug Flash</h2>
<p>LoRA provides a practical way to adapt Smaug Flash while preserving the underlying model. For a model intended to handle long-running enterprise workflows, compact adapters can target tool-use behavior without requiring a complete rewrite of all model parameters.</p>
<p>This approach is especially attractive when organizations need multiple specialized deployments or want to iterate quickly on workflow behavior.</p>
<h2>Reasoning-Preserving Supervised Fine-Tuning for Smaug Agentic</h2>
<p>Smaug Agentic uses supervised fine-tuning designed to improve tool behavior while preserving native reasoning. The methodology masks reasoning tokens from the loss, reducing the risk that training will overwrite the model’s internal deliberation for the sake of surface-level compliance.</p>
<p>This reflects an important principle: a model can follow formatting instructions more closely while becoming less capable at solving difficult problems. Strong fine-tuning should improve behavior without unnecessarily replacing the model’s existing reasoning process.</p>
<h2>Dense-Model Adaptation for Smaug Mini</h2>
<p>Smaug Mini applies the broader methodology to a dense 27-billion-parameter base. Its results show why dense models should not be evaluated only through agentic coding scores.</p>
<p>The model’s specialization toward instruction following, multimodal reasoning, and job completion produced meaningful gains even though some benchmark categories declined slightly. The correct question is whether the model became more useful for its intended deployment.</p>
<h2>Why Capability Gains Follow Base-Model Weaknesses</h2>
<p>Fine-tuning gains often appear where a model has the most room to improve. If a base model is already excellent at one task, additional training may produce only small gains. If it struggles with tool coordination or context retention, targeted examples can produce larger improvements.</p>
<p>This resembles repairing a building. Strengthening an already solid wall changes little; fixing a weak foundation can improve the entire structure.</p>
<h2>Benchmark Comparison: Task Scores Versus Production Reliability</h2>
<p>Benchmark scores remain useful for selecting candidate models, but they should not be treated as interchangeable. Each benchmark measures a different slice of capability.</p>
<p>Agentic coding evaluates more than code generation. It may involve planning, editing files, executing commands, interpreting test results, and recovering from failures.</p>
<p>Automation benchmarks emphasize tool use and job completion. Repository-level benchmarks test whether a model can understand and modify a larger software context. Smaug Flash’s gains across these areas suggest that its training addressed connected workflow weaknesses rather than isolated answer patterns.</p>
<p>Instruction-following benchmarks measure precision in complying with user constraints. Multimodal reasoning tests the ability to combine visual and textual information. Job-completion benchmarks evaluate whether the model finishes a broader task.</p>
<p>Smaug Mini’s results show that a model can improve substantially in these areas without improving every coding metric. That is why evaluation must match deployment goals.</p>
<h2>How AI Model Capability Specialization Shapes Evaluation</h2>
<p>Specialization requires specialized evaluation. A coding agent should be tested on repository changes, test execution, and recovery. A document agent should be evaluated on extraction accuracy, citation consistency, and completion time. An automation agent should be measured on tool selection, error handling, and successful job closure.</p>
<p>The future of evaluation will therefore involve test suites built around workflows rather than isolated prompts.</p>
<h2>Forecast: The Future of Transfer Learning for LLMs</h2>
<p>Transfer learning for LLMs is moving toward a more surgical model of adaptation. Teams will increasingly begin with capability diagnostics, select a training method, and validate improvements under realistic operating conditions.</p>
<h2>From General-Purpose Models to Workload-Specific AI Systems</h2>
<p>General-purpose models will remain valuable as foundations, but specialized descendants may offer better cost, latency, and reliability for specific roles.</p>
<p>Organizations could maintain portfolios of open-weight models, each optimized for a different operating point: fast customer service, long-context research, coding, data extraction, or multi-step automation.</p>
<h2>Why Training Data Will Focus More on Difficult Agent Traces</h2>
<p>Easy demonstrations provide limited learning value. The most useful data will increasingly come from difficult traces involving failed tool calls, ambiguous instructions, long contexts, recovery decisions, and objective drift.</p>
<p>Human-curated examples will remain important because they capture judgment. Synthetic data can expand coverage, but it must be generated from genuine failure modes rather than superficial benchmark patterns.</p>
<h2>Production Evaluation Will Outrank Isolated Benchmark Gains</h2>
<p>A model that scores well but fails unpredictably in deployment may be less valuable than a model with smaller benchmark gains and stronger reliability.</p>
<p>Teams should measure whether the model completes tasks and whether it produces consistent results across repeated runs. Reproducibility is especially important for business workflows where unpredictable behavior creates review costs.</p>
<p>Long-context performance must be measured at realistic input sizes. A model that performs well on short prompts may become too slow or expensive when an agent accumulates tool outputs over several hours.</p>
<h2>How Model-Specific Fine-Tuning Strategy May Improve Open-Weight Models</h2>
<p>Open-weight models provide flexibility for private deployment, custom data, and hardware-specific optimization. A model-specific fine-tuning strategy can turn that flexibility into a practical advantage by tailoring the model to a company’s workflows.</p>
<p>Over time, open-weight ecosystems may produce specialized variants for sectors such as software engineering, finance, healthcare operations, and enterprise search. The strongest models will not necessarily be those with the highest general benchmark scores, but those with the best measured fit for a defined workload.</p>
<h2>Call to Action: Build Your Own Model-Specific Fine-Tuning Strategy</h2>
<p>Organizations can adopt a structured process rather than immediately beginning training.</p>
<h2>1. Identify the Base Model’s Capability Gaps</h2>
<p>Establish a baseline using realistic tasks. Test context retention, tool use, instruction precision, reasoning, latency, and failure recovery.</p>
<p>Avoid relying on a single score. A capability profile is more informative than a leaderboard position.</p>
<h2>2. Match the Fine-Tuning Recipe to the Target Workload</h2>
<p>Choose LoRA or another PEFT method when targeted, resource-efficient adaptation is appropriate. Consider deeper supervised fine-tuning when the model needs substantial behavioral change.</p>
<p>Protect capabilities that already work well, especially native reasoning and general instruction understanding.</p>
<h2>3. Compare Models on Task-Specific Benchmarks</h2>
<p>Create evaluation sets that reflect the intended job. Include normal cases, edge cases, long workflows, ambiguous requests, and tool failures.</p>
<p>A model optimized for automation should not be selected solely because it leads on a coding benchmark.</p>
<h2>4. Validate Agent Behavior in Production-Like Tests</h2>
<p>Run extended simulations with realistic context growth, tool latency, API errors, and incomplete information. Measure whether the agent remains aligned with the objective after many steps.</p>
<p>This is where operational differences often become visible.</p>
<h2>5. Track Quality, Cost, Latency, and Reliability Together</h2>
<p>A useful model must balance all four. A small quality improvement may not justify a major increase in inference cost. Conversely, a modest benchmark score may be worthwhile if it eliminates repeated failures and human intervention.</p>
<h2>Conclusion: Choose Fine-Tuning by Model and Mission</h2>
<p>Fine-tuning is becoming less about applying a universal recipe and more about engineering a precise match between a model and a mission.</p>
<h2>The Core Lesson From Smaug AI Model Specialization</h2>
<p>The Smaug AI model family shows that a shared training philosophy can produce different results across different bases. Smaug Flash improved strongly in long-context tool use and agentic coding. Smaug Mini specialized in instruction following, multimodal reasoning, and automation. Smaug Agentic emphasized endurance and stability during long-running coding workflows.</p>
<p>These outcomes demonstrate that capability gains follow both the training data and the weaknesses of the underlying model.</p>
<h2>A Practical Decision Rule for Transfer Learning for LLMs</h2>
<p>Choose a model-specific fine-tuning strategy when you can clearly define the target workload, diagnose the base model’s gaps, and measure improvement using production-relevant metrics.</p>
<p>Use benchmarks to select candidates, but use realistic workflows to make the final decision. The future of transfer learning for LLMs will belong to systems that are not merely more capable in general, but more dependable for the work they are designed to perform.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>Sketchpad: certified exact arithmetic with a small vision-language model</title>
		<link>https://blog.jefersonjeronimoai.online/2026/11/09/hf-paper-2611-04434/</link>
		
		<dc:creator><![CDATA[Tech News]]></dc:creator>
		<pubDate>Mon, 09 Nov 2026 15:19:27 +0000</pubDate>
				<category><![CDATA[AI Agents]]></category>
		<category><![CDATA[Technology]]></category>
		<category><![CDATA[Exact computation]]></category>
		<category><![CDATA[Lean theorem proving]]></category>
		<category><![CDATA[LoRA adapters]]></category>
		<category><![CDATA[Model certification]]></category>
		<category><![CDATA[Neural arithmetic]]></category>
		<category><![CDATA[vision-language models]]></category>
		<guid isPermaLink="false">https://blog.jefersonjeronimoai.online/2026/11/09/hf-paper-2611-04434/</guid>

					<description><![CDATA[Sketchpad turns a small vision-language model into a certifiable arithmetic engine by limiting each step to a finite worksheet view, then checking every possible view.]]></description>
										<content:encoded><![CDATA[<h2>Sketchpad: certified exact arithmetic with a small vision-language model</h2>
<p>Language models can produce impressive answers to long-addition and multiplication tasks, but benchmark accuracy is not a proof of correctness. A model may fail on an unseen input, and the failure can look like a finished, plausible answer. External calculators provide exactness, but they move the arithmetic outside the model.</p>
<p>In <em>Sketchpad: certified exact arithmetic with a small vision-language model</em>, Mostafa ‘Neo’ Mohsenvand proposes a different division of labor. A small vision-language model performs local arithmetic steps on a rendered worksheet, while a conventional runtime controls the procedure and moves a bounded window across the sheet. The authors’ central claim is that this structure turns an unbounded arithmetic problem into a finite certification problem.</p>
<h2>Why a bounded view matters</h2>
<p>Each model call sees only three cells, an operation token, and output placeholders. It emits a digit or intermediate value, a carry or borrow, and the next operation. The runtime writes those outputs to the sheet and advances the window. No hidden reasoning trace, earlier answer, or other state is passed from one step to the next.</p>
<p>That restriction is important. Under the paper’s formal model, every possible input to a step belongs to a finite set of views. If the model produces the correct output for every view, induction shows that the complete procedure is correct for runs of any length. The authors formalized this argument in Lean.</p>
<p>The paper also proves a converse-style limitation: if a step can read the entire history of earlier steps, as in ordinary chain-of-thought decoding, no finite test suite can guarantee correctness on all longer runs. A decoder can be constructed that passes every finite collection of tests and then fails on a longer context. The result does not say that chain of thought always fails; it says that finite testing alone cannot provide the same universal guarantee.</p>
<h2>How Sketchpad is trained and checked</h2>
<p>Sketchpad adapts Qwen3.5-0.8B with a low-rank adapter trained directly on exhaustive local-operation tables. The tables cover column addition, long multiplication, subtraction, quotient estimation, and decrementing an estimate. Together they contain 2,446 views.</p>
<p>The model is trained not only to select the correct output token, but also to maintain a positive logit margin. The deployed certificate evaluates every view on the target execution path: an Apple M4 laptop GPU running the model in a browser through WebGPU and 16-bit arithmetic. All 2,446 views passed. The smallest reported margin was 3.02 logits, which the authors say provides tolerance for bounded numerical changes below half that margin.</p>
<p>This is stronger than checking a random sample of large arithmetic problems, but it is not a proof that every part of the system is correct. The theorem depends on the runtime presenting only valid table views and implementing the reference algorithms correctly. The paper states that these runtime obligations were asserted and tested, but the concrete layouts and procedures were not fully formalized in Lean.</p>
<h2>What the experiments showed</h2>
<p>The certified procedures cover signed addition and subtraction, long multiplication and division, square roots, and greatest common divisors at arbitrary input length, subject to the runtime and representation assumptions. In a pre-registered browser test, Sketchpad solved 200 of 200 problems exactly: 50 each for division, square roots, greatest common divisors, and signed sums or differences. Median times ranged from 2.0 seconds for signed arithmetic to 9.8 seconds for square roots on the reported laptop setup. The slowest problem took 16.5 seconds.</p>
<p>In a separate comparison involving 220 problems, the system solved all 220 sums and products. The strongest of ten other tested models, GPT-6.1 Sol, solved 217. The paper reports that the alternative models’ errors included silent mistakes on long sums, while several open models also exhausted their output budgets. These results demonstrate the benefit of the worksheet design on this test, not a general ranking of mathematical ability.</p>
<p>The authors further composed the certified arithmetic operations into a symbolic runtime for polynomial expansion, equation solving, and factorization. The runtime checks roots and some derived expressions, but those checks are not complete proofs that every root or coefficient was found. The demonstrated algebra system is also limited: the paper describes low-degree polynomial tasks and linear systems of up to three equations.</p>
<h2>Preserving general language ability</h2>
<p>Fine-tuning an arithmetic adapter initially damaged instruction-following performance, illustrating a familiar catastrophic-forgetting problem. Sketchpad addresses this with a scalar gate on the adapter. When the gate is zero, the adapter branch contributes an exact floating-point zero, and the authors report bit-for-bit recovery of the base model in their tests. When the gate is active, the model executes the sheet procedures.</p>
<p>A dialogue adapter was trained to decide when to invoke the sheet. On held-out invocation tasks, the paper reports precision of 0.994 and recall of 0.989. However, the broader result is mixed: on fresh GSM8K problems, the dialogue adapter scored 5.8 points below the base model. The authors attribute this loss to multi-step reasoning rather than arithmetic execution. This distinction matters: certified arithmetic does not automatically certify the model’s interpretation of a word problem or its decision to call the tool.</p>
<h2>Practical implications and open constraints</h2>
<p>Sketchpad’s most useful idea is architectural. If an AI system must perform a narrow, high-stakes computation, giving a model unrestricted scratch space may make universal guarantees impossible. A bounded external memory, exhaustive local certification, and a conventional controller can instead make the model’s step-level behavior auditable.</p>
<p>That approach could be relevant to browser-based calculators, educational software, or agent workflows that need exact numerical subroutines without sending every calculation to an external service. It also suggests a practical certificate format tied to model-file hashes, runtime versions, device precision, and glyph definitions.</p>
<p>There are clear trade-offs. The model requires a specialized rendered input format, each step takes roughly a tenth of a second on the reported laptop GPU, and the guarantee applies only to the certified execution path. New hardware may require renewed checking. Parsing, transcription, runtime control, and natural-language reasoning remain outside the arithmetic certificate. The authors therefore demonstrate a credible pattern for certifying a useful neural component—not a general solution to reliable mathematical reasoning.</p>
<hr>
<h2>Sources</h2>
<ul>
<li><a href="https://arxiv.org/pdf/2611.04434" rel="noopener">Read the original paper (PDF)</a></li>
<li><a href="https://arxiv.org/abs/2611.04434" rel="noopener">arXiv paper</a></li>
<li><a href="https://huggingface.co/papers/2611.04434" rel="noopener">Hugging Face paper page</a></li>
</ul>
<p><em>AI-assisted research summary. Check the original paper for complete methods, results, and limitations.</em></p>
]]></content:encoded>
					
		
		
			</item>
	</channel>
</rss>
