https://hello-diana.github.io/tags/prompt-optimization/Prompt OptimizationHugo Blox Builder (https://hugoblox.com)en-usWed, 20 May 2026 00:00:00 +0000https://hello-diana.github.io/media/icon_hu_3520ea6f5cfedd63.pnghttps://hello-diana.github.io/tags/prompt-optimization/https://hello-diana.github.io/project/textreg/Wed, 20 May 2026 00:00:00 +0000https://hello-diana.github.io/project/textreg/<p>Large language models are highly sensitive to their prompts, and iterative prompt-optimization methods that rewrite prompts from model feedback tend to overfit: prompts grow longer, accumulate narrow sample-specific rules, and generalize poorly beyond the training distribution. We frame this failure mode as <strong>prompt distributional overfitting</strong> and measure it through <em>representational inefficiency</em>, which decomposes prompt inefficiency into capacity cost and scope narrowness.</p> <p><strong>TextReg</strong> is a regularization framework that realizes a soft-penalty objective through regularized textual gradients, combining three stages: Dual-Evidence Gradient Purification, Semantic Edit Regularization, and Regularization-Guided Prompt Update.</p> <p> <figure > <div class="flex justify-center "> <div class="w-100" ><img alt="The TextReg pipeline: Dual-Evidence Gradient Purification, Semantic Edit Regularization, and Regularization-Guided Prompt Update (adapted from the paper)." srcset=" /project/textreg/framework_hu_2dc2964a137b7d46.webp 400w, /project/textreg/framework_hu_42bb8845c72621b5.webp 760w, /project/textreg/framework_hu_de2a3ee30da79edc.webp 1200w" src="https://hello-diana.github.io/project/textreg/framework_hu_2dc2964a137b7d46.webp" width="760" height="330" loading="lazy" data-zoomable /></div> </div></figure> </p> <p><strong>My role.</strong> I am a co-author on this collaboration. Across multiple reasoning benchmarks, TextReg substantially improves out-of-distribution generalization, with accuracy gains of up to +11.8% over TextGrad and +16.5% over REVOLVE.</p>https://hello-diana.github.io/publication/textreg/Wed, 20 May 2026 00:00:00 +0000https://hello-diana.github.io/publication/textreg/<h2 id="abstract">Abstract</h2> <p>LLMs are highly sensitive to their prompts, and iterative prompt-optimization methods often overfit: prompts grow longer, accumulate sample-specific rules, and generalize poorly. We study this as <strong>prompt distributional overfitting</strong> and introduce <strong>TextReg</strong>, a regularization framework that realizes a soft-penalty objective through regularized textual gradients — combining Dual-Evidence Gradient Purification, Semantic Edit Regularization, and Regularization-Guided Prompt Update. Across reasoning benchmarks, TextReg improves out-of-distribution generalization by up to +11.8% over TextGrad and +16.5% over REVOLVE.</p>