Is prompt tuning still a thing today? Anybody still using the likes of DSPy?
Just a random thought that came up when I was discussing AI engineering challenges with some folks. Prompt engineering/tuning was definitely "a thing" during the ChatGPT-3.5 era.
There were all sorts of fancy rule of thumb people were advocating, as well as formal approaches like GEPA that were then incorporated into frameworks like DSPy which were also briefly popular.
Having said that, it's been a while since I've come across conversations on this topic, which means it's either dead or it's already working well in production.
Anybody still has prompt tuning with the likes of DSPy as a standard part of your development process and stack?
The core bet of DSPy is that it functions kind of like a compiler. You define your task as a signature, with typed inputs and outputs, and provide examples and/or a metric of what good looks like. DSPy then manages the implementation of how the task is performed, given the current state of the art.
So things like GEPA have evolved beyond prompt writing. DSPy.Flex exposes the code your program, as well as the prompt, to the optimizer allowing it to decompose or route inputs in optimal ways. For decision models, the ReAnchor optimizer fits your thresholds of the Nouls and Choices to the optimal levels.
When DSPy first launched, optimizing was about selecting the right in-context examples. Then models got great at prompting, so prompt optimizers emerged. Then models got great at writing code, so harness optimizers emerged. (They're still meh at decomposition though – hoping that improves). Decision models exposed confidence levels, so prob fitting emerged. You can even use weight space optimizers, which we've prototyped, since the larges models can now interpret and edit weights.
What's wonderful about this is that the DSPy API has basically remained the same. My programs are still used during testing, ones I wrote years ago.