TokMem: packing a whole procedure into a single token
Compiling each reusable task procedure into a single trainable token, with the language model kept frozen.
Blog
Notes on my research in NLP and machine learning, and the occasional aside.
ULPT: tuning a soft prompt in two dimensions
Tuning prompts in an ultra-low-dimensional space and letting a frozen random matrix do the rest, for ~98% fewer trainable parameters.
Carrying a soft prompt from one model to another
Describing a soft prompt by what it's near, so its task semantics carry across different language models.