Do Prompt Optimizers Actually Work? We Tested Our Own, and It Often Lost

The short answer: rewording alone didn't help. When we sent AI the same request written two ways, rewritten prompts with their blanks left empty lost to the person's original wording 24 to 9. Once a simulated user filled in those blanks, the rewrites won 25 to 8. The improvement came from the added details, not from polished wording.

We make a prompt optimizer, so this wasn't the result we wanted. We changed the product because of it.

#What we tested

We took 33 prompts from our test set of 132, spanning industries from ecommerce and law to software and everyday questions. They're written the way people really type, including vague ones. For each one:

1. PromptSaver rewrote the prompt.
2. Claude Sonnet 5.5 answered the original and the rewrite separately.
3. Claude Opus 5.5 read both answers, in random order, and picked the one a senior professional in that field would prefer.

We ran that comparison two ways. In the first, details the rewrite asked for were left out. In the second, a simulated user filled them in with realistic details, as you would before sending.

#The results

Before you read much into that last row, here's a check we ran on our own judge. We sent it two answers to the exact same prompt, 18 times. It split them 9 to 8, with one tie. Answers vary, and the judge almost always finds a reason to pick one. So small gaps in a small test are noise. A 4 to 6 split tells you only that those rewrites made no clear difference.

#Why rewrites lost when details were left out

• Models already fill gaps well. Given a loose request, a good model makes sensible assumptions and gives you something usable. The rewrite's empty slots gave it less room to do that. We tried marking them "not specified" and leaving them in brackets with a note, and neither helped when nothing was filled in.
• Length limits can backfire. Our early rewrites often capped the length of an answer. That saves money, but a judge reading for usefulness preferred the fuller answer. If you want the most complete answer, don't cap it. If you're paying per token, do.
• Bad rewrites do real damage. Our first version once replaced someone's pasted meeting notes with a blank. It also added claims the person never made. We've fixed the rules behind both and added an automatic check for them, but every rewriter can make these mistakes, so read what you're about to send.

#Why rewrites won when details were filled in

When people are prompted for the facts that matter, the AI stops guessing. In the winning cases, the judge kept praising the same thing: answers that used the person's real figures, dates and policies, where the answers to the original prompts made up prices, guarantees and product claims.

You don't need an optimizer to get this benefit. Before you send anything that matters, ask yourself what the AI would have to guess, and tell it.

#What about just making prompts shorter?

Plenty of tools sell shorter prompts, so we tested that too. We took 12 wordy prompts, full of greetings, thanks and "if it's not too much trouble," and cut the filler while keeping every fact.

Given that the judge split identical prompts 9 to 8, 7 to 5 is no real difference. The prompts got about half as long and the answers didn't measurably change. That's the honest case for shortening: it saves tokens on what you send.

There are two limits. Most of the prompts in our main test set were already short, and shortening those saved about 2%. And what you send is usually the smaller part of the bill, because answers tend to run longer than prompts. Cutting 100 tokens from a prompt is worth little unless you do it thousands of times. Shortening matters most for repeated prompts and for instructions you resend with every message.

#What we changed

• Two modes, each with one job. Stubby cuts the filler and asks you nothing. Full asks for what's missing and builds a complete prompt. We test each against its own promise.
• Stubby can't make a prompt longer. If it can't shorten safely, you get back exactly what you wrote, and we tell you.
• We ask before we rewrite your message. If Full needs details, it shows a few short boxes first. Nothing goes in your message box until you choose.
• Skip every detail, and you get your own prompt back. A follow-up test on 32 prompts gave the same answer as the first: with no details, rewrites lost 8 to 24. So we don't hand you a worse prompt just because we rewrote it.
• A few details are enough. In that same test, rewrites with about half their details filled in beat the original 31 to 1. Those details were simulated, and a prompt with only one blank counts as fully filled, but it means you don't have to complete the form.
• Save your details once. Since the details are what help, you can now save facts about you and your business, and Full uses them instead of asking. In a test on 27 prompts, it asked for 1.0 details on average instead of 2.3, and rewrites beat the original 23 to 4 with any other questions skipped. Those saved details were simulated, written by a model that had seen each request, so real results may be lower. When we gave it another person's unrelated details, none got into the rewrites.
• Every rewrite gets checked before you see it. Plain code looks for lost numbers, links, code and names, and for figures you never gave. If it finds one, the rewrite runs again.
• We test ourselves every day. A rotating set of prompts runs through the real product daily, and a judged comparison runs weekly. If quality drops below our targets, we get an alert.
• We don't lead with wording. The surest savings come from the model. Across our prompt library, the improved prompt on the model we recommend costs under half of the original on Claude Sonnet 5.5 at list prices. That's a cost result, separate from this test of answer quality.

#How far to trust this

This is one test, and it has real limits.

• It's a small sample. 33 prompts show a direction, and 12 show less. Neither gives a precise rate.
• The details were simulated, and they helped by design. The filled-in version has information the original never had, and the judge was told to treat it as true. That's the point of asking for details, but it means the test can't show that rewording itself helps. It suggests the opposite.
• There's one judge and one answering model. Another model might score these differently, and an AI judge isn't a human expert.
• Some answers were cut off. Four of the 33 comparisons had an empty or cut-off answer on one side, because of our test's length limit. Those count as they fell.
• We wrote the 12 wordy prompts ourselves to give shortening something to work on. Your prompts may have less to cut.
• We're not neutral. It's our product and our test set.

An earlier run of all 130 prompts we had then, with a tighter limit on answer length and blanks left unfilled, went 112 to 17 for the original. That run had flaws of its own, which is why we rebuilt the test, but it points the same way.

#Should you use a prompt optimizer?

Use one as a checklist, not as magic. It's most useful when:

• The work matters, and you aren't sure what the AI needs to know.
• You repeat the same kind of task, so a good prompt pays off many times.
• You want to know what a prompt costs, and whether a cheaper model would do.

Skip it for quick questions. Ask them the way you'd normally ask. Be wary of any tool that promises better answers from rewording alone, or that advertises token cuts without showing you the answers. Fewer tokens don't help if you have to ask again.

#Frequently asked questions

#Do prompt optimizers improve AI answers?

In our test, only when they brought out details the AI didn't have. With details filled in, rewrites won 25 of 33 comparisons. With details left out, they won 9 of 33. Filling in about half of them was enough to win in a follow-up test.

#Does a shorter prompt get a worse answer?

Not in our small test, when only the filler was cut and every fact stayed. Prompts got 53% smaller, and answers stayed about the same length and quality. Cutting facts is different: the AI has to guess what you removed.

#Is it better to just ask AI normally?

For everyday questions, usually yes. For work that depends on your facts, add those facts. That helps with or without a tool.

#What should I add to a prompt to get a better answer?

Whatever the AI would otherwise have to guess: real figures, dates, names, your policies, who it's for, and how it'll be used. See how to write better AI prompts.

#Does a rewritten prompt save money?

A little, on what you send. The bigger saving is usually using a smaller model for the job. See how to reduce AI costs.

#Can I run this test myself?

Yes. Take a prompt you use often, write a version with the details filled in, send both to your AI, and compare them. PromptSaver's "Check it yourself" step does a side-by-side comparison too.

#How we checked this

We ran these comparisons on November 3, 2026 with PromptSaver as it stood that day, using every fourth prompt in our test set, plus 12 wordy prompts for the shortening test. Answers came from Claude Sonnet 5.5 and judgments from Claude Opus 5.5, and the order of the two answers was shuffled. PromptSaver's rewrites run on Claude Haiku 5.5. We didn't test other companies' optimizers, so we can't say how they'd do.