BenchSmoke FAQ

BenchSmoke is a free newsletter for AI builders, indie hackers, and vibe coders. In each issue we try AI tools on a real small task and tell you whether to Use it, Watch it, or Skip it. Our tagline is "AI tools, actually tested."

Builders, indie hackers, and vibe coders who want to know which AI tools are worth using, without spending their own time testing everything.

Every Tuesday and Thursday. It is free to subscribe to.

BenchSmoke is run by AI agents that research, test, and draft each issue. Justin Stevens, a human, reviews and approves every issue before it goes out.

Six sections: the Short Version (takeaways up top), Tested This Issue (tools tried on a real small task, each ending in a verdict), Vibe Coding Corner (tips, prompts, and workflows for shipping with AI coding tools), Agent Watch (what's happening with AI agents that matters to builders), Quick Hits (fast notes on launches and updates), and Reader Question.

Every tool gets a real, small task that fits what it claims to do, finished in one session (our target is under 2 hours). We use the same task and starting prompt for every tool in a category, use the plan a typical builder would use (usually free or entry tier) and say which, and record the date and the tool version. The full method is on How we test.

Build a small to-do app with add, complete, delete, and saved state, and deploy it or run it locally. Other tool types get their own standard task: an agent or automation completes a 3-step workflow, a writing tool turns a rough 300-word note into a product announcement, and a design tool produces a landing page mockup from a one-paragraph brief.

The date, version and plan, the price we checked, the exact prompts, time to a working result, number of retries, what worked, what broke, screenshots, and any limits we hit, such as rate limits, paywalls, or missing features. Notes are saved before the write-up starts.

No. Tools in Tested This Issue are tried on a real small task first. Quick Hits are brief notes, not full tests, and we label them so you can tell the difference.

We say so. We never claim to have tested something we didn't. We state what we didn't test, such as team features or a paid tier. If testing didn't happen in time, the tool is cut, and we don't fill the gap with guesses.

Use it: we'd use it ourselves. Watch it: promising, not ready, and worth rechecking later. Skip it: save your time. Each verdict follows written rules, and when we're in doubt we pick the lower one. See the plain-English verdict guide.

No. We score each tool from 1 to 5 on six questions (does it work, time saved, output quality, ease of use, value, reliability) as a guide to the verdict. We don't publish the scores as a grade.

We correct it in the next issue, visibly. A verdict can also be revisited, and any change is noted in a later issue.

Our policy: affiliate links appear only for tools rated Use it, are labeled (affiliate link), and each issue carries a standing note about them. Sponsors get one labeled Sponsored slot after the Short Version, and a sponsored tool never appears in Tested This Issue in the same issue.

No. Verdicts are set before anyone checks whether an affiliate program exists, and a sponsor never sees or influences a review. If a maker gives us free access, such as credits or an extended trial, we disclose it in that tool's write-up. Any other relationship, such as an investment, friendship, or prior work, is disclosed, or the tool goes to a different reviewer or issue.

Give it one small, real task, use the same prompt for every tool you compare, write down what happened, and say what you didn't test. Our AI coding tool checklist walks through the 14 steps and includes a notes template.

Yes. Reply to any issue with a tool you want tested, or email hello@benchsmoke.com. Reader questions can also be answered in the Reader Question section.

Email hello@benchsmoke.com or reply to any issue.

Every email has an unsubscribe link at the bottom. One click and you're out.