Show HN: AI agents that validate your product idea by talking to real users(app.holyshift.ai)

5 pointsby Matzalar13 hours ago5 comments

lovrok2313 hours ago
I'm curious about the guardrails here. In my experience trying to use LLMs for user research, they tend to be "yes man" often hallucinating features or agreeing to user requests that aren't actually on the roadmap just to keep the conversation flowing.
how do you constrain the agent to stick strictly to the facts of the product hypothesis without making stuff up to please the potential customer?
- Matzalar13 hours ago
  We ran into the same issue early on. Our fix was to lock each agent to a small JSON snapshot of the idea (no other knowledge), plus strict response templates. They can only ask questions, never describe features or promise anything. If a user asks for something outside scope, the agent replies with “not in the current hypothesis, why is that important to you?” rather than making stuff up. We also have a human review step before anything goes live.
tene80i12 hours ago
Interesting idea. Nice design. But usability issue: on mobile I hit your yellow chat CTA thinking it was submitting the app text input. You might want to move that out of the way.
- Matzalar3 hours ago
  Good point. Thanks for your suggestion. We’re very early, a lot is changing as we move on and get more feedback.
13 hours ago
undefined
thebiggodzzila12 hours ago
Chat always boosts my confidence, but reality isn’t always as kind. How can I really tell if my idea is any good beyond what Chat says, and how many people do you actually interact with?
- Matzalar3 hours ago
  Chat is OK at making anything sound promising, but it will not talk to real people and give you the real market signals. We do both, we do the “synthetic” data analysis, but more importantly we talk to real people and ask them what they think. Depending on the product (or project), we’ll usually get few hundred real interactions. The goal is to get enough real, unfiltered feedback to see whether there’s a consistent signal to make a decision.
likethejade8713 hours ago
Are agent pitching ideas or do actual research? Sounds super interesting though
- Matzalar13 hours ago
  They’re not pitching or selling anything they only do research. The agents ask structured questions in relevant communities and collect real reactions, pain points, objections ... No selling, no marketing language.