Ask HN: What do you do to receive honest feedback for your products?

43 points | by ayushchat 14 days ago

28 comments

  • testmasterflex 13 days ago
    I’ve noticed that asking leading _negative_ questions opens up people a lot.

    Me: “The lights are too bright, right?” (Hardware product)

    User: “No they are fine but I wish the display was larger.”

    I think questioning like this makes user comfortable criticising you.

    • tr1ll10nb1ll 13 days ago
      I've been doing this for a while now. Good thing is, my audience is developers, so the feedback is detailed whenever I point something out to start a conversation.

      The next step is to just differentiate between actual feedback and opinions that are intertwined in so much of the information the user could pass to you haha.

      • testmasterflex 9 days ago
        I remedy this by quantity; I say the same thing to multiple users and listen attentively to the feedback. If enough people say the same thing (usually >2) then I know it’s something there that needs fixing.
      • silisili 13 days ago
        I was going to mention something similar! People by nature (usually) don't want to offend anyone, hence white lies and such.

        I've found by starting with my own insult/complaint, people feel more comfortable to open up a bit.

        Not just specific to products, btw, but anything you want an honest opinion on.

        • egfx 11 days ago
          It’s interesting that you picked up on this. I tweeted about it https://twitter.com/chamILYAn/status/1379140215407747073
          • ayushchat 13 days ago
            Oh this is genius.. thanks for sharing.
            • fendy3002 13 days ago
              Cunningham's law in action
              • nickff 13 days ago
                That's a brilliant tactic! May I ask where/how you learned it?
                • testmasterflex 9 days ago
                  I’m very disagreeable and judging in my temperament by nature so it’s natural for me.

                  Unfortunately it also means I can seldom enjoy anything and other people find me annoying. :/

              • eimrine 14 days ago
                I am an owner of grocery shop and for me it is super easy to have a talk with my clients or to observe their behaviour. Somebody don't like our goods or service, somebody use to fall down before even entering because tiles on stairs become slippery on winters. Somebody use to steal something as expensive as possible without any real needness. Well, software business is different but find a way to get some talks with your clients and try to find a way to observe what they are actually doing (please look at people but not spyware trackers).
                • ayushchat 14 days ago
                  This is very interesting.

                  So basically, have a similar approach to a physical business even for a software business.

                  Thanks for sharing.

                • this2shallPass 14 days ago
                  Have you read The Mom Test? Highly recommended.

                  Here's a summary: https://www.slideshare.net/xamde/summary-of-the-mom-test

                  The closer you are to the target user and target buyer (not necessarily the same person), the better. You can learn a lot from anyone, but it might be at a pretty abstract level and less useful.

                  Focus as much as you can on users and buyers. Speak with them often, observe what they do often, wherever they are - conversations with you, chatting with each other online, complaining about current pain points with existing solutions, including in reviews for products that solve similar problems.

                  Good luck!

                  • nlitened 13 days ago
                    Its audiobook version on Audible is narrated by author himself, which I enjoyed a lot. All the emphases are placed as author intended.
                    • ayushchat 14 days ago
                      Thanks a lot.

                      I'm a big fan of The Mom Test. Fantastic book.

                      But I was more curious about how are people actually applying it's lessons in their actual business.

                      • michepriest 14 days ago
                        +1 for The Mom Test

                        Something I did which was highly effective was set up a Discord server to watch users talk about my product with each other. Doesn’t have to be Discord. Can be a Facebook group, sub-Reddit, etc

                        • michepriest 14 days ago
                          Also try user experience testing. You pay a nominal fee to watch users use your product and say their thoughts out loud. I was always surprised by what they said and thought about what we created. The only thing is you need to set up a good testing plan.
                          • ayushchat 14 days ago
                            Yes, that makes sense, thanks for the answer.
                      • muzani 14 days ago
                        I recommend getting the actual book. Most of the summaries get some gist of it but miss the mark.
                      • gnicholas 13 days ago
                        I have an exit survey on my Chrome extension, which loads automatically upon uninstall. People who hate the extension, or hate aspects about it, are more than happy to provide such feedback. Sometimes with obscenities.

                        One thing that people complained about was price, so I decided to ask people what price they thought was fair, from a list of three options (50% off, 75% off, and 99¢/yr). I then offered them a coupon code at whatever price they indicated was fair.

                        After offering this for years, and with over a thousand people going through this branch of the survey (which required saying that cost was the primary reason for uninstall), only a handful took me up on the discount. My takeaway from this is that lots of people will say that something should be X% cheaper, but nearly all of those people would not buy it even if it were that price.

                        Having learned this lesson, I no longer offer the uninstall discount, or many discounts at all.

                        • ayushchat 13 days ago
                          Yes, this makes so much sense, true earned insight, thanks for sharing
                        • softwaredoug 14 days ago
                          Super users and dogfooding

                          When we started out with Quepid, we found a small handful of champions that loved the idea and advocated heavily for adopting the tool in their orgs. They became an extension of the team. We engaged with them all the time. We did everything we could to take their feedback seriously. Even if they never even converted their orgs to heavy users, their feedback was worth the high level of investment.

                          We also offered consulting services and would use the tool on our own projects. Here we were the customer to ourselves, and had to ensure the product was stable during engagements.

                          During consulting, we would train up teams on the tool. We could see where they fell in love with it and where it wasn’t a good fit.

                          As a SaaS product, we weren’t successful. We decided to focus on just the consulting business model. In the end Quepid became open source, and the community built around it seeded the open source community.

                          • ayushchat 13 days ago
                            What an amazing story, thanks for sharing.
                          • exolymph 13 days ago
                            Try to actually sell them something, that's the best way to get genuine feedback :P Until money is involved (or the hassle of account creation, I guess) it's all play to them.

                            Perhaps more useful answer is to be part of a practitioner community, as you noted.

                            • ayushchat 13 days ago
                              Yes, I think trying to sell something is probably the best approach. Going to try something soon.
                            • KBeyo 13 days ago
                              1) find the people who have the problem you’re solving / creating value for

                              2) have them use / experience the product and watch them

                              3) have them think out loud while their doing that and keenly observe how they use experience the product

                              4) the observations should point to low hanging fruit for improvement

                              5) ask open ended unbiased questions about the experience and listen to them talk about it. Example: what did you think?

                              6) you don’t necessarily need to use a big sample size to get good feedback. Getting feedback from 10 people, you will see six that show the same pattern and you will then know to focus on that

                              7) hypothesize on improvements from what you learn, build, release again a and repeat feedback with real users

                              8) you will know you have something great when people show they are impressed. Be careful if people just saying it’s great as everyone will do this, so you will only really know if it’s great through observing.

                              9) if you can’t do this and are a solo founder, you probably will have a very difficult time with sales once you find product market fit since you always have to be in front of people and pitching

                              • ayushchat 13 days ago
                                Wow, thanks for the detailed step by step playbook. This helps a lot.
                              • analog31 13 days ago
                                There are a handful of web forums where people discuss the hobby that my product is related to. They're pretty frank. In addition to reading comments about my product, I also read both the praise and horror stories about other vendors, and try to form a picture in my mind of what really went wrong with the bad customer experiences.
                                • ayushchat 13 days ago
                                  Yes, this is an interesting approach, thanks for sharing.
                                • Jugurtha 14 days ago
                                  >Especially when you're just starting out, and have no users yet.

                                  Add analytics, even rudimentary. Something like PostHog is good.

                                  Add monitoring (Sentry) so you catch exceptions in your code when soon to be people and systems use your product without them having to report them. You'll fix problems faster and prioritize (most frequent, most impactful, etc).

                                  Create a Slack workspace where your users could ask questions.

                                  Get users.

                                  Observe behavior and usage. Keep an eye on non-consumption. Schedule calls with them and ask about what they're trying to accomplish with your product.

                                  If they're not using a feature you know they need or have stopped using it (from analytics), ask why they're not using it or not using it anymore (the load time is just too long, it's buggy, it doesn't support their use case, they moved on to something else or to another role).

                                  Look at where people are coming from (referrers in analytics) but also ask them where they're coming from (community? subreddit? HN? Word of mouth? Someone mentioned it? Who? Ask them. Rinse and repeat. Investigate everything.)

                                  Look at email addresses and their domains. Are they university accounts? Are there many from the same organization? Investigate. Ask questions on how they signed up and what they're trying to do.

                                  You want to get users. You'll get all the feedback you need, either active or passive (them not using something, not doing something, dropping something mid-action).

                                  Ask questions. Don't really ask for "feedback" because you're putting them on the spot. Ask "Have you tried x?". If you're aware of an issue in something, let it out, they may laugh and be relieved because they didn't want to break it to you, but you did and now they can go on about why something sucks for them because they now know you won't be butthurt if they say something.

                                  Ask questions to know the problem they are trying to solve and you'll get reasons or usage or non-usage (work in a very sensitive organization and they can't, regulations, formats, etc).

                                  Asking for "feedback" in the form it is most known is like asking someone what they think of you. Mostly useless and uncomfortable. You want people to be comfortable sharing with you, and they're most comfortable talking about the problems they're having.

                                • tr1ll10nb1ll 13 days ago
                                  If you have a product but no users at all, I think you should go back to the step of idea validation. Make sure what you're building is something that people might use in the first place.

                                  Multiple ways to do this.

                                  Th easiest, imo, is to set up a simple landing page for your idea. Know before-hand who might be interested in your product, if it's just a handful of people, that works too. Just prepare for a lot of manual work (getting in touch with people and conversing about your product in different ways without making it seem like some sort of an ad), at least until your first 1000 users, more so for the first 500 (assuming your end-users are individuals and not businesses).

                                  • ayushchat 13 days ago
                                    Yes, this makes sense, thanks for sharing.
                                  • ak_un 13 days ago
                                    It's part of my job, so my advice would be:

                                    1. You talk with real people, it's way more complex than doing a survey, but you'll get better data. 2. Talk to your customers or ask your potential customers if you don't have them.

                                    3. Don't ask your friends, especially your mum - she'll tell you've got a great product because she loves you.

                                    4. Important thing is to ask open-ended questions that don't suggest the answers. 5. You should see patterns after interviewing about 5-20 people. 6. Then, you use the answers and create a survey that will be statistically significant (about 384 respondents) - there're paid services that let you do that.

                                    That's it.

                                    • ayushchat 13 days ago
                                      Wow.. thanks for sharing, I will try this approach.
                                    • jensneuse 13 days ago
                                      It shouldn't happen like this in the first place. Before building something, you should already have access to your target audience. There are many ways of solving this. E.g. doing consulting and building a list of clients with a similar problem. Starting out to build a solution without this makes it just so much more likely that you build something that doesn't solve a real problem or there's no buying interest. Ideally, you can build for a group of clients who are ready to pay when you give them an MVP.
                                      • ayushchat 13 days ago
                                        Yes, there's truth in that. Thanks for sharing.
                                      • muzani 14 days ago
                                        A survey honestly helps a lot. Also Hotjar has a tool thar records users using a site. You can see people's frustrations on it.

                                        Most people should avoid friends and family. Or rather "normies" who aren't your target market. You'll get dishonest replies by people who want to support you, people who are blindly envious, people who want to sound clever or helpful. You'll often get feedback like "this costs too much" or "this is too ugly", when your actual target market is happy for it.

                                        • ayushchat 13 days ago
                                          Oh yes, for sure, thanks for sharing.
                                        • Chris86 14 days ago
                                          This book was really important for me early in my career, and I still recommend it to others quite regularly https://www.goodreads.com/book/show/9039673-undercover-user-... it will give you plenty of tactics you can use to conduct user research and gather feedback when you have no users or low budget etc.
                                          • ayushchat 14 days ago
                                            Great, thanks, will check it out.
                                          • DamonHD 14 days ago
                                            Definitely lots of feedback from everyone possible. Try and have someone other than you ask in some cases, because people will be deferential and kind to you as inventor/creator.

                                            Sometimes you can get small grants to cover professional advice, which can include feedback, or sometimes there will be events where you can get such advice from experts for free.

                                            I also help run an invetors' club, and that is another good venue for advice.

                                          • chunkyks 14 days ago
                                            I tell people that I'm not precious about my stuff, and if something sucks I want them to tell me so that I can make it better.

                                            The other thing that I find works really well is "what is your least favorite thing about this tool?"

                                            • ayushchat 14 days ago
                                              Yea, need to do that to get brutally honest feedback.
                                            • aruanavekar 13 days ago
                                              We are a B2B SaaS, we do - End-of-year survey for users(with giveaway giftcards) to encourage feedback and completion. We ask them to rate features, also provide suggestions on feature enhancement/improvement - End-of-year survey for client Admin/Managers to learn what admin related improvements we should undertake - Host a quick access form under the user's profile page, so that at any time they can send us suggestions/feedback

                                              When we started in 2017, the pattern was similar but just more 1x1 basis and more frequent(catch up each week)

                                              Not sure best practices for B2C

                                              • justinbaker84 13 days ago
                                                I hire UI/UX professionals on UpWork. That is the best solution I have come up with. It is totally worth it to pay somebody $70-$100 to review your site or app for an hour and give you actionable advice.
                                                • brudgers 14 days ago
                                                  Not having users is a strong feedback on your product.

                                                  Finding users is a the priority.

                                                  It is hard painful work.

                                                  Good luck.

                                                • encoderer 13 days ago
                                                  Get like a hundred users and you will likely get a couple hero’s who take the time to tell you their problems and give you truly valuable feedback.
                                                  • Markoff 13 days ago
                                                    Show pop-up windows when launching app/opening website at least 3 times a day. Guaranteed honest feedback!
                                                    • aristofun 13 days ago
                                                      Find Israeli friends or colleagues. They will tell you the brutal truth.

                                                      Any culture where people tend to be direct will do.

                                                      • karaterobot 13 days ago
                                                        Presumably if you are building something for a domain, you know some people who are working in that domain. Don't ask those friends, ask them to recommend their least tactful colleague, the one who knows everything about everything, and can't keep their opinions to themself.
                                                        • NotAnEnemy 13 days ago
                                                          I would ask an UX Researcher to help you out via platforms like Fiverr. This is literally their job.

                                                          Source : I am a UX Researcher :)

                                                          • 2thamoon 13 days ago
                                                            Who also happens to be on Fiverr, seemingly? :)
                                                          • rokhayakebe 13 days ago
                                                            You ask people "Help me save time. I know why this will succeed, tell me why it will fail."
                                                            • andsoitis 14 days ago
                                                              Be curious.

                                                              Never get defensive.

                                                            • whoomp12342 13 days ago
                                                              dont ask for feedback. observe how they use it.
                                                              • FintechRisen 13 days ago
                                                                Following