Ask HN: Is there a site where a crowd can vote on which A/B site looks better?

8 points | by EGreg 13 days ago


  • Tagbert 13 days ago
    Normally, how an A/B test works is that you present those options to different users and then you compare their behavior to some designed outcome. It is almost never a matter of asking users to choose between options. You are just giving them the different versions and then measuring their engagement levels. Engagement often measured by sales or click through rates. The one with the most engagement wins.
  • ianmabie 12 days ago - you can upload screenshots / prototypes / the actual webpages (if they exist) and have users go through a series of questions. You can also set it to scramble the order to see how that (if at all) changes preferences.

    As other folks called out, this can give you a sense of customer comprehension / stated preferences & reasons why, but may not give you the level of confidence a true A/B test would (launching both iterations & splitting traffic amongst & measuring conversion difference)

  • dalmo3 13 days ago
    I don't know anyone who would do that for free, but there are tons of paid services that do just that. E.g
  • jtokoph 13 days ago
    Amazon Mechanical Turk is probably your best bet:
    • BlameKaneda 13 days ago
      I did Mechanical Turk surveys on and off for about 8 months and all it netted me was about $25. But, I bought a nonstick pan with it.
  • rgavuliak 13 days ago
    What looks better and what drives more revenue/conversions are often very different. The data collected this way might not be useful at all.
  • shoo 12 days ago
    see also: Kohavi, Ron, Diane Tang, and Ya Xu. 2020. Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing. Cambridge University Press;
  • terrycody 12 days ago
    I think you can apply for Google Search Quality Raters, these people exactly doing this every day.
  • savrajsingh 13 days ago but they’ll harass you forever w their sales team
  • tchock23 12 days ago might be what you’re looking for…
  • 35mm 13 days ago
    Google Ads
  • mv4 13 days ago