3 pointsby SquirrelFan5 hours ago1 comment
  • halcyon-past5 hours ago
    Hi, do you mind sharing how do you fetch or scrape these data
    • SquirrelFan2 hours ago
      Sure! It's mainly a mix of APIs to discover publicly available conversations.

      Most of the data currently comes from Reddit, particularly as election of subreddits i curated myself. I also collect data from Hacker News, LinkedIn, X, and a few other sources using third party API.

      The dataset likely has a bias toward tech, partly because of the subreddits I monitor. I'm working on expanding coverage to make the rankings more representative.