48 pointsby leephillips8 hours ago2 comments
  • Legend24407 hours ago
    A lot of researchers think their job is to build models. They don't want to collect their own data, so they go find whatever dataset they can on kaggle or from a previous paper or wherever.

    This is backwards. The model is the easy part. Getting good data is 99% of the job, and nearly any clown can make a good model once you hand them a good dataset.

    • skvmb7 hours ago
      As a clown, I can confirm.

      If you hand me a clean, well-labeled, representative dataset, I can make the model do a respectable little dance by lunch.

      If you hand me a Kaggle CSV with duplicated rows, target leakage, mislabeled outcomes, and columns named final_final_v2_REAL, suddenly I’m not doing ML anymore. I’m doing archaeology with a red nose on.

      The model is the balloon animal. The dataset is the elephant you had to drag into the tent.

    • steve_adams_866 hours ago
      This holds in software as well. I encounter people trying to build solutions for problems that might not even exist, even in the context of addressing a specific bug. The act of measuring and collecting data is hard work, pretty boring sometimes, and often prescriptive in ways that aren't appealing. It's like we'd rather guess and use the ambiguity to allow ourselves to explore solutions we're more interested in. The alternative is manually profiling and poring through logs, so, I kind of get it.
    • i7lan hour ago
      So true and it's been like that for ages. It's why I called these people rogue data scientists five years ago:

      https://ianreppel.org/how-to-spot-a-rogue-data-scientist/

    • nradov7 hours ago
      For a lot of clinical decision support use cases you don't even need fancy AI models to get accurate results. If you have good quality cleansed data you can literally just import it into Excel and run a simple linear regression analysis. But unfortunately that won't get you a reputation as an "AI thought leader".
      • kenjackson6 hours ago
        Actually a simple flow-chart works for a large number of use cases. That said, there are a lot of use cases where we don't have a simple way to run a linear regression model to get reasonable results where "AI" does seem to work well.
      • QuercusMax7 hours ago
        You just need to figure out a way to brand that as a new, resource-conserving AI model.
        • actionfromafar6 hours ago
          I think it needs a cool name.
          • QuercusMax6 hours ago
            We'll call it SSLRM. Spread Sheet Linear Regression Modeling, pronounced SLURM. Sounds fancy and business friendly.
  • matusp5 hours ago
    Dataset quality is a huge issue in ML in general. You can often list a few dozen random samples from any given dataset and you will find out something weird going on instantly.