Is this the end of human code review?(arxiv.org)
1 pointby bonjourjoelan hour ago2 comments
  • alertchecker42 minutes ago
    Think we're pretty close to the point where we'll no longer need to prompt AI to refactor or review the refactoring if it's done within tight constraints i.e. absolutely no change to functionality. But when functionality changes may be acceptable as part of it, it's less clear - in my experience AI isn't at a point yet where it can always be trusted to "understand" what the requirements are (including to what extent they can be allowed to evolve), what is or isn't acceptable in terms of user experience etc. That's where human review is still useful and in many cases, essential.
    • bonjourjoel37 minutes ago
      Requirements as product owner are not covered at all by AI, 100% agree.

      But in this programming-only example study, the refactor changes the functionnality very seriously, and it has to go deep in the core of the system, find all the traps, plan everything. And the agent succeeds without human code review. Code review was impossible because of the complexity and size of the task.

      Code review was not necessary, nor useful, nor essential, and even practically impossible.

  • kubban hour ago
    This title doesn’t match the article.
    • bonjourjoelan hour ago
      How? The agent does something incredibly difficult, and there is no human code review. Why do you think it doesn't match ?
      • ben_w34 minutes ago
        "Here's a case study where it worked once" != "the end of human code review".

        Don't get me wrong, this is impressive and capabilities do still seem to be on a generally upward trend with nothing "hitting a wall" despite all the parroting of that phrase, but there's a huge gap between the first time a machine manages something impressive enough to document, and that machine becoming so good at that task that humans need not apply.

        • bonjourjoel30 minutes ago
          Agreed. But also this inequality:

          "Is this the end of human code review?" != "the end of human code review"

          Now since the problem in this study is more complex than 90 to 99.5% of what a normal ticket is, the question makes sense. The agent does not have knowledge of the project, at the begining of each session, and manages to handle something more complicated than virtually anything a developper has to do. The question holds IMO.