Claude Code can be tricked simply by asking it to summarize a website(www.theregister.com)
11 pointsby galaxyLogic8 hours ago4 comments
  • science4sail5 hours ago
    > It starts off by asking the agentic coding model to summarize a malicious website that presents itself as an archive of notebook records, and then tricking Claude into using curl instead of its WebFetch tool to retrieve the contents of the page – but without directly telling the model to use curl.

    There seems to be a tug-of-war here. Harness authors want models to use the harness's specialized tools, but AI labs and agent "users" would rather have full access to just one tool: bash.

    • cyanydeez2 hours ago
      The models want bash because that's the training. Tools are synthetic constructs that have only conceptual weight where bash, curl, have real weight.

      What should be doing is a vm/shell that dumbs down model calls via another llm. Then you just translate.

      After all: a model is text, it cant do anything without the harness. The focus on arbitrary tools rather than just dummy shells is silly. The models know all about curl, so just mock up a compiant REST api and tell them the endpoints.

      Even MCP gets this wrong.

  • g42gregory5 hours ago
    If you are trying to hack into a website, using a software tool (Claude Code in this case), you are breaking the law. It doesn’t matter if it’s a Claude Code or a text editor, you are the one responsible. And you are using a text editor in an “unsafe” manner. CC is not there to babysit anyone.
    • pavel_lishin4 hours ago
      The article is not about using Claude to hack into a website.
  • galaxyLogic8 hours ago
    “The solution is something we talked about for many years,” he wrote. “Do not trust the model output.”
  • anr_pGan hour ago
    [flagged]