"Let's outsource critical decision making to a language model."
Small scale of this can be seen when attempting to single-shot a whole SaaS by the means of a severely underspecified prompt. This is the first-order blind reliance situation: our SaaS sure is implemented in some way.
Really fiery tarpits put the LLM higher in the stack: let the language model decide what the feature should be. Our SaaS sure does something in some way.
Having a human in the loop at all times doesn't scale. Not having a human scales, but doesn't work.