1 pointby shadab_nazar2 hours ago1 comment
  • shadab_nazar2 hours ago
    Summer Yue's OpenClaw agent deleted 200 emails despite "confirm before action." We tested the gog skill (Google Workspace) and saw the same behavior — the skill teaches the agent how to bulk-delete but not when to stop.

      That was 1 of 11 security failures in that single skill. Others: data exfiltration, unauthorized forwarding, contact harvesting, impersonation via  
      calendar events.                                                                                                                                    
                                                                                                                                                        
      We tested 10 OpenClaw skills across 186 security properties, with and without each skill loaded. 9 of 10 show the same pattern — some properties
      improve, others degrade. The skill adds domain knowledge that shifts security behavior in ways nobody tested for.
    
      We hardened all 10. 84% fix rate. Each guardrail traces to a specific regression. Open source, drop-in replacements.
    
      Per-skill scorecards: https://faberlens.ai/report
      Research: https://faberlens.ai/blog/jagged-surface
    
      N=10, two models, limitations published. Happy to go deeper on methodology.