looks like the public bench link in the paper was taken down. <a href="https:&#x2F;&#x2F;hub.harborframework.com&#x2F;datasets&#x2F;orca-bench&#x2F;ORCA-bench&#x2F;latest" rel="nofollow">https:&#x2F;&#x2F;hub.harborframework.com&#x2F;datasets&#x2F;orca-bench&#x2F;ORCA-ben...</a>This doesn&#x27;t work anymore. Is there a newer link?

Attackers advantage in the iterative fast feedback loop?It’s harder to have a loop to ensure you are defending all possible attacks?I guess the loop is you need to attack yourself and fix. But attackers only need a single opening.Finding all possible attacks and patching them against yourself is inherently more expensive?

That is why I built <a href="https:&#x2F;&#x2F;safebots.ai&#x2F;safebox.html" rel="nofollow">https:&#x2F;&#x2F;safebots.ai&#x2F;safebox.html</a>Your strategy can’t be patch AFTER an intrusion. Only to build a hardened environment from scratch and be ready in advance.

Seems like there&#x27;s a big attack-defence asymmetry at present: models are great at exploiting systems and poor at fixing them.

Orca-Bench: How Ready Are Language Model Agents for Oncall?