@drwhax oh I've been talking about smaller open-weights models for a long time now. A leaked Google memo ("we have no moat") mentioned them as a massive problem for them years ago. I have much less problem with using small, specialized, self-hosted, open-weights models.
In fact I know of at least one small company that already does this for vulnerability testing of their own code, avoiding most of the BS.
That said, it is still immensely expensive for them to run it on their own infra.
@drwhax I am very very doubtful we will in any meaningful way.
In the end coding is an exercise in translating intentions into machine-readable code, and also an exercise in communication between those whose intentions are enshrined in code, and those who then need to maintain it.
We will be able to automate some things slightly better, though. But then the question is: at what cost?
@drwhax the problem is this works better for certain tasks (finding vulnerabilities) and much worse for other tasks (vibe-coding) because of the shape of these tasks.
"Human in the loop" is not the get-out-of-LLM-problems-free card people try to pretend it is.
Human in the loop works for vulnerability findings because there is a reliable way of verifying the finding. It clearly does not work well for vibe-coding at all because there is no such reliable way of verifying code correctness.
@drwhax yup. All I am saying is that this distinction is important and needs to be constantly stressed, because people will end up creating more problems by vibe-coding.
@drwhax and in the end this will come down to managing limited resources and incentives, as always.
If a company can spend $1m in tokens on their codebase, are they incentivized to spend it on finding vulnerabilities, or churning out new vibe-coded features? I know what my guess is.
@drwhax finding vulnerabilities can be stochastic because it has a very clear and effective verification function: either the exploit works or it does not. Exploit code can be messy and convoluted, as it is not going to be maintained after the vulnerability is fixed.
Vibe-coding fixes does not have that kind of verification function: the fix must not only close the specific vulnerability, but *also* not introduce new ones or re-introduce old ones, and it has to be maintainable in the future.
@DekOfTheYautja I don't think my bike has an alienator 🤣
The alternator is fine. I knew the battery was low when I turned it over this morning, but I thought it had enough to kick it to life again. The level crossing was down, so I turned the bike off and it wouldn't kick. Jumper pack sorted it and I took the long way home - bike would then kick over when I got back.
