[1] "Linnaeus: A highly reusable and adaptable ML based log classification pipeline", https://arxiv.org/abs/2103.06927
NOTE: the work was actually accepted at the International Conference on Software Engineering (ICSE) in 2021.
The key issue I encountered in industry after was that companies would not allow use for training on their data across customers. Understandably so, as SVM dimensions are readily interpretable given some thought.
So the amount of data you could train on was always rather limited unless you were Google with an own huge dataset.
LLMs "solved" that by just ignoring the previous rules and training on everything, which lets the needed world knowledge seep back in.
I'm sure we'll see many combinations of LLMs and regular machine intelligence in the near future.
Building an SVM isn't a tricky job, but you probably need a data scientist involved, and to think carefully about a labeled dataset.
I think we're seeing the modelling equivalent of something like electron apps. Where they waste a tonne of resources, but make an easier developement experience.
Will be interesting to see how much or little this changes with LLM costs.
With the LLM you don't even need a data scientist to train the model, you just need someone who can connect to APIs. And the LLM is fine with adding and removing categories. No retraining.
It makes the initial faces incredibly fast. Then it is slow and expensive to run, so once the system settles, it becomes the target for model optimization, e.g, a dedicated, or semi-dedicated model.
I initially thought of tickets as just customer support, or internal JIRAs, but the article mentions alarms, security findings, and capacity increase requests, so that makes the scale feel more plausible. Still surprising so many tickets go through the system.
I built a real time character story. In which players interact and integrate fictional characters. There is no delay in response and the characters are all in character. Oh and it’s all voice.
It’s shocking how effective this is.
— "Let's build efficient classifiers"
— "Forget about the silly classifiers, LET'S BUILD A MACHINE GOD"
— "Ah, that machine god is often too expensive and underwhelming. Let's build efficient classifiers"