Automated triage is genuinely useful for the volume problem. It clusters related alerts, summarises what happened and closes the obvious noise, which frees analysts for work that needs judgement. The risk is not that it makes mistakes, since people do too. The risk is that its mistakes are invisible, because a closed alert produces no evidence that anything was missed.
Where it performs well
Repetitive, well understood alerts. A failed login from a known travelling user, a blocked malware detection, a scanner triggering a rule for the fortieth time this week. Summarising a long alert chain into readable narrative is another genuine strength, and so is enrichment: pulling the asset owner, the user’s role and recent related events into one place saves an analyst several minutes on every ticket. Multiply that across a queue of several hundred alerts a day and the improvement is real, particularly for a small team that would otherwise triage by guesswork.
Where it struggles
Slow, quiet activity that looks legitimate in each individual event. An attacker reading files at a normal rate, using valid credentials during working hours, generates alerts that resemble ordinary work at every step. Novel techniques are the other weakness, because the model has been trained on what came before. NIST’s AI Risk Management Framework, published in 2023, is built around exactly this question of knowing where a system’s judgement is reliable and where it is not, and treating that boundary as something you manage rather than assume.
“Ask what proportion of automatically closed alerts gets sampled by a person, and how often that sampling finds something. If nobody knows the answer, the automation is not being supervised, it is being trusted. Sample five per cent every week and record what you find, because that number is the only evidence you will have that the triage is working.”
William Fieldhouse, Director, Aardwolf Security Ltd
Automation bias in the team
The subtler risk is what it does to analysts. When a system suggests a verdict with a confident summary, people agree with it more often than they should, particularly under pressure at the end of a shift. Guard against that by having the tool present evidence rather than conclusions where the decision matters, by rotating the analysts who review automated closures, and by making it easy to disagree without justifying the decision at length. A team that never overrides the automation is not a well tuned team.
Testing the whole detection chain
The way to know whether any of this works is to generate real activity and see what happens. Run controlled exercises that produce the behaviour you expect to detect, then measure what was alerted, what was triaged automatically and what reached a person. Penetration testing providers in the UKwill often include a detection assessment alongside the technical testing, which gives you the outside view. Continuous vulnerability managementreduces the volume of genuine problems in the first place, which is the most reliable way to make any triage process perform better.
Frequently asked questions about AI in security operations
These questions come up when a security team evaluates automation.
Can automation replace tier one analysts?
It changes the role rather than removing it. Somebody still has to supervise the automation, handle what it escalates and investigate the cases it cannot classify, and that work needs the judgement tier one used to develop.
Is feeding logs to an external service a risk?
It is a data processing decision that deserves the same review as any other. Understand what leaves your environment, where it is stored and for how long, and confirm the arrangement fits your obligations before enabling it.
