Infosec.Pub
  • Communities
  • Create Post
  • heart
    Support Lemmy
  • search
    Search
  • Login
  • Sign Up
digicatM to blueteamsecEnglish · 17 days ago

Beyond the Hype: Evaluating LLM Integration and Practical Limitations in Security Operation Centers

arxiv.org

external-link
message-square
0
link
fedilink
3
external-link

Beyond the Hype: Evaluating LLM Integration and Practical Limitations in Security Operation Centers

arxiv.org

digicatM to blueteamsecEnglish · 17 days ago
message-square
0
link
fedilink
Large Language Models (LLMs) are increasingly being explored within Security Operation Centers (SOCs) to support text-heavy analytical work such as alert contextualization, incident summarization, and drafting investigative artifacts. Despite this interest, practitioners describe critical operational concerns, most notably hallucinations (plausible but incorrect outputs), opaque reasoning, and the verification effort required to safely use model-generated content in security workflows. In this paper, we present findings from semi-structured interviews with 20 SOC practitioners spanning frontline analysts, SOC managers, and tool developers. Participants report perceived time savings for low-stakes tasks that are quickly verifiable (e.g., summarizing logs or drafting initial investigative leads), but they consistently frame LLM outputs as preliminary drafts and suggestions rather than decision-grade conclusions. Participants also describe limited trust in LLMs for high-stakes security decisions due to unreliable outputs and unclear model reasoning, and they report relying primarily on ad-hoc verification norms and continuous human oversight rather than standardized mitigation procedures. Based on these interview-grounded accounts, we introduce a maturity rubric to characterize readiness for LLM integration and outline a research agenda emphasizing auditability and transparent explanation mechanisms to support safer adoption in SOC workflows.
alert-triangle
You must log in or # to comment.

blueteamsec

blueteamsec

Subscribe from Remote Instance

Create a post
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !blueteamsec@infosec.pub

For [Blue|Purple] Teams in Cyber Defence - covering discovery, detection, response, threat intelligence, malware, offensive tradecraft and tooling, deception, reverse engineering etc.

Visibility: Public
globe

This community can be federated to other instances and be posted/commented in by their users.

  • 49 users / day
  • 118 users / week
  • 330 users / month
  • 1.04K users / 6 months
  • 252 local subscribers
  • 792 subscribers
  • 4.67K Posts
  • 352 Comments
  • Modlog
  • mods:
  • digicat
  • UI: 0.19.19
  • BE: 0.19.19-jemalloc
  • Modlog
  • Instances
  • Docs
  • Code
  • join-lemmy.org