Databricks Lakewatch

July 21, 2026

Security Operations Center Modernization with Databricks Lakewatch SIEM
Table of Contents

The market of SIEM is in an inflection point. Enterprises are drowning in security telemetry they cannot afford to store, fighting AI-powered attackers with manual workflows, and losing visibility at the exact moment they need it most. Over the years, security operations teams have been left with an impossible decision to make: pay crippling ingestion fees to store all their data, or to log less and accept dangerous blind spots. Databricks has already joined this market with Lakewatch – an agentic SIEM powered by the Databricks Lakehouse designed to redefine the economics and capabilities of enterprise security operations.

Royal Cyber, a Databricks Certified Partner, has led in enterprise Lakehouse implementation in financial services, healthcare, retail and manufacturing. Having extensive practical knowledge in the fields of data engineering, artificial intelligence/machine learning, and cloud infrastructure, Royal Cyber has assisted hundreds of companies in realizing the full potential of the Databricks platform. With Lakewatch reinventing the way enterprise security operations can be structured, Royal Cyber is the trusted partner that can help organizations navigate the entire lifecycle, starting with the primordial architecture design and pipeline construction, all the way to a full-scale implementation, detection fine-tuning, and AI-assisted SOC enablement.
Explore What Lakewatch Can Do for Your Security Program
Talk toExpert

What Is a SIEM — and Why Is It Breaking?

A Security Information and Event Management (SIEM) platform serves as the brain of enterprise cybersecurity – it gathers logs of firewalls, endpoints, identity providers, and cloud services and correlates that data to identify threats and address compliance requirements. The SIEM market is estimated to cost the world a total of $10.67billion in 2025, and is estimated to cost the world a total of 20.78billion in 2031. However, legacy SIEM design was developed around predictable data rates, and slow adversaries. That era is over.
The statistics speak volumes. The security teams are stretched in all directions: attackers are accelerating, the amount of data is growing rapidly, and the old per-GB pricing approach makes each line of logs a cost reduction choice. The outcome is a security stance that is full of holes, not necessarily because of lax design, but because of architectural limits inherent in the products produced 10 years ago.
Figure 1: The SIEM crisis — key industry statistics driving the shift to lakehouse-native security.
What Is a SIEM
Figure 2: Legacy SIEM vs. Databricks Lakewatch — a direct capability and cost comparison.
Is Your SIEM Creating Blind Spots?
Request Your Free Assessment

Why Databricks Is Entering Security

Databricks serves more than 20,000 organizations — including over 60% of the Fortune 500 — and has crossed a $5.4 billion revenue run rate. More importantly, it already owns the storage, compute, governance, and AI infrastructure that security operations require. Enterprises already keep HR records, application logs, and business data in the Lakehouse; traditional SIEMs cannot access this context without expensive duplication. Lakewatch flips the model entirely: security analysis runs where the data already lives, eliminating the duplication tax and unlocking cross-domain correlation that was previously impossible.
The timing is also driven by market structure. SIEM growth has decelerated from 20% to just 4% annually as buyers lose confidence in legacy architectures. Regulatory mandates like NIS2 and DORA are demanding longer data retention at scale — something that per-GB pricing makes economically untenable. And the industry is consolidating rapidly around AI-native security platforms. Lakewatch arrives precisely at the moment the market is ready for a fundamentally different approach.

Lakewatch: The Open, Agentic SIEM

Launched March 24, 2026 (currently in Private Preview), Lakewatch is a Tier 1 security analytics platform that unifies security, IT, and business data for AI-driven detection and response. It is built on open formats — Delta Lake, Iceberg, and OCSF — with compute-based pricing that eliminates the ingestion tax entirely. The medallion architecture below shows how raw telemetry flows from ingestion through enrichment to AI-ready detection, all governed by Unity Catalog:
Lakewatch The Open, Agentic SIEM
Figure 3: Bronze → Silver → Gold medallion architecture, governed end-to-end by Unity Catalog.

Key capabilities

  • Agent Bricks — Custom AI security agents that autonomously triage telemetry across hundreds of log formats, dramatically reducing mean time to detect (MTTD) and mean time to respond (MTTR). Analysts direct strategy; agents handle the volume.
  • Genie AI — Natural language threat hunting across petabytes of data. Analysts can query years of security history in plain English, and Genie automates detection authoring, false-positive tuning, and SQL translation.
  • Detection-as-Code — YAML and Python rules managed in Git and deployed via CI/CD pipelines, bringing the rigor of DevOps to the Security Operations Center.
  • Anthropic Claude Integration — Advanced reasoning for signal correlation across security, IT, and business data, powering threat triage and executive-ready reporting at machine speed.
  • Unity Catalog — Fine-grained access control at the table, row, and column level, with full audit lineage for NIS2, DORA, and SOX compliance built in from day one.
  • Multimodal Ingestion — Structured logs plus chat, audio, and video content — capturing the unstructured channels where social engineering and insider threats most often hide.

Ecosystem and Early Validation

Databricks launched an Open Security Lakehouse Ecosystem alongside Lakewatch, bringing together Akamai, Arctic Wolf, Cribl, Deloitte, Okta, Palo Alto Networks, Proofpoint, Wiz, and Zscaler. The company also made two strategic acquisitions: Antimatter, for AI agent authentication, and SiftD.ai — whose founder created Splunk’s SPL query language — to accelerate detection engineering on the platform.
Enterprise adoption is already accelerating. Early defenders include Adobe, NAB, Mercedes-Benz, Atlassian, Mars, and NBC Universal. Arctic Wolf alone processes 8 trillion security events per week on lakehouse architecture, validating the platform at extraordinary scale. The Atlassian experience captures the problem Lakewatch was built to solve:

“As we were rehydrating our logs, the legacy SIEM simply couldn’t do it. We had to switch gears and go to Databricks. Without Databricks, we wouldn’t have been able to rehydrate and analyze the data.”

— Niels Heijmans, Sr. Principal Security Architect, Atlassian

Royal Cyber’s Lakewatch Implementation Approach

As a certified Databricks Consulting Partner, Royal Cyber has developed a structured four-phase methodology for deploying Lakewatch across enterprise environments — whether you are starting from scratch or migrating from Splunk, Microsoft Sentinel, or IBM QRadar. Each phase is designed to deliver value quickly while building toward a fully agentic, lakehouse-native SOC.

PHASE 1: Telemetry Inventory

  • Map data sources: Identify and catalog all enterprise security log sources.
  • Design pipelines: Architect the flow of data from ingestion to the Lakehouse.
  • Normalize to OCSF:  Standardize raw telemetry into the Open Cybersecurity Schema Framework.

PHASE 2: Detection Tuning

  • Calibrate AI models: Fine-tune machine learning models for accurate threat detection.
  • Custom rule Authoring: Develop YAML and Python rules specific to your environment.
  • Backtest vs history: Run new detection logic against historical data to ensure efficacy.

PHASE 3: Agentic Workflows

  • Deploy Agent Bricks: Activate custom AI agents to autonomously triage security telemetry. 
  • Genie Spaces setup: Configure natural language interfaces for analyst threat hunting.
  • SOAR/ITSM connect: Integrate Lakewatch with existing ticketing and orchestration platforms.

PHASE 4: Migration & Hybrid

  • Splunk/Sentinel/QRadar move: Execute the transition from legacy platforms to a native Lakehouse architecture.
  • QRadar migration: Specifically manage the migration of existing security workflows.
  • Analyst enablement: Provide hands-on training to SOC teams on AI-driven hunting and response

Bring Security Intelligence Into Your Lakehouse

The transition to lakehouse-native security is not a distant goal, it is already underway, and those organizations that go first will have a long-term edge in the speed of detection, data coverage, and overall cost of ownership. The financials are strong, the technology is tested at scale, and the regulatory pressure is constantly growing. Royal Cyber possesses the accredited expertise, the established implementation approach, and the comprehensive Databricks alliance to assist your organization in doing so with secureness. The Lakewatch team is here to help with your initial assessment, a legacy SIEM migration, or to make the most of your existing Databricks investment.

 The future of enterprise security runs on the Lakehouse — let Royal Cyber help you get there.

Start Your Lakewatch Journey Today
Schedule Your Architecture Review

Frequently Asked Questions

Yes, as long as organizations have already operational Databricks workloads, Lakewatch can be used as the main SIEM. In the case of full-fledged Splunk or Microsoft Sentinel shops, a hybrid solution is the most viable point of departure: Lakewatch takes over analytics, long-term retention and AI-based threat hunting, whereas the current SIEM deals with the alerting and workflow continuity over the migration duration. The route will be based on your existing data maturity and operational capabilities.

All telemetry is stored in Delta Lake or Apache Iceberg format within your own cloud storage account — you own it entirely. You control tiering (hot, warm, cold), and multi-year retention is cost-effective by default with no per-day or per-GB storage surcharge as data ages. This directly addresses one of the biggest pain points with legacy SIEMs, where long retention windows become prohibitively expensive.

Lakewatch entered Private Preview in April 2026. General availability is yet to be announced. Any organization planning to get an early access needs to start the preparation of the readiness by evaluating Databricks footprint, data source inventory, and compliance needs now so that they can get on board as soon as GA is called.

At Royal Cyber, we provide a guided onboarding journey that starts with a SIEM Cost and Coverage Assessment, during which our certified Databricks engineers chart your existing telemetry lapses, ingestion expenses, and compliance requirements against the Lakewatch architecture. There, we work up a phased deployment strategy that fits your environment, be it a greenfield deployment or a migration of Splunk, Sentinel, or QRadar. The majority of clients recognize a significant value in the first 60 days of interaction.

Royal Cyber is an authorized Databricks Consulting Partner with a history of successful enterprise Lakehouse implementations in financial services, healthcare, retail, and manufacturing. Our team consists of cross-functional data engineering and AI/ML, cloud security, and compliance (that is, we do not bootstrap the technology, but rather make it fit your regulatory requirements (NIS2, DORA, SOX) and business environment). Our Databricks practice is committed, immersive, and constantly renewed like the platform itself, unlike generalist SIs.

Author

Haider Jan

AI Engineer

Zainab Batool

Content Writer

Talk To Our Experts

    [recaptcha]

    Recent Blogs

    Websites used to be something you built once and basically…

    Read More »
    ServiceNow Agentic AI: How Intelligent Agents Are Transforming IT, HR, and CSM
    July 21, 2026

    Websites used to be something you built once and basically…

    Read More »

    Websites used to be something you built once and basically…

    Read More »