Unify your productivity and AI data with Lakeflow Connect

This is the second post in our series on how Lakeflow Connect brings fully managed data ingestion to every line of business. Following our deep dive into unifying marketing data, this post focuses on unlocking the enterprise knowledge trapped within your organization's productivity stack. We’ll explore how native connectors for wikis, project management tools, and AI foundation models give your downstream AI agents the complete business context they need.

One platform, any source

Lakeflow Connect provides a comprehensive suite of fully managed native connectors across SaaS applications, databases, and file sources that ingest your data directly into the Databricks Platform. Each connector can be set up via a point-and-click UI or a simple API. From there, your data is incrementally ingested as governed, managed tables in Unity Catalog, with no infrastructure for you to provision, manage, or scale.

Prioritized by customer demand, Lakeflow Connect's ingestion connectors span every line of business, including Customer 360, Operations, Finance, and Productivity—the stack closest to day-to-day work.

One platform, every productivity tool

The average company's productivity stack is a sprawl of project trackers, wikis, document stores, inboxes, and AI tools, each holding only a slice of how work gets done. Getting that data out has historically required a patchwork of custom pipelines that are slow to build and painful to maintain. The bar is high enough that the most impactful use cases stay out of reach, for example: AI agents powered by your enterprise's internal knowledge or visibility into operational analytics.

Lakeflow Connect resolves this by providing built-in connectors to ingest seamlessly from across the productivity stack:

• COLLABORATION: How does work get tracked and shipped? Jira, Monday.com, Smartsheet, Aha!, and GitHub
• ENTERPRISE KNOWLEDGE: What collective expertise does your company run on? Confluence, Notion, Glean, SharePoint, and Google Drive
• EMAIL: How do your teams communicate? Outlook and Gmail
• AI: How is your organization adopting AI? OpenAI and Anthropic

With this breadth of connectors, Lakeflow Connect becomes the ingestion engine that brings scattered productivity data into one governed foundation, without the integration bottlenecks that traditionally slow teams down.

For example, Toyota Finance Australia, the financing arm of one of the world's largest automakers, uses Lakeflow Connect to bring its Jira data into Databricks as a fully managed, zero-maintenance pipeline.

“Lakeflow Connect for Jira has transformed how we onboard and consume Jira data within our enterprise data management platform. Moving to the fully managed, UI-driven experience offered by Databricks provided a seamless, zero-maintenance pipeline that automatically handles API complexity and schema changes while delivering governed, analytics-ready Delta tables in the Databricks Lakehouse with minimal effort and maximum reliability.”—Charbel Khoury, Head of Data Office, Toyota Finance Australia Limited

Another customer, VaxCare, a vaccine management company runs the largest primary-care vaccination network in the U.S., and uses Lakeflow Connect's Confluence connector to feed its internal knowledge base into Databricks and ground an AI agent in it.

"Lakeflow Connect's Confluence connector gives us a seamless way to feed our internal knowledge base into Databricks, making that data searchable and actionable for our chatbot today—and ready to power future AI solutions as we build them." —Yasaman Malone, VP of Optimization and AI, VaxCare

Unstructured content, AI-ready out of the box

AI agents can provide instant, grounded answers from your company's collective knowledge without employees having to hunt across wikis, drives, and inboxes. But an agent is only as good as the data behind it, and most of that is unstructured. With Lakeflow Connect, this is not a problem; our connectors can land unstructured content like files, pages, and messages as governed tables in Unity Catalog. The exact data, and the format it arrives in, varies by source:

• SharePoint: The SharePoint connector brings in the unstructured files themselves (PDFs, Word documents, slide decks, and images). Each unstructured file lands as one row, with its raw bytes stored in a content column (either as BINARY or FILE type), with per-file metadata stored alongside. Structured files (Excel files, CSVs, etc.) can also be landed directly as Delta tables.
• Google Drive: The Google Drive connector works the same way for unstructured files, landing each document’s raw bytes as one row in a content column. Structured files (Google Sheets, CSVs, etc.) can also be landed directly as Delta tables.
• Confluence: The Confluence connector brings in the content of every page and blog post as STRUCT type in a body field of the pages and blogposts tables.
• Outlook: The Outlook connector stores the email body in a full_body_content column as a text string (either HTML or plain text), and it stores the attachments in an attachments column as BINARY type. Both live in the email_messages table.
• Gmail: The Gmail connector brings in the full message body as a text string, held in the messages table. It also ingests a reference to each attachment as an attachmentId.
• GitHub: The GitHub connector brings in repository text files (source code, Markdown, and config), stored in the content column of the repo_contents table.

Whatever the source, the content lands in Unity Catalog as a governed table, where you can begin to transform it for downstream AI use cases.

• Text that comes in (like Confluence pages, email bodies, and GitHub source files) is ready to chunk and embed after light cleanup.
• On the other hand, document files that arrive as raw bytes (like the files from SharePoint and Google Drive or the attachments from Outlook), are first turned into text with ai_parse_document, Databricks' built-in document intelligence.

Either way you end up with text you can index for retrieval and put behind an agent. For a deeper look at document processing, watch this on-demand session, Intelligent Document Processing with Lakeflow; and for more on how to get started, read the downstream RAG guide.

M3 Accounting, a leading provider of hospitality software, uses Lakeflow Connect's GitHub and SharePoint connectors to unify its engineering and product documentation into Databricks.

"We had the Lakeflow Connect GitHub connector up and running in minutes. It's saving us about an hour a week by automatically blending our GitHub data with our other engineering operations data, giving us a full picture of developer activity we can actually act on. We pointed the Lakeflow Connect SharePoint connector at our product documentation library and had our internal knowledge assistant pulling from live, up-to-date content within a week. No custom crawlers, no file sync scripts, just a connector and a pipeline." —Ross Serven, VP of Data & AI, M3 Accounting

Govern what you spend on AI with Lakeflow Connect

As AI agents and generative tools spread across teams, so do the questions about the models powering them: what are we spending, who's using what, is it compliant? To help answer these, Lakeflow Connect provides connectors for the biggest foundation model providers:

• OpenAI: The OpenAI connector brings in usage, costs, API keys, projects, users, and audit logs, so you can attribute AI spend by team and track adoption.
• Anthropic: The Anthropic connector brings in audit logs, your organization's users and roles, and the content of claude.ai chats and messages, so you can govern how the tool is used.

Concretely, this enables a single, unified view of how AI usage and adoption are trending within your organiztion, so you can measure and report the ROI of your AI investments to stakeholders.

Build a complete picture of your Atlassian data

Lakeflow Connect’s depth of ecosystem coverage also extends to Atlassian—one of the most widely used suites in any engineering or product org:

• Jira: The Jira connector brings in issues, comments, sprints, boards, and custom fields, so you can track delivery and build dashboards across projects.
• Confluence: The Confluence connector brings in the full text of pages and blog posts, along with their spaces, labels, and attachments, so your team's written knowledge comes in ready to search and analyze.

With both connectors, you get a complete picture of your Atlassian data, the delivery and the documentation that surrounds it, side by side. From there you can track launches, bottlenecks, and resolution times across projects. You can also build AI/BI dashboards, and ask Genie about any of it in natural language.

For example, Digicert, a global leader in digital security, uses Lakeflow Connect’s Confluence connector to reliably power its internal AI Knowledge Assistants without the overhead of custom pipelines.

"Lakeflow Connect's Confluence connector has been a great help in keeping our Knowledge Assistants in Databricks continuously in sync with our latest documentation. It gives us a simple, reliable way to bring Confluence data into Databricks without the overhead of building and maintaining custom pipelines. This has been especially valuable for our Product Bots, helping them stay current and provide our teams with accurate, up-to-date knowledge." —Nikita Raje, Senior Director, Data Engineering, DigiCert

Another customer, Orizon, a Brazilian healthcare technology company, uses Lakeflow Connect's Jira connector to feed its leadership dashboards.

"With the Lakeflow Connect Jira connector, we brought over one million tickets from 168 projects into Databricks, all synchronized daily with 100% success. From this data, we built AI/BI dashboards that automatically feed our Steering Committee's cockpit. Genie has been fundamental: we ask questions in natural language and it delivers answers instantly." —Gustavo Lima, Data Engineering Manager, Orizon

For a step-by-step setup and tuning guide for the Jira connector, see Unify Your Engineering Data: Jira Ingestion Best Practices.

One foundation, every analytics or AI question

Because every connector lands its data as governed Delta tables in Unity Catalog, the productivity and AI data you ingest becomes part of the same foundation as the rest of your enterprise data. From there you can query it directly in Databricks SQL alongside your other sources, ask questions of it in natural language with Genie One, and ground the agents you build with Agent Bricks in current company knowledge.

And because governance, lineage, and access control all come from Unity Catalog, access stays consistent regardless of the team or tool from where the data originally came.

TRegs, a financial technology platform for treasury, risk, and regulatory reporting, is already building on that foundation.

"We needed to see all our costs and projections in one place. By bringing our Monday.com and QuickBooks data into Databricks with Lakeflow Connect, we can now power our Genie AI/BI dashboards to get a full 360-degree view of our current and projected expenses, making our lives super easy." —Smriti Raj, President and CTO, TRegs

Lakeflow Connect is actively rolling out new connectors across every line of business, with support for even more connectors coming soon! Try Lakeflow Connect for free now.

Watch for the next post in this Lakeflow Connect blog series to keep learning how to bring all your enterprise data into Databricks and give your AI agents the full context they need.