A Martech Perspective on Databricks’ New CDP Capability

When Databricks announced that it was launching a CDP product, it created a frenzy of activity on LinkedIn. It was an unusual move that created more questions than answers. Every expert chimed in with their two cents, and few added anything beyond hot takes. Now that the dust has settled a little bit, let’s give this a closer look.

To understand why the announcement landed the way it did, it’s worth remembering that the market was just barely coming to terms with the fact that traditional CDPs could barely keep their heads above water. The composable, zero-copy and data-warehouse-native approach was becoming the de-facto approach in a backlash against years of never-ending and expensive CDP projects that had mostly failed to deliver clear ROI. Even that composable approach had started to show its limitations and most activation products (Campaign Management) had muddied the water by professing that they had a built-in CDP. So it quite rightly got a lot of people wondering whether the Databricks announcement was yet again going to upset the applecart.

It’s not a bad question. Composability is pragmatic and can enable businesses to make the most of what they have invested in, as well as build a “best of breed” architecture. At the same time, native functionality can bring architectural advantages in terms of closeness, more effective data transfer protocols and overall cost. Put simply, composing a stack with too many tools means as many UIs, as many contracts, as many integration points, additional maintenance, different tech stack, etc. Lately it’s also become very clear that AI and decisioning had to be somewhat central. There is a point at which the downside of composability outweighs the benefits. As proven by the trail of failed projects, a CDP just by itself has little value except when part of a broader solution that supports use-cases with value and ROI. So yes, why not embed it into Databricks?

So businesses that have selected Databricks as their data-warehouse of choice can now activate a CDP capability. It’s an easy choice in principle, but so far the solution clearly does not have the breadth of capabilities of a traditional CDP (notably in terms of real-time capabilities), even if it could be “enough” for a number of use-cases. The unintended side effect is that it could discourage third party vendors to work with Databricks. Clearly for businesses like Hightouch, it looks like an 80% overlap of capabilities so that relationship is unlikely to thrive. After a few years of filling the reverse ETL gap, Hightouch is getting squeezed by its former partners, Databricks on one side and Braze/Iterable on the other.

As illustrated above, all the solutions adjacent to the CDP have for years been edging in its direction, encroaching on capabilities and increasing the overlap between solutions. So Databricks launching a CDP was not really surprising from that perspective. But the news that really threw me was the announcement that Bloomreach was a launch partner! I remember my remarkably eloquent reaction at the news: “Wait, what?” If you don’t see the massive irony in this announcement, I’ll remind you that the foundation of Bloomreach was the Exponea CDP, a positioning that is still being held through paid SEO and reflected on their website. I know vendors get creative with their positioning but this is reaching unprecedented levels of dissonance.

I wondered what the Bloomreach customers who have been sold a CDP are thinking of this… Using Databricks’ CDP to feed your campaign management’s CDP sounds ridiculous and maybe the term CDP is being misused on both sides. But the issue of overlap in orchestration and segmentation is real. It also exists with reverse ETLs, and once again it is staring straight at us as if asking the market “What you gonna do about it?”

The question to ask yourself is, “Who is actually the user?” or more pointedly “Who owns segmentation and audience building?”. This question highlights the architectural issue with most of the leading CRM solutions: the synchronisation between a fundamentally relational model and a proprietary NoSQL, schema-less customer + event store.

According to Databricks’ documentation on the underlying multi-agent approach, Genie translates the natural-language brief into a precise query, providing the SQL logic behind it for full transparency. So anyone without SQL knowledge can create a query that they’ll be incapable of checking for accuracy. I have no doubt that the “Find me people who haven’t bought in the last 2 months” will work, but what will happen when this is extended to real life scenarios? When the request spans over 4 or 5 tables, needs to take into account the total amount of spend on products in a specific category, exclude transactions that were returned (which can be in a separate table), etc.

Independent benchmarks tell a sobering story. BIRD (a benchmark built from real, messy databases) puts leading models at 73% accuracy. When moving to enterprise-scale complexity (Spider 2.0’s multi-step, agentic, advanced schemas), accuracy falls to 21.3%. Take a few seconds to let those numbers sink in… What head of CRM is going to launch a campaign with an audience that has this level of accuracy? We’re not talking about affinity scoring, we’re talking about messages or journeys that will only make sense if the customer is in the right audience. Expected accuracy is 100%! If it isn’t, someone’s made one of those mistakes that used to get you fired.

Full disclosure: this is where imagino plays. I truly believe that a Query Builder, like the one provided by imagino, is currently the best way to bridge the gap between CRM users and the SQL world. It allows the user to understand the query logic and to tweak the parameters, even when they start from natural language. The truth is that the user of Databricks’ CDP won’t be in marketing, they’ll be in IT and that’s ok as long as expectations are set accordingly.

To conclude this article, I’ll try to briefly tackle one of the underlying questions, specifically from a data-warehouse perspective: does it provide Databricks an edge or is it a distraction from its core focus. Snowflake has always privileged its partner network and constantly invests in those relationships. Databricks has probably made a few enemies but most importantly, that’s a totally different product line that they are not experienced in selling, and a product that they’ll need to defend, maintain and improve. I’m not saying it won’t be fit for purpose in some IT teams but I do doubt that they will convince the marketing crowds.

From a Snowflake perspective, there are two important points to remember:

• They have created a formidable ecosystem of solutions, CDPs just being one of them.
• Snowpark enables some of those solutions to be run fully as Native Apps.

Bearing in mind those two points, I think Snowflake will remain undeterred by the move. There is little in Databricks’ CDP that cannot be achieved by a solution like imagino, purchased via the Snowflake Marketplace, and deployed as a native app. Very little.

Mathieu Lavedrine has spent his career at the intersection of martech and data. 8 years at Neolane/Adobe, Tinyclues (a pre-LLM AI company), and now running partnerships and competitive strategy at imagino, a Snowflake-native CDP/CEP.

Originally published at https://www.linkedin.com.

A Martech Perspective on Databricks’ New CDP Capability was originally published in Snowflake Builders Blog: Data Engineers, App Developers, AI, & Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.