Modern Data Stack with Chad Sanderson

Data-Driven Podcast

Listen to this podcast with Chad Sanderson, head of the data platform team at Convoy, which includes Convoy’s Data Engineering team, Data Warehouse, Data Pipeline tooling, Experimentation Platform, Machine Learning Platform, Analytics Platform, and Streaming. Hear how Chad’s team is building out one of the most advanced Experimentation and Machine Learning Platforms in the world from the ground up, servicing thousands of carriers every day to ship freight more efficiently.

See All Podcasts Button Arrow
Quote icon

The biggest thing that’s affecting us is how the direction of the modern data stack was a pretty clear initial divergence from some of the original data warehousing philosophies. It seems like horseshoe theory is in play where we’re kind of circling around back to a lot of the stuff that was really popular in, like the nineties when it came to data warehouse, design, entity, relationship diagrams, things like that.

Anytime something breaks the data scientists have to go and act like Sherlock Holmes and figure that out or try to throw it to the data engineering team so they can go and figure it out. So, what we’re trying to do is create that clear concept of ownership from the source, having a software engineer, actually own the production code, the events in this case. So we’re using Kafka streams and those Kafka events tied directly to the semantic events that are most relevant to our business, the Semantic events, and the semantic entities.

Be Data-Driven At Scale