Remember cable, the offer that combines phone and internet has ever arrived in our mailbox? These offers are highly optimized for conversions, and the offer type and monthly price can vary significantly between two neighboring homes or even between apartments in the same building.
I know this because I used to be a data engineer and built the extract-transform-load (ETL) data pipeline for this kind of offer optimization. Part of my job involves unpacking encrypted feeds, deleting rows or columns with missing data, and mapping fields to our internal data models. Our statistical team then used clean, up-to-date data to model the best offer for each household.
That was almost a decade ago. If you take that process and run it on steroids for a data set 100 times larger today, you’ll achieve the scale that medium and large organizations are dealing with today.
Every step of the data analysis process is ripe for disruption.
For example, a video conference call can generate a log that requires hundreds of storage tables. The cloud has fundamentally changed the way of doing business because of the unlimited storage and scalable computing resources you can get at an affordable price.
Simply put, here is the difference between the old and modern stack:
Image credits: Ashish Kakran, Thomvest Ventures
Why are today’s data leaders interested in the modern data stack?
Self-service analytics
Citizen-developers want access to important business dashboards in real time. They want to automatically update dashboards built on their customer and activity data.
For example, a product team can use real-time product usage data and customer renewal data to make decisions. The cloud makes data truly accessible to anyone, but requires self-service analytics versus legacy, static, on-demand reports and dashboards.
