Retail applications store day-to-day data such as products, customers, orders, shipments, payments, inventory, returns, and refunds inside databases.
But this data cannot always be used directly for reporting or analytics from the same application database. In real projects, data is usually moved into a central analytics platform first, so other teams and pipelines can safely use it.
In this pipeline, you will build that first step on GCP.
You will take retail data from a relational database, load it into BigQuery, and use watermarks to load only new or changed records instead of reloading everything every time.
After completing this pipeline, you will be able to: