Gold Dimension and Fact Tables with AWS Glue

Transform trusted Silver healthcare data into Gold dimension and fact tables using AWS Glue, Apache Iceberg, Athena, and MWAA.
AWS • Lakehouse • Intermediate • Healthcare

Architecture Diagram

Build this project

  • Source code
  • Setup files
  • Architecture walkthrough
  • Implementation videos
  • Execution guidance
  • Support
Get Project Access

Included in Data0to1 Project Library

What this project solves

Silver data is clean and trusted, but analytics teams usually need data shaped into business-friendly tables.

For reporting and downstream analytics, healthcare data often needs to be organized into dimensions such as patients, providers, departments, facilities, and dates, along with fact tables for encounters, vitals, labs, medications, diagnoses, procedures, charges, allergies, problems, and discharge summaries.

In this pipeline, you will create the Gold Core layer of an AWS lakehouse. You will transform Silver healthcare tables into business-ready dimension and fact tables, store them as Apache Iceberg tables on Amazon S3, track pipeline runs and quality checks, and validate the results using Athena.

What you’ll build

  • Build business-ready Gold tables from Silver data
  • Create dimension tables for facilities, departments, providers, patients, and dates
  • Create fact tables for encounters, vitals, labs, medications, charges, diagnoses, procedures, allergies, problems, and discharge summaries
  • Store Gold tables as Iceberg tables on Amazon S3
  • Track pipeline runs using audit tables
  • Create data quality metrics for important Silver tables
  • Validate Gold tables using Athena
  • Run the Gold pipeline using MWAA and Airflow

How you’ll build it

  • Read trusted Silver Iceberg tables from Amazon S3
  • Use AWS Glue to transform healthcare data into dimensions and facts
  • Write Gold tables as Apache Iceberg tables
  • Add audit logs and basic data quality checks
  • Orchestrate the Gold pipeline using MWAA/Airflow and validate with Athena

Tools you’ll use

Amazon S3 • AWS Glue • Apache Iceberg • Amazon Athena • Amazon MWAA • Apache Airflow • AWS Glue Catalog • Apache Parquet

What you’ll walk away with

After completing this pipeline, you will be able to:

  1. Explain how Silver data becomes a Gold analytics layer
  2. Create dimension and fact tables for analytics use cases
  3. Store curated Gold tables using Apache Iceberg on Amazon S3
  4. Track pipeline runs and basic data quality metrics
  5. Orchestrate and validate Gold layer processing using MWAA and Athena

Ready to build this project?

Get access to the project code, setup files, architecture walkthrough, implementation videos, and support.

Get Project Access