Detailed Course Outline
Challenge 1: Select and provision storage for an enterprise data lake
In this challenge, you will…
Learning objectives:
- Compare and contrast Azure storage offerings
- Provision the selected Azure storage service
Challenge 2: Ingest data from cloud sources
In this challenge, you will….
Learning objectives:
- Orchestrate the ingestion of data from multiple cloud-based sources to a single cloud-based store
- Ensure the protection of specific customer data at all times leveraging the current technology set and solution architecture
Challenge 3: Pull data from on-premises and establish source control
In this challenge, you will…
Learning objectives:
- Orchestrate the ingestion of data specifically from maintained “on-premises” solutions
- Implement a cloud-based source control repository for the developed solution
Challenge 4: Transform and normalize data within the lake and establish branch policies
In this challenge, you will…
Learning objectives:
- Transform data into a normalized schema for downstream consumption
- Create new policies to make certain all future changes leverage an appropriate review process
Challenge 5: Populate a data warehouse and implement unit tests
In this challenge, you will…
Learning objectives:
- Transform the data from the various source systems into a common data warehouse schema to support the generation of specific reports mandated by the business
- Orchestrate the dataflow into the data warehouse in an automated manner
- Build out unit tests across core components of the data pipeline
- Integrate automated testing into the code review process
Challenge 6: Differential data loads and telemetry
In this challenge, you will….
Learning objectives:
- Modify the solution to include doing differential data loads as well as the original bulk load
- Automate data load and processing to run daily
- Implement rich telemetry into the dataflow and deployment pipelines
- Add error handling to raise pipeline issues in real-time
Challenge 7: Automated deployment with validation and approval
In this challenge, you will…
Learning objectives:
- Operationalize the solution deployment process through automation
- Create and implement a testing environment
- Implement automated deployment processes and policies