LEAD:FUH
How it works
Ingestion → Transformation → Intelligence → Delivery → Upstream
Technical architecture
Consent Gateway
What ist the Learning Analytics Backbone?
Control the dot movements to show the data flow at a calmer or more dynamic pace.
Data Sources
- SAP
- CM (HIS)
- vStpl
- Studyport
- Moodle
- OÜS
- JELAI
- COFFEE
- Etherpad
How it works · Ingestion & Upstream
In this phase, Moodle logs, matriculations, or survey responses flow into the Landing Zone. They remain here for up to 24 hours until consent is processed – each interaction feeds new events back and refines future analyses.
- Landing Zone (temporary): Raw data, automatic deletion after 24 h.
- Upstream: Interactions with feedback and recommendations generate new datasets.
- Example: An accepted learning resource immediately provides new inputs for the next recommendation.
Animation step sequence 3 Steps
- Start landing: Click “Ingest Data” to flow raw data into the Landing Zone and catalog metadata.
- Transformation & Consent: Then click “Transform (dbt)” to reveal consent filters and governance steps.
- Explore use cases: Finally, click “Consume Data” and the ↗ chips such as “Students in Focus” to open the concrete applications. This is also where the exemplary upstream flow begins (Moodle dashboards).
LEAD:FUH Platform
SAP BW
Nightly/Monthly Transfer
(e.g. Assignment Grades)
Ingestion Layer
Landing Zone
raw data, TTL 24h, ALL students
- Moodle Activity Data
- Enrollments (Raw)
- Student Master Data (Raw)
🗑️
Transformation
Federated Governance (consent filter, quality checks, …)
Transformation & Data Quality
Data is cleansed, unified, and enriched – course and module merge into a single entity. dbt and Great Expectations document and monitor every pipeline.
- Transformation: Harmonization, consent filters, governance.
- Data quality: Great Expectations monitors all products.
Core Zone
- Activity (Consented)
- Enrollments (Consented)
- Student 360° Insight
- Course Progress Monitor
- Activity Features
Data Zones · Storage
The Core Zone holds only cleansed data with valid consent – ClickHouse processes behavioral data, PostgreSQL manages consent and results.
Data Catalog
Metadata & Discovery
- Enterprise Glossary
Governance & Self-Service
DataHub provides lineage, glossary, and a self-service marketplace – analysts always know where data originates and who is responsible for it.
Intelligence
- Dropout Risk Scores
- Support Recommendations
Intelligence
Containerized Python services and ML models generate dropout risk predictions, deliver aggregations for dashboards, and feed their learnings back into the system.
Serving & Privacy Layer
Serving & Consent Gateway
FastAPI checks in real time with every request whether active consent is in place. Without opt-in, no data is served – revocations immediately block output and trigger physical deletion.
- Opt-in as default, revocation takes effect immediately.
- Physical deletion removes historical data of revoked consents.
- Full transparency: DataHub documents lineage for audits.
- Serving: Insights are delivered directly to students, instructors, advisors, and researchers.
Technical Architecture
The backbone follows a modern data stack with clearly separated responsibilities – each layer is documented and maintained as a product.
- Ingestion: PeerDB (CDC / Real-time) and Meltano (Batch ELT).
- Storage: ClickHouse for logs, PostgreSQL for results & consent.
- Transformation: dbt with documented models.
- Data quality: Great Expectations monitors products.
- Intelligence: Python ML services for scoring & recommendations.
- Serving: FastAPI gateway with live consent.
- Orchestration: Dagster manages workflows.
- Governance: DataHub provides lineage and self-service.
Data Users
Research
Anonymized Data
- Research Dataset (Anonymous)
- Research 360° View
Student Advisory
Student Success
- Students in Focus
Student/Teacher Dashboard
- MoLA
- My Learning Progress
Faculty
Self-Service BI
- Faculty Dashboard
Data Mesh Principles & Use Cases
Domain ownership remains with the source system teams, data is served as products with owners and quality metrics, and a self-service platform gives analysts direct access – federated governance ensures that data privacy rules apply everywhere.
- Early warning system: detects study dropout risks and delivers supporting recommendations.
- Personalized learning: adaptive feedback and individual learning paths.
- Self-monitoring: dashboards display student progress and activity.
Operations & Sustainability
Long-term Operations
Infrastructure as Code and backups allow for complete recovery. After the project, LEAD:FUH hands operations over to ZDI, while the team continues to maintain ML models and data products – Renovate and similar tools keep all components up to date.
