Technology
Built With Modern Data Technologies
The stack
Technologies we work with
Technologies Datalloy works with. No vendor partnership, certification or endorsement is implied.
Microsoft Azure
Cloud platform
Azure Data Factory
Ingestion & orchestration
Azure Databricks
Data engineering & processing
Azure Data Lake
Lake storage
Azure Synapse Analytics
Analytical warehouse
SQL Server
Relational database
Azure SQL
Managed relational database
Power BI
Semantic layer & reporting
Talend
Data integration
Python
Engineering & automation
REST APIs
Source integration
By layer
Where each technology sits
Every layer of the architecture has a job. These are the tools Datalloy typically uses to do it, and what they are actually used for.
Cloud Platform
The environment the data platform is built and governed in.
Microsoft Azure
Cloud platform
Resource organisation, environment separation, networking and identity for the data platform.
Ingestion & Orchestration
Moving data from source systems on a reliable schedule.
Azure Data Factory
Ingestion & orchestration
Source connectivity, incremental extraction, parameterised pipelines and dependency-aware scheduling.
Talend
Data integration
Integration and ETL jobs where an existing Talend estate is in place or a portable tool is preferred.
Processing & Engineering
Transforming raw data into curated, business-ready structures.
Azure Databricks
Data engineering & processing
Large-scale transformation, cleansing and curation of lake data using Spark and notebooks under source control.
Storage
Where raw, curated and historical data lands.
Azure Data Lake
Lake storage
Raw, curated and archive zones with a folder and file-format convention that keeps reprocessing possible.
Warehouse & Databases
Modelled, query-optimised storage for analytics.
Azure Synapse Analytics
Analytical warehouse
Warehouse workloads, distribution and partitioning strategies for modelled fact and dimension tables.
SQL Server
Relational database
On-premises warehouses, staging layers and source extraction from existing operational databases.
Azure SQL
Managed relational database
Managed warehouse and mart workloads where a relational engine is the right fit for the volume.
Business Intelligence
The semantic layer and the reporting people actually open.
Power BI
Semantic layer & reporting
Semantic models, governed measures, DAX calculations, row-level security and report distribution.
Languages & Interfaces
The tools used to build, integrate and calculate.
Python
Engineering & automation
Custom extraction, API integration, data validation routines and platform automation.
REST APIs
Source integration
Authenticated, paginated and rate-limited extraction from SaaS platforms and internal services.
Reference architecture
How it fits together
A typical Azure-based Datalloy platform. The shape stays consistent; the components change with volume, budget and the systems already in place.
Source Systems
- ERP
- CRM
- APIs
- Databases
- SaaS
Data Ingestion
Azure Data Factory
Data Lake
Raw and curated zones
Data Engineering
Databricks transformation
Data Warehouse
Azure Synapse
Data Modeling
Facts and dimensions
Semantic Model
Governed measures and hierarchies
Power BI
Dashboards and reporting
Business Insights
Decisions people can act on
Working with a different stack?
Datalloy works with the estate that already exists, not the one a slide deck would prefer.