Python Data Engineer
Our client is building an operations management platform that handles workforce planning, rostering, dispatching, and performance analytics for a large-scale field operations environment. You will own the Python engine layer, responsible for the algorithms and data processing that power the core operational workflows used daily by operations teams.
- This is not a typical web backend role. You will be working on constraint-based planning algorithms, large-scale data processing pipelines, and event-driven dispatch engines that operate in real time against live operational data.
- WHAT YOU'LL OWN:
- Planning engine:
- processing large volumes of operational schedule data through constraint evaluation, business rules, and engagement profiles to produce workforce planning demands. Built on Celery, Polars, and OR-Tools running on Azure App Service
- Rostering engine: generating periodic staff rosters by solving constraint problems against planning demands, employee certifications, and leave data pulled from third-party HR systems. Running as an Azure Function App
- Dispatch engine:
- real-time task assignment using proximity scoring against live employee location data, attendance records, and operational schedule data. Running as an always-on Azure Function App or App service.
- Performance engine:
- scheduled KPI aggregation and SLA scoring across all modules. Running as a timer-triggered Azure Function App
- WHY'S THIS ROLE INTERESTING:
The planning engine is the core of this platform. The algorithm you build determines how efficiently an operations team deploys its workforce against a complex schedule of demands. When the platform matures, the rule-based constraint engine you build will become the foundation for an ML-driven prediction model. You are building something that runs in a live operational environment and directly impacts how a team performs on the ground every day.
WHAT YOU'LL WORK WITH:
Python 3.11
Celery with Redis as broker for long-running jobs
- Polars for large-scale data processing
- OR-Tools for constraint solving and optimisation
- Azure Functions for event-driven and scheduled workloads
- Azure SQL via SQLAlchemy for output persistence
- Azure Blob Storage for intermediate file handling
- Azure Service Bus for async message queuing between modules
- Third-party HR and workforce management API integrations
- Real-time location data streams for proximity-based task assignment
- Requirements
- WHAT WE ARE LOOKING FOR:
- Required:
5 or more years of Python backend or data engineering experience
- Experience with constraint solving, optimization, or scheduling algorithms (OR-Tools, PuLP, or equivalent)
- Strong data processing experience with large datasets (Polars, pandas, or Apache Spark)
- Experience with task queues and async processing (Celery, RQ, or equivalent)
Familiarity with Azure
- Comfortable working with SQL databases and writing performant queries
- Experience consuming REST APIs and message queues
- Preferred:
Experience with Azure Functions or equivalent serverless compute
- Experience with OR-Tools specifically
- Knowledge of scikit-learn or ML model integration (ONNX, Azure ML) for a future ML upgrade path
- Experience with Pydantic for data validation and schema enforcement
- Nice to have:
- Experience with large-scale workforce management or scheduling systems
- Familiarity with SLA-based operational frameworks
- Experience working in fast-paced, operationally critical environments
- WHAT YOU "DO NOT NEED" TO KNOW:
- This is a pure Python engineering role. You will not be expected to write TypeScript, work on the API layer, or contribute to the frontend.
- Your responsibility sits clearly within the Python processing layer. The integration boundary between your work and the wider platform is
- Redis and Azure SQL. You will write clean, reliable outputs into these systems, and the surrounding services will consume them from there.
- In simple terms:
- you own the Python logic and data processing. The rest of the platform reads from what you produce.