Unlock to reveal name
Senior Data Engineer
- Location:
- Sri Lanka
- Sector:
- Information Technology - Software
- Profile Created:
- 4 weeks ago
Education
Experience
-
Veracity Digital, Sri Lanka 2026 - Present
Senior Data Engineer
Contributed to designing and maintaining scalable ETL pipelines, Airflow orchestration, data validation frameworks, and PostgreSQL data marts for performance marketing analytics, processing nearly $40M+ in campaign records and 100GB+ of weekly data across production pipelines. Contributed to multi-source ETL pipelines integrating marketing data sources and email data into PostgreSQL, and supported production Apache Airflow DAG management on Astronomer with hourly and daily scheduling workflows. Migrated legacy Cake pipelines to V18 hourly-sync connectors processing 13.9M+ click tracking records, and resolved data consistency issues related to payout calculations, timezone normalization, and ingestion workflows. Tech-Stack: Python, Apache Airflow, Astronomer, PostgreSQL, Airbyte, ETL Pipelines
-
Software Engineer 2025 - 2026
Tensorglade, Sri Lanka
At Tensorglade, I worked on a project for a US-based media company, developing an AI-powered lyric video generation platform. My work focused on speech-to-text forced alignment using deep learning, custom animation rendering, and deploying a scalable streaming system on AWS. Implemented an AI based custom animation rendering pipeline for word-level lyric videos with accurate audio-text synchronization. Implemented speech-to-text forced alignment using NVIDIA NeMo and PyTorch to accurately align lyrics at word level with audio. Deployed the system on AWS ECS using containerized services, configuring task definitions, load balancers, and autoscaling for high availability and production readiness. Tech-Stack: Python, NVIDIA NeMo, FastAPI, AWS S3, DynamoDB, ECS, EC2, Docker
-
Network Softwarization and Security Labs, Ireland 2025 - 2026
Research Assistant
At NetSLab , I worked as a Research Assistant on securing large-scale Federated Learning systems. My work focused on designing and deploying distributed FL infrastructures, developing LLM-based defense mechanisms against poisoning attacks. Conducted a research on Federated Learning security, focusing on detection of poisoning attacks in distributed systems. Designed and developed DistilGuard, a novel LLM-based defense mechanism for identifying malicious client updates in Federated Learning systems. . Deployed a distributed Federated Learning (FL) system using Docker, Kubernetes, and gRPC-based orchestration (Flower framework) to enable rapid testing and simulation of federated learning workflows. Developed a novel defense mechanism for Federated Learning, resulting in a first-author publication at IEEE CNS 2025.
-
London Stock Exchange Group, Sri Lanka 2024 - 2025
Software Engineer Intern - Data Engineering Team
LSEG Trade Surveillance is a production platform for detecting market abuse and financial crime, I worked within Team Spark (Data Engineering Team) on production-grade financial data ingestion systems. Designed and validated scalable, cloud-based data ingestion pipelines for high-volume financial datasets using AWS S3, SQS, RDS, EC2 and Apache Airflow. Developed and orchestrated Airflow DAGs to monitor workflows by querying the Airflow metadata database. Reduced Airflow DAG triggering bottlenecks by optimizing AWS SQS message retrieval logic, implemented dynamic message consumption based on real-time Airflow metrics. Developed a Ingestion Performance Testing Tool to benchmark data ingestion pipelines, SQS queue processing, and AWS EMR job execution to evaluate system throughput . Tech-Stack: Python, Apache Airflow, SQL, AWS (MWAA, S3, EC2, RDS,SQS )