SOLUTIONS ARCHITECT

I architect data platforms
that survive the real world.

Thirteen years designing scalable AWS, Snowflake and Databricks platforms — lakehouse architectures, streaming ingestion, governance models, and the migrations that get organisations there.

01Cloud Architecture
02Data Platforms
03Streaming & CDC
04Data Modeling
05Governance & Security
06DataOps & IaC
Sources Ingest Stream Landing · S3 Bronze Silver Gold Curation Warehouse Catalog Analytics

I turn complex technical problems into systems that are reliable, scalable, understandable, and built for change.

Certified across AWS and Snowflake. I own architecture end to end — reference architectures, HLD and LLD, data modeling, governance, and the migration path from where a business is to where it needs to be.

0
Years architecting data platforms
0
Pipeline uptime, streaming banking data
0
ELT load time reduced, Snowflake platform
0
AWS & Snowflake certifications
03

Selected Architectures

Systems I've designed

01 / 05

Governed Snowflake Data Platform ↗

WASTE MANAGEMENT · phDATA

Problem

A large waste-management enterprise needed a governed, self-serve analytics platform: batch and real-time ingestion in one architecture, with data products analytics teams could trust without asking a data engineer.

Role

Solutions Architect — reference architecture, HLD & LLD, dimensional models, governance model.

Result

Load time reduced 60%. Governed self-serve data products with lineage built in.

Snowflake · dbt · Airflow · AWS Lambda · S3 · Glue · SNS · SQS · Terraform · GitHub Actions · Great Expectations · Docker

Sources Lambda S3 Airflow Snowflake dbt models GREAT EXPECTATIONS RBAC / MASKING

02 / 05

Enterprise Lakehouse Modernization ↗

DATABRICKS ON AWS · phDATA

Problem

Multiple disconnected sources, no ACID guarantees, and no single place to enforce access or lineage. The business needed incremental, versioned data it could reason about.

Role

Solutions Architect — designed and built the medallion lakehouse end to end.

Result

Versioned, incrementally processed data with governance enforced at the catalog layer.

Databricks · Delta Lake · Delta Live Tables · Unity Catalog · AWS Glue · S3 · Auto Loader · PySpark · Airflow · Terraform

Sources Auto Loader S3 · Delta Bronze Silver · Gold Consumers DELTA LIVE TABLES UNITY CATALOG

03 / 05

Real-Time Banking Ingestion Platform ↗

LLOYDS BANK · PUBLICIS SAPIENT

Problem

High-volume transactional banking data required low-latency processing, strict auditability, and pipelines that could not lose or duplicate a record.

Role

Data Engineering Lead — streaming ingestion architecture and fault-tolerance model.

Result

99.9% uptime, with Spark jobs tuned for both cost and performance.

Databricks · Delta Lake · Kafka · Kinesis · Spark Structured Streaming · Scala · PySpark · Airflow · AWS EMR · S3

TXN Src Kafka Streaming Delta Lake Curated Reporting S3 LINEAGE · IDEMPOTENT RETRY · BACKFILL

04 / 05

Legacy ETL to Snowflake Migration ↗

ALBERTSONS · PUBLICIS SAPIENT

Problem

Legacy ETL workflows had become a performance and maintainability ceiling, with no reliable way to test or document changes.

Role

Data Engineering Lead — led the migration and defined the target architecture, with HLD and LLD to align stakeholders.

Result

Significantly improved performance and maintainability; infrastructure and transformations both version-controlled and testable.

Snowflake · dbt · Airflow · Terraform · AWS S3 · Lambda · Glue

Legacy Assessment Snowflake dbt models Tests Documented TERRAFORM IaC AIRFLOW · RETRIES · ALERTING

05 / 05

Medallion Data Lake on AWS EMR ↗

PRESORT · PITNEY BOWES

Problem

Raw operational data with no governance, no catalog, and query costs that scaled with carelessness rather than value.

Role

Data Engineering Lead — lake layout, governance model and operational runbooks.

Result

Governed, discoverable, cost-aware lake with runbooks for on-call and backfill.

AWS EMR · Glue · Athena · S3 · Glue Data Catalog · Lake Formation · Step Functions · Lambda · IAM · KMS · PySpark · Parquet · Snowflake · Deequ

Sources EMR PySpark Raw · Staging Curated Lake Formation Athena DEEQU QUALITY GATE GLUE CATALOG · KMS
04

Interactive

Every architecture is a trade-off

There is no best design — only the one that fits the constraint you're actually under. Drag the point.

RELIABILITY COST LATENCY —

DRAG THE POINT OR PICK A PRESET

Sources Batch Load Runs on a schedule Streaming As events arrive Data Lake Transform Warehouse Replica Multi-AZ standby Cache Serves hot reads Second region Failover target Analytics SURVIVES A ZONE LOSS BATCH WINDOW 6H · CHEAPER, STALER SPOT COMPUTE · CHEAPER, INTERRUPTIBLE ⚠ SINGLE POINT OF FAILURE
COST $$ p95 240ms RPO 5m RTO 15m COMPONENTS 7

05

Failure analysis

Click anything. Watch what breaks.

Designing for failure means knowing the blast radius before it happens. This is a representative topology assembled from platforms I've built — pick a component and kill it. RTO and RPO figures are design targets, not measured production SLAs.

Failed
—
Blast radius
—
RTO
—
RPO
—
Mitigation
Select a component to simulate its failure.

RESILIENCE · 12 of 13 components survive their own failure with RPO 0

06

Method

How I think about systems

01 / 07

01

Understand the problem

Before any technology decision, I want the business outcome in plain language, and the constraint that makes it hard. Most bad architectures are correct answers to the wrong question.

02

Identify constraints

Latency, compliance, budget, team skill, and the systems already in place. Constraints are not obstacles to the design — they are the design. Architecture that ignores them looks elegant in a document and fails in delivery.

03

Design the simplest viable architecture

Start with the least architecture that satisfies the constraints. Every additional component is another thing to secure, monitor, pay for, and explain to the person on call at 3 a.m.

04

Plan for failure

Design the failure modes before the happy path is finished. Idempotency, retries with backoff, replay from the lake, and a documented recovery path — decided at design time, not discovered in production.

05

Secure the system

Access control, encryption, masking and lineage belong in the architecture diagram, not in a hardening phase afterwards. Retrofitted security is expensive and usually incomplete.

06

Measure and observe

A pipeline you cannot observe is a pipeline you cannot operate. Data quality gates, lineage, SLAs and alerting are delivered with the pipeline, not after it.

07

Scale only when necessary

Scale in response to evidence. Premature scaling adds cost and complexity to solve a problem that may never arrive — and makes the real bottleneck harder to find when it does.

07

How I decide

Principles

01

Design for failure. Everything else is optimism.

02

Complexity must earn its place.

03

Security belongs in the architecture, not after it.

04

Observability is a feature, not an afterthought.

05

Scale intentionally, on evidence.

06

Choose boring technology when boring works.

08

Capability map

Technical landscape

Organised by architectural responsibility, not by logo.

Cloud · AWS

S3 · Glue · Athena · Redshift Spectrum · API Gateway

Compute

Lambda · EMR · Databricks (Photon) · Docker

Data Platforms

Snowflake · Snowpark · Databricks · Delta Lake · Redshift · S3

Ingestion & Stream

Kafka · Kinesis · Flink · Auto Loader · DMS · CDC

Orchestration

Airflow · Dagster · Step Functions

Transformation

dbt · PySpark · Spark · Scala · SQL

Modeling

Kimball · Data Vault 2.0 · Inmon 3NF · Medallion · SCD

Governance

Unity Catalog · Lake Formation · Glue Data Catalog · RBAC · PII masking · Data contracts

Quality

Great Expectations · Deequ · Quality SLAs

Platform & IaC

Terraform · CloudFormation · Docker · Jenkins · GitHub Actions

Observability

CloudWatch · Pipeline observability · Lineage

Patterns

Lambda · Kappa · Medallion · Lakehouse · Data mesh · Event driven · Microservices

09

Trajectory

Experience

Solutions Architect

phData

02.2024 — PRESENT

Responsibility

Own end-to-end platform architecture: reference architectures, HLD and LLD, dimensional and governance models, and the delivery of governed self-serve data products.

Systems

Governed Snowflake platform on AWS (Waste Management); enterprise medallion lakehouse on Databricks with Delta Live Tables and Unity Catalog.

Impact

ELT load time reduced 60%. Quality gates, RBAC, masking and lineage governance delivered as part of the platform, provisioned through Terraform and GitHub Actions.

Data Engineering Lead

Publicis Sapient

06.2021 — 02.2024

Responsibility

Led architecture and delivery for streaming and migration programmes across banking and retail.

Systems

Real-time ingestion on Kafka, Kinesis and Spark Structured Streaming into Delta Lake on Databricks (Lloyds Bank); legacy ETL migration to Snowflake with modular dbt models (Albertsons).

Impact

99.9% pipeline uptime on fault-tolerant, idempotent pipelines meeting banking audit compliance. Spark jobs tuned for cost and performance.

MLOps Lead

Pitney Bowes

04.2017 — 04.2021

Responsibility

Designed production ML pipelines, model deployment workflows and monitoring practices; owned reliability and operational governance across the ML lifecycle.

Systems

AWS-based ML pipelines using S3 and PySpark on EMR for data processing; containerised model services with Flask and API Gateway, with automated deployment and monitoring workflows.

Impact

Improved model delivery through repeatable deployment pipelines and production monitoring. Secured ML workloads with IAM and KMS while strengthening data quality and operational reliability.

Senior Data Engineer

Cognizant Technologies

07.2016 — 03.2017

Responsibility

Led migration of Hadoop Hive pipelines to AWS Redshift and simplified the reporting architecture.

Systems

Unilever analytics platform on AWS EMR, Redshift, S3, Hadoop, Hive, Sqoop and Oozie.

Impact

Improved query performance through optimised SQL and Redshift data models; version-controlled migration scripts for safe handover.

Data Engineer

Hewlett Packard Enterprise

07.2015 — 07.2016

Responsibility

Built customer analytics pipelines and the reporting layer above them.

Systems

Python and Pandas ETL processing customer analytics data from Teradata; Tableau dashboards for sales and support KPIs (Avaya).

Impact

Delivered the sales and support KPI reporting layer end to end, from extraction through to dashboard.

Project Engineer

Wipro Infotech

01.2013 — 06.2015

Responsibility

Built and automated large-scale batch ETL on Hadoop.

Systems

Hive-based ETL pipelines on Hadoop for large-scale sales data (Kellogg's).

Impact

Automated ingestion and validation with Python and Shell, materially reducing manual error.

Education

B.Tech
Dr. M.G.R. Educational and
Research Institute University
2008 — 2012

Certifications — Snowflake

SnowPro Core
Data Engineer
Solution Architect

Certifications — AWS

Solutions Architect Associate
Developer Associate
Data Engineer Associate

10

A little about me

About

Priyank Mishra, Solutions Architect, in a dark jacket against a dark background.
PRIYANK MISHRA · BANGALORE, KARNATAKA

I enjoy working at the intersection of technology and business — understanding what a system actually needs to achieve, identifying the constraints that matter, and designing something that stays understandable as it grows.

Thirteen years in, most of the interesting problems have turned out not to be technical. They are questions of what to model, what to guarantee, what to leave out, and how to hand a platform to a team that has to run it after you leave.

I work across AWS, Snowflake and Databricks, and I care most about the part of the job that happens before any code is written: the reference architecture, the data model, the failure plan, and the conversation that gets everyone to agree on them.

Based
Bangalore, Karnataka, India
Focus
AWS · Snowflake · Databricks
Practice
Reference architecture · HLD & LLD · Migration strategy
Cost optimisation (FinOps) · Presales & estimation
Download résumé PDF

Have a difficult system to design?

Let's make it simpler.

Start a conversation ↗
me@priyankmishra.in LinkedIn Follow on X