Key Skills

Job Description

About the team & role The Data Platform team builds and operates the shared infrastructure that powers how data is stored, moved, and consumed across Alpaca. Our platform supports financial transactions, customer data, API and system events, enriched datasets, and third-party sources that power critical decisions and products for internal teams and external stakeholders. We process hundreds of millions of events each day and that scale continues to grow as Alpaca serves more customers and launches new products. As a Senior Data Platform Engineer, you'll design, build, and operate reliable systems across Alpaca's data platform, from distributed query engines and lakehouse storage to streaming infrastructure, governance, and data serving. You'll also work closely with our AI and Data Science teams to enable agentic workflows and machine learning capabilities. As part of a small team, you'll contribute across multiple initiatives, own projects from design through production, and help shape the future of our platform. Our team is 100% distributed and remote. What you’ll do Design, build, and operate core data platform infrastructure, including distributed query engines, lakehouse storage, orchestration, and streaming pipelines. Contribute to regional platform deployments, building scalable and repeatable infrastructure using Kubernetes and infrastructure-as-code. Develop real-time data processing, analytics, and alerting capabilities for operational and product use cases. Partner with AI and Data Science teams on the platform foundations for agentic workflows, evaluations, and ML lifecycle tooling. Evolve our data catalog, governance, and access controls so data is discoverable and properly governed. Own platform reliability through observability, on-call participation, and operational improvements while collaborating with cross-functional teams. Must-haves 5+ years of experience in data infrastructure, platform engineering, or software engineering, with demonstrated end-to-end ownership of production systems. Strong Python and SQL skills, with experience building reliable pipelines, services, or platform tooling. Experience building or operating distributed data systems, such as query engines, lakehouses, or streaming platforms. Experience operating cloud infrastructure using Kubernetes, infrastructure-as-code, and deployment automation. Sound operational judgment, clear technical communication, and the ability to collaborate across engineering teams. Nice to Haves: Experience with distributed query engines such as Trino, Presto, or Spark, and open table formats such as Apache Iceberg. Experience with streaming and real-time data infrastructure such as Kafka, Redpanda, Flink, or Debezium. Experience with AI and agent infrastructure, including MCP servers, retrieval systems, agent observability, or LLM evaluation. Experience supporting ML infrastructure using technologies such as MLflow, Vertex AI, or similar platforms. Experience with real-time analytics engines such as ClickHouse, Pinot, or Druid. Experience designing and operating infrastructure across multiple cloud regions, including workload isolation and cost-performance trade-offs.

Core Responsibilities

Design, build, and operate data platform infrastructure spanning distributed query engines, lakehouse storage, orchestration, streaming, regional deployments, and real-time analytics. Improve platform reliability, governance, and access controls, and partner with AI and Data Science teams to support agentic workflows and machine learning capabilities.

Requirements

Requires 5+ years of experience in data infrastructure, platform engineering, or software engineering, with end-to-end ownership of production systems. Candidates should have strong Python and SQL skills, experience with distributed data systems and cloud infrastructure using Kubernetes and infrastructure-as-code, and sound operational judgment and communication skills.