About AquilaClouds
AquilaClouds is a Silicon Valley-based startup revolutionizing AI-powered cloud management. We’re building the next generation of autonomous cloud platforms—driven by AI and intelligent automation—that self-optimize, proactively protect, and seamlessly migrate workloads across on-premises, multi-cloud, and containerized environments.
We foster a collaborative culture focused on collective growth, delivering exceptional value to our customers, and tackling ambitious challenges. We believe in asking tough questions, supporting each other with critical thinking and innovative solutions, and winning together.
If you’re ready to join a team driving exponential innovation and growth, let’s talk.
About the role
As a Principal Senior Software Engineer on the Cloud Billing Collectors & Data Ingestion team, you will build and scale the foundational pipelines that power Aquila’s data platform. Every number our enterprise customers rely on begins as a raw billing artifact across heterogeneous cloud providers—multi-gigabyte AWS CUR parquet drops, Azure EA/MCA consumption exports, GCP BigQuery tables, OCI cost reports, and Databricks system tables.
Your mission is to own the architecture that ingests, normalizes into a FOCUS-aligned model, and lands this mission-critical data into high-throughput analytical PostgreSQL infrastructure.
This role is directly impactful to the business: you will solve fundamental challenges in data correctness, sub-cent invoice reconciliation, late-arriving restatements, and provider API quotas at immense scale. By ensuring high reliability and ultra-low ingestion latency, your work directly enables enterprise leaders to trust their FinOps and billing analytics every single day.
Responsibilities and Duties
- Own one or more cloud collectors end to end — connector onboarding, credential/auth model, extraction, normalization, load, reconciliation, and monitoring.
- Build and maintain ingestion pipelines for AWS (CUR / CUR 2.0 / FOCUS parquet exports), Azure (EA and MCA consumption details), GCP (BigQuery billing export), OCI cost and usage reports, and Databricks system tables.
- Design the normalization layer that maps heterogeneous provider schemas onto Aquila’s common cost model, including resource ID parsing, tag extraction, currency and amortization handling, and FOCUS specification compliance.
- Build bulk-load paths into partitioned PostgreSQL tables based staging, upserts, idempotent reprocessing, and partition management.
- Own reconciliation: automated checks that our ingested totals tie back to provider invoices, plus the tooling to root-cause a variance when they don’t.
- Handle the auth surface across providers — cross-account IAM roles and STS AssumeRole, Azure Service Principals and managed identities, GCP service accounts and Workload Identity Federation, OCI API signing keys.
- Instrument pipelines for observability: run status, row counts, duration, cost of ingestion, and alerting on drift or silent failure.
- Work with product and customer-facing teams to onboard new enterprise accounts and debug ingestion issues in live customer environments.
Skills & Qualifications
Must-Haves:
- 6-10+ years of hands-on experience building scalable enterprise software, focused on data engineering, ETL/ELT pipelines, or data ingestion.
- Strong Java and Spring Boot expertise (REST APIs, JSON, JPA/Hibernate, HikariCP, scheduled/long-running batch processes).
- Strong Python proficiency for cloud SDKs and production data pipeline maintenance.
- Solid PostgreSQL experience (bulk loading, table partitioning, upserts, indexing, and high-volume query optimization).
- Hands-on experience with at least two major cloud providers (AWS, Azure, GCP, OCI)—specifically storage, IAM, and billing APIs.
- Expertise in cross-account cloud authentication and security protocols (AssumeRole, Service Principals, Workload Identity Federation).
- Solid understanding of data correctness principles: invoice reconciliation, idempotency, deduplication, schema evolution, and restatement handling.
- Strong technical written and verbal communication skills for cross-team collaboration and technical documentation.
Nice-to-Haves:
- Familiarity with BigQuery (Storage Read API), Databricks/Spark, DuckDB, or columnar storage formats (Parquet/ORC).
- Experience with Airflow or equivalent orchestration platforms, Kubernetes, and CI/CD automation.
- Domain knowledge in FinOps, FOCUS specifications, or multi-cloud billing standards.
Qualification: BTech/BE/MTech/MS/MCA or equivalent.