Anthropic· Software Engineering - Infrastructure· San Francisco, CA | Seattle, WA
Staff + Sr. Software Engineer, Cloud Inference
Classified Tasks (22)
Automate 0%Augment 64%Human-Only 36%
Augment (14)
AI assists, human decides
Build backend services and infrastructure to enable Claude serving across AWS, GCP, Azure, and additional cloud platforms
technical
Integrate APIs between Claude and cloud provider platforms
technical
Implement intelligent request routing to direct traffic across providers and regions
technical
Operate inference execution stacks to run model inferences reliably on cloud accelerators
operational
Manage capacity planning and provisioning to match compute supply with demand
operational
Develop and apply autoscaling strategies for inference workloads
technical
Design and implement workload routing strategies that direct requests to the most cost-effective accelerator and region
technical
Stand up the full serving stack on new cloud platforms
operational
Build and evolve CI/CD automation systems, including validation and deployment pipelines, to reliably ship new model versions across cloud platforms
technical
Validate and deploy new model versions and features without regressions to millions of users
operational
Analyze observability and telemetry data across providers to identify performance bottlenecks, cost anomalies, and regressions
analytical
Drive remediation actions based on real-world production workloads and observability findings
operational
Scale and optimize Claude serving infrastructure to support large developer and enterprise audiences
technical
Optimize compute utilization and cost across cloud providers
analytical
Human-Only (8)
Requires human judgment
Design backend services and infrastructure that serve Claude across multiple cloud service providers, accounting for differences in compute hardware, networking, APIs, and operational models
technical
Own the end-to-end product of Claude on each cloud platform, including API integration, request routing, inference execution, capacity management, and day-to-day operations
leadership
Collaborate cross-functionally with internal inference, product API, systems, and security teams and with cloud provider partners to deploy and operate the serving stack
communication
Resolve operational issues across cloud platforms and production services
operational
Influence cloud service provider roadmaps through partnership and technical feedback
communication
Design interfaces and tooling abstractions across cloud providers to enable cost-effective inference management and reduce per-platform complexity
technical
Make architectural decisions to maintain reliability and cost-effectiveness at massive scale
technical
Ensure deployed LLMs meet safety, performance, and security standards
operational
Job description
About Anthropic Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems. About the Role The Cloud Inference team scales and optimizes Claude to serve the massive audiences of developers and enterprise companies across AWS, GCP, Azure, and future cloud service providers (CSPs). We own the end-to-end product of Claude on each cloud platform, from API integration and intelligent request routing to inference execution, capacity management, and day-to-day operations. Our engineers are extremely high leverage: we simultaneously drive multiple major revenue streams while optimizing one of Anthropic's most precious resources: compute. As we expand to more cloud platforms, the complexity of managing inference efficiently across providers with different hardware, networking stacks, and operational models grows significantly. We need product-minded backend engineers who can navigate these platform differences, design the services and abstractions that work across providers, and make architectural decisions that keep us reliable and cost-effective at massive scale. Your work will increase the scale at which our services operate, accelerate our ability to reliably launch new frontier models and innovative features to customers across all platforms, and ensure our LLMs meet rigorous safety, performance, and security standards. What You'll Do Design, build, and own backend services and infrastructure that serve Claude across multiple CSPs, accounting for differences in compute hardware, networking, APIs, and operational models Work cross-functionally with internal inference, product API, systems, and security teams, among others, and with CSP partners to stand up the full serving stack on new cloud platforms, resolve operational issues, and influence provider roadmaps Build and evolve CI/CD automation systems, including validation and deployment pipelines, that reliably ship new model versions to millions of users across cloud platforms without regressions Design interfaces and tooling abstractions across CSPs that enable cost-effective inference management, scale across providers, and reduce per-platform complexity Contribute to capacity planning, autoscaling, and workload routing strategies that match supply with demand and direct requests to the most cost-effective accelerator and region Analyze observability data across providers to identify performance bottlenecks, cost anomalies, and regressions, and drive remediation based on real-world production workloads You May Be a Good Fit If You: Have significant software engineering experience, with a strong background in high-performance, large-scale distributed systems serving millions of users Have experience building or operating services on at least one major cloud platform (AWS, GCP, or Azure), with exposure to Kubernetes, Infrastructure as Code, or container orchestration Are curious about LLM serving; prior inference or ML experience is not required Thrive in cross-functional collaboration with both internal teams and external partners Are a fast learner who can quickly ramp up on new technologies, hardware platforms, and provider ecosystems Are highly autonomous and take ownership of problems end-to-end, including work that falls outside your job description Strong Candidates May Also Have Experience With Direct experience working with CSPs to scale infrastructure or products across multi