Member of Technical Staff, Inference & Energy Efficiency
Energy is the #1 constraint on new AI datacenters. Neuralwatt makes every kilowatt produce more AI. We build software that tunes GPU power at the server level (Neuralwatt Optimize), serves energy-efficient inference (Neuralwatt Cloud), and lets datacenters put flexible and stranded power to work on more GPUs (Neuralwatt Deploy). We're a VC-backed, seed-stage startup working with neoclouds, datacenter operators, and a major utility.
We're hiring a founding Member of Technical Staff to make Neuralwatt Cloud serve more tokens per kilowatt than anyone else. You'll work across the stack, from serving infrastructure to the models that drive our optimizations.
What you'll do:
Technically:
- Build and run our inference serving stack across a mixed NVIDIA and AMD fleet
- Design energy-aware scheduling, routing, and autoscaling that treat power as a first-class constraint next to latency and cost
- Tune vLLM and SGLang (parallelism, batching, KV cache, prefill/decode disaggregation), measured in tokens per joule, not just tokens per second
- Train and improve the energy-efficiency models behind our AI-led optimizations
- Launch new open models fast, and build reproducible benchmarks for latency, throughput, and energy
- Write Python and Rust
Non-technically:
- Work directly with customers: developers on Cloud and operators running Deploy
- Publish benchmarks, write, and present at conferences on inference efficiency
- Shape the product roadmap
Requirements:
- 5-10+ years of software development experience, with 2+ years on LLM serving or ML infrastructure
- Hands-on with vLLM, SGLang, TensorRT-LLM, or similar
- Deep understanding of inference internals: prefill vs. decode, KV cache behavior, batching, parallelism strategies
- Comfortable debugging across the stack: CUDA or ROCm, drivers, containers, networking
- Clear communication and strong leadership
- Thrive in ambiguous, outcome-driven environments
Nice to have:
- Disaggregated serving with NVIDIA Dynamo or similar
- GPU power and thermal behavior under real inference load
- Kubernetes at GPU scale; multi-region serving
- Open-source contributions to serving or inference projects
What success looks like in year one:
- A published, reproducible tokens-per-kilowatt lead on the models customers use most
- Cloud scaling to many more customers with clear latency and uptime targets met
- Energy-aware scheduling running in production across heterogeneous GPUs
Location: Candidates in the Seattle or Denver metro areas are strongly preferred. We work together in person regularly. Exceptional candidates elsewhere in US time zones will be considered.
Salary: $220k-$300k depending upon experience, plus meaningful founding equity
Benefits: 401(k), health, dental, & vision