Software Engineer III, AI System Engineer

Vision Connection:

Position Overview:

We're looking for an AI systems engineer to own the performance, cost, and architecture of our multi-agent clinical documentation platform. Our system runs 15+ specialized AI agents on AWS Bedrock AgentCore (built with Strands Agents SDK), orchestrating them in parallel to turn clinical transcripts into structured medical notes. You'll optimize how these agents run, build the evaluation frameworks that measure quality, architect RAG systems for clinical decision support, and manage the AWS infrastructure that hosts both managed and open-source models. This is not a traditional ML role — you won't be training models from scratch. You'll be engineering the systems around LLMs: making them faster, cheaper, more accurate, and more capable at scale.

##

Responsibilities:

- Own cost and performance optimization of the multi-agent system — prompt caching strategies, model routing (right-sizing models to task complexity), token reduction, parallelization, and latency profiling
- Extend and maintain the agent orchestrator and individual section agents (Python, Strands SDK, Bedrock)
- Build and improve evaluation frameworks (accuracy benchmarks, agent quality scoring, regression detection) to measure output quality across agent changes
- Architect and implement RAG infrastructure for clinical decision support — vector search over clinical knowledge bases, retrieval pipelines, and integration with the agent system
- Manage AI/ML infrastructure on AWS: SageMaker endpoints (Whisper transcription), Bedrock provisioned throughput, Agent Core runtime configuration, model hosting
- Set up infrastructure to run open-source models (clinical NER, specialized summarization, cheaper inference tiers) on SageMaker or ECS Work with Serverless Framework and Terraform to deploy and configure inference infrastructure (Lambdas, VPC networking, secrets, MSK integration)
- Collaborate with the agent team on prompt engineering, structured output schemas, and MCP tool integrations
- Monitor production costs and performance, identify optimization opportunities, and implement them.

Education:

- Bachelor’s degree in computer science or equivalent combination of education and experience.

Experience:

- 4+ years of software engineering experience with significant AI/LLM systems work Strong Python skills — you've built production systems, not just notebooks
- Hands-on experience with LLM APIs and multi-agent architectures (prompt design, structured output, tool use, orchestration patterns)
- Experience with AWS AI services (Bedrock, SageMaker, or equivalent managed inference platforms)
- Understanding of cost dynamics for LLM workloads — token economics, caching, model selection tradeoffs
- Comfort with infrastructure-as-code (Terraform, Serverless Framework, CloudFormation) for deploying and managing ML/AI infrastructure
- Experience building evaluation pipelines for AI systems (benchmarking, quality metrics, automated regression testing)
- Ability to profile and optimize latency in distributed async system

Preferred:

- Experience with RAG architectures (vector databases, embedding models, retrieval optimization)
- Familiarity with Strands Agents SDK or similar agent frameworks (LangChain, CrewAI, AutoGen)
- Experience hosting open-source models on SageMaker (Whisper, Llama, Mistral, domain specific models)
- Healthcare domain knowledge (clinical terminology, medical documentation standards) Experience with prompt caching and cross-request cache sharing strategies
- Background in real-time audio/speech-to-text pipelines
- Familiarity with MCP (Model Context Protocol) for tool integration

Team Member Competencies:

Demonstrate and support the organization's core values.

- Give honest and constructive feedback to other team members.
- Maintains a positive attitude. Tackle challenges as they come, and don’t let setbacks get you down.
- Provide assistance, information, or other support to others to build or maintain relationships.
- Take responsibility for your mistakes and look for solutions. Understand how your actions impact the team.
- Understands role on the team and works to achieve goals to the best of your ability.
- When recognizing a problem, take action to solve it.
- Working within a team means there will be varying opinions and ideas. Active listening and thoughtfully responding to what your team member says.  Every team member exhibits our core values: Team First Lift others up Share openly Set and Crush goals Delight the Client   Security Access (High):    Based on an annual analysis, the expected level of exposure to Protected Health Information (PHI) for anyone performing the duties required by this position is: HIGH. This level of classification indicates that workforce members performing duties required in this position are expected to have elevated exposure to software, databases, network storage, and physical locations that contain PHI.Based on an annual analysis, the level of access to Protected Health Information (PHI) for anyone performing the duties required by this position is: HIGH. This indicates that potential access to PHI is high. The level of access can never be lower than the level of exposure.Employee access to locations, programs, systems or data is based on least privilege principle and assigned at a level deemed appropriate by the manager or appropriate company personnel per the Security Access Procedures document.

Equal Opportunity Employer
This employer is required to notify all applicants of their rights pursuant to federal employment laws. For further information, please review the Know Your Rights notice from the Department of Labor.