Senior AI Engineer | Careers at Alyra Technology
Own the retrieval, evaluation and agent orchestration behind every engagement. Depth over proximity, and the standards the whole firm builds on.
Senior AI Engineer
Engineering. 6+ years. Charlotte, NC. full time.
Own the retrieval, evaluation and agent orchestration behind every engagement. Depth over proximity, and the standards the whole firm builds on.
The mission
Where the forward deployed roles are defined by proximity to the client, this one is defined by depth. You build the harder pieces: the retrieval architecture, the evaluation harnesses, the agent orchestration that has to behave the same way on the thousandth run as it did on the first. In the work we do, an answer that cannot name its source is not a weaker answer, it is an unusable one, so the bar is systems that can show their reasoning months later. You also decide what becomes reusable rather than rebuilt.
Responsibilities
- Own retrieval end to end: ingestion, chunking, embeddings, hybrid and multi-hop search, reranking, and the evaluation that says whether any of it actually helped
- Design the agent orchestration at the centre of client systems, including tool scoping, validation and guardrails
- Treat evaluation as a gate rather than a report: regression suites that block a release when answer quality or traceability drops
- Make live systems observable and debuggable, so that months later anyone can answer why it said that
- Make the accuracy, cost and latency tradeoffs explicit, and write them down so a client can audit the choice
- Build the internal patterns and tooling that keep engagements from starting at zero
- Set the standards for model use, retrieval and agent design across the firm
- Partner with the forward deployed engineers who carry this work into client environments
What we look for
- Six or more years in software engineering, including at least two shipping and operating LLM systems in production
- Depth in retrieval: chunking, embeddings, hybrid and multi-hop search, reranking, and the ways each of them fails
- Production experience with agent orchestration and tool calling, including what happens when a tool call goes wrong
- Systematic evaluation: offline benchmarks, regression suites, grounding and hallucination measurement, and online monitoring
- Strong Python, and the judgement to say out loud when a model is the wrong tool for a problem
- Experience making non-deterministic systems observable and debuggable once they are live
- Able to explain a technical tradeoff to someone who is not an engineer and will be held accountable for the result
- Based in Charlotte or willing to move there
Nice to have
- Self-hosted model serving such as vLLM, or fine-tuning and distillation work
- Open source contributions or published work in applied AI
- Background in data platform or infrastructure engineering
- Work in an environment where an auditor could ask to see your evaluation history
In person in Charlotte, North Carolina. Hybrid or remote can be discussed for the right person.
Salary plus performance bonus, discussed openly on the first call.
Start a conversation