RL environments, agent benchmarks and the verifiers that score them, plus the agents and RAG systems themselves. My favourite part is finding out when a score is lying.
I'm an applied AI engineer who's a little obsessed with whether AI systems actually work. At Turing I build agent benchmarks in Harbor format (terminal, multi-agent and enterprise-tool tasks with deterministic verifiers), and I led an evaluation pod on Terminal Bench, OS Bench and SWE-Bench.
Outside of that I build my own things: an RL agent for a Kaggle simulation competition (provisional silver), hand-written ARM NEON quantization kernels, an LLM judge calibrated against a human grader, and a multimodal RAG app. Proof beats vibes.
An RL agent for a farming simulation competition: provisional silver, around the top 1% of 10,246 teams. I built a C++ replica of the game engine (move-for-move parity, ~1.5M games an hour) and trained a late-game policy with PPO that added about $400 a game on 2,400 fresh games. Biggest lesson: a behaviour-cloned copy of the #1 player was 81% accurate offline and still lost live.
From-scratch 4-bit (Q4_0) quantization kernels in C++ with ARM NEON SIMD and a GGUF loader, benchmarked on real SmolLM2 weights: 3.4x faster dequantization than llama.cpp's scalar reference and a 4.8x faster SDOT dot product, with bit-identical output.
A multi-pass LLM judge for plans and PRDs (extractor, five per-criterion scorers, a deterministic citation verifier, synthesizer). On a 6-plan calibration set: 9887/10000, perfect rank correlation with the human grader, 96.5% citation precision and 99.6% test-retest stability. About 30 CI tests keep it honest.
Multimodal RAG over PDF, image, audio and video. A LangGraph agent (plan, retrieve, synthesize, reflect) over two ChromaDB stores with Gemini and CLIP embeddings, plus MLflow tracking, Airflow jobs and CI. Live on Hugging Face Spaces.
A portable harness for running Harbor multi-agent benchmark tasks (oracle, single-agent and multi-agent runs) on free GitHub Codespaces with Docker-in-Docker. No local Docker, no cloud bill.
A reliability scorecard for MCP servers: schema conformance, tool reliability and hygiene, with LLM-judged agent tasks on the roadmap. First reference run scored 5750/7000 with 13/13 tools succeeding.
Rhesis (an AI testing platform): stopped empty test sets from being scheduled for execution, with tests (PR #2173). OGX / Llama Stack: fixed the Gemini provider auth docs to match the mutually exclusive api_key / access_token behaviour (PR #6341).
I'd rather find the failure mode myself at 2am than let a user find it in prod.
Spent two months building an RL farmer for Kaggle. Ended around the top 1%, still annoyed it wasn't gold.
Hand-wrote NEON quantization kernels just to feel how the bytes actually move.
I think "is it actually good?" is the most underrated question in AI.
Best ideas arrive with the second cup of chai, usually.
I'd rather tell you what a system can't do than oversell what it can.
Open to applied AI, evals and RL-environment roles, full-time or contract. I reply fastest to email, and yes, the button does something.