Filip Makraduli

Hi, I'm Filip. ML engineer and DevRel, Imperial grad, based in London. Short version of what's here: I've spent the last few years building, benchmarking, and explaining exactly the layer you sell (inference, retrieval, and more and more the agent-execution loop), all of it in public.

One thing worth knowing before you dig in: almost everything below was done for a brand nobody had heard of when I started. At Superlinked I had to be resourceful from day one, talking my way into events, co-organizing hackathons to earn a seat at the table, building partnerships one maintainer and one meetup at a time. I'm proud of that, and it taught me how to make something from nothing. But it does mean all of this was done on hard mode. Put a household name behind the same energy and it stops starting from zero. It compounds. I've shown the motion works without leverage; I'd love to run it with leverage.

1. Research and technical depth

If you want to see me dive deep into the math, start with my Haystack EU 2025 talk: watch here. The arc: I open with Dante and the analogy of Virgil as a guide, go into the deep matrix math of embeddings, and end by pushing back on Google DeepMind's conclusions about single-vector retrieval.

If you prefer reading: "What actually makes embedding inference fast?" on my Substack. A dissection of transformer inference internals for senior AI engineers: attention mechanisms, flash attention, where the milliseconds go.

2. FlashNorm: from paper to kernels to ecosystem adoption

This is the one I'd point at if you want the full pipeline in one place: paper → benchmarks → published checkpoints → PRs into the engines you serve. Basically the forward-deployed loop run against open source: find the bottleneck, ship the fix, measure it end-to-end, publish, upstream.

• The paper: co-author on FlashNorm (OpenMachine's transformer-tricks). The idea: fold RMSNorm's weights into the next linear layer and defer normalization so it overlaps with the GEMM. 15+ merged PRs on record, the strongest being the GPU benchmarks, the "flashify-your-own-model" recipe, norm cancellation with eligibility audit, and strict Gemma-4 modeling. Conference submission pending.
• The headline number I produced: +12.77% end-to-end speedup in HF Transformers on Llama-3.2-1B (A100, bf16). That's the full generation loop, not a microbenchmark.
• Custom kernel development, in both serious stacks: I wrote the custom Triton kernel and the CUTLASS fused kernel, including a CUTLASS 3.x SM90 version validated on H100 against a FlashInfer baseline. That's the same Hopper-generation kernel API family your famous kernel reverse-engineering post dissects. I've written against it, not just read about it.
• The notebooks each answer one question: gpu_benchmark asks is the speedup real on a GPU? (+12–14% at 7B scale via CUDA-streams overlap); decode_profile asks how much decode time does RMSNorm actually eat?; hf_a100 is the +12.77% run; quality_evals asks does the model stay just as good?; flashify_and_publish is the recipe that produced the published checkpoints.
• Published "flashified" models on Hugging Face: Llama-3.2-1B, Llama-3.1-8B, Qwen3-1.7B, Qwen3-8B, SmolLM2-135M, gemma-4-E2B. Community GGUF quants of these have hundreds of downloads.
• Upstream work and the ecosystem campaign: my PRs in Transformers #45742 and vLLM #41431, plus my issue vLLM #41430 diagnosing why the fused kernel couldn't realize the speedup, which a community contributor fixed with new CUDA kernels in #44109. Alongside that, Nils Graef (lead author) and I ran a coordinated push for weightless-RMSNorm support across the ecosystem: feature requests in Modular, FlashInfer, Ollama, LM Studio's MLX engine, Unsloth, SGLang, TensorRT-LLM, nano-vllm, and DeepMind's Gemma repo (the most interesting open thread: if DeepMind takes it, a trick from our paper ends up in how a Google model family is modeled, and my strict Gemma-4 implementation is the working proof it's safe). Third parties are already shipping it: FlashInfer #3280, vLLM #44109, Tenstorrent tt-metal #41834, nano-vllm #250, an Unsloth fork. This is developer relations run against the hardest audience there is, kernel maintainers, who only move for measured numbers and runnable code. Nils invented the technique and led the push; my lanes were the benchmarks, the checkpoints, the Gemma-4 implementation, and the vLLM/Transformers work. Evidence first, adoption follows: the same motion your content runs on.

3. Conference talks

AI Engineer Europe 2026: "The Small Model Infrastructure Nobody Built (So We Did)"

The main one, and the talk I'm happiest about. I did this end to end from scratch: the writing, the directing, reading the audience, the stage presence, all of it tuned to earn views, traction, and brand. Landing the likes, comments, and view count it did made me genuinely proud. Watch it here. (Two more AI Engineer talks, from San Francisco, are pending release.)

Haystack EU: a longer lecture in three parts (the math one)

Almost a lecture. Three parts, and it's where I go deepest on theory:

• Part 1: the math, and an analysis of DeepMind's paper. I open with Dante and the analogy of Virgil (from 2:07), then go into the deep dive (from 8:28 to ~23:00): sign-rank theory, the satellite analogy, NP-hard problems, and a walk through DeepMind's paper on the theoretical limits of retrieval.
• Part 2 (24:00–30:00): what a mixture of encoders is (the previous-generation Superlinked embedding-generation framework).
• Part 3: beyond text-only benchmarks (from 30:13): evaluating queries in ways the standard benchmarks miss.

Inference talks (EuroPython, Berlin, Kraków, and more)

• Intro to inference, prefill and decode, for engineers who haven't dug into inference but want to: from 1:49 to 8:45. Prompt processing is compute-bound, token generation is memory-bandwidth-bound, and that changes which hardware and engine you pick. The same distinction your GPU glossary teaches, from a conference stage.
• Superlinked's PID-based batching, shared queue vs non-shared queue, then a one-off competitor-comparison slide and a showcase of integrations and early partnerships: from 18:37.

A Qdrant + LlamaIndex + LangChain partnership talk (Berlin), under tight constraints

An honest example of a talk where I didn't get to pick the topic or the framing. I had to explain specific code and integrations with LlamaIndex and LangChain. In an ideal world I wouldn't structure a stage talk this way, but it's still a solid dive into real code for a more technical, retrieval-focused audience, and salvaging the positives out of a constrained brief is itself a skill. From 3:28.

Coming next (end of October, San Francisco)

A talk on model routing, inspired and developed from a hackathon user's entry, for ODSC West. It's built on darwin-router (see the next section).

A note on how I speak: I adapt the content to the event and the crowd, live. I'll happily say "that makes sense" or "that doesn't make sense" on stage and adjust. The talk serves the room, not the slides.

4. The router: my favorite recent build, and the most agent-relevant

darwin-router, 🏆 first prize at the Superlinked × Qwen Hackathon (Aug 2026). A training-free, self-improving semantic model router: routing is a kNN lookup over labelled exemplars, and misroutes caught by an LLM judge are written back into the index, so it improves online with no training loop. Built in one day, measured on an immutable 40-query holdout: accuracy 75% → 85% in eight hours while cost per request halved, with total spend ~65% below sending everything to the big model. 2-minute demo video in the repo.

This started as a hackathon entry and it's now the basis of an upcoming talk I'm giving; the entry sparked the whole line of thinking. For you it's the "which model does this agent call?" problem, the connective tissue between sandboxes (where agents act) and endpoints (what they call), solved the way your platform would solve it: no training runs, adapt at the speed of an index write.

5. The MCP model-routing demo

At Superlinked I built and wrote up serving private document tools to any LLM via the SIE MCP server (Superlinked blog). Combine that with darwin-router and you get the demo I'd love to show live: an MCP server that routes each request to the right model, learns from its misroutes, and executes anything agentic in a sandbox. On your platform that's sandboxes plus endpoints plus a key-value store for the exemplar index: one platform, one file. Happy to build it as a take-home.

6. What we did internally: the benchmarking record

All public in superlinked/sie (~3.2k stars, self-hosted inference for agents; I led its GTM and worked on it as engineer and de-facto forward-deployed engineer).

The top 3 things we tested on your platform. It was our qualification and benchmarking rig, with your public LLM-engine benchmark as the reference for what normal performance looks like:

• Whether a new engine version actually works on our GPUs and models. We tested SGLang 0.5.10 with CUDA 12.9 and Qwen3.6-27B on a rented H100. The server started and loaded the model fine, but one of SGLang's GPU kernels failed to compile the first time a real request came in, because the newer CUDA compiler is stricter about a bug in SGLang's code. We caught it in an afternoon of H100 time instead of in production, stayed on the older version, and wrote the finding into our bundle file.
• Whether FlashAttention-4 works in our stack. Our CUDA-13 lane pins the FA4 beta (flash-attn-4==4.0.0b11); before trusting it we checked it builds and runs alongside TensorRT-LLM and our pinned dependencies. The same kernel your team reverse-engineered, tested from the user's side.
• Which GPU for which model. Throughput and latency checks across model classes: detection plus OCR runs comfortably on a cheap L4 (the 5.27s OCR timing in our retail-audit evidence), and TEI vs Infinity vs Fastembed comparisons decided our adapter roadmap. Your benchmark told us what normal looks like so we didn't re-derive it.

This testing is where my Europe product feedback (§7) comes from. Testing from London, you feel the US-east control-plane latency directly; setting up EU region pinning, you hit the surcharges and discover large payloads still pass through US storage. I evaluated the platform the way a European customer would.

The rest of the record:

• Retrieval-ablation study: six controlled conditions (BM25 → dense → hybrid → cross-encoder rerank → ColBERT rerank → multi-vector direct) on real financial filings. 2,942 pages, 1,854 queries, 8,766 graded judgments, non-goals stated up front. Your benchmark's discipline, applied to retrieval quality.
• The tensorrt-llm bundle shows what the "engineering lift" warning looks like in a real repo: every dependency pinned to a version we proved works, with comments explaining why, because with TensorRT-LLM anything less breaks.
• Your platform in our production: the retail-shelf-audit case ships hash-verified evidence from direct GPU runs (Grounding DINO plus LightOnOCR on an L4, pinned checkpoints, and a command that rechecks every file hash).
• Partnerships and ecosystem distribution, as a repeatable motion: a custom retriever in LangChain (PyPI package; merged PR) and a LlamaIndex retriever on LlamaHub (merged PR); here's a video where I explain how the retrievers work. Plus merged integration pages in Chroma, Qdrant, Weaviate, LanceDB, and Haystack, with CrewAI in review. The same motion produced co-marketing on partners' own blogs: "Beyond Multimodal Vectors: Hotel Search With Superlinked and Qdrant". Honest note: the retrievers were for the previous-generation framework, since deprecated in favour of SIE. The point is seven partner teams merging my work (an eighth in review), not the specific product.
• I maintain the current docs and examples: README passes, quickstart fixes, and retitling are my merged PRs, and I authored document-ocr (swap OCR models with one identifier change, hosted on HF Spaces) and stripe-link-fraud (a fraud-risk gate inside a Stripe payment round-trip) end-to-end.
• Process-level thinking, because we had few employees and lots of tokens: I built repo automations and reusable AI skills that mine our codebase and community for content ideas, draft from them, and (the important part) edit their own instructions as we learn what works. Content production as a pipeline that improves, not a heroic effort that repeats.
• The two gallery pieces I'd walk you through: contract-review-agent, an OpenAI Agents SDK agent where every model call (triage, vision, OCR, embeddings, rerank, extraction, text-to-SQL, reasoning, and a safety guardrail) runs on one self-hosted cluster with per-model observability; the "agent on fully-owned inference" story end to end. And agent-action-monitor, a behavioural gate that catches hijacked agents by their actions, not their credentials, checking each proposed action against the agent's learned baseline before it reaches a downstream system. A working prototype of the agent-security layer your sandbox and RBAC roadmap points at.

7. Product feedback for Europe

From selling inference into European teams:

• Residency is the #1 enterprise objection: even with EU region pinning, large payloads transit US-east storage, and there's no EU entity or ISO 27001 story. The EU-native clouds lead with this. Fixing it unlocks DACH and regulated UK/FR.
• The input-plane latency floor lands harder in Europe: ~150ms p95 with the control path in US-east is a transatlantic tax for EU callers. Regional input planes should be the loud story.
• The EU AI Act is a product opportunity: eval-gated model promotion on endpoints doubles as the audit trail GPAI-era customers must produce. Nobody sells that framing yet.
• Multi-LoRA serving on managed endpoints: per-company adapted models are the wave, and I've built the LoRA lifecycle seam in SIE (hot-reload vs blocking-load is a decision I've already had to make). Per-tenant adapters on shared fleets is the missing piece.

8. Strategy

I wrote a full EMEA strategy for you: the "trinity" thesis (sandboxes plus endpoints plus RL and fine-tuning as one self-upselling loop, with the sandbox as the verification layer that keeps agent improvement honest), a London/EMEA community ground game, forward-deployed demo blueprints (an agent that learns your customer's repo via sandbox-forked GRPO rollouts, with eval-gated LoRA promotion), and a 90-day plan. Happy to walk through it; it's the job talk, essentially.

9. Partnerships I already have warm, and the ones I'd build

Communities: {Tech: Europe} (hackathon organizer), DeepMind and Qwen/Alibaba (co-organized hackathons; darwin-router came out of the Qwen one), AI Tinkerers, EuroPython, PyData London, ODSC, Haystack, Berlin Buzzwords. The same venues a London office will want a presence at, and I arrive with organizer relationships, not cold emails.

Universities, at the level of specific labs: I've spoken at Imperial (my alma mater; targets: I-X, the RL groups, ICAIS for distribution) and can open a lecture-plus-workshop circuit at Edinburgh (Autonomous Agents group, the RL reading group), Oxford (the Foerster lab lineage, OxAI), Cambridge (CBL, CUAI), and UCL's DARK lab orbit. Speak, seed the RL groups with credits and starter repos (sandbox-forked rollouts are their wall-clock bottleneck), and tie credits to ICLR/NeurIPS/EurIPS deadlines so your compute shows up in acknowledgments.

Quant finance, the untold London segment: your press coverage already lists unnamed hedge funds as customers. London is Europe's densest quant hub (Squarepoint Capital, XTX, G-Research, Man Group, Qube RT) with a perfect workload shape: bursty research compute, strict isolation, LLM research agents needing sandboxes. They hire from the labs above and buy through in-person technical credibility, which is the demo-led motion I've closed European clients with. A named segment, not ambient enterprise.

Conferences: EWRL (Lille) → EurIPS (Paris, Dec) → ICLR/NeurIPS. Credits tied to paper deadlines and "reproduce your RL paper" desks are the cheapest researcher-mindshare play available. I'd own it end to end.