Kog Aims to Squeeze 30x Faster LLM Inference from Existing GPUs
Kog, a French startup established by Gaël Delalleau, is focused on enhancing traditional GPUs for AI inference, thereby questioning the necessity for specialized chips. In May, the company presented a technology preview on Hacker News, achieving a remarkable 3,000 tokens per second (TPS) with the Laneformer 2B model utilizing AMD MI300X and NVIDIA H200 GPUs. Kog claims to deliver 30 times faster LLM inference, catering to software engineering clients who are dissatisfied with delays in tools such as Claude Code. The startup has shifted its focus to larger models after discovering that clients were hesitant to fine-tune smaller ones. Supported by Scaleway, Bpifrance, and French Tech 2030, Kog intends to unveil a significant model at 10x speed by September to secure Series A funding. Delalleau possesses a background in solid-state physics and cybersecurity.
Key facts
- Kog is a French startup focused on software optimization for faster AI inference on conventional GPUs.
- Founded by Gaël Delalleau, who studied solid-state physics at École Polytechnique and worked in offensive cybersecurity.
- In May, Kog's tech preview on Hacker News demonstrated fast decoding on AMD MI300X and NVIDIA H200 GPUs.
- The demo achieved 3,000 tokens per second (TPS) using Laneformer 2B, a small open-sourced model.
- Kog promises 30x faster LLM inference, but has not yet proven it on large models.
- The company received 200 tangible business leads after the preview.
- Kog is focusing on software engineering as the first use case, targeting users of tools like Claude Code.
- Delalleau expects to demonstrate a major model at 10x speed by September to raise Series A funding.
- Kog is backed by Scaleway, Bpifrance, and French Tech 2030.
- The team of 11 dedicates weeks or months to each new GPU, limiting scalability.
- Kog plans to use agent-based pipelines to support more chips and models in the future.
Entities
Artists
- Gaël Delalleau
Institutions
- Kog
- Cerebras
- AMD
- NVIDIA
- Anthropic
- ZML
- Stanford University
- Hazy Research
- École Polytechnique
- DEF CON
- Varsity VC
- Scaleway
- Bpifrance
- French Tech 2030
- TechCrunch
Locations
- France