Compute & infrastructure
The hardware and platforms that train and serve models: AI chips, local inference engines, GPU clouds, and sandboxes for running agent-generated code.
AI chips and hardware
Processors purpose-built for training and inference, from GPUs to wafer-scale and photonic chips.
- Arago - Deeptech building a photonic AI chip that runs matrix math with light for order-of-magnitude lower energy use.
- Cerebras - Builder of wafer-scale chips that hold an entire model on a single processor for ultra-fast training and inference.
- Groq - Designer of the LPU inference chip and cloud known for sub-hundred-millisecond token generation.
- NVIDIA - Dominant maker of AI GPUs and the CUDA software stack that most frontier training and inference runs on.
- Sesterce - AI-factory company operating large, sustainable GPU clusters for training and inference.
- VSORA - Startup building an inference chip to challenge GPU makers on performance per watt.
Local inference
Engines and apps for running models on your own CPUs and GPUs.
- llama.cpp - LLM inference engine in pure C/C++ that runs GGUF-format models across CPUs and GPUs on a wide range of hardware.
- LM Studio - Desktop app to browse, download, and run local models with a built-in chat interface and local server.
- MLX - Apple's array and machine-learning framework optimized for Apple silicon, for on-device training and inference.
- Ollama - Tool that packages and runs LLMs locally via a simple command line and local API.
- vLLM - High-throughput inference and serving library using PagedAttention and continuous batching for production deployment.
- ZML - Zig-based inference stack that compiles models into standalone native binaries across NVIDIA, AMD, TPU, and Trainium with zero Python dependencies.
GPU clouds
Providers renting raw GPU capacity, on demand or reserved, for training and inference workloads.
- CoreWeave - GPU cloud purpose-built for AI, offering large-scale NVIDIA clusters for training and inference.
- Koyeb - Serverless platform for deploying AI inference and apps on autoscaling CPUs and GPUs worldwide.
- Lambda - GPU cloud offering on-demand and reserved NVIDIA clusters for training and inference.
- Nebius - AI-focused cloud providing GPU compute, managed inference, and ML tooling.
- RunPod - GPU cloud with on-demand and serverless GPUs for training and inference.
Hosted inference and fine-tuning
Managed platforms that serve, fine-tune, and post-train models behind an API, with no infrastructure to operate.
- Adaptive ML - Reinforcement-learning platform for post-training, evaluating, and serving enterprise-specialized open models.
- DeepInfra - Serverless inference cloud for open-weight models across text, embeddings, speech, image, and video, billed per token with no minimums and no idle GPU charges.
- Fireworks AI - Inference and fine-tuning platform for open-weight models with low-latency routing.
- FlexAI - Universal AI compute platform that abstracts heterogeneous hardware so developers can train, fine-tune, and serve models without managing infrastructure.
- Hugging Face - The hub for open models, datasets, and Spaces, plus the Transformers library and hosted inference that anchor the open-source AI ecosystem.
- Replicate - Hosted registry and inference layer for open-source models.
- Together AI - Inference, fine-tuning, and training platform for open-weight models.
Serving and orchestration
Self-hosted stacks that schedule, scale, and coordinate model serving across nodes.
- llm-d - Red Hat-led, Kubernetes-native distributed inference framework built on vLLM for serving LLMs at scale.
- NVIDIA Dynamo - Open-source datacenter-scale orchestration layer that coordinates vLLM, SGLang, and TensorRT-LLM into distributed multi-node inference.
- NVIDIA NIM - Prebuilt, containerized inference microservices for deploying optimized model endpoints across cloud, data center, and workstation.
Gateways and routing
Proxies sitting in front of model providers to route, meter, and govern traffic.
- Edgee - Agent gateway that sits between coding agents and LLM providers, applying budget-driven routing, gateway-side token compression, and per-developer cost attribution across Claude Code, Codex, Cursor, and Copilot.
- OpenRouter - Unified API and marketplace routing requests across hundreds of models and providers.
Sandboxes
Isolated, ephemeral environments for safely executing AI-generated code.
- Daytona - Secure, elastic infrastructure for running AI-generated code, with agent sandboxes that spin up in tens of milliseconds.
- E2B - Open-source runtime that gives AI agents isolated cloud sandboxes to execute code.
- Modal - Serverless Python platform offering on-demand sandboxes for running AI-generated code, model inference, and batch jobs.
- Vercel Sandbox - Ephemeral, isolated microVMs for running untrusted AI-generated code, from Vercel.