Compute & infrastructure
The hardware and platforms that train and serve models: AI chips, local inference engines, GPU clouds, and sandboxes for running agent-generated code.
AI chips and hardware
Processors purpose-built for training and inference, from GPUs to wafer-scale and photonic chips.
- Arago - Deeptech building a photonic AI chip that runs matrix math with light for order-of-magnitude lower energy use.
- Cerebras - Builder of wafer-scale chips that hold an entire model on a single processor for ultra-fast training and inference.
- Groq - Designer of the LPU inference chip and cloud known for sub-hundred-millisecond token generation.
- NVIDIA - Dominant maker of AI GPUs and the CUDA software stack that most frontier training and inference runs on.
- Sesterce - AI-factory company operating large, sustainable GPU clusters for training and inference.
- VSORA - Startup building an inference chip to challenge GPU makers on performance per watt.
Local inference
Engines and apps for running models on your own CPUs and GPUs.
- llama.cpp - LLM inference engine in pure C/C++ that runs GGUF-format models across CPUs and GPUs on a wide range of hardware.
- LM Studio - Desktop app to browse, download, and run local models with a built-in chat interface and local server.
- MLX - Apple's array and machine-learning framework optimized for Apple silicon, for on-device training and inference.
- Ollama - Tool that packages and runs LLMs locally via a simple command line and local API.
- vLLM - High-throughput inference and serving library using PagedAttention and continuous batching for production deployment.
- ZML - Zig-based inference stack that compiles models into standalone native binaries across NVIDIA, AMD, TPU, and Trainium with zero Python dependencies.
Developer infrastructure
GPU clouds, hosted inference, and fine-tuning platforms for deploying models at scale.
- Adaptive ML - Reinforcement-learning platform for post-training, evaluating, and serving enterprise-specialized open models.
- CoreWeave - GPU cloud purpose-built for AI, offering large-scale NVIDIA clusters for training and inference.
- Fireworks AI - Inference and fine-tuning platform for open-weight models with low-latency routing.
- FlexAI - Universal AI compute platform that abstracts heterogeneous hardware so developers can train, fine-tune, and serve models without managing infrastructure.
- Hugging Face - The hub for open models, datasets, and Spaces, plus the Transformers library and hosted inference that anchor the open-source AI ecosystem.
- Koyeb - Serverless platform for deploying AI inference and apps on autoscaling CPUs and GPUs worldwide.
- Lambda - GPU cloud offering on-demand and reserved NVIDIA clusters for training and inference.
- llm-d - Red Hat-led, Kubernetes-native distributed inference framework built on vLLM for serving LLMs at scale.
- Nebius - AI-focused cloud providing GPU compute, managed inference, and ML tooling.
- NVIDIA Dynamo - Open-source datacenter-scale orchestration layer that coordinates vLLM, SGLang, and TensorRT-LLM into distributed multi-node inference.
- NVIDIA NIM - Prebuilt, containerized inference microservices for deploying optimized model endpoints across cloud, data center, and workstation.
- OpenRouter - Unified API and marketplace routing requests across hundreds of models and providers.
- Replicate - Hosted registry and inference layer for open-source models.
- RunPod - GPU cloud with on-demand and serverless GPUs for training and inference.
- Together AI - Inference, fine-tuning, and training platform for open-weight models.
Sandboxes
Isolated, ephemeral environments for safely executing AI-generated code.
- Daytona - Secure, elastic infrastructure for running AI-generated code, with agent sandboxes that spin up in tens of milliseconds.
- E2B - Open-source runtime that gives AI agents isolated cloud sandboxes to execute code.
- Modal - Serverless Python platform offering on-demand sandboxes for running AI-generated code, model inference, and batch jobs.
- Vercel Sandbox - Ephemeral, isolated microVMs for running untrusted AI-generated code, from Vercel.