Loading…
Generate YAML configurations for PyTorch serving with TensorRT-LLM.
The Trtllm Serve Config Guide helps users generate a source-backed YAML configuration for the `trtllm-serve --config` command, tailored for single-node PyTorch serving. It aligns with the official TensorRT-LLM configurations and deployment documentation, ensuring that latency, balanced, and throughput objectives are preserved while excluding speculative and multi-node configurations. This guide is particularly useful for users looking to optimize their serving setups based on specific performance goals.