Use a publisher GLM 5.2 checkpoint on a capable multi-GPU Linux host, connect VS Code/Cline privately from Windows or Mac, and verify a small code change.
Publisher hardware requirements for the complete GLM 5.2 checkpoint, not per-token active parameters. FP8 is the preferred eight-GPU route.
| Machine | GLM 5.2 · publisher FP8 | GLM 5.2 · publisher BF16 |
|---|---|---|
| Rented NVIDIA · 8× H200 141 GB | Publisher cookbook supports FP8 on one node; start at 32K context. | BF16 cannot fit on this single node; choose eight B300 GPUs instead. |
| Rented AMD · 8× MI300X 192 GB | Publisher cookbook supports FP8 with compatible ROCm and DSA tilelang. | BF16 cannot fit weights plus cache on one MI300X node; choose FP8. |
| Rented NVIDIA · 8× B300 288 GB | FP8 fits with ample capacity; compare rental price. | Publisher cookbook verifies BF16 on one node; start at 32K context. |
| Rented AMD · 8× MI325X 256 GB | Publisher cookbook supports FP8 with compatible ROCm. | Publisher cookbook says BF16 fits on MI325X with a compatible ROCm build. |
Provider stock and rates change. The SGLang cookbook also documents AMD MI355X and other precision options; check the current source before renting.
Pick the source-documented FP8 eight-GPU route or the much larger BF16 B300 route, keep the SGLang API private, connect local VS Code/Cline, verify a small code change and stop billing.
Select the weights you want. The instructions, downloads and settings change with your choice.
Save your machine in My Hardware to get an automatic starting choice. You can always choose any package yourself.
Start at 32K context, well below the publisher 1M limit. Recommended published precision. Start at 32K context; eight H200 or MI300X GPUs are source-documented, subject to provider availability. Increasing context increases KV-cache memory. This is a multi-GPU package and cannot run on one consumer card.
Choose your computer first, then its GPU path. The instructions below update to match your selection.
Rent an eight-GPU H200 node only if the current provider offers it. The model executes on Linux; Windows/Mac clients use SSH and Cline.
Choose a Runpod eight-H200 node with at least 1 TB free disk, enough host RAM for downloads, direct SSH and a current CUDA/PyTorch image. Check the full eight-GPU hourly price before clicking deploy.
Runpod · affiliate SGLang verified hardware matrix Runpod pricingIn the rented Linux Pod Terminal, install SGLang in the compatible image and start the exact GLM 5.2 · publisher FP8 checkpoint below. The first download is About 756 GB FP8 weights and can take substantial time. Keep the API on Pod loopback. This command starts at 32K context and follows the cookbook’s eight-GPU parallelism, with AMD-specific DSA flags where applicable.
Official SGLang installation Exact publisher weights Hardware and runtime cookbookpython -m pip install -U uv
uv pip install --system --prerelease=allow sglang
python -m sglang.launch_server --model-path zai-org/GLM-5.2-FP8 --served-model-name zai-org/GLM-5.2-FP8 --tp 8 --context-length 32768 --mem-fraction-static 0.8 --host 127.0.0.1 --port 30000In Runpod Connect copy the direct SSH command. On your own Windows use PowerShell; on Mac use Terminal. Append -N -L 127.0.0.1:30000:127.0.0.1:30000 to the SSH command and keep that terminal open. Visit http://127.0.0.1:30000/v1/models locally. Do not expose the unauthenticated SGLang API publicly.
Runpod direct SSH Windows OpenSSH clientInstall VS Code and the Cline extension on your Windows or Mac computer. Open a small local repository and review what Cline may read and edit; leave automatic command approval off.
VS Code download Cline extensionIn Cline select the OpenAI-compatible provider, Base URL http://127.0.0.1:30000/v1, exact model ID zai-org/GLM-5.2-FP8, and 32K context to match the server. This is much smaller than the publisher’s 1M maximum; increase only after measuring KV-cache headroom. A local placeholder API key is needed only if Cline requires it.
Ask Cline to add an add(a,b) function and a test for add(2,3)==5 in a tiny local Python repository. Require it to show the diff and run the test. Inspect both the code and test result in VS Code before accepting the edit.
Copy any useful result to your own computer. Then stop or terminate the Pod in Runpod; an idle eight-GPU Pod is still billed even when your SSH tunnel or browser is closed.
Runpod Pod managementCline connects through your SSH tunnel, proposes a local add(a,b) function and test, and you inspect the diff and passing add(2,3)==5 test in VS Code. Then stop the eight-GPU rental.
This is a large multi-GPU deployment, not a single 5090 or DGX Spark setup. The supported hardware claims come from SGLang’s GLM 5.2 cookbook. YouRunAI has not physically rented and benchmarked these nodes.
Check current Runpod capacity and the full rental price before deploying; choose the FP8 package or a smaller local coding model. Do not start a partial cluster that cannot hold these weights.
Verify the exact checkpoint and all eight GPUs. Keep the initial 32K context and check the cookbook’s matching NVIDIA or AMD memory/ROCm settings; BF16 cannot fit a single eight-H200 node.
Keep SGLang bound to Pod loopback, reopen the private SSH tunnel to local port 30000, and confirm /v1/models responds on your desktop before editing Cline’s model ID.
Inspect and save the local diff, then stop or terminate the Runpod Pod. Closing the browser or Cline does not stop compute billing.
Original instructions, model files, and compatibility notes behind this setup.
Source-backed package and platform-route plan for private local chat with Qwen3.5 · 9B. Preserves the official Ollama 9B package, a smaller 4B family fallback, and a clearly labeled community uncensored GGUF option.
Run Qwen3.8 27B with Ollama, connect it to Cline in VS Code, and make one reviewable code change in a tiny local project.
Generate images from text prompts in ComfyUI with the fast distilled or more adjustable base 4B model. Save your results and reuse the visual workflow.