Rent two NVIDIA GPUs to serve Mistral Small 4 privately through vLLM, connect Cline in your local VS Code, and create and test a small local Python file.
Rented NVIDIA capacity estimates for 70 GB NVFP4 repository and 32K starting context. These are not benchmarks or guaranteed fits; check live GPU availability and peak usage.
| Machine | Mistral Small 4 · publisher quantized |
|---|---|
| Rented NVIDIA · 2× H100 80 GB | Possible but tight; keep context at 32K and watch memory during startup. |
| Rented NVIDIA · 2× H200 141 GB | Recommended starting rental with more room for model cache and context. |
| Rented NVIDIA · 2× B200 180 GB | More capacity; compare the live hourly price before choosing it. |
The Windows or Mac client only runs VS Code, Cline and SSH; the model runs on rented Linux GPUs.
Rent two GPUs for Mistral Small 4, serve its exact quantized checkpoint through vLLM, connect local Cline through a private tunnel, create and test a small local Python file, inspect the result and stop billing.
This setup uses the package documented for this task. Review its source and supported platforms before starting.
Save your machine in My Hardware to get an automatic starting choice. You can always choose any package yourself.
Start at 32K context on two H200 141 GB GPUs. The quantized weights, runtime and KV cache consume more than the repository size; increasing context increases memory use. Two H100 80 GB GPUs are a tighter estimate.
Choose where you want to run it. The instructions below update to match your selection.
The GPUs run Linux on Runpod. Your own Windows PowerShell or Mac Terminal opens a private SSH tunnel to the localhost API; VS Code and Cline remain on your own computer.
In Runpod choose an on-demand Pod with two NVIDIA H200 141 GB GPUs, at least 128 GB system RAM and enough persistent disk for the model cache. Two H100 80 GB GPUs are a tighter estimated alternative. Check the live hourly rate, storage cost and available stock before deploying; use a current PyTorch/CUDA template and direct SSH.
Runpod · affiliate Runpod pricingIn the rented Linux Pod Web Terminal, install current vLLM in the template's Python environment, then run this publisher-derived command. It serves mistralai/Mistral-Small-4-119B-2603-NVFP4 on the Pod loopback at 127.0.0.1:8000; do not expose an unauthenticated model API publicly. First download may take time. Start at 32K context even if the publisher supports longer contexts.
Official vLLM GPU installation Publisher checkpointpython -m pip install -U 'vllm>=0.15.0'
vllm serve mistralai/Mistral-Small-4-119B-2603-NVFP4 --host 127.0.0.1 --port 8000 --tensor-parallel-size 2 --max-model-len 32768 --attention-backend TRITON_MLA --tool-call-parser mistral --enable-auto-tool-choice --reasoning-parser mistral --gpu-memory-utilization 0.8From your own Windows PowerShell or Mac Terminal, copy the direct SSH command from Runpod Connect, then add -N -L 127.0.0.1:8000:127.0.0.1:8000. Keep that terminal open. In another local terminal open http://127.0.0.1:8000/v1/models to confirm the server responds; your code stays in the local VS Code workspace.
Runpod direct SSH instructions Windows OpenSSH setupDownload VS Code and install the Cline extension on your Windows or Mac computer. Open only the local test repository you intend Cline to read, review its permissions and keep automatic command approval off for the first task.
VS Code download Cline extensionIn Cline choose the OpenAI-compatible provider, Base URL http://127.0.0.1:8000/v1, model ID mistralai/Mistral-Small-4-119B-2603-NVFP4, and a 32K context limit to match the server. The SSH tunnel must remain open. Use a placeholder local API key only if the client requires a value; the unauthenticated vLLM endpoint remains private through the tunnel.
Ask Cline to create a tiny calculator.py with add(a,b), add one test, and run the test before accepting its diff.
The new calculator.py and test file appear in your local VS Code folder, the test passes, and you inspect the diff before accepting any change. Save any useful output on your own computer. Then stop or terminate the Runpod Pod in the console; closing the browser, terminal or VS Code does not stop rental charges.
Manage Runpod PodsThe new calculator.py and test file appear in your local VS Code folder, the test passes, and you inspect the diff before accepting any change. Stop or terminate the Pod after saving the result.
Source-backed commands and capacity estimates. No YouRunAI benchmark or physical rental test is claimed. Only Runpod is linked as a GPU referral.
Confirm both GPUs are visible, the selected quantized checkpoint is correct and the Pod has enough disk and RAM. Keep the 32K starting context and check the publisher runtime version before changing flags.
Keep vLLM bound to 127.0.0.1 on the Pod, keep the private SSH tunnel open, and check http://127.0.0.1:8000/v1/models on your own computer. Confirm the selected model ID exactly matches the vLLM server.
Save or copy the output and review your local files, then stop or terminate the Runpod Pod. Closing VS Code or the SSH tunnel does not stop GPU billing.
Original instructions, model files, and compatibility notes behind this setup.
Source-backed package and platform-route plan for private local chat with Qwen3.5 · 9B. Preserves the official Ollama 9B package, a smaller 4B family fallback, and a clearly labeled community uncensored GGUF option.
Run Qwen3.8 27B with Ollama, connect it to Cline in VS Code, and make one reviewable code change in a tiny local project.
Generate images from text prompts in ComfyUI with the fast distilled or more adjustable base 4B model. Save your results and reuse the visual workflow.