Use the official gpt-oss 120B Ollama package for a private reasoning assistant. Choose a starting context that fits your memory, ask a checkable question, and verify the answer.
Practical local starting points for the official 65 GB MXFP4 package. Spark and Ryzen use shared memory, not discrete VRAM; the AMD test used LM Studio.
| Machine | gpt-oss · 120B |
|---|---|
| NVIDIA H100 · 80 GB VRAM | Runs locally; publisher target. Start with 32K context and monitor remaining VRAM. |
| NVIDIA RTX PRO 6000 Blackwell · 96 GB VRAM | Should run; 65 GB weights leave room for context. Speed not measured here. |
| NVIDIA H200 · 141 GB VRAM | Should run with generous headroom; choose the compatible Ollama GPU backend. |
| DGX Spark ×1 · 128 GB unified memory | Runs well; Ollama measured about 41 generated tokens/s in its published test. |
| AMD Ryzen AI Max+ 395 · 128 GB unified memory | AMD ran 120B in LM Studio with 96 GB graphics allocation; verify this Ollama route. |
| AMD Instinct MI300X · 192 GB VRAM | Fits with ample headroom; confirm the ROCm build and model load. |
| Intel Arc GPU · 24 GB VRAM | Does not fit entirely in GPU memory; use 20B or expect heavy CPU offload. |
| Apple Silicon · 96 GB unified memory | Likely fits; 65 GB weights leave limited room for the OS and long context. |
| Apple Silicon · 128 GB unified memory | Should run; start with 32K context and check for memory pressure or swap. |
| CPU system RAM · 128 GB | Can fit in RAM, but CPU-only replies may be slow; use 20B for a lighter setup. |
Spark ×2–×4 do not pool memory automatically: this 120B setup already fits one Spark. Memory-fit estimates are not YouRunAI speed tests. The ~41 tokens/s Spark figure is Ollama’s own short-prompt measurement; AMD’s Ryzen result used LM Studio.
Start with a modest active context, then raise it after confirming the model is loaded without swapping. These are setup suggestions, not measured maximums.
| Machine | gpt-oss · 120B |
|---|---|
| NVIDIA H100 · 80 GB VRAM | Start at 32K; 128K model limit is not a promise of GPU memory fit. |
| NVIDIA RTX PRO 6000 Blackwell · 96 GB VRAM | Start at 32K; increase only after checking GPU memory use. |
| NVIDIA H200 · 141 GB VRAM | Start at 64K; check memory use before raising context. |
| DGX Spark ×1 · 128 GB unified memory | Start at 32K; memory is shared with the OS. Check Ollama allocation. |
| AMD Ryzen AI Max+ 395 · 128 GB unified memory | Start at 32K in the AMD-demonstrated LM Studio route; verify Ollama separately. |
| AMD Instinct MI300X · 192 GB VRAM | Start at 64K; raise context if the full model stays on GPU. |
| Intel Arc GPU · 24 GB VRAM | Use the 20B package for a practical local context. |
| Apple Silicon · 96 GB unified memory | Start at 16K–32K; stop if the system begins swapping. |
| Apple Silicon · 128 GB unified memory | Start at 32K; monitor unified-memory pressure before increasing. |
| CPU system RAM · 128 GB | Start at 16K–32K; longer contexts raise memory use and latency. |
The model supports up to 128K tokens, but the active context you can use depends on the full memory footprint and runtime. Confirm with ollama ps.
Set up the exact gpt-oss · 120B Ollama MXFP4 package or the community uncensored HauhauCS Aggressive MXFP4 GGUF, then verify a reasoning response against a known problem.
This setup uses the package documented for this task. Review its source and supported platforms before starting.
Save your machine in My Hardware to get an automatic starting choice. You can always choose any package yourself.
OpenAI recommends at least 60 GB GPU or unified memory for this MXFP4 package. Ollama tested gpt-oss:120b on one DGX Spark; 80–96 GB dedicated GPUs and 128 GB Apple Silicon are other local options. AMD demonstrated the Ryzen AI Max+ 395 with 128 GB shared memory and 96 GB graphics allocation in LM Studio; confirm Ollama support separately.
Choose where you want to run it. The instructions below update to match your selection.
Ollama requires macOS Sonoma 14 or newer; Apple M-series or x86 CPU, with Metal acceleration on Apple Silicon. The 120B MXFP4 package targets a single 80 GB GPU.
Ollama for macOS requires Sonoma 14 or newer. Download the macOS build from the official Ollama download page, open the .dmg, drag Ollama into Applications, and launch it so the background service starts. The app verifies the `ollama` CLI is on your PATH and offers to create the link in /usr/local/bin if it is missing.
Download Ollama for Mac Ollama macOS documentationOpen a new Terminal window (Command+Space, type Terminal). The Ollama app must stay running, since it hosts the local service on port 11434. Confirm the CLI resolves and check the version.
ollama --versionThe publisher package on the Ollama library is gpt-oss:120b (65 GB, 128K advertised context window, native MXFP4 weights). Pulling it first separates the download from the chat launch so you can see progress and disk usage. This needs roughly 65 GB free plus cache overhead.
gpt-oss on the Ollama libraryollama pull gpt-oss:120bLaunch the interactive session. The model's maximum position embedding is 131072 tokens, but the runtime context is a separate setting. Ollama reads OLLAMA_CONTEXT_LENGTH only when the server starts, and the desktop app already owns that server, so an inline environment variable on `ollama run` would be ignored. Instead set the context from inside the chat prompt with `/set parameter num_ctx 32768`. Start at 32768 on Apple Silicon with 128 GB unified memory; on a 192 GB AMD MI300X or equivalent you can try 65536, and on memory-tight or CPU-only systems use 16384 or lower. To apply a different default at launch instead, quit the Ollama app, then start the server yourself in this terminal with the variable set, and run the model from a second terminal.
Ollama context length documentation Ollama FAQ: specifying context window sizeollama run gpt-oss:120bLeave the `ollama run` session open in Terminal 1. Open Terminal 2 and inspect the loaded model: the CONTEXT column shows the context actually allocated and PROCESSOR shows whether weights sit on Metal GPU or CPU. If memory pressure is high, return to Terminal 1, run `/set parameter num_ctx 16384`, and confirm again. `ollama list` proves the exact tag is installed.
Ollama CLI referenceollama ps
ollama listIn the Terminal 1 chat prompt, enter a short multi-step problem with a verifiable answer: "A train travels 60 miles in 1.5 hours. What is its average speed in miles per hour? Show the calculation." The correct final answer is 40 mph from 60 ÷ 1.5 = 40. Check both the final value and the intermediate step before raising context or reasoning effort. This is the first local model response for this path.
A train travels 60 miles in 1.5 hours. What is its average speed in miles per hour? Show the calculation.Stay in the interactive `ollama run gpt-oss:120b` session for further prompts — history is kept for the session. For programmatic use, the same app-hosted service exposes an OpenAI-compatible endpoint at http://localhost:11434/v1/chat/completions and a native endpoint at http://localhost:11434/api/chat. Example native request from Terminal 2 (keep the app or run session alive): curl http://localhost:11434/api/chat -d '{"model":"gpt-oss:120b","messages":[{"role":"user","content":"Hello!"}]}' Example OpenAI-compatible call with the dummy key "ollama": curl http://localhost:11434/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"gpt-oss:120b","messages":[{"role":"user","content":"Hello!"}]}' For a browser chat UI, run `ollama launch` from a terminal and pick the integration, or use the Ollama app window with the same local service; type your message and press send to get a reply.
Run gpt-oss locally with Ollama (OpenAI Cookbook)curl http://localhost:11434/api/chat -d '{"model":"gpt-oss:120b","messages":[{"role":"user","content":"Hello!"}]}'
curl http://localhost:11434/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"gpt-oss:120b","messages":[{"role":"user","content":"Hello!"}]}'
ollama launchAsk a short multi-step reasoning question with a verifiable answer. Confirm the final answer and key intermediate steps are correct before increasing context or reasoning effort.
This guide covers the official runnable package. A community uncensored GGUF is not offered here because its exact Harmony chat-template import path has not been verified. Large contexts increase memory use; start small and increase only after the first response succeeds.
Verify total VRAM or unified memory. The publisher 120B MXFP4 package targets a single 80 GB GPU; close other GPU applications or choose the smaller 20B variant if your hardware is below that.
Confirm the exact GGUF file is used and that its Harmony template metadata is applied. Do not substitute an invented Jinja or Go TEMPLATE. Check the import output for architecture or template errors.
Reduce context length or reasoning effort, then restart the session.
Use the exact 120B package tag or the exact imported model name for your runtime, and verify the loaded model identity before relying on output.
Original instructions, model files, and compatibility notes behind this setup.
Source-backed package and platform-route plan for private local chat with Qwen3.5 · 9B. Preserves the official Ollama 9B package, a smaller 4B family fallback, and a clearly labeled community uncensored GGUF option.
Run Qwen3.8 27B with Ollama, connect it to Cline in VS Code, and make one reviewable code change in a tiny local project.
Generate images from text prompts in ComfyUI with the fast distilled or more adjustable base 4B model. Save your results and reuse the visual workflow.