Build a local browser search tool for your text and Markdown documents. Get ranked source passages, keep a reusable index, and export results for the reranking workflow.
Conservative starting points for this small CPU-first app. GPU memory is not required; the listed memory is system/unified RAM.
| Machine | Qwen3 Embedding 0.6B |
|---|---|
| NVIDIA PC · 8 GB RAM | Runs on CPU; 16 GB RAM gives more room for other apps |
| AMD PC · 8 GB RAM | Runs on CPU; no ROCm or Vulkan dependency |
| Intel PC · 8 GB RAM | Runs on CPU; an Arc GPU is not required |
| Apple Silicon · 8 GB unified memory | Runs on CPU; 16 GB unified memory gives more headroom |
| CPU only · 8 GB RAM | Suitable for the supplied small collection; larger batches use more memory |
These are memory estimates for short passages and one local user, not speed measurements. The guide uses CPU on every OS for a consistent first run.
| Machine | Qwen3 Embedding 0.6B |
|---|---|
| CPU / Apple Silicon · 8–16 GB RAM | Start at 1,024 tokens per passage; 200-word chunks, batch 4 |
The published model supports longer inputs. This starter chooses short passages for useful search and modest memory use; increasing the limit raises RAM use.
Build a local browser search tool for your text and Markdown documents. Get ranked source passages, keep a reusable index, and export results for the reranking workflow.
This setup uses the package documented for this task. Review its source and supported platforms before starting.
Save your machine in My Hardware to get an automatic starting choice. You can always choose any package yourself.
Start at 1,024 tokens per passage. Batch size 4 keeps memory modest; larger batches and longer passages consume more RAM.
Choose where you want to run it. The instructions below update to match your selection.
CPU inference on this route. Short local documents and one user; no GPU-specific driver setup.
Install uv from its official instructions, then open a new PowerShell window. The commands below create an isolated Python 3.11 environment; there is no need to change your system Python.
uv installationpowershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"Use a new folder named yourunai-search in your home directory. These commands run in PowerShell. This starter deliberately uses CPU inference: its small model works with NVIDIA, AMD, Intel, Apple or no GPU. No CUDA installation is needed. Leave this window in the project folder for the remaining commands.
uv virtual environments PyTorch installation choicesNew-Item -ItemType Directory -Force -Path "$env:USERPROFILE\yourunai-search" | Out-Null
Set-Location "$env:USERPROFILE\yourunai-search"
uv venv --python 3.11
uv pip install --python .\.venv\Scripts\python.exe torch --index-url https://download.pytorch.org/whl/cpu
uv pip install --python .\.venv\Scripts\python.exe sentence-transformers==6.1.0 gradio==6.28.0Right-click the download link below and choose Save Link As. Save the .py file directly in the project folder created above; keep its .py extension, not .txt. This complete YouRunAI starter creates the browser interface and output folders. You can inspect it in a text editor. Model weights download separately when the runtime needs them.
Download private-search.pyIn PowerShell, run the app and leave it open. It downloads Qwen/Qwen3-Embedding-0.6B, creates three sample documents, and builds a persistent index. Open http://127.0.0.1:7860 after the local URL appears.
.\.venv\Scripts\python.exe private-search.pyClick Search with “How do I return an item?” in the question box. Confirm returns.md is the most relevant passage and that it includes the 30-day return rule. Similarity scores compare results within this query; they are not confidence percentages.
Open the documents folder inside yourunai-search. Add UTF-8 .txt or .md files, then click Rebuild index. Search again with a question answered by one of your files. This app splits files into 200-word passages with overlap and bounds them to 1,024 tokens. PDFs, scans and Word files must first be converted to text; the app does not silently skip their extraction.
Read the returned filename and source passage against your original document. Click Download candidates for reranking to save candidates.json. The persistent files are search-index/passages.json and search-index/vectors.npy; the last query is search-index/candidates.json. Results are retrieved text, not a generated answer.
Press Ctrl+C in the running PowerShell window to stop the app. Your files stay in yourunai-search. Next time run the commands below, then reopen http://127.0.0.1:7860. Dependencies and downloaded weights are reused. Keep this address local; there is no public sharing link.
Set-Location "$env:USERPROFILE\yourunai-search"
.\.venv\Scripts\python.exe private-search.pyA query returns a relevant filename and exact passage from your collection. New files appear after rebuilding, and restarting reuses the saved index. You can hand candidates.json to the reranker.
Keep the Python process open and wait until it prints the local URL. If another app uses that port, stop that app or run these examples one at a time.
The first run downloads the publisher model to the Hugging Face cache. Watch the Terminal/PowerShell progress and allow the download to finish; later starts reuse it.
Run the installation command from this guide again in the project folder. Launch with the .venv Python path shown here so you use the same environment.
Close memory-heavy apps and other models. Keep the supplied short input limits and batch size. The small-model CPU route needs system RAM, not GPU VRAM.
Click Rebuild index after adding, editing or deleting a document. The app prevents stale results from being presented as current.
Export it to UTF-8 text or Markdown first. Scanned documents need OCR/vision extraction; this example indexes text files only.
Original instructions, model files, and compatibility notes behind this setup.
Source-backed package and platform-route plan for private local chat with Qwen3.5 · 9B. Preserves the official Ollama 9B package, a smaller 4B family fallback, and a clearly labeled community uncensored GGUF option.
Run Qwen3.8 27B with Ollama, connect it to Cline in VS Code, and make one reviewable code change in a tiny local project.
Generate images from text prompts in ComfyUI with the fast distilled or more adjustable base 4B model. Save your results and reuse the visual workflow.