Manual setup
For people who would rather not run the trailSailServer package. Everything here runs on the machine that serves the models; nothing is exposed except through tailscale serve, and each service is published on your tailnet on its own HTTPS port.
Use a tailnet hostname such as <tailnet-host>.ts.net (your machine's tailscale status shows it).
| Service | Local port | Tailnet URL |
|---|---|---|
| Splash | 8000 | https://<tailnet-host>.ts.net |
| llama.cpp router | 8081 | https://<tailnet-host>.ts.net:8443 |
| SearXNG | 8888 | https://<tailnet-host>.ts.net:8444 |
| Strata (optional) | 8082 | https://<tailnet-host>.ts.net:8445 |
SearXNG (web search)
There is no Homebrew formula, so it runs as a container — OrbStack is the lightest Docker runtime on macOS; Docker Desktop works the same way.
brew install --cask orbstack # open it once to finish setup
mkdir -p ~/searxng
Write the settings file before the first start. Two lines matter: the json format (the app needs it) and limiter: false (the bot limiter needs Valkey and would otherwise block API calls).
printf 'use_default_settings: true\nserver:\n secret_key: "%s"\n limiter: false\n image_proxy: false\nsearch:\n formats:\n - html\n - json\n safe_search: 1\n' "$(openssl rand -hex 32)" > ~/searxng/settings.yml
Then run it bound to localhost, restarting on reboot, and publish it:
docker run -d --name searxng --restart unless-stopped -p 127.0.0.1:8888:8080 \
-v ~/searxng:/etc/searxng \
-e BASE_URL=https://<tailnet-host>.ts.net:8444/ searxng/searxng
curl -s 'http://127.0.0.1:8888/search?q=tailscale&format=json' | head -c 300
tailscale serve --bg --https=8444 8888
In TrailSail → Settings → Web search: provider SearXNG, base URL https://<tailnet-host>.ts.net:8444, then Test search. To update later: docker pull searxng/searxng && docker rm -f searxng, then run the docker run line again. Logs: docker logs -f searxng.
Splash (native MLX)
Splash is Inco AI's open-source engine for Apple silicon. It needs an Apple M3 or newer and macOS 26.4 or later. Install it from its makers' Homebrew tap:
brew install incoai/tap/splash
Splash serves without authentication unless you give it a key, and then ignores any key a client sends. Before publishing it on your tailnet, make a key and start Splash with it:
export SPLASH_API_KEY="$(openssl rand -hex 32)"
echo "$SPLASH_API_KEY" # this is the key to paste into TrailSail
splash serve --model mlx-community/Qwen3.8-27B-4bit --allowed-host <tailnet-host>.ts.net
The first start downloads the model. --allowed-host is required — without it Splash refuses requests addressed to the tailnet hostname. Tailscale publishes it with tailscale serve --bg 8000. Health: curl -s http://127.0.0.1:8000/health.
If its inference engine crashes three times in a minute (usually memory pressure from another large model) it stops auto-restarting and must be started by hand — so save a launchd agent to survive reboots:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key><string>co.uk.yard8.splash</string>
<key>ProgramArguments</key>
<array>
<string>/opt/homebrew/bin/splash</string>
<string>serve</string>
<string>--model</string><string><hf repo></string>
<string>--allowed-host</string><string><tailnet-host>.ts.net</string>
</array>
<key>EnvironmentVariables</key>
<dict>
<key>SPLASH_API_KEY</key><string>YOUR-KEY</string>
</dict>
<key>RunAtLoad</key><true/>
<key>KeepAlive</key><true/>
<key>StandardOutPath</key><string>/tmp/splash.log</string>
<key>StandardErrorPath</key><string>/tmp/splash.log</string>
</dict>
</plist>
Save that as ~/Library/LaunchAgents/co.uk.yard8.splash.plist, make it yours alone with chmod 600 (it holds the key), then launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/co.uk.yard8.splash.plist. To change the key later, edit it there and reload the agent. Memory budget on 32 GB: Splash's 27B uses about 20 GB when busy, the router's about 18 GB loaded — do not keep both at once.
llama.cpp router
For your own quantisations, with models loaded on demand from TrailSail's model manager.
brew install llama.cpp
mkdir -p ~/.config/llama && openssl rand -hex 32 > ~/.config/llama/api.key && chmod 600 ~/.config/llama/api.key
Download a model into the cache once (press Ctrl+C when it says it is listening): llama-server -hf <repo:QUANT> --port 8089. Then run the router:
llama-server --models-max 1 --no-models-autoload --host 127.0.0.1 --port 8081 \
--api-key-file ~/.config/llama/api.key --metrics --jinja -c 32768 -ctk q8_0 -ctv q8_0 -fa on
Publish it with tailscale serve --bg --https=8443 8081, and save the same flags in a launchd agent (co.uk.yard8.llama-router) to keep it running after reboot — tailscale serve --bg entries persist on their own.
Strata (optional, experimental on the Mac)
Niko1221/Strata (MIT) is a local inference engine with OpenAI- and Anthropic-compatible APIs; version 0.1.44+ added a native Metal backend for M1–M5, built from source. It runs one model family only — a ~125B mixture-of-experts Qwen (about 6B active per token) in low-bit packs — so Splash and the llama.cpp router stay the way to run the 27B. On 32 GB it streams experts from the SSD: about 14 tok/s decode / 90 tok/s prompt on an M4, one request at a time, ~32K context, roughly 110 GB of disk. Strata and Splash's 27B cannot both be loaded at once.
Check the machine first:
sw_vers # macOS 15 or newer
xcode-select --install # Command Line Tools
df -h / # need about 120 GB free
sysctl hw.memsize # 34359738368 = 32 GB
Then clone, build and generate a key:
git clone https://github.com/Niko1221/Strata ~/Strata && cd ~/Strata
./setup.sh --setup # builds; picks the Q2_0 pack
mkdir -p ~/.config/strata && openssl rand -hex 32 > ~/.config/strata/api.key && chmod 600 ~/.config/strata/api.key
./run-q2_0.sh # start (later starts are just this)
Test, publish and add a launchd agent (co.uk.yard8.strata, with RunAtLoad and KeepAlive off so it never competes with Splash for memory at boot):
curl -s http://127.0.0.1:8082/v1/models -H "Authorization: Bearer $(cat ~/.config/strata/api.key)"
tailscale serve --bg --https=8445 8082
Add a TrailSail profile: name, URL https://<tailnet-host>.ts.net:8445, key from ~/.config/strata/api.key; if it does not report a context length, set 32,768 for that chat under Settings → Context & memory.
Adding any of them to TrailSail
Settings → Connections → Add: a name, the URL, the API key, Test — the models it lists confirm the connection. Splash and the router profiles are created for you if the machine was set up with trailSailServer; by hand, add each one.
Found a problem, or is something here unclear? Report it.