Manual setup

For people who would rather not run the trailSailServer package. Everything here runs on the machine that serves the models; nothing is exposed except through tailscale serve, and each service is published on your tailnet on its own HTTPS port.

Use a tailnet hostname such as <tailnet-host>.ts.net (your machine's tailscale status shows it).

ServiceLocal portTailnet URL
Splash8000https://<tailnet-host>.ts.net
llama.cpp router8081https://<tailnet-host>.ts.net:8443
SearXNG8888https://<tailnet-host>.ts.net:8444
Strata (optional)8082https://<tailnet-host>.ts.net:8445

There is no Homebrew formula, so it runs as a container — OrbStack is the lightest Docker runtime on macOS; Docker Desktop works the same way.

brew install --cask orbstack   # open it once to finish setup
mkdir -p ~/searxng

Write the settings file before the first start. Two lines matter: the json format (the app needs it) and limiter: false (the bot limiter needs Valkey and would otherwise block API calls).

printf 'use_default_settings: true\nserver:\n  secret_key: "%s"\n  limiter: false\n  image_proxy: false\nsearch:\n  formats:\n    - html\n    - json\n  safe_search: 1\n' "$(openssl rand -hex 32)" > ~/searxng/settings.yml

Then run it bound to localhost, restarting on reboot, and publish it:

docker run -d --name searxng --restart unless-stopped -p 127.0.0.1:8888:8080 \
  -v ~/searxng:/etc/searxng \
  -e BASE_URL=https://<tailnet-host>.ts.net:8444/ searxng/searxng

curl -s 'http://127.0.0.1:8888/search?q=tailscale&format=json' | head -c 300
tailscale serve --bg --https=8444 8888

In TrailSail → Settings → Web search: provider SearXNG, base URL https://<tailnet-host>.ts.net:8444, then Test search. To update later: docker pull searxng/searxng && docker rm -f searxng, then run the docker run line again. Logs: docker logs -f searxng.

Splash (native MLX)

Splash is Inco AI's open-source engine for Apple silicon. It needs an Apple M3 or newer and macOS 26.4 or later. Install it from its makers' Homebrew tap:

brew install incoai/tap/splash

Splash serves without authentication unless you give it a key, and then ignores any key a client sends. Before publishing it on your tailnet, make a key and start Splash with it:

export SPLASH_API_KEY="$(openssl rand -hex 32)"
echo "$SPLASH_API_KEY"     # this is the key to paste into TrailSail
splash serve --model mlx-community/Qwen3.8-27B-4bit --allowed-host <tailnet-host>.ts.net

The first start downloads the model. --allowed-host is required — without it Splash refuses requests addressed to the tailnet hostname. Tailscale publishes it with tailscale serve --bg 8000. Health: curl -s http://127.0.0.1:8000/health.

If its inference engine crashes three times in a minute (usually memory pressure from another large model) it stops auto-restarting and must be started by hand — so save a launchd agent to survive reboots:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key><string>co.uk.yard8.splash</string>
  <key>ProgramArguments</key>
  <array>
    <string>/opt/homebrew/bin/splash</string>
    <string>serve</string>
    <string>--model</string><string>&lt;hf repo&gt;</string>
    <string>--allowed-host</string><string>&lt;tailnet-host&gt;.ts.net</string>
  </array>
  <key>EnvironmentVariables</key>
  <dict>
    <key>SPLASH_API_KEY</key><string>YOUR-KEY</string>
  </dict>
  <key>RunAtLoad</key><true/>
  <key>KeepAlive</key><true/>
  <key>StandardOutPath</key><string>/tmp/splash.log</string>
  <key>StandardErrorPath</key><string>/tmp/splash.log</string>
</dict>
</plist>

Save that as ~/Library/LaunchAgents/co.uk.yard8.splash.plist, make it yours alone with chmod 600 (it holds the key), then launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/co.uk.yard8.splash.plist. To change the key later, edit it there and reload the agent. Memory budget on 32 GB: Splash's 27B uses about 20 GB when busy, the router's about 18 GB loaded — do not keep both at once.

llama.cpp router

For your own quantisations, with models loaded on demand from TrailSail's model manager.

brew install llama.cpp
mkdir -p ~/.config/llama && openssl rand -hex 32 > ~/.config/llama/api.key && chmod 600 ~/.config/llama/api.key

Download a model into the cache once (press Ctrl+C when it says it is listening): llama-server -hf <repo:QUANT> --port 8089. Then run the router:

llama-server --models-max 1 --no-models-autoload --host 127.0.0.1 --port 8081 \
  --api-key-file ~/.config/llama/api.key --metrics --jinja -c 32768 -ctk q8_0 -ctv q8_0 -fa on

Publish it with tailscale serve --bg --https=8443 8081, and save the same flags in a launchd agent (co.uk.yard8.llama-router) to keep it running after reboot — tailscale serve --bg entries persist on their own.

Strata (optional, experimental on the Mac)

Niko1221/Strata (MIT) is a local inference engine with OpenAI- and Anthropic-compatible APIs; version 0.1.44+ added a native Metal backend for M1–M5, built from source. It runs one model family only — a ~125B mixture-of-experts Qwen (about 6B active per token) in low-bit packs — so Splash and the llama.cpp router stay the way to run the 27B. On 32 GB it streams experts from the SSD: about 14 tok/s decode / 90 tok/s prompt on an M4, one request at a time, ~32K context, roughly 110 GB of disk. Strata and Splash's 27B cannot both be loaded at once.

Check the machine first:

sw_vers                     # macOS 15 or newer
xcode-select --install      # Command Line Tools
df -h /                     # need about 120 GB free
sysctl hw.memsize           # 34359738368 = 32 GB

Then clone, build and generate a key:

git clone https://github.com/Niko1221/Strata ~/Strata && cd ~/Strata
./setup.sh --setup          # builds; picks the Q2_0 pack
mkdir -p ~/.config/strata && openssl rand -hex 32 > ~/.config/strata/api.key && chmod 600 ~/.config/strata/api.key
./run-q2_0.sh               # start (later starts are just this)

Test, publish and add a launchd agent (co.uk.yard8.strata, with RunAtLoad and KeepAlive off so it never competes with Splash for memory at boot):

curl -s http://127.0.0.1:8082/v1/models -H "Authorization: Bearer $(cat ~/.config/strata/api.key)"
tailscale serve --bg --https=8445 8082

Add a TrailSail profile: name, URL https://<tailnet-host>.ts.net:8445, key from ~/.config/strata/api.key; if it does not report a context length, set 32,768 for that chat under Settings → Context & memory.

Adding any of them to TrailSail

Settings → Connections → Add: a name, the URL, the API key, Test — the models it lists confirm the connection. Splash and the router profiles are created for you if the machine was set up with trailSailServer; by hand, add each one.

Found a problem, or is something here unclear? Report it.