TL;DR: The call kept saying “waiting for media” because the browser never opened a WebSocket to LiveKit. The root cause was duplicate
Access-Control-Allow-Originheaders on/sfu/get(CORS), which stopped the JWT response. Fixing CORS and ensuring the WS proxy worked (HTTP 101 in logs) solved it.
What I’m building
- A Matrix homeserver (Synapse) at
matrix.example.com(replace with your domain). - TURN/STUN (coTURN) for NAT traversal.
- Element Call backed by LiveKit, fronted by
rtc.example.com. - Nginx (host) as the single reverse proxy for everything.
- Cloudflare DNS (with
rtc.*set to DNS-only, no orange cloud). - UFW firewall opened for Matrix federation, TURN, and LiveKit media ports.
I used Docker for Synapse, PostgreSQL, LiveKit and the JWT helper. I used host Nginx (not Nginx in Docker) to avoid port binding conflicts on 80/443/8448.
Prereqs
- DNS A/AAAA:
matrix.example.com→ your server (v4/v6)rtc.example.com→ your server (v4/v6)
- Certificates:
matrix.example.comandrtc.example.comvia Let’s Encrypt on the host
- Cloudflare: DNS-only (grey cloud) for
rtc.example.comso WebSockets & UDP work without interference. - UFW / firewall open:
- 80/tcp, 443/tcp
- 8448/tcp (Matrix federation)
- 3478/tcp, 3478/udp and 5349/tcp (TURN/TLS)
- LiveKit: 7881/tcp and 50100–50200/udp (or your chosen range)
- Docker + docker compose installed.
Synapse + PostgreSQL
1) The PostgreSQL collation gotcha
Synapse prefers the database collation C. If your Postgres cluster was initialized with en_US.utf8, Synapse will error like:
Two ways to resolve:
- Preferred (clean): Re-initialize the Postgres cluster with
CandUTF-8:
# docker-compose.yml (excerpt for Postgres)
services:
db:
image: postgres:16
environment:
POSTGRES_DB: synapse
POSTGRES_USER: synapse
POSTGRES_PASSWORD: <strong-password>
POSTGRES_INITDB_ARGS: "--locale=C --encoding=UTF8 --lc-collate=C --lc-ctype=C"
volumes:
- ./pgdata:/var/lib/postgresql/data
Requires wiping the volume and recreating the DB.
- Pragmatic (works quickly): In Synapse’s DB config, set
allow_unsafe_locale: true. This bypasses the check. It’s fine for hobby use; for production, prefer the cleanCcluster.
2) Start Synapse and generate config
docker compose up -d db synapse
# Logs
docker compose logs --tail=200 synapse
Ensure Synapse prints your server_name and public base URL and stays up.
3) Create an admin user
# Exec into the running Synapse container:
docker compose exec synapse register_new_matrix_user \
-c /data/homeserver.yaml -u <username> -p <password> \
-a -k
If you see “Unknown execution mode”, you probably ran the binary with the wrong entrypoint. Use
docker compose exec synapse …against the running container.
TURN (coTURN)
1) Avoid bad inline comments
If you see errors like:
…it means a # comment is on the same line as a boolean directive. Move comments to their own lines.
2) Minimal turnserver.conf
listening-port=3478
tls-listening-port=5349
fingerprint
use-auth-secret
static-auth-secret=<shared-secret> # also set in Synapse
realm=example.com # used in creds generation
total-quota=0
bps-capacity=0
cli-password=<admin-pass>
no-cli
cert=/etc/letsencrypt/live/turn.example.com/fullchain.pem
pkey=/etc/letsencrypt/live/turn.example.com/privkey.pem
# If behind NAT:
# external-ip=<public-ip>/<internal-ip>
3) Wire TURN into Synapse
In homeserver.yaml:
turn_uris:
- "turn:turn.example.com?transport=udp"
- "turn:turn.example.com?transport=tcp"
- "turns:turn.example.com:5349?transport=tcp"
turn_shared_secret: "<shared-secret>"
turn_user_lifetime: "1d"
Verify the homeserver issues TURN creds:
TOKEN='<your matrix access token>'
curl -s -H "Authorization: Bearer $TOKEN" \
https://matrix.example.com/_matrix/client/v3/voip/turnServer | jq
You should see uris, a time-limited username and password.
Nginx (host) for Synapse and federation
Create /etc/nginx/conf.d/matrix.conf:
# Client traffic on 443
server {
listen 443 ssl; listen [::]:443 ssl;
server_name matrix.example.com;
ssl_certificate /etc/letsencrypt/live/matrix.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/matrix.example.com/privkey.pem;
# Proxy client & admin APIs to Synapse (container) on 8008
location ~ ^(/_matrix|/_synapse/client) {
proxy_pass http://127.0.0.1:8008;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $host;
client_max_body_size 50M;
}
# Advertise homeserver base + RTC focus (MSC4143) via .well-known
location = /.well-known/matrix/client {
default_type application/json;
add_header Access-Control-Allow-Origin "*" always;
return 200 '{"m.homeserver":{"base_url":"https://matrix.example.com"},"org.matrix.msc4143.rtc_foci":[{"type":"livekit","livekit_service_url":"https://rtc.example.com"}]}';
}
}
# Federation on 8448
server {
listen 8448 ssl http2; listen [::]:8448 ssl http2;
server_name matrix.example.com;
ssl_certificate /etc/letsencrypt/live/matrix.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/matrix.example.com/privkey.pem;
location / {
proxy_pass http://127.0.0.1:8008;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $host;
client_max_body_size 50M;
}
}
Reload and sanity check:
sudo nginx -t && sudo systemctl reload nginx
curl -s https://matrix.example.com/.well-known/matrix/client | jq
Also check Synapse supports RTC signaling (MSC4140) so clients actually use it:
curl -s https://matrix.example.com/_matrix/client/versions | jq '.unstable_features."org.matrix.msc4140"'
# expect: true
Element Call + LiveKit
I’ll run LiveKit and the small JWT helper (Element’s elementcall_jwt) in Docker. LiveKit handles media; the JWT helper mints access tokens for WebSocket connects.
1) LiveKit config (/etc/livekit.yaml inside container)
port: 7880
bind_addresses: ["0.0.0.0"]
rtc:
tcp_port: 7881
port_range_start: 50100
port_range_end: 50200
use_external_ip: true
logging:
level: info
turn:
enabled: false
keys:
lk_prod_1: "REPLACE_WITH_A_64_CHAR_RANDOM_SECRET___________________________________"
Important: the secret must be >= 32 chars. The default
devkeywill triggersecret is too shortwarnings and won’t work with the JWT helper.
2) elementcall_jwt env
Run it with:
LIVEKIT_URL=wss://rtc.example.com(root WS URL, no/livekit/sfupath)LIVEKIT_KEY=lk_prod_1LIVEKIT_SECRET=<the long secret above>LIVEKIT_JWT_PORT=8080(internal HTTP port the proxy will hit)- Optionally:
LIVEKIT_FULL_ACCESS_HOMESERVERS=*during setup
Check logs on start; it prints the LIVEKIT_URL it will advertise.
3) Nginx (host) for rtc.example.com
Create /etc/nginx/conf.d/rtc.conf:
server {
listen 443 ssl; listen [::]:443 ssl;
server_name rtc.example.com;
ssl_certificate /etc/letsencrypt/live/rtc.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/rtc.example.com/privkey.pem;
access_log /var/log/nginx/rtc.access.log combined;
# 3a) JWT endpoint with clean CORS (avoid duplicate ACAO)
location = /sfu/get {
proxy_hide_header Access-Control-Allow-Origin;
add_header Access-Control-Allow-Origin $http_origin always;
add_header Vary "Origin" always;
add_header Access-Control-Allow-Methods "POST, OPTIONS" always;
add_header Access-Control-Allow-Headers "Accept, Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization" always;
if ($request_method = OPTIONS) { return 204; }
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://127.0.0.1:8070/sfu/get; # elementcall_jwt
}
# 3b) LiveKit WS & HTTP (catch-all)
location / {
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Sec-WebSocket-Protocol $http_sec_websocket_protocol; # "livekit"
proxy_set_header Origin $http_origin;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering off;
proxy_read_timeout 3600s;
proxy_pass http://127.0.0.1:7880; # livekit
}
}
Reload and basic checks:
sudo nginx -t && sudo systemctl reload nginx
# JWT preflight (should return a single ACAO header)
curl -si -X OPTIONS https://rtc.example.com/sfu/get \
-H 'Origin: https://app.element.io' \
-H 'Access-Control-Request-Method: POST' \
-H 'Access-Control-Request-Headers: authorization, content-type' | sed -n '1,30p'
# Expected: one Access-Control-Allow-Origin and 200/204
Why I did this: I initially had two
Access-Control-Allow-Originheaders (one added by the upstream, one by Nginx). Browsers reject that with “Access-Control-Allow-Origin cannot contain more than one origin”, so the JWT response never reached the client. Fixing CORS fixed everything.
The “waiting for media” debugging story (how I found it)
Symptom: Element Call created rooms, but calls stayed on “waiting for media.”
What worked:
elementcall_jwtcould CreateRoom in LiveKit (seen in logs).- TURN creds endpoint returned time-limited credentials.
What didn’t appear:
- No HTTP 101 lines in
rtc.access.log→ the browser never established a WebSocket to LiveKit.
Step 1: prove the WS vhost works (even without auth)
curl -v --http1.1 \
-H 'Connection: Upgrade' -H 'Upgrade: websocket' \
-H 'Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==' \
-H 'Sec-WebSocket-Version: 13' \
https://rtc.example.com/rtc -o /dev/null
Result: I got a 401 (expected), but importantly I saw a log line in rtc.access.log. So Nginx WS proxying was fine.
Step 2: check the browser console
The smoking gun in DevTools:
The browser refused the JWT call due to duplicated ACAO headers, so no token → no WebSocket connect.
Fix: in Nginx I added:
proxy_hide_header Access-Control-Allow-Origin;
add_header Access-Control-Allow-Origin $http_origin always;
add_header Vary "Origin" always;
…and ensured no other add_header created duplicates. After that, /sfu/get succeeded and the WebSocket to wss://rtc.example.com immediately followed (I saw HTTP 101 in the logs).
Step 3: confirm LiveKit side
Once the WS was up, LiveKit logs showed participants joining (not just RoomService.CreateRoom), and calls were established.
Useful verification commands
# Synapse features (expect MSC4140 true)
curl -s https://matrix.example.com/_matrix/client/versions | jq '.unstable_features."org.matrix.msc4140"'
# Well-known with RTC focus
curl -s https://matrix.example.com/.well-known/matrix/client | jq
# TURN creds (with your access token)
curl -s -H "Authorization: Bearer $TOKEN" \
https://matrix.example.com/_matrix/client/v3/voip/turnServer | jq
# JWT health
curl -si -X POST https://rtc.example.com/sfu/get
# LiveKit simple HTTP probe
curl -si https://rtc.example.com | head
# Nginx logs (look for 101 Switching Protocols when a call starts)
sudo tail -f /var/log/nginx/rtc.access.log | grep ' 101 '
Common pitfalls (I hit these so you don’t have to)
- Host Nginx vs Docker Nginx: If you already run Nginx on the host, don’t also bind 80/443/8448 in a Docker Nginx — you’ll get
bind() ... already in useand restart loops. Use host Nginx to reverse proxy to containers. - Nginx
http2directive: Old Nginx may not support thehttp2directive onlisten. Uselisten 443 ssl;(and addhttp2if your version supports it). - Certificate name mismatch: Make sure
rtc.example.com’s vhost uses a certificate for that exact hostname (initially I had thematrix.*cert onrtc.*and curl complained). - Postgres collation: Either initialize the cluster with
Cor useallow_unsafe_locale: truein Synapse DB config to get running quickly. - CORS duplication on
/sfu/get: Only ONEAccess-Control-Allow-Originheader. If the upstream adds it too, useproxy_hide_header Access-Control-Allow-Origin;on the Nginx location. - Cloudflare: Use DNS-only for
rtc.*. Proxies can interfere with WS and UDP paths. - Firewall: Open the LiveKit UDP range and TURN ports on both v4 and v6.
Final checklist (print me)
-
https://matrix.example.com/.well-known/matrix/clientreturns bothm.homeserver.base_urlandorg.matrix.msc4143.rtc_focipointing tohttps://rtc.example.com. -
/_matrix/client/versionsshows"org.matrix.msc4140": true. -
/sfu/getpreflight returns oneAccess-Control-Allow-Originand 200/204. - Starting a call creates HTTP 101 entries to
wss://rtc.example.cominrtc.access.log. - LiveKit logs show participants joining (not just CreateRoom).
-
/voip/turnServerreturns time-limited TURN credentials. - Cloudflare set to DNS-only for
rtc.*. UFW allows 7881/tcp and your LiveKit UDP range.
Credits & tooling
- Matrix Synapse, coTURN, LiveKit, Element Call.
curl,jq,docker compose logs, Nginx access logs. These are your best friends.- The debugging breakthrough was catching CORS errors in the browser console and looking for HTTP 101 in Nginx logs.
Happy calling! 🎉
Downloads: Markdown · Signature (.asc)
View OpenPGP signature
-----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEVaKl3oR5K6zGyu4/tYgoUOBMjSoFAmpkbuQACgkQtYgoUOBM jSpglxAArFNgxheR8m2r6QvtoEuHJwI51WQteu+0un0FiIfHKtFPcXd2HiVqzODu YZvIGCAecMURQMRvrgB8cO3ebY8Yi9EECnC1sHsluE05GOuro5Htja7TbsGxwTc3 PIe+zju7lIhTHonqLogUyyunhWxOfg1RCaSjp3n6k/r2iEamgKu6Cihrv54wstGa SVJbwubie1D9TPcXU3ynC+ynNBcmVevFl7g/X7Ie8Pw0SP1dJF+we5iVqUrZgPO2 AudHlWRm13j7Xifv/JxqymkZV1XiShIY8Mb0Ju8m5+HjkoIaZDtSfFyt+AwPdHDQ m3sMXA7yZUvy+pXtziwrOnHFAez22goAr9Ar9KcwGQgRvyxKuuKIuTq+yxtCuXBF fPWo5pL0rMtIfwRyaiiX9bwV+WbBXNhghTPnaxuQ3CWkLdiwaycI7xPDAg8FzFAR 7yoN0vqhKSvZlAL1OQS+4yRcXnguq7UY9UF+drG0f0QpC3aht1QgrJ04gvDp2BOk ymmlxCxUWQrFSqDThjv7WFCclamKTimCODKWvIG6sjQcJuLCg9CXRl+ZMvwobQqH Tv8cm8PMimqJppUodB3Ig5zP3ZkVcK8uFm5XqoUnasqDVLLJaRcCu+Qt4h9gZ2w6 Q3Q6K/zPZcKEIrwJfczWotSgG0g8dnuMUUYALWTbRrGjN0mgCss= =yHZZ -----END PGP SIGNATURE-----
Verify locally:
curl -fSLO https://blog.alipour.eu/sources/posts/matrix_setup.md curl -fSLO https://blog.alipour.eu/sources/posts/matrix_setup.md.asc gpg --verify matrix_setup.md.asc matrix_setup.md
