Commit graph

18 commits

Author SHA1 Message Date
Derick Phan
1bc695f508
Harden network and privacy defaults; fix several bugs
Security:
- Bind HTTP gateway to 127.0.0.1 by default; add --bind for LAN opt-in
- Restrict Reticulum mesh surface to GET /api/sites only (CSRF cannot
  authenticate mesh callers, so gate by whitelist)
- Cap request body size at 16 MiB to prevent memory DoS
- Redact /bookmark query strings from request logs so the bookmark token
  and URLs do not land in stdout / docker / journal logs
- Tighten FTS5 sanitizer: strip colon, drop AND/OR/NOT/NEAR operator words
- Expand .dockerignore; document trust model in README

Features:
- Add sharing mode toggle (share everything except private vs share only
  public-tagged) with /share/preview so users can see what subscribers
  would receive before enabling sharing

Bugs:
- handle_export() crashed on every call (missing query kwarg)
- Dead float16 decompression branch in embeddings.py silently corrupted
  the HNSW index when compress_embeddings was on
- GATEWAY_PORT staleness: --port and find_available_port had no effect
  on the actual bind
- semantic_search default mismatched between db.py ("1") and the rest of
  the app ("0"), causing embeddings to be generated when the UI said off
- Connection pool returned connections with uncommitted transactions to
  the next consumer
- Gateway POST body decode 502'd on non-UTF-8 input
- ensure_rns_config clobbered user-edited ~/.reticulum/config; now only
  rewrites files it authored (sentinel-tagged)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 15:37:45 -07:00
Derick Phan
ce50150363
Add LoRa support with background sync and settings UI
- Progressive retry in rns_client.py: fast timeout (15s) then slow (60s+)
  for LoRa/multi-hop links, with automatic fallback
- Background sync threads so subscriptions page returns immediately
  with syncing/error status indicators per subscription
- LoRa RNode configuration in settings page with serial port and
  expandable advanced radio settings (frequency, bandwidth, etc.)
- Internet transport now toggleable alongside LoRa — users can
  enable one, the other, or both
- Reticulum config auto-generated from settings on startup

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-22 08:47:09 -07:00
lichenblankie
c844e2c81e Disabled semantic search and reranker by default
Some checks are pending
/ build (push) Waiting to run
/ release (push) Blocked by required conditions
2026-04-08 05:21:08 +00:00
Test User
57a79e5e8e Add PyInstaller builds, AGPLv3 license, transport node selection, and rmap.world link
- Add pyinstaller.spec and GitHub/Forgejo CI workflows for cross-platform builds
- Add AGPLv3 license
- Move data storage to ~/.tinyweb/
- Add --version and --port CLI flags
- Add transport node selection in /style (smart regeneration preserves Reticulum config)
- Add discover more nodes link to rmap.world
2026-04-08 04:36:28 +00:00
Derick Phan
c959ee98ae
Make semantic search and reranking optional, use site meta descriptions for snippets
- Add semantic_search setting to toggle AI-powered search on/off
- Skip embedding generation, hybrid search, and model preloading when disabled
- Use site owner's meta description as snippet instead of heuristic extraction
- Remove _generate_summary() and snippet() - no more generated snippets
- Show reranker/reindex controls grayed out when semantic search is off
- AI dependencies (onnxruntime, hnswlib, etc.) are now fully optional

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-28 20:58:04 -07:00
Derick Phan
299735f816
Add junimo theme and increase browse page size to 50
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 10:59:37 -07:00
Derick Phan
395fc17092
Add hybrid semantic search with optional cross-encoder reranking
Implements a three-stage search pipeline:
1. BM25 keyword search via FTS5 with column weights
2. Semantic search via Snowflake arctic-embed-s bi-encoder + HNSW index
3. Optional cross-encoder reranking (on by default, toggleable in settings)

Top 20 results are reranked for precision, next 10 appended from RRF
for coverage, giving 30 total results across 3 pages.

- New embeddings.py with ONNX Runtime inference, text chunking, HNSW
  index management, RRF fusion, and cross-encoder reranking
- Meta description extraction for authentic page snippets with centroid
  extractive fallback
- Stopword filtering in FTS5 queries to avoid overly strict matching
- /reindex page for batch embedding of existing pages
- Semantic embedding of remote pages during subscription sync
- ~125MB dependency footprint (onnxruntime, tokenizers, hnswlib, numpy)
- Models: 34MB bi-encoder + 22MB cross-encoder (downloaded on first use)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 03:24:41 -07:00
Derick Phan
ffdfb821c8
Set share_instance = No for reliable mesh announces
With share_instance = Yes, announces weren't being sent over TCP
in Docker environments. Setting it to No ensures each TinyWeb
instance manages its own Reticulum interfaces directly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-26 21:39:37 -07:00
Derick Phan
fe0e15edc4
Add delay before announce to ensure TCP interface is ready
The announce was firing before the TCP transport connection was fully
established, causing Docker instances to never announce over the mesh.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-26 21:32:11 -07:00
Derick Phan
2adef21ec6
Add default internet transport node for zero-config mesh connectivity
New TinyWeb instances now auto-connect to reticulum.derickphan.com:4242
so users get internet mesh connectivity out of the box without any
manual Reticulum configuration. Env var overrides still supported.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-26 20:13:35 -07:00
Derick Phan
5f8863ce77
Add entrypoint script for configurable Reticulum networking in Docker
Replaces static CMD with an entrypoint that generates RNS config from
environment variables (RNS_TCP_HOST/PORT), enabling TCP transport for
environments without LAN auto-discovery (e.g. Docker on macOS).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-26 18:44:26 -07:00
Derick Phan
c10aa7955c
Fix SSRF redirect bypass, identity permissions, error leakage, and DB connection leaks
- SSRF: disable automatic redirects, manually follow up to 5 hops with
  IP re-validation at each step to prevent redirect-to-localhost bypass
- Identity file: enforce 0600 permissions on tinyweb_identity at load
  and creation to prevent other users from reading the private key
- Error messages: replace raw exception strings with generic messages
  to avoid leaking internal paths/hostnames to the UI
- DB connections: wrap all get_db() usage in try/finally to guarantee
  close() even when handlers throw mid-operation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-26 11:18:47 -07:00
Derick Phan
20de39fc46
Bind HTTP server to 0.0.0.0 for remote access
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 23:38:15 -07:00
Derick Phan
175582914d
Add /about landing page with slow web philosophy
Shows instance stats, destination hash for subscribing, and explains
the slow web movement and how TinyWeb works. Destination hash is
stored in settings on startup so the about page can display it.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 23:21:02 -07:00
Derick Phan
4e4cc69e0f
Single-command startup and fix bookmarklet
app.py now auto-starts the gateway HTTP server in a daemon thread,
so users only need `python app.py` to get everything running. The
gateway calls dispatch_request directly when co-located (local mode)
instead of trying to establish an RNS link to itself. Bookmarklet
hardcoded to localhost:8080. gateway.py still works standalone for
connecting to remote instances.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 23:01:54 -07:00
Derick Phan
f609f867ef
Migrate TinyWeb to Reticulum mesh network
Replace HTTP server with Reticulum-native architecture. The server
now speaks only Reticulum, with a client-side gateway providing
browser access by translating HTTP to/from RNS requests.

- Extract db layer (db.py), templates (templates.py), handlers (handlers.py)
- app.py is now the RNS server with persistent identity and destination
- gateway.py bridges HTTP on localhost:8080 to RNS link requests
- Add rns dependency, add .gitignore

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 22:18:24 -07:00
Derick Phan
c30fccb7a1
Bind to 0.0.0.0 and use dynamic Host header for bookmarklet
Makes the server accessible from other devices on the network
instead of only localhost. The bookmarklet now uses the Host header
from the request so it works regardless of how the server is accessed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-24 20:45:14 -07:00
blankie
3bd8601c63 init 2026-03-24 20:35:20 -07:00