Migration This is my new public git source viewer. Source coming soon. Please contact with concerns/bugs. Contact me

llm_readme.md

d658fcc4ea9f3bb562fd00dce8f863934560c56c · 20.9 KB · 503 lines raw

1 # gsg — a modular git browser
2
3 cgit's weight, Hugo's composability, served dynamically. A **module**
4 computes one piece of a view; a **route** binds a renderer to a URL pattern.
5 Adding a feature means adding a module and calling it from a template —
6 `internal/core` never changes.
7
8 One dependency (`goldmark`, which itself has none) plus the `git` binary. Go 1.22+ (uses `net/http`
9 pattern wildcards, so there is no router dependency either).
10
11 ```
12 go build -o gsg .
13 ./gsg -root /srv/git -addr :8080 -name "rschanz.org"
14 ./gsg -root /srv/git -templates ./templates # re-parse the theme per request
15 ./gsg -root /srv/git -notices /etc/gsg/notices.yaml -admin-addr 127.0.0.1:8081
16 ```
17
18 ## The idea
19
20 ```go
21 type Module func(c *Ctx, args ...string) (any, error)
22 ```
23
24 Templates reach modules through one funcmap entry:
25
26 ```gotmpl
27 {{ with mod "meta" }}<h1>{{ .Title }}</h1>{{ end }}
28 {{ $d := mod "diff" }}
29 {{ with modOr "readme" }}{{ .HTML }}{{ end }}
30 ```
31
32 - **Lazy.** A template that calls `mod "authors"` is what *causes* authors to
33 be computed. Views that don't ask don't pay.
34 - **Memoized twice.** Once per request (`Ctx.cache`), and once process-wide
35 against the resolved commit SHA (`core.LRU`).
36 - **No manifest.** Dependencies are declared by use. There is no module list
37 to keep in sync with the templates.
38
39 `internal/modules/example_authors.go` is a live demonstration: 30 lines, an
40 `init()`, and one `{{ with mod "authors" }}` block. Nothing else moved.
41
42 ## Pages and endpoints are the same mechanism
43
44 A route is `{pattern, renderer}`. `core.View` renders a template,
45 `core.JSON` serializes an endpoint. Both take a `*Ctx`, both call modules.
46
47 ```go
48 {Name: "commit", Immutable: true,
49 Pattern: "GET /{repo}/commit/{rev}",
50 Render: core.View{Template: "commit.html"}},
51
52 {Name: "api-commit", Immutable: true,
53 Pattern: "GET /{repo}/api/commit/{rev}",
54 Render: core.JSON{Build: api.CommitEndpoint}},
55 ```
56
57 ### Endpoints use DTOs, not module values
58
59 `internal/api` defines explicit structs and never marshals a module's return
60 value. Go marshals every exported field by default, so serializing a module
61 type means the day someone adds `Body` to `gitx.Commit`, commit messages are
62 public. The allowlist is one readable file.
63
64 ```
65 GET /api/repos.json site index
66 GET /{repo}/api/repo.json slug, title, description, owner, section,
67 clone_url, default_branch, last_commit
68 GET /{repo}/api/commit/{sha} sha, short, authored, parents, and
69 file/line counts — no subject, no body
70 ```
71
72 ## Routes
73
74 | Route | URL |
75 |---|---|
76 | index | `/` |
77 | repo | `/{repo}/` — default branch, last update, latest commit, README |
78 | files | `/{repo}/tree/{ref}/{path}` |
79 | file | `/{repo}/blob/{ref}/{path}` |
80 | **raw bytes** | `/{repo}/raw/{ref}/{path}` |
81 | log | `/{repo}/log/{ref}` |
82 | refs | `/{repo}/refs/` |
83 | commit | `/{repo}/commit/{sha}` |
84 | **compare** | `/{repo}/compare/{base}..{head}` or `{base}...{head}` |
85 | **branch graph** | `/{repo}/graph/` |
86
87 ### Compare
88
89 `a..b` is the literal endpoint difference. `a...b` diffs from the merge
90 base, which is what you want when reviewing a branch — it excludes changes
91 that landed on the base since the branch forked. Verified against
92 `git merge-base` on a diverged fixture: `topic..main` lists `topic.txt` as
93 removed, `topic...main` does not.
94
95 ### Raw files
96
97 A repository can contain an `.html` file with a `<script>` tag. Serving that
98 as `text/html` on your own origin turns any push into stored XSS against the
99 whole site — which is why GitHub puts raw content on
100 `raw.githubusercontent.com`, a separate origin. Same-host serving is only
101 safe if the response cannot be interpreted as a document, so `/raw/` sends
102 `text/plain` (or `application/octet-stream` for binaries, plus
103 `Content-Disposition: attachment`), with `X-Content-Type-Options: nosniff`
104 and `Content-Security-Policy: default-src \'none\'; sandbox` as a backstop.
105
106 The exception is a validated raster image (PNG, JPEG, GIF, WebP, BMP, ICO,
107 AVIF): when the extension and the magic bytes agree, `/raw/` declares the
108 real `image/*` type inline so the blob page can preview it. An extension by
109 itself is not enough — an `.png` whose contents are HTML still downloads as
110 `application/octet-stream`. SVG is deliberately excluded: it is an XML
111 document that can execute script, so it stays `text/plain`.
112
113 If you would rather have real isolation, serve `/raw/` from a second
114 hostname and the CSP becomes belt-and-braces rather than the only line.
115
116 ### Images and structured config files
117
118 An image blob shows an inline preview with a `download` link to the same
119 `/raw/` URL; images over 10 MiB skip the preview and offer the download only.
120
121 JSON and YAML blobs get per-level indent guides, fold controls, key/value
122 search, and YAML anchor/alias links. The search box also accepts a value
123 path: typing `obj1.property3.items[4].val` resolves it against the parsed
124 document and highlights that line and, for a container, its subtree. A
125 query containing `.` or `[` is tried as a path first and falls back to
126 plain substring search when it does not resolve, so dotted text stays
127 searchable. Paths quote keys that need it (`obj1["a.b"]`) and attach indices
128 directly; SVG is never previewed, and files past 512 KiB skip the parsing
129 tooling entirely.
130
131 ### Branch graph
132
133 `modules/graph.go` lays out the DAG in lanes in one pass over
134 `git log --all --topo-order`, which guarantees a child is emitted before its
135 parents. `active` holds, per lane, the SHA that lane is waiting to draw; a
136 commit claims the lane expecting it (or a fresh one), hands that lane to its
137 first parent, and opens a new lane per additional parent — which is what
138 makes a merge fork visually.
139
140 The module emits the per-row SVG because the geometry *is* the data, but
141 every stroke refers to a CSS custom property (`--lane-0` … `--lane-5`), so
142 the theme still owns the palette. Row height in the SVG must stay in sync
143 with `table.graph td { height }` in the stylesheet.
144
145 ## Caching, which is the whole reason this works dynamically
146
147 Every module result is a pure function of `(repo, commit SHA, path, args)`.
148 Refs move; SHAs do not. So the process-wide cache is keyed on the resolved
149 SHA and needs **no invalidation logic at all** — a stale entry is impossible
150 by construction.
151
152 `Ctx.Vars` are part of the cache key, not just the commit. They were not
153 originally, and the bug was exactly the kind this cache should be immune to:
154 `/compare/a..b` and `/compare/a...b` resolve to the same head commit, so the
155 second request was served the first one\'s diff. Anything that changes what a
156 module computes has to be in the key.
157
158 ETags are derived from the resolved SHA plus a digest of every template
159 file, *not* from the response body. A conditional request costs one ref
160 lookup instead of a render, and restyling the theme invalidates every ETag
161 at once (without the template digest, a restyled page keeps serving 304s).
162
163 Two URL classes, two policies:
164
165 | URL shape | Example | Cache-Control |
166 |---|---|---|
167 | SHA-addressed | `/demo/commit/64b90fff…` | `max-age=31536000, immutable` |
168 | ref-addressed | `/demo/tree/main` | `no-cache` (revalidate via ETag) |
169
170 ## Ref-vs-path ambiguity
171
172 `/{repo}/tree/{rest...}` fuses them, and `gitx.SplitRefPath` separates them
173 by **longest matching prefix against the repo's actual ref list**, so
174 `feature/x/src/main.go` resolves correctly. Splitting at the first `/` is
175 wrong because branch names contain slashes; cgit dodges this with `?h=`
176 query params, GitHub does what this does.
177
178 ## Serving is an attack surface; static generation was not
179
180 - **Ref input never reaches `git rev-parse`.** rev-parse accepts an
181 expression language — `:/regex` searches history, `@{...}` reads the
182 reflog — and git subcommands treat a leading `-` as an option.
183 `gitx.Resolve` accepts either a hex SHA that already exists or an exact
184 match against the repo's own ref list. Verified: `--upload-pack=…`,
185 `:/secret`, `HEAD@{0}` and `-n` all return 404 with no side effects.
186 - **Every git call is `exec.CommandContext`** with a per-request deadline
187 (`-timeout`, default 10s) and a scrubbed environment
188 (`GIT_CONFIG_NOSYSTEM=1`, no `HOME`, no terminal prompt).
189 - **Output is capped** at `MaxGit` (8 MiB) per command; a diff that exceeds
190 it renders truncated rather than eating the heap.
191
192 ## Layout
193
194 ```
195 main.go flags, repo discovery, embedded assets
196 internal/gitx/ the ONLY package that knows git exists
197 internal/core/core.go Ctx, RepoInfo, module registry, Store
198 internal/core/route.go Route, View, JSON, route request parsing
199 internal/core/server.go request -> Ctx, ETags, cache policy
200 internal/core/cache.go LRU keyed on commit SHAs
201 internal/core/funcs.go template funcmap (mod, modOr, dict, date, bytes)
202 internal/access/ visibility and share-grant policy (imports no core)
203 internal/authn/ login/logout endpoints
204 internal/httpx/ shared admin/share HTTP security helpers
205 internal/modules/ meta refs log tree readme repos commit diff blob notices
206 internal/notices/ notices YAML store + admin UI (imports no core)
207 internal/api/ JSON DTOs — the serialization allowlist
208 internal/site/routes.go the route table (the file you edit most)
209 internal/site/request.go route-specific path parsers (compare ranges)
210 templates/ layouts/ + views/
211 ```
212
213 `gitx` shells out to plumbing, one process per query, `git log` in a single
214 pass. Swapping in go-git means rewriting that one package.
215
216 ## Sections
217
218 `gsg.section` groups repositories on the index. The `sections` module builds
219 on `repos` rather than re-walking the repo set, so the index pays for its log
220 lookups once. When no repo sets a section, it returns a single unnamed group
221 and the template renders a flat list — no config, no headings.
222
223 ```sh
224 git --git-dir=/srv/git/demo.git config gsg.section "Infrastructure"
225 ```
226
227 ## Notices
228
229 Operator-authored banners at the top of every page, site-wide and per
230 repository. They live in one YAML file that is watched for changes, so
231 editing it never needs a restart or a rebuild:
232
233 ```yaml
234 global:
235 - type: critical # critical | warning | info | fix
236 title: Maintenance window
237 body: |
238 Saturday 02:00-04:00 UTC.
239 The site is read-only during the window.
240 link: https://status.example.org
241 link_text: status page
242 repos:
243 demo:
244 - type: warning
245 title: Migrating to a new remote
246 ```
247
248 ```sh
249 ./gsg -root /srv/git -notices /etc/gsg/notices.yaml
250 ```
251
252 Global notices render above every repository's; on the site index only the
253 global list appears. The file is re-read on write (atomic-rename saves from
254 an editor included); a parse error keeps the last good content rather than
255 blanking the banners.
256
257 ### Editing from the web
258
259 `-admin-addr` starts a second listener with a small HTML form to add, edit,
260 and delete notices. It is deliberately separate from the read-only server:
261 the admin routes are not reachable on the public port even if the reverse
262 proxy is misconfigured.
263
264 ```sh
265 ./gsg -root /srv/git -notices /etc/gsg/notices.yaml -admin-addr 127.0.0.1:8081
266 ```
267
268 Protect it with Caddy `forward_auth` and proxy it to the admin listener. gsg
269 trusts the network in front of it, so the admin port must not be published
270 directly. In Compose, put 8081 on the internal network only and pass
271 `-admin-allow-remote`, because a container address is not loopback. Note
272 that gsg performs no authentication itself -- the `forward_auth` block below
273 is the only thing between the internet and the admin form.
274
275 ```caddyfile
276 example.com {
277 # Admin: authenticate first, then proxy to the separate admin listener.
278 # handle, not handle_path: the admin mux matches the full /admin/...
279 # path, so stripping the prefix would 404 every request.
280 handle /admin/* {
281 forward_auth authentik:9000 {
282 uri /outpost.goauthentik.io/auth/caddy
283 copy_headers X-Authentik-Username
284 }
285 reverse_proxy gsg:8081
286 }
287
288 # Public, read-only.
289 handle {
290 reverse_proxy gsg:8080
291 }
292 }
293 ```
294
295 The `X-Authentik-Username` header is used only to attribute the mutation in
296 the log; it is never consulted for authorization. By default Authentik
297 decides who may reach the handler at all -- anyone forward_auth lets through
298 `/admin/*` can edit notices.
299
300 Mutations are POST-only, CSRF-protected (double-submit cookie plus
301 `Origin`/`Sec-Fetch-Site`), and logged with the `X-Authentik-Username`
302 header, which is used for audit only, never for authorization.
303
304 ### Gating notices editing to a group
305
306 `/admin/*` is also where [Login](#login) mints the `gsg_user` cookie, so
307 forward_auth for the whole prefix is usually left open to everyone you want
308 able to log in (for share links, hidden-repo visibility, etc). To let more
309 people log in without also handing them notices editing, add `-admin-group`:
310
311 ```sh
312 ./gsg -root /srv/git -notices /etc/gsg/notices.yaml \
313 -admin-addr 127.0.0.1:8081 -admin-group gsg-admins
314 ```
315
316 This checks the `X-Authentik-Groups` header (override with
317 `-admin-group-header`) for `gsg-admins` among its `|`-delimited entries,
318 Authentik's own format for that header, and answers `403` to any request
319 under `/admin/notices` from a caller that isn't a member -- before the
320 handler does anything else, including setting the CSRF cookie. Add the group
321 header to the same `forward_auth` block used above:
322
323 ```caddyfile
324 forward_auth authentik:9000 {
325 uri /outpost.goauthentik.io/auth/caddy
326 copy_headers X-Authentik-Username X-Authentik-Groups
327 }
328 ```
329
330 Leaving `-admin-group` unset keeps today's behavior: anyone forward_auth
331 admits to `/admin/*` can edit notices.
332
333 **Immutable commit pages carry no notices.** `/{repo}/commit/{rev}` is
334 served `Cache-Control: public, max-age=31536000, immutable`, so a banner
335 rendered into it would stay frozen in the reader's browser. Notices are
336 suppressed there by design; every other page picks them up on the next
337 request.
338
339 ## Login
340
341 Logging in reveals hidden repos whose `gsg.visible-to` list includes your
342 username; everyone else, logged in or not, sees exactly what an anonymous
343 visitor sees. It reuses the same Authentik `forward_auth` setup as the
344 notices admin UI above, and needs nothing extra beyond `-admin-addr`
345 (notices editing is not required — `-admin-addr` alone is enough to enable
346 login).
347
348 ```sh
349 ./gsg -root /srv/git -admin-addr 127.0.0.1:8081
350 ```
351
352 ```caddyfile
353 example.com {
354 # Login: same trust boundary as /admin/*, one hop mints the cookie
355 # and redirects back. This must be the exact path, not /auth/*: gsg
356 # also serves /auth/logout, but on the *public* listener, since
357 # logging out only clears gsg's own cookie and needs no Authentik
358 # round trip at all. Routing /auth/* here would send logout to the
359 # admin listener too, where no such handler exists, and gate it
360 # behind an Authentik check it doesn't need.
361 handle /auth/login {
362 forward_auth authentik:9000 {
363 uri /outpost.goauthentik.io/auth/caddy
364 copy_headers X-Authentik-Username
365 }
366 reverse_proxy gsg:8081
367 }
368
369 # Public, read-only (also handles /auth/logout, /share/new, /share/view/*).
370 handle {
371 reverse_proxy gsg:8080
372 }
373 }
374 ```
375
376 Visiting `/auth/login` (through Caddy, so `X-Authentik-Username` is set and
377 trustworthy by the time gsg sees the request) mints a signed, stateless
378 `gsg_user` cookie and redirects back to `/`. The cookie is verified on the
379 public listener with an in-memory key — nothing is written to disk, and a
380 process restart signs everyone out. `/auth/logout` clears it and needs no
381 round trip through Authentik, since it only has to forget gsg's own cookie.
382
383 **There is no "log in" link anywhere in the UI.** `/auth/login` is not
384 discoverable from the site; people who need it are told the URL out of
385 band. Once logged in, the nav bar shows who you are and a log-out link.
386
387 ## Share links
388
389 Once logged in, `/share/new` lets you pick from your own hidden repos and
390 mint a link that grants view access to exactly those repos to anyone who
391 has it — no login, no Authentik involvement. It's entirely stateless: the
392 link is a signed, self-contained token, and gsg keeps no record of what's
393 been shared. An optional password can gate it.
394
395 ```
396 git --git-dir=/srv/git/demo.git config gsg.share-max-age "720h"
397 ```
398
399 `gsg.share-max-age` (Go duration syntax, default 30 days if unset) is both
400 the ceiling no link's expiry may exceed and the default when the minting
401 form doesn't specify one — you can always ask for less, never more. Sharing
402 several repos at once uses the smallest ceiling among them.
403
404 **Passwords are never stored, not even hashed.** The password is folded
405 directly into the link's signature instead of being embedded in it: nobody
406 holding just the URL has anything to brute-force offline, and checking a
407 guess requires asking the server, which rate-limits attempts. This is a
408 soft gate appropriate for a small self-hosted tool, not a cryptographic
409 secret protecting high-value data.
410
411 **Revocation.** A stateless token can't be individually revoked — there's
412 no list of what's been issued. The one lever:
413
414 ```
415 git --git-dir=/srv/git/demo.git config gsg.share-revoked-after "$(date +%s)"
416 ```
417
418 Any share grant for that repo issued at or before this timestamp stops
419 working immediately, including in a browser that already redeemed the
420 link — not just future redemptions. Two things worth knowing before you
421 rely on this:
422
423 - There's no per-recipient state, so revoking means invalidating *every*
424 outstanding share of that repo, then re-sharing with whoever should still
425 have it.
426 - Removing a username from `gsg.visible-to` does **not** revoke a share
427 link that user already minted for that repo. The two mechanisms are
428 independent; only `gsg.share-revoked-after` invalidates outstanding
429 shares.
430
431 ## README rendering
432
433 `modules/markdown.go` uses goldmark with GFM, Typographer, and automatic
434 heading IDs — Hugo\'s own goldmark defaults, so a README renders identically
435 to the same file on your blog.
436
437 `WithUnsafe` is deliberately not set, which is also Hugo\'s default. Anyone
438 who can push can write a README, so raw HTML in markdown is untrusted input;
439 goldmark drops it. A `<script>` committed to a README comes out as
440 `<!-- raw HTML omitted -->`.
441
442 Relative links are rewritten against the file\'s own directory: links become
443 the blob view, images become `/raw/` (an `<img>` needs bytes, not an HTML
444 page). Absolute URLs, site-absolute paths, fragments and `mailto:` are left
445 alone, and anything resolving above the repo root is not rewritten.
446
447 ## Per-repository config
448
449 Read from each repo's own git config, cgit-style, with `gitweb.*` and the
450 legacy `description` file as fallbacks, so an existing cgit tree works as-is:
451
452 ```sh
453 git --git-dir=/srv/git/demo.git config gsg.title "Demo Project"
454 git --git-dir=/srv/git/demo.git config gsg.description "A demo repository"
455 git --git-dir=/srv/git/demo.git config gsg.clone "git://rschanz.org/demo"
456 git --git-dir=/srv/git/demo.git config gsg.hide true
457 git --git-dir=/srv/git/demo.git config gsg.visible-to "alice,bob"
458 git --git-dir=/srv/git/demo.git config gsg.share-max-age "720h"
459 git --git-dir=/srv/git/demo.git config gsg.share-revoked-after "$(date +%s)"
460 ```
461
462 Add a key by adding a field to `core.RepoInfo` and a line in `LoadRepoInfo`.
463
464 A repo's slug is its directory name (minus `.git`), and slugs share the
465 public listener's URL namespace with a few reserved top-level paths:
466 `static`, `auth`, and `share`. A repo named e.g. `share.git` would collide
467 with the share-link endpoints below and is not supported.
468
469 `gsg.visible-to` is a comma-separated allowlist of usernames (matched
470 case-insensitively against a logged-in session, see "Login" below) who may
471 see this repo even though `gsg.hide` is set. It's a separate key from
472 `gsg.owner` on purpose: `Owner` is a free-text display field that falls
473 back to `gitweb.owner` for cgit-migrated repos, so treating it as an access
474 list could silently grant visibility nobody intended.
475
476 ## Theme
477
478 Monospace-first: every value on these pages is an identifier, hash, path or
479 count, so the interface type is the same face as the code, and the sans face
480 appears only inside rendered README prose. A cool slate ramp, one indigo
481 accent, and a six-colour lane ramp shared by the commit graph and its branch
482 chips. Light and dark both defined via `prefers-color-scheme`.
483
484 Diffs mark the gutter rather than flooding whole rows with green and red —
485 the signal stays readable when a hunk is mostly additions.
486
487 No webfonts; system stacks only, so there is no third-party request. All
488 colour lives in `:root` custom properties in `static/style.css`.
489
490 ## Known gaps
491
492 - **No syntax highlighting.** `blob` returns numbered plain lines; Chroma
493 drops into the same module.
494 - **No pagination on `log` or `graph`.** They cap at 200 and 150 commits.
495 - **The graph loads all refs into memory** before laying out. Fine to a few
496 thousand commits; beyond that it needs windowing.
497 - **The LRU is bounded by entry count, not bytes.** A giant diff and a ref
498 list cost the same slot. The `Store` interface is the only thing a
499 size-aware replacement has to satisfy.
500 - **Repo discovery is one level deep** and happens once at startup; a repo
501 created later needs a restart.
502 - **Module errors surface at render time**, not load time. Add a CI pass
503 that renders every view against a fixture repo.