Migration
This is my new public git source viewer. Source coming soon. Please contact with concerns/bugs.
Contact me
README.md
38fa4e6cd9bb9dbf0ae23ef02e50b1c75cd84ebb
· 14.8 KB · 362 lines
raw
| 1 | # gsg — a modular git browser |
| 2 | |
| 3 | cgit's weight, Hugo's composability, served dynamically. A **module** |
| 4 | computes one piece of a view; a **route** binds a renderer to a URL pattern. |
| 5 | Adding a feature means adding a module and calling it from a template — |
| 6 | `internal/core` never changes. |
| 7 | |
| 8 | One dependency (`goldmark`, which itself has none) plus the `git` binary. Go 1.22+ (uses `net/http` |
| 9 | pattern wildcards, so there is no router dependency either). |
| 10 | |
| 11 | ``` |
| 12 | go build -o gsg . |
| 13 | ./gsg -root /srv/git -addr :8080 -name "rschanz.org" |
| 14 | ./gsg -root /srv/git -templates ./templates # re-parse the theme per request |
| 15 | ./gsg -root /srv/git -notices /etc/gsg/notices.yaml -admin-addr 127.0.0.1:8081 |
| 16 | ``` |
| 17 | |
| 18 | ## The idea |
| 19 | |
| 20 | ```go |
| 21 | type Module func(c *Ctx, args ...string) (any, error) |
| 22 | ``` |
| 23 | |
| 24 | Templates reach modules through one funcmap entry: |
| 25 | |
| 26 | ```gotmpl |
| 27 | {{ with mod "meta" }}<h1>{{ .Title }}</h1>{{ end }} |
| 28 | {{ $d := mod "diff" }} |
| 29 | {{ with modOr "readme" }}{{ .HTML }}{{ end }} |
| 30 | ``` |
| 31 | |
| 32 | - **Lazy.** A template that calls `mod "authors"` is what *causes* authors to |
| 33 | be computed. Views that don't ask don't pay. |
| 34 | - **Memoized twice.** Once per request (`Ctx.cache`), and once process-wide |
| 35 | against the resolved commit SHA (`core.LRU`). |
| 36 | - **No manifest.** Dependencies are declared by use. There is no module list |
| 37 | to keep in sync with the templates. |
| 38 | |
| 39 | `internal/modules/example_authors.go` is a live demonstration: 30 lines, an |
| 40 | `init()`, and one `{{ with mod "authors" }}` block. Nothing else moved. |
| 41 | |
| 42 | ## Pages and endpoints are the same mechanism |
| 43 | |
| 44 | A route is `{pattern, renderer}`. `core.View` renders a template, |
| 45 | `core.JSON` serializes an endpoint. Both take a `*Ctx`, both call modules. |
| 46 | |
| 47 | ```go |
| 48 | {Name: "commit", Immutable: true, |
| 49 | Pattern: "GET /{repo}/commit/{rev}", |
| 50 | Render: core.View{Template: "commit.html"}}, |
| 51 | |
| 52 | {Name: "api-commit", Immutable: true, |
| 53 | Pattern: "GET /{repo}/api/commit/{rev}", |
| 54 | Render: core.JSON{Build: api.CommitEndpoint}}, |
| 55 | ``` |
| 56 | |
| 57 | ### Endpoints use DTOs, not module values |
| 58 | |
| 59 | `internal/api` defines explicit structs and never marshals a module's return |
| 60 | value. Go marshals every exported field by default, so serializing a module |
| 61 | type means the day someone adds `Body` to `gitx.Commit`, commit messages are |
| 62 | public. The allowlist is one readable file. |
| 63 | |
| 64 | ``` |
| 65 | GET /api/repos.json site index |
| 66 | GET /{repo}/api/repo.json slug, title, description, owner, section, |
| 67 | clone_url, default_branch, last_commit |
| 68 | GET /{repo}/api/commit/{sha} sha, short, authored, parents, and |
| 69 | file/line counts — no subject, no body |
| 70 | ``` |
| 71 | |
| 72 | ## Routes |
| 73 | |
| 74 | | Route | URL | |
| 75 | |---|---| |
| 76 | | index | `/` | |
| 77 | | repo | `/{repo}/` — default branch, last update, latest commit, README | |
| 78 | | files | `/{repo}/tree/{ref}/{path}` | |
| 79 | | file | `/{repo}/blob/{ref}/{path}` | |
| 80 | | **raw bytes** | `/{repo}/raw/{ref}/{path}` | |
| 81 | | log | `/{repo}/log/{ref}` | |
| 82 | | refs | `/{repo}/refs/` | |
| 83 | | commit | `/{repo}/commit/{sha}` | |
| 84 | | **compare** | `/{repo}/compare/{base}..{head}` or `{base}...{head}` | |
| 85 | | **branch graph** | `/{repo}/graph/` | |
| 86 | |
| 87 | ### Compare |
| 88 | |
| 89 | `a..b` is the literal endpoint difference. `a...b` diffs from the merge |
| 90 | base, which is what you want when reviewing a branch — it excludes changes |
| 91 | that landed on the base since the branch forked. Verified against |
| 92 | `git merge-base` on a diverged fixture: `topic..main` lists `topic.txt` as |
| 93 | removed, `topic...main` does not. |
| 94 | |
| 95 | ### Raw files |
| 96 | |
| 97 | A repository can contain an `.html` file with a `<script>` tag. Serving that |
| 98 | as `text/html` on your own origin turns any push into stored XSS against the |
| 99 | whole site — which is why GitHub puts raw content on |
| 100 | `raw.githubusercontent.com`, a separate origin. Same-host serving is only |
| 101 | safe if the response cannot be interpreted as a document, so `/raw/` sends |
| 102 | `text/plain` (or `application/octet-stream` for binaries, plus |
| 103 | `Content-Disposition: attachment`), with `X-Content-Type-Options: nosniff` |
| 104 | and `Content-Security-Policy: default-src \'none\'; sandbox` as a backstop. |
| 105 | |
| 106 | The exception is a validated raster image (PNG, JPEG, GIF, WebP, BMP, ICO, |
| 107 | AVIF): when the extension and the magic bytes agree, `/raw/` declares the |
| 108 | real `image/*` type inline so the blob page can preview it. An extension by |
| 109 | itself is not enough — an `.png` whose contents are HTML still downloads as |
| 110 | `application/octet-stream`. SVG is deliberately excluded: it is an XML |
| 111 | document that can execute script, so it stays `text/plain`. |
| 112 | |
| 113 | If you would rather have real isolation, serve `/raw/` from a second |
| 114 | hostname and the CSP becomes belt-and-braces rather than the only line. |
| 115 | |
| 116 | ### Images and structured config files |
| 117 | |
| 118 | An image blob shows an inline preview with a `download` link to the same |
| 119 | `/raw/` URL; images over 2 MiB skip the preview and offer the download only. |
| 120 | |
| 121 | JSON and YAML blobs get per-level indent guides, fold controls, key/value |
| 122 | search, and YAML anchor/alias links. The search box also accepts a value |
| 123 | path: typing `obj1.property3.items[4].val` resolves it against the parsed |
| 124 | document and highlights that line and, for a container, its subtree. A |
| 125 | query containing `.` or `[` is tried as a path first and falls back to |
| 126 | plain substring search when it does not resolve, so dotted text stays |
| 127 | searchable. Paths quote keys that need it (`obj1["a.b"]`) and attach indices |
| 128 | directly; SVG is never previewed, and files past 512 KiB skip the parsing |
| 129 | tooling entirely. |
| 130 | |
| 131 | ### Branch graph |
| 132 | |
| 133 | `modules/graph.go` lays out the DAG in lanes in one pass over |
| 134 | `git log --all --topo-order`, which guarantees a child is emitted before its |
| 135 | parents. `active` holds, per lane, the SHA that lane is waiting to draw; a |
| 136 | commit claims the lane expecting it (or a fresh one), hands that lane to its |
| 137 | first parent, and opens a new lane per additional parent — which is what |
| 138 | makes a merge fork visually. |
| 139 | |
| 140 | The module emits the per-row SVG because the geometry *is* the data, but |
| 141 | every stroke refers to a CSS custom property (`--lane-0` … `--lane-5`), so |
| 142 | the theme still owns the palette. Row height in the SVG must stay in sync |
| 143 | with `table.graph td { height }` in the stylesheet. |
| 144 | |
| 145 | ## Caching, which is the whole reason this works dynamically |
| 146 | |
| 147 | Every module result is a pure function of `(repo, commit SHA, path, args)`. |
| 148 | Refs move; SHAs do not. So the process-wide cache is keyed on the resolved |
| 149 | SHA and needs **no invalidation logic at all** — a stale entry is impossible |
| 150 | by construction. |
| 151 | |
| 152 | `Ctx.Vars` are part of the cache key, not just the commit. They were not |
| 153 | originally, and the bug was exactly the kind this cache should be immune to: |
| 154 | `/compare/a..b` and `/compare/a...b` resolve to the same head commit, so the |
| 155 | second request was served the first one\'s diff. Anything that changes what a |
| 156 | module computes has to be in the key. |
| 157 | |
| 158 | ETags are derived from the resolved SHA plus a digest of every template |
| 159 | file, *not* from the response body. A conditional request costs one ref |
| 160 | lookup instead of a render, and restyling the theme invalidates every ETag |
| 161 | at once (without the template digest, a restyled page keeps serving 304s). |
| 162 | |
| 163 | Two URL classes, two policies: |
| 164 | |
| 165 | | URL shape | Example | Cache-Control | |
| 166 | |---|---|---| |
| 167 | | SHA-addressed | `/demo/commit/64b90fff…` | `max-age=31536000, immutable` | |
| 168 | | ref-addressed | `/demo/tree/main` | `no-cache` (revalidate via ETag) | |
| 169 | |
| 170 | ## Ref-vs-path ambiguity |
| 171 | |
| 172 | `/{repo}/tree/{rest...}` fuses them, and `gitx.SplitRefPath` separates them |
| 173 | by **longest matching prefix against the repo's actual ref list**, so |
| 174 | `feature/x/src/main.go` resolves correctly. Splitting at the first `/` is |
| 175 | wrong because branch names contain slashes; cgit dodges this with `?h=` |
| 176 | query params, GitHub does what this does. |
| 177 | |
| 178 | ## Serving is an attack surface; static generation was not |
| 179 | |
| 180 | - **Ref input never reaches `git rev-parse`.** rev-parse accepts an |
| 181 | expression language — `:/regex` searches history, `@{...}` reads the |
| 182 | reflog — and git subcommands treat a leading `-` as an option. |
| 183 | `gitx.Resolve` accepts either a hex SHA that already exists or an exact |
| 184 | match against the repo's own ref list. Verified: `--upload-pack=…`, |
| 185 | `:/secret`, `HEAD@{0}` and `-n` all return 404 with no side effects. |
| 186 | - **Every git call is `exec.CommandContext`** with a per-request deadline |
| 187 | (`-timeout`, default 10s) and a scrubbed environment |
| 188 | (`GIT_CONFIG_NOSYSTEM=1`, no `HOME`, no terminal prompt). |
| 189 | - **Output is capped** at `MaxGit` (8 MiB) per command; a diff that exceeds |
| 190 | it renders truncated rather than eating the heap. |
| 191 | |
| 192 | ## Layout |
| 193 | |
| 194 | ``` |
| 195 | main.go flags, repo discovery, embedded assets |
| 196 | internal/gitx/ the ONLY package that knows git exists |
| 197 | internal/core/core.go Ctx, RepoInfo, module registry, Store |
| 198 | internal/core/route.go Route, View, JSON |
| 199 | internal/core/server.go request -> Ctx, ETags, cache policy |
| 200 | internal/core/cache.go LRU keyed on commit SHAs |
| 201 | internal/core/funcs.go template funcmap (mod, modOr, dict, date, bytes) |
| 202 | internal/modules/ meta refs log tree readme repos commit diff blob notices |
| 203 | internal/notices/ notices YAML store + admin UI (imports no core) |
| 204 | internal/api/ JSON DTOs — the serialization allowlist |
| 205 | internal/site/routes.go the route table (the file you edit most) |
| 206 | templates/ layouts/ + views/ |
| 207 | ``` |
| 208 | |
| 209 | `gitx` shells out to plumbing, one process per query, `git log` in a single |
| 210 | pass. Swapping in go-git means rewriting that one package. |
| 211 | |
| 212 | ## Sections |
| 213 | |
| 214 | `gsg.section` groups repositories on the index. The `sections` module builds |
| 215 | on `repos` rather than re-walking the repo set, so the index pays for its log |
| 216 | lookups once. When no repo sets a section, it returns a single unnamed group |
| 217 | and the template renders a flat list — no config, no headings. |
| 218 | |
| 219 | ```sh |
| 220 | git --git-dir=/srv/git/demo.git config gsg.section "Infrastructure" |
| 221 | ``` |
| 222 | |
| 223 | ## Notices |
| 224 | |
| 225 | Operator-authored banners at the top of every page, site-wide and per |
| 226 | repository. They live in one YAML file that is watched for changes, so |
| 227 | editing it never needs a restart or a rebuild: |
| 228 | |
| 229 | ```yaml |
| 230 | global: |
| 231 | - type: critical # critical | warning | info | fix |
| 232 | title: Maintenance window |
| 233 | body: | |
| 234 | Saturday 02:00-04:00 UTC. |
| 235 | The site is read-only during the window. |
| 236 | link: https://status.example.org |
| 237 | link_text: status page |
| 238 | repos: |
| 239 | demo: |
| 240 | - type: warning |
| 241 | title: Migrating to a new remote |
| 242 | ``` |
| 243 | |
| 244 | ```sh |
| 245 | ./gsg -root /srv/git -notices /etc/gsg/notices.yaml |
| 246 | ``` |
| 247 | |
| 248 | Global notices render above every repository's; on the site index only the |
| 249 | global list appears. The file is re-read on write (atomic-rename saves from |
| 250 | an editor included); a parse error keeps the last good content rather than |
| 251 | blanking the banners. |
| 252 | |
| 253 | ### Editing from the web |
| 254 | |
| 255 | `-admin-addr` starts a second listener with a small HTML form to add, edit, |
| 256 | and delete notices. It is deliberately separate from the read-only server: |
| 257 | the admin routes are not reachable on the public port even if the reverse |
| 258 | proxy is misconfigured. |
| 259 | |
| 260 | ```sh |
| 261 | ./gsg -root /srv/git -notices /etc/gsg/notices.yaml -admin-addr 127.0.0.1:8081 |
| 262 | ``` |
| 263 | |
| 264 | Protect it with Caddy `forward_auth` and proxy it to the admin listener. gsg |
| 265 | trusts the network in front of it, so the admin port must not be published |
| 266 | directly. In Compose, put 8081 on the internal network only and pass |
| 267 | `-admin-allow-remote`, because a container address is not loopback. Note |
| 268 | that gsg performs no authentication itself -- the `forward_auth` block below |
| 269 | is the only thing between the internet and the admin form. |
| 270 | |
| 271 | ```caddyfile |
| 272 | example.com { |
| 273 | # Admin: authenticate first, then proxy to the separate admin listener. |
| 274 | # handle, not handle_path: the admin mux matches the full /admin/... |
| 275 | # path, so stripping the prefix would 404 every request. |
| 276 | handle /admin/* { |
| 277 | forward_auth authentik:9000 { |
| 278 | uri /outpost.goauthentik.io/auth/caddy |
| 279 | copy_headers X-Authentik-Username |
| 280 | } |
| 281 | reverse_proxy gsg:8081 |
| 282 | } |
| 283 | |
| 284 | # Public, read-only. |
| 285 | handle { |
| 286 | reverse_proxy gsg:8080 |
| 287 | } |
| 288 | } |
| 289 | ``` |
| 290 | |
| 291 | The `X-Authentik-Username` header is used only to attribute the mutation in |
| 292 | the log; it is never consulted for authorization. Authentik decides who may |
| 293 | reach the handler at all. |
| 294 | |
| 295 | Mutations are POST-only, CSRF-protected (double-submit cookie plus |
| 296 | `Origin`/`Sec-Fetch-Site`), and logged with the `X-Authentik-Username` |
| 297 | header, which is used for audit only, never for authorization. |
| 298 | |
| 299 | **Immutable commit pages carry no notices.** `/{repo}/commit/{rev}` is |
| 300 | served `Cache-Control: public, max-age=31536000, immutable`, so a banner |
| 301 | rendered into it would stay frozen in the reader's browser. Notices are |
| 302 | suppressed there by design; every other page picks them up on the next |
| 303 | request. |
| 304 | |
| 305 | ## README rendering |
| 306 | |
| 307 | `modules/markdown.go` uses goldmark with GFM, Typographer, and automatic |
| 308 | heading IDs — Hugo\'s own goldmark defaults, so a README renders identically |
| 309 | to the same file on your blog. |
| 310 | |
| 311 | `WithUnsafe` is deliberately not set, which is also Hugo\'s default. Anyone |
| 312 | who can push can write a README, so raw HTML in markdown is untrusted input; |
| 313 | goldmark drops it. A `<script>` committed to a README comes out as |
| 314 | `<!-- raw HTML omitted -->`. |
| 315 | |
| 316 | Relative links are rewritten against the file\'s own directory: links become |
| 317 | the blob view, images become `/raw/` (an `<img>` needs bytes, not an HTML |
| 318 | page). Absolute URLs, site-absolute paths, fragments and `mailto:` are left |
| 319 | alone, and anything resolving above the repo root is not rewritten. |
| 320 | |
| 321 | ## Per-repository config |
| 322 | |
| 323 | Read from each repo's own git config, cgit-style, with `gitweb.*` and the |
| 324 | legacy `description` file as fallbacks, so an existing cgit tree works as-is: |
| 325 | |
| 326 | ```sh |
| 327 | git --git-dir=/srv/git/demo.git config gsg.title "Demo Project" |
| 328 | git --git-dir=/srv/git/demo.git config gsg.description "A demo repository" |
| 329 | git --git-dir=/srv/git/demo.git config gsg.clone "git://rschanz.org/demo" |
| 330 | git --git-dir=/srv/git/demo.git config gsg.hide true |
| 331 | ``` |
| 332 | |
| 333 | Add a key by adding a field to `core.RepoInfo` and a line in `LoadRepoInfo`. |
| 334 | |
| 335 | ## Theme |
| 336 | |
| 337 | Monospace-first: every value on these pages is an identifier, hash, path or |
| 338 | count, so the interface type is the same face as the code, and the sans face |
| 339 | appears only inside rendered README prose. A cool slate ramp, one indigo |
| 340 | accent, and a six-colour lane ramp shared by the commit graph and its branch |
| 341 | chips. Light and dark both defined via `prefers-color-scheme`. |
| 342 | |
| 343 | Diffs mark the gutter rather than flooding whole rows with green and red — |
| 344 | the signal stays readable when a hunk is mostly additions. |
| 345 | |
| 346 | No webfonts; system stacks only, so there is no third-party request. All |
| 347 | colour lives in `:root` custom properties in `static/style.css`. |
| 348 | |
| 349 | ## Known gaps |
| 350 | |
| 351 | - **No syntax highlighting.** `blob` returns numbered plain lines; Chroma |
| 352 | drops into the same module. |
| 353 | - **No pagination on `log` or `graph`.** They cap at 200 and 150 commits. |
| 354 | - **The graph loads all refs into memory** before laying out. Fine to a few |
| 355 | thousand commits; beyond that it needs windowing. |
| 356 | - **The LRU is bounded by entry count, not bytes.** A giant diff and a ref |
| 357 | list cost the same slot. The `Store` interface is the only thing a |
| 358 | size-aware replacement has to satisfy. |
| 359 | - **Repo discovery is one level deep** and happens once at startup; a repo |
| 360 | created later needs a restart. |
| 361 | - **Module errors surface at render time**, not load time. Add a CI pass |
| 362 | that renders every view against a fixture repo. |