ryan77627/guix
1
0
Fork 0
mirror of https://git.in.rschanz.org/ryan77627/guix.git synced 2024-12-25 22:08:16 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions

gnu: racket-minimal: Bootstrap from C.

Browse source 
This commit bootstraps the Racket compiler and runtime system from source,
including Racket CS as well as both variants of Racket BC. (One remaining
limitation is discussed in comments added to gnu/packages/racket.scm.)

In the process, it moves to building minimal Racket from the Git repository,
rather than the packaged source tarballs. The Git repository is slightly
better as the ``corresponding source'':

 1. A few packages especially closely tied to the Racket core implementation
    (like "compiler-lib", "base", and "racket-doc") are developed in the
    same Git repository. Having them use the same Guix origin, too, will
    help to keep them in sync.

 2. The top-level Makefile in the Git repository is an important
    ``script[] used to control compilation and installation.''
    In particular, it cooperates with the "distro-build" package to
    create the source tarballs and installers for a Racket distribution.
    (Racket supports a notion of custom distributions.)

 3. It is ``the preferred form ... for making modifications'' to the core
    Racket implementation.

Racket releases are tagged in the Git repository (e.g. "v8.1"). At the
beginning of each release cycle, a branch is created to stabilizer a version
for extra testing. Active development happens on the "master" branch.

* gnu/packages/racket-minimal-sh-via-rktio.patch: Adjust for extra directory
layer.
* gnu/local/racket.scm (cfg-flag:sh-for-rktio, cfg-flag:enable-lt,
cfg-flag:enable-racket, unpack-nanopass+stex,
%main-repo-main-distribution-pkgs): New private variables.
* gnu/local/racket.scm (racket-minimal)[source]: Use Git.
[source](snippet): Unbundle nanopass, stex, and libffi.
[native-inputs]: Use racket-bootstrap-chez-bootfiles, plus its
dependencies (for Chez, plus a Racket for bootstrap pig).
[arguments]: Revise extensively.
* gnu/local/racket.scm (racket-minimal-bc-3m, racket-minimal-bc-cgc): New
packages, hidden at least for now.
(racket-bootstrap-chez-bootfiles): Another new package, but this one is
especially likely to stay hidden.
* gnu/local/racket.scm (racket)[origin](snippet): Unbundle packages
developed in the main Git repository, but leave their links.rktd and
pkgs.rktd entries in place.
[native-inputs]: Add the main Racket Git repository.
[arguments](#:phases): Adjust 'unpack-packages to also unpack package
sources from the main Racket Git repository.

Signed-off-by: Ludovic Courtès <ludo@gnu.org>
This commit is contained in:
Philip McGrath 2021-07-19 02:31:43 -04:00 committed by Ludovic Courtès
parent 65bad4d036
commit db2c7e70ad
No known key found for this signature in database
GPG key ID: 090B11993D9AEBB5
2 changed files with 377 additions and 155 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
gnu/packages/patches/racket-minimal-sh-via-rktio.patch
View file

@ -35,13 +35,13 @@ making this change at the C level is both:
conditional and a runtime check that the file in the store exists,
we make it much less likely that it will "leak" out of Guix.
---
src/rktio/rktio_process.c | 21 ++++++++++++++++++++-
racket/src/rktio/rktio_process.c | 21 ++++++++++++++++++++-
1 file changed, 20 insertions(+), 1 deletion(-)
diff --git a/src/rktio/rktio_process.c b/src/rktio/rktio_process.c
diff --git a/racket/src/rktio/rktio_process.c b/racket/src/rktio/rktio_process.c
index 89202436c0..465ebdd5c5 100644
--- a/src/rktio/rktio_process.c
+++ b/src/rktio/rktio_process.c
--- a/racket/src/rktio/rktio_process.c
+++ b/racket/src/rktio/rktio_process.c
@@ -1224,12 +1224,14 @@ int rktio_process_allowed_flags(rktio_t *rktio)
/*========================================================================*/

524
gnu/packages/racket.scm
View file

@ -31,7 +31,9 @@ (define-module (gnu packages racket)
#:use-module (srfi srfi-1)
#:use-module (ice-9 match)
#:use-module (gnu packages)
#:use-module (gnu packages autotools)
#:use-module (gnu packages bash)
#:use-module (gnu packages chez)
#:use-module (gnu packages compression)
#:use-module (gnu packages databases)
#:use-module (gnu packages fontutils)
@ -40,11 +42,333 @@ (define-module (gnu packages racket)
#:use-module (gnu packages gtk)
#:use-module (gnu packages image)
#:use-module (gnu packages libedit)
#:use-module (gnu packages libffi)
#:use-module (gnu packages multiprecision)
#:use-module (gnu packages sqlite)
#:use-module (gnu packages tls)
#:use-module (gnu packages xorg))
;; Commentary:
;;
;; Here's how bootstrapping minimal Racket works:
;;
;; - Racket BC [CGC] can be built with only a C compiler (except for
;; one caveat discussed below).
;; - Racket BC [3M] needs an existing Racket to run "xform",
;; which transforms its own C source code to add additional annotations
;; for the precise garbage collector.
;; - Racket CS needs (bootfiles for) Racket's fork of Chez Scheme.
;; It also needs an existing Racket to compile Racket-implemented
;; parts of the runtime system to R6RS libraries.
;; - Chez Scheme also needs bootfiles for itself, but Racket can simulate
;; enough of Chez Scheme to load Racket's fork of the Chez Scheme compiler
;; purely from source into Racket and apply the compiler to itself,
;; producing the needed bootfiles (albeit very slowly).
;; Any variant of Racket since version 7.1 can run the simulation.
;;
;; So, we build CGC to build 3M to build bootfiles and CS.
;;
;; One remaining bootstrapping limitation is that Racket's reader, module
;; system, and macro expander are implemented in Racket. For Racket CS,
;; they are compiled to R6RS libraries as discussed above. This note from the
;; README file applies to all such subsystems:
;;
;; The Racket version must be practically the same as the current Racket
;; verson, although it can be the Racket BC implementation (instead of
;; the Racket CS implementation).
;;
;; Unlike Chez Scheme boot files, the files generated in "schemified"
;; are human-readable and -editable Scheme code. That provides a way
;; out of bootstrapping black holes, even without BC.
;;
;; However, other Racket subsystems implemented in Racket for Racket CS
;; use older C implementations for Racket BC, whereas the reader, expander,
;; and module system were completely replaced with the Racket implementation
;; as of Racket 7.0.
;;
;; For Racket BC, the compiled "linklet" s-expressions (primitive modules)
;; are embeded in C as a static string constant. Eventually, they are further
;; compiled by the C-implemented Racket BC bytecode and JIT compilers.
;; (On platforms where Racket BC's JIT is not supported, yet another compiler
;; instead compiles the linklets to C code, but this is not a bootstrapping
;; issue.)
;;
;; Code:
(define cfg-flag:sh-for-rktio
`(string-append "CPPFLAGS=-DGUIX_RKTIO_PATCH_BIN_SH="
(assoc-ref %build-inputs "sh")
"/bin/sh"))
(define cfg-flag:enable-lt
`(string-append "--enable-lt="
(assoc-ref %build-inputs "libtool")
"/bin/libtool"))
(define cfg-flag:enable-racket
`(let ((racket (assoc-ref %build-inputs "racket")))
(string-append "--enable-racket="
racket
"/bin/racket")))
(define unpack-nanopass+stex
;; Copied from chez-scheme.
;; TODO: Eventually, we should refactor Chez Scheme
;; enough to share more directly, so that we can make
;; Racket's version of Chez avalable as a Guix package,
;; e.g. for architectures not supported upstream.
;; For now, we let Racket drive the Chez build process
;; other than this step.
`(for-each (lambda (dep)
(define src
(assoc-ref (or native-inputs inputs) dep))
(copy-recursively src dep
#:keep-mtime? #t))
'("nanopass" "stex")))
(define-public racket-minimal
(package
(name "racket-minimal")
(version "8.2") ; note: remember to also update racket!
(source
(origin
(method git-fetch)
(uri (git-reference
(url "https://github.com/racket/racket")
(commit (string-append "v" version))))
(sha256
"061bhiyjlvazph0dj9i3i3x2q5z53rp8h5cjwg3frjimkr45lncn")
(file-name (git-file-name name version))
(patches (search-patches "racket-minimal-sh-via-rktio.patch"))
(snippet
(with-imported-modules '((guix build utils))
#~(begin
(use-modules (guix build utils))
;; unbundle Chez submodules
(with-directory-excursion "racket/src/ChezScheme"
#$(origin-snippet (package-source chez-scheme)))
;; unbundle libffi
(for-each
delete-file-recursively
'("racket/src/bc/foreign/libffi")))))))
(inputs
`(;; common to all racket-minimal variants:
("openssl" ,openssl)
("sqlite" ,sqlite)
("sh" ,bash-minimal)
;; only for CS
("zlib" ,zlib)
("zlib:static" ,zlib "static")
("lz4" ,lz4)
("lz4:static" ,lz4 "static")))
(native-inputs
`(("bootfiles" ,racket-bootstrap-chez-bootfiles)
,@(package-native-inputs racket-bootstrap-chez-bootfiles)))
(build-system gnu-build-system)
(arguments
`(#:configure-flags
(list "--enable-csonly"
"--enable-libz"
"--enable-liblz4"
,cfg-flag:enable-racket
,cfg-flag:sh-for-rktio)
#:out-of-source? #true
;; Tests are in packages like racket-test-core and
;; main-distribution-test that aren't part of the main distribution.
#:tests? #f
#:modules ((ice-9 match)
(guix build gnu-build-system)
(guix build utils))
#:phases
(modify-phases %standard-phases
(add-after 'unpack 'unpack-nanopass+stex
(lambda* (#:key inputs native-inputs #:allow-other-keys)
(with-directory-excursion "racket/src/ChezScheme"
,unpack-nanopass+stex)
#t))
(add-after 'unpack-nanopass+stex 'unpack-bootfiles
(lambda* (#:key inputs #:allow-other-keys)
(with-directory-excursion "racket/src/ChezScheme"
(copy-recursively
(string-append (assoc-ref inputs "bootfiles") "/boot")
"boot"))
#t))
(add-before 'configure 'initialize-config.rktd
(lambda* (#:key inputs #:allow-other-keys)
(define (write-racket-hash alist)
;; inside must use dotted pair notation
(display "#hash(")
(for-each (match-lambda
((k . v)
(format #t "(~s . ~s)" k v)))
alist)
(display ")\n"))
(mkdir-p "racket/etc")
(with-output-to-file "racket/etc/config.rktd"
(lambda ()
(write-racket-hash
`((lib-search-dirs
. (#f ,@(map (lambda (lib)
(string-append (assoc-ref inputs lib)
"/lib"))
'("openssl"
"sqlite"))))
(catalogs
. (,(string-append
"https://download.racket-lang.org/releases/"
,version
"/catalog/")
#f))))))
#t))
(add-before 'configure 'change-directory
(lambda _
(chdir "racket/src")
#t))
(add-after 'install 'remove-pkgs-directory
;; If the configured pkgs-dir exists, "pkgs.rktd" does not
;; exist, and a lock file does not exist, commands like
;; `raco pkg show` will try to create a lock file and fail
;; due to the read-only store.
;; Arguably this may be a bug in `pkg/private/lock`:
;; see <https://github.com/racket/racket/issues/3851>.
;; As a workaround, remove the directory.
(lambda* (#:key outputs #:allow-other-keys)
;; rmdir because we want an error if it isn't empty
(rmdir (string-append (assoc-ref outputs "out")
"/share/racket/pkgs"))
#t)))))
(home-page "https://racket-lang.org")
(synopsis "Racket without bundled packages such as DrRacket")
(description
"Racket is a general-purpose programming language in the Scheme family,
with a large set of libraries and a compiler based on Chez Scheme. Racket is
also a platform for language-oriented programming, from small domain-specific
languages to complete language implementations.
The ``minimal Racket'' distribution includes just enough of Racket for you to
use @command{raco pkg} to install more. Bundled packages, such as the
DrRacket IDE, are not included.")
;; https://download.racket-lang.org/license.html
;; The LGPL components are only used by Racket BC.
(license (list asl2.0 expat))))
(define-public racket-minimal-bc-3m
(hidden-package
(package
(inherit racket-minimal)
(name "racket-minimal-bc-3m")
(inputs
`(("libffi" ,libffi) ;; <- only for BC variants
,@(fold alist-delete
(package-inputs racket-minimal)
'("zlib" "zlib:static" "lz4" "lz4:static"))))
(native-inputs
`(("libtool" ,libtool)
("racket" ,(if (%current-target-system)
racket-minimal
racket-minimal-bc-cgc))))
(arguments
(substitute-keyword-arguments (package-arguments racket-minimal)
((#:configure-flags _ '())
`(list "--enable-bconly"
,cfg-flag:enable-racket
,cfg-flag:enable-lt
,cfg-flag:sh-for-rktio))
((#:phases usual-phases)
`(modify-phases ,usual-phases
(delete 'unpack-nanopass+stex)
(delete 'unpack-bootfiles)))))
(synopsis "Minimal Racket with the BC [3M] runtime system")
(description "The Racket BC (``before Chez'' or ``bytecode'')
implementation was the default before Racket 8.0. It uses a compiler written
in C targeting architecture-independent bytecode, plus a JIT compiler on most
platforms. Racket BC has a different C API and supports a slightly different
set of architectures than the current default runtime system, Racket CS (based
on ``Chez Scheme'').
This package is the normal implementation of Racket BC with a precise garbage
collector, 3M (``Moving Memory Manager'').")
;; https://download.racket-lang.org/license.html
;; The LGPL components are only used by Racket BC.
(license (list lgpl3+ asl2.0 expat)))))
(define-public racket-minimal-bc-cgc
(package
(inherit racket-minimal-bc-3m)
(name "racket-minimal-bc-cgc")
(native-inputs
(alist-delete "racket" (package-native-inputs racket-minimal-bc-3m)))
(arguments
(substitute-keyword-arguments (package-arguments racket-minimal-bc-3m)
((#:configure-flags _ '())
`(list "--enable-cgcdefault"
,cfg-flag:enable-lt
,cfg-flag:sh-for-rktio))))
(synopsis "Old Racket implementation used for bootstrapping")
(description "This variant of the Racket BC (``before Chez'' or
``bytecode'') implementation is not recommended for general use. It uses
CGC (a ``Conservative Garbage Collector''), which was succeeded as default in
PLT Scheme version 370 (which translates to 3.7 in the current versioning
scheme) by the 3M variant, which in turn was succeeded in version 8.0 by the
Racket CS implementation.
Racket BC [CGC] is primarily used for bootstrapping Racket BC [3M]. It may
also be used for embedding applications without the annotations needed in C
code to use the 3M garbage collector.")))
(define-public racket-bootstrap-chez-bootfiles
(hidden-package
(package
(inherit racket-minimal)
(name "racket-bootstrap-chez-bootfiles")
(inputs `())
(native-inputs
`(("racket" ,(if (%current-target-system)
racket-minimal
racket-minimal-bc-3m))
("stex" ,@(assoc-ref (package-native-inputs chez-scheme) "stex"))
("nanopass" ,@(assoc-ref (package-native-inputs chez-scheme)
"nanopass"))))
(arguments
`(#:phases
(modify-phases %standard-phases
(add-after 'unpack 'unpack-nanopass+stex
(lambda* (#:key inputs native-inputs #:allow-other-keys)
(with-directory-excursion "racket/src/ChezScheme"
,unpack-nanopass+stex)
#t))
(delete 'configure)
(delete 'patch-generated-file-shebangs)
(replace 'build
(lambda* (#:key inputs outputs #:allow-other-keys)
(with-directory-excursion "racket/src/ChezScheme"
(invoke (string-append (assoc-ref inputs "racket")
"/bin/racket")
"rktboot/main.rkt"
"--dest" (assoc-ref outputs "out")))
#t))
(delete 'check)
(delete 'install))))
(synopsis "Chez Scheme bootfiles bootstrapped by Racket")
(description "Chez Scheme is a self-hosting compiler: building it
requires ``bootfiles'' containing the Scheme-implemented portions compiled for
the current platform. (Chez can then cross-compile bootfiles for all other
supported platforms.)
The Racket package @code{cs-bootstrap} (part of the main Racket Git
repository) implements enough of a Chez Scheme simulation to load the Chez
Scheme compiler purely from source into Racket and apply the compiler to
itself, thus bootstrapping Chez Scheme. Bootstrapping takes about 10 times as
long as using an existing Chez Scheme, but @code{cs-bootstrap} supports Racket
7.1 and later, including the Racket BC variant.
Note that the generated bootfiles are specific to Racket's fork of Chez
Scheme, and @code{cs-bootstrap} does not currently support building upstream
Chez Scheme.")
(license (list asl2.0)))))
(define %installer-mirrors
;; Source:
@ -60,114 +384,28 @@ (define %installer-mirrors
"https://racket.infogroep.be/"
))
(define-public racket-minimal
(package
(name "racket-minimal")
(version "8.2") ; note: remember to also update racket!
(source
(origin
(method url-fetch)
(uri (map (lambda (base)
(string-append base version "/racket-minimal-src.tgz"))
%installer-mirrors))
(sha256 "13qfg56w554vdj5iwa8lpacy83s7bzhhyr44pjns68mkhj69ring")
(patches (search-patches
"racket-minimal-sh-via-rktio.patch"))))
(home-page "https://racket-lang.org")
(synopsis "Racket without bundled packages such as DrRacket")
(inputs
`(("openssl" ,openssl)
("sqlite" ,sqlite)
("sh" ,bash-minimal)
("zlib" ,zlib)
("zlib:static" ,zlib "static")
("lz4" ,lz4)
("lz4:static" ,lz4 "static")))
(build-system gnu-build-system)
(arguments
`(#:configure-flags
`(,(string-append "CPPFLAGS=-DGUIX_RKTIO_PATCH_BIN_SH="
(assoc-ref %build-inputs "sh")
"/bin/sh")
"--enable-libz"
"--enable-liblz4")
#:modules
((guix build gnu-build-system)
(guix build utils)
(srfi srfi-1))
#:phases
(modify-phases %standard-phases
(add-before 'configure 'pre-configure-minimal
(lambda* (#:key inputs #:allow-other-keys)
(chdir "src")
#t))
(add-after 'build 'patch-config.rktd-lib-search-dirs
(lambda* (#:key inputs outputs #:allow-other-keys)
;; We do this between the `build` and `install` phases
;; so that we have racket to read and write the hash table,
;; but it comes before `raco setup`, when foreign libraries
;; are needed to build the documentation.
(define out (assoc-ref outputs "out"))
(apply invoke
"./cs/c/racketcs"
"-e"
,(format #f
"~s"
'(let* ((args
(vector->list
(current-command-line-arguments)))
(file (car args))
(extra-lib-search-dirs (cdr args)))
(write-to-file
(hash-update
(file->value file)
'lib-search-dirs
(lambda (dirs)
(append dirs extra-lib-search-dirs))
'(#f))
#:exists 'truncate/replace
file)))
"--"
"../etc/config.rktd"
(filter-map (lambda (lib)
(cond
((assoc-ref inputs lib)
=> (lambda (pth)
(string-append pth "/lib")))
(else
#f)))
'("cairo"
"fontconfig"
"glib"
"glu"
"gmp"
"gtk+"
"libjpeg"
"libpng"
"libx11"
"mesa"
"mpfr"
"openssl"
"pango"
"sqlite"
"unixodbc"
"libedit")))
#t)))
;; Tests are in packages like racket-test-core and
;; main-distribution-test that aren't part of the main distribution.
#:tests? #f))
(description
"Racket is a general-purpose programming language in the Scheme family,
with a large set of libraries and a compiler based on Chez Scheme. Racket is
also a platform for language-oriented programming, from small domain-specific
languages to complete language implementations.
The ``minimal Racket'' distribution includes just enough of Racket for you to
use @command{raco pkg} to install more. Bundled packages, such as the
DrRacket IDE, are not included.")
;; https://download.racket-lang.org/license.html
(license (list lgpl3+ asl2.0 expat))))
(define %main-repo-main-distribution-pkgs
;; These are the packages developed in the main Racket Git repository
;; that are part of the main distribution.
'("at-exp-lib"
"base"
"compiler-lib"
;; NOT "compiler-test"
"compiler"
"net-doc"
"net-lib"
;; NOT "net-test"
"net"
;; NOT "plt-services"
;; NOT "racket-benchmarks"
;; NOT "racket-build-guide"
"racket-doc"
"racket-index"
"racket-lib"
;; NOT "racket-test-core"
;; NOT "racket-test-extra"
;; NOT "racket-test"
"zo-lib"))
(define-public racket
@ -177,7 +415,7 @@ (define-public racket
(version (package-version racket-minimal)) ; needed for origin uri to work
(source
(origin
(inherit (package-source racket-minimal))
(method url-fetch)
(uri (map (lambda (base)
(string-append base version "/racket-src.tgz"))
%installer-mirrors))
@ -197,36 +435,10 @@ (define-public racket
"README"
"src"))
;; unbundle package sources included elsewhere
(define (substitute/delete file pattern)
(substitute
file
(list (cons pattern
(lambda (line matches)
;; must match exactly once
(match matches
((m)
(string-append (match:prefix m)
(match:suffix m)))))))))
(define (unbundle-pkg pkg)
(define quoted-pkg (regexp-quote pkg))
(with-directory-excursion "share"
(substitute/delete
"links.rktd"
(string-append
"[(][^()]+[(]#\"pkgs\" #\""
quoted-pkg
"\"[)][)]"))
(with-directory-excursion "pkgs"
(substitute/delete
"pkgs.rktd"
(string-append
"[(]\""
quoted-pkg
"\" \\. #s[(]"
"(pkg-info|[(]sc-pkg-info pkg-info 3[)])"
" [(][^()]+[)] [^()]+[)][)]"))
(delete-file-recursively pkg))))
(unbundle-pkg "racket-lib")))))
(with-directory-excursion "share/pkgs"
(for-each delete-file-recursively
'#+%main-repo-main-distribution-pkgs))
#t))))
(inputs
`(("cairo" ,cairo)
("fontconfig" ,fontconfig)
@ -244,22 +456,32 @@ (define quoted-pkg (regexp-quote pkg))
("libedit" ,libedit)))
(native-inputs
`(("racket" ,racket-minimal)
("extend-layer" ,extend-layer)))
("extend-layer" ,extend-layer)
("main-repo" ,(package-source racket-minimal))))
(arguments
`(#:phases
(modify-phases %standard-phases
(add-before 'configure 'unpack-packages
(lambda* (#:key native-inputs inputs outputs #:allow-other-keys)
(let ((racket (assoc-ref (or native-inputs inputs) "racket"))
(prefix (assoc-ref outputs "out")))
(mkdir-p (string-append prefix "/share/racket/pkgs"))
(copy-recursively
"share/links.rktd"
(string-append prefix "/share/racket/links.rktd"))
(copy-recursively
"share/pkgs"
(string-append prefix "/share/racket/pkgs"))
#t)))
(let ((unpack (assoc-ref %standard-phases 'unpack)))
(lambda* (#:key native-inputs inputs outputs #:allow-other-keys)
(let* ((racket (assoc-ref (or native-inputs inputs) "racket"))
(prefix (assoc-ref outputs "out"))
(pkgs-dir (string-append prefix "/share/racket/pkgs")))
(mkdir-p pkgs-dir)
(copy-recursively
"share/links.rktd"
(string-append prefix "/share/racket/links.rktd"))
(copy-recursively "share/pkgs" pkgs-dir)
;; NOTE: unpack changes the working directory
(unpack #:source (assoc-ref (or native-inputs inputs)
"main-repo"))
(for-each (lambda (pkg)
(define dest (string-append pkgs-dir "/" pkg))
(mkdir-p dest)
(copy-recursively (string-append "pkgs/" pkg)
dest))
',%main-repo-main-distribution-pkgs)
#t))))
(replace 'configure
(lambda* (#:key native-inputs inputs outputs #:allow-other-keys)
(let ((racket (assoc-ref (or native-inputs inputs) "racket"))