Partly fixes <https://bugs.gnu.org/47867>.
Reported by Florian Pelz <pelzflorian@pelzflorian.de>.
The non-tail recursive call to 'connect' could cause requests to be
processed twice, with 'p' possibly closed the second time.
Regression introduced in 205833b72c and
carried over in 45fce38fb0.
* guix/http-client.scm (http-multiple-get): Remove call to 'close-port'
and recursive call to 'connect' when the 'write-request' block returns #f.
Partly fixes <https://bugs.gnu.org/47867>.
Reported by Florian Pelz <pelzflorian@pelzflorian.de>.
In GnuTLS up to 3.7.1 included, GNUTLS_E_AGAIN and GNUTLS_E_INTERRUPTED
are not handled by 'write_to_session_record_port' and could be thrown at
the caller. This patch works around that by dropping connections
altogether and restarting when this happens.
* guix/http-client.scm (false-if-networking-error): Swallow ERROR/AGAIN
and ERROR/INTERRUPTED.
* guix/scripts/substitute.scm (call-with-cached-connection): Likewise.
Reported by Christopher Baines <mail@cbaines.net>.
* guix/http-client.scm (http-multiple-get): Change 2nd argument to
'drop' to PROCESSED when (false-if-networking-error ...) returns #f.
Fixes <https://bugs.gnu.org/47283>.
Commit 205833b72c made 'http-multiple-get'
non-tail-recursive. Each recursive call would install an exception
handler. As the number of iterations grows beyond 1,000, quadratic
complexity of 'raise-exception' would show and we'd spend most of our
time there.
* guix/http-client.scm (false-if-networking-error): New macro.
(http-multiple-get): Use it around 'write-request' and 'put-bytevector'
calls, and around 'read-response' call, in lieu of the inline 'catch'
forms.
So that an alternative procedure can be passed in, perhaps to perform
connection caching.
* guix/http-client.scm (http-fetch): Add an #:open-connection keyword
argument.
Making sure to close the port if it looks to be unusable. This closing of the
port will allow for caching connections, without caching broken connections,
as the cache can avoid handing out closed ports.
* guix/http-client.scm (http-multiple-get): Try to catch exceptions that
happen if the port is unusable, this is a adaptation of code within the (guix
scripts substitute) module.
From (guix scripts substitute). This will make it easier to reuse this code.
* guix/scripts/substitute.scm (http-multiple-get): Remove, and move to…
* guix/http-client.scm (http-multiple-get): …here.
That way, when fetching a series of substitutes from the same server(s),
the connection is reused instead of being closed/opened for each
substitutes, which saves on network round trips and TLS handshakes.
* guix/http-client.scm (http-fetch): Add #:keep-alive? and honor it.
* guix/progress.scm (progress-report-port): Add #:close? parameter and
honor it.
* guix/scripts/substitute.scm (at-most): Return the tail as a second
value.
(fetch): Add #:port and #:keep-alive? and honor them.
(%max-cached-connections): New variable.
(open-connection-for-uri/cached, call-with-cached-connection): New
procedures.
(with-cached-connection): New macro.
(process-substitution): Wrap 'fetch' call in 'with-cached-connection'.
Pass #:close? to 'progress-report-port'.
* guix/http-client.scm (http-fetch): Add #:timeout and pass it to
'guix:open-connection-for-uri'.
(http-fetch/cached): Add #:timeout parameter and pass it to
'http-fetch'.
Fixes <https://bugs.gnu.org/20145>.
* guix/build/download.scm (%tls-ports, register-tls-record-port): Remove.
(tls-wrap): Remove call to 'register-tls-record-port'. Return a custom
binary input/output port instead. This is a backport of what Guile
2.2's (web client) module has been doing.
(close-connection): Define as an alias for 'close-port'.
* guix/http-client.scm (http-fetch): Remove #:keep-alive? parameter,
which was ignored and unused.
Pass #:keep-alive? #f to 'http-get'.
* guix/lint.scm (probe-uri): Use 'close-port' instead of 'close-connection'.
* guix/scripts/substitute.scm (http-multiple-get): Likewise.
That way CVE fetching benefits from 'If-Modified-Since' handling.
* guix/http-client.scm (http-fetch/cached): Add #:write-cache and
#:cache-miss parameters and honor them.
* guix/cve.scm (%current-year-ttl, %past-year-ttl): Reduce.
(call-with-cve-port): Remove.
(write-cache): New procedure.
(fetch-vulnerabilities): Rewrite in terms of 'http-fetch/cached'.
* guix/http-client.scm (http-fetch/cached)[update-cache]: Add
'cache-port' parameter. Check its mtime and compute 'if-modified-since'
header accordingly. Guard 'http-get-error?' and honor 304.
Adjust callers of 'update-cache'.
* guix/gnu-maintenance.scm (ftp.gnu.org-files): Set #:ttl to 15m.
* guix/build/download.scm (http-fetch): Complete the hard-coded list of HTTP
redirection status codes.
* guix/http-client.scm (http-fetch): Likewise.
* guix/scripts/lint.scm (probe-uri): Likewise.
* guix/build/download.scm (open-connection-for-uri): Add note about
same-named binding in Guile 2.2.0.
* guix/http-client.scm: Use 'guix:open-connection-for-uri' for the
procedure coming from (guix build download).
* guix/scripts/lint.scm: Likewise.
* guix/scripts/substitute.scm: Likewise.
* guix/http-client.scm (cache-file-for-uri): New procedure.
(http-fetch/cached): Use it. Remove 'directory' variable.
[update-cache]: Make the 'dirname' of FILE.
Previously, code using directly (guix build download) was still affected
by <http://bugs.gnu.org/15368>. This includes source derivations, the
'guix download' command, and (guix gnu-maintenance).
'guix substitute' was unaffected since it used (guix http-client), which
already had the fix.
* guix/http-client.scm (open-socket-for-uri): Remove.
(http-fetch): Remove #:buffered? argument to 'open-socket-for-uri';
use 'setvbuf' instead.
* guix/scripts/substitute.scm (fetch): Likewise.
* guix/build/download.scm (open-socket-for-uri): New procedure, taken
from guix/http-client.scm, but without the #:buffered? parameter.
Reported by Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de>.
* guix/http-client.scm (make-delimited-input-port): New procedure.
Install it in (web response) for Guile <= 2.0.9.
Fixes <http://bugs.gnu.org/19840>.
Reported by Ricardo Wurmus <rekado@elephly.net>.
* guix/build/download.scm: On Guile 2.0.11 or earlier, redefine the http
"Location" header to accept relative URIs.
(resolve-uri-reference): New exported procedure.
(http-fetch): Use 'resolve-uri-reference' to resolve redirections.
* guix/http-client.scm (http-fetch): Use 'resolve-uri-reference'
* guix/http-client.scm (&http-get-error): New condition type.
(http-fetch): Raise it instead of using 'error'.
* guix/scripts/substitute-binary.scm (fetch) <http>: Wrap body into
'guard' form; gracefully handle 'http-get-error?' conditions.
This fixes Guile version comparisons when (version) has a
vendor-specific suffix.
Reported by Andreas Enge <andreas@enge.fr>.
* guix/utils.scm (guile-version>?): New procedure.
* tests/utils.scm ("guile-version>? 1.8", "guile-version>? 10.5"): New
tests.
* guix/scripts/substitute-binary.scm (fetch, progress-report-port): Use
`guile-version>?' instead of `version>?'.
* guix/http-client.scm (when-guile<=2.0.5, http-fetch): Likewise.
