How to build GNUTLS Guile bindings on Xubuntu

Discussion:

Zelphir Kaltstahl

2018-11-15 00:49:13 UTC

Hi,

I read in the docs at
https://www.gnu.org/software/guile/manual/html_node/Web-Client.html that
Guile will dynamically make use of GNUTLS for HTTPS requests. However, I
don't seem to have the bindings installed. There is a link to a guide on
the same docs page, but the link is dead:
https://www.gnu.org/software/guile/manual/gnutls-guile/Guile-Preparations.html#Guile-Preparations
. More searching showed me the following:

There seems to have been some bug which prevented some tests on Debian
from completing:

Bug discussions seem to indicate the bug was fixed?:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=821457
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=805863

Then there is a commit removing Guile bindings from the gnutls-dev
package on Debian:

https://salsa.debian.org/gnutls-team/gnutls/commit/ebb7130b47dc08311c1de2c189758a73bbaeca27#58ef006ab62b83b4bec5d81fe5b32c3b4c2d1cc2_35_35

Discussion about adding it again:

https://groups.google.com/forum/#!topic/linux.debian.bugs.dist/3DTLVkTvME8

Apart from that, I could not find a download link or repository of the
bindings, like I could for other libraries I so far used in Guile and
run configure make make install in it. So I do not know how to get the
bindings for my system, so that my Guile will be able to make requests
using HTTPS as well as HTTP.

The bindings seem to be available on GUIX:

https://gitlab.digitalcourage.de/htgoebel/guix/commit/1dbe3a8db0a3e5a8e5f9b30e6f6a6bbfb699275b

How can I make it work on a Xubuntu 18.04?

Regards,

Zelphir

Mark H Weaver

2018-11-15 07:28:02 UTC

Permalink

Hi Zelphir,

Post by Zelphir Kaltstahl
I read in the docs at
https://www.gnu.org/software/guile/manual/html_node/Web-Client.html that
Guile will dynamically make use of GNUTLS for HTTPS requests. However, I
don't seem to have the bindings installed. There is a link to a guide on
https://www.gnu.org/software/guile/manual/gnutls-guile/Guile-Preparations.html#Guile-Preparations
There seems to have been some bug which prevented some tests on Debian
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=821457
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=805863
Then there is a commit removing Guile bindings from the gnutls-dev
https://salsa.debian.org/gnutls-team/gnutls/commit/ebb7130b47dc08311c1de2c189758a73bbaeca27#58ef006ab62b83b4bec5d81fe5b32c3b4c2d1cc2_35_35

If you look closely, those two bugs were "fixed" by the above commit,
i.e. by simply disabling the Guile bindings in Debian's GnuTLS package.

Post by Zelphir Kaltstahl
Apart from that, I could not find a download link or repository of the
bindings, like I could for other libraries I so far used in Guile and
run configure make make install in it. So I do not know how to get the
bindings for my system, so that my Guile will be able to make requests
using HTTPS as well as HTTP.

The bindings in question are not a separate package. They are built and
installed by GnuTLS itself, unless explicitly disabled by passing
"--disable-guile" to the GnuTLS configure script, as Debian now does.

Post by Zelphir Kaltstahl
https://gitlab.digitalcourage.de/htgoebel/guix/commit/1dbe3a8db0a3e5a8e5f9b30e6f6a6bbfb699275b

Note that the repository above is not the official Guix repository, but
rather someone's personal variant. Here's the same commit in the
official Guix repository:

https://git.sv.gnu.org/cgit/guix.git/commit/?id=1dbe3a8db0a3e5a8e5f9b30e6f6a6bbfb699275b

Post by Zelphir Kaltstahl
How can I make it work on a Xubuntu 18.04?

Yes, an easy solution would be to install Guix on top of Xubuntu, and
installing the 'guile' and 'gnutls' packages from Guix. Note that the
binaries from Guix are self-contained and independent of the host
system, in the sense that they are installed in different directories
(within /gnu/store) and don't use any libraries from the host system,
not even the C library. Guix installs only in /gnu, /var/guix,
/var/log/guix, /etc/guix, ~/.config/guix, and ~/.guix-profile. You must
set your environment variables to point within ~/.guix-profile, e.g. add
~/.guix-profile/bin to your PATH, to enable the use of software from
Guix. If you'd like to try it, the instructions are here:

https://www.gnu.org/software/guix/manual/en/html_node/Installation.html

Mark

Zelphir Kaltstahl

2018-11-15 22:41:11 UTC

Permalink

Hi Mark,

I took another look at the page you linked to. The issue is, that I
would like to try Guix package manager to install GNUTLS + Guile
bindings, but Guix itself has the requirement of GNUTLS + Guile
bindings. See:
https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html#Requirements

So I guess I would need to somehow get GNUTLS and Guile bindings working
before getting Guix and then when I have Guix, I can probably somehow
link to the Guix installed GNUTLS and Guile bindings.

Now I don't know where I would get the Guile bindings from. Are they
included in the releases on https://gnutls.org/ ?

Regards,

Zelphir

Post by Mark H Weaver
Hi Zelphir,

If you look closely, those two bugs were "fixed" by the above commit,
i.e. by simply disabling the Guile bindings in Debian's GnuTLS package.

Post by Zelphir Kaltstahl
https://gitlab.digitalcourage.de/htgoebel/guix/commit/1dbe3a8db0a3e5a8e5f9b30e6f6a6bbfb699275b

Note that the repository above is not the official Guix repository, but
rather someone's personal variant. Here's the same commit in the
https://git.sv.gnu.org/cgit/guix.git/commit/?id=1dbe3a8db0a3e5a8e5f9b30e6f6a6bbfb699275b

Post by Zelphir Kaltstahl
How can I make it work on a Xubuntu 18.04?

Yes, an easy solution would be to install Guix on top of Xubuntu, and
installing the 'guile' and 'gnutls' packages from Guix. Note that the
binaries from Guix are self-contained and independent of the host
system, in the sense that they are installed in different directories
(within /gnu/store) and don't use any libraries from the host system,
not even the C library. Guix installs only in /gnu, /var/guix,
/var/log/guix, /etc/guix, ~/.config/guix, and ~/.guix-profile. You must
set your environment variables to point within ~/.guix-profile, e.g. add
~/.guix-profile/bin to your PATH, to enable the use of software from
https://www.gnu.org/software/guix/manual/en/html_node/Installation.html
Mark

Mark H Weaver

2018-11-16 02:10:44 UTC

Permalink

Post by Zelphir Kaltstahl
I took another look at the page you linked to. The issue is, that I
would like to try Guix package manager to install GNUTLS + Guile
bindings, but Guix itself has the requirement of GNUTLS + Guile
https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html#Requirements
So I guess I would need to somehow get GNUTLS and Guile bindings working
before getting Guix and then when I have Guix, I can probably somehow
link to the Guix installed GNUTLS and Guile bindings.

If you install Guix using the binary installation method, this will not
be an issue, because the binary install tarball is self-contained, i.e.
it includes all of Guix's dependencies, including Guile, GnuTLS,
sqlite3, guile-sqlite3, guile-json, libgc, libffi, etc, all the way down
to the C library, even low-level things like the dynamic linker, all in
/gnu/store. You should be able to unpack it on any system with a Linux
kernel on one of our supported architectures, no matter how bare its
userspace environment is, follow the remaining install steps described
on the following page, and it should just work.

https://www.gnu.org/software/guix/manual/en/html_node/Binary-Installation.html

Mark

Alex Vong

2018-11-16 08:44:20 UTC

Permalink

Hi Zelphir,

Post by Zelphir Kaltstahl
Hi Mark,
I took another look at the page you linked to. The issue is, that I
would like to try Guix package manager to install GNUTLS + Guile
bindings, but Guix itself has the requirement of GNUTLS + Guile
https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html#Requirements
So I guess I would need to somehow get GNUTLS and Guile bindings working
before getting Guix and then when I have Guix, I can probably somehow
link to the Guix installed GNUTLS and Guile bindings.
Now I don't know where I would get the Guile bindings from. Are they
included in the releases on https://gnutls.org/ ?

As Mark has pointed out, if you use the binary installation of guix,
then you can install gnutls using guix. However, I prefer to build guix
from source. Therefore, I use the standard "configure, make, make
install" method with appropriate CPPFLAGS, CFLAGS, CXXFLAGS and LDFLAGS
(I include some hardending flags[0][1]):

./configure --with-included-libtasn1 --with-included-unistring --with-guile-site-dir=/usr/local/share/guile/site/2.2
make
make check
make install

The above works in Debian. Apart from it, you also have to build
scheme-bytestructure, guile-git and guile-sqlite3 from source using the
same method. Besides, it is important to keep gnutls up to date (since
it is a piece of security-sensitive software).

Post by Zelphir Kaltstahl
Regards,
Zelphir

Cheers,
Alex

[0]: https://wiki.debian.org/Hardening
[1]: https://security.stackexchange.com/questions/24444/what-is-the-most-hardened-set-of-options-for-gcc-compiling-c-c

Post by Zelphir Kaltstahl

Post by Mark H Weaver
Hi Zelphir,

If you look closely, those two bugs were "fixed" by the above commit,
i.e. by simply disabling the Guile bindings in Debian's GnuTLS package.

Post by Zelphir Kaltstahl
https://gitlab.digitalcourage.de/htgoebel/guix/commit/1dbe3a8db0a3e5a8e5f9b30e6f6a6bbfb699275b

Post by Zelphir Kaltstahl
How can I make it work on a Xubuntu 18.04?

Yes, an easy solution would be to install Guix on top of Xubuntu, and
installing the 'guile' and 'gnutls' packages from Guix. Note that the
binaries from Guix are self-contained and independent of the host
system, in the sense that they are installed in different directories
(within /gnu/store) and don't use any libraries from the host system,
not even the C library. Guix installs only in /gnu, /var/guix,
/var/log/guix, /etc/guix, ~/.config/guix, and ~/.guix-profile. You must
set your environment variables to point within ~/.guix-profile, e.g. add
~/.guix-profile/bin to your PATH, to enable the use of software from
https://www.gnu.org/software/guix/manual/en/html_node/Installation.html
Mark

Mark H Weaver

2018-11-16 18:58:59 UTC

Permalink

Hi Alex,

Post by Alex Vong

As Mark has pointed out, if you use the binary installation of guix,
then you can install gnutls using guix. However, I prefer to build guix
from source. Therefore, I use the standard "configure, make, make
install" method with appropriate CPPFLAGS, CFLAGS, CXXFLAGS and LDFLAGS
./configure --with-included-libtasn1 --with-included-unistring --with-guile-site-dir=/usr/local/share/guile/site/2.2
make
make check
make install
The above works in Debian. Apart from it, you also have to build
scheme-bytestructure, guile-git and guile-sqlite3 from source using the
same method. Besides, it is important to keep gnutls up to date (since
it is a piece of security-sensitive software).

Interesting. I haven't tried building Guix from source on Debian in a
few years, but it's not clear to me how your suggestion above addresses
the original problem that Zelphir mentioned, namely that Guix requires
the Guile bindings for GnuTLS, which are not included in Debian. I
don't see any mention of building those bindings above. Am I missing
something?

Mark

Alex Vong

2018-11-16 23:19:44 UTC

Permalink

Hello Mark,

Post by Mark H Weaver
Hi Alex,

Post by Alex Vong

As Mark has pointed out, if you use the binary installation of guix,
then you can install gnutls using guix. However, I prefer to build guix
from source. Therefore, I use the standard "configure, make, make
install" method with appropriate CPPFLAGS, CFLAGS, CXXFLAGS and LDFLAGS
./configure --with-included-libtasn1 --with-included-unistring
--with-guile-site-dir=/usr/local/share/guile/site/2.2
make
make check
make install
The above works in Debian. Apart from it, you also have to build
scheme-bytestructure, guile-git and guile-sqlite3 from source using the
same method. Besides, it is important to keep gnutls up to date (since
it is a piece of security-sensitive software).

Maybe what I wrote is not clear. What I mean is that since Debian build
of gnutls does not include the guile bindings, we have to build gnutls
from the source tarball ourselves.

The flag "--with-guile-site-dir=/usr/local/share/guile/site/2.2"
instructs the build script to install the guile bindings into
"/usr/local/share/guile/site/2.2".

Post by Mark H Weaver
Mark

Mark H Weaver

2018-11-17 01:30:13 UTC

Permalink

Hi Alex,

Post by Alex Vong
Maybe what I wrote is not clear. What I mean is that since Debian build
of gnutls does not include the guile bindings, we have to build gnutls
from the source tarball ourselves.
The flag "--with-guile-site-dir=/usr/local/share/guile/site/2.2"
instructs the build script to install the guile bindings into
"/usr/local/share/guile/site/2.2".

Sure. This is reasonable, but for the sake of completeness, I'll point
out two disadvantages with this approach, namely that (1) it entails
effectively overriding Debian's GnuTLS library with your manually built
version, which potentially affects the operation of any Debian package
that links to GnuTLS and (2) it means staying on top of security updates
yourself, i.e. recompiling and installing new versions of GnuTLS or the
bundled copies of libtasn1 and libunistring when security flaws are
discovered in those versions. Ditto for the other packages that you
build and install manually.

Regards,
Mark

Alex Vong

2018-11-17 08:26:38 UTC

Permalink

Post by Mark H Weaver
Hi Alex,

Agree, while (1) seems to not causing problems for me in practice, (2)
is a very important point to keep in mind. Btw, this bug report[0]
explains why the guile bindings were removed. Although it's mark as
wontfix, should we encourage the maintainers to re-enable the guile
bindings in the experimental repository? The problems could perhaps be
solved by building the bindings with guile 2.2.

[0]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=863147

Post by Mark H Weaver
Regards,
Mark

Mark H Weaver

2018-11-17 21:34:07 UTC

Permalink

Hi Alex,

Post by Alex Vong
Btw, this bug report[0]
explains why the guile bindings were removed. Although it's mark as
wontfix, should we encourage the maintainers to re-enable the guile
bindings in the experimental repository?

Based on:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=863147#10

I think the Debian maintainers have made it clear that they will not
re-enable the Guile bindings until the underlying bug is fixed, which
apparently causes intermittent failures in GnuTLS's test suite when
Guile is built with certain compile flags, as described here:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=805863#84

Post by Alex Vong
The problems could perhaps be solved by building the bindings with
guile 2.2.

Sure, this would be a worthwhile thing to try, but we should probably
try it ourselves before asking the Debian developers to spend time on
it. I can't try it easily because I use GuixSD, but if there are
interested users of Debian and Guile here, it would be a great help to
try reproducing the problem described in Debian bug 805863, and seeing
if the problem can be avoided by building GnuTLS with Debian's Guile-2.2
package.

Regards,
Mark

Post by Alex Vong
[0]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=863147

Ludovic Courtès

2018-11-20 20:50:54 UTC

Permalink

Hello,

Post by Zelphir Kaltstahl
Now I don't know where I would get the Guile bindings from. Are they
included in the releases on https://gnutls.org/ ?

Yes, the Guile bindings are part of GnuTLS itself. See also the
instructions at
<https://gnutls.org/manual/gnutls-guile/Guile-Preparations.html>.

Now, as Mark notes, if you want to install Guix, I strongly recommend
using the binary installation tarball.

Thanks,
Ludo’.