How to make GNU Guile more successful

[Amirouche]

I don't know why Racket is successful probably because
it has a very good documentation and also a documentation
generator written in scheme.

=> This is a long debate and core principle of GNU project
so I don't want to enter that debate

And it's easy to packages.

=> Another pain topic. In reality it's very easy to package
all sort of guile programs for guix. But guix itself needs
to be more stable.

Python is successful because it has a massive number
of libraries. In particular it has dozens of:

- web framework
- library to generate diagrams and graphical stuff
- scientific libraries (machine learning and others)
- books which teach python (the good enough way)
- 2D and 3D libraries for games

=> We (all the people reading this mail) can not just
create all those projects. It requires to many domain
knowledge for a handful of people to be able to compete.
But what we can do is *share*, *share* and *share*
again *all* what we do (on mailing list, blogs, on gitlab
or whatever) whether it is a full blown project with a
website made by a professional designer with tests suite,
a tutorial documentation reference doc and a community or
a simple *snippet* or broken experiments. I have lost
(read deleteted) at least the code of 5 or 6 experiments
only to recognize later that it could be useful for future
me.

=> In terms of software I really think, we need something like
ActiveState Python Cookbook [0]. I still go there sometime even
I could simply ask the question on StackOverflow and get an
answer in minutes.

[0] http://code.activestate.com/recipes/langs/python/

During a long time I was said that Pythonist were
among the smartest developer, because simply they had
the guts to learn a language that was not learned in
school.

=> this can be true for schemers;

Python *was* easy to learn.

=> Scheme is easy to learn if you stick in the "garden"
(garden must be defined but it's basically what I
introduce in my tutorial [1]) Which is totally the
same for the Python of today (read Python 3.5+).

[1] http://hyperdev.fr

PHP was successful back in the days because it was easy
to install,

=> A scheme web application is much easier to run!

Now PHP is successful because there is a few successful
software written in it. By successful I mean they GTD.
They aren't perfect, you get bitten etc. But the basic
are there. I think about Drupal, Wordpress and Prestashop.

=> While you might not be interested to build something
like prestashop. You very much be interested to build
something like Drupal or Wordpress to allow mom and
dad and others create a personnal experience on the
web.

Another thing, Python has GNU Media Goblin

=> We must do better!

WDYT?

What block you from contributing to the wide ecosystem of GNU Guile?

Best regards,

Amirouche

[Amirouche]

Post by Amirouche
WDYT?
What block you from contributing to the wide ecosystem of GNU Guile?

I heared a lot around me about the projects I was doing before in my
previous life in as Pythonista. That I had NIH syndrom. Yeah, I rebuild
stuff that already exists hence according to the "haters" that was not
useful.

=> THIS IS A FALSE. Fully FUD! Two ways you can learn a programming
a) coding b) reading code. Basically in Guile, if there is no code to read
you only have the other option.

=> Remember they were dozen wikis before wikipedia with similar intents,
dozen search engines before Google with similar intents; You never reinvent
the wheel. You update it little by little, it might not be revolution.
Look at paintings for instance, it's seems some painters do not know proper
painting but they are still successful in their niche. Why? Because they tried.
My advice is to aim big or small whatever, but don't be shy to share.

I personnaly don't believe that code pollution exists. My theory is that
this concept was invented by techno oligarcs to limit the creativity of
those that can make. Basicly techno overlords supply the tools and you provide
the finished product that they are interested in. This is not in the interest
of creativity or liberty. Code what you want to code and think you need to code.
Don't be shy. Javascript fatigue is another name for code pollution.

Another thing i heard a lot is: It's impossible.

=> I was said that about Python->Javascript translator dozen of times. About
my project to create a new web framework too. Basically Django is too
mainstream. I don't want to be bigger than Django I want to have the
*correct* solution. In many situations, for me, django was not up to the
mark.

Another thing I hear a lot: nothing. Yeah. Nothing.

=> It take a lot of courage to do free and open source software. First,
you might find it difficult to find people that have the same interest
as you. Second they probably perfer to fork and do it from scratch by
themself. Maybe they are shy and do it secretly aka. they don't want to
disturb the matrix.

You will also maybe meet people that hate open source or worst that hate
GNU. Leave if possible.

That said I don't say HACK ALL DAY until something BAD happens. I say love
the code and the community share what you did.

[Arne Babenhauserheide]

Hi Amirouche,

Thank you for your nice writeup!

Post by Amirouche
I don't know why Racket is successful probably because
it has a very good documentation and also a documentation
generator written in scheme.
=> This is a long debate and core principle of GNU project
so I don't want to enter that debate

But I want to add something: I’m getting the hang of just checking the
info-page to look something up. And I realize that with well-written
info-pages this is faster than googling for stack overflow.

Post by Amirouche
And it's easy to packages.
=> Another pain topic. In reality it's very easy to package
all sort of guile programs for guix. But guix itself needs
to be more stable.

I wish I had guildhall ready. Got hit by
time-eaten-by-other-project-because-guildhall-was-priority-three.

It needs to be easy to not just package some code, but also to share
that package without hitting a bottleneck.

Post by Amirouche
Python is successful because it has a massive number
of libraries.

…

Post by Amirouche
=> We (all the people reading this mail) can not just
create all those projects. It requires to many domain
knowledge for a handful of people to be able to compete.
But what we can do is *share*, *share* and *share*
again *all* what we do (on mailing list, blogs, on gitlab
or whatever) whether it is a full blown project with a
website made by a professional designer with tests suite,
a tutorial documentation reference doc and a community or
a simple *snippet* or broken experiments.

A really simple way to share a project would be nice here. Nowadays many
people just push something with a README to github, but github is
unfree.

Post by Amirouche
=> In terms of software I really think, we need something like
ActiveState Python Cookbook [0]. I still go there sometime even
I could simply ask the question on StackOverflow and get an
answer in minutes.
[0] http://code.activestate.com/recipes/langs/python/
During a long time I was said that Pythonist were
among the smartest developer, because simply they had
the guts to learn a language that was not learned in
school.

I never heard that one :)

Post by Amirouche
Python *was* easy to learn.
=> Scheme is easy to learn if you stick in the "garden"
(garden must be defined but it's basically what I
introduce in my tutorial [1]) Which is totally the
same for the Python of today (read Python 3.5+).

Or look at py2guile: http://draketo.de/py2guile

Python is no longer easy due to integrating additional syntactic forms,
each of which makes a specific kind of code much nicer but complicates
learning.

We have something similar in foof-loop:

(define (count-matching-items list predicate)
(loop ((for item (in-list list))
(with count 0
(if (predicate item)
(+ count 1)
count)))
=> count))
; ^^ here

This looks really simple in this place, but it introduces special syntax
which will be unintelligible for someone who does did not explicitly
learn foof-loop.

And guildhall is filled with special forms I had to understand just to
be able to do small changes.

This is a social problem. For a deeper discussion why it is a problem,
see http://winestockwebdesign.com/Essays/Lisp_Curse.html
"Lisp is so powerful that problems which are technical issues in other
programming languages are social issues in Lisp."

We can counter this with easy tutorials and with writing something like
canonical Scheme. But for this, we need to define a canonical Scheme
which is hits the performance and readability sweet-spot for
Guile. Canonical code must be close to the fastest code.

Practically put: We need Andy Wingo to nitpick the tutorial about things
which will cause overheads the compiler cannot fix easily — including
expensive use of macros.

Post by Amirouche
[1] http://hyperdev.fr
PHP was successful back in the days because it was easy
to install,

Or because it was the only thing which existed. Didn’t it make it easy
for the hoster to sandbox it — both in terms of access and in terms of
resource requirements?

Post by Amirouche
=> A scheme web application is much easier to run!

Sadly not on my cheap webhoster which provides unlimited bandwidth for
a fixed cost.

Post by Amirouche
=> While you might not be interested to build something
like prestashop. You very much be interested to build
something like Drupal or Wordpress to allow mom and
dad and others create a personnal experience on the
web.
Another thing, Python has GNU Media Goblin
=> We must do better!

Or, maybe collaborate with it? I’m sure that there’s lots of stuff which
could be shared, because it is not language-specific (more exactly: it’s
HTML, CSS, Javascript and images).

Post by Amirouche
What block you from contributing to the wide ecosystem of GNU Guile?

- publishing a program for non-Linux platforms (well, Windows, OSX and
mobile phones) so most of my friends can use it.

- an alternative to `python setup.py register sdist upload`

Best wishes,
Arne

[Arne Babenhauserheide]

Post by Arne Babenhauserheide
- an alternative to `python setup.py register sdist upload`

Since we are GNUfolks, we can leverage autotools for that. I have a
pet-project which automates the setup of autotools-projects in the style
of Rust Cargo (but without the NIH). I should share it …

Best wishes,
Arne

[Amirouche]

Post by Arne Babenhauserheide
Hi Amirouche,
Thank you for your nice writeup!

Post by Amirouche
I don't know why Racket is successful probably because
it has a very good documentation and also a documentation
generator written in scheme.
=> This is a long debate and core principle of GNU project
so I don't want to enter that debate

But I want to add something: I’m getting the hang of just checking the
info-page to look something up. And I realize that with well-written
info-pages this is faster than googling for stack overflow.

My peers have this habit during system programming looking at man
pages and other stuff. I don't know much command line-fu. My experience,
about programming:

- looking up the API referennce (procedure index)
- looking up ansers in stack overflow

In the case of Guile programming I also look general scheme solution like
in racket. Last time I checked they had a Racket->JS translator, but not
much graphdb stuff.

The ability to generate multiple format is what makes TEXINFO powerful.

Post by Arne Babenhauserheide

Post by Amirouche
And it's easy to packages.
=> Another pain topic. In reality it's very easy to package
all sort of guile programs for guix. But guix itself needs
to be more stable.

I wish I had guildhall ready. Got hit by
time-eaten-by-other-project-because-guildhall-was-priority-three.
It needs to be easy to not just package some code, but also to share
that package without hitting a bottleneck.

I got stuck with guile-git; because of new way of bindings C. I use
guile-bytestructures.

Post by Arne Babenhauserheide

Post by Amirouche
Python is successful because it has a massive number
of libraries.

…

Post by Amirouche
=> We (all the people reading this mail) can not just
create all those projects. It requires to many domain
knowledge for a handful of people to be able to compete.
But what we can do is *share*, *share* and *share*
again *all* what we do (on mailing list, blogs, on gitlab
or whatever) whether it is a full blown project with a
website made by a professional designer with tests suite,
a tutorial documentation reference doc and a community or
a simple *snippet* or broken experiments.

A really simple way to share a project would be nice here. Nowadays many
people just push something with a README to github, but github is
unfree.

I try to move to framagit. but I still use github sometimes.

Post by Arne Babenhauserheide

Post by Amirouche
=> In terms of software I really think, we need something like
ActiveState Python Cookbook [0]. I still go there sometime even
I could simply ask the question on StackOverflow and get an
answer in minutes.
[0] http://code.activestate.com/recipes/langs/python/
During a long time I was said that Pythonist were
among the smartest developer, because simply they had
the guts to learn a language that was not learned in
school.

I never heard that one :)

Post by Amirouche
Python *was* easy to learn.
=> Scheme is easy to learn if you stick in the "garden"
(garden must be defined but it's basically what I
introduce in my tutorial [1]) Which is totally the
same for the Python of today (read Python 3.5+).

Or look at py2guile: http://draketo.de/py2guile
Python is no longer easy due to integrating additional syntactic forms,
each of which makes a specific kind of code much nicer but complicates
learning.
(define (count-matching-items list predicate)
(loop ((for item (in-list list))
(with count 0
(if (predicate item)
(+ count 1)
count)))
=> count))
; ^^ here
This looks really simple in this place, but it introduces special syntax
which will be unintelligible for someone who does did not explicitly
learn foof-loop.
And guildhall is filled with special forms I had to understand just to
be able to do small changes.
This is a social problem. For a deeper discussion why it is a problem,
see http://winestockwebdesign.com/Essays/Lisp_Curse.html
"Lisp is so powerful that problems which are technical issues in other
programming languages are social issues in Lisp."

I didn't know that one

Post by Arne Babenhauserheide
We can counter this with easy tutorials and with writing something like
canonical Scheme. But for this, we need to define a canonical Scheme
which is hits the performance and readability sweet-spot for
Guile. Canonical code must be close to the fastest code.
Practically put: We need Andy Wingo to nitpick the tutorial about things
which will cause overheads the compiler cannot fix easily — including
expensive use of macros.

Right now, I don't have performance issues. Except maybe with the crawler code.

I improved stream srfi-41 performance, if you want the code.

Post by Arne Babenhauserheide

Post by Amirouche
[1] http://hyperdev.fr
PHP was successful back in the days because it was easy
to install,

Or because it was the only thing which existed. Didn’t it make it easy
for the hoster to sandbox it — both in terms of access and in terms of
resource requirements?

Post by Amirouche
=> A scheme web application is much easier to run!

Sadly not on my cheap webhoster which provides unlimited bandwidth for
a fixed cost.

Ok, I have a vps, I simply spawn the guile application in a screen and
point nginx to it. It's like configuring nginx for gunicorn. nginx forward
the traffic from :80 to :8080 or whatever based on some rules. It's like
a router for the ports being smart about http protocol.

Post by Arne Babenhauserheide

Post by Amirouche
=> While you might not be interested to build something
like prestashop. You very much be interested to build
something like Drupal or Wordpress to allow mom and
dad and others create a personnal experience on the
web.
Another thing, Python has GNU Media Goblin
=> We must do better!

Or, maybe collaborate with it?

There is the protocol Chris is working at w3c which is interesting.
I forgot the name :p but it looked cool. At some point, I'd like to talk
some kind of federation protocol. The issue is that they require some
identification, so that people can name you unlike 4chan. Yes my current
project is like a text 4chan with a search engine.

Post by Arne Babenhauserheide
I’m sure that there’s lots of stuff which
could be shared, because it is not language-specific (more exactly: it’s
HTML, CSS, Javascript and images).

I love their website and logo!

Post by Arne Babenhauserheide

Post by Amirouche
What block you from contributing to the wide ecosystem of GNU Guile?

- publishing a program for non-Linux platforms (well, Windows, OSX and
mobile phones) so most of my friends can use it.
- an alternative to `python setup.py register sdist upload`
Best wishes,
Arne

[Arne Babenhauserheide]

Post by Amirouche
My peers have this habit during system programming looking at man
pages and other stuff. I don't know much command line-fu. My experience,
- looking up the API referennce (procedure index)
- looking up ansers in stack overflow

I used to do that. But Emacs C-h i C-s <search phrase> is becoming more
and more efficient for me (since I know more and more keywords).

I still google stuff, but less than in Python. It feels like the info
documentation is actually better.

Post by Amirouche
In the case of Guile programming I also look general scheme solution like
in racket. Last time I checked they had a Racket->JS translator, but not
much graphdb stuff.
The ability to generate multiple format is what makes TEXINFO powerful.

Yes.

Post by Amirouche

Post by Arne Babenhauserheide
It needs to be easy to not just package some code, but also to share
that package without hitting a bottleneck.

I got stuck with guile-git; because of new way of bindings C. I use
guile-bytestructures.

It would be great if you could simply run "make binary" and get binaries
for multiple platforms which you can provide for different people.

But that’s just what blocks me the most.

Post by Amirouche

Post by Arne Babenhauserheide
A really simple way to share a project would be nice here. Nowadays many
people just push something with a README to github, but github is
unfree.

I try to move to framagit. but I still use github sometimes.

I did not hear from that until today. I prefer Mercurial with my
self-written site extension[1] for static http repositories — at least
as a backup.

[1]: https://www.mercurial-scm.org/wiki/SiteExtension

Post by Amirouche

Post by Arne Babenhauserheide
Practically put: We need Andy Wingo to nitpick the tutorial about things
which will cause overheads the compiler cannot fix easily — including
expensive use of macros.

Right now, I don't have performance issues. Except maybe with the crawler code.
I improved stream srfi-41 performance, if you want the code.

Sounds good. Can you get that into Guile? Having the default
implementation more efficient would help me the most (in case I need
streams at some point).

Post by Amirouche

Post by Arne Babenhauserheide

Post by Amirouche
=> A scheme web application is much easier to run!

Sadly not on my cheap webhoster which provides unlimited bandwidth for
a fixed cost.

Ok, I have a vps, I simply spawn the guile application in a screen and
point nginx to it. It's like configuring nginx for gunicorn. nginx forward
the traffic from :80 to :8080 or whatever based on some rules. It's like
a router for the ports being smart about http protocol.

That’s like a remote server with full access. What I have is FTP
access. But in exchange my sites can cope with serving several hundreds
of GiB of data per month.

Post by Amirouche
There is the protocol Chris is working at w3c which is interesting.
I forgot the name :p but it looked cool. At some point, I'd like to talk
some kind of federation protocol. The issue is that they require some
identification, so that people can name you unlike 4chan. Yes my current
project is like a text 4chan with a search engine.

In GNU social you can simply use ***@instance.tld

But this is again standard proliferation. I already moaned when
identi.ca turned away from status.net. The new pump.io interface never
worked well for me :(

Post by Amirouche
I love their website and logo!

:)

Best wishes,
Arne

[Panicz Maciej Godek]

Post by Arne Babenhauserheide
Hi Amirouche,
Thank you for your nice writeup!

Post by Amirouche
I don't know why Racket is successful probably because
it has a very good documentation and also a documentation
generator written in scheme.
=> This is a long debate and core principle of GNU project
so I don't want to enter that debate

But I want to add something: I’m getting the hang of just checking the
info-page to look something up. And I realize that with well-written
info-pages this is faster than googling for stack overflow.

Once I was strucked by this:
http://www.wilfred.me.uk/blog/2016/07/30/example-driven-development/

There's also a similar project by William Byrd that could make the
difference
https://github.com/webyrd/Barliman

I think that Python is popualr because you don't need Emacs to work with it.
And that the milestone to the success of Racket, apart from intensive
propaganda, is Dr Racket.
(also, an important factor is that you can install both Python and Racket
on Windows, Linux
and OSX, and it Just Works)

The problem with Emacs is that it is that, although you probably can do
everything imaginable with it,
it is difficult to approach. The problem with the problem wth Emacs is that
once you get it, you usually
have better ideas in mind than making Emacs more approachable.

There's also this problem with Scheme that it is a very diverse community
with plethora
of implementations. And while I use Guile, I don't mind using Chicken or
Chez or Gambit
for various purposes. Or even Stalin sometimes. Maybe, instead of getting a
better compiler,
we should focus on integrating existing compilers?

Post by Arne Babenhauserheide
And it's easy to packages.

Post by Amirouche
=> Another pain topic. In reality it's very easy to package
all sort of guile programs for guix. But guix itself needs
to be more stable.

I wish I had guildhall ready. Got hit by
time-eaten-by-other-project-because-guildhall-was-priority-three.
It needs to be easy to not just package some code, but also to share
that package without hitting a bottleneck.

What happened to initiatives like Scheme NOW, that tried to integrate
various groups of developers.
Or why can't we just "steal" eggs from Chicken? (I mean it. They already
have that problem solved)

Post by Arne Babenhauserheide

Post by Amirouche
Python is successful because it has a massive number
of libraries.

This is a social problem. For a deeper discussion why it is a problem,
see http://winestockwebdesign.com/Essays/Lisp_Curse.html
"Lisp is so powerful that problems which are technical issues in other
programming languages are social issues in Lisp."
We can counter this with easy tutorials and with writing something like
canonical Scheme. But for this, we need to define a canonical Scheme
which is hits the performance and readability sweet-spot for
Guile. Canonical code must be close to the fastest code.

Practically put: We need Andy Wingo to nitpick the tutorial about things

Post by Arne Babenhauserheide
which will cause overheads the compiler cannot fix easily — including
expensive use of macros.

I definitely oppose. If Chez has something solved better, why not use Chez?
The ultimate goal is not to optimize programs, but programmers.
But I agree on the point of the canonical code. I think that the "zen of
python"
is one of important factors of making it popular. I think it could be the
same
with Scheme, but here the problem is that people have various ideas
with regard to how the "ideal" program should look like.

For example, I have a very strong opinion with regard to the use of pattern
matching
and restructuring -- I try to use destructuring almost everywhere (from
what I've seen,
Clojure programmers are alike with this regard). However, I am aware that
there are
people who don't like it that much, like Taylan or Barry Margolin.

Also, I think that the astounding success of R provides hints with regard
what is needed
for a programming language to gain popularity. R has a large repository of
packages
that fills a certain niche. When I advertised my Pamphlet on Hacker News,
someone
responded critically: "are there out of the box libraries to estimate a
zero inflated negative
binomial regression model in guile". Of course, if I knew what a
zero-inflated
negative binomial regression model, I could deliver an implementation by
just explaining
the notions used in that phrase. (But then again, I try to write functional
programs whenever
possible, often consciously sacrificing time complexity of my solutions,
and I'm aware
that not everybody will like it.)

Thanks,
Panicz

[Amirouche]

Post by Arne Babenhauserheide
Hi Amirouche,
Thank you for your nice writeup!

Post by Amirouche
I don't know why Racket is successful probably because
it has a very good documentation and also a documentation
generator written in scheme.
=> This is a long debate and core principle of GNU project
so I don't want to enter that debate

But I want to add something: I’m getting the hang of just checking the
info-page to look something up. And I realize that with well-written
info-pages this is faster than googling for stack overflow.
http://www.wilfred.me.uk/blog/2016/07/30/example-driven-development/
There's also a similar project by William Byrd that could make the
difference
https://github.com/webyrd/Barliman

This is awesome!

[Marko Rauhamaa]

Post by Panicz Maciej Godek
I think that Python is popualr because you don't need Emacs to work with it.

You do need a special editor to work with Python, and emacs is perfect
for it.

Python is popular because it is a very traditional, dynamic, high-level
programming language. It is conceptually very similar to Lisp
(especially coupled with JSON). Definitely a gateway drug to Lisp.

Post by Panicz Maciej Godek
The problem with Emacs is that it is that, although you probably can
do everything imaginable with it, it is difficult to approach.

I never noticed that problem with emacs. Emacs is really the only
reasonable way to type text. For example, when I enter text of any
length to a web app, I first type it in emacs and then copy it to the
web form.

I have typed this message in emacs.

Post by Panicz Maciej Godek
There's also this problem with Scheme that it is a very diverse
community with plethora of implementations.

Scheme might have a chance as people's tastes evolve. Python is an
example of that evolution.

In my opinion one of the worst problems with Scheme is the Schemers:
Scheme lovers are often far too enthusiastic with defining new, esoteric
syntax instead of solving practical problems.

Then, there's GOOPS, which in my opinion is simply an unnatural way to
go about object-oriented programming. It does violence both to ordinary
OO way of thinking and classic Lisp idioms.

Continuations and multiple values are obstacles instead of enablers.

BTW, Python seems to be suffering from quite many self-inflicted wounds:
Python2/Python3 incompatibility, asyncio, type annotation, decorators,
dunder jungle, meta-object programming, Unicode illusion. (Guile has
fallen into that last trap as well, unfortunately.) In its eagerness to
please everyone, Python might well be jumping the shark.

There's one thing Python has over Guile, still: Python's system call
support is outstanding. For example, I managed to implement a full-blown
shell using Python. That was the first time I ever ran into
terminal-related system calls, and Python had them all.


Marko

[Arne Babenhauserheide]

Post by Marko Rauhamaa
I have typed this message in emacs.

Same for me, but getting people to use Emacs is harder. It might not
*be* that complicated, but it *feels* different.

Post by Marko Rauhamaa
Scheme lovers are often far too enthusiastic with defining new, esoteric
syntax instead of solving practical problems.
Then, there's GOOPS, which in my opinion is simply an unnatural way to
go about object-oriented programming. It does violence both to ordinary
OO way of thinking and classic Lisp idioms.

GOOPS works pretty well for me where I use it (for dispatch by
type). Could you clarify your criticism: Do you think it is bad or is it
just different?

Post by Marko Rauhamaa
Continuations and multiple values are obstacles instead of enablers.

I think multiple values are nice. But they are not well-integrated (I
have to import something to use them). Why do you think them enablers?

Why do you think that continuations are an obstacle (do you mean general
continuations or do you mean prompts)?

Post by Marko Rauhamaa
Python2/Python3 incompatibility,

Py2 vs. Py3 really hurts, yes.

It still really hurts and will likely hurt for the next 10 years. I
think that should be a lesson for all other language-implementors: Once
you start to become popular, you cannot do any backwards-incompatible
changes without endangering your position. Once you start to get
traction, you have to stick to your horse.

Post by Marko Rauhamaa
asyncio, type annotation,

type annotations should provide nice ways to optimize for pypy and other
more optimizing Python implementations.

Post by Marko Rauhamaa
decorators,

Decorators are really cool to use:

@with_progress
def something():
time.sleep(10)

That’s almost like

(with-progress
(define (something)
(sleep 10)))

But decorators are actually definition time, so they bring the tower of
meta-levels to Python 


Post by Marko Rauhamaa
dunder jungle,

What’s that?

Post by Marko Rauhamaa
meta-object programming,

That’s needed to make some things elegant. "Different 20%"-rule again.

Post by Marko Rauhamaa
Unicode illusion. (Guile has fallen into that last trap as well,
unfortunately.)

I’m using Python and Guile for physics and math (to some degree), and
having good Unicode-support is great for that. What’s your practical
criticism here?

Post by Marko Rauhamaa
In its eagerness to please everyone, Python might well
be jumping the shark.
There's one thing Python has over Guile, still: Python's system call
support is outstanding. For example, I managed to implement a full-blown
shell using Python. That was the first time I ever ran into
terminal-related system calls, and Python had them all.

Which ones are missing in Guile?

Best wishes,
Arne

[David Kastrup]

Post by Arne Babenhauserheide

Post by Marko Rauhamaa
I have typed this message in emacs.

Same for me, but getting people to use Emacs is harder. It might not
*be* that complicated, but it *feels* different.

Post by Marko Rauhamaa
Scheme lovers are often far too enthusiastic with defining new, esoteric
syntax instead of solving practical problems.
Then, there's GOOPS, which in my opinion is simply an unnatural way to
go about object-oriented programming. It does violence both to ordinary
OO way of thinking and classic Lisp idioms.

GOOPS works pretty well for me where I use it (for dispatch by
type). Could you clarify your criticism: Do you think it is bad or is it
just different?

My main beef with GOOPS is that it does not help with narrowing down on
a solution but rather with extending the problem space. It is too
generic to provide guidance and a cohesive framework: if two different
people solve problems using GOOPS, the likelihood that those solutions
can be connected in interface or design better than non-GOOPS solutions
is slim.

[Arne Babenhauserheide]

Post by David Kastrup

Post by Arne Babenhauserheide

Post by Marko Rauhamaa
I have typed this message in emacs.

Same for me, but getting people to use Emacs is harder. It might not
*be* that complicated, but it *feels* different.

Post by Marko Rauhamaa
Scheme lovers are often far too enthusiastic with defining new, esoteric
syntax instead of solving practical problems.
Then, there's GOOPS, which in my opinion is simply an unnatural way to
go about object-oriented programming. It does violence both to ordinary
OO way of thinking and classic Lisp idioms.

GOOPS works pretty well for me where I use it (for dispatch by
type). Could you clarify your criticism: Do you think it is bad or is it
just different?

My main beef with GOOPS is that it does not help with narrowing down on
a solution but rather with extending the problem space. It is too
generic to provide guidance and a cohesive framework: if two different
people solve problems using GOOPS, the likelihood that those solutions
can be connected in interface or design better than non-GOOPS solutions
is slim.

That’s a good point.

A framework which provides that guidance could be built on top of GOOPS,
though. Or would that carry around too much burden from the generic
approach it’s based on?

Best wishes,
Arne

[Christopher Allan Webber]

Post by David Kastrup

Post by Arne Babenhauserheide

Post by Marko Rauhamaa
I have typed this message in emacs.

Same for me, but getting people to use Emacs is harder. It might not
*be* that complicated, but it *feels* different.

Post by Marko Rauhamaa
Scheme lovers are often far too enthusiastic with defining new, esoteric
syntax instead of solving practical problems.
Then, there's GOOPS, which in my opinion is simply an unnatural way to
go about object-oriented programming. It does violence both to ordinary
OO way of thinking and classic Lisp idioms.

GOOPS works pretty well for me where I use it (for dispatch by
type). Could you clarify your criticism: Do you think it is bad or is it
just different?

My main beef with GOOPS is that it does not help with narrowing down on
a solution but rather with extending the problem space. It is too
generic to provide guidance and a cohesive framework: if two different
people solve problems using GOOPS, the likelihood that those solutions
can be connected in interface or design better than non-GOOPS solutions
is slim.

Generic methods help somewhat with this though right? That's part of
the goal of generic methods even; the portion of SICP where they are
introduced is even about getting two programmers who don't necessarily
get along to get their code to work together.

[sirgazil]

Post by Amirouche
What block you from contributing to the wide ecosystem of GNU Guile?

I think about Guile (and GuixSD) almost every day, but I don't contribute as much as I'd like mostly because of lack of resources to do so. Sadly, it's hard to find the opportunity to immerse yourself in libre projects.

As for how to make GNU Guile more successful, I'd like to see what I call a "Hello Guile" series of guides targeted at people who just want to start programming. The series would include:

+ Hello Guile: Getting Started (An overview of programming, and how to start with Guile)
+ Hello Guile: Let's develop a console application
+ Hello Guile: Let's develop a desktop application
+ Hello Guile: Let's develop a web application
+ Hello Guile: Let's develop a 2D game
+ Hello Guile: Let's package your software for distribution

Additionally, I'd like the following:

+ More high level libraries [1]
+ Add Guile to the GNOME Developer Center [2][3][4]
+ Package more guile libraries in Guix

Finally, we need to clone a bunch of people to do all this work :)


[1]: For example, I'm following the Systematic Program Design course with
Gregor Kiczales. The course uses DrRacket for interactive
programming and a language called BSL (Beginning Student
Language).

When I started, my plan was to follow the course in Guile instead,
since the course is about the design method, and not the tools, but
I had to stop very early when I had to do some projects involving
graphics and animation.

Personally, I don't feel the need for something like DrRacket,
but it would have been nice to be able to find a Guile "canvas" or a
2D game library to play easily with basic graphics and animations.
[2]: https://developer.gnome.org/
[3]: https://developer.gnome.org/gnome-devel-demos/stable/
[4]: https://developer.gnome.org/references (see Language Bindings section)



---
https://sir

[Amirouche]

Post by sirgazil

Post by Amirouche
What block you from contributing to the wide ecosystem of GNU Guile?

I think about Guile (and GuixSD) almost every day, but I don't contribute as much as I'd like mostly because of lack of resources to do so. Sadly, it's hard to find the opportunity to immerse yourself in libre projects.
+ Hello Guile: Getting Started (An overview of programming, and how to start with Guile)
+ Hello Guile: Let's develop a console application
+ Hello Guile: Let's develop a desktop application
+ Hello Guile: Let's develop a web application
+ Hello Guile: Let's develop a 2D game
+ Hello Guile: Let's package your software for distribution
+ More high level libraries [1]
+ Add Guile to the GNOME Developer Center [2][3][4]
+ Package more guile libraries in Guix
Finally, we need to clone a bunch of people to do all this work :)
[1]: For example, I'm following the Systematic Program Design course with
Gregor Kiczales. The course uses DrRacket for interactive
programming and a language called BSL (Beginning Student
Language).
When I started, my plan was to follow the course in Guile instead,
since the course is about the design method, and not the tools, but
I had to stop very early when I had to do some projects involving
graphics and animation.
Personally, I don't feel the need for something like DrRacket,
but it would have been nice to be able to find a Guile "canvas" or a
2D game library to play easily with basic graphics and animations.
[2]: https://developer.gnome.org/
[3]: https://developer.gnome.org/gnome-devel-demos/stable/
[4]: https://developer.gnome.org/references (see Language Bindings section)
---
https://sirgazil.bitbucket.io/

I have to +1 all of this post.

[Marko Rauhamaa]

Post by Marko Rauhamaa
Then, there's GOOPS, which in my opinion is simply an unnatural way
to go about object-oriented programming. It does violence both to
ordinary OO way of thinking and classic Lisp idioms.

GOOPS works pretty well for me where I use it (for dispatch by type).
Could you clarify your criticism: Do you think it is bad or is it just
different?

GOOPS starts by defining slots. Slots are an objects internal business
and should not be visible to the outside. Instead, objects should be
black boxes that interact with methods. For example, the port interface
is a nice, classic OO API. You don't know anything about the slots of a
port.

Furthermore, I think an extensive type system is un-Lisp-like. OO is not
about things belonging to a type but objects interacting through
methods. In fact, the concept of a "class" could be done away with.

Post by Marko Rauhamaa
Continuations and multiple values are obstacles instead of enablers.

I think multiple values are nice. But they are not well-integrated (I
have to import something to use them). Why do you think them enablers?

Yes, the multiple values API is a problem in and of itself, but more
generally, what does:

(values a b c)

give you on top of, say:

(list a b c)

Why do you think that continuations are an obstacle (do you mean
general continuations or do you mean prompts)?

Continuations prevent you from implementing Python's try/finally and
emacs' (unwind-protect).

On the other hand, I don't think any application developer would come to
a point of thinking, "Hey, I know, I'll use a continuation!"

Post by Marko Rauhamaa
asyncio, type annotation,

type annotations should provide nice ways to optimize for pypy and
other more optimizing Python implementations.

You can't have it bothways. A dynamic language liberates you from
shackles and boilerplate even if it destroys your performance. The
moment you bring in the boilerplate, you are back to Java.
Have yet to find a need for one.

Post by Marko Rauhamaa
dunder jungle,

What’s that?

<URL: https://docs.python.org/3/genindex-_.html>

Post by Marko Rauhamaa
meta-object programming,

That’s needed to make some things elegant. "Different 20%"-rule again.

Again, I haven't yet found a need to deviate from the regular object
semantics.

Post by Marko Rauhamaa
Unicode illusion. (Guile has fallen into that last trap as well,
unfortunately.)

I’m using Python and Guile for physics and math (to some degree), and
having good Unicode-support is great for that. What’s your practical
criticism here?

In Linux, all pathnames, files, sockets, environment variables and
command-line arguments are, or deal with, bytes. The bytes are often
encoded with UTF-8, but a programming language cannot assume that, even
if LOCALE promises otherwise.

It would be better to leave Unicode out of Guile's system interface and
have the application encode and decode explicitly where needed.

Post by Marko Rauhamaa
There's one thing Python has over Guile, still: Python's system call
support is outstanding. For example, I managed to implement a
full-blown shell using Python. That was the first time I ever ran
into terminal-related system calls, and Python had them all.

Which ones are missing in Guile?

For example tcgetattr(3). (Ok, not a system call.)

Missing epoll(2) is inconvenient. Not being able to transfer an open
file descriptor via sendmsg(2) is really bad.


Marko

[Linas Vepstas]

Hey,

Post by Marko Rauhamaa

Post by Marko Rauhamaa
Then, there's GOOPS, which in my opinion is simply an unnatural way
to go about object-oriented programming. It does violence both to
ordinary OO way of thinking and classic Lisp idioms.

GOOPS works pretty well for me where I use it (for dispatch by type).
Could you clarify your criticism: Do you think it is bad or is it just
different?

GOOPS starts by defining slots. Slots are an objects internal business
and should not be visible to the outside. Instead, objects should be
black boxes that interact with methods. For example, the port interface
is a nice, classic OO API. You don't know anything about the slots of a
port.

Yes, but ... maybe goops should be renamed guijs ? Javascript has this
... wonderful ... ability to just attach new data and new methods to any
"object" at runtime. Now, I've never used goops in any serious way, so
maybe I'm quite off the mark, but it seems to me that maybe this is what
slots are trying to accomplish.

That is, when you say "classic OO API", it seems you really have C++ in
the back of your mind. But the early days of OO had all sorts of weird ways
of saying things and thinking about things (e.g. "sending messages to objects")
that was a bigger, more inclusive tent. Perhaps GOOPS is just OO in this
older, and now deprecated sense.

My naive impression is that some javascrtipt-ish extension to guile would
be very cool. Perhaps goops could be pushed further in that direction.

Post by Marko Rauhamaa

Post by Marko Rauhamaa
Continuations and multiple values are obstacles instead of enablers.

I think multiple values are nice. But they are not well-integrated (I
have to import something to use them). Why do you think them enablers?

Yes, the multiple values API is a problem in and of itself, but more
(values a b c)
(list a b c)

+1 every time I think I need multiple values, I try it, and then back
away from it.
Its a cool-sounding idea that is a consistent obstacle.

Post by Marko Rauhamaa

Why do you think that continuations are an obstacle (do you mean
general continuations or do you mean prompts)?

Continuations prevent you from implementing Python's try/finally and
emacs' (unwind-protect).
On the other hand, I don't think any application developer would come to
a point of thinking, "Hey, I know, I'll use a continuation!"

Continuations are stunningly-badly explained, almost everywhere. More
or less every example I've ever seen is pointlessly obtuse and opaque,
seemingly written to demonstrate how smart the author is, and how stupid
the reader is.

There is one place where an application programmer could say "Hey, I know,
I'll use a continuation!" -- whenever they need a single "global" "OO object"
with a "global variable" in it. I use them for implementing statistics
printing and
profiling. The idea is simple, the solution is elegant:

-- I want to have a single "global" counter which I will increment by one from
now until forever.
-- I want to have a function to increment it, and I want to call that function
at any time, from anywhere at all.
-- every hundreth time that it is called, it will print something like
"xyz operations per second"

Turns out that continuations offer a marvelously simple, easy-to-understand
easy-to-use solution to this problem. You'd think that there would be some
tutorial or some common "here's the cookbook way of implementing this"
but I sure haven't seen one. But once you get the hang of the above example,
one can see how it could be fleshed out into a full-fledged OO-like
message-passing system, complete with frames and slots.

Continuations do not need to be a swear word; its just that they're badly
explained. They are a stumbling block, but that's in part because there's
no analogous concept in proceedural languages; so most programmers
never have to learn it.

Post by Marko Rauhamaa
Have yet to find a need for one.

Sounds like gain-saying. Just because you don't need it, doesn't mean its
not useful. "A typical user uses 20% of an application. The problem is,
different users use a different 20%". "recall there were dozens of wikis before
wikipedia, dozens of search engines before google, dozens of social media
sites before facebook"

I conclude that the difference between a successful(popular) and a failed
(unpopular) tech is often very subtle. For all I know, decorators "done right"
in guile could skyrocket it to the top of the popularity ranking. Or not. Just
be careful in dismissing some of these ideas.

Post by Marko Rauhamaa

Post by Marko Rauhamaa
Unicode illusion. (Guile has fallen into that last trap as well,
unfortunately.)

I’m using Python and Guile for physics and math (to some degree), and
having good Unicode-support is great for that. What’s your practical
criticism here?

Unicode support in guile remains rocky and buggy, as I once again
rediscovered recently when pumping Chinese through it.

Post by Marko Rauhamaa
It would be better to leave Unicode out of Guile's system interface and
have the application encode and decode explicitly where needed.

Huh? That would suck. Unless I misunderstand, in which case it would
be great. Unicode is just a bunch of bytes that are null-terminated. Guile
does very stupidly try to convert unicode strings into arrays of wide chars
which is absolutely idiotic, and a total bottleneck for what I do.
Its a useless,
pointless processing step. I keep planning on submitting a patch to fix this
but never have time.

I regularly have strings that are 10 Mbytes or more, that contain mixtures of
scheme code and chinese characters (in utf8), that I have to send to guile
for eval. Anything that would make that faster would be awesome.

--linas

[Marko Rauhamaa]

Post by Linas Vepstas
Unicode is just a bunch of bytes that are null-terminated.

That is not what Unicode is. Maybe you are thinking of UTF-8, an
ingenious way to map a Unicode sequence onto a byte sequence. Trouble
is, there are byte sequences that are illegal UTF-8. While they are
illegal for Unicode use, they are perfectly possible and valid as far as
the (Linux) operating system is concerned.


Marko

[Arne Babenhauserheide]

Post by Marko Rauhamaa

Post by Marko Rauhamaa
Then, there's GOOPS, which in my opinion is simply an unnatural way
to go about object-oriented programming. It does violence both to
ordinary OO way of thinking and classic Lisp idioms.

GOOPS works pretty well for me where I use it (for dispatch by type).
Could you clarify your criticism: Do you think it is bad or is it just
different?

GOOPS starts by defining slots. Slots are an objects internal business
and should not be visible to the outside. Instead, objects should be
black boxes that interact with methods. For example, the port interface
is a nice, classic OO API. You don't know anything about the slots of a
port.
Furthermore, I think an extensive type system is un-Lisp-like. OO is not
about things belonging to a type but objects interacting through
methods. In fact, the concept of a "class" could be done away with.

Post by Marko Rauhamaa
Continuations and multiple values are obstacles instead of enablers.

I think multiple values are nice. But they are not well-integrated (I
have to import something to use them). Why do you think them enablers?

Yes, the multiple values API is a problem in and of itself, but more
(values a b c)
(list a b c)

Practically it gives me (let-values (((a b c) (values 1 2 3))) a)

I could get the same by using match on a list, but then I have to use
match on a list instead of just assigning the values. let-values could
be designed to work with lists, though.

So conceptually I assume that the real difference is in implementation:
avoiding the construction of a list.

Post by Marko Rauhamaa

Why do you think that continuations are an obstacle (do you mean
general continuations or do you mean prompts)?

Continuations prevent you from implementing Python's try/finally and
emacs' (unwind-protect).

If I understand it correctly, this is only true, if there are resources
in use which do not get created in the try. I hit that problem when
implementing 'with':
https://bitbucket.org/ArneBab/wisp/src/3a654cfe6632af4b0002ce98c753c07c400aac55/examples/with.w#with.w-8

with (open-file "with.w" "r") as port
format #t "~a\n" (read port)

^ working Guile code which ensures that the port is closed after it is
no longer used. But it can break if a continuation is used. To fix
that, I’d have to add guards which save the state of the port when
code creates a continuation and reconstructs that state when the
continuation is used.

Post by Marko Rauhamaa
On the other hand, I don't think any application developer would come to
a point of thinking, "Hey, I know, I'll use a continuation!"

Error-handling is simply a specific case of that. In Python people use
exceptions for flow control, which can be efficient when the
non-exception case is much, much more common than the exception case
(because in the first case the overhead of that is almost zero).

Post by Marko Rauhamaa

Post by Marko Rauhamaa
asyncio, type annotation,

type annotations should provide nice ways to optimize for pypy and
other more optimizing Python implementations.

You can't have it bothways. A dynamic language liberates you from
shackles and boilerplate even if it destroys your performance. The
moment you bring in the boilerplate, you are back to Java.

It’s optional boilerplate which you only add where you really need it.

Post by Marko Rauhamaa
Have yet to find a need for one.

I used them quite a bit. The most efficient one was a caching decorator:

@cached
def fun(a):
# something expensive
return a

In Python 3.2 you can use the standard lru-cache for that:
https://docs.python.org/3/library/functools.html#functools.lru_cache

Post by Marko Rauhamaa

Post by Marko Rauhamaa
dunder jungle,

What’s that?

<URL: https://docs.python.org/3/genindex-_.html>

Ah, yes :)

I think these are a good way to show that I as developer am venturing
into regions I should only enter when I am really, really sure I want
to, because they can cause problems I may not see right away.

Sadly this rule is broken with if __name__ == "__main__": ...
and for def __init__(self)

So I think it’s good to have that, but not good that programmers need to
use that during typical programming tasks.

Post by Marko Rauhamaa

Post by Marko Rauhamaa
meta-object programming,

That’s needed to make some things elegant. "Different 20%"-rule again.

Again, I haven't yet found a need to deviate from the regular object
semantics.

I saw some, like actually accessing and changing the bytecode at runtime
to add tracing commands.

Post by Marko Rauhamaa

Post by Marko Rauhamaa
Unicode illusion. (Guile has fallen into that last trap as well,
unfortunately.)

I’m using Python and Guile for physics and math (to some degree), and
having good Unicode-support is great for that. What’s your practical
criticism here?

In Linux, all pathnames, files, sockets, environment variables and
command-line arguments are, or deal with, bytes. The bytes are often
encoded with UTF-8, but a programming language cannot assume that, even
if LOCALE promises otherwise.
It would be better to leave Unicode out of Guile's system interface and
have the application encode and decode explicitly where needed.

I’m not sure about that. Python3 requires explicit encode/decode, and
it’s a lot of hassle to port from Py2 to Py3 if your application does
lots of socket-operations. I have firsthand experience with that, and
the debugging is gruesome.

But it might have been much better if the program had been created with
Python3 from the start.

Post by Marko Rauhamaa

Post by Marko Rauhamaa
There's one thing Python has over Guile, still: Python's system call
support is outstanding. For example, I managed to implement a
full-blown shell using Python. That was the first time I ever ran
into terminal-related system calls, and Python had them all.

Which ones are missing in Guile?

For example tcgetattr(3). (Ok, not a system call.)
Missing epoll(2) is inconvenient. Not being able to transfer an open
file descriptor via sendmsg(2) is really bad.

Do you know whether there is a specific reason for that or whether these
could be added?

Best wishes,
Arne

[Arne Babenhauserheide]

Post by Panicz Maciej Godek
There's also this problem with Scheme that it is a very diverse
community with plethora of implementations. And while I use Guile, I
don't mind using Chicken or Chez or Gambit for various purposes. Or
even Stalin sometimes. Maybe, instead of getting a better compiler, we
should focus on integrating existing compilers?

You’ll have to ask Andy Wingo about all the hidden complexity in just
wiring in a compilier. I assume it’s pretty big.

I think the r7rs folks are working in that regard, to make it possible
to write code which you can simply run through another Scheme
implementation if you need those strengths.

I started to use (import (...)) to get closer to that: Getting used to
the portable stuff (also it’s shorter than (use-modules (...)). But not
everything is portable, and performance critical stuff most definitely
isn’t.

And for the stuff I do, most things get performance critical at some
point. Even looking for null-bytes in a file becomes a bottleneck if you
have a few hundred TiB to check.

Though I mainly need to write Python at work (Scheme isn’t widespread
enough for the folks there — Python is everywhere in science
nowadays). It’s still efficient to do that and I enjoy it, but not as
much as writing Scheme.

Post by Panicz Maciej Godek

Post by Amirouche
And it's easy to packages.

Post by Amirouche
=> Another pain topic. In reality it's very easy to package
all sort of guile programs for guix. But guix itself needs
to be more stable.

I wish I had guildhall ready. Got hit by
time-eaten-by-other-project-because-guildhall-was-priority-three.
It needs to be easy to not just package some code, but also to share
that package without hitting a bottleneck.

What happened to initiatives like Scheme NOW, that tried to integrate
various groups of developers.

I did not hear from that.

Post by Panicz Maciej Godek
Or why can't we just "steal" eggs from Chicken? (I mean it. They already
have that problem solved)

Do you mean, integrating eggs into Guile so we could just use them?
Conceptually what would be needed for that is a metadata field: "Works
with Guile". And practically someone to write the integration.

We could also have guildhall working, and add a tool for converting
chicken eggs into guildhall packages (and ideally vice versa). Then
people could do the required Guile integration steps once and provide
the packages to all who might want them.

Post by Panicz Maciej Godek

Post by Amirouche

Post by Amirouche
Python is successful because it has a massive number
of libraries.

This is a social problem. For a deeper discussion why it is a problem,
see http://winestockwebdesign.com/Essays/Lisp_Curse.html
"Lisp is so powerful that problems which are technical issues in other
programming languages are social issues in Lisp."
We can counter this with easy tutorials and with writing something like
canonical Scheme. But for this, we need to define a canonical Scheme
which is hits the performance and readability sweet-spot for
Guile. Canonical code must be close to the fastest code.

Practically put: We need Andy Wingo to nitpick the tutorial about things

Post by Amirouche
which will cause overheads the compiler cannot fix easily — including
expensive use of macros.

I definitely oppose. If Chez has something solved better, why not use Chez?

It’s not "why not use Chez", but rather "what should I teach new
people?"

They are already learning a new language. When I now go and point them
towards many different implementations which differ in details of stuff
I teach them, I’m throwing more complexity at them. More things to
understand before even starting to write working code.

Maybe it would already help to mark the code which will work the same in
all (r7rs) Schemes. Is there stuff which is well optimized in all
Schemes? Who knows that? The r7rs benchmarks look like this is very much
not the case: http://ecraven.github.io/r7rs-benchmarks/benchmark.html

(Or rather, the benchmarks would ask: "why should I *not* use Chez or
stalin for everything I do?" Because those are the fastest for most
tasks.)

But that might mean to write less elegant or efficient code to keep it
cross-implementation. Should I rather teach new people to solve a
problem as well as possible, or teach them to solve a problem in a
portable way?

In my case, I decided to use Guile by checking Scheme
implementations. There were three with roughly equal activity and
features. I chose Guile, because it’s a GNU project and it has a long
history of surviving changing maintainers, so my skills are most likely
to stay useful.

Post by Panicz Maciej Godek
The ultimate goal is not to optimize programs, but programmers.

I’m not sure that I want to optimize them, I want to teach them tools to
be more efficient and enjoy their work more (and teach myself the same).

Post by Panicz Maciej Godek
But I agree on the point of the canonical code. I think that the "zen
of python" is one of important factors of making it popular. I think
it could be the same with Scheme, but here the problem is that people
have various ideas with regard to how the "ideal" program should look
like.

I think one important point for Scheme would be to gather some consensus
points. The Zen of Python is part of what made that one community
strong. There is no reason why there should not be a Zen of Scheme,
along with implementation-specific Koans which extend it.

Do you have ideas for principles which could be shared by most Schemers?

Post by Panicz Maciej Godek
For example, I have a very strong opinion with regard to the use of
pattern matching and restructuring -- I try to use destructuring
almost everywhere (from what I've seen, Clojure programmers are alike
with this regard). However, I am aware that there are people who don't
like it that much, like Taylan or Barry Margolin.
Also, I think that the astounding success of R provides hints with
regard what is needed for a programming language to gain popularity. R
has a large repository of packages that fills a certain niche. When I
"are there out of the box libraries to estimate a zero inflated
negative binomial regression model in guile". Of course, if I knew
what a zero-inflated negative binomial regression model, I could
deliver an implementation by just explaining the notions used in that
phrase.

I think there’s a quote which applies here: "Most people only use 20% of
the capabilities of any program, but they do not use the same 20%."
— Joel Spolksy, if I remember correctly.

And to make people really dig into it, a programming language must work
well for all tasks they might encounter. That’s why a comprehensive
standard library is important: It gives the assurance that the skills
you learn will stay useful wherever you go.

There’s a nice quote from Rust folks: "people started to feel safe when
we told them about unsafe" — someone on GNU social.
This essentially says: they then saw that Rust could work in the
uncomfortable way some of their tasks might require. That they could
take it into harsher territory, if needed.

Post by Panicz Maciej Godek
(But then again, I try to write functional programs whenever
possible, often consciously sacrificing time complexity of my
solutions, and I'm aware that not everybody will like it.)

I look for ease of understanding, with functional programming techniques
being a useful tool to reduce complexity and to simplify refactoring.

But then, this is a point where we actually move into a similar
direction out of different motives. I think the only problem here is the
danger of assuming contradiction where there does not need to be
any. Just different nuances of Zen :)

Best wishes,
Arne

[Panicz Maciej Godek]

Post by Panicz Maciej Godek
There's also this problem with Scheme that it is a very diverse
community with plethora of implementations. And while I use Guile, I
don't mind using Chicken or Chez or Gambit for various purposes. Or
even Stalin sometimes. Maybe, instead of getting a better compiler, we
should focus on integrating existing compilers?

You’ll have to ask Andy Wingo about all the hidden complexity in just
wiring in a compilier. I assume it’s pretty big.
I think the r7rs folks are working in that regard, to make it possible
to write code which you can simply run through another Scheme
implementation if you need those strengths.
I started to use (import (...)) to get closer to that: Getting used to
the portable stuff (also it’s shorter than (use-modules (...)). But not
everything is portable, and performance critical stuff most definitely
isn’t.
And for the stuff I do, most things get performance critical at some
point. Even looking for null-bytes in a file becomes a bottleneck if you
have a few hundred TiB to check.
Though I mainly need to write Python at work (Scheme isn’t widespread
enough for the folks there — Python is everywhere in science
nowadays). It’s still efficient to do that and I enjoy it, but not as
much as writing Scheme.

Post by Panicz Maciej Godek

Post by Amirouche
And it's easy to packages.

Post by Amirouche
=> Another pain topic. In reality it's very easy to package
all sort of guile programs for guix. But guix itself needs
to be more stable.

I wish I had guildhall ready. Got hit by
time-eaten-by-other-project-because-guildhall-was-priority-three.
It needs to be easy to not just package some code, but also to share
that package without hitting a bottleneck.

What happened to initiatives like Scheme NOW, that tried to integrate
various groups of developers.

I did not hear from that.

Post by Panicz Maciej Godek
Or why can't we just "steal" eggs from Chicken? (I mean it. They already
have that problem solved)

Do you mean, integrating eggs into Guile so we could just use them?
Conceptually what would be needed for that is a metadata field: "Works
with Guile". And practically someone to write the integration.

There's surely many ways to approach that issue. The point is that for some
reason, Schemers from various tribes prefer to reinvent the wheel,
rather than use an existing one. (Not that I am any different)
However, I also think that these CPAN-alike solutions are far from optimal:
ideally, programming could be made an experience similar the game
"The Journey" by thatgamecompany (where you travel to your goal and
sometimes
encounter other players heading the same direction), and the repository
itself could look
more like Wikipedia (but first we'd need to start perceiveing programming as
a compact way of representing knowledge)

Practically put: We need Andy Wingo to nitpick the tutorial about things

Post by Panicz Maciej Godek

Post by Amirouche
which will cause overheads the compiler cannot fix easily — including
expensive use of macros.

I definitely oppose. If Chez has something solved better, why not use Chez?

It’s not "why not use Chez", but rather "what should I teach new
people?"
They are already learning a new language. When I now go and point them
towards many different implementations which differ in details of stuff
I teach them, I’m throwing more complexity at them. More things to
understand before even starting to write working code.
I totally agree. Programmers shouldn't be concerned whether they're

using Chez or Guile or Gambit or Stalin or whatever else. Ideally, they
should do well without even knowing that such things exist.
There are Schemes, such as Kawa or Biwa or IronScheme, that are tied
to their environment, but most of them just run on the PC

Maybe it would already help to mark the code which will work the same in
all (r7rs) Schemes. Is there stuff which is well optimized in all
Schemes? Who knows that? The r7rs benchmarks look like this is very much
not the case: http://ecraven.github.io/r7rs-benchmarks/benchmark.html
(Or rather, the benchmarks would ask: "why should I *not* use Chez or
stalin for everything I do?" Because those are the fastest for most
tasks.)
But that might mean to write less elegant or efficient code to keep it
cross-implementation. Should I rather teach new people to solve a
problem as well as possible, or teach them to solve a problem in a
portable way?

I think that in principle, the programmer should only focus on writing
most elegant code

In my case, I decided to use Guile by checking Scheme
implementations. There were three with roughly equal activity and
features. I chose Guile, because it’s a GNU project and it has a long
history of surviving changing maintainers, so my skills are most likely
to stay useful.

That's interesting. I chose it by trying to add a scripting language to my
game engine written in C++. But I guess that if I was starting now, I'd
write
the whole engine in Chez (which wasn't opensource until last year)

Post by Panicz Maciej Godek
The ultimate goal is not to optimize programs, but programmers.

I’m not sure that I want to optimize them, I want to teach them tools to
be more efficient and enjoy their work more (and teach myself the same).

Yeah, maybe I used a harsh word, but I meant exactly optimizing
programmers' efficiency (or if it sounds too bad, "giving them tools
to be more efficient")

I think one important point for Scheme would be to gather some consensus
points. The Zen of Python is part of what made that one community
strong. There is no reason why there should not be a Zen of Scheme,
along with implementation-specific Koans which extend it.
Do you have ideas for principles which could be shared by most Schemers?

It depends on the most Schemers, not me :)
However, you can have a look at the (grand scheme) glossary that I've been
maintaining for some time:

https://github.com/plande/grand-scheme

of course, the intent is to make the glossary usable for someone else than
me,
and I can boast that I have at least one user distinct from the maintainer.
You can have a look and let me know what you think. (for now it only works
with Guile). In particular, are there any factors that could convince you to
adapt it as a base for your library, or what would you want changed
(I assume you won't stop at the README file).
Anyways, the rules (of thumb) are:


- functions should be pure
- language constructs that cause side-effects should be encapsulated
with macros that limit the extent of those side-effects
- if possible, use the quasiquote macro, rather than cons, append or list
- prefer the use of match <http://synthcode.com/scheme/match.scm> to cond,
if the patterns are simple
- never use abbreviations
- use commented prepositions to separate arguments

Also, I recently added infix syntax to Scheme, but (unlike in Racket or
David Wheeler's solutions) in the way it is still a prefix syntax. So for
example,
instead of writing (< a b), one ought to write (is a < b). The rationale is
that
prefix notation is terrible for non-symmetrical binary predicates, because
it
obscures the roles of respective arguments (in addition, (is a < b <= c)
expands
to (and (< a b) (<= b c))). I'm still not sure whether it is a good idea,
but I have
no reasons to think that it is bad either :]
[Curiously enough, I think that prefix notation in arithmetic is just fine:
(+ a b c) reads as "sum of a, b and c"]

Also, I think that the astounding success of R provides hints with

Post by Panicz Maciej Godek
regard what is needed for a programming language to gain popularity. R
has a large repository of packages that fills a certain niche. When I
"are there out of the box libraries to estimate a zero inflated
negative binomial regression model in guile". Of course, if I knew
what a zero-inflated negative binomial regression model, I could
deliver an implementation by just explaining the notions used in that
phrase.

I think there’s a quote which applies here: "Most people only use 20% of
the capabilities of any program, but they do not use the same 20%."
— Joel Spolksy, if I remember correctly.
And to make people really dig into it, a programming language must work
well for all tasks they might encounter. That’s why a comprehensive
standard library is important: It gives the assurance that the skills
you learn will stay useful wherever you go.
There’s a nice quote from Rust folks: "people started to feel safe when
we told them about unsafe" — someone on GNU social.
This essentially says: they then saw that Rust could work in the
uncomfortable way some of their tasks might require. That they could
take it into harsher territory, if needed.
I like it that you and Amirouche look a lot at the sociocultural

perspective
of programming

Post by Panicz Maciej Godek
(But then again, I try to write functional programs whenever
possible, often consciously sacrificing time complexity of my
solutions, and I'm aware that not everybody will like it.)

I look for ease of understanding, with functional programming techniques
being a useful tool to reduce complexity and to simplify refactoring.
But then, this is a point where we actually move into a similar
direction out of different motives. I think the only problem here is the
danger of assuming contradiction where there does not need to be
any. Just different nuances of Zen :)
Yes, to me the ease of understanding is also crucial. (I'd even go

further and say that undestranding is the very point of programming)

Best regards,
Panicz

[Arne Babenhauserheide]

Post by Panicz Maciej Godek
There's surely many ways to approach that issue. The point is that for
some reason, Schemers from various tribes prefer to reinvent the
wheel, rather than use an existing one. (Not that I am any different)
However, I also think that these CPAN-alike solutions are far from
optimal: ideally, programming could be made an experience similar the
game "The Journey" by thatgamecompany (where you travel to your goal
and sometimes encounter other players heading the same direction), and
the repository itself could look more like Wikipedia (but first we'd
need to start perceiveing programming as a compact way of representing
knowledge)

That sounds somehow like stackoverflow. We might already be going there :)

Post by Panicz Maciej Godek

Post by Arne Babenhauserheide
Practically put: We need Andy Wingo to nitpick the tutorial about things

Post by Panicz Maciej Godek

Post by Amirouche
which will cause overheads the compiler cannot fix easily — including
expensive use of macros.

I definitely oppose. If Chez has something solved better, why not use Chez?

It’s not "why not use Chez", but rather "what should I teach new
people?"
They are already learning a new language. When I now go and point them
towards many different implementations which differ in details of stuff
I teach them, I’m throwing more complexity at them. More things to
understand before even starting to write working code.

I totally agree. Programmers shouldn't be concerned whether they're
using Chez or Guile or Gambit or Stalin or whatever else. Ideally, they
should do well without even knowing that such things exist.
There are Schemes, such as Kawa or Biwa or IronScheme, that are tied
to their environment, but most of them just run on the PC

I think this is a point which cannot be reached, because the design of
the Scheme system itself must make certain tradeoffs — especially for
the default environment (what to present a new user). If that is too
large, then users cannot start quickly. If it is too small, then users
cannot solve problems easily. But different problem spaces require
different base tools.

Another points are the basic datastructures: If they are too complex,
then the system cannot be used easily on tiny computers.

Essentially you have to find reasons for decisions like this:
https://docs.python.org/3/faq/design.html#how-are-lists-implemented
and this:
https://wiki.python.org/moin/TimeComplexity

Post by Panicz Maciej Godek

Post by Arne Babenhauserheide
Maybe it would already help to mark the code which will work the same in
all (r7rs) Schemes. Is there stuff which is well optimized in all
Schemes? Who knows that? The r7rs benchmarks look like this is very much
not the case: http://ecraven.github.io/r7rs-benchmarks/benchmark.html
(Or rather, the benchmarks would ask: "why should I *not* use Chez or
stalin for everything I do?" Because those are the fastest for most
tasks.)
But that might mean to write less elegant or efficient code to keep it
cross-implementation. Should I rather teach new people to solve a
problem as well as possible, or teach them to solve a problem in a
portable way?

I think that in principle, the programmer should only focus on writing
most elegant code

I think there are several different kinds of elegance. There’s technical
elegance (combining existing things to build something unexpectedly
powerful with minimal effort), readability (being easy to understand for
newcomers), minimal code (solving a problem with the minimal amount of
code — often close to mathematics), uniformity (using the same basic
structure everywhere), efficiency (solving a problem with minimal
resource-consumption), ...
I’m sure we’ll find more.

Sadly these elegances contradict each other in some points.

Post by Panicz Maciej Godek

Post by Arne Babenhauserheide
In my case, I decided to use Guile by checking Scheme
implementations. There were three with roughly equal activity and
features. I chose Guile, because it’s a GNU project and it has a long
history of surviving changing maintainers, so my skills are most likely
to stay useful.

That's interesting. I chose it by trying to add a scripting language
to my game engine written in C++. But I guess that if I was starting
now, I'd write the whole engine in Chez (which wasn't opensource until
last year)

Chez becoming free software is an interesting development.

Post by Panicz Maciej Godek

Post by Arne Babenhauserheide

Post by Panicz Maciej Godek
The ultimate goal is not to optimize programs, but programmers.

I’m not sure that I want to optimize them, I want to teach them tools to
be more efficient and enjoy their work more (and teach myself the same).

Yeah, maybe I used a harsh word, but I meant exactly optimizing
programmers' efficiency (or if it sounds too bad, "giving them tools
to be more efficient")

Sounds good :)

Post by Panicz Maciej Godek

Post by Arne Babenhauserheide
I think one important point for Scheme would be to gather some consensus
points. The Zen of Python is part of what made that one community
strong. There is no reason why there should not be a Zen of Scheme,
along with implementation-specific Koans which extend it.
Do you have ideas for principles which could be shared by most Schemers?

It depends on the most Schemers, not me :)

That’s why its a social problem :)

Finding a set of basic koans which most Schemers agree with requires
talking to and knowing a lot of Schemers.

Post by Panicz Maciej Godek
However, you can have a look at the (grand scheme) glossary that I've been
https://github.com/plande/grand-scheme

This looks interesting.

Post by Panicz Maciej Godek
of course, the intent is to make the glossary usable for someone else than
me,
and I can boast that I have at least one user distinct from the maintainer.
You can have a look and let me know what you think. (for now it only works
with Guile). In particular, are there any factors that could convince you to
adapt it as a base for your library, or what would you want changed
(I assume you won't stop at the README file).

I’d mainly need practical examples. Tell me how I’d solve a problem
with (grand scheme) and how it would be solved without it — showing me
why using it is better.

That means: Documentation.

Also I’d need to know whether some of these methods have performance
problems: Will this scale as well as the method without (grand scheme)?

Post by Panicz Maciej Godek
- functions should be pure
- language constructs that cause side-effects should be encapsulated
with macros that limit the extent of those side-effects
- if possible, use the quasiquote macro, rather than cons, append or list
- prefer the use of match <http://synthcode.com/scheme/match.scm> to cond,
if the patterns are simple
- never use abbreviations
- use commented prepositions to separate arguments

As guidelines for easy-to-use libraries, that sounds good.

Post by Panicz Maciej Godek
Also, I recently added infix syntax to Scheme, but (unlike in Racket
or David Wheeler's solutions) in the way it is still a prefix
syntax. So for example, instead of writing (< a b), one ought to write
(is a < b). The rationale is that prefix notation is terrible for
non-symmetrical binary predicates, because it obscures the roles of
respective arguments (in addition, (is a < b <= c) expands to (and (<
a b) (<= b c))). I'm still not sure whether it is a good idea, but I
have no reasons to think that it is bad either :] [Curiously enough, I
think that prefix notation in arithmetic is just fine: (+ a b c) reads
as "sum of a, b and c"]

This is an old idea, but using (is) as prefix looks
interesting. Typically the prefix was something like (nfx), which
doesn’t say anything semantic (other than that what follows uses infix
syntax). It’s like an expanded equal?

(if (is a = 5)
#t
#f)

(cond
((is q = p)
q)
((is p > 5)
p)
(else
q))

This seems pretty close to natural language. Can I also use (is A 5)
instead of (equal? A 5)?

Though (is? a = 5) might be closer to canonical Scheme.

Post by Panicz Maciej Godek
I like it that you and Amirouche look a lot at the sociocultural
perspective of programming

I’m glad to hear that :)

Post by Panicz Maciej Godek
Yes, to me the ease of understanding is also crucial. (I'd even go
further and say that undestranding is the very point of programming)

I would not go that far: We’re programming both for the future
maintainers of the code and for the computer. This restricts us in
expressing things, because we always have to keep two targets audiences
in mind (many different systems *and* many different people).

Best wishes,
Arne

[Amirouche]

Post by Arne Babenhauserheide

Post by Panicz Maciej Godek
There's surely many ways to approach that issue. The point is that for
some reason, Schemers from various tribes prefer to reinvent the
wheel, rather than use an existing one. (Not that I am any different)
However, I also think that these CPAN-alike solutions are far from
optimal: ideally, programming could be made an experience similar the
game "The Journey" by thatgamecompany (where you travel to your goal
and sometimes encounter other players heading the same direction), and
the repository itself could look more like Wikipedia (but first we'd
need to start perceiveing programming as a compact way of representing
knowledge)

That sounds somehow like stackoverflow. We might already be going there :)

That's a very good idea!

Post by Arne Babenhauserheide

Post by Panicz Maciej Godek

Post by Arne Babenhauserheide
Practically put: We need Andy Wingo to nitpick the tutorial about things

Post by Panicz Maciej Godek

Post by Arne Babenhauserheide
which will cause overheads the compiler cannot fix easily — including
expensive use of macros.

I definitely oppose. If Chez has something solved better, why not use Chez?

It’s not "why not use Chez", but rather "what should I teach new
people?"
They are already learning a new language. When I now go and point them
towards many different implementations which differ in details of stuff
I teach them, I’m throwing more complexity at them. More things to
understand before even starting to write working code.

I totally agree. Programmers shouldn't be concerned whether they're
using Chez or Guile or Gambit or Stalin or whatever else. Ideally, they
should do well without even knowing that such things exist.
There are Schemes, such as Kawa or Biwa or IronScheme, that are tied
to their environment, but most of them just run on the PC

I think this is a point which cannot be reached, because the design of
the Scheme system itself must make certain tradeoffs — especially for
the default environment (what to present a new user). If that is too
large, then users cannot start quickly. If it is too small, then users
cannot solve problems easily. But different problem spaces require
different base tools.
Another points are the basic datastructures: If they are too complex,
then the system cannot be used easily on tiny computers.
https://docs.python.org/3/faq/design.html#how-are-lists-implemented
https://wiki.python.org/moin/TimeComplexity

Post by Panicz Maciej Godek

Post by Arne Babenhauserheide
Maybe it would already help to mark the code which will work the same in
all (r7rs) Schemes. Is there stuff which is well optimized in all
Schemes? Who knows that? The r7rs benchmarks look like this is very much
not the case: http://ecraven.github.io/r7rs-benchmarks/benchmark.html
(Or rather, the benchmarks would ask: "why should I *not* use Chez or
stalin for everything I do?" Because those are the fastest for most
tasks.)
But that might mean to write less elegant or efficient code to keep it
cross-implementation. Should I rather teach new people to solve a
problem as well as possible, or teach them to solve a problem in a
portable way?

I think that in principle, the programmer should only focus on writing
most elegant code

I think there are several different kinds of elegance. There’s technical
elegance (combining existing things to build something unexpectedly
powerful with minimal effort), readability (being easy to understand for
newcomers), minimal code (solving a problem with the minimal amount of
code — often close to mathematics), uniformity (using the same basic
structure everywhere), efficiency (solving a problem with minimal
resource-consumption), ...
I’m sure we’ll find more.
Sadly these elegances contradict each other in some points.

Post by Panicz Maciej Godek

Post by Arne Babenhauserheide
In my case, I decided to use Guile by checking Scheme
implementations. There were three with roughly equal activity and
features. I chose Guile, because it’s a GNU project and it has a long
history of surviving changing maintainers, so my skills are most likely
to stay useful.

That's interesting. I chose it by trying to add a scripting language
to my game engine written in C++. But I guess that if I was starting
now, I'd write the whole engine in Chez (which wasn't opensource until
last year)

Chez becoming free software is an interesting development.

Post by Panicz Maciej Godek

Post by Arne Babenhauserheide

Post by Panicz Maciej Godek
The ultimate goal is not to optimize programs, but programmers.

I’m not sure that I want to optimize them, I want to teach them tools to
be more efficient and enjoy their work more (and teach myself the same).

Yeah, maybe I used a harsh word, but I meant exactly optimizing
programmers' efficiency (or if it sounds too bad, "giving them tools
to be more efficient")

Sounds good :)

Post by Panicz Maciej Godek

Post by Arne Babenhauserheide
I think one important point for Scheme would be to gather some consensus
points. The Zen of Python is part of what made that one community
strong. There is no reason why there should not be a Zen of Scheme,
along with implementation-specific Koans which extend it.
Do you have ideas for principles which could be shared by most Schemers?

It depends on the most Schemers, not me :)

That’s why its a social problem :)
Finding a set of basic koans which most Schemers agree with requires
talking to and knowing a lot of Schemers.

Post by Panicz Maciej Godek
However, you can have a look at the (grand scheme) glossary that I've been
https://github.com/plande/grand-scheme

This looks interesting.

Post by Panicz Maciej Godek
of course, the intent is to make the glossary usable for someone else than
me,
and I can boast that I have at least one user distinct from the maintainer.
You can have a look and let me know what you think. (for now it only works
with Guile). In particular, are there any factors that could convince you to
adapt it as a base for your library, or what would you want changed
(I assume you won't stop at the README file).

I’d mainly need practical examples. Tell me how I’d solve a problem
with (grand scheme) and how it would be solved without it — showing me
why using it is better.
That means: Documentation.
Also I’d need to know whether some of these methods have performance
problems: Will this scale as well as the method without (grand scheme)?

Post by Panicz Maciej Godek
- functions should be pure
- language constructs that cause side-effects should be encapsulated
with macros that limit the extent of those side-effects
- if possible, use the quasiquote macro, rather than cons, append or list
- prefer the use of match <http://synthcode.com/scheme/match.scm> to cond,
if the patterns are simple
- never use abbreviations
- use commented prepositions to separate arguments

As guidelines for easy-to-use libraries, that sounds good.

Post by Panicz Maciej Godek
Also, I recently added infix syntax to Scheme, but (unlike in Racket
or David Wheeler's solutions) in the way it is still a prefix
syntax. So for example, instead of writing (< a b), one ought to write
(is a < b). The rationale is that prefix notation is terrible for
non-symmetrical binary predicates, because it obscures the roles of
respective arguments (in addition, (is a < b <= c) expands to (and (<
a b) (<= b c))). I'm still not sure whether it is a good idea, but I
have no reasons to think that it is bad either :] [Curiously enough, I
think that prefix notation in arithmetic is just fine: (+ a b c) reads
as "sum of a, b and c"]

This is an old idea, but using (is) as prefix looks
interesting. Typically the prefix was something like (nfx), which
doesn’t say anything semantic (other than that what follows uses infix
syntax). It’s like an expanded equal?
(if (is a = 5)
#t
#f)
(cond
((is q = p)
q)
((is p > 5)
p)
(else
q))
This seems pretty close to natural language. Can I also use (is A 5)
instead of (equal? A 5)?
Though (is? a = 5) might be closer to canonical Scheme.

Post by Panicz Maciej Godek
I like it that you and Amirouche look a lot at the sociocultural
perspective of programming

I’m glad to hear that :)

Post by Panicz Maciej Godek
Yes, to me the ease of understanding is also crucial. (I'd even go
further and say that undestranding is the very point of programming)

I would not go that far: We’re programming both for the future
maintainers of the code and for the computer. This restricts us in
expressing things, because we always have to keep two targets audiences
in mind (many different systems *and* many different people).
Best wishes,
Arne

[Amirouche]

Post by Amirouche

Post by Arne Babenhauserheide

Post by Panicz Maciej Godek
There's surely many ways to approach that issue. The point is that for
some reason, Schemers from various tribes prefer to reinvent the
wheel, rather than use an existing one. (Not that I am any different)
However, I also think that these CPAN-alike solutions are far from
optimal: ideally, programming could be made an experience similar the
game "The Journey" by thatgamecompany (where you travel to your goal
and sometimes encounter other players heading the same direction), and
the repository itself could look more like Wikipedia (but first we'd
need to start perceiveing programming as a compact way of representing
knowledge)

That sounds somehow like stackoverflow. We might already be going there :)

That's a very good idea!

There is trusting issue tho. Do you trust all what is written in
wikipedia right? You
can not simply execute code from the programming cloud like that. I try
to review
at least once the code I use. But I trust most of it to be safe.

[sirgazil]

Post by sirgazil
When I started, my plan was to follow the course in Guile instead,
since the course is about the design method, and not the tools, but
I had to stop very early when I had to do some projects involving
graphics and animation.
Personally, I don't feel the need for something like DrRacket,
but it would have been nice to be able to find a Guile "canvas" or a
2D game library to play easily with basic graphics and animations.

You might want to try these:
https://dthompson.us/projects/chickadee.html
https://dthompson.us/projects/sly.html

Hi, Rodger.

Yes, I know those. I tried Sly, but then realized it requires OpenGL, which is broken in the system I use (nvidia seems incompatible with free software). I haven't tried chickadee, but, if I remember correctly, it has the same requirement.

So I still need something that works without OpenGL, like Racket's universe module or Py

[sirgazil]

-------- Original Message --------
Subject: Re: How to make GNU Guile more successful
Local Time: 17 de febrero de 2017 9:18 PM
UTC Time: 18 de febrero de 2017 2:18

Post by sirgazil

Post by sirgazil

Post by sirgazil
When I started, my plan was to follow the course in Guile

instead,

Post by sirgazil
since the course is about the design method, and not the tools,

but

Post by sirgazil
I had to stop very early when I had to do some projects involving
graphics and animation.
Personally, I don't feel the need for something like DrRacket,
but it would have been nice to be able to find a Guile "canvas"

or a

Post by sirgazil
2D game library to play easily with basic graphics and

animations.
https://dthompson.us/projects/chickadee.html
https://dthompson.us/projects/sly.html

Hi, Rodger.
Yes, I know those. I tried Sly, but then realized it requires OpenGL,
which is broken in the system I use (nvidia seems incompatible with
free software). I haven't tried chickadee, but, if I remember
correctly, it has the same requirement.
So I still need something that works without OpenGL, like Racket's
universe module or Pygame :)

I'm not familiar with Racket's universe or Pygame, but I have a couple
more suggestions that you might consider.

There is Guile-Cairo if you just want a basic canvas.
http://www.nongnu.org/guile-cairo/
See also: https://www.cairographics.org/

Maybe you can use Guile-SDL or Guile-SDL2, which is what Sly and
Chickadee are built on, but like normal SDL, you can use it with or
without OpenGL.
And it will also provides more of the API needed for simple games.
https://www.gnu.org/software/guile-sdl/
https://dthompson.us/projects/guile-sdl2.html
See also: http://libsdl.org/


Well, I considered both when I started the course :)

The problem is their level is not as high as I'd like. The course I'm following is for beginner programmers, so they use higher level libraries for drawing and animating. For example, you can write a checker pattern function like this:

(define (checker-pattern color-a color-b)
(above (beside (square 20 "solid" color-a) (square 20 "solid" color-b))
(beside (square 20 "solid" color-b) (square 20 "solid" color-a))))

The functions "above", "beside", and "square" come with a Racket's image library. Guile cairo's interface is lower level, it feels like reading the SVG specification, but without the geometric shapes.

I think the situation with SDL is similar.

Maybe when I finish the course I can start designing a very high level library using the existin

[David Pirotte]

Heya,

Post by sirgazil
There is Guile-Cairo if you just want a basic canvas.
http://www.nongnu.org/guile-cairo/
See also: https://www.cairographics.org/

Which you can use from Guile-Clutter [1], a must. I maintain a personal toolbox,
called grip, which as some guile-clutter examples [2], some using cairo (the
spline, the clock ...)

https://www.gnu.org/software/guile-gnome/clutter/ [1]
http://www.nongnu.org/grip/examples.html [2]

Post by sirgazil
The problem is their level is not as high as I'd like. The course I'm following is
for beginner programmers, so they use higher level libraries for drawing and
(define (checker-pattern color-a color-b)
(above (beside (square 20 "solid" color-a) (square 20 "solid" color-b))
(beside (square 20 "solid" color-b) (square 20 "solid" color-a))))
The functions "above", "beside", and "square" come with a Racket's image library.
Guile cairo's interface is lower level, it feels like reading the SVG
specification, but without the geometric shapes.

I'm working on Guile-CV, a computer vision functional programming library for Guile,
which provides a high level API. I uploaded a tarball here (for GNU evaluators),
here:

http://download.savannah.gnu.org/releases/grip/guile-cv/guile-cv-0.1.0.tar.gz
http://download.savannah.gnu.org/releases/grip/guile-cv/guile-cv-0.1.0.tar.gz.sig

[ GPG Key: A3057AD7
[ gpg --keyserver keys.gnupg.net --recv-keys A3057AD7

[ I will upload a new version within the next few days, stay tuned...

Chers,
David

[sirgazil]

-------- Original Message --------
Subject: Re: How to make GNU Guile more successful
Local Time: 18 de febrero de 2017 11:55 AM
UTC Time: 18 de febrero de 2017 16:55
From: ***@altosw.be
To: sirgazil <***@protonmail.com>
Rodger Fox <***@openmailbox.org>, guile-***@gnu.org <guile-***@gnu.org>

Heya,

Post by sirgazil
There is Guile-Cairo if you just want a basic canvas.
http://www.nongnu.org/guile-cairo/
See also: https://www.cairographics.org/

Which you can use from Guile-Clutter [1], a must. I maintain a personal toolbox,
called grip, which as some guile-clutter examples [2], some using cairo (the
spline, the clock ...)

https://www.gnu.org/software/guile-gnome/clutter/ [1]
http://www.nongnu.org/grip/examples.html [2]

Post by sirgazil
The problem is their level is not as high as I'd like. The course I'm following is
for beginner programmers, so they use higher level libraries for drawing and
(define (checker-pattern color-a color-b)
(above (beside (square 20 "solid" color-a) (square 20 "solid" color-b))
(beside (square 20 "solid" color-b) (square 20 "solid" color-a))))
The functions "above", "beside", and "square" come with a Racket's image library.
Guile cairo's interface is lower level, it feels like reading the SVG
specification, but without the geometric shapes.

I'm working on Guile-CV, a computer vision functional programming library for Guile,
which provides a high level API. I uploaded a tarball here (for GNU evaluators),
here:

http://download.savannah.gnu.org/releases/grip/guile-cv/guile-cv-0.1.0.tar.gz
http://download.savannah.gnu.org/releases/grip/guile-cv/guile-cv-0.1.0.tar.gz.sig

[ GPG Key: A3057AD7
[ gpg --keyserver keys.gnupg.net --recv-keys A3057AD7

[ I will upload a new version within the next few days, stay tuned...

Chers,
David

Thanks, David

[David Pirotte]

Hello,

Thanks, David, I'll take a look at those :)

Welcome, for info, I've uploaded a new version:

http://download.savannah.gnu.org/releases/grip/guile-cv/
guile-cv-0.1.1.tar.gz
guile-cv-0.1.1.tar.gz.sig

[ GPG Key: A3057AD7
[ gpg --keyserver keys.gnupg.net --recv-keys A3057AD7

See NEWS and INSTALL...

Cheers,
David

[Michael Vehrs]

As a late-comer to this discussion, here are my two cents. The thing I
miss most is a central package repository like Eggs Unlimited
(http://wiki.call-cc.org/chicken-projects/egg-index-4.html), or the
Racket Package List (http://pkgs.racket-lang.org/), or CPAN, of course.
Sure, a bespoke package manager might be nifty, but a single curated
list of packages would be a game-changer.


Regards


Michael

[Arne Babenhauserheide]

Post by Michael Vehrs
As a late-comer to this discussion, here are my two cents. The thing I
miss most is a central package repository like Eggs Unlimited
(http://wiki.call-cc.org/chicken-projects/egg-index-4.html), or the
Racket Package List (http://pkgs.racket-lang.org/), or CPAN, of course.
Sure, a bespoke package manager might be nifty, but a single curated
list of packages would be a game-changer.

In theory we have guildhall for this: https://github.com/ijp/guildhall

In practice it does not provide a web interface for uploading packages.

If you want to do something truly exciting, you could take wingos fibers
and build a high performance web interface for guildhall with them.

This is something I’d love to do but I fear that it’s not high enough in
my todo list that I’ll actually get it done.¹

Best wishes,
Arne

¹: I became Freenet Release Manager a few months ago. That took away the
last free time I could allocate to new challenging projects.

[Michael Vehrs]

Post by Arne Babenhauserheide

Post by Michael Vehrs
As a late-comer to this discussion, here are my two cents. The thing I
miss most is a central package repository like Eggs Unlimited
(http://wiki.call-cc.org/chicken-projects/egg-index-4.html), or the
Racket Package List (http://pkgs.racket-lang.org/), or CPAN, of course.
Sure, a bespoke package manager might be nifty, but a single curated
list of packages would be a game-changer.

In theory we have guildhall for this: https://github.com/ijp/guildhall

In theory, yes. But there isn't actually very much available in the
repository.

Post by Arne Babenhauserheide
In practice it does not provide a web interface for uploading packages.
If you want to do something truly exciting, you could take wingos fibers
and build a high performance web interface for guildhall with them.

High performance is not really important in this case. We are not
talking about gazillions of npm packages.

Post by Arne Babenhauserheide
This is something I’d love to do but I fear that it’s not high enough in
my todo list that I’ll actually get it done.¹
Best wishes,
Arne
¹: I became Freenet Release Manager a few months ago. That took away the
last free time I could allocate to new challenging projects.

If someone had a realistic plan of building something like that, I might
be able to contribute. I am not going to tackle it alone.


Regards

Michael

[Arne Babenhauserheide]

Post by Michael Vehrs

Post by Arne Babenhauserheide

Post by Michael Vehrs
As a late-comer to this discussion, here are my two cents. The thing I
miss most is a central package repository like Eggs Unlimited
(http://wiki.call-cc.org/chicken-projects/egg-index-4.html), or the
Racket Package List (http://pkgs.racket-lang.org/), or CPAN, of course.
Sure, a bespoke package manager might be nifty, but a single curated
list of packages would be a game-changer.

In theory we have guildhall for this: https://github.com/ijp/guildhall

In theory, yes. But there isn't actually very much available in the
repository.

I think the main reasons for that are (a) that it’s unmaintained (you
can’t just get your package added), (b) that it’s underdocumented (it
doesn’t say how to add a package or create a repo in less than 5 steps),
and (c) that there is no web interface for uploading a package.

Post by Michael Vehrs

Post by Arne Babenhauserheide
In practice it does not provide a web interface for uploading packages.
If you want to do something truly exciting, you could take wingos fibers
and build a high performance web interface for guildhall with them.

High performance is not really important in this case. We are not
talking about gazillions of npm packages.

That’s true. That’s why I wrote exciting :)

You could also use the existing (web server) tools to build such a site.

Best wishes,
Arne

[Amirouche]

Post by Arne Babenhauserheide

Post by Michael Vehrs

Post by Arne Babenhauserheide

Post by Michael Vehrs
As a late-comer to this discussion, here are my two ce

In practice it does not provide a web interface for uploading packages.
If you want to do something truly exciting, you could take wingos fibers
and build a high performance web interface for guildhall with them.

High performance is not really important in this case. We are not
talking about gazillions of npm packages.

That’s true. That’s why I wrote exciting :)
You could also use the existing (web server) tools to build such a site.

We can make it exciting and use fibers web server nonetheless.

The interface of the *fibers* web server is the same as guile web
server interface.

Here is it [0]:

```

(use-modules (fibers web server))

(define (handler request body)
(values '((content-type . (text/plain)))
"Hello, World!"))

(run-server handler)

```

Basically you just need to change the use-modules line in your current
project.

[0]

[Mike Gran]

Post by Arne Babenhauserheide

Post by Michael Vehrs

Post by Arne Babenhauserheide
If you want to do something truly exciting, you could take wingos fibers
and build a high performance web interface for guildhall with them.

High performance is not really important in this case. We are not
talking about gazillions of npm packages.

That’s true. That’s why I wrote exciting :)
You could also use the existing (web server) tools to build such a site.

The fastest thing would be to create a git project with a
{meta}/{maintainer}/{project} type structure and let people hack away unsupervised
in their {maintainer} subdirectories. Then add a thin libraries that
allows one to pick a specific Git commit UUID as a version number.

It would be chaos, but, we don't need professionalism when there is nothing
about which to be professional.


Or, to be fancier, clone metacpan. It exists.


-Mike

[Amirouche]

Post by Mike Gran

Post by Arne Babenhauserheide

Post by Michael Vehrs

Post by Arne Babenhauserheide
If you want to do something truly exciting, you could take wingos fibers
and build a high performance web interface for guildhall with them.

High performance is not really important in this case. We are not
talking about gazillions of npm packages.

That’s true. That’s why I wrote exciting :)
You could also use the existing (web server) tools to build such a site.

The fastest thing would be to create a git project with a
{meta}/{maintainer}/{project} type structure and let people hack away unsupervised
in their {maintainer} subdirectories. Then add a thin libraries that
allows one to pick a specific Git commit UUID as a version number.
It would be chaos, but, we don't need professionalism when there is nothing
about which to be professional.
Or, to be fancier, clone metacpan. It exists.

Sorry, I don't understand you want to use git as the package index?

[Mike Gran]

Post by Amirouche
Sorry, I don't understand you want to use git as the package index?

Just have a top-level README.md that everyone edits as the package index.
It would suffice until so long as the number of packages remains in the low
hundreds.

[Amirouche]

Post by Michael Vehrs

Post by Arne Babenhauserheide

Post by Michael Vehrs
As a late-comer to this discussion, here are my two cents. The thing I
miss most is a central package repository like Eggs Unlimited
(http://wiki.call-cc.org/chicken-projects/egg-index-4.html), or the
Racket Package List (http://pkgs.racket-lang.org/), or CPAN, of course.
Sure, a bespoke package manager might be nifty, but a single curated
list of packages would be a game-changer.

In theory we have guildhall for this: https://github.com/ijp/guildhall

In theory, yes. But there isn't actually very much available in the
repository.

Post by Arne Babenhauserheide
In practice it does not provide a web interface for uploading packages.
If you want to do something truly exciting, you could take wingos fibers
and build a high performance web interface for guildhall with them.

High performance is not really important in this case. We are not
talking about gazillions of npm packages.

Post by Arne Babenhauserheide
This is something I’d love to do but I fear that it’s not high enough in
my todo list that I’ll actually get it done.¹
Best wishes,
Arne
¹: I became Freenet Release Manager a few months ago. That took away the
last free time I could allocate to new challenging projects.

If someone had a realistic plan of building something like that, I
might be able to contribute. I am not going to tackle it alone.

What's the status of guildhall, does it work? where are the latest changes?

amirouche ~ amz3

[Arne Babenhauserheide]

Post by Amirouche

Post by Michael Vehrs
If someone had a realistic plan of building something like that, I
might be able to contribute. I am not going to tackle it alone.

What's the status of guildhall, does it work? where are the latest changes?

What we need is someone who regularly puts the package folder onto a
static webserver (starnge that I did not think of just using a repo
instead of implementing a full web interface with access rights and
such).

Guildhall works with guile-2.0, and there are no changes (though I have
a local version in which `guild hall` outputs the default installation
locations).

However I get an error with guile 2.1.5:

$ make
./env /home/arne/.local/bin//guile-tools compile -Wunbound-variable -Warity-mismatch -Wformat -o "guildhall/cli/config.go" "guildhall/cli/config.scm"
Backtrace:
In ice-9/boot-9.scm:
2860:10 19 (define-module* _ #:filename _ #:pure _ #:version _ # _ 
)
2799:17 18 (resolve-interface (guildhall config) #:select _ #:hide 
)
2724:10 17 (_ (guildhall config) _ _ #:ensure _)
3000:16 16 (try-module-autoload _ _)
2336:4 15 (save-module-excursion #<procedure 6d5d80 at ice-9/boot
>)
3020:22 14 (_)
In unknown file:
13 (primitive-load-path "guildhall/config" #<procedure d47
>)
In guildhall/config.scm:
25:0 12 (_)
In ice-9/boot-9.scm:
2799:17 11 (resolve-interface (guildhall destination fhs) #:select 
)
2724:10 10 (_ (guildhall destination fhs) _ _ #:ensure _)
3000:16 9 (try-module-autoload _ _)
2336:4 8 (save-module-excursion #<procedure da8930 at ice-9/boot
>)
3020:22 7 (_)
In unknown file:
6 (primitive-load-path "guildhall/destination/fhs" #<proc
>)
In guildhall/destination/fhs.scm:
152:0 5 (_)
In guildhall/ext/inc/irregex-r6rs.scm:
1505:17 4 (sre->irregex _ . _)
2448:15 3 (sre->nfa _ 0)
2391:29 2 (lp ((* any) (: "." ($ (+ (~ numeric))))) 1 0 0)
2301:52 1 (lp ("." ($ (+ (~ numeric)))) 1 0 _)
In ice-9/boot-9.scm:
762:26 0 (dispatch-exception _ _ _)

Best wishes,
Arne

[Amirouche]

Post by Arne Babenhauserheide

Post by Amirouche

Post by Michael Vehrs
If someone had a realistic plan of building something like that, I
might be able to contribute. I am not going to tackle it alone.

What's the status of guildhall, does it work? where are the latest changes?

What we need is someone who regularly puts the package folder onto a
static webserver

What does it mean?

Post by Arne Babenhauserheide
(starnge that I did not think of just using a repo
instead of implementing a full web interface with access rights and
such).

The issue I have with the web repository (I assume you meant git repo),
is that it requires for the maintainer of the repository to push a button,
accept new packages (at the very least). Which would mean that I offer
some kind of guarantees over the content of the software that you
download. Which maybe right now I can assume, but if there is as much
as patch in the repo as there is in guix I will need help...

Post by Arne Babenhauserheide
Guildhall works with guile-2.0, and there are no changes (though I have
a local version in which `guild hall` outputs the default installation
locations).
$ make
./env /home/arne/.local/bin//guile-tools compile -Wunbound-variable -Warity-mismatch -Wformat -o "guildhall/cli/config.go" "guildhall/cli/config.scm"
2860:10 19 (define-module* _ #:filename _ #:pure _ #:version _ # _ …)
2799:17 18 (resolve-interface (guildhall config) #:select _ #:hide …)
2724:10 17 (_ (guildhall config) _ _ #:ensure _)
3000:16 16 (try-module-autoload _ _)
2336:4 15 (save-module-excursion #<procedure 6d5d80 at ice-9/boot…>)
3020:22 14 (_)
13 (primitive-load-path "guildhall/config" #<procedure d47…>)
25:0 12 (_)
2799:17 11 (resolve-interface (guildhall destination fhs) #:select …)
2724:10 10 (_ (guildhall destination fhs) _ _ #:ensure _)
3000:16 9 (try-module-autoload _ _)
2336:4 8 (save-module-excursion #<procedure da8930 at ice-9/boot…>)
3020:22 7 (_)
6 (primitive-load-path "guildhall/destination/fhs" #<proc…>)
152:0 5 (_)
1505:17 4 (sre->irregex _ . _)
2448:15 3 (sre->nfa _ 0)
2391:29 2 (lp ((* any) (: "." ($ (+ (~ numeric))))) 1 0 0)
2301:52 1 (lp ("." ($ (+ (~ numeric)))) 1 0 _)
762:26 0 (dispatch-exception _ _ _)
Best wishes,
Arne

[Nala Ginrut]

I think we have to elaborate the question clearer.

1. How to make guile-scheme more successful?
I think this is similar to ask "how to make scheme more successful".
This is the big question to the whole scheme community.

2. How to make guile platform more successful?
I this case, let me ask a question, if we have guile-python3
(compatible with most of Python3 features), and provide more
convenient FFI helper function to help bind more libraries in short
time, is it possible to attract more people from Python land? Given
we'll have good JIT compiler in the future.

Post by Amirouche

Post by Arne Babenhauserheide

Post by Amirouche

Post by Michael Vehrs
If someone had a realistic plan of building something like that, I
might be able to contribute. I am not going to tackle it alone.

What's the status of guildhall, does it work? where are the latest changes?

What we need is someone who regularly puts the package folder onto a
static webserver

What does it mean?

Post by Arne Babenhauserheide
(starnge that I did not think of just using a repo
instead of implementing a full web interface with access rights and
such).

The issue I have with the web repository (I assume you meant git repo),
is that it requires for the maintainer of the repository to push a button,
accept new packages (at the very least). Which would mean that I offer
some kind of guarantees over the content of the software that you
download. Which maybe right now I can assume, but if there is as much
as patch in the repo as there is in guix I will need help...

Post by Arne Babenhauserheide
Guildhall works with guile-2.0, and there are no changes (though I have
a local version in which `guild hall` outputs the default installation
locations).
$ make
./env /home/arne/.local/bin//guile-tools compile -Wunbound-variable
-Warity-mismatch -Wformat -o "guildhall/cli/config.go"
"guildhall/cli/config.scm"
2860:10 19 (define-module* _ #:filename _ #:pure _ #:version _ # _ …)
2799:17 18 (resolve-interface (guildhall config) #:select _ #:hide …)
2724:10 17 (_ (guildhall config) _ _ #:ensure _)
3000:16 16 (try-module-autoload _ _)
2336:4 15 (save-module-excursion #<procedure 6d5d80 at ice-9/boot…>)
3020:22 14 (_)
13 (primitive-load-path "guildhall/config" #<procedure d47…>)
25:0 12 (_)
2799:17 11 (resolve-interface (guildhall destination fhs) #:select …)
2724:10 10 (_ (guildhall destination fhs) _ _ #:ensure _)
3000:16 9 (try-module-autoload _ _)
2336:4 8 (save-module-excursion #<procedure da8930 at ice-9/boot…>)
3020:22 7 (_)
6 (primitive-load-path "guildhall/destination/fhs" #<proc…>)
152:0 5 (_)
1505:17 4 (sre->irregex _ . _)
2448:15 3 (sre->nfa _ 0)
2391:29 2 (lp ((* any) (: "." ($ (+ (~ numeric))))) 1 0 0)
2301:52 1 (lp ("." ($ (+ (~ numeric)))) 1 0 _)
762:26 0 (dispatch-exception _ _ _)
Best wishes,
Arne

[David Kastrup]

Post by Nala Ginrut
I think we have to elaborate the question clearer.
1. How to make guile-scheme more successful?
I think this is similar to ask "how to make scheme more successful".
This is the big question to the whole scheme community.
2. How to make guile platform more successful?
I this case, let me ask a question, if we have guile-python3
(compatible with most of Python3 features), and provide more
convenient FFI helper function to help bind more libraries in short
time, is it possible to attract more people from Python land? Given
we'll have good JIT compiler in the future.

Frankly, I doubt that migration of large Python-based applications is
going to be a thing when nobody can even be bothered with immersing
himself in the problems with migrating LilyPond from Guile-1.8 to
Guile-2.

[Nala Ginrut]

Post by David Kastrup
Frankly, I doubt that migration of large Python-based applications is
going to be a thing when nobody can even be bothered with immersing
himself in the problems with migrating LilyPond from Guile-1.8 to
Guile-2.

No, I don't think so.
If we have guile-python3, the migration work becomes attractive to
Guile community. Because each time you migrate a library, it can be
used in all languages implemented on Guile platform. Write only once
for all supported languages. Even if we don't care about Python (do
we?), we have to write many libraries for Guile to make it more
useful. It's the work Guile community has to do anyway.

And LilyPond is not a good case here, since not everybody needs it. I
think the best way to push a community is to provide convenient way to
let users who care certain library to contribute it. But we don't have
it now. For example, the documentation or tools to help 1.8->2.0.
Python has tools for Python2->Python3 and documents for it. It is the
management of Guile community, not technical problem.
Fortunately, it's just management problem, and it's easier to improve
than technical one, only if we found a persistent way to push and
there's enough contributors.

[David Kastrup]

Post by Nala Ginrut

Post by David Kastrup
Frankly, I doubt that migration of large Python-based applications is
going to be a thing when nobody can even be bothered with immersing
himself in the problems with migrating LilyPond from Guile-1.8 to
Guile-2.

No, I don't think so.
If we have guile-python3, the migration work becomes attractive to
Guile community. Because each time you migrate a library, it can be
used in all languages implemented on Guile platform.

The .go organization and call gate costs (for example constant string
conversions) and memory organization and foreign string hardiness issues
bogging down LilyPond will affect interfacing to every external library
with a high call rate for processing.

Post by Nala Ginrut
I think the best way to push a community is to provide convenient way
to let users who care certain library to contribute it. But we don't
have it now. For example, the documentation or tools to help 1.8->2.0.
Python has tools for Python2->Python3 and documents for it. It is the
management of Guile community, not technical problem. Fortunately,
it's just management problem, and it's easier to improve than
technical one, only if we found a persistent way to push and there's
enough contributors.

The technical problems won't go away by themselves.

So which migration of a large Python-based application do you expect to
become a thing without addressing significant amounts of technical
problems first? Or how do I have to interpret your "No, I don't think
so."?

[Nala Ginrut]

Post by David Kastrup
The .go organization and call gate costs (for example constant string
conversions) and memory organization and foreign string hardiness issues
bogging down LilyPond will affect interfacing to every external library
with a high call rate for processing.

I respect LilyPond contributors and people who put effort on it. But
it's not on my hack-line.
If someone really care about it, it'll be done someday definitely.

Post by David Kastrup
The technical problems won't go away by themselves.
So which migration of a large Python-based application do you expect to
become a thing without addressing significant amounts of technical
problems first? Or how do I have to interpret your "No, I don't think
so."?

If I don't say "no, I don't think so", should I say "well, yes, Guile
has no chance, let's remove it".? ;-)
I think most of the people in Guile community knows the problems and
difficulties, it's unnecessary to repeat them again, what we need is
to find the solution.
I gave part of solution, and I'm enjoying to go with it.
If you're afraid of "large python based apps", how about get a seat
and see how others solve it?
Anyway, talking the problems and difficulties people already know
again and again is a contribution too. Then we don't need to write
TODO.

Do you think I know nothing about the complexity? I'll tell you that I
love that complexity so I'm in.
When I came to Guile community 7 years ago, I know nothing as a pure newbie.
But now I know I can learn many things each time when I challenge the
difficulties.
If you feel depressed when you face LilyPond, please don't forget I
feel frustrated when I'm debugging the Artanis server-core and our JIT
compiler too.
No one is facing the easy job.

Frankly, I have better idea than migrating all Python pacakges: just
migrate when you need it, or write a better one.
I'm trying to use Guile things in real products, so I'm serious.

I still want to ask "why do we have to compare to Python?"
I'm going to write guile-python3 because some young people in my team
has difficult to use Scheme directly, and we don't want them to
rewrite whole things in Scheme again.
But Guile will bring many potential advantages rather than our current
Python and C++14 combination.

Be more confident, folks.
That's why the answer from me is "no, I don't think so".

Best regards.

[Arne Babenhauserheide]

Post by David Kastrup

Post by Nala Ginrut
I think we have to elaborate the question clearer.
1. How to make guile-scheme more successful?
I think this is similar to ask "how to make scheme more successful".
This is the big question to the whole scheme community.
2. How to make guile platform more successful?
I this case, let me ask a question, if we have guile-python3
(compatible with most of Python3 features), and provide more
convenient FFI helper function to help bind more libraries in short
time, is it possible to attract more people from Python land? Given
we'll have good JIT compiler in the future.

Frankly, I doubt that migration of large Python-based applications is
going to be a thing when nobody can even be bothered with immersing
himself in the problems with migrating LilyPond from Guile-1.8 to
Guile-2.

I worked on testing Lilypond with user installed Guile 2.x, does that
count?

Just to have it recorded somewhere, here’s a patch to lilypond along
with a copy of the bash history of the setup (cleaned up, it was many
times as long):

## patch

From bd2ffea6f4c4c1ede13f5ac82d0a8ce31ccfe3c7 Mon Sep 17 00:00:00 2001
Subject: [PATCH] Build fixes for Guile 2.1.x (not yet functional)

---
configure.ac | 7 ++++++-
lily/pdf-scheme.cc | 4 ++++
scm/memory-trace.scm | 3 ++-
3 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/configure.ac b/configure.ac
index d77ea15..393976b 100644
--- a/configure.ac
+++ b/configure.ac
@@ -267,7 +267,12 @@ STEPMAKE_FREETYPE2(freetype2, REQUIRED, 2.1.10)
STEPMAKE_WINDOWS

# guile executable for some scripts
-STEPMAKE_GUILE(OPTIONAL, 1.8.2, 1.9.0)
+if test "$GUILEv2" = "yes"
+then
+ STEPMAKE_GUILE(OPTIONAL, 2.0.7, 2.2.0)
+else
+ STEPMAKE_GUILE(OPTIONAL, 1.8.2, 1.9.0)
+fi

# perl for help2man and for mf2pt1.pl
STEPMAKE_PERL(REQUIRED)
diff --git a/lily/pdf-scheme.cc b/lily/pdf-scheme.cc
index da2ce2c..f5ae70c 100644
--- a/lily/pdf-scheme.cc
+++ b/lily/pdf-scheme.cc
@@ -91,7 +91,11 @@ LY_DEFINE (ly_encode_string_for_pdf, "ly:encode-string-for-pdf",
* (string->utf16 str 'big)
*/
if (g)
+#if GUILEV2
+ return scm_take_locale_stringn (g, bytes_written); // scm_take_str eventually frees g!
+#else
return scm_take_str (g, bytes_written); // scm_take_str eventually frees g!
+#endif
else
return str;
}
diff --git a/scm/memory-trace.scm b/scm/memory-trace.scm
index d8ffeb9..9ebd722 100644
--- a/scm/memory-trace.scm
+++ b/scm/memory-trace.scm
@@ -2,7 +2,8 @@

(define-module (scm memory-trace))
(use-modules (lily)
- (ice-9 format))
+ (ice-9 format)
+ (ice-9 threads))

(define-public (mtrace:start-trace freq)
(set! usecond-interval (inexact->exact (/ 1000000 freq)))
--
2.10.2


## bash history

sudo emerge media-fonts/tex-gyre
sudo nano /etc/portage/package.keywords/sonstiges
sudo emerge media-fonts/tex-gyre
sudo pmerge dblatex
git clone git://git.sv.gnu.org/lilypond.git
cd lilypond/
./autogen.sh --prefix ~/.local --enable-guile2
mkdir -p ~/texmf/lh
cp ~/Downloads/lhfnt35g-source.zip ~/texmf/lh/
unzip lhfnt35g-source.zip
cd ~/texmf/source/latex/lh
latex lcyfonts.ins
latex ot2fonts.ins; latex t2ccfonts.ins
ls ~/texmf/examples/
mkdir ~/texmf/lh
cp *sty ~/texmf/lh/
./autogen.sh --prefix ~/.local --enable-guile2
sudo emerge dev-texlive/texlive-langcyrillic
sudo pmerge dev-texlive/texlive-langcyrillic
make out/lilypond-invoke-editor
cd scripts/
/home/arne/lilypond/scripts/build/out/help2man out/lilypond-invoke-editor --no-discard-stderr
/home/arne/lilypond/scripts/build/out/help2man out/lilypond-invoke-editor
./autogen.sh --prefix ~/.local --enable-guile2
make
LD_LIBRARY_PATH=/home/arne/.local/lib/guile/2.2/ g++ -o out/lilypond ./out/modified-font-metric.o ./out/sequential-iterator.o ./out/hara-kiri-group-spanner.o ./out/completion-note-heads-engraver.o ./out/slur-score-parameters.o ./out/flag.o ./out/span-arpeggio-engraver.o ./out/beam-quanting.o ./out/dynamic-performer.o ./out/parse-scm.o ./out/tie.o ./out/bend-engraver.o ./out/staff-spacing.o ./out/rest-collision-engraver.o ./out/simple-spacer-scheme.o ./out/audio-item.o ./out/pdf-scheme.o ./out/paper-score.o ./out/slur-configuration.o ./out/constrained-breaking.o ./out/score-engraver.o ./out/unpure-pure-container.o ./out/multi-measure-rest-engraver.o ./out/system-start-delimiter-engraver.o ./out/context-property.o ./out/freetype.o ./out/interval-minefield.o ./out/kievan-ligature.o ./out/completion-rest-engraver.o ./out/pure-from-neighbor-engraver.o ./out/horizontal-bracket-engraver.o ./out/grob-closure.o ./out/default-bar-line-engraver.o ./out/control-track-performer.o ./out/file-name-map.o ./out/audio-staff.o ./out/line-spanner.o ./out/grob-info.o ./out/spring.o ./out/slur.o ./out/music-scheme.o ./out/program-option.o ./out/fretboard-engraver.o ./out/hyphen-engraver.o ./out/page-marker.o ./out/global-vars.o ./out/stream-event-scheme.o ./out/script-column-engraver.o ./out/bezier-bow.o ./out/page-breaking-scheme.o ./out/lily-version.o ./out/repeat-tie-engraver.o ./out/tweak-engraver.o ./out/staff-symbol-referencer.o ./out/moment-scheme.o ./out/note-heads-engraver.o ./out/paper-outputter.o ./out/music-wrapper.o ./out/grob.o ./out/performance-scheme.o ./out/bar-line.o ./out/page-turn-engraver.o ./out/dimensions-scheme.o ./out/tab-note-heads-engraver.o ./out/piano-pedal-bracket.o ./out/spaceable-grob.o ./out/event-iterator.o ./out/lyric-combine-music-iterator.o ./out/percent-repeat-iterator.o ./out/audio-element-info.o ./out/midi-cc-announcer.o ./out/font-metric.o ./out/note-spacing-engraver.o ./out/duration-scheme.o ./out/accidental-placement.o ./out/music-iterator.o ./out/rhythmic-head.o ./out/grob-array.o ./out/tab-tie-follow-engraver.o ./out/context.o ./out/dispatcher.o ./out/scale.o ./out/chord-name-engraver.o ./out/auto-change-iterator.o ./out/tie-engraver.o ./out/grid-line-interface.o ./out/forbid-break-engraver.o ./out/note-column.o ./out/lookup.o ./out/global-ctor.o ./out/custos-engraver.o ./out/piano-pedal-engraver.o ./out/simple-music-iterator.o ./out/pitch-scheme.o ./out/mensural-ligature-engraver.o ./out/articulations.o ./out/horizontal-bracket.o ./out/script-column.o ./out/global-context.o ./out/slur-performer.o ./out/guile-init.o ./out/stencil-scheme.o ./out/font-size-engraver.o ./out/rest-engraver.o ./out/skyline.o ./out/percent-repeat-item.o ./out/enclosing-bracket.o ./out/module-scheme.o ./out/input.o ./out/paper-outputter-scheme.o ./out/spacing-interface.o ./out/break-align-engraver.o ./out/chord-tremolo-iterator.o ./out/general-scheme.o ./out/least-squares.o ./out/audio-element.o ./out/paper-system.o ./out/lily-lexer-scheme.o ./out/beam.o ./out/music-sequence.o ./out/music.o ./out/stem.o ./out/relative-octave-check.o ./out/keyword.o ./out/translator-ctors.o ./out/stencil-integral.o ./out/arpeggio-engraver.o ./out/extender-engraver.o ./out/prob-scheme.o ./out/ligature-engraver.o ./out/lyric-hyphen.o ./out/simple-spacer.o ./out/output-property-engraver.o ./out/drum-note-performer.o ./out/fingering-column-engraver.o ./out/midi-item.o ./out/undead.o ./out/repeat-acknowledge-engraver.o ./out/grace-engraver.o ./out/figured-bass-position-engraver.o ./out/tie-formatting-problem.o ./out/context-mod-scheme.o ./out/pfb.o ./out/fingering-engraver.o ./out/profile.o ./out/break-alignment-interface.o ./out/trill-spanner-engraver.o ./out/multi-measure-rest.o ./out/collision-engraver.o ./out/system-start-delimiter.o ./out/one-line-auto-height-breaking.o ./out/minimal-page-breaking.o ./out/dot-column-engraver.o ./out/note-name-engraver.o ./out/breathing-sign.o ./out/paper-book.o ./out/sustain-pedal.o ./out/tie-configuration.o ./out/staff-grouper-interface.o ./out/output-def.o ./out/nested-property.o ./out/timing-translator.o ./out/protected-scm.o ./out/volta-bracket.o ./out/beam-performer.o ./out/beaming-pattern.o ./out/prob.o ./out/part-combine-engraver.o ./out/smobs.o ./out/system.o ./out/figured-bass-engraver.o ./out/paper-def.o ./out/performer-group.o ./out/rhythmic-column-engraver.o ./out/volta-engraver.o ./out/paper-column-engraver.o ./out/scm-hash.o ./out/beam-collision-engraver.o ./out/apply-context-iterator.o ./out/skyline-pair.o ./out/gdb.o ./out/spacing-spanner.o ./out/stanza-number-align-engraver.o ./out/line-interface.o ./out/measure-grouping-spanner.o ./out/staff-symbol-referencer-scheme.o ./out/stem-engraver.o ./out/font-metric-scheme.o ./out/beam-engraver.o ./out/freetype-error.o ./out/instrument-name-engraver.o ./out/partial-iterator.o ./out/lily-lexer.o ./out/metronome-engraver.o ./out/warn-scheme.o ./out/includable-lexer.o ./out/item.o ./out/spacing-engraver.o ./out/relocate.o ./out/pointer-group-interface.o ./out/measure-grouping-engraver.o ./out/grace-music.o ./out/staff-performer.o ./out/slur-engraver.o ./out/function-documentation.o ./out/dots.o ./out/performer.o ./out/page-spacing.o ./out/separating-line-group-engraver.o ./out/axis-group-interface-scheme.o ./out/box.o ./out/spacing-determine-loose-columns.o ./out/duration.o ./out/stencil-expression.o ./out/phrasing-slur-engraver.o ./out/side-position-interface.o ./out/pitch-interval.o ./out/accidental.o ./out/pure-from-neighbor-interface.o ./out/optimal-page-breaking.o ./out/spanner-break-forbid-engraver.o ./out/drum-note-engraver.o ./out/stanza-number-engraver.o ./out/midi-walker.o ./out/ly-module.o ./out/span-bar-stub-engraver.o ./out/book-scheme.o ./out/grid-point-engraver.o ./out/scheme-engraver.o ./out/dot-column.o ./out/note-head-scheme.o ./out/lily-imports.o ./out/font-config-scheme.o ./out/open-type-font-scheme.o ./out/hairpin.o ./out/quote-iterator.o ./out/ledger-line-engraver.o ./out/auto-beam-engraver.o ./out/pango-font.o ./out/misc.o ./out/repeated-music.o ./out/figured-bass-continuation.o ./out/grace-iterator.o ./out/cue-clef-engraver.o ./out/pango-select.o ./out/main.o ./out/music-wrapper-iterator.o ./out/pitch-squash-engraver.o ./out/tuplet-iterator.o ./out/stencil-interpret.o ./out/key-signature-interface.o ./out/pitched-trill-engraver.o ./out/score.o ./out/context-mod.o ./out/fingering-column.o ./out/music-function-scheme.o ./out/bar-number-engraver.o ./out/break-substitution.o ./out/sources.o ./out/piano-pedal-performer.o ./out/performance.o ./out/volta-repeat-iterator.o ./out/slash-repeat-engraver.o ./out/font-config.o ./out/lily-guile.o ./out/balloon.o ./out/text-spanner-engraver.o ./out/bezier.o ./out/stencil.o ./out/clef.o ./out/spacing-basic.o ./out/dynamic-engraver.o ./out/time-signature-engraver.o ./out/ottava-engraver.o ./out/music-output.o ./out/lily-parser.o ./out/ligature-bracket-engraver.o ./out/translator-dispatch-list.o ./out/lily-parser-scheme.o ./out/engraver-scheme.o ./out/dynamic-align-engraver.o ./out/part-combine-iterator.o ./out/custos.o ./out/translator-scheme.o ./out/slur-scoring.o ./out/accidental-engraver.o ./out/rest-collision.o ./out/staff-symbol-engraver.o ./out/spanner.o ./out/note-column-scheme.o ./out/piano-pedal-align-engraver.o ./out/clef-modifier.o ./out/note-performer.o ./out/pango-font-scheme.o ./out/melody-spanner.o ./out/tie-column.o ./out/tuplet-bracket.o ./out/episema-engraver.o ./out/lyric-extender.o ./out/grob-property.o ./out/lyric-combine-music.o ./out/vaticana-ligature.o ./out/cluster-engraver.o ./out/instrument-switch-engraver.o ./out/part-combine-part-iterator.o ./out/audio-column.o ./out/key-performer.o ./out/gregorian-ligature-engraver.o ./out/context-specced-music-iterator.o ./out/pango-select-scheme.o ./out/translator-group.o ./out/book.o ./out/stream-event.o ./out/staff-symbol.o ./out/font-interface.o ./out/lyric-engraver.o ./out/page-breaking.o ./out/engraver-group.o ./out/grob-smob.o ./out/grob-array-scheme.o ./out/rod.o ./out/moment.o ./out/dispatcher-scheme.o ./out/lilypond-version.o ./out/tab-staff-symbol-engraver.o ./out/font-select.o ./out/grob-interface-scheme.o ./out/tuplet-engraver.o ./out/balloon-engraver.o ./out/template5.o ./out/paper-column.o ./out/script-row-engraver.o ./out/clef-engraver.o ./out/one-page-breaking.o ./out/midi-chunk.o ./out/context-def.o ./out/paper-score-scheme.o ./out/axis-group-engraver.o ./out/cluster.o ./out/translator.o ./out/item-scheme.o ./out/double-percent-repeat-engraver.o ./out/separation-item.o ./out/key-engraver.o ./out/keep-alive-together-engraver.o ./out/source-file.o ./out/all-font-metrics-scheme.o ./out/self-alignment-interface.o ./out/axis-group-interface.o ./out/midi-cc-performer.o ./out/input-scheme.o ./out/lyric-performer.o ./out/grid-line-span-engraver.o ./out/glissando-engraver.o ./out/event-chord-iterator.o ./out/chord-name.o ./out/spring-smob.o ./out/score-performer.o ./out/paper-system-scheme.o ./out/coherent-ligature-engraver.o ./out/context-scheme.o ./out/tie-performer.o ./out/semi-tie.o ./out/chord-tremolo-engraver.o ./out/pitch.o ./out/listener.o ./out/percent-repeat-engraver.o ./out/note-head.o ./out/relative-octave-music.o ./out/note-spacing.o ./out/dimension-cache.o ./out/spacing-loose-columns.o ./out/melody-engraver.o ./out/page-spacing-result.o ./out/script-engraver.o ./out/ttf.o ./out/dot-formatting-problem.o ./out/all-font-metrics.o ./out/ledger-line-spanner.o ./out/staff-collecting-engraver.o ./out/spacing-options.o ./out/time-signature-performer.o ./out/pfb-scheme.o ./out/property-iterator.o ./out/vaticana-ligature-engraver.o ./out/script-interface.o ./out/score-scheme.o ./out/one-line-page-breaking.o ./out/page-marker-scheme.o ./out/spanner-scheme.o ./out/music-function.o ./out/ambitus-engraver.o ./out/new-fingering-engraver.o ./out/bar-check-iterator.o ./out/change-iterator.o ./out/context-handle.o ./out/parenthesis-engraver.o ./out/align-interface.o ./out/change-sequence-iterator.o ./out/column-x-positions.o ./out/output-def-scheme.o ./out/global-context-scheme.o ./out/tie-details.o ./out/rest.o ./out/note-collision.o ./out/dot-configuration.o ./out/tempo-performer.o ./out/translator-group-ctors.o ./out/ottava-bracket.o ./out/stem-tremolo.o ./out/mensural-ligature.o ./out/grob-pq-engraver.o ./out/lily-modules.o ./out/page-turn-page-breaking.o ./out/gregorian-ligature.o ./out/engraver.o ./out/rhythmic-music-iterator.o ./out/mark-engraver.o ./out/page-layout-problem-scheme.o ./out/vertical-align-engraver.o ./out/kievan-ligature-engraver.o ./out/tie-specification.o ./out/tuplet-number.o ./out/laissez-vibrer-engraver.o ./out/dots-engraver.o ./out/directional-element-interface.o ./out/concurrent-hairpin-engraver.o ./out/line-interface-scheme.o ./out/paper-book-scheme.o ./out/pointer-group-interface-scheme.o ./out/grob-interface.o ./out/open-type-font.o ./out/span-bar-engraver.o ./out/midi-stream.o ./out/page-layout-problem.o ./out/input-smob.o ./out/note-head-line-engraver.o ./out/bar-engraver.o ./out/text-interface.o ./out/text-engraver.o ./out/grob-scheme.o ./out/arpeggio.o ./out/semi-tie-column.o ./out/program-option-scheme.o ./out/footnote-engraver.o ./out/simultaneous-music-iterator.o ./out/breathing-sign-engraver.o ./out/grace-spacing-engraver.o ./out/lexer.o ./out/parser.o ./out/../../flower/out/library.a -ldl -L/home/arne/.local/lib -L/home/arne/.local/lib/guile/2.2 -lguile-2.2 -lgc -lpangoft2-1.0 -lpango-1.0 -lgobject-2.0 -lglib-2.0 -lfontconfig -lfreetype -lfontconfig -lfreetype -lfreetype -v 2>&1 | grep -i lib
nm out/lilypond
LD_LIBRARY_PATH=~/.local/lib/ out/lilypond
GUILE_LOAD_PATH=$GUILE_LOAD_PATH:../:../scm LD_LIBRARY_PATH=~/.local/lib/ out/lilypond
wget http://infinite-hands.draketo.de/infinite_hands_sheet.ly
wget -O mf/out/emmentaler-20.otf https://github.com/saebekassebil/subito/raw/master/resources/gonville/lilyfonts/otf/emmentaler-20.otf
GUILE_LOAD_PATH=$GUILE_LOAD_PATH:.:scm LD_LIBRARY_PATH=~/.local/lib/ lily/out/lilypond -I ly/ -I mf/out/ -I ps/ test.ly
cp lilypond-data_2.18.2-4.1_all.deb /tmp
scp kav:/usr/share/lilypond/2.16.2/fonts/otf/*otf mf/out/
GUILE_LOAD_PATH=$GUILE_LOAD_PATH:.:scm LD_LIBRARY_PATH=~/.local/lib/ lily/out/lilypond -I ly/ -I mf/out/ -I ps/ test.ly
git add configure.ac lily/general-scheme.cc lily/pdf-scheme.cc scm/memory-trace.scm
time GUILE_LOAD_PATH=$GUILE_LOAD_PATH:.:scm LD_LIBRARY_PATH=~/.local/lib/ lily/out/lilypond -I ly/ -I mf/out/ -I ps/ infinite_hands_sheet.ly
cat 0001-Build-fixes-for-Guile-2.1.x-not-yet-functional.patch
cat > 1 <<EOF
diff --git a/lily/general-scheme.cc b/lily/general-scheme.cc
index 1168ee9..2df63fc 100644
--- a/lily/general-scheme.cc
+++ b/lily/general-scheme.cc
@@ -275,7 +275,8 @@ LY_DEFINE (ly_protects, "ly:protects",
#if SCM_MAJOR_VERSION < 2 || SCM_MAJOR_VERSION == 2 && SCM_MINOR_VERSION < 1
return scm_protects;
#else
- return programming_error ("ly:protects is not supported in Guile 2.1");
+ // return programming_error ("ly:protects is not supported in Guile 2.1");
+ return 0; // FIXME: Evil hack just to get this to build.
#endif
}
EOF

Best wishes,
Arne

PS: If this provides at least a small step towards guile 2 in lilypond,
it’s worth its while.

[Thomas Morley]

Post by Arne Babenhauserheide

Post by David Kastrup

Post by Nala Ginrut
I think we have to elaborate the question clearer.
1. How to make guile-scheme more successful?
I think this is similar to ask "how to make scheme more successful".
This is the big question to the whole scheme community.
2. How to make guile platform more successful?
I this case, let me ask a question, if we have guile-python3
(compatible with most of Python3 features), and provide more
convenient FFI helper function to help bind more libraries in short
time, is it possible to attract more people from Python land? Given
we'll have good JIT compiler in the future.

Frankly, I doubt that migration of large Python-based applications is
going to be a thing when nobody can even be bothered with immersing
himself in the problems with migrating LilyPond from Guile-1.8 to
Guile-2.

I worked on testing Lilypond with user installed Guile 2.x, does that
count?
Just to have it recorded somewhere, here’s a patch to lilypond along
with a copy of the bash history of the setup (cleaned up, it was many
## patch
From bd2ffea6f4c4c1ede13f5ac82d0a8ce31ccfe3c7 Mon Sep 17 00:00:00 2001
Subject: [PATCH] Build fixes for Guile 2.1.x (not yet functional)
---
configure.ac | 7 ++++++-
lily/pdf-scheme.cc | 4 ++++
scm/memory-trace.scm | 3 ++-
3 files changed, 12 insertions(+), 2 deletions(-)
diff --git a/configure.ac b/configure.ac
index d77ea15..393976b 100644
--- a/configure.ac
+++ b/configure.ac
@@ -267,7 +267,12 @@ STEPMAKE_FREETYPE2(freetype2, REQUIRED, 2.1.10)
STEPMAKE_WINDOWS
# guile executable for some scripts
-STEPMAKE_GUILE(OPTIONAL, 1.8.2, 1.9.0)
+if test "$GUILEv2" = "yes"
+then
+ STEPMAKE_GUILE(OPTIONAL, 2.0.7, 2.2.0)
+else
+ STEPMAKE_GUILE(OPTIONAL, 1.8.2, 1.9.0)
+fi
# perl for help2man and for mf2pt1.pl
STEPMAKE_PERL(REQUIRED)
diff --git a/lily/pdf-scheme.cc b/lily/pdf-scheme.cc
index da2ce2c..f5ae70c 100644
--- a/lily/pdf-scheme.cc
+++ b/lily/pdf-scheme.cc
@@ -91,7 +91,11 @@ LY_DEFINE (ly_encode_string_for_pdf, "ly:encode-string-for-pdf",
* (string->utf16 str 'big)
*/
if (g)
+#if GUILEV2
+ return scm_take_locale_stringn (g, bytes_written); // scm_take_str eventually frees g!
+#else
return scm_take_str (g, bytes_written); // scm_take_str eventually frees g!
+#endif
else
return str;
}
diff --git a/scm/memory-trace.scm b/scm/memory-trace.scm
index d8ffeb9..9ebd722 100644
--- a/scm/memory-trace.scm
+++ b/scm/memory-trace.scm
@@ -2,7 +2,8 @@
(define-module (scm memory-trace))
(use-modules (lily)
- (ice-9 format))
+ (ice-9 format)
+ (ice-9 threads))
(define-public (mtrace:start-trace freq)
(set! usecond-interval (inexact->exact (/ 1000000 freq)))
--
2.10.2
## bash history
sudo emerge media-fonts/tex-gyre
sudo nano /etc/portage/package.keywords/sonstiges
sudo emerge media-fonts/tex-gyre
sudo pmerge dblatex
git clone git://git.sv.gnu.org/lilypond.git
cd lilypond/
./autogen.sh --prefix ~/.local --enable-guile2
mkdir -p ~/texmf/lh
cp ~/Downloads/lhfnt35g-source.zip ~/texmf/lh/
unzip lhfnt35g-source.zip
cd ~/texmf/source/latex/lh
latex lcyfonts.ins
latex ot2fonts.ins; latex t2ccfonts.ins
ls ~/texmf/examples/
mkdir ~/texmf/lh
cp *sty ~/texmf/lh/
./autogen.sh --prefix ~/.local --enable-guile2
sudo emerge dev-texlive/texlive-langcyrillic
sudo pmerge dev-texlive/texlive-langcyrillic
make out/lilypond-invoke-editor
cd scripts/
/home/arne/lilypond/scripts/build/out/help2man out/lilypond-invoke-editor --no-discard-stderr
/home/arne/lilypond/scripts/build/out/help2man out/lilypond-invoke-editor
./autogen.sh --prefix ~/.local --enable-guile2
make
LD_LIBRARY_PATH=/home/arne/.local/lib/guile/2.2/ g++ -o out/lilypond ./out/modified-font-metric.o ./out/sequential-iterator.o ./out/hara-kiri-group-spanner.o ./out/completion-note-heads-engraver.o ./out/slur-score-parameters.o ./out/flag.o ./out/span-arpeggio-engraver.o ./out/beam-quanting.o ./out/dynamic-performer.o ./out/parse-scm.o ./out/tie.o ./out/bend-engraver.o ./out/staff-spacing.o ./out/rest-collision-engraver.o ./out/simple-spacer-scheme.o ./out/audio-item.o ./out/pdf-scheme.o ./out/paper-score.o ./out/slur-configuration.o ./out/constrained-breaking.o ./out/score-engraver.o ./out/unpure-pure-container.o ./out/multi-measure-rest-engraver.o ./out/system-start-delimiter-engraver.o ./out/context-property.o ./out/freetype.o ./out/interval-minefield.o ./out/kievan-ligature.o ./out/completion-rest-engraver.o ./out/pure-from-neighbor-engraver.o ./out/horizontal-bracket-engraver.o ./out/grob-closure.o ./out/default-bar-line-engraver.o ./out/control-track-performer.o ./out/file-name-map.o ./out/audio-staff.o ./out/line-spanner.o ./out/grob-info.o ./out/spring.o ./out/slur.o ./out/music-scheme.o ./out/program-option.o ./out/fretboard-engraver.o ./out/hyphen-engraver.o ./out/page-marker.o ./out/global-vars.o ./out/stream-event-scheme.o ./out/script-column-engraver.o ./out/bezier-bow.o ./out/page-breaking-scheme.o ./out/lily-version.o ./out/repeat-tie-engraver.o ./out/tweak-engraver.o ./out/staff-symbol-referencer.o ./out/moment-scheme.o ./out/note-heads-engraver.o ./out/paper-outputter.o ./out/music-wrapper.o ./out/grob.o ./out/performance-scheme.o ./out/bar-line.o ./out/page-turn-engraver.o ./out/dimensions-scheme.o ./out/tab-note-heads-engraver.o ./out/piano-pedal-bracket.o ./out/spaceable-grob.o ./out/event-iterator.o ./out/lyric-combine-music-iterator.o ./out/percent-repeat-iterator.o ./out/audio-element-info.o ./out/midi-cc-announcer.o ./out/font-metric.o ./out/note-spacing-engraver.o ./out/duration-scheme.o ./out/accidental-placement.o ./out/music-iterator.o ./out/rhythmic-head.o ./out/grob-array.o ./out/tab-tie-follow-engraver.o ./out/context.o ./out/dispatcher.o ./out/scale.o ./out/chord-name-engraver.o ./out/auto-change-iterator.o ./out/tie-engraver.o ./out/grid-line-interface.o ./out/forbid-break-engraver.o ./out/note-column.o ./out/lookup.o ./out/global-ctor.o ./out/custos-engraver.o ./out/piano-pedal-engraver.o ./out/simple-music-iterator.o ./out/pitch-scheme.o ./out/mensural-ligature-engraver.o ./out/articulations.o ./out/horizontal-bracket.o ./out/script-column.o ./out/global-context.o ./out/slur-performer.o ./out/guile-init.o ./out/stencil-scheme.o ./out/font-size-engraver.o ./out/rest-engraver.o ./out/skyline.o ./out/percent-repeat-item.o ./out/enclosing-bracket.o ./out/module-scheme.o ./out/input.o ./out/paper-outputter-scheme.o ./out/spacing-interface.o ./out/break-align-engraver.o ./out/chord-tremolo-iterator.o ./out/general-scheme.o ./out/least-squares.o ./out/audio-element.o ./out/paper-system.o ./out/lily-lexer-scheme.o ./out/beam.o ./out/music-sequence.o ./out/music.o ./out/stem.o ./out/relative-octave-check.o ./out/keyword.o ./out/translator-ctors.o ./out/stencil-integral.o ./out/arpeggio-engraver.o ./out/extender-engraver.o ./out/prob-scheme.o ./out/ligature-engraver.o ./out/lyric-hyphen.o ./out/simple-spacer.o ./out/output-property-engraver.o ./out/drum-note-performer.o ./out/fingering-column-engraver.o ./out/midi-item.o ./out/undead.o ./out/repeat-acknowledge-engraver.o ./out/grace-engraver.o ./out/figured-bass-position-engraver.o ./out/tie-formatting-problem.o ./out/context-mod-scheme.o ./out/pfb.o ./out/fingering-engraver.o ./out/profile.o ./out/break-alignment-interface.o ./out/trill-spanner-engraver.o ./out/multi-measure-rest.o ./out/collision-engraver.o ./out/system-start-delimiter.o ./out/one-line-auto-height-breaking.o ./out/minimal-page-breaking.o ./out/dot-column-engraver.o ./out/note-name-engraver.o ./out/breathing-sign.o ./out/paper-book.o ./out/sustain-pedal.o ./out/tie-configuration.o ./out/staff-grouper-interface.o ./out/output-def.o ./out/nested-property.o ./out/timing-translator.o ./out/protected-scm.o ./out/volta-bracket.o ./out/beam-performer.o ./out/beaming-pattern.o ./out/prob.o ./out/part-combine-engraver.o ./out/smobs.o ./out/system.o ./out/figured-bass-engraver.o ./out/paper-def.o ./out/performer-group.o ./out/rhythmic-column-engraver.o ./out/volta-engraver.o ./out/paper-column-engraver.o ./out/scm-hash.o ./out/beam-collision-engraver.o ./out/apply-context-iterator.o ./out/skyline-pair.o ./out/gdb.o ./out/spacing-spanner.o ./out/stanza-number-align-engraver.o ./out/line-interface.o ./out/measure-grouping-spanner.o ./out/staff-symbol-referencer-scheme.o ./out/stem-engraver.o ./out/font-metric-scheme.o ./out/beam-engraver.o ./out/freetype-error.o ./out/instrument-name-engraver.o ./out/partial-iterator.o ./out/lily-lexer.o ./out/metronome-engraver.o ./out/warn-scheme.o ./out/includable-lexer.o ./out/item.o ./out/spacing-engraver.o ./out/relocate.o ./out/pointer-group-interface.o ./out/measure-grouping-engraver.o ./out/grace-music.o ./out/staff-performer.o ./out/slur-engraver.o ./out/function-documentation.o ./out/dots.o ./out/performer.o ./out/page-spacing.o ./out/separating-line-group-engraver.o ./out/axis-group-interface-scheme.o ./out/box.o ./out/spacing-determine-loose-columns.o ./out/duration.o ./out/stencil-expression.o ./out/phrasing-slur-engraver.o ./out/side-position-interface.o ./out/pitch-interval.o ./out/accidental.o ./out/pure-from-neighbor-interface.o ./out/optimal-page-breaking.o ./out/spanner-break-forbid-engraver.o ./out/drum-note-engraver.o ./out/stanza-number-engraver.o ./out/midi-walker.o ./out/ly-module.o ./out/span-bar-stub-engraver.o ./out/book-scheme.o ./out/grid-point-engraver.o ./out/scheme-engraver.o ./out/dot-column.o ./out/note-head-scheme.o ./out/lily-imports.o ./out/font-config-scheme.o ./out/open-type-font-scheme.o ./out/hairpin.o ./out/quote-iterator.o ./out/ledger-line-engraver.o ./out/auto-beam-engraver.o ./out/pango-font.o ./out/misc.o ./out/repeated-music.o ./out/figured-bass-continuation.o ./out/grace-iterator.o ./out/cue-clef-engraver.o ./out/pango-select.o ./out/main.o ./out/music-wrapper-iterator.o ./out/pitch-squash-engraver.o ./out/tuplet-iterator.o ./out/stencil-interpret.o ./out/key-signature-interface.o ./out/pitched-trill-engraver.o ./out/score.o ./out/context-mod.o ./out/fingering-column.o ./out/music-function-scheme.o ./out/bar-number-engraver.o ./out/break-substitution.o ./out/sources.o ./out/piano-pedal-performer.o ./out/performance.o ./out/volta-repeat-iterator.o ./out/slash-repeat-engraver.o ./out/font-config.o ./out/lily-guile.o ./out/balloon.o ./out/text-spanner-engraver.o ./out/bezier.o ./out/stencil.o ./out/clef.o ./out/spacing-basic.o ./out/dynamic-engraver.o ./out/time-signature-engraver.o ./out/ottava-engraver.o ./out/music-output.o ./out/lily-parser.o ./out/ligature-bracket-engraver.o ./out/translator-dispatch-list.o ./out/lily-parser-scheme.o ./out/engraver-scheme.o ./out/dynamic-align-engraver.o ./out/part-combine-iterator.o ./out/custos.o ./out/translator-scheme.o ./out/slur-scoring.o ./out/accidental-engraver.o ./out/rest-collision.o ./out/staff-symbol-engraver.o ./out/spanner.o ./out/note-column-scheme.o ./out/piano-pedal-align-engraver.o ./out/clef-modifier.o ./out/note-performer.o ./out/pango-font-scheme.o ./out/melody-spanner.o ./out/tie-column.o ./out/tuplet-bracket.o ./out/episema-engraver.o ./out/lyric-extender.o ./out/grob-property.o ./out/lyric-combine-music.o ./out/vaticana-ligature.o ./out/cluster-engraver.o ./out/instrument-switch-engraver.o ./out/part-combine-part-iterator.o ./out/audio-column.o ./out/key-performer.o ./out/gregorian-ligature-engraver.o ./out/context-specced-music-iterator.o ./out/pango-select-scheme.o ./out/translator-group.o ./out/book.o ./out/stream-event.o ./out/staff-symbol.o ./out/font-interface.o ./out/lyric-engraver.o ./out/page-breaking.o ./out/engraver-group.o ./out/grob-smob.o ./out/grob-array-scheme.o ./out/rod.o ./out/moment.o ./out/dispatcher-scheme.o ./out/lilypond-version.o ./out/tab-staff-symbol-engraver.o ./out/font-select.o ./out/grob-interface-scheme.o ./out/tuplet-engraver.o ./out/balloon-engraver.o ./out/template5.o ./out/paper-column.o ./out/script-row-engraver.o ./out/clef-engraver.o ./out/one-page-breaking.o ./out/midi-chunk.o ./out/context-def.o ./out/paper-score-scheme.o ./out/axis-group-engraver.o ./out/cluster.o ./out/translator.o ./out/item-scheme.o ./out/double-percent-repeat-engraver.o ./out/separation-item.o ./out/key-engraver.o ./out/keep-alive-together-engraver.o ./out/source-file.o ./out/all-font-metrics-scheme.o ./out/self-alignment-interface.o ./out/axis-group-interface.o ./out/midi-cc-performer.o ./out/input-scheme.o ./out/lyric-performer.o ./out/grid-line-span-engraver.o ./out/glissando-engraver.o ./out/event-chord-iterator.o ./out/chord-name.o ./out/spring-smob.o ./out/score-performer.o ./out/paper-system-scheme.o ./out/coherent-ligature-engraver.o ./out/context-scheme.o ./out/tie-performer.o ./out/semi-tie.o ./out/chord-tremolo-engraver.o ./out/pitch.o ./out/listener.o ./out/percent-repeat-engraver.o ./out/note-head.o ./out/relative-octave-music.o ./out/note-spacing.o ./out/dimension-cache.o ./out/spacing-loose-columns.o ./out/melody-engraver.o ./out/page-spacing-result.o ./out/script-engraver.o ./out/ttf.o ./out/dot-formatting-problem.o ./out/all-font-metrics.o ./out/ledger-line-spanner.o ./out/staff-collecting-engraver.o ./out/spacing-options.o ./out/time-signature-performer.o ./out/pfb-scheme.o ./out/property-iterator.o ./out/vaticana-ligature-engraver.o ./out/script-interface.o ./out/score-scheme.o ./out/one-line-page-breaking.o ./out/page-marker-scheme.o ./out/spanner-scheme.o ./out/music-function.o ./out/ambitus-engraver.o ./out/new-fingering-engraver.o ./out/bar-check-iterator.o ./out/change-iterator.o ./out/context-handle.o ./out/parenthesis-engraver.o ./out/align-interface.o ./out/change-sequence-iterator.o ./out/column-x-positions.o ./out/output-def-scheme.o ./out/global-context-scheme.o ./out/tie-details.o ./out/rest.o ./out/note-collision.o ./out/dot-configuration.o ./out/tempo-performer.o ./out/translator-group-ctors.o ./out/ottava-bracket.o ./out/stem-tremolo.o ./out/mensural-ligature.o ./out/grob-pq-engraver.o ./out/lily-modules.o ./out/page-turn-page-breaking.o ./out/gregorian-ligature.o ./out/engraver.o ./out/rhythmic-music-iterator.o ./out/mark-engraver.o ./out/page-layout-problem-scheme.o ./out/vertical-align-engraver.o ./out/kievan-ligature-engraver.o ./out/tie-specification.o ./out/tuplet-number.o ./out/laissez-vibrer-engraver.o ./out/dots-engraver.o ./out/directional-element-interface.o ./out/concurrent-hairpin-engraver.o ./out/line-interface-scheme.o ./out/paper-book-scheme.o ./out/pointer-group-interface-scheme.o ./out/grob-interface.o ./out/open-type-font.o ./out/span-bar-engraver.o ./out/midi-stream.o ./out/page-layout-problem.o ./out/input-smob.o ./out/note-head-line-engraver.o ./out/bar-engraver.o ./out/text-interface.o ./out/text-engraver.o ./out/grob-scheme.o ./out/arpeggio.o ./out/semi-tie-column.o ./out/program-option-scheme.o ./out/footnote-engraver.o ./out/simultaneous-music-iterator.o ./out/breathing-sign-engraver.o ./out/grace-spacing-engraver.o ./out/lexer.o ./out/parser.o ./out/../../flower/out/library.a -ldl -L/home/arne/.local/lib -L/home/arne/.local/lib/guile/2.2 -lguile-2.2 -lgc -lpangoft2-1.0 -lpango-1.0 -lgobject-2.0 -lglib-2.0 -lfontconfig -lfreetype -lfontconfig -lfreetype -lfreetype -v 2>&1 | grep -i lib
nm out/lilypond
LD_LIBRARY_PATH=~/.local/lib/ out/lilypond
GUILE_LOAD_PATH=$GUILE_LOAD_PATH:../:../scm LD_LIBRARY_PATH=~/.local/lib/ out/lilypond
wget http://infinite-hands.draketo.de/infinite_hands_sheet.ly
wget -O mf/out/emmentaler-20.otf https://github.com/saebekassebil/subito/raw/master/resources/gonville/lilyfonts/otf/emmentaler-20.otf
GUILE_LOAD_PATH=$GUILE_LOAD_PATH:.:scm LD_LIBRARY_PATH=~/.local/lib/ lily/out/lilypond -I ly/ -I mf/out/ -I ps/ test.ly
cp lilypond-data_2.18.2-4.1_all.deb /tmp
scp kav:/usr/share/lilypond/2.16.2/fonts/otf/*otf mf/out/
GUILE_LOAD_PATH=$GUILE_LOAD_PATH:.:scm LD_LIBRARY_PATH=~/.local/lib/ lily/out/lilypond -I ly/ -I mf/out/ -I ps/ test.ly
git add configure.ac lily/general-scheme.cc lily/pdf-scheme.cc scm/memory-trace.scm
time GUILE_LOAD_PATH=$GUILE_LOAD_PATH:.:scm LD_LIBRARY_PATH=~/.local/lib/ lily/out/lilypond -I ly/ -I mf/out/ -I ps/ infinite_hands_sheet.ly
cat 0001-Build-fixes-for-Guile-2.1.x-not-yet-functional.patch
cat > 1 <<EOF
diff --git a/lily/general-scheme.cc b/lily/general-scheme.cc
index 1168ee9..2df63fc 100644
--- a/lily/general-scheme.cc
+++ b/lily/general-scheme.cc
@@ -275,7 +275,8 @@ LY_DEFINE (ly_protects, "ly:protects",
#if SCM_MAJOR_VERSION < 2 || SCM_MAJOR_VERSION == 2 && SCM_MINOR_VERSION < 1
return scm_protects;
#else
- return programming_error ("ly:protects is not supported in Guile 2.1");
+ // return programming_error ("ly:protects is not supported in Guile 2.1");
+ return 0; // FIXME: Evil hack just to get this to build.
#endif
}
EOF
Best wishes,
Arne
PS: If this provides at least a small step towards guile 2 in lilypond,
it’s worth its while.
--
Unpolitisch sein
heißt politisch sein
ohne es zu merken

Hi Arne,

many thanks for your work!
I'll have a closer look tomorrow (it's in the middle of the night here)...

The main problem is not that we can't build lilypond with 2.0.14 or 2.1.7
Checkout the branch dev/guile-v2-work in the lilypond-repository (you
will need to rebase it against current master) and you'll be able to
build LilyPond with guile-2.0.13, so I'm pretty confident it will work
with 2.0.14. For building with guile 2.1.7 an additional patch is
needed, currently I've simply deleted all about ly:protects
(brute-force, I know...).

Though the perfomance of LilyPond is dramatically slowed down. Making
it effectively unusable for any longer input.
I'm currently testing this for builds with guile 1.8.8, 2.0.14 and
2.1.7 and will report tomorrow.

Cheers,
Harm

[Thomas Morley]

Hi Arne,

as promised here my thoughts/findings.
All tested with a LilyPond build from the rebased branch
dev/guile-v2-work, with some additional patches on top of it:

diff --git a/lily/general-scheme.cc b/lily/general-scheme.cc
index 1168ee9..db376d11 100644
--- a/lily/general-scheme.cc
+++ b/lily/general-scheme.cc
@@ -267,6 +267,7 @@ LY_DEFINE (ly_dimension_p, "ly:dimension?", 1, 0,
0, (SCM d),
/*
Debugging mem leaks:
*/
+/*
LY_DEFINE (ly_protects, "ly:protects",
0, 0, 0, (),
"Return hash of protected objects.")
@@ -278,6 +279,7 @@ LY_DEFINE (ly_protects, "ly:protects",
return programming_error ("ly:protects is not supported in Guile 2.1");
#endif
}
+*/

LY_DEFINE (ly_gettext, "ly:gettext",
1, 0, 0, (SCM original),
diff --git a/scm/lily.scm b/scm/lily.scm
index d3164e4..23b1647 100644
--- a/scm/lily.scm
+++ b/scm/lily.scm
@@ -836,10 +836,11 @@ messages into errors.")

(define-public (dump-gc-protects)
(set! gc-protect-stat-count (1+ gc-protect-stat-count))
- (let* ((protects (sort (hash-table->alist (ly:protects))
- (lambda (a b)
- (< (object-address (car a))
- (object-address (car b))))))
+ (let* (;(protects (sort (hash-table->alist (ly:protects))
+ ; (lambda (a b)
+ ; (< (object-address (car a))
+ ; (object-address (car b))))))
+ (protects '())
(out-file-name (string-append
"gcstat-" (number->string gc-protect-stat-count)
".scm"))


diff --git a/ly/scheme-sandbox.ly b/ly/scheme-sandbox.ly
index 7dec0dc..1f233c0 100644
--- a/ly/scheme-sandbox.ly
+++ b/ly/scheme-sandbox.ly
@@ -13,4 +13,8 @@
% requirements may be different.

#(newline)
-#(scm-style-repl)
+#(if (guile-v2)
+ (begin
+ (use-modules (system repl repl))
+ (start-repl))
+ (scm-style-repl))



The ly-file used for testing can be found at
http://lists.gnu.org/archive/html/lilypond-user/2016-11/msg00948.html
As the author says, this file was not finished, so several errors and
warnings because of suboptimal user-input happen.
Quite typical for ongoing work and good for testings.
Compiling the file Urtext.ly returns a 159-pages-pdf.

Post by Thomas Morley

Post by Arne Babenhauserheide
I worked on testing Lilypond with user installed Guile 2.x, does that
count?
Just to have it recorded somewhere, here’s a patch to lilypond along
with a copy of the bash history of the setup (cleaned up, it was many
## patch
From bd2ffea6f4c4c1ede13f5ac82d0a8ce31ccfe3c7 Mon Sep 17 00:00:00 2001
Subject: [PATCH] Build fixes for Guile 2.1.x (not yet functional)
---
configure.ac | 7 ++++++-
lily/pdf-scheme.cc | 4 ++++
scm/memory-trace.scm | 3 ++-
3 files changed, 12 insertions(+), 2 deletions(-)
diff --git a/configure.ac b/configure.ac
index d77ea15..393976b 100644
--- a/configure.ac
+++ b/configure.ac
@@ -267,7 +267,12 @@ STEPMAKE_FREETYPE2(freetype2, REQUIRED, 2.1.10)
STEPMAKE_WINDOWS
# guile executable for some scripts
-STEPMAKE_GUILE(OPTIONAL, 1.8.2, 1.9.0)
+if test "$GUILEv2" = "yes"
+then
+ STEPMAKE_GUILE(OPTIONAL, 2.0.7, 2.2.0)
+else
+ STEPMAKE_GUILE(OPTIONAL, 1.8.2, 1.9.0)
+fi

The above is already in, see
$ git log -p 91ff9563ebe1c1cd720ad1a44890e7375fd83da8

Post by Thomas Morley

Post by Arne Babenhauserheide
# perl for help2man and for mf2pt1.pl
STEPMAKE_PERL(REQUIRED)
diff --git a/lily/pdf-scheme.cc b/lily/pdf-scheme.cc
index da2ce2c..f5ae70c 100644
--- a/lily/pdf-scheme.cc
+++ b/lily/pdf-scheme.cc
@@ -91,7 +91,11 @@ LY_DEFINE (ly_encode_string_for_pdf, "ly:encode-string-for-pdf",
* (string->utf16 str 'big)
*/
if (g)
+#if GUILEV2
+ return scm_take_locale_stringn (g, bytes_written); // scm_take_str eventually frees g!
+#else
return scm_take_str (g, bytes_written); // scm_take_str eventually frees g!
+#endif
else
return str;
}

Instead of the above, currently this code is in:

diff --git a/lily/pdf-scheme.cc b/lily/pdf-scheme.cc
index da2ce2c..61cf382 100644
--- a/lily/pdf-scheme.cc
+++ b/lily/pdf-scheme.cc
@@ -84,14 +84,16 @@ LY_DEFINE (ly_encode_string_for_pdf,
"ly:encode-string-for-pdf",
free (p);

/* Convert back to SCM object and return it */
- /* FIXME guile-2.0: With guile 2.0 the internal representation of a string
- * has changed (char vector rather than binary bytes in
- * UTF-8). However, with guile 2.0, ly:encode-string-for-pdf
- * is no longer needed and can be replaced by the new
- * (string->utf16 str 'big)
- */
if (g)
- return scm_take_str (g, bytes_written); // scm_take_str eventually frees g!
+ {
+ /*
+ * Return the raw byte representation of the UTF-16BE encoded string,
+ * in a locale independent way.
+ */
+ SCM string = scm_from_latin1_stringn (g, bytes_written);
+ free(g);
+ return string;
+ }
else
return str;
}
See:
git log -p d15c38c0ddd4c04edcf82cda50ca30f6dc4941fa

Though, I tested your suggestion as well.
It changes one of our regtests, utf-8.ly. But I have to admit I tested
only this one (not all regtests) and did not try to build the docs.
(All this stuff is pretty heavy for my admittedly weak laptop, it
lasts ages...)
Compiling the above mentioned Urtext.ly-file with your proposal shows
no significant difference for timings (see below)

Post by Thomas Morley

Post by Arne Babenhauserheide
diff --git a/scm/memory-trace.scm b/scm/memory-trace.scm
index d8ffeb9..9ebd722 100644
--- a/scm/memory-trace.scm
+++ b/scm/memory-trace.scm
@@ -2,7 +2,8 @@
(define-module (scm memory-trace))
(use-modules (lily)
- (ice-9 format))
+ (ice-9 format)
+ (ice-9 threads))
(define-public (mtrace:start-trace freq)
(set! usecond-interval (inexact->exact (/ 1000000 freq)))
--
2.10.2

Thanks for the above.
It fixes an annoying warning when LilyPond uses guile2.
I'll have to test, but I guess it's compatible with guile-1.
If so I'll likely check it in, ofcourse giving proper credits ;)

[...]

Post by Thomas Morley

Post by Arne Babenhauserheide
PS: If this provides at least a small step towards guile 2 in lilypond,
it’s worth its while.
--
Unpolitisch sein
heißt politisch sein
ohne es zu merken

Hi Arne,
many thanks for your work!
I'll have a closer look tomorrow (it's in the middle of the night here)...
The main problem is not that we can't build lilypond with 2.0.14 or 2.1.7
Checkout the branch dev/guile-v2-work in the lilypond-repository (you
will need to rebase it against current master) and you'll be able to
build LilyPond with guile-2.0.13, so I'm pretty confident it will work
with 2.0.14. For building with guile 2.1.7 an additional patch is
needed, currently I've simply deleted all about ly:protects
(brute-force, I know...).
Though the perfomance of LilyPond is dramatically slowed down. Making
it effectively unusable for any longer input.
I'm currently testing this for builds with guile 1.8.8, 2.0.14 and
2.1.7 and will report tomorrow.

Here some timing values

(1)
lilypond-2.19.52 using guile 1.8.7
(I would have prefered to build lilypond with a guile-1.8.8 build from
the guile-repository. Though my try to build it from the
branch_release-1-8 failed. Instead attempting to fix it, I then used a
released lilypond-version)

real 8m16.191s
user 6m39.864s
sys 0m10.860s

(2)
guile-2.0.14 build from guile-git-repository, branch remotes/origin/stable-2.0
lilypond-2.19.56, build from local branch dev/guile-v2.2-work

real 34m11.762s
user 45m11.316s
sys 0m5.604s

(3)
guile-2.1.7 build from guile-git-repository, branch master
(I've got this warning:
configure: WARNING: *** GNU Readline is too old on your system.
configure: WARNING: *** You need readline version 2.1 or later.
No idea whether this may have an impact on lilyponds compiling-time
I'll have to test.)
lilypond-2.19.56, build from local branch dev/guile-v2.2-work

real 67m29.132s
user 93m14.812s
sys 0m7.332s


Thanks,
Harm

[David Kastrup]

Post by Thomas Morley
Here some timing values
(1)
lilypond-2.19.52 using guile 1.8.7
(I would have prefered to build lilypond with a guile-1.8.8 build from
the guile-repository. Though my try to build it from the
branch_release-1-8 failed. Instead attempting to fix it, I then used a
released lilypond-version)
real 8m16.191s
user 6m39.864s
sys 0m10.860s
(2)
guile-2.0.14 build from guile-git-repository, branch remotes/origin/stable-2.0
lilypond-2.19.56, build from local branch dev/guile-v2.2-work
real 34m11.762s
user 45m11.316s
sys 0m5.604s
(3)
guile-2.1.7 build from guile-git-repository, branch master
configure: WARNING: *** GNU Readline is too old on your system.
configure: WARNING: *** You need readline version 2.1 or later.
No idea whether this may have an impact on lilyponds compiling-time
I'll have to test.)
lilypond-2.19.56, build from local branch dev/guile-v2.2-work
real 67m29.132s
user 93m14.812s
sys 0m7.332s

Same compilation options? This looks like a surprisingly serious
regression compared to 2.0.

[Thomas Morley]

Post by David Kastrup

Post by Thomas Morley
Here some timing values
(1)
lilypond-2.19.52 using guile 1.8.7
(I would have prefered to build lilypond with a guile-1.8.8 build from
the guile-repository. Though my try to build it from the
branch_release-1-8 failed. Instead attempting to fix it, I then used a
released lilypond-version)
real 8m16.191s
user 6m39.864s
sys 0m10.860s
(2)
guile-2.0.14 build from guile-git-repository, branch remotes/origin/stable-2.0
lilypond-2.19.56, build from local branch dev/guile-v2.2-work
real 34m11.762s
user 45m11.316s
sys 0m5.604s
(3)
guile-2.1.7 build from guile-git-repository, branch master
configure: WARNING: *** GNU Readline is too old on your system.
configure: WARNING: *** You need readline version 2.1 or later.
No idea whether this may have an impact on lilyponds compiling-time
I'll have to test.)
lilypond-2.19.56, build from local branch dev/guile-v2.2-work
real 67m29.132s
user 93m14.812s
sys 0m7.332s

Same compilation options?

Yep.
To get comparable results I always did exactly the same, for building
the guile-versions as well as for building lilypond.
Doing all tests with a fresh restarted computer.


Cheers,
Harm

Post by David Kastrup
This looks like a surprisingly serious
regression compared to 2.0.
--
David Kastrup

[Thomas Morley]

Post by Thomas Morley
The above is already in, see
$ git log -p 91ff9563ebe1c1cd720ad1a44890e7375fd83da8

[...}

Post by Thomas Morley
git log -p d15c38c0ddd4c04edcf82cda50ca30f6dc4941fa

Aargh, those are from my local _rebased_ branch.
Try instead:
e23d7b1763fdc2e7815c7069d5d1702f1a132383
59197ffbf370b12986ef3f3ee225322048110477

Sorry,
Harm

[Andy Wingo]

Post by Thomas Morley
Here some timing values
(1)
lilypond-2.19.52 using guile 1.8.7
(I would have prefered to build lilypond with a guile-1.8.8 build from
the guile-repository. Though my try to build it from the
branch_release-1-8 failed. Instead attempting to fix it, I then used a
released lilypond-version)
real 8m16.191s
user 6m39.864s
sys 0m10.860s
(2)
guile-2.0.14 build from guile-git-repository, branch remotes/origin/stable-2.0
lilypond-2.19.56, build from local branch dev/guile-v2.2-work
real 34m11.762s
user 45m11.316s
sys 0m5.604s
(3)
guile-2.1.7 build from guile-git-repository, branch master
configure: WARNING: *** GNU Readline is too old on your system.
configure: WARNING: *** You need readline version 2.1 or later.
No idea whether this may have an impact on lilyponds compiling-time
I'll have to test.)
lilypond-2.19.56, build from local branch dev/guile-v2.2-work
real 67m29.132s
user 93m14.812s
sys 0m7.332s

Oh, that's interesting. IME the only thing that is slower on 2.2
compared to 2.0 is the compiler; everything else is significantly
faster. Could it be that Lilypond is spending time compiling things
somehow? Perhaps via scm_load without disabling autocompilation or
something? Guessing at this point tho.

But that's getting ahead of myself -- do you have a document that
exhibits a similar performance series (1.8 < 2.0 < 2.2) but which
doesn't take so long to run? That can make perf investigation a bit
more tractable :)

Andy

[Thomas Morley]

Post by Andy Wingo

Post by Thomas Morley
Here some timing values
(1)
lilypond-2.19.52 using guile 1.8.7
(I would have prefered to build lilypond with a guile-1.8.8 build from
the guile-repository. Though my try to build it from the
branch_release-1-8 failed. Instead attempting to fix it, I then used a
released lilypond-version)
real 8m16.191s
user 6m39.864s
sys 0m10.860s
(2)
guile-2.0.14 build from guile-git-repository, branch remotes/origin/stable-2.0
lilypond-2.19.56, build from local branch dev/guile-v2.2-work
real 34m11.762s
user 45m11.316s
sys 0m5.604s
(3)
guile-2.1.7 build from guile-git-repository, branch master
configure: WARNING: *** GNU Readline is too old on your system.
configure: WARNING: *** You need readline version 2.1 or later.
No idea whether this may have an impact on lilyponds compiling-time
I'll have to test.)
lilypond-2.19.56, build from local branch dev/guile-v2.2-work
real 67m29.132s
user 93m14.812s
sys 0m7.332s

Oh, that's interesting. IME the only thing that is slower on 2.2
compared to 2.0 is the compiler; everything else is significantly
faster. Could it be that Lilypond is spending time compiling things
somehow? Perhaps via scm_load without disabling autocompilation or
something? Guessing at this point tho.

At least git-grepping for scm_load inthe lilypond-repository returns nothing.

Post by Andy Wingo
But that's getting ahead of myself -- do you have a document that
exhibits a similar performance series (1.8 < 2.0 < 2.2) but which
doesn't take so long to run? That can make perf investigation a bit
more tractable :)
Andy

Attached you'll find short-lily-guile-test.ly.
Two overrides, foo and fooI, are defined to show embedding lily in
guile and vice versa as well as some simple string-operations.
(The overrides themselves print silly things, but it's typical
user-generated code, just to demonstrate the method)
Below you'll find timing values for lilypond with guile 1.8.7, 2.0.14 and 2.1.7.
Those three lily-versions are tested without foo and fooI, with foo
and with foo _and_ fooI

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
lilypond 2.19.56 guile 1.8.7
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

real 0m2.207s
user 0m2.120s
sys 0m0.076s

with foo:

real 0m6.707s
user 0m6.336s
sys 0m0.372s

with foo and fooI:

real 0m9.749s
user 0m9.268s
sys 0m0.472s


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
lilypond 2.19.57 guile 2.0.14
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

real 0m11.843s
user 0m12.452s
sys 0m0.172s

with foo:

real 0m34.020s
user 0m35.284s
sys 0m0.168s


with foo and fooI:

real 0m38.819s
user 0m40.020s
sys 0m0.396s


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
lilypond 2.19.57 guile 2.1.7
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

real 0m8.594s
user 0m9.424s
sys 0m0.180s

with foo:

real 0m14.626s
user 0m15.924s
sys 0m0.204s

with foo and fooI:

real 0m19.422s
user 0m20.960s
sys 0m0.304s


So guile 2.1.7 is indeed faster than 2.0.14 with this test-file, otoh
I've redone testings with the other file and can confirm 2.1.7 being
slower there.
Currently I've no clue why.


Btw, I've improved my local setup to be able to test lilypond more
quickly with different guile versions. Though I wasn't able to compile
1.8.8, neither from the repository nor from the tarball downloaded
from
https://www.gnu.org/software/guile/download/
Due to:
async.c: In function 'scm_i_queue_async_cell':
async.c:243:14: error: variable 'count' set but not used
[-Werror=unused-but-set-variable]
size_t count;
^

Am I missing something?
I'm aware noone is interested in developing 1.8.8 further, though I
would have prefered to build lilypond with that version as well, like
the other test-versions.

Cheers,
Harm

[David Kastrup]

Post by Thomas Morley
So guile 2.1.7 is indeed faster than 2.0.14 with this test-file, otoh
I've redone testings with the other file and can confirm 2.1.7 being
slower there.
Currently I've no clue why.

Lot's of output? The output files are generated in
lily/paper-outputter.cc with

SCM
Paper_outputter::dump_string (SCM scm)
{
return scm_display (scm, file ());
}

SCM
Paper_outputter::scheme_to_string (SCM scm)
{
return scm_eval (scm, output_module_);
}

SCM
Paper_outputter::module () const
{
return output_module_;
}

SCM
Paper_outputter::output_scheme (SCM scm)
{
SCM str = scheme_to_string (scm);
if (scm_is_string (str))
dump_string (str);
return str;
}

SCM
paper_outputter_dump (void *po, SCM x)
{
Paper_outputter *me = (Paper_outputter *) po;
return me->output_scheme (x);
}

void
Paper_outputter::output_stencil (Stencil stil)
{
interpret_stencil_expression (stil.expr (), paper_outputter_dump,
(void *) this, Offset (0, 0));
}

So every output item is generated by running a humongous expression
through scm_eval and then calling scm_display on the expression (when it
turns out it is a string).

For PDF output, those strings are generated in the PostScript backend in
scm/output-ps.scm, typically with stuff like:

(define (char font i)
(ly:format "~a (\\~a) show"
(ps-font-command font)
(ly:inexact->string i 8)))

(define (circle radius thick fill)
(ly:format
"~a ~4f ~4f draw_circle"
(if fill
"true"
"false")
radius thick))

with ly:format defined in C and consequently ping-ponging strings
through the SCM API (using scm_to_locale_stringbuf and
scm_take_locale_stringn).

So the basic question is whether the output generation phase (after all
typesetting and page breaking has finished) is significantly involved in
the total slowdown or not.

Post by Thomas Morley
Btw, I've improved my local setup to be able to test lilypond more
quickly with different guile versions. Though I wasn't able to compile
1.8.8, neither from the repository nor from the tarball downloaded
from
https://www.gnu.org/software/guile/download/
async.c:243:14: error: variable 'count' set but not used
[-Werror=unused-but-set-variable]
size_t count;
^
Am I missing something?

Remove compilation option -Wall here?

Post by Thomas Morley
I'm aware noone is interested in developing 1.8.8 further,

There was just a question on the developer list (I think) how to best
maintain a fork of 1.8.

[Thien-Thi Nguyen]

() Thomas Morley <***@gmail.com>
() Thu, 9 Mar 2017 00:17:12 +0100

Btw, I've improved my local setup to be able to test lilypond
more quickly with different guile versions. Though I wasn't
able to compile 1.8.8, neither from the repository

Strange, because the repo should have...

nor from the tarball downloaded from
https://www.gnu.org/software/guile/download/
Due to:
async.c: In function 'scm_i_queue_async_cell':
async.c:243:14: error: variable 'count' set but not used
[-Werror=unused-but-set-variable]
size_t count;
^

Am I missing something?

...a fix installed from 2012-05-01:

http://git.savannah.gnu.org/cgit/guile.git/commit/?h=branch_release-1-8&id=e2547476441

among other, more recent, changes. Note that this is on branch
‘branch_release-1-8’. Perhaps you were on a different branch?

[Ludovic Courtès]

Hello,

Post by Thomas Morley
Btw, I've improved my local setup to be able to test lilypond more
quickly with different guile versions. Though I wasn't able to compile
1.8.8, neither from the repository nor from the tarball downloaded
from
https://www.gnu.org/software/guile/download/
async.c:243:14: error: variable 'count' set but not used
[-Werror=unused-but-set-variable]
size_t count;
^
Am I missing something?

Could you try configuring like this:

./configure --disable-error-on-warning

?

Post by Thomas Morley
I'm aware noone is interested in developing 1.8.8 further, though I
would have prefered to build lilypond with that version as well, like
the other test-versions.

The performance gap in LilyPond between 1.8 and 2.0 is terrible. I
suppose LilyPond uses ‘eval’ to run Scheme code? What fraction of the
Scheme code being run for this benchmark is pre-compiled (as a .go
file)? Is auto-compilation enabled, and could it be that the figures
include auto-compilation time?

Thanks,
Ludo’.

[Paul]

What fraction of the Scheme code being run for this benchmark is pre-compiled (as a .go file)?

I don't think any of LilyPond's Scheme code is pre-compiled at this point...

Yep, as David Kastrup wrote in the "GNU Guile 2.1.7 released (beta)" thread on Feb 28, 2017:

<quote>
Regular read and eval, however, is used a lot during the parsing of
files and startup of LilyPond. But at least under Guile-1.8, the
parsing and preprocessing took up a rather small part of the overall
runtime (in the order of 15% or so), so it is unlikely to be responsible
for the bulk of the slowdown.

My personal guess is that the largest performance impact at the moment
will be due to an absence of generation and installation of .go files.
Since .go files are target-dependent (if I am not mistaken) and LilyPond
is cross-compiled for a number of architectures with different byte
orders and type sizes, it seems tricky to get this under wraps.

The next largest performance impact will be redecoding issues.
</quote>

-Paul

[Ludovic Courtès]

Post by Paul

What fraction of the Scheme code being run for this benchmark is pre-compiled (as a .go file)?

I don't think any of LilyPond's Scheme code is pre-compiled at this point...
<quote>
Regular read and eval, however, is used a lot during the parsing of
files and startup of LilyPond. But at least under Guile-1.8, the
parsing and preprocessing took up a rather small part of the overall
runtime (in the order of 15% or so), so it is unlikely to be responsible
for the bulk of the slowdown.
My personal guess is that the largest performance impact at the moment
will be due to an absence of generation and installation of .go files.
Since .go files are target-dependent (if I am not mistaken) and LilyPond
is cross-compiled for a number of architectures with different byte
orders and type sizes, it seems tricky to get this under wraps.
The next largest performance impact will be redecoding issues.
</quote>

Thanks. As Andy wrote in that thread, it would be beneficial if
LilyPond could pre-compile as much as possible of its core Scheme code.

Ludo’.

[Paul]

Post by Ludovic Courtès
Thanks. As Andy wrote in that thread, it would be beneficial if
LilyPond could pre-compile as much as possible of its core Scheme code.

Hi, Yeah, it seems like that would be the next step in addressing the
performance questions. That brings up how to compile for different
targets (from that other thread):

David: Since .go files are target-dependent (if I am not mistaken) and LilyPond
is cross-compiled for a number of architectures with different byte
orders and type sizes, it seems tricky to get this under wraps.

Andy: Yeah. In both 2.0 and 2.2 there are only four "targets" really (32-bit
and 64-bit, big- and little-endian), so it's somewhat manageable.
"guild compile" does support cross-compilation, and I think there are
some projects that do so; but yep, wiring that up can be tricky like you
say.


So maybe sorting out and/or documenting the "best practices" for
addressing this trickiness would be a mutually beneficial thing for both
LilyPond and Guile? (For anyone looking for ways to help.) Anyone know
of projects that are examples of how to manage this kind of
cross-compilation?

(I'm an occasional LilyPond contributor who learned Scheme from hacking
with/on LilyPond, and I'd love to see LilyPond working well with
Guile2. Ultimately, it seems like that would be in the best interests
of both projects, especially in the long run.)

Cheers,
-Paul

[Thomas Morley]

Hi,

Post by David Pirotte
Hello,

Post by Thomas Morley
Btw, I've improved my local setup to be able to test lilypond more
quickly with different guile versions. Though I wasn't able to compile
1.8.8, neither from the repository nor from the tarball downloaded
from
https://www.gnu.org/software/guile/download/
async.c:243:14: error: variable 'count' set but not used
[-Werror=unused-but-set-variable]
size_t count;
^
Am I missing something?

./configure --disable-error-on-warning
?

Works.
Thanks for the hint.

Post by David Pirotte

Post by Thomas Morley
I'm aware noone is interested in developing 1.8.8 further, though I
would have prefered to build lilypond with that version as well, like
the other test-versions.

The performance gap in LilyPond between 1.8 and 2.0 is terrible. I
suppose LilyPond uses ‘eval’ to run Scheme code? What fraction of the
Scheme code being run for this benchmark is pre-compiled (as a .go
file)? Is auto-compilation enabled, and could it be that the figures
include auto-compilation time?

I think/hope Paul answered already sufficiently.
Let me add, I'd be interested in examples of cross-compiled
applications having already done so, as well.
The guile2-manual says nothing about the best practise or perhaps I missed it.


Thanks,
Harm

[Ludovic Courtès]

Post by Thomas Morley
Let me add, I'd be interested in examples of cross-compiled
applications having already done so, as well.

It boils down to having a makefile rule along the lines of:

%.go: %.scm
guild compile --target="$(host)" -o $@ $<

where $host is the cross-compilation target triplet, such as
arm-linux-gnueaihf. See
<https://gitlab.com/gnutls/gnutls/blob/master/guile/Makefile.am#L78> for
an example.

HTH,
Ludo’.

[Matt Wette]

If lilypond is performing a lot of eval or lambda generation would turning off optimization help?

(compile expr #:opts ‘(#:partial-eval? #f #:cse? #f))

Matt

[Andy Wingo]

Post by Matt Wette
If lilypond is performing a lot of eval or lambda generation would turning off optimization help?
(compile expr #:opts ‘(#:partial-eval? #f #:cse? #f))

I think Lilypond is currently not going through the compiler at all, so
no partial evaluation or anything else is running -- it's just
macroexpand-then-interpret.

Incidentally in 2.2 the options you need to turn off optimization are a
little more complicated. We have "guild compile -O0" which will produce
the right set of options but nothing like #:optimize-level 0 or
something that you can pass to `compile'.

Andy

[Ludovic Courtès]

Post by Andy Wingo
Incidentally in 2.2 the options you need to turn off optimization are a
little more complicated. We have "guild compile -O0" which will produce
the right set of options but nothing like #:optimize-level 0 or
something that you can pass to `compile'.

What would you recommend as the main optimization to turn off in 2.2 if
one is to reduce compile time?

I’m asking in the context of Guix, where there’s no much to optimize in
files that just define packages.

Ludo’.

[Matt Wette]

Post by Nala Ginrut
2. How to make guile platform more successful?
I this case, let me ask a question, if we have guile-python3
(compatible with most of Python3 features), and provide more
convenient FFI helper function to help bind more libraries in short
time, is it possible to attract more people from Python land? Given
we'll have good JIT compiler in the future.

Now that the C preprocessor in NYACC/C99 is more robust I could start looking at generating some sort of “FFI helper” functionality. I have been thinking about this problem on and off for a while. Completely automating things might be tough, but a “helper” would probably go a long way.

Matt

[Amirouche]

Post by Matt Wette

Post by Nala Ginrut
2. How to make guile platform more successful?
I this case, let me ask a question, if we have guile-python3
(compatible with most of Python3 features), and provide more
convenient FFI helper function to help bind more libraries in short
time, is it possible to attract more people from Python land? Given
we'll have good JIT compiler in the future.

Now that the C preprocessor in NYACC/C99 is more robust I could start looking at generating some sort of “FFI helper” functionality. I have been thinking about this problem on and off for a while. Completely automating things might be tough, but a “helper” would probably go a long way.
Matt

Tx for working on that!

[Amirouche]

Post by Matt Wette

Post by Nala Ginrut
2. How to make guile platform more successful?
I this case, let me ask a question, if we have guile-python3
(compatible with most of Python3 features), and provide more
convenient FFI helper function to help bind more libraries in short
time, is it possible to attract more people from Python land? Given
we'll have good JIT compiler in the future.

Now that the C preprocessor in NYACC/C99 is more robust I could start looking at generating some sort of “FFI helper” functionality. I have been thinking about this problem on and off for a while. Completely automating things might be tough, but a “helper” would probably go a long way.
Matt

Are you aware of scheme-bytestructures [0] that's what we are using
in guile-git. If you target that API to wrap C structs, it would be helpful.

[0] https://github.com/TaylanUB/scheme-bytestructures

[Mike Gran]

Post by Matt Wette
Now that the C preprocessor in NYACC/C99 is more robust I could start looking at generating some sort of “FFI helper” functionality. I have been thinking about this problem on and off for a while. Completely automating things might be tough, but a “helper” would probably go a long way.

Are you aware of scheme-bytestructures [0] that's what we are using> in guile-git. If you target that API to wrap C structs, it would be helpful.

SWIG used to work for Guile. It was a rather complete solution to
the wrapping problem.

[Matt Wette]

Post by Mike Gran

Post by Matt Wette
Now that the C preprocessor in NYACC/C99 is more robust I could start looking at generating some sort of “FFI helper” functionality. I have been thinking about this problem on and off for a while. Completely automating things might be tough, but a “helper” would probably go a long way.

Are you aware of scheme-bytestructures [0] that's what we are using> in guile-git. If you target that API to wrap C structs, it would be helpful.

SWIG used to work for Guile. It was a rather complete solution to
the wrapping problem.

I had forgotten about that. Seems targeted to 1.X but might be worth trying to autogenerate swig template or equivalent to start.

[Matt Wette]

Post by Amirouche

Post by Matt Wette
Now that the C preprocessor in NYACC/C99 is more robust I could start looking at generating some sort of “FFI helper” functionality. I have been thinking about this problem on and off for a while. Completely automating things might be tough, but a “helper” would probably go a long way.

Are you aware of scheme-bytestructures [0] that's what we are using
in guile-git. If you target that API to wrap C structs, it would be helpful.
[0] https://github.com/TaylanUB/scheme-bytestructures <https://github.com/TaylanUB/scheme-bytestructures>

Cloned it. Thanks for the reference. — Matt

[Michael Vehrs]

Post by Michael Vehrs

Post by Arne Babenhauserheide

Post by Michael Vehrs
As a late-comer to this discussion, here are my two cents. The thing I
miss most is a central package repository like Eggs Unlimited
(http://wiki.call-cc.org/chicken-projects/egg-index-4.html), or the
Racket Package List (http://pkgs.racket-lang.org/), or CPAN, of course.
Sure, a bespoke package manager might be nifty, but a single curated
list of packages would be a game-changer.

In theory we have guildhall for this: https://github.com/ijp/guildhall

In theory, yes. But there isn't actually very much available in the
repository.

Post by Arne Babenhauserheide
In practice it does not provide a web interface for uploading packages.
If you want to do something truly exciting, you could take wingos fibers
and build a high performance web interface for guildhall with them.

High performance is not really important in this case. We are not
talking about gazillions of npm packages.

Post by Arne Babenhauserheide
This is something I’d love to do but I fear that it’s not high enough in
my todo list that I’ll actually get it done.¹
Best wishes,
Arne
¹: I became Freenet Release Manager a few months ago. That took away the
last free time I could allocate to new challenging projects.

If someone had a realistic plan of building something like that, I
might be able to contribute. I am not going to tackle it alone.
Regards
Michael

By the way, this (http://sph.mn/content/3e73) is the most complete list
of guile projects I have seen so far.


Regards

Michael

[Alejandro Sanchez]

If I may add my two cents as a Scheme layman: the question is not so much about making Guile more popular, but about making Scheme itself more popular.

One big reason for Python’s popularity is something I haven’t seen mentioned in this thread so far: if you know pseudocode you basically know Python already. Of course this is hyperbolic, there are a lot of finer details to Python, but the superficial simplicity of Python makes you think you already know the language and that you can get started right away. By the time you encounter the more arcane aspects of Python you have already invested enough time into it that you will put up with learning something new.

Scheme on the other hand is weird; weird in a good way, but still weird. For me the reason for picking up Scheme was working my way through SICP, but even before that I had heard of the Lisp family. Every time I tried to find out what the fuss was about Lisp all I could find was nebulous concepts like “it’s a programmable programming language” without explaining what that even meant and why I would ever want to use it.

Of course once I got over the weird parentheses and I understood why they are actually a beneficial aspect I liked it. Yet, even after getting into Scheme there are problems. The r5rs standard is pretty limited, and so every implementation does its own thing. There was not even a standard module system until r6rs. Unfortunately r6rs never got fully adopted by any of the major implementations, so we are back to where we were with r5rs more or less. r7rs-small looks like it could finally fill in the most glaring gaps, and r7rs-large is still nowhere done.

All this is very frustrating, and getting into a new language is a long-term commitment to learn something new. When faced with Python, Ruby, JavaScript, Lua and Scheme, it’s easy to go with the flow and just use what everyone else is using.

I think someone here mentioned the lack of a proper Scheme IDE, other than Dr. Racket for Racket. I don’t use IDEs anymore, but I can see how that can be a problem for other people who only want to do intermediate scripting rather than write entire applications. Writing a full IDE would be quite a lot of work, so perhaps integrating with existing IDEs would be better. Think something like Geiser for programs other than Emacs. There is the “language server protocol" by Microsoft, the idea is to have a standardised protocol that can be supported by a language server and an editor application.
https://github.com/Microsoft/language-server-protocol

I know it’s Microsoft, but this is just a specification. With this protocol one would write a Scheme server that receives instructions form the editor via RPC and sends the result back to the server via RPC as well. This would allow using the same Scheme server regardless of the editor (Emacs, Vim, Neovim, Eclipse, …). I had wanted to write something like this for Neovim myself, but I don’t know enough of Scheme yet.

I hope these ramblings of mine offer something useful from an outsider perspective to you guys.

[Arne Babenhauserheide]

Post by Alejandro Sanchez
If I may add my two cents as a Scheme layman: the question is not so much about making Guile more popular, but about making Scheme itself more popular.
One big reason for Python’s popularity is something I haven’t seen
mentioned in this thread so far: if you know pseudocode you basically
know Python already. Of course this is hyperbolic, there are a lot of
finer details to Python, but the superficial simplicity of Python
makes you think you already know the language and that you can get
started right away. By the time you encounter the more arcane aspects
of Python you have already invested enough time into it that you will
put up with learning something new.

This is basically what wisp¹ is intended to address:

define : hello who
format #t "Hello ~a!\n" who

hello "World"

And it’s still full scheme. Just looking like pseudocode.

I documented that in py2guile: http://draketo.de/py2guile

¹: http://draketo.de/english/wisp

Post by Alejandro Sanchez
I think someone here mentioned the lack of a proper Scheme IDE, other
than Dr. Racket for Racket. I don’t use IDEs anymore, but I can see
how that can be a problem for other people who only want to do
intermediate scripting rather than write entire applications.

I also see lots of people at work using Python IDEs, which all bring
their own problems but provide the kind of information I get from the
commandline.

Just having a geiser setup for Emacs properly documented — or maybe an
Emacs customized for Scheme development — would help a lot, I think.

"This is our canonical interface for writing Scheme" ← That is what
people need to be able to find.

Best wishes,
Arne

[Thien-Thi Nguyen]

() Arne Babenhauserheide <***@web.de>
() Sun, 05 Mar 2017 01:23:59 +0100

Just having a geiser setup for Emacs properly documented — or
maybe an Emacs customized for Scheme development — would help
a lot, I think.

Could you please summarize (or point to a summary of) Geiser
documentation deficiencies?

[Arne Babenhauserheide]

Post by Thien-Thi Nguyen
() Sun, 05 Mar 2017 01:23:59 +0100
Just having a geiser setup for Emacs properly documented — or
maybe an Emacs customized for Scheme development — would help
a lot, I think.
Could you please summarize (or point to a summary of) Geiser
documentation deficiencies?

I can’t just say "give me the canonical Guile development environment"
and have that work without any additional setup needed from me. This is
not about the documentation of Geiser itself but rather about the
documentation of "how to make Emacs the best tool for hacking Scheme".

The following contains user experience questions of coming to
geiser. These will sound unfair, because they assume a user who did not
read any documentation and just wants to work on a Scheme file given to
him/her by a co-worker, likely with a sentence like "here’s the file,
you’ll need to adapt it for your usecase".

This is how I come to geiser:


First I use M-x package-list-packages to find and install geiser (Why is
geiser not shipped with Emacs (like org-mode)?).

Then I open a Scheme file, then I type M-x geiser followed by guile (why
doesn’t geiser start by default? Why do I have to know that the best way
to write scheme is called geiser?).

Finally I add M-x geiser-mode in the file (why does this not happen
automatically? — or: why is there no information in the REPL that I can
do so?).


Now I have a Geiser repl and a function tool tip in the echo area.

Is this the canonical setup people use? What else is there I do not get
right away? From the IDEs co-workers use I know display of local
variables (and their values). And warnings and errors shown in the
fringe (I get that in Python with flycheck mode). Is this also provided
by geiser (maybe harnessing some other tool)?


Best wishes,
Arne

[Erik Edrosa]

Post by Alejandro Sanchez
If I may add my two cents as a Scheme layman: the question is not so much about making Guile more popular, but about making Scheme itself more popular.
One big reason for Python’s popularity is something I haven’t seen mentioned in this thread so far: if you know pseudocode you basically know Python already. Of course this is hyperbolic, there are a lot of finer details to Python, but the superficial simplicity of Python makes you think you already know the language and that you can get started right away. By the time you encounter the more arcane aspects of Python you have already invested enough time into it that you will put up with learning something new.
Scheme on the other hand is weird; weird in a good way, but still weird. For me the reason for picking up Scheme was working my way through SICP, but even before that I had heard of the Lisp family. Every time I tried to find out what the fuss was about Lisp all I could find was nebulous concepts like “it’s a programmable programming language” without explaining what that even meant and why I would ever want to use it.
Of course once I got over the weird parentheses and I understood why they are actually a beneficial aspect I liked it. Yet, even after getting into Scheme there are problems. The r5rs standard is pretty limited, and so every implementation does its own thing. There was not even a standard module system until r6rs. Unfortunately r6rs never got fully adopted by any of the major implementations, so we are back to where we were with r5rs more or less. r7rs-small looks like it could finally fill in the most glaring gaps, and r7rs-large is still nowhere done.
All this is very frustrating, and getting into a new language is a long-term commitment to learn something new. When faced with Python, Ruby, JavaScript, Lua and Scheme, it’s easy to go with the flow and just use what everyone else is using.
I think someone here mentioned the lack of a proper Scheme IDE, other than Dr. Racket for Racket. I don’t use IDEs anymore, but I can see how that can be a problem for other people who only want to do intermediate scripting rather than write entire applications. Writing a full IDE would be quite a lot of work, so perhaps integrating with existing IDEs would be better. Think something like Geiser for programs other than Emacs. There is the “language server protocol" by Microsoft, the idea is to have a standardised protocol that can be supported by a language server and an editor application.
https://github.com/Microsoft/language-server-protocol
I know it’s Microsoft, but this is just a specification. With this protocol one would write a Scheme server that receives instructions form the editor via RPC and sends the result back to the server via RPC as well. This would allow using the same Scheme server regardless of the editor (Emacs, Vim, Neovim, Eclipse, …). I had wanted to write something like this for Neovim myself, but I don’t know enough of Scheme yet.
I hope these ramblings of mine offer something useful from an outsider perspective to you guys.

I think one issue when you want to compare languages like Python to
Scheme is that these languages are dominated by a single implementation.
Python, Ruby, Go, Rust, Clojure, and many others have a single dominate
implementation. The compatibility of libraries isn't something devs
usually have to worry about (except for breaking changes in the language
like Python 2 to Python 3). I agree with how r5rs standard is pretty
limited, languages with multiple implementations tend to have pretty
large standards so that these implementations are somewhat compatible. I
am liking the direction of r7rs and I hope it will bring scheme
implementations closer together. Probably the other option for scheme
would be for everyone to get behind a single implementation like Chez
Scheme.

Of course that isn't the only thing needed to make Scheme or Guile more
popular. I believe a programming language needs a somewhat unique and
powerful library or framework, an awesome application to show off the
power of the language, good tooling, and some luck.

The language server protocol is pretty cool and I would like to see a
lisp implement it. An IDE for a lisp obviously needs to implement more
features to be useful, I believe being able to easily interact with a
REPL is one of those.

[Jan Wedekind]

According to Alan Kay's research, the future might be a lot of domain specific mini languages. I.e. languages tomorrow are going to be like libraries today. Guile supports custom languages which makes it future proof at least to some extent.

Myself, I am using Scheme because of it's unique metaprogramming facilities (Scheme macros) and GNU Guile because it has multiple-dispatch object oriented programming (unlike Racket). Also Guild's native interface with the Boehm Garbage Collector is quite easy to use.

Post by Alejandro Sanchez

Post by Alejandro Sanchez
If I may add my two cents as a Scheme layman: the question is not so

much about making Guile more popular, but about making Scheme itself
more popular.

Post by Alejandro Sanchez
One big reason for Python’s popularity is something I haven’t seen

mentioned in this thread so far: if you know pseudocode you basically
know Python already. Of course this is hyperbolic, there are a lot of
finer details to Python, but the superficial simplicity of Python makes
you think you already know the language and that you can get started
right away. By the time you encounter the more arcane aspects of Python
you have already invested enough time into it that you will put up with
learning something new.

Post by Alejandro Sanchez
Scheme on the other hand is weird; weird in a good way, but still

weird. For me the reason for picking up Scheme was working my way
through SICP, but even before that I had heard of the Lisp family.
Every time I tried to find out what the fuss was about Lisp all I could
find was nebulous concepts like “it’s a programmable programming
language” without explaining what that even meant and why I would ever
want to use it.

Post by Alejandro Sanchez
Of course once I got over the weird parentheses and I understood why

they are actually a beneficial aspect I liked it. Yet, even after
getting into Scheme there are problems. The r5rs standard is pretty
limited, and so every implementation does its own thing. There was not
even a standard module system until r6rs. Unfortunately r6rs never got
fully adopted by any of the major implementations, so we are back to
where we were with r5rs more or less. r7rs-small looks like it could
finally fill in the most glaring gaps, and r7rs-large is still nowhere
done.

Post by Alejandro Sanchez
All this is very frustrating, and getting into a new language is a

long-term commitment to learn something new. When faced with Python,
Ruby, JavaScript, Lua and Scheme, it’s easy to go with the flow and
just use what everyone else is using.

Post by Alejandro Sanchez
I think someone here mentioned the lack of a proper Scheme IDE, other

than Dr. Racket for Racket. I don’t use IDEs anymore, but I can see how
that can be a problem for other people who only want to do intermediate
scripting rather than write entire applications. Writing a full IDE
would be quite a lot of work, so perhaps integrating with existing IDEs
would be better. Think something like Geiser for programs other than
Emacs. There is the “language server protocol" by Microsoft, the idea
is to have a standardised protocol that can be supported by a language
server and an editor application.

Post by Alejandro Sanchez
https://github.com/Microsoft/language-server-protocol
I know it’s Microsoft, but this is just a specification. With this

protocol one would write a Scheme server that receives instructions
form the editor via RPC and sends the result back to the server via RPC
as well. This would allow using the same Scheme server regardless of
the editor (Emacs, Vim, Neovim, Eclipse, …). I had wanted to write
something like this for Neovim myself, but I don’t know enough of
Scheme yet.

Post by Alejandro Sanchez
I hope these ramblings of mine offer something useful from an

outsider perspective to you guys.
I think one issue when you want to compare languages like Python to
Scheme is that these languages are dominated by a single
implementation.
Python, Ruby, Go, Rust, Clojure, and many others have a single dominate
implementation. The compatibility of libraries isn't something devs
usually have to worry about (except for breaking changes in the
language
like Python 2 to Python 3). I agree with how r5rs standard is pretty
limited, languages with multiple implementations tend to have pretty
large standards so that these implementations are somewhat compatible. I
am liking the direction of r7rs and I hope it will bring scheme
implementations closer together. Probably the other option for scheme
would be for everyone to get behind a single implementation like Chez
Scheme.
Of course that isn't the only thing needed to make Scheme or Guile more
popular. I believe a programming language needs a somewhat unique and
powerful library or framework, an awesome application to show off the
power of the language, good tooling, and some luck.
The language server protocol is pretty cool and I would like to see a
lisp implement it. An IDE for a lisp obviously needs to implement more
features to be useful, I believe being able to easily interact with a
REPL is one of those.

[Erik Edrosa]

Post by Jan Wedekind
According to Alan Kay's research, the future might be a lot of domain
specific mini languages. I.e. languages tomorrow are going to be like
libraries today. Guile supports custom languages which makes it future
proof at least to some extent.

I think I can see how domain specific mini languages might be the
future. You can look at what has happened in web development to see this
explosion of JavaScript transpilers, hoping to either extend or provide
new ways to design web applications.

Post by Jan Wedekind
Myself, I am using Scheme because of it's unique metaprogramming
facilities (Scheme macros) and GNU Guile because it has
multiple-dispatch object oriented programming (unlike Racket). Also
Guild's native interface with the Boehm Garbage Collector is quite easy
to use.
--
Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet.

I started using GNU Guile after reading about rms, emacs, and GNU Guile
where GNU Guile would be used as a similar way to extend software. Then
I fell in love with scheme.

[Arne Babenhauserheide]

Post by Erik Edrosa
I think one issue when you want to compare languages like Python to
Scheme is that these languages are dominated by a single
implementation.

I read this argument repeatedly, but it does not apply to C and C++, two
of the most widely used languages. Therefore that argument might just
interpret too much into a coincidence — or rather: into a situation
which happens in the beginning of language development.

Even Java nowadays has several different implementations.

Post by Erik Edrosa
Of course that isn't the only thing needed to make Scheme or Guile more
popular. I believe a programming language needs a somewhat unique and
powerful library or framework

From what I know, that’s what ruby had: many people wanted to use rails
and after they already knew ruby well, they also used it for other
tasks.

For Python I’m not sure.

Java had that big company behind it, which marketed it as the best
choice for companies.

Post by Erik Edrosa
an awesome application to show off the
power of the language, good tooling, and some luck.

And zealots, never forget zealots. I’d dare to say that people, who
believe that the language or tool is the best thing in existence and
give lots of talks about it, are a far too easily underestimated factor
in the success of any tool.

Best wishes,
Arne

[Nala Ginrut]

Folks, I have to claim that life is not a popularity contest. That's why I
always dislike to compare to Python on popularity topic. Of course, it's
good to make Guile more successful, but please don't forget the popularity
is the result, and the productive and convenient (both in technics and
community management) is the reason.
For me, I have already started projects to solve part of productive issues.
And I think a more usable Guildhall is very important on the other hand.
I'll appreciate if someone could put effort on it. :-)

Best regards.

[Alejandro Sanchez]

Thank you for you responses, I will try to respond to them all in one email.

About Emacs + Geiser as the default development environment

Emacs is a power tool. Giving someone Emacs to write a simple script is like handing someone a jackhammer to nail plywood together. OK, maybe I’m stretching the analogy a bit, but when I think IDE I imagine something that handholds people, for the better or worse. Sure, you could offer some sort of pre-configured Emacs setup, but what’s the point of that?

This is why I had brought up the Language Server protocol, integrating Scheme with an existing IDE allows people to jump right in with whatever tool they are familiar with. It should also be less work to write a server or modify Geiser to follow the Language Server protocol than to maintain a separate distribution of Emacs.

About the lack of a standard implementation

As Arne Babenhauserheide pointed out, there is no standard implementation of C or C++ either, and yet those languages manage just fine. The difference is in *what* is undefined. Undefined in C/C++ usually means “do not rely on this, demons might fly out of your nose” and you are not supposed to touch it unless you really know what your compiler will do. Most of the undefined parts of C/C++ can be avoided in a portable way.

Undefined in Scheme before r6rs basically meant “we couldn’t all agree on one thing, so whatever”. So you have to pick one implementation and roll with it. My understanding is that’s how PLT-Scheme turned into Racket and became its own thing. This makes my code completely dependent on the implementation’s lifespan, and that’s a commitment that’s hard to sell.

From what I understand, with r6rs and r7rs the voting has been changed from unanimous to majory-vote, so this should become less of an issue over time.

About the syntax

I know about wisp, but that looks like an awkward cross-breed between Python and Lisp. Lisp syntax is simply something one needs to get over. For me it really clicked when I understood that Lisp syntax is not really a syntax, it’s the AST (sort of). Wisp reminds me of those people who use the arrow keys Vim and never leave insert-mode, in the end they just have a convoluted Nano. Maybe Wisp will help people, I don’t know, but it didn’t do anything for me.

Closing remark

The question for someone shopping around for languages is: what is the selling point for Guile (or Scheme)? Being a programmable programming language sounds cool, but what does it let me do? Nothing on Guile's website tells me “look at how easy this is to solve in Guile, you cannot do this in other languages”. What would that be? I don’t know, I’m just a layman. The topic of the thread is “how to make Guile more successful”, and I most of those are technical points, so I’m afraid that discussing popularity is all I can contribute.

[Arne Babenhauserheide]

Post by Alejandro Sanchez
Thank you for you responses, I will try to respond to them all in one email.

About Emacs + Geiser as the default development environment

Emacs is a power tool. Giving someone Emacs to write a simple script is like handing someone a jackhammer to nail plywood together. OK, maybe I’m stretching the analogy a bit, but when I think IDE I imagine something that handholds people, for the better or worse. Sure, you could offer some sort of pre-configured Emacs setup, but what’s the point of that?

The point of that is that every developer starts by doing small
things. Therefore the small startup things should work well.

I know many people at work who’ve been using Emacs for years but mostly
limit their use to what it provides by default.

What works well from the start is where most people will stay. And
that’s not unreasonable: If it doesn’t work now, how are the chances
that it will be better than something which works well right away?

Post by Alejandro Sanchez

About the syntax

I know about wisp, but that looks like an awkward cross-breed between Python and Lisp. Lisp syntax is simply something one needs to get over. For me it really clicked when I understood that Lisp syntax is not really a syntax, it’s the AST (sort of). Wisp reminds me of those people who use the arrow keys Vim and never leave insert-mode, in the end they just have a convoluted Nano. Maybe Wisp will help people, I don’t know, but it didn’t do anything for me.

The difference to vim in insert mode is that wisp provides all features
of scheme. The opening and closing parens just look different :)

(or rather: they are merged into the existing indentation, so this
reduces duplication of information)

Post by Alejandro Sanchez

Closing remark

The question for someone shopping around for languages is: what is the selling point for Guile (or Scheme)? Being a programmable programming language sounds cool, but what does it let me do? Nothing on Guile's website tells me “look at how easy this is to solve in Guile, you cannot do this in other languages”. What would that be? I don’t know, I’m just a layman. The topic of the thread is “how to make Guile more successful”, and I most of those are technical points, so I’m afraid that discussing popularity is all I can contribute.

You can for example build wisp ;)

And "natural script writing" would have been much harder (and much more
mechanical work) in other languages. In most languages it would have
required parsing the text by hand and explicitly extracting code and
handing it to some function to execute it.

Best wishes,
Arne

[Matt Wette]

Scheme is not for everyone. But getting more people to use it is going to mean people seeing what it offers over other languages. I do a fair amount of Python programming. Python code is fairly portable: my Mac comes with it installed and so I can send code to anyone with a Mac and it will probably run. Python also has a *huge* set of libraries. Writing extension code (in C) for Python sucks, though. And Python `lambda’ is weak.

And Python does not work well on tasks that Scheme excels at: tree structures, little-languages, list processing, etc. If you want more people to use Scheme/Guile you need to write applications that demonstrate it’s strengths over other languages. I know of nothing that beats Scheme macros for building your own language extensions: that capability is so awesome. And Scheme looks pretty promising to me for handling XML-based tree structures (e.g., SXML with foldts and friends).

Matt

[Erik Edrosa]

Post by Arne Babenhauserheide

Post by Erik Edrosa
I think one issue when you want to compare languages like Python to
Scheme is that these languages are dominated by a single
implementation.

I read this argument repeatedly, but it does not apply to C and C++, two
of the most widely used languages. Therefore that argument might just
interpret too much into a coincidence — or rather: into a situation
which happens in the beginning of language development.
Even Java nowadays has several different implementations.

C, C++, and Java have very large specifications and standards which are
meant to keep the various implementations as compatible as possible.
Although implementations do provide their own extensions and there is
issues between OSes. I agree this argument isn't too strong, but I see
it brought up once in awhile. One issue is it does fracture the
community a bit. If implementations have greater compatibility you see
developers use them in interesting ways. For example in Common Lisp,
I've heard some developers will use CCL in development because it
compiles faster and produces easier to debug code and use SBCL for
building releases because it produces faster code.

Post by Arne Babenhauserheide

Post by Erik Edrosa
an awesome application to show off the
power of the language, good tooling, and some luck.

And zealots, never forget zealots. I’d dare to say that people, who
believe that the language or tool is the best thing in existence and
give lots of talks about it, are a far too easily underestimated factor
in the success of any tool.
Best wishes,
Arne

I think we have many of those. ;)

- Erik

[David Kastrup]

Post by Alejandro Sanchez
Scheme on the other hand is weird; weird in a good way, but still
weird. For me the reason for picking up Scheme was working my way
through SICP, but even before that I had heard of the Lisp
family. Every time I tried to find out what the fuss was about Lisp
all I could find was nebulous concepts like “it’s a programmable
programming language” without explaining what that even meant and why
I would ever want to use it.
Of course once I got over the weird parentheses and I understood why
they are actually a beneficial aspect I liked it.

Everything falls into place once you understand that Scheme does not
offer a programming language as user interface. Instead, you directly
write down the parse trees in Scheme's compact list representation.

That's not exactly readable but makes structure-preserving code
manipulation a breeze.

[Vítor De Araújo]

Hi,

I'm relatively new to Guile and new to this list, but I saw this
thread in the archives and I'd like to make some comments.

First, I think it would be nice to have a Guile wiki, in the likes of
cliki.net. A publicly-editable reference place where people can share
knowledge about Guile and Guile libraries and projects. Sure, having a
centralized package manager/repository would be nice, but just having
a common place where people can look up what libraries exist and where
they can be downloaded would be nice. (I don't know how easy would it
be to set up a wiki at the Guile website.)

On the topic of package management, I think that, more important than
a central repository, is a standard format in which to distribute
packages for easy installation. I've never used guildhall, and I guess
it defines something like this, but I think just having an
installation command where you can pass the URL of a package to
download and install (say, guild install http://some.url/somepackage.zip,
or even a git URL) would already go a long way towards making it
easier for people to use third-party libraries in Guile.

(Or maybe package _names_ could be URLs pointing to an index in a
well-defined format, which tells how to download each version of a
package; then we could have even dependency management without a
central repository. I'm pipe-dreaming here, though.)

Best regards,

Vítor De Araújo

[Arne Babenhauserheide]

Post by Vítor De Araújo
(Or maybe package _names_ could be URLs pointing to an index in a
well-defined format, which tells how to download each version of a
package; then we could have even dependency management without a
central repository. I'm pipe-dreaming here, though.)

You’re not just dreaming:

https://github.com/plande/compan/blob/master/compan.scm

;; clone the repository and load (lib) module from the top level:
(load-modules ("https://bitbucket.org/panicz/dogma" (lib)))

;; as above, but load modules from "libraries" directory:
(load-modules (("https://github.com/panicz/pamphlet" "libraries") (pamphlet)))

;; one can also fix on a particular tag/branch/commit:
(load-modules (("https://bitbucket.org/panicz/dogma" "." "3884445191c2") (lib)))

We’d just have to decide to use this.

Best wishes,
Arne

[Panicz Maciej Godek]

Post by Arne Babenhauserheide

Post by Vítor De Araújo
(Or maybe package _names_ could be URLs pointing to an index in a
well-defined format, which tells how to download each version of a
package; then we could have even dependency management without a
central repository. I'm pipe-dreaming here, though.)

https://github.com/plande/compan/blob/master/compan.scm
(load-modules ("https://bitbucket.org/panicz/dogma" (lib)))
(load-modules (("https://github.com/panicz/pamphlet" "libraries") (pamphlet)))
(load-modules (("https://bitbucket.org/panicz/dogma" "."
"3884445191c2") (lib)))
We’d just have to decide to use this.
I think that the situation isn't quite as good as one could think. The

code works fine from
the interpreter, but if fails to work for compiled code, and I think that
we'd need to come up
with some solution of delaing module imports in compiled code until they're
available.

(Note also that the source code of Compan is extremely straightforward, as
it boils down
to a few calls of the "system" function which in turns calls some "hg"
commands)

So while I think that the author's idea was very nice, it clearly needs
some elaboration :)

[v***@sapo.pt]

Post by Panicz Maciej Godek

Post by Arne Babenhauserheide

Post by Vítor De Araújo
(Or maybe package _names_ could be URLs pointing to an index in a
well-defined format, which tells how to download each version of a
package; then we could have even dependency management without a
central repository. I'm pipe-dreaming here, though.)

https://github.com/plande/compan/blob/master/compan.scm
(load-modules ("https://bitbucket.org/panicz/dogma" (lib)))
(load-modules (("https://github.com/panicz/pamphlet" "libraries") (pamphlet)))
(load-modules (("https://bitbucket.org/panicz/dogma" "."
"3884445191c2") (lib)))
We’d just have to decide to use this.

That's very cool :) And also supports my point that there should be something
like a wiki for publishing information about Guile packages. It'd be hard
for me to find this on my own.

Post by Panicz Maciej Godek

Post by Arne Babenhauserheide
I think that the situation isn't quite as good as one could think. The

code works fine from
the interpreter, but if fails to work for compiled code, and I think that
we'd need to come up
with some solution of delaing module imports in compiled code until they're
available.
(Note also that the source code of Compan is extremely straightforward, as
it boils down
to a few calls of the "system" function which in turns calls some "hg"
commands)
So while I think that the author's idea was very nice, it clearly needs
some elaboration :)

Yeah, for one I don't think the URLs should go directly into the code, but
rather in a package metadata file. The reason is I don't think we should
hard-code the package manager in the code. People should be able to use
the code without having the package manager, if they already have the
dependencies installed by some other means. But the basic idea is cool. :)

--
Vítor De Araújo

[Panicz Maciej Godek]

Post by v***@sapo.pt
That's very cool :) And also supports my point that there should be something
like a wiki for publishing information about Guile packages. It'd be hard
for me to find this on my own.
[...]
Yeah, for one I don't think the URLs should go directly into the code, but
rather in a package metadata file. The reason is I don't think we should
hard-code the package manager in the code. People should be able to use
the code without having the package manager, if they already have the
dependencies installed by some other means. But the basic idea is cool. :)

There is a reason why I like the idea of having URLs directly in the code,
namely that it makes you independent from some big institutions/particular
maintainers. Of course it is risky if your code refers to repositories that
you do not control, but you can always make your private forks.

Essentially the power of this idea is that it suddenly turns the whole web
into a Guile Incomprehensive Archive Network (clearly, a word beginning
with a letter T is missing to get a nice acronym).

Of course, it should be very easy to build a translation layer that
would map package names to addresses, but it requires a middleman
which always makes the costs higher (but perhaps having a proxy that
would make the necessary forks automatically would be a nice idea.
Plus if it would be able to check whether the changes between
some commits break compatibility then, well, that would really be
something. Which brings to mind the recent talk by Rich Hickey:

)

I do agree that something wiki-like would be a lovely thing.

[Thien-Thi Nguyen]

() Panicz Maciej Godek <***@gmail.com>
() Fri, 10 Mar 2017 16:08:23 +0100

Essentially the power of this idea is that it suddenly turns
the whole web into a Guile Incomprehensive Archive Network
(clearly, a word beginning with a letter T is missing to get
a nice acronym).

Cool. Some suggestions:

tactic
tap
tapestry
tatter
tattoo
taunt
teacup
teardrop
tech
tee
temperament
tendency
tenet
thanks
thicket
thing
thrall
thwack
tine
tofu
toke
torpor
torrent
toy
tracks
traipse
travail
trawl
tree
tremor
trick
trinkets
triumph
troth
truffle
tsunami
tub
tufts
twig
twists
typo

[Nala Ginrut]

Well, I like tofu :-)

Post by Thien-Thi Nguyen
() Fri, 10 Mar 2017 16:08:23 +0100
Essentially the power of this idea is that it suddenly turns
the whole web into a Guile Incomprehensive Archive Network
(clearly, a word beginning with a letter T is missing to get
a nice acronym).
tactic
tap
tapestry
tatter
tattoo
taunt
teacup
teardrop
tech
tee
temperament
tendency
tenet
thanks
thicket
thing
thrall
thwack
tine
tofu
toke
torpor
torrent
toy
tracks
traipse
travail
trawl
tree
tremor
trick
trinkets
triumph
troth
truffle
tsunami
tub
tufts
twig
twists
typo
--
Thien-Thi Nguyen -----------------------------------------------
(defun responsep (query)
(pcase (context query)
(`(technical ,ml) (correctp ml))
...)) 748E A0E8 1CB8 A748 9BFA
--------------------------------------- 6CE4 6703 2224 4C80 7502

[Panicz Maciej Godek]

My mind has actually been oscillating between "trick", "toy", "thing" and
"torrent".
The last one would go particularly nice with the concept of "GitTorrent"

http://blog.printf.net/articles/2015/05/29/announcing-gittorrent-a-decentralized-github/

Post by Nala Ginrut
Well, I like tofu :-)

Post by Thien-Thi Nguyen
() Fri, 10 Mar 2017 16:08:23 +0100
Essentially the power of this idea is that it suddenly turns
the whole web into a Guile Incomprehensive Archive Network
(clearly, a word beginning with a letter T is missing to get
a nice acronym).
tactic
tap
tapestry
tatter
tattoo
taunt
teacup
teardrop
tech
tee
temperament
tendency
tenet
thanks
thicket
thing
thrall
thwack
tine
tofu
toke
torpor
torrent
toy
tracks
traipse
travail
trawl
tree
tremor
trick
trinkets
triumph
troth
truffle
tsunami
tub
tufts
twig
twists
typo
--
Thien-Thi Nguyen -----------------------------------------------
(defun responsep (query)
(pcase (context query)
(`(technical ,ml) (correctp ml))
...)) 748E A0E8 1CB8 A748 9BFA
--------------------------------------- 6CE4 6703 2224 4C80 7502

[Amirouche]

Héllo!

Post by Vítor De Araújo
Hi,
I'm relatively new to Guile and new to this list, but I saw this
thread in the archives and I'd like to make some comments.

Welcome!

Post by Vítor De Araújo
First, I think it would be nice to have a Guile wiki, in the likes of
cliki.net. A publicly-editable reference place where people can share
knowledge about Guile and Guile libraries and projects.

I agree.

Post by Vítor De Araújo
Sure, having a
centralized package manager/repository would be nice, but just having
a common place where people can look up what libraries exist and where
they can be downloaded would be nice. (I don't know how easy would it
be to set up a wiki at the Guile website.)

The current most comprehensive list of GNU Guile package is
available at http://sph.mn/content/3e73

There is also the github org:

https://github.com/guildhall

Post by Vítor De Araújo
On the topic of package management, I think that, more important than
a central repository, is a standard format in which to distribute
packages for easy installation.

I agree. The standard could be to have a tarball named using
semantic versionning, from which one can extract the PACKAGE-NAME.
Inside the tarball, the package manager will expect a PACKAGE-NAME.scm
or a PACKAGE-NAME directory which can be installed into ~/.local/lib/guile

WDYT?

Post by Vítor De Araújo
I've never used guildhall, and I guess
it defines something like this,

Probably but it's not documented.

Post by Vítor De Araújo
but I think just having an
installation command where you can pass the URL of a package to
download and install (say, guild install http://some.url/somepackage.zip,
or even a git URL)

This will still require the user to set GUILE_LOAD_PATH. Maybe we can
add ~/.local/lib/guile/ to the default GUILE_LOAD_PATH to make things
easier.

A simple guild install https://url/to/patch can be done in mater of hours
(if not minutes).

WDYT?

Post by Vítor De Araújo
would already go a long way towards making it
easier for people to use third-party libraries in Guile.

I agree.

Post by Vítor De Araújo
(Or maybe package _names_ could be URLs pointing to an index in a
well-defined format, which tells how to download each version of a
package; then we could have even dependency management without a
central repository. I'm pipe-dreaming here, though.)

This seems fun, but at the same time odd.

FWIW, I've started a new effort to build a package manager for GNU
Guile. You can find a demo at http://guildhall.hypermove.net/

It started as a guildhall web frontend but I noticed that guildhall:

1) doesn't work with guile 2.2 and I can not make it work
2) The solver is too complicated, even if it's based on aptitude algorithm
it's still complicated.
3) use a lot of other scheme libraries [0] (like foof-loop) which
doesn't help noobs like me to dive into the code. Maybe those
libraries are *very* neat but why not include them in Guile proper
then and make them default.

[0] https://github.com/ijp/guildhall/tree/master/guildhall/ext

For all this reasons I forked the effort.

I address each previous point as follow:

1) I only use guile 2.2, right now guile 2.1 is not supported. I think it's
bad this should be addressed at some point. AFAIR the issue is in
guile-wiredtiger (or it's an issue with setlocale (anyway this must be
addressed)))

2) I use a logic programming library called minikanren to solve the
dependency problem. This is the first real problem I can use minikanren
to solve a problem that is craft. This is the logic programming
library in scheme. While it's not very advanced compared to core.logic,
the version I use called microkaren has a straight forward
implementation that can
hold all at once in a single head. Logic programming, like probabilistic
programming (cf. ) are two research areas in programming languages
that could improve the way we craft algorithms today. The logic approach
being more useful to me (but someone doing machine learning stuff,
might find the probabilistic scheme very useful. Ask me for a link!).
OpenCog
does have a logic language embedded in it's language in their hyper
graph database
(AFAIU it works using backward / forward inference but implements on top
of it higher logic constructs, like abduction, anaphora resolution...). It's
something they call PLN as Probabilistic Logic Network. AFAIK it's some kind
of probabilistic Datalog (which is one of the API Datomic support).
There is
also probabilistic minikanren (albeit not useable). So, to sum up (because
I can (because I am not a robot)):

programming language research + guile + logic = minikanren.

Mind the fact that there is also a guile-log. But I still don't get what
guile-log does (hints: http://c-lambda.se/) whereas I understand how
microkanren does its stuff (I still need to benchmark it).

3) I don't use foof-loop but I bring my own dependencies. I use my
own database library that is based on wiredtiger. I've been working
on this database library for 3 years, it's well documented and various
example codes. That said, AFAIK, I am the only one to use it. I've built
a clone of Datomic (with the persistence part, patch welcome) which
use a query engine similar to Datomic I guess based on minikanren.

It's performance on a middle end laptop run Guile 2.2 are the followings:

- 1500 document reads per seconds
- 1000 to 500 document write per second

Documents are scheme assoc, and those are inserted 1000 per 1000 and
read 1000 per 1000 until it reach 50Go of data. At that point writes
take 1/500
seconds or 0.002 seconds.

The biggest dataset I *loaded* into that database is wikidata which is
50Go, I
don't remember how much time it takes to load it.

I had issues with wiredtiger using the Python bindings but only during reads
over a gigantic dataset of 60Giga. I say gigantic for a blog not for Google
scale. Also upstream can solve issues if we can have a way to reproduce the
issues (which I plan to do once guile 2.2 is out (which means I will
redo the
benchmarks against wikidata and read/write)). I understand the problem
that wiredtiger being part of MongoDB is problematic as MongoDB might not
care much for the same problem as ours. They will always be interested by
bigger free software database, tho. As it make free publicity of how
successful
is their software.

Also there is much documentation about this library. I created several
Guile projects
using it (albeit not big) they document several layers of the library
and one
Guile user reported using it for doing human/social science research.

Search for guile wiredtiger in you favorite duckduckgo search engine.

Some people claim that PostgreSQL has all the required feature that
someone wants
to store data and that this mongodb/wiredtiger is a bad. I recognize
that wiredtiger
can be poor man's database right now. I don't have the required
expertise to verify
whether it's good enough for your usecase. PostgreSQL is used in
"production" in
all the world using multiple workload and stuff. PostgreSQL is good.

1) There is no dynamic ffi bindings for PostgreSQL yet, otherwise said you
can't use pgsql from Guile RIGHT NOW.
2) wiredtiger is GOOD enough, benchmark it before saying it's not good
ENOUGH.
3) it's not SQL, it's Schemey! It's inspired from datomic which the goto
database
for clojure with multiple implementation in the browser. This is
killer feature
of the clojure ecosystem that is closed source.
4) I find it more funny to fiddle with logic programming that set pseudo
theory of SQL.
5) The first thing I do when pgsql bindings will be out, is to port
feature space to PostgreSQL
and compare both pgsql and wiredtiger.
6) wiredtiger will always be simpler to use that Postgresql it's like
comparing sqlite
and Datomic.

What happens next? You choose to hack on this project and find something
interesting in this
TODO list:

- fix guile-wiredtiger to work on guile 2.0

- make it possible in guile-wiredtiger to stream the results of a look
up using traversi using multiple cursors

- implement disequality in microkanren (for implementing the package
dependency resolver
and for fun because logic is awesome).

- implement a pastebin service using GNU Guile, yes because I think this
a good bad reason to start
another simple project that is not a static blog generator.

- Port the wiki of http://culturia.one to use feature space library.

- implement or find a scheme library that does the diff two lists (to
finish wiki implementation
and for the package dependency resolver).

- Help Matt Wette to complete his ffi helper
http://lists.gnu.org/archive/html/guile-user/2017-03/msg00085.html
(because we will need it for guix).

- add guile-fibers as a submodule and use it

- there is also a simple boolean keyword search engine that must be
ported to feature space

- find the code that implements trigram transderivational search and put
it in the repository. Bridging
the gap with tsearch2.

- index the wiki

- index the packages

- index the pastebin

- index the web

And add gnunet to the mix.

There is not a lot of get-together pure guile project out there. There
is guix. I hope this new project
can be the occasion for new guilers to submit patches or ideas.

The code is currently hosted at
https://framagit.org/a-guile-mind/culturia.one/tree/guildhall/

The codename is "primary".

Also I'd like to point out, that I don't need primary. I do it for the
community only. The
road is fascinating, tho. I could make mistakes so please input your
feedback if you want
to use a tool like that.

You can:

guix package -i wiredtiger
git clone https://framagit.org/a-guile-mind/culturia.one.git
git checkout guildhall
emacs culturia.one/src/webui.scm

or

cd culturia.one/src/ && guile -L . webui.scm

[Amirouche]

There is compan, there is guildhall, there is guix. They shall be primary.



-------- Message transféré --------
Sujet : Re: How to make GNU Guile more successful
Date : Fri, 10 Mar 2017 21:17:37 +0100
De : Amirouche <***@hypermove.net>
Pour : guile-***@gnu.org



Héllo!

Post by Vítor De Araújo
Hi,
I'm relatively new to Guile and new to this list, but I saw this
thread in the archives and I'd like to make some comments.

Welcome!

Post by Vítor De Araújo
First, I think it would be nice to have a Guile wiki, in the likes of
cliki.net. A publicly-editable reference place where people can share
knowledge about Guile and Guile libraries and projects.

I agree.

Post by Vítor De Araújo
Sure, having a
centralized package manager/repository would be nice, but just having
a common place where people can look up what libraries exist and where
they can be downloaded would be nice. (I don't know how easy would it
be to set up a wiki at the Guile website.)

The current most comprehensive list of GNU Guile package is
available at http://sph.mn/content/3e73

There is also the github org:

https://github.com/guildhall

Post by Vítor De Araújo
On the topic of package management, I think that, more important than
a central repository, is a standard format in which to distribute
packages for easy installation.

I agree. The standard could be to have a tarball named using
semantic versionning, from which one can extract the PACKAGE-NAME.
Inside the tarball, the package manager will expect a PACKAGE-NAME.scm
or a PACKAGE-NAME directory which can be installed into ~/.local/lib/guile

WDYT?

Post by Vítor De Araújo
I've never used guildhall, and I guess
it defines something like this,

Probably but it's not documented.

Post by Vítor De Araújo
but I think just having an
installation command where you can pass the URL of a package to
download and install (say, guild install http://some.url/somepackage.zip,
or even a git URL)

This will still require the user to set GUILE_LOAD_PATH. Maybe we can
add ~/.local/lib/guile/ to the default GUILE_LOAD_PATH to make things
easier.

A simple guild install https://url/to/patch can be done in mater of hours
(if not minutes).

WDYT?

Post by Vítor De Araújo
would already go a long way towards making it
easier for people to use third-party libraries in Guile.

I agree.

Post by Vítor De Araújo
(Or maybe package _names_ could be URLs pointing to an index in a
well-defined format, which tells how to download each version of a
package; then we could have even dependency management without a
central repository. I'm pipe-dreaming here, though.)

This seems fun, but at the same time odd.

FWIW, I've started a new effort to build a package manager for GNU
Guile. You can find a demo at http://guildhall.hypermove.net/

It started as a guildhall web frontend but I noticed that guildhall:

1) doesn't work with guile 2.2 and I can not make it work
2) The solver is too complicated, even if it's based on aptitude algorithm
it's still complicated.
3) use a lot of other scheme libraries [0] (like foof-loop) which
doesn't help noobs like me to dive into the code. Maybe those
libraries are *very* neat but why not include them in Guile proper
then and make them default.

[0] https://github.com/ijp/guildhall/tree/master/guildhall/ext

For all this reasons I forked the effort.

I address each previous point as follow:

1) I only use guile 2.2, right now guile 2.1 is not supported. I think it's
bad this should be addressed at some point. AFAIR the issue is in
guile-wiredtiger (or it's an issue with setlocale (anyway this must be
addressed)))

2) I use a logic programming library called minikanren to solve the
dependency problem. This is the first real problem I can use minikanren
to solve a problem that is craft. This is the logic programming
library in scheme. While it's not very advanced compared to core.logic,
the version I use called microkaren has a straight forward
implementation that can
hold all at once in a single head. Logic programming, like probabilistic
programming (cf. ) are two research areas in programming languages
that could improve the way we craft algorithms today. The logic approach
being more useful to me (but someone doing machine learning stuff,
might find the probabilistic scheme very useful. Ask me for a link!).
OpenCog
does have a logic language embedded in it's language in their hyper
graph database
(AFAIU it works using backward / forward inference but implements on top
of it higher logic constructs, like abduction, anaphora resolution...). It's
something they call PLN as Probabilistic Logic Network. AFAIK it's some kind
of probabilistic Datalog (which is one of the API Datomic support).
There is
also probabilistic minikanren (albeit not useable). So, to sum up (because
I can (because I am not a robot)):

programming language research + guile + logic = minikanren.

Mind the fact that there is also a guile-log. But I still don't get what
guile-log does (hints: http://c-lambda.se/) whereas I understand how
microkanren does its stuff (I still need to benchmark it).

3) I don't use foof-loop but I bring my own dependencies. I use my
own database library that is based on wiredtiger. I've been working
on this database library for 3 years, it's well documented and various
example codes. That said, AFAIK, I am the only one to use it. I've built
a clone of Datomic (with the persistence part, patch welcome) which
use a query engine similar to Datomic I guess based on minikanren.

It's performance on a middle end laptop run Guile 2.2 are the followings:

- 1500 document reads per seconds
- 1000 to 500 document write per second

Documents are scheme assoc, and those are inserted 1000 per 1000 and
read 1000 per 1000 until it reach 50Go of data. At that point writes
take 1/500
seconds or 0.002 seconds.

The biggest dataset I *loaded* into that database is wikidata which is
50Go, I
don't remember how much time it takes to load it.

I had issues with wiredtiger using the Python bindings but only during reads
over a gigantic dataset of 60Giga. I say gigantic for a blog not for Google
scale. Also upstream can solve issues if we can have a way to reproduce the
issues (which I plan to do once guile 2.2 is out (which means I will
redo the
benchmarks against wikidata and read/write)). I understand the problem
that wiredtiger being part of MongoDB is problematic as MongoDB might not
care much for the same problem as ours. They will always be interested by
bigger free software database, tho. As it make free publicity of how
successful
is their software.

Also there is much documentation about this library. I created several
Guile projects
using it (albeit not big) they document several layers of the library
and one
Guile user reported using it for doing human/social science research.

Search for guile wiredtiger in you favorite duckduckgo search engine.

Some people claim that PostgreSQL has all the required feature that
someone wants
to store data and that this mongodb/wiredtiger is a bad. I recognize
that wiredtiger
can be poor man's database right now. I don't have the required
expertise to verify
whether it's good enough for your usecase. PostgreSQL is used in
"production" in
all the world using multiple workload and stuff. PostgreSQL is good.

1) There is no dynamic ffi bindings for PostgreSQL yet, otherwise said you
can't use pgsql from Guile RIGHT NOW.
2) wiredtiger is GOOD enough, benchmark it before saying it's not good
ENOUGH.
3) it's not SQL, it's Schemey! It's inspired from datomic which the goto
database
for clojure with multiple implementation in the browser. This is
killer feature
of the clojure ecosystem that is closed source.
4) I find it more funny to fiddle with logic programming that set pseudo
theory of SQL.
5) The first thing I do when pgsql bindings will be out, is to port
feature space to PostgreSQL
and compare both pgsql and wiredtiger.
6) wiredtiger will always be simpler to use that Postgresql it's like
comparing sqlite
and Datomic.

What happens next? You choose to hack on this project and find something
interesting in this
TODO list:

- fix guile-wiredtiger to work on guile 2.0

- make it possible in guile-wiredtiger to stream the results of a look
up using traversi using multiple cursors

- implement disequality in microkanren (for implementing the package
dependency resolver
and for fun because logic is awesome).

- implement a pastebin service using GNU Guile, yes because I think this
a good bad reason to start
another simple project that is not a static blog generator.

- Port the wiki of http://culturia.one to use feature space library.

- implement or find a scheme library that does the diff two lists (to
finish wiki implementation
and for the package dependency resolver).

- Help Matt Wette to complete his ffi helper
http://lists.gnu.org/archive/html/guile-user/2017-03/msg00085.html
(because we will need it for guix).

- add guile-fibers as a submodule and use it

- there is also a simple boolean keyword search engine that must be
ported to feature space

- find the code that implements trigram transderivational search and put
it in the repository. Bridging
the gap with tsearch2.

- index the wiki

- index the packages

- index the pastebin

- index the web

And add gnunet to the mix.

There is not a lot of get-together pure guile project out there. There
is guix. I hope this new project
can be the occasion for new guilers to submit patches or ideas.

The code is currently hosted at
https://framagit.org/a-guile-mind/culturia.one/tree/guildhall/

The codename is "primary".

Also I'd like to point out, that I don't need primary. I do it for the
community only. The
road is fascinating, tho. I could make mistakes so please input your
feedback if you want
to use a tool like that.

You can:

guix package -i wiredtiger
git clone https://framagit.org/a-guile-mind/culturia.one.git
git checkout guildhall
emacs culturia.one/src/webui.scm

or

cd culturia.one/src/ && guile -L . webui.scm

[Nala Ginrut]

Hi Amiroiche!
FWIW, usually there are 3 kind s of pkg-manager:

1. SPM(system pkg manager)
It's used for building a distro, like RPM, deb, Guix.

2. LPM(language pkg manager)
It's used for specific language to install optional packages. Like npm,
gem, pip, etc.
Guix could do that too, IMO.

3. APM(application pkg manager)
It's used for specific framework to install packages in local application
directory. This is important for deployment.
Ruby has "bundle" for that, and Python usually use pip+virtualenv for that,
and npm is flexible enough to do that for JS too. Of course there are
alternative tools do the similar thing in each community.

For now, I'm concerning an APM for Artanis.
I've discussed with @Ludo the possibility to take advantage of Guix (as a
core lib) to implement it. If you want to write yet another pkg manager,
it's better to confirm which kind are you going to do, and don't forget to
reuse some cool features exists in Guix. :-)

Post by Amirouche
There is compan, there is guildhall, there is guix. They shall be primary.
-------- Message transféré --------
Sujet : Re: How to make GNU Guile more successful
Date : Fri, 10 Mar 2017 21:17:37 +0100
Héllo!

Post by Vítor De Araújo
Hi,
I'm relatively new to Guile and new to this list, but I saw this
thread in the archives and I'd like to make some comments.

Welcome!

Post by Vítor De Araújo
First, I think it would be nice to have a Guile wiki, in the likes of
cliki.net. A publicly-editable reference place where people can share
knowledge about Guile and Guile libraries and projects.

I agree.

Post by Vítor De Araújo
Sure, having a
centralized package manager/repository would be nice, but just having
a common place where people can look up what libraries exist and where
they can be downloaded would be nice. (I don't know how easy would it
be to set up a wiki at the Guile website.)

The current most comprehensive list of GNU Guile package is
available at http://sph.mn/content/3e73
https://github.com/guildhall

Post by Vítor De Araújo
On the topic of package management, I think that, more important than
a central repository, is a standard format in which to distribute
packages for easy installation.

I agree. The standard could be to have a tarball named using
semantic versionning, from which one can extract the PACKAGE-NAME.
Inside the tarball, the package manager will expect a PACKAGE-NAME.scm
or a PACKAGE-NAME directory which can be installed into ~/.local/lib/guile
WDYT?

Post by Vítor De Araújo
I've never used guildhall, and I guess
it defines something like this,

Probably but it's not documented.

Post by Vítor De Araújo
but I think just having an
installation command where you can pass the URL of a package to
download and install (say, guild install http://some.url/somepackage.zip

,

Post by Vítor De Araújo
or even a git URL)

This will still require the user to set GUILE_LOAD_PATH. Maybe we can
add ~/.local/lib/guile/ to the default GUILE_LOAD_PATH to make things
easier.
A simple guild install https://url/to/patch can be done in mater of hours
(if not minutes).
WDYT?

Post by Vítor De Araújo
would already go a long way towards making it
easier for people to use third-party libraries in Guile.

I agree.

Post by Vítor De Araújo
(Or maybe package _names_ could be URLs pointing to an index in a
well-defined format, which tells how to download each version of a
package; then we could have even dependency management without a
central repository. I'm pipe-dreaming here, though.)

This seems fun, but at the same time odd.
FWIW, I've started a new effort to build a package manager for GNU
Guile. You can find a demo at http://guildhall.hypermove.net/
1) doesn't work with guile 2.2 and I can not make it work
2) The solver is too complicated, even if it's based on aptitude algorithm
it's still complicated.
3) use a lot of other scheme libraries [0] (like foof-loop) which
doesn't help noobs like me to dive into the code. Maybe those
libraries are *very* neat but why not include them in Guile proper
then and make them default.
[0] https://github.com/ijp/guildhall/tree/master/guildhall/ext
For all this reasons I forked the effort.
1) I only use guile 2.2, right now guile 2.1 is not supported. I think it's
bad this should be addressed at some point. AFAIR the issue is in
guile-wiredtiger (or it's an issue with setlocale (anyway this must be
addressed)))
2) I use a logic programming library called minikanren to solve the
dependency problem. This is the first real problem I can use minikanren
to solve a problem that is craft. This is the logic programming
library in scheme. While it's not very advanced compared to core.logic,
the version I use called microkaren has a straight forward
implementation that can
hold all at once in a single head. Logic programming, like probabilistic
programming (cf. ) are two research areas in programming languages
that could improve the way we craft algorithms today. The logic approach
being more useful to me (but someone doing machine learning stuff,
might find the probabilistic scheme very useful. Ask me for a link!).
OpenCog
does have a logic language embedded in it's language in their hyper
graph database
(AFAIU it works using backward / forward inference but implements on top
of it higher logic constructs, like abduction, anaphora resolution...). It's
something they call PLN as Probabilistic Logic Network. AFAIK it's some kind
of probabilistic Datalog (which is one of the API Datomic support).
There is
also probabilistic minikanren (albeit not useable). So, to sum up (because
programming language research + guile + logic = minikanren.
Mind the fact that there is also a guile-log. But I still don't get what
guile-log does (hints: http://c-lambda.se/) whereas I understand how
microkanren does its stuff (I still need to benchmark it).
3) I don't use foof-loop but I bring my own dependencies. I use my
own database library that is based on wiredtiger. I've been working
on this database library for 3 years, it's well documented and various
example codes. That said, AFAIK, I am the only one to use it. I've built
a clone of Datomic (with the persistence part, patch welcome) which
use a query engine similar to Datomic I guess based on minikanren.
- 1500 document reads per seconds
- 1000 to 500 document write per second
Documents are scheme assoc, and those are inserted 1000 per 1000 and
read 1000 per 1000 until it reach 50Go of data. At that point writes
take 1/500
seconds or 0.002 seconds.
The biggest dataset I *loaded* into that database is wikidata which is
50Go, I
don't remember how much time it takes to load it.
I had issues with wiredtiger using the Python bindings but only during reads
over a gigantic dataset of 60Giga. I say gigantic for a blog not for Google
scale. Also upstream can solve issues if we can have a way to reproduce the
issues (which I plan to do once guile 2.2 is out (which means I will
redo the
benchmarks against wikidata and read/write)). I understand the problem
that wiredtiger being part of MongoDB is problematic as MongoDB might not
care much for the same problem as ours. They will always be interested by
bigger free software database, tho. As it make free publicity of how
successful
is their software.
Also there is much documentation about this library. I created several
Guile projects
using it (albeit not big) they document several layers of the library
and one
Guile user reported using it for doing human/social science research.
Search for guile wiredtiger in you favorite duckduckgo search engine.
Some people claim that PostgreSQL has all the required feature that
someone wants
to store data and that this mongodb/wiredtiger is a bad. I recognize
that wiredtiger
can be poor man's database right now. I don't have the required
expertise to verify
whether it's good enough for your usecase. PostgreSQL is used in
"production" in
all the world using multiple workload and stuff. PostgreSQL is good.
1) There is no dynamic ffi bindings for PostgreSQL yet, otherwise said you
can't use pgsql from Guile RIGHT NOW.
2) wiredtiger is GOOD enough, benchmark it before saying it's not good
ENOUGH.
3) it's not SQL, it's Schemey! It's inspired from datomic which the goto
database
for clojure with multiple implementation in the browser. This is
killer feature
of the clojure ecosystem that is closed source.
4) I find it more funny to fiddle with logic programming that set pseudo
theory of SQL.
5) The first thing I do when pgsql bindings will be out, is to port
feature space to PostgreSQL
and compare both pgsql and wiredtiger.
6) wiredtiger will always be simpler to use that Postgresql it's like
comparing sqlite
and Datomic.
What happens next? You choose to hack on this project and find something
interesting in this
- fix guile-wiredtiger to work on guile 2.0
- make it possible in guile-wiredtiger to stream the results of a look
up using traversi using multiple cursors
- implement disequality in microkanren (for implementing the package
dependency resolver
and for fun because logic is awesome).
- implement a pastebin service using GNU Guile, yes because I think this
a good bad reason to start
another simple project that is not a static blog generator.
- Port the wiki of http://culturia.one to use feature space library.
- implement or find a scheme library that does the diff two lists (to
finish wiki implementation
and for the package dependency resolver).
- Help Matt Wette to complete his ffi helper
http://lists.gnu.org/archive/html/guile-user/2017-03/msg00085.html
(because we will need it for guix).
- add guile-fibers as a submodule and use it
- there is also a simple boolean keyword search engine that must be
ported to feature space
- find the code that implements trigram transderivational search and put
it in the repository. Bridging
the gap with tsearch2.
- index the wiki
- index the packages
- index the pastebin
- index the web
And add gnunet to the mix.
There is not a lot of get-together pure guile project out there. There
is guix. I hope this new project
can be the occasion for new guilers to submit patches or ideas.
The code is currently hosted at
https://framagit.org/a-guile-mind/culturia.one/tree/guildhall/
The codename is "primary".
Also I'd like to point out, that I don't need primary. I do it for the
community only. The
road is fascinating, tho. I could make mistakes so please input your
feedback if you want
to use a tool like that.
guix package -i wiredtiger
git clone https://framagit.org/a-guile-mind/culturia.one.git
git checkout guildhall
emacs culturia.one/src/webui.scm
or
cd culturia.one/src/ && guile -L . webui.scm

[Amirouche]

Post by Nala Ginrut
Hi Amiroiche!

Héllo Nala!

Post by Nala Ginrut
1. SPM(system pkg manager)
It's used for building a distro, like RPM, deb, Guix.
2. LPM(language pkg manager)
It's used for specific language to install optional packages. Like
npm, gem, pip, etc.
Guix could do that too, IMO.
3. APM(application pkg manager)
It's used for specific framework to install packages in local
application directory. This is important for deployment.
Ruby has "bundle" for that, and Python usually use pip+virtualenv for
that, and npm is flexible enough to do that for JS too. Of course
there are alternative tools do the similar thing in each community.
For now, I'm concerning an APM for Artanis.

I think right now I will do Language Package Manager.
I'd like to focus on community building feature that helps
document GNU Guile, like:

- wiki
- discussion pages
- pastebin

Code is always code but I need to choose which feature I will
implement given I have not an infinite set of time.

I have all those feature in some project, but now I need to put
them together.

Post by Nala Ginrut
(as a core lib) to implement it.
If you want to write yet another pkg manager, it's better to confirm
which kind are you going to do, and don't forget to reuse some cool
features exists in Guix. :-)

I don't think Guix can help right now. The only issue that intersect with
guix is dependency resolution, and AFAIK Guix does hardcode dependencies
in the sens it's a simple breath first search. And what I planned to do is
implement semantic versioning à la npmjs (or aptitude)
https://docs.npmjs.com/misc/semver

I think guix web, is enough for guix. I am not a regular guixsd user
anymore,
so I am not sure how to help guix/sd.

A far reaching goal can be to link against guile-git to mix both package
management with forge.

Post by Nala Ginrut
There is compan, there is guildhall, there is guix. They shall be primary.
-------- Message transféré --------
Sujet : Re: How to make GNU Guile more successful
Date : Fri, 10 Mar 2017 21:17:37 +0100
Héllo!

Post by Vítor De Araújo
Hi,
I'm relatively new to Guile and new to this list, but I saw this
thread in the archives and I'd like to make some comments.

Welcome!

Post by Vítor De Araújo
First, I think it would be nice to have a Guile wiki, in the

likes of

Post by Vítor De Araújo
cliki.net <http://cliki.net>. A publicly-editable reference

place where people can share

Post by Vítor De Araújo
knowledge about Guile and Guile libraries and projects.

I agree.

Post by Vítor De Araújo
Sure, having a
centralized package manager/repository would be nice, but just

having

Post by Vítor De Araújo
a common place where people can look up what libraries exist and

where

Post by Vítor De Araújo
they can be downloaded would be nice. (I don't know how easy

would it

Post by Vítor De Araújo
be to set up a wiki at the Guile website.)

The current most comprehensive list of GNU Guile package is
available at http://sph.mn/content/3e73
https://github.com/guildhall

Post by Vítor De Araújo
On the topic of package management, I think that, more important

than

Post by Vítor De Araújo
a central repository, is a standard format in which to distribute
packages for easy installation.

I agree. The standard could be to have a tarball named using
semantic versionning, from which one can extract the PACKAGE-NAME.
Inside the tarball, the package manager will expect a PACKAGE-NAME.scm
or a PACKAGE-NAME directory which can be installed into
~/.local/lib/guile
WDYT?

Post by Vítor De Araújo
I've never used guildhall, and I guess
it defines something like this,

Probably but it's not documented.

Post by Vítor De Araújo
but I think just having an
installation command where you can pass the URL of a package to
download and install (say, guild install

http://some.url/somepackage.zip,

Post by Vítor De Araújo
or even a git URL)

This will still require the user to set GUILE_LOAD_PATH. Maybe we can
add ~/.local/lib/guile/ to the default GUILE_LOAD_PATH to make things
easier.
A simple guild install https://url/to/patch can be done in mater of hours
(if not minutes).
WDYT?

Post by Vítor De Araújo
would already go a long way towards making it
easier for people to use third-party libraries in Guile.

I agree.

Post by Vítor De Araújo
(Or maybe package _names_ could be URLs pointing to an index in a
well-defined format, which tells how to download each version of a
package; then we could have even dependency management without a
central repository. I'm pipe-dreaming here, though.)

This seems fun, but at the same time odd.
FWIW, I've started a new effort to build a package manager for GNU
Guile. You can find a demo at http://guildhall.hypermove.net/
1) doesn't work with guile 2.2 and I can not make it work
2) The solver is too complicated, even if it's based on aptitude algorithm
it's still complicated.
3) use a lot of other scheme libraries [0] (like foof-loop) which
doesn't help noobs like me to dive into the code. Maybe those
libraries are *very* neat but why not include them in Guile proper
then and make them default.
[0] https://github.com/ijp/guildhall/tree/master/guildhall/ext
For all this reasons I forked the effort.
1) I only use guile 2.2, right now guile 2.1 is not supported. I think it's
bad this should be addressed at some point. AFAIR the issue is in
guile-wiredtiger (or it's an issue with setlocale (anyway this must be
addressed)))
2) I use a logic programming library called minikanren to solve the
dependency problem. This is the first real problem I can use minikanren
to solve a problem that is craft. This is the logic programming
library in scheme. While it's not very advanced compared to core.logic,
the version I use called microkaren has a straight forward
implementation that can
hold all at once in a single head. Logic programming, like
probabilistic
programming (cf. ) are two research areas in programming languages
that could improve the way we craft algorithms today. The logic approach
being more useful to me (but someone doing machine learning stuff,
might find the probabilistic scheme very useful. Ask me for a link!).
OpenCog
does have a logic language embedded in it's language in their hyper
graph database
(AFAIU it works using backward / forward inference but implements on top
of it higher logic constructs, like abduction, anaphora
resolution...). It's
something they call PLN as Probabilistic Logic Network. AFAIK it's some kind
of probabilistic Datalog (which is one of the API Datomic support).
There is
also probabilistic minikanren (albeit not useable). So, to sum up (because
programming language research + guile + logic = minikanren.
Mind the fact that there is also a guile-log. But I still don't get what
guile-log does (hints: http://c-lambda.se/) whereas I understand how
microkanren does its stuff (I still need to benchmark it).
3) I don't use foof-loop but I bring my own dependencies. I use my
own database library that is based on wiredtiger. I've been working
on this database library for 3 years, it's well documented and various
example codes. That said, AFAIK, I am the only one to use it. I've built
a clone of Datomic (with the persistence part, patch welcome) which
use a query engine similar to Datomic I guess based on minikanren.
- 1500 document reads per seconds
- 1000 to 500 document write per second
Documents are scheme assoc, and those are inserted 1000 per 1000 and
read 1000 per 1000 until it reach 50Go of data. At that point writes
take 1/500
seconds or 0.002 seconds.
The biggest dataset I *loaded* into that database is wikidata which is
50Go, I
don't remember how much time it takes to load it.
I had issues with wiredtiger using the Python bindings but only during reads
over a gigantic dataset of 60Giga. I say gigantic for a blog not
for Google
scale. Also upstream can solve issues if we can have a way to reproduce the
issues (which I plan to do once guile 2.2 is out (which means I will
redo the
benchmarks against wikidata and read/write)). I understand the problem
that wiredtiger being part of MongoDB is problematic as MongoDB might not
care much for the same problem as ours. They will always be interested by
bigger free software database, tho. As it make free publicity of how
successful
is their software.
Also there is much documentation about this library. I created several
Guile projects
using it (albeit not big) they document several layers of the library
and one
Guile user reported using it for doing human/social science research.
Search for guile wiredtiger in you favorite duckduckgo search engine.
Some people claim that PostgreSQL has all the required feature that
someone wants
to store data and that this mongodb/wiredtiger is a bad. I recognize
that wiredtiger
can be poor man's database right now. I don't have the required
expertise to verify
whether it's good enough for your usecase. PostgreSQL is used in
"production" in
all the world using multiple workload and stuff. PostgreSQL is good.
1) There is no dynamic ffi bindings for PostgreSQL yet, otherwise said you
can't use pgsql from Guile RIGHT NOW.
2) wiredtiger is GOOD enough, benchmark it before saying it's not good
ENOUGH.
3) it's not SQL, it's Schemey! It's inspired from datomic which the goto
database
for clojure with multiple implementation in the browser. This is
killer feature
of the clojure ecosystem that is closed source.
4) I find it more funny to fiddle with logic programming that set pseudo
theory of SQL.
5) The first thing I do when pgsql bindings will be out, is to port
feature space to PostgreSQL
and compare both pgsql and wiredtiger.
6) wiredtiger will always be simpler to use that Postgresql it's like
comparing sqlite
and Datomic.
What happens next? You choose to hack on this project and find something
interesting in this
- fix guile-wiredtiger to work on guile 2.0
- make it possible in guile-wiredtiger to stream the results of a look
up using traversi using multiple cursors
- implement disequality in microkanren (for implementing the package
dependency resolver
and for fun because logic is awesome).
- implement a pastebin service using GNU Guile, yes because I think this
a good bad reason to start
another simple project that is not a static blog generator.
- Port the wiki of http://culturia.one to use feature space library.
- implement or find a scheme library that does the diff two lists (to
finish wiki implementation
and for the package dependency resolver).
- Help Matt Wette to complete his ffi helper
http://lists.gnu.org/archive/html/guile-user/2017-03/msg00085.html
(because we will need it for guix).
- add guile-fibers as a submodule and use it
- there is also a simple boolean keyword search engine that must be
ported to feature space
- find the code that implements trigram transderivational search and put
it in the repository. Bridging
the gap with tsearch2.
- index the wiki
- index the packages
- index the pastebin
- index the web
And add gnunet to the mix.
There is not a lot of get-together pure guile project out there. There
is guix. I hope this new project
can be the occasion for new guilers to submit patches or ideas.
The code is currently hosted at
https://framagit.org/a-guile-mind/culturia.one/tree/guildhall/
The codename is "primary".
Also I'd like to point out, that I don't need primary. I do it for the
community only. The
road is fascinating, tho. I could make mistakes so please input your
feedback if you want
to use a tool like that.
guix package -i wiredtiger
git clone https://framagit.org/a-guile-mind/culturia.one.git
git checkout guildhall
emacs culturia.one/src/webui.scm
or
cd culturia.one/src/ && guile -L . webui.scm

[Vítor De Araújo]

Post by Amirouche
Héllo!

Post by Vítor De Araújo
Hi,
I'm relatively new to Guile and new to this list, but I saw this
thread in the archives and I'd like to make some comments.

Welcome!

Thanks :)

[...]

Post by Amirouche

Post by Vítor De Araújo
On the topic of package management, I think that, more important than
a central repository, is a standard format in which to distribute
packages for easy installation.

I agree. The standard could be to have a tarball named using
semantic versionning, from which one can extract the PACKAGE-NAME.
Inside the tarball, the package manager will expect a PACKAGE-NAME.scm
or a PACKAGE-NAME directory which can be installed into ~/.local/lib/guile
WDYT?

That's good, but it'd also have to contain a metadata file containing at
least the package's dependencies and what commands should be run to
install (for example, ./configure && make && make install, for packages
which need some compilation step before copying to ~/.local/lib/guile).

Post by Amirouche

Post by Vítor De Araújo
I've never used guildhall, and I guess
it defines something like this,

Probably but it's not documented.

Post by Vítor De Araújo
but I think just having an
installation command where you can pass the URL of a package to
download and install (say, guild install http://some.url/somepackage.zip,
or even a git URL)

This will still require the user to set GUILE_LOAD_PATH. Maybe we can
add ~/.local/lib/guile/ to the default GUILE_LOAD_PATH to make things
easier.

Yes, ~/.local/lib/guile/ (or other user-level directory) definitely
should be in the default GUILE_LOAD_PATH.

Post by Amirouche
A simple guild install https://url/to/patch can be done in mater of hours
(if not minutes).
WDYT?

Sure. Adding dependency handling would make things somewhat more
complex, though.

I thought a little bit about the packages-as-urls idea and I had some
ideas I'll just throw out here.

First, I realized using the URL as the package _name_ is problematic,
because it hard-codes the source of the package; that complicates things
if someone wants to make a fork of the package and install it in place
of the original. However, packages can still have a normal name and a
"canonical URL" for fetching it.

The package would come with a metadata file (named something like
metadata.foopkg, where 'foopkg' is a placeholder for the name of the
package manager (and I'd rather have the name of the package manager as
an extension, because I'd rather not assume I'm the only package manager
in the world)). This metadata would contain the package name, version,
and dependencies. For each dependency, it would specify the package
name, the version, *and the canonical URL* for that package. Upoin
installation, the package manager would go through each dependency and
check if a package of that name and compatible version (assuming
semantic versioning) is already installed; if it's not, it would fetch
it from the canonical URL.

In this way, we can have decentralized package management which can
handle dependencies.

Post by Amirouche
Yeah, for one I don't think the URLs should go directly into the
code, but
rather in a package metadata file. The reason is I don't think we

should

Post by Amirouche
hard-code the package manager in the code. People should be able

to use

Post by Amirouche
the code without having the package manager, if they already have the
dependencies installed by some other means. But the basic idea is
cool. :)
There is a reason why I like the idea of having URLs directly in the code,
namely that it makes you independent from some big institutions/particular
maintainers. Of course it is risky if your code refers to repositories

that

Post by Amirouche
you do not control, but you can always make your private forks.

I'm totally in favor of independence from particular institutions or
maintainers, but I think this can be achieved by putting the URLs in the
metadata file rather than directly in the code. Code would just use
(use-modules ...) as usual to load code, and the business of dowloading
and installing would be solely the task of the package manager. I think
that enables a friendlier co-existence with other package managers, and
allows the same library to be packaged for multiple package managers at
the same time (e.g., guix, compan, guildhall, etc.). It also allows
people to just unpack and install the code manually if they want to.

(A problem that still remains if packages have names separate from the
URLs is avoiding name clashes. The wiki would go some way towards that
(people could check more easily if the name they want is already in
use), but maybe some conventions for globally unique names (e.g.,
prepending a domain prefix like Java people do) could ameliorate this. I
don't think this is really important at this point.)

[...]

Post by Amirouche
FWIW, I've started a new effort to build a package manager for GNU
Guile. You can find a demo at http://guildhall.hypermove.net/
1) doesn't work with guile 2.2 and I can not make it work
2) The solver is too complicated, even if it's based on aptitude algorithm
it's still complicated.
3) use a lot of other scheme libraries [0] (like foof-loop) which
doesn't help noobs like me to dive into the code. Maybe those
libraries are *very* neat but why not include them in Guile proper
then and make them default.
[0] https://github.com/ijp/guildhall/tree/master/guildhall/ext
For all this reasons I forked the effort.
1) I only use guile 2.2, right now guile 2.1 is not supported. I think it's
bad this should be addressed at some point. AFAIR the issue is in
guile-wiredtiger (or it's an issue with setlocale (anyway this must be
addressed)))
2) I use a logic programming library called minikanren to solve the
dependency problem. This is the first real problem I can use minikanren
to solve a problem that is craft. This is the logic programming
library in scheme. While it's not very advanced compared to core.logic,
the version I use called microkaren has a straight forward
implementation that can
hold all at once in a single head. Logic programming, like probabilistic
programming (cf. ) are two research areas in programming languages
that could improve the way we craft algorithms today. The logic approach
being more useful to me (but someone doing machine learning stuff,
might find the probabilistic scheme very useful. Ask me for a link!).
OpenCog
does have a logic language embedded in it's language in their hyper
graph database
(AFAIU it works using backward / forward inference but implements on top
of it higher logic constructs, like abduction, anaphora resolution...). It's
something they call PLN as Probabilistic Logic Network. AFAIK it's some kind
of probabilistic Datalog (which is one of the API Datomic support).
There is
also probabilistic minikanren (albeit not useable). So, to sum up (because
programming language research + guile + logic = minikanren.
Mind the fact that there is also a guile-log. But I still don't get what
guile-log does (hints: http://c-lambda.se/) whereas I understand how
microkanren does its stuff (I still need to benchmark it).
3) I don't use foof-loop but I bring my own dependencies. I use my
own database library that is based on wiredtiger. I've been working
on this database library for 3 years, it's well documented and various
example codes. That said, AFAIK, I am the only one to use it. I've built
a clone of Datomic (with the persistence part, patch welcome) which
use a query engine similar to Datomic I guess based on minikanren.
- 1500 document reads per seconds
- 1000 to 500 document write per second
Documents are scheme assoc, and those are inserted 1000 per 1000 and
read 1000 per 1000 until it reach 50Go of data. At that point writes
take 1/500
seconds or 0.002 seconds.
The biggest dataset I *loaded* into that database is wikidata which is
50Go, I
don't remember how much time it takes to load it.
I had issues with wiredtiger using the Python bindings but only during reads
over a gigantic dataset of 60Giga. I say gigantic for a blog not for Google
scale. Also upstream can solve issues if we can have a way to reproduce the
issues (which I plan to do once guile 2.2 is out (which means I will
redo the
benchmarks against wikidata and read/write)). I understand the problem
that wiredtiger being part of MongoDB is problematic as MongoDB might not
care much for the same problem as ours. They will always be interested by
bigger free software database, tho. As it make free publicity of how
successful
is their software.
Also there is much documentation about this library. I created several
Guile projects
using it (albeit not big) they document several layers of the library
and one
Guile user reported using it for doing human/social science research.
Search for guile wiredtiger in you favorite duckduckgo search engine.
Some people claim that PostgreSQL has all the required feature that
someone wants
to store data and that this mongodb/wiredtiger is a bad. I recognize
that wiredtiger
can be poor man's database right now. I don't have the required
expertise to verify
whether it's good enough for your usecase. PostgreSQL is used in
"production" in
all the world using multiple workload and stuff. PostgreSQL is good.
1) There is no dynamic ffi bindings for PostgreSQL yet, otherwise said you
can't use pgsql from Guile RIGHT NOW.
2) wiredtiger is GOOD enough, benchmark it before saying it's not good
ENOUGH.
3) it's not SQL, it's Schemey! It's inspired from datomic which the goto
database
for clojure with multiple implementation in the browser. This is
killer feature
of the clojure ecosystem that is closed source.
4) I find it more funny to fiddle with logic programming that set pseudo
theory of SQL.
5) The first thing I do when pgsql bindings will be out, is to port
feature space to PostgreSQL
and compare both pgsql and wiredtiger.
6) wiredtiger will always be simpler to use that Postgresql it's like
comparing sqlite
and Datomic.
What happens next? You choose to hack on this project and find something
interesting in this
- fix guile-wiredtiger to work on guile 2.0
- make it possible in guile-wiredtiger to stream the results of a look
up using traversi using multiple cursors
- implement disequality in microkanren (for implementing the package
dependency resolver
and for fun because logic is awesome).
- implement a pastebin service using GNU Guile, yes because I think this
a good bad reason to start
another simple project that is not a static blog generator.
- Port the wiki of http://culturia.one to use feature space library.
- implement or find a scheme library that does the diff two lists (to
finish wiki implementation
and for the package dependency resolver).
- Help Matt Wette to complete his ffi helper
http://lists.gnu.org/archive/html/guile-user/2017-03/msg00085.html
(because we will need it for guix).
- add guile-fibers as a submodule and use it
- there is also a simple boolean keyword search engine that must be
ported to feature space
- find the code that implements trigram transderivational search and put
it in the repository. Bridging
the gap with tsearch2.
- index the wiki
- index the packages
- index the pastebin
- index the web
And add gnunet to the mix.
There is not a lot of get-together pure guile project out there. There
is guix. I hope this new project
can be the occasion for new guilers to submit patches or ideas.
The code is currently hosted at
https://framagit.org/a-guile-mind/culturia.one/tree/guildhall/
The codename is "primary".
Also I'd like to point out, that I don't need primary. I do it for the
community only. The
road is fascinating, tho. I could make mistakes so please input your
feedback if you want
to use a tool like that.
guix package -i wiredtiger
git clone https://framagit.org/a-guile-mind/culturia.one.git
git checkout guildhall
emacs culturia.one/src/webui.scm
or
cd culturia.one/src/ && guile -L . webui.scm

There are certainly lots of fun subprojects to tackle here, but I'm
afraid it would take really long to get ready. I'd rather have a really
simple package manager, preferably with zero dependencies other than
Guile itself. But then again I think multiple package managers should be
able to co-exist, so there's no problem for different people to tackle
multiple approaches to the problem.

[Vítor De Araújo]

On 10/03/2017 21:50, Vítor De Araújo wrote:
[...]

Post by Vítor De Araújo
I thought a little bit about the packages-as-urls idea and I had some
ideas I'll just throw out here.
First, I realized using the URL as the package _name_ is problematic,
because it hard-codes the source of the package; that complicates things
if someone wants to make a fork of the package and install it in place
of the original. However, packages can still have a normal name and a
"canonical URL" for fetching it.
The package would come with a metadata file (named something like
metadata.foopkg, where 'foopkg' is a placeholder for the name of the
package manager (and I'd rather have the name of the package manager as
an extension, because I'd rather not assume I'm the only package manager
in the world)). This metadata would contain the package name, version,
and dependencies. For each dependency, it would specify the package
name, the version, *and the canonical URL* for that package. Upoin
installation, the package manager would go through each dependency and
check if a package of that name and compatible version (assuming
semantic versioning) is already installed; if it's not, it would fetch
it from the canonical URL.
In this way, we can have decentralized package management which can
handle dependencies.

[...]

Just one more little thought on package names vs. URLs. Guile already
has a standard module system, and modules already have names which are
supposed to be globally unique, and even they even have version numbers.
So module names can double as package names. The advantage is that then
you can check whether a dependency is installed *whether or not it was
installed via the package manager*. Basically you'd describe your
dependencies in terms of modules, and specify where the package that
implements that module can be fetched from.

I have to think this through a bit more, though.

[Thien-Thi Nguyen]

() Amirouche <***@hypermove.net>
() Fri, 10 Mar 2017 21:17:37 +0100

1) There is no dynamic ffi bindings for PostgreSQL yet,
otherwise said you can't use pgsql from Guile RIGHT NOW.

Plug: Guile-PG[0] does not provide ffi bindings for PostgreSQL,
but it does work w/ Guile 2.0 today (last i checked: 2.0.11).
Paren lovers might like the query construction DSL[1], too.

[0] http://www.nongnu.org/guile-pg/
[1] http://www.nongnu.org/guile-pg/doc/Query-Construction.html

Please report any problems you find w/ it to this mailing list.
I'm especially interested in experiences w/ Guile 2.x (x > 0).