How to notice abrupt tcp connection losses in server/client?

Discussion:

Zelphir Kaltstahl

2018-06-21 06:22:36 UTC

Hello Guile users,

I wrote some TCP server and client in Guile which I have uploaded here:

https://gitlab.com/zelphir-kaltstahl-projects/guile-scheme-tutorials-and-examples/raw/dev/network-programming/tcp-client.scm

and here:

https://gitlab.com/zelphir-kaltstahl-projects/guile-scheme-tutorials-and-examples/raw/dev/network-programming/tcp-server.scm

or normal GitLab view:

https://gitlab.com/zelphir-kaltstahl-projects/guile-scheme-tutorials-and-examples/tree/dev/network-programming

(Is it OK to post these as links, or always better to include all
relevant source code on the mailing list? On other e-mail lists I have
experienced that my message was too long and got truncated, so I posted
the code as links to the raw file on GitLab.)

The loop for reacting on messages from a client on the server looks like
this:

(while #t
(let* ([bytes-count (recv! in-out-sock receive-buffer)]
[message-received (byte-vector->utf8-message receive-buffer bytes-count)])
(message-handler client-connection message-received))))))

But this has a problem: When I run both client and server in two
terminals and then exit the client using Ctrl-D, the server somehow gets
stuck in the loop always receiving the empty string. Since that is 0
Bytes long, it does not really take anything from the socket (if I
understand correctly), but instead can recv! in the next iteration again
immediately. The only thing that works then for stopping this loop is to
hold down Ctrl-C on the server for a second or so.
The same happens for the client receiving message loop, because that one
also does not detect me suddenly interrupting or killing the server and
then loops on the empty string.
At first I thought if I caught eof-object? and then (break) the loop it
would solve the problem, but apparently it does not.
Basically I would like the server and client to be prepared for non
proper shutdown of either.

How can I handle / detect abrupt connection losses, so that I can break
the message handling loop?

Send guile-user mailing list submissions to
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.gnu.org/mailman/listinfo/guile-user
or, via email, send a message with subject or body 'help' to
You can reach the person managing the list at
When replying, please edit your Subject line so it is more specific
than "Re: Contents of guile-user digest..."
1. Re: lat? and atom? not in guile? (Thompson, David)
----------------------------------------------------------------------
Message: 1
Date: Wed, 20 Jun 2018 10:55:43 -0400
Subject: Re: lat? and atom? not in guile?
Content-Type: text/plain; charset="UTF-8"

Oh hey David! Thanks for the explanation! Maybe I need to take another
look at that book then. I found it a little repetitive, but I am
probably not quite grasping some of the fundamentals.

It's meant to be repetitive. It's explicitly compared in the introduction
to Hanon's finger exercises for the piano: C-E-F-G-A-G-F-E,
D-F-G-A-B-A-G-F, E-G-A-B-C-B-A-D and so on forever, up the scale and down
in every key. Boring as hell, but just the thing to get fluency into your
fingers.

It's repetitive but honestly I found that the comedic writing made the
exercises very entertaining. I was rarely bored. I don't play the
piano, but I do play drums, and the equivalent to Hanon's finger
exercises is an old book called Stick Control, which is definitely in
the boring as hell category! I'm glad that Lisp beginners have
something more fun available. :)
- Dave
------------------------------
Subject: Digest Footer
_______________________________________________
guile-user mailing list
https://lists.gnu.org/mailman/listinfo/guile-user
------------------------------
End of guile-user Digest, Vol 187, Issue 14
*******************************************

Joshua Branson

2018-06-21 13:49:12 UTC

Permalink

I'll have to check these out! Thanks for mentioning them!

Chris Vine

2018-06-21 15:08:20 UTC

Permalink

On Thu, 21 Jun 2018 08:22:36 +0200

Post by Zelphir Kaltstahl
Hello Guile users,
https://gitlab.com/zelphir-kaltstahl-projects/guile-scheme-tutorials-and-examples/raw/dev/network-programming/tcp-client.scm
https://gitlab.com/zelphir-kaltstahl-projects/guile-scheme-tutorials-and-examples/raw/dev/network-programming/tcp-server.scm
https://gitlab.com/zelphir-kaltstahl-projects/guile-scheme-tutorials-and-examples/tree/dev/network-programming
(Is it OK to post these as links, or always better to include all
relevant source code on the mailing list? On other e-mail lists I have
experienced that my message was too long and got truncated, so I posted
the code as links to the raw file on GitLab.)
The loop for reacting on messages from a client on the server looks like
(while #t
(let* ([bytes-count (recv! in-out-sock receive-buffer)]
[message-received (byte-vector->utf8-message receive-buffer bytes-count)])
(message-handler client-connection message-received))))))
But this has a problem: When I run both client and server in two
terminals and then exit the client using Ctrl-D, the server somehow gets
stuck in the loop always receiving the empty string. Since that is 0
Bytes long, it does not really take anything from the socket (if I
understand correctly), but instead can recv! in the next iteration again
immediately. The only thing that works then for stopping this loop is to
hold down Ctrl-C on the server for a second or so.
The same happens for the client receiving message loop, because that one
also does not detect me suddenly interrupting or killing the server and
then loops on the empty string.
At first I thought if I caught eof-object? and then (break) the loop it
would solve the problem, but apparently it does not.
Basically I would like the server and client to be prepared for non
proper shutdown of either.
How can I handle / detect abrupt connection losses, so that I can break
the message handling loop?

The POSIX recv() function returns 0 on end of file (connection closed)
so I expect the scheme recv! procedure does the same. So on end-of-file
your code appears to be producing an endless supply of empty strings.

More generally, you will find it easier to use something like the
get-line procedure to read text, or something like get-bytevector-n or
get-bytevector-n! for reading binary records. recv! only really becomes
important when the flags argument is meaningful.

Rather than starting a new thread for each connection, you will get much
better scalability if you use something like fibers
( https://github.com/wingo/fibers/ ), guile-a-sync2
( https://github.com/ChrisVine/guile-a-sync2/ ) or 8sync
( https://www.gnu.org/software/8sync/ ).

Chris

Zelphir Kaltstahl

2018-06-21 22:19:49 UTC

Permalink

Message: 5
Date: Thu, 21 Jun 2018 16:08:20 +0100
Subject: Re: How to notice abrupt tcp connection losses in
server/client?
Content-Type: text/plain; charset=US-ASCII
On Thu, 21 Jun 2018 08:22:36 +0200

I actually tried to (break) the loop when the received message is
(eof-object? ...), but it might be that I used this predicate at the
wrong place (directly inside the loop when the message was already
copied from the receive-buffer, instead of when recv!, I believe), now
that I think about it … and maybe that is why (break)ing the loop did
not work. Need to investigate more.

More generally, you will find it easier to use something like the
get-line procedure to read text, or something like get-bytevector-n or
get-bytevector-n! for reading binary records. recv! only really becomes
important when the flags argument is meaningful.

Ah ok, thanks for the advice! I thought recv! was the way to go, as it
worked so nicely so far.

Rather than starting a new thread for each connection, you will get much
better scalability if you use something like fibers
( https://github.com/wingo/fibers/ ), guile-a-sync2
( https://github.com/ChrisVine/guile-a-sync2/ ) or 8sync
( https://www.gnu.org/software/8sync/ ).

Yep =) I am aware. I already wrote a comment inside the code noting that
this new thread thing is probably not so good. Not sure I should
introduce fibers into this example code yet though. And looking at
fibers and 8sync is on my to-do list ; )

Chris

I will try these things and see how it goes. Thanks!

~ Zelphir

Chris Vine

2018-06-21 23:09:16 UTC

Permalink

On Fri, 22 Jun 2018 00:19:49 +0200
[snip]

Post by Zelphir Kaltstahl

Post by Chris Vine
The POSIX recv() function returns 0 on end of file (connection closed)
so I expect the scheme recv! procedure does the same. So on end-of-file
your code appears to be producing an endless supply of empty strings.

It looks as if recv! returns 0 on end-of-file, not an eof-object, so
that may be your problem. That is not the case with get-line or
get-bytevector-n. Note that recv! may also return less than the
requested number of bytes - it does not guarantee to block until the
entire request has been met or end-of-file is reached, as it is just a
wrapper for POSIX/windows recv().

I think you will need to use recv! with windows, because you cannot
read from sockets in windows using POSIX read(). With other OS's it
would be more normal to use the R5RS/R6RS port procedures, because they
are easier to use and are buffered.

For asynchronous non-blocking ports, I am not sure whether recv! is safe
with guile-2.2's suspendable ports or not. The get-line and
get-bytevector-n procedures are. The get-bytevector-n! procedure is
not. I suspect recv! may not be also.

Post by Zelphir Kaltstahl

Post by Chris Vine
More generally, you will find it easier to use something like the
get-line procedure to read text, or something like get-bytevector-n or
get-bytevector-n! for reading binary records. recv! only really becomes
important when the flags argument is meaningful.

Ah ok, thanks for the advice! I thought recv! was the way to go, as it
worked so nicely so far.

A call to unix read() on a socket is equivalent to a call to recv()
with default flags. At the end of it, the buffered guile port reading
procedures generally involve a call to read().

Post by Zelphir Kaltstahl

Post by Chris Vine
Rather than starting a new thread for each connection, you will get much
better scalability if you use something like fibers
( https://github.com/wingo/fibers/ ), guile-a-sync2
( https://github.com/ChrisVine/guile-a-sync2/ ) or 8sync
( https://www.gnu.org/software/8sync/ ).

Post by Chris Vine
Chris

I will try these things and see how it goes. Thanks!

The guile-a-sync/guile-a-sync2 libraries (of which I am the author) have
an example of a client and server in the doc directory, which might be
of interest to you. fibers is definitely worth a look also, but is
linux only, and also guile-2.2 only. I think it also has some examples
(not sure).

Chris

Eli Zaretskii

2018-06-22 06:27:30 UTC

Permalink

Date: Fri, 22 Jun 2018 00:09:16 +0100
I think you will need to use recv! with windows, because you cannot
read from sockets in windows using POSIX read().

What makes you say that? It isn't true; Gawk does that on Windows,
and it works very well. The only two issues, which are easy to
overcome with wrappers or macros, are:

. you need to create the socket with 'WSASocket; rather than
'socket', since the latter creates overlapped sockets that cannot
be used with file I/O APIs;
. you need to convert the SOCKET type (which is a handle in
disguise) into a file descriptor and back using a pair of library
functions

Perhaps Guile doesn't yet do that (I didn't look, it could be in
Gnulib functions), but it would be easy to add if so.

Chris Vine

2018-06-22 10:07:39 UTC

Permalink

On Fri, 22 Jun 2018 09:27:30 +0300

Post by Eli Zaretskii

Date: Fri, 22 Jun 2018 00:09:16 +0100
I think you will need to use recv! with windows, because you cannot
read from sockets in windows using POSIX read().

What makes you say that? It isn't true; Gawk does that on Windows,
and it works very well. The only two issues, which are easy to
. you need to create the socket with 'WSASocket; rather than
'socket', since the latter creates overlapped sockets that cannot
be used with file I/O APIs;
. you need to convert the SOCKET type (which is a handle in
disguise) into a file descriptor and back using a pair of library
functions
Perhaps Guile doesn't yet do that (I didn't look, it could be in
Gnulib functions), but it would be easy to add if so.

OK, my misunderstanding then. Looking at the wrappers for windows
sockets (lib/socket.c) it looks as if it may work - since you use
windows and I don't, why not try it and tell us?

Zelphir Kaltstahl

2018-06-22 20:17:51 UTC

Permalink

Message: 2
Date: Fri, 22 Jun 2018 00:09:16 +0100
Subject: Re: How to notice abrupt tcp connection losses in
server/client?
Content-Type: text/plain; charset=UTF-8
On Fri, 22 Jun 2018 00:19:49 +0200
[snip]

Post by Zelphir Kaltstahl

I actually tried to (break) the loop when the received message is
(eof-object? ...), but it might be that I used this predicate at the
wrong place (directly inside the loop when the message was already
copied from the receive-buffer, instead of when recv!, I believe), now
that I think about it ? and maybe that is why (break)ing the loop did
not work. Need to investigate more.

I think you are exactly right. If it returns 0 for end of file (what is
the difference between this and end of file object?), then 0 bytes will
be copied to the receive byte vector. When converting that to utf8
string, it will be the empty string and that is printed. However, since
there is still no new message (client connection is no more) there is
still only EOF and the same will happen over and over again.

I think you will need to use recv! with windows, because you cannot
read from sockets in windows using POSIX read(). With other OS's it
would be more normal to use the R5RS/R6RS port procedures, because they
are easier to use and are buffered.
For asynchronous non-blocking ports, I am not sure whether recv! is safe
with guile-2.2's suspendable ports or not. The get-line and
get-bytevector-n procedures are. The get-bytevector-n! procedure is
not. I suspect recv! may not be also.

Post by Zelphir Kaltstahl

Ah ok, thanks for the advice! I thought recv! was the way to go, as it
worked so nicely so far.

A call to unix read() on a socket is equivalent to a call to recv()
with default flags. At the end of it, the buffered guile port reading
procedures generally involve a call to read().

Post by Zelphir Kaltstahl

Post by Chris Vine
Chris

I will try these things and see how it goes. Thanks!

I now use get-bytevector-n with the count which was the size of the
receive-buffer and it does not have the endless looping behavior. All
seems fine now. Thanks!

Chris Vine

2018-06-23 11:42:52 UTC

Permalink

On Fri, 22 Jun 2018 22:17:51 +0200
Zelphir Kaltstahl <***@gmail.com> wrote:
[snip]

... it returns 0 for end of file (what is the difference between this
and end of file object?

R6RS states that "The end-of-file object is returned by various I/O
procedures when they reach end of file". It can be obtained by invoking
the R6RS eof-object procedure and can be queried with the eof-object?
predicate, so '(eof-object? (eof-object))' will evaluate to true. R6RS
also says "The end-of-file object is not a datum value, and thus has no
external representation".

0 is an integer. POSIX read() and recv() indicate end-of-file by
unblocking and returning 0. Subsequent calls to read() or recv() might
return -1 indicating an error, in which case recv! would throw an
exception.

Zelphir Kaltstahl

2018-07-08 11:49:04 UTC

Permalink

Hi!

I decided to take a look at how one can parse command line arguments in
Guile and was looking for something like argparse in Python. It seems
that (use-modules (ice-9 getopt-long)) does the job, except that I hit
one problem and don't know what the mistake I am making is. It seems to
be connected to the usage of `predicate` in my code.

The following is an example program, which I wrote to try the
getopt-long facilities, adapted from the docs example at
https://www.gnu.org/software/guile/manual/html_node/getopt_002dlong-Example.html#getopt_002dlong-Example:

;; ===== EXAMPLE START =====
(add-to-load-path (dirname (current-filename)))

(use-modules (ice-9 getopt-long))

(define (string-exact-integer? str)
(exact-integer? (string->number str)))

(define option-spec
'((version (single-char #\v) (value #f))
    (help    (single-char #\h) (value #f))
    (user-name (value #t) (required? #f))
    (times-hello (value #t)
                 (single-char #\n)
                 (required? #f)
                 (predicate string-exact-integer?))))

(define options (getopt-long (command-line) option-spec))

(option-ref options 'help #f)

(define (main)
(let* ([help-wanted (option-ref options 'help #f)]
         [version-wanted (option-ref options 'version #f)]
         [user-name (option-ref options 'user-name #f)]
         [times-say-hello (option-ref options 'times-hello 1)])
    (cond [(or version-wanted help-wanted)
           (when version-wanted
             (display "command-line-arguments.scm 1.0.0\n"))
           (when help-wanted
             (display
              (string-join '(""
                             "getopt-long-example [options]"
                             "-v, --version Display version"
                             "-h, --help     Display this help"
                             "")
                           "\n")))]
          [else
           (do ([i 0 (1+ i)])
               ([>= i times-say-hello])
             (display (simple-format #f "Hello, ~a!\n" (if user-name
                                                           user-name
                                                           "World"))))])))

(main)
;; ===== EXAMPLE END =====

This program I call as follows in Emacs' Eshell:

;; ===== COMMAND START =====
guile command-line-arguments.scm -n 3 --user-name test
;; ===== COMMAND END =====

Then I get the following error:

;; ===== ERROR START =====
Backtrace:
           9 (apply-smob/1 #<catch-closure a98b80>)
In ice-9/boot-9.scm:
    705:2 8 (call-with-prompt ("prompt") #<procedure a9a0a0 at
ice-9/eval.scm:330:13 ()> #<procedure…>)
In ice-9/eval.scm:
    619:8 7 (_ #(#(#<directory (guile-user) b2e140>)))
In ice-9/boot-9.scm:
   2312:4 6 (save-module-excursion #<procedure ad6330 at
ice-9/boot-9.scm:3827:3 ()>)
3822:12 5 (_)
In
/home/xiaolong/development/Guile/examples-and-convenience-procedures/command-line-arguments/command-line-arguments.scm:
    30:16 4 (_)
In ice-9/getopt-long.scm:
    350:6 3 (getopt-long _ _ #:stop-at-first-non-option _)
In ice-9/boot-9.scm:
   260:13 2 (for-each #<procedure acd1e0 at
ice-9/getopt-long.scm:350:16 (spec)> _)
In ice-9/getopt-long.scm:
   209:28 1 (_ "times-hello" _)
In unknown file:
           0 (_ "3")

ERROR: Wrong type to apply: string-exact-integer?
;; ===== ERROR END =====

Maybe I am doing something stupid, but as far as I can see, I am just
doing what the docs say, I am providing a function that takes the
option's value as string and returns #t or #f. Everything worked up to
when I tried the predicate thing.

What am I doing wrong?

Matt Wette

2018-07-08 15:44:22 UTC

Permalink

Post by Zelphir Kaltstahl
Hi!
I decided to take a look at how one can parse command line arguments in
Guile and was looking for something like argparse in Python. It seems
that (use-modules (ice-9 getopt-long)) does the job, except that I hit
one problem and don't know what the mistake I am making is. It seems to
be connected to the usage of `predicate` in my code.

You probably want to use quasi-quote + unquote:
`((version ... (predicate ,string-exact-integer?))))

I believe the module (srfi srfi-37), args-fold, is now recommended over
getopt-long.

Matt