gcl-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Gcl-devel] Re: Parallel GCL


From: Camm Maguire
Subject: [Gcl-devel] Re: Parallel GCL
Date: 25 Oct 2005 09:38:35 -0400
User-agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.2

Greetings!

Robert Boyer <address@hidden> writes:

> Well, I've started trying to understand your code involving one-result, etc.
> Always wanted to know about sockets, and perhaps now I will.  But you can see
> from the transcript below I didn't get very far.  GCL exited with the message
> 
>         "couldn't open socket"!.
> 

Don't you just love this informative and robust error messsage :-).

> What am I doing wrong?
> 

Nothing as far as I can see.  I've assumed perhaps too much thus far
in putting this together -- it is possible that some sites disallow
server sockets in a certain port range at the kernel level, etc.  I've
managed to build cvs head on one of your new ubuntu machines and could
not reproduce the error you report, but ran into other
dysfunctionality which I will try to investigate this evening.
Perhaps you could point me to the machine you are using.  

Also, for comparison purposes, I've included the output below of the
intended behavior as it is currently on my machine.  You can see there
that this sequence creates 10 sub gcl processes to read some input
from the parent, run #'complicated-function and return its output
before quiting.  The parent then waits for the results and provides
the completed list when done. #'complicated-function is totally
artificial and contrived to take a random amount of time to complete
-- any other similiar function would suffice.  It is just that all
children inherit the exact memory image of the parent as of the time
the sockets were created, and thus have the same default random state,
so their calls to #'random return identical results.

> By the way, and this is just a matter of taste. I claim it is much harder to
> read the line
> 
>    (mapcar (lambda (x) (read-byte s)) (make-list 10)) 
> 
> than
> 
>    (loop for i below 10 collect (read-byte s))
> 
> I suspect that the latter is vastly more efficient in GCL, unless you have
> put in some amazing optimizers to actually avoid doing the make-list.
> 

I agree what I wrote should be replaced by your version.  I've somehow
migrated toward avoiding the loop macro almost entirely, clearly for
irrational reasons.  I've been influenced by the functional nature of
acl2 and the like, and as well become a little confused when looking
at loop as it is never completely clear to me what is evaluated and
when, and what are special syntax keywords.  It really does appear to
break standard lisp syntax, but packages itself in parenthesis to
confuse the matter.  Of course, one could say the same for the other
special lisp forms which bind variables (like let) to a lesser extent
as well.  Perhaps it just takes getting used to -- I will endeavor to
do so.

Please remind me to discuss sloop/loop with you at some point.

> > Though currently quite crude, you can see how
> > a parallel-let, "parallel-and" and "parallel-or" would be implemented
> > (I believe) from this example -- if not please let me know and I will
> > try to elaborate.
> 
> Sorry, I don't yet have a clue on these matters of p-let, p-and, and p-or in
> GCL via sockets.

OK, with all the baggage of the current implementation
notwithstanding and in rough sketch form:

(plet ((x (foo y)) (z (bar w))) (baz x z)) -> (via macroexpansion)

        (let* ((p1 (unique-socket-id))
               (p2 (unique-socket-id)) 
               (l1 (si::socket p1 :server (lambda nil (foo y))  :daemon t))
               (l2 (si::socket p2 :server (lambda nil (bar w))  :daemon t))
               (s1 (si::socket p1 :host "localhost"))
               (s2 (si::socket p2 :host "localhost"))
               v1 v2)
           (do nil ((not (or s1 s2)))     ;; This is a poor-man's 'select
                (when (and s1 (listen s1));; and can be replaced by the
                      (setq v1 (read s1)) ;; latter much more efficiently
                      (close s1)(setq s1 nil))
                (when (and s2 (listen s2))
                      (setq v2 (read s2)) 
                      (close s2)(setq s2 nil))
                (when (or s1 s2) (sleep 1)))
           (let ((x v1) (z v2) (baz x z))))

pand would not wait for all results, but return nil on receiving the
first nil and t after receiving the last non-nil. I.e. wrap the whole
thing in a block and issue a conditionalized return-from right after
the read in the above.  por inverts the return logic.  More explicitly:

(pand (foo y) (bar w)) -> (via macroexpansion)

        (let* ((p1 (unique-socket-id))
               (p2 (unique-socket-id)) 
               (l1 (si::socket p1 :server (lambda nil (foo y))  :daemon t))
               (l2 (si::socket p2 :server (lambda nil (bar w))  :daemon t))
               (s1 (si::socket p1 :host "localhost"))
               (s2 (si::socket p2 :host "localhost"))
               v1 v2)
           (do nil ((not (or s1 s2)) t)   
                (when (and s1 (listen s1))
                      (unless (read s1) (return nil))
                      (close s1)(setq s1 nil))
                (when (and s2 (listen s2))
                      (unless (read s2) (return nil))
                      (close s2)(setq s2 nil))
                (when (or s1 s2) (sleep 1))))

(por  (foo y) (bar w)) -> (via macroexpansion)

        (let* ((p1 (unique-socket-id))
               (p2 (unique-socket-id)) 
               (l1 (si::socket p1 :server (lambda nil (foo y))  :daemon t))
               (l2 (si::socket p2 :server (lambda nil (bar w))  :daemon t))
               (s1 (si::socket p1 :host "localhost"))
               (s2 (si::socket p2 :host "localhost"))
               v1 v2)
           (do nil ((not (or s1 s2)) nil)
                (when (and s1 (listen s1))
                      (when   (read s1) (return t  ))
                      (close s1)(setq s1 nil))
                (when (and s2 (listen s2))
                      (when   (read s2) (return t  ))
                      (close s2)(setq s2 nil))
                (when (or s1 s2) (sleep 1))))


One could alternatively define these to return the actual first
non-nil value received, etc., but then the code might not run the same
way when executed again later.

                
> 
> Thanks so very much for dealing with my pedestrian questions.  How did you
> know to use socket number 1927, by the way.  Is that a magic GCL number of
> some sort?
> 

Traditionally, ports below 1000 are reserved for root services.  ssh
listens on 'port' 22, for example.  I just picked a free port on my
machine at the time in the unrestricted range.

Take care,


> 
> Bob
> 

=============================================================================
address@hidden:/fix/t1/camm/debian/gcl/tmp/tmp/foo$ 
address@hidden:/fix/t1/camm/debian/gcl/tmp/tmp/foo$ unixport/saved_ansi_gcl
GCL (GNU Common Lisp)  2.7.0 ANSI  profiling  Oct 17 2005 22:43:00
Source License: LGPL(gcl,gmp), GPL(unexec,bfd)
Binary License:  GPL due to GPL'ed components: (READLINE BFD UNEXEC)
Modifications of this banner must retain notice of a compatible license
Dedicated to the memory of W. Schelter

Use (help) to get some basic information on how to use GCL.

>(defvar *waits* (with-open-file 
                 (s "/dev/random" :element-type '(unsigned-byte 32))
                 (mapcar (lambda (x) (read-byte s)) (make-list 10))))
(defun complicated-function (x) 
  (let ((y (nth (mod (si::getpid) 10) *waits*)))
    (sleep (mod y 30))
    (cons x (mod y x))))
(defun one-result (s) 
  (format s "~s~%" (complicated-function (read s nil 10)))
  (read s nil 'eof)
  (bye))
(defun many-results (s) 
  (let ((x (read s nil 'eof)))
    (unless (eq x 'eof)
      (format s "~s~%" (complicated-function x))
      (many-results s))
    (bye)))
(defun watch (x) 
  (cond ((some 'streamp x) 
         (print x)
         (sleep 1)
         (watch (mapcar (lambda (x) (if (and (streamp x) (listen x)) (read x 
nil 'eof) x)) x)))
        (x)))



(setq l1 (si::socket 1927 :server #'one-result :daemon t))

(setq x (mapcar (lambda (x) (si::socket 1927 :host "localhost")) (make-list 
10)))
(mapcar (lambda (x) (format x "~s~%" (+ 10 (random 20)))) x)
(watch x)


*WAITS*

>
COMPLICATED-FUNCTION

>
ONE-RESULT

>
MANY-RESULTS

>
WATCH

>
20967

>
(#<two-way stream 01200e80> #<two-way stream 01200e20>
 #<two-way stream 01200da0> #<two-way stream 01200d40>
 #<two-way stream 01200c80> #<two-way stream 01200c20>
 #<two-way stream 01200bc0> #<two-way stream 01200b60>
 #<two-way stream 01200b00> #<two-way stream 01200aa0>)

>
(NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL)

>
(#<two-way stream 01200e80> #<two-way stream 01200e20>
 #<two-way stream 01200da0> #<two-way stream 01200d40>
 #<two-way stream 01200c80> #<two-way stream 01200c20>
 #<two-way stream 01200bc0> #<two-way stream 01200b60>
 #<two-way stream 01200b00> #<two-way stream 01200aa0>) 
((27 . 6) #<two-way stream 01200e20> #<two-way stream 01200da0>
 #<two-way stream 01200d40> (14 . 4) #<two-way stream 01200c20>
 #<two-way stream 01200bc0> #<two-way stream 01200b60>
 #<two-way stream 01200b00> (13 . 11)) 
((27 . 6) #<two-way stream 01200e20> #<two-way stream 01200da0>
 #<two-way stream 01200d40> (14 . 4) #<two-way stream 01200c20>
 #<two-way stream 01200bc0> #<two-way stream 01200b60>
 #<two-way stream 01200b00> (13 . 11)) 
((27 . 6) #<two-way stream 01200e20> #<two-way stream 01200da0>
 #<two-way stream 01200d40> (14 . 4) #<two-way stream 01200c20>
 #<two-way stream 01200bc0> #<two-way stream 01200b60>
 #<two-way stream 01200b00> (13 . 11)) 
((27 . 6) #<two-way stream 01200e20> #<two-way stream 01200da0>
 #<two-way stream 01200d40> (14 . 4) #<two-way stream 01200c20>
 #<two-way stream 01200bc0> #<two-way stream 01200b60>
 #<two-way stream 01200b00> (13 . 11)) 
((27 . 6) #<two-way stream 01200e20> #<two-way stream 01200da0>
 #<two-way stream 01200d40> (14 . 4) #<two-way stream 01200c20>
 #<two-way stream 01200bc0> #<two-way stream 01200b60>
 #<two-way stream 01200b00> (13 . 11)) 
((27 . 6) #<two-way stream 01200e20> #<two-way stream 01200da0>
 #<two-way stream 01200d40> (14 . 4) #<two-way stream 01200c20>
 #<two-way stream 01200bc0> #<two-way stream 01200b60>
 #<two-way stream 01200b00> (13 . 11)) 
((27 . 6) (29 . 10) #<two-way stream 01200da0> (23 . 1) (14 . 4)
 #<two-way stream 01200c20> #<two-way stream 01200bc0>
 #<two-way stream 01200b60> #<two-way stream 01200b00> (13 . 11)) 
((27 . 6) (29 . 10) #<two-way stream 01200da0> (23 . 1) (14 . 4)
 #<two-way stream 01200c20> #<two-way stream 01200bc0>
 #<two-way stream 01200b60> #<two-way stream 01200b00> (13 . 11)) 
((27 . 6) (29 . 10) #<two-way stream 01200da0> (23 . 1) (14 . 4)
 #<two-way stream 01200c20> #<two-way stream 01200bc0>
 #<two-way stream 01200b60> #<two-way stream 01200b00> (13 . 11)) 
((27 . 6) (29 . 10) #<two-way stream 01200da0> (23 . 1) (14 . 4)
 #<two-way stream 01200c20> #<two-way stream 01200bc0>
 #<two-way stream 01200b60> #<two-way stream 01200b00> (13 . 11)) 
((27 . 6) (29 . 10) #<two-way stream 01200da0> (23 . 1) (14 . 4)
 #<two-way stream 01200c20> #<two-way stream 01200bc0>
 #<two-way stream 01200b60> #<two-way stream 01200b00> (13 . 11)) 
((27 . 6) (29 . 10) #<two-way stream 01200da0> (23 . 1) (14 . 4)
 #<two-way stream 01200c20> #<two-way stream 01200bc0>
 #<two-way stream 01200b60> (26 . 4) (13 . 11)) 
((27 . 6) (29 . 10) #<two-way stream 01200da0> (23 . 1) (14 . 4)
 #<two-way stream 01200c20> #<two-way stream 01200bc0>
 #<two-way stream 01200b60> (26 . 4) (13 . 11)) 
((27 . 6) (29 . 10) #<two-way stream 01200da0> (23 . 1) (14 . 4)
 #<two-way stream 01200c20> #<two-way stream 01200bc0>
 #<two-way stream 01200b60> (26 . 4) (13 . 11)) 
((27 . 6) (29 . 10) #<two-way stream 01200da0> (23 . 1) (14 . 4)
 #<two-way stream 01200c20> #<two-way stream 01200bc0>
 #<two-way stream 01200b60> (26 . 4) (13 . 11)) 
((27 . 6) (29 . 10) #<two-way stream 01200da0> (23 . 1) (14 . 4)
 #<two-way stream 01200c20> #<two-way stream 01200bc0>
 #<two-way stream 01200b60> (26 . 4) (13 . 11)) 
((27 . 6) (29 . 10) #<two-way stream 01200da0> (23 . 1) (14 . 4)
 #<two-way stream 01200c20> #<two-way stream 01200bc0>
 #<two-way stream 01200b60> (26 . 4) (13 . 11)) 
((27 . 6) (29 . 10) #<two-way stream 01200da0> (23 . 1) (14 . 4)
 #<two-way stream 01200c20> #<two-way stream 01200bc0> (28 . 4)
 (26 . 4) (13 . 11)) 
((27 . 6) (29 . 10) #<two-way stream 01200da0> (23 . 1) (14 . 4)
 #<two-way stream 01200c20> (24 . 13) (28 . 4) (26 . 4) (13 . 11)) 
((27 . 6) (29 . 10) #<two-way stream 01200da0> (23 . 1) (14 . 4)
 #<two-way stream 01200c20> (24 . 13) (28 . 4) (26 . 4) (13 . 11)) 
((27 . 6) (29 . 10) #<two-way stream 01200da0> (23 . 1) (14 . 4)
 #<two-way stream 01200c20> (24 . 13) (28 . 4) (26 . 4) (13 . 11)) 
((27 . 6) (29 . 10) #<two-way stream 01200da0> (23 . 1) (14 . 4)
 #<two-way stream 01200c20> (24 . 13) (28 . 4) (26 . 4) (13 . 11)) 
((27 . 6) (29 . 10) (16 . 3) (23 . 1) (14 . 4)
 #<two-way stream 01200c20> (24 . 13) (28 . 4) (26 . 4) (13 . 11)) 
((27 . 6) (29 . 10) (16 . 3) (23 . 1) (14 . 4)
 #<two-way stream 01200c20> (24 . 13) (28 . 4) (26 . 4) (13 . 11)) 
((27 . 6) (29 . 10) (16 . 3) (23 . 1) (14 . 4) (26 . 15) (24 . 13)
 (28 . 4) (26 . 4) (13 . 11))

>=============================================================================

> -------------------------------------------------------------------------------
> 
> 
> 
> % g
> GCL (GNU Common Lisp)  2.7.0 ANSI    Oct 24 2005 10:28:12
> Source License: LGPL(gcl,gmp), GPL(unexec,bfd)
> Binary License:  GPL due to GPL'ed components: (BFD UNEXEC)
> Modifications of this banner must retain notice of a compatible license
> Dedicated to the memory of W. Schelter
> 
> Use (help) to get some basic information on how to use GCL.
> 
> >(defvar *waits* (with-open-file 
>                (s "/dev/random" :element-type '(unsigned-byte 32))
>                (mapcar (lambda (x) (read-byte s)) (make-list 10))))
> (defun complicated-function (x) 
>   (let ((y (nth (mod (si::getpid) 10) *waits*)))
>     (sleep (mod y 30))
>     (cons x (mod y x))))
> (defun one-result (s) 
>   (format s "~s~%" (complicated-function (read s nil 10)))
>   (bye))
> (defun many-results (s) 
>   (let ((x (read s nil 'eof)))
>     (unless (eq x 'eof)
>       (format s "~s~%" (complicated-function x))
>       (many-results s))
>     (bye)))
> (defun watch (x) 
>   (cond ((some 'streamp x) 
>        (print x)
>        (sleep 1)
>        (watch (mapcar (lambda (x) (if (and (streamp x) (listen x)) (read x) 
> x)) x)))
>       (x)))
> 
> 
> 
> *WAITS*
> 
> >
> COMPLICATED-FUNCTION
> 
> >
> ONE-RESULT
> 
> >
> MANY-RESULTS
> 
> >
> WATCH
> 
> >(setq l1 (si::socket 1927 :server #'one-result :daemon t))
> 
> 30272
> 
> >(setq x (mapcar (lambda (x) (si::socket 1927 :host "localhost")) (make-list 
> >10)))
> couldn't open socket:% 
> 
> 
> 

-- 
Camm Maguire                                            address@hidden
==========================================================================
"The earth is but one country, and mankind its citizens."  --  Baha'u'llah




reply via email to

[Prev in Thread] Current Thread [Next in Thread]