Re: propose deprecation of generalized-vector-*

[Daniel Llorens]

On Mar 1, 2013, at 03:42, Noah Lavine wrote:

> There should still be an operator that splits in other ways, but I agree that 
> we can shortcut that in many cases.

One think I like about the frame / cell split is that you know that the result 
will have the same frame. So I imagine an operator

(do-with-frame array-op (arg0 2 1) (arg1 ...) ...)

that transposes 2 1 to  0 1 before the array op, and when the array op is done 
you know that the output shape will be 2 1 ... axes of the output cell ...  
However you still can't transpose the frame back without knowing what the array 
op did to the cell. There're specific types of operations where this works 
because the frame would be gone (reductions, or any time the output rank is 0) 
or it would be unaffected (rank 1 to rank 1 operations, like 'shuffle this 
axis'), but not in general. That's one reason why the axes specification is ad 
hoc in those other languages. If array functions know their ranks, we could 
make this work automatically often enough, but sometimes you'd still have to 
say 'I don't know how to do that'.

> A slow array reduction is easy enough to add, but I'm guessing that's not 
> what we need. :-) Perhaps we should have some sort of (ice-9 array) module 
> where we put useful array functions.

That's a good idea. I have my own slow reduction and I imagine that the 
expedient vector->list / array->list and back is done way too often.

> think this comes down to a more fundamental difference - I still don't think 
> that functions should automatically map over arrays, and you do. If they did 
> automatically map, then I would agree with you about array-ref, because then 
> arrays wouldn't be fundamentally different types from the objects they 
> contained.

I actually agree here! I don't want regular scheme functions to have things 
done to them around their back, it would be another language. I can accept why 
you want array-ref to be strict. Indeed my approach tends to a confusion 
between a 2-array of 2-arrays and a 4-array. In guile-ploy you can see this in 
collapse-array ---if the verb doesn't provide an output shape, I make an 
assumption. I also banish 0-rank arrays.

The idea is to make special array-functions to get the array behavior. Like 
ufuncs in NumPy.

> I agree that the compiler should be better, but as one of the people working 
> on it, there are lots of things that it should do and presently doesn't. I 
> don't know when we'll get nice array optimizations.

Guile has gotten a lot better recently. I wish I had a clue about compilers 
though.

> library like that in Guile - but I think that this should be optional, and 
> that not *every* function should have rank information. This is because while 
> it is fairly natural for programs that involve a lot of array processing, I 
> don't think it is as natural for, for example, networking code or the web 
> server.

I totally agree with this.

> I really like the ply function that lets you connect functions with rank 
> information to functions without it.

My idea was to define +., -., etc. as array versions of +, -, and so on. Then 
when the library was mature enough, (+. ...) would be transformed into (ply +. 
...). However I've fallen into the vice of doing things on the spot, like (ply 
(make-verb + ...) ...). There's a lot of this in the test script.

> I definitely agree that we should provide enough primitive operators to write 
> fast array code in Guile. Your README says that certain functions can't be 
> implemented efficiently without access to the underlying array representation 
> - that should certainly change.

I've thought about what could be provided on the Scheme side. The problem is 
that if you give strides / lbnd / ubnd directly, you still need to create a 
bunch of lists for any slicing, although maybe not as many as with 
make-shared-array. This is worse for ops that are going to be used inside loops 
a lot, like array-ref. The recent scm_c_array_ref_1 functions help, but it's 
just a specific case. All array operations are in C, maybe that's why they are 
opaque? I mean

scheme@(guile-user)> ,optimize (vector-ref #(1 2 3) 0)
$1 = 1
scheme@(guile-user)> ,optimize (array-ref #(1 2 3) 0)
$2 = (array-ref '#(1 2 3) 0)

> However, I don't think that we should make every function have rank 
> information, when it's not really used in most areas of programming. I think 
> the library you've presented is a great compromise, because it lets you put 
> rank annotations on some functions, but not all.
> 
> What do you think?

Agreed, I'm sorry I wasn't clear in the other messages. This array extension 
should be a library. What I'd like from Guile is the facilities to allow it to 
be efficient.

Best regards,
        
        Daniel