[p-dev] Resources and higher-order calls

[Paul Bone]

On Fri, Sep 01, 2017 at 12:09:12AM +1000, Paul Bone wrote:
> 
> I think some of these questions would be easier to answer if we actually had
> some experience writing Plasma and Wybe programs.  Particularly since Plasma
> (I don't remember for Wybe) includes loops.  So people are more-likely to
> use a loop rather than map to print out a list of items.
> 
> > I'd be happy to get together to brainstorm this some time if you like. 
> > It's an interesting question, and could possibly make an interesting
> > paper (maybe for PADL?).
> 
> Yep, See you tomorrow :D

Peter and I met today and we spent about 3 hours trying to figure out
exactly how much common ground there is between Plasma and Wybe before we
tackled higher order resources.  In talking through this stuff we discussed
many future ideas, like linking resources to values (like file descriptors),
or whether statements should be permitted in the condition of an
if-then-else and whether they behave like semidet code (or the Maybe monad
if you like) in that context.  There's a very wide and deep range of things
that Peter and I _almost_ but not quite agree about, or perhaps do agree but
are using different language to describe.

We each gave our thoughts about higher order resources.  But we didn't get
to anything like a conclusion.

We both want to avoid people needing to write their higher order code more
than once.  So the resource-aware version needs to work with
non-resource-using arguments.  So we both agree and that's easy to do.

Implicit resources
------------------

Peter feels that developers that don't imagine their code being used with
resources simply wont write the resource declarations or ! on the calls.  I
agree.  Peter continues by saying that therefore the resource unaware code
should work with resources anyway, but the ordering of how the resource is
used is either undefined or defined in a possibly not-obvious way.  So
developers wouldn't have to use ! or say "uses r" and yet their code would
handle resources implicitly.  I'm leaning towards "no".

Evaluation order
----------------

Another idea is that just as statements have the declarative semantics of
being executed one at a time, top to bottom, but that this is not
necessarily their operational semantics, that expressions could have a
similar declarative semantics.  And therefore you could have multiple uses
of a single resource in a single statement, it'd be difficult to read and
maybe your linter will give you a cleanliness warning, but it would be well
defined.  Therefore:

    func map(f : a -> b uses r, l : List(a)) -> List(b) uses r {
        return switch (l) {
            case [] -> []
            case [x | xs] -> [f!(x) | map!(f, xs)]
        }
    }

Is valid.

Force developers to deal with resources explicitly
--------------------------------------------------

In combination with this idea, we could also require that every higher order
value has a "uses" or "observes" clause attached.  This defeats the YAGNI
problem you mentioned
(https://en.wikipedia.org/wiki/You_aren%27t_gonna_need_it taken to its
logical conclusion people will never put in resource declarations until
needed, but that need might arrise much later when you're using a 3rd party
library).  However it may have the problem of turning people away from
higher order code, or Plasma/Wybe.

We could take this idea further and suggest that a higher order value must
always have a "uses", "observes" or "pure" annotation.  So the developer
is always forced to think about it.  Then if they wish to omit the !
symbols, they can say:

    func map(f : a -> b pure, l : List(a)) -> List(b) {
        return switch (l) {
            case [] -> []
            case [x | xs] -> [f(x) | map(f, xs)]
        }
    }

The difference is it's not that they "forgot" to add a polymorphic resource,
this changes it to a choice.  So it still suffers from the YAGNI problem,
people will just say "pure" all the time, but a lot more code will be
succinct.

Placement of !
--------------

There are two other ideas I discussed Peter and others recently.

The ! referring to an affect could be written before the statement as a
whole, out-dented from the current block:

    x = calc_something()
  ! do_something(x)
    y = calc_something_else()
  ! ok = do_something_with_result(x, y)

But in our map example this is far away from the actual effects.

    func map(f : a -> b uses r, l : List(a)) -> List(b) uses r {
      ! return switch (l) {
            case [] -> []
            case [x | xs] -> [f(x) | map(f, xs)]
        }
    }

There's only one per statement and it still means "there's something to be
aware of here".

Placement of uses/observes
--------------------------

The other idea is that the uses/observes clause should be moved closer to
the 'arrow'.

    func map(f: (a) uses r -> b, l : List(a)) uses r -> List(a)

It doesn't 'look' right yet, but I think this helps.  This also helps when a
function returns a function.

Higher-order and structures
---------------------------

One thing we didn't discuss is what happens when I higher order value is
stored into a data type.  What does that data type's signature look like.

    data Foo = Foo (f : Int -> Int uses R)

If it's polymorphic then you have to write the type arguments on the LHS:

    data Foo(a, b) = Foo (f : a -> b uses R)

But what if its resource polymorphic.

    data Foo(a, b) with r = Foo (f : a -> b uses r)

That is probably required to make something sound, but I'm not sure.  If we
adopt one of the ideas above, for example that all higher order calls
implicitly use a polymorphic resource (resource-agnostic code is implicitly
resource-safe).  That what implicit types happen when you put a
resource-agnostic higher order value in a data structure?  Or is it just
always pure in this context if you don't write a resource?

Higher order specialisation
---------------------------

Oh Peter, here's a reason why you won't be able to optimize away all the
higher order calls: people can put them into structures then map over them
applying them to some single data term:

    x = ...
    ys = map(\f -> f(x), functions)



I'm sorry this e-mail seemed a bit scattered.  In my defense that correctly
reflects our meeting today (there were many things to discuss so we switched
between them often).

Thanks.


-- 
Paul Bone
http://paul.bone.id.au