Re: Automatic Poke

[Jose E. Marchesi]

Hi Philippe.

> I am starting a new project that will make use of libPoke. It is a tool
> that will try to automatically reverse engineer the structure of a
> collection of opaque blobs. The idea is that if you have enough blobs with
> the same structure, you can have the computer try grammar permutations that
> fit the blob collection. The tool will generate a portion of Poke grammar
> for you to modify and improve on.
>
> The workflow I have in mind is this:
> 1-Organize your blobs by grammar.
> 2-Run the tool, it will generate grammar. Maybe with different choices?
> 3-Edit the generated grammar
> 4-Repeat step 2
>
> Nice Features:
> - User provides hints of data you know is in the blob (from log files,
> visual inspection, ...)
> - Automatically detect embedded files with magic headers
> -...
>
> I hope to generate the grammars in the Poke DSL, and use the libPoke VM to
> interpret and apply the grammars to multiple blobs and score how they
> perform. How should I go about this? Is this even a good idea? Does it make
> sense?
>
> Thank you for your feedback!

Interesting domain.

Using libpoke you can evaluate any Poke code.  This means that you
should be able to, for example, easily permutate fields of different
types with different constraints in them and rely on poke's data
integrity checking to determine whether the guess worked.

Additionally, you could also achieve run-time permutation by using
unions and labels.

I find the scoring part and the goal-oriented process very interesting.
We could maybe write a pickle with useful utilities for that kind of
things (discover.pk?).

Do you have some particular example of the kind of data structures of
these blobs?