Re: Backward compatibility of next beta

[Derek Atkins]

Hi Frank,

Frank Crawford <address@hidden> writes:

> Okay, to get into the technical details, as I tracked them down, and
> there may be other issues over and above these, but much of the data
> between remote servers is passed in pickle format.
>
> One of the options in a pickle is to not just pass the data but the
> object, or more specifically the data and the function call to process
> that data.  These functions are essentially assumed to be the same
> function on both sides.  These functions are essentially looked up in
> the python symbol table.
>
> Unfortunately in Python2 the functions in the symbol table are in
> ascii, while in Python3 they are in unicode, and so the pickle data
> doesn't match on the lookup in the symbol table, and hence most objects
> passed between the two return a "function not found" error.
>
> Without somehow modifying the symbol table (which would probably break
> other things) you can't get around it, and you would need to modify it
> on both sides scripts.
>
> There are a number of other items like just the strings transition from
> ascii to unicode for the actual data needs to be fixed, on to new
> pickle formats available in Python3, that would be a headache, but they
> could be addressed.
>
> Unfortunately they are minor compared to the symbol table issue.

According to the docs I just found on class pickle.Pickler[0], there is
a fix_imports and Protocol version number in there that lets you set the
compatibility level:

    The optional protocol argument, an integer, tells the pickler to use
    the given protocol; supported protocols are 0 to
    HIGHEST_PROTOCOL. If not specified, the default is
    DEFAULT_PROTOCOL. If a negative number is specified,
    HIGHEST_PROTOCOL is selected.

    If fix_imports is true and protocol is less than 3, pickle will try
    to map the new Python 3 names to the old module names used in Python
    2, so that the pickle data stream is readable with Python 2.

SO..  It SHOULD be possible to create a pickle in Py3 that is readable
by Py2.  Have we tried this?

-derek

[0] https://docs.python.org/3/library/pickle.html#pickle.Pickler

-- 
       Derek Atkins                 617-623-3745
       address@hidden             www.ihtfp.com
       Computer and Internet Security Consultant