[Octave-patch-tracker] [patch #9980] JSON encoder and decoder, alternative to object2json

[anonymous]

Follow-up Comment #3, patch #9980 (project octave):

On input validation, Ok.  But "toJSON.m" is intended to take any and ALL
objects and convert to JSON string, so I'm not sure what you have in minded
there.  For "fromJSON.m", it works only on string, so that is no prob!   As
for malformed JSON string, not sure how to valid other than just parsing the
string.

Also, I don't know what you mean "yet input validation BIST tests are
lacking"...as compared to what is already there.  The testing includes all the
different Octave object classes I know off. More elaboration would be
instructive.

On Octave version support, I don't know how to test for that, other than
loading all versions on my computer.  The first iteration was develop on 4 but
refactored on 5.  In refactoring, the only new feature added was default
function args, so I'm 99.9999% sure it will still work on 4.  It uses
strcut2cell, str2func, is_function_handle, cell2mat, regexp, regexprep and
subsref/subsasgn, among other more mundane functions.

On JSON expertise, I don't know any.  I usually consult a web-browser's
developer console (see JSON.parse).

On 'jsonlab', as well as the 'jsonencode' and 'jsondecode', I think 'toJSON'
and 'fromJSON' still has some advantages.   I have very limited (<2hrs)
experience with these so I cannot say expertly, but:

jsonlab seem okay, but it doesn't seem to be made to work well with other
languages.   It seems to be made primarly for data storage, not data
transmission.  Besides the fact that it's loaded with filesaving and
compression (requiring external java library), it relies a lot on metadata. 
For example, with 2D matrix, 'savejson.m' converts to a conventional JSON
string, readily parsed to identical form in Javescript, python, etc., but not
for ND arrays. For ND array, 'savejson' flattens the data into 1D array and
includes metadata on array shape, variable name and class.  While still JSON
compliant, it is not a conventional format for arrays, and require additional
parsing on receiving end (_i.e._ non-trivial for Javascript, the JS in json). 
 

Also, setting aside the fact that I was unaware of the forthcoming
'jsonencode/decode', I also want to solve the problem of output data size and
transmitting Inf and NaN, which I think neither 'jsonlab' nor 'jsonencode'
really address properly.

First, 'toJSON' allows floating numbers to be truncated (precision), for more
compact data transmission for large datasets.  For complex structured data,
this is non-trivial either before or after encoding.

Second, I did not like their ('jsonlab', 'jsonencode', Firefox, Chrome,
python, etc.) handling of Inf or NaN.   Either they turns *both* into 'null'
(making them indistinguishable), or replace it with quoted-strings
'"Infinity"' or '"NaN"'.  This require additional (computationally expensive
for Javascript) parsing on receiving end.

The problem stems from the fact that JSON spec pretends to be IEEE754
complaint but doesn't actually make allowance for Infinity or NaN.  So, those
numbers get trashed.  Instead 'toJSON' turns Inf into '1e999'.  While not a
valid IEEE754 number, it *automatically* turns itself into Infinity when
parsed by Javascript, Python, Ruby, Octave, Matlab, etc.  No fuss. No muss. 
Nothing can be done about NaN, but with Infinity automatically parse, dealing
with NaN is easier. 

Likewise, 'fromJSON' cheats a little with regards to receiving Inf, it treats
any floating number >= 1e308 as Inf.  Natively, Octave, JS, python, Ruby, etc.
treat >= 2e308 as Inf, but by nudging this threshold down a tad, there's less
work for transmitting end (JavaScript) to encode Inf, and it allows 'fromJSON'
to very easily differentiate Inf from NaN.

Finally, 'toJSON' and 'fromJSON' handles complex number, which is slated to be
*unsupported* in 'jsonencode' and 'jsondecode' (per MATLAB).  'jsonlab' uses
more metadata.  Complex number are not native in Javascript nor supported by
JSON, so structured object are the norm.  'jsonlab' does not follow this norm.
 'toJSON', on the hand, handles complex number in a manner consistent with
that norm. 'object2json' does similarly support complex numbers, however,
where as it makes an array of structure object, 'toJSON' makes (for speed and
output compactness) a structure object of arrays.  It's a design choice that
super easy to change (add 2 lines, thanks to judicious use of recursion). 
Additionally, 'object2json' is significantly (very!) slower, wrecks ND arrays,
and there is no 'json2object'.  In contrast, 'fromJSON' reads in that
structure object and converts it back into a Octave complex number. It works
with either struct of array (small,fast) or array of struct (big,slow...for
JSON).

When I make the requested change, do I upload it here again?  how do I name or
version number the new files?


    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/patch/?9980>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/