[Octave-bug-tracker] [bug #58147] Matlab 2020 compatibility: dec2bin, dec2hex shouldn't error on negative inputs

[Nicholas Jankowski]

Follow-up Comment #8, bug #58147 (project octave):

that's close, and avoids the bitcmp issue, but need to +1 each one to get
two's complement.  


if (d < intmin ("int64"))
  error ("out of range");
elseif (d < intmin ("int32"))
  d += double (intmax ("uint64")) + 1 ;
elseif (d < intmin ("int16"))
  d += double (intmax ("uint32")) + 1;
elseif (d < intmin ("int8"))
  d += double (intmax ("uint16"))+ 1;
elseif (d < 0)
  d += double (intmax ("uint8")) + 1;
endif


That then works for all test cases but one. looking at the bit length
transitions:

>> dec2bin(-128)
ans = 10000000
>> dec2bin(-129)
ans = 1111111101111111
>> dec2bin(-2^15)
ans = 1000000000000000
>> dec2bin(-2^15-1)
ans = 11111111111111110111111111111111
>> dec2bin(-2^31)
ans = 10000000000000000000000000000000
>> dec2bin(-2^31-1)
ans = 1111111111111111111111111111111110000000000000000000000000000000
>> dec2bin(-2^63)
ans = 1000000000000000000000000000000000000000000000000000000000000000
>> dec2bin(-2^63-1)
ans = 1000000000000000000000000000000000000000000000000000000000000000


those all are correct except the (-2^31-1) case.


Matlab:
>> dec2bin(-2^31-1)
ans =
    '1111111111111111111111111111111101111111111111111111111111111111'

Octave:
>> dec2bin(-2^31-1)
ans = 1111111111111111111111111111111110000000000000000000000000000000
>>


so need to peek at what's happening there.

Also, need to adjust it to handling both arrays and cell arrays, which could
include a mix of pos and neg values, complicates things a bit. Apparently
Octave handling cell arrays in an extension beyond Matlab, which throws an
error if d is not numeric. would probably also throw an 'if d<0' around that
to avoid the checking cascade when not needed.

    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?58147>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/