handling of large constants

[Jean-Marc Saffroy]

Hi,

I just noticed a change between 6.7 and 6.8 in how gdb handles the 
implicit type of some large constants. I have the following script:

$ cat gdb-regtest-1
set $foo = (1 << 46)-1
p/x $foo

file  ~/prog/c/misc/sizeof
p/x $foo

set $foo = (1 << 46)-1
p/x $foo

"sizeof" is a dummy program which does nothing fancy. It is however worth 
noting that it's a native x86-64 binary, ie. the platform I'm running on.

So if I feed gdb-6.7 with the script above, I see this:

$ ./gdb-6.7/gdb/gdb -batch -x gdb-regtest-1
$1 = 0xffffffff
Using host libthread_db library "/lib/libthread_db.so.1".
$2 = 0xffffffff
$3 = 0x3fffffffffff

So setting $foo before and after loading the binary gives different 
results!

Now with gdb-6.8:

$ ./gdb-6.8/gdb/gdb -batch -x gdb-regtest-1
$1 = 0xffffffff
$2 = 0xffffffff
$3 = 0xffffffff

At least this looks consistent, except that it's not the value I want for 
my constant $foo.

Now I try again my test, this time with a cast into a long:

$ cat gdb-regtest-2
set $foo = ((long)1 << 46)-1
p/x $foo

file  ~/prog/c/misc/sizeof
p/x $foo

set $foo = ((long)1 << 46)-1
p/x $foo

Then gdb-6.8 shows inconsistent results again:

$ ./gdb-6.8/gdb/gdb -batch -x gdb-regtest-2
$1 = 0xffffffff
$2 = 0xffffffff
$3 = 0x3fffffffffff

This looks definitely wrong to me.


--
address@hidden