Re: ABI Compatibility (was Re: Installation woes for the average user...)

[Xavier Glattard]

David Chisnall a écrit :
> On 10 Mar 2009, at 14:27, Xavier Glattard wrote:
>
> > David Chisnall a écrit :
> > > The extra bytes are allocated after the object, which would be 
> > > completely useless.  Imagine:
> > (...)
> > > Now you add another ivar to A and remove the extra bytes, and you 
> > > have this layout:
> > >    id isa;    // offset 0
> > >    int a;    // offset 4
> > >    int a1;    // offset 8
> > >    int b;    // offset 12 <- This has now moved and class B needs 
> > > recompiling, as do any subclasses of B.
> >
> > Yes, but this is the case we want to avoid with the use of extra bytes!
> > --> " do not insert ivar, add it in extra bytes "
>
> Right.
>
> > > Alternatively, you could try storing the data in the extra bytes, so 
> > > the layout would be:
> > (...)
> > > We now have three different classes, with offsets of 8, 12 and 24, 
> > > respectively for the a1 ivar.  Every method that attempted to access 
> > > this ivar would need to determine which class it is and calculate the 
> > > offset.  This would need a macro like this:
> > > #define a1 (*((int*)(((char*)self) +isa->instance_size)))
> > > This would be really horrible and error-prone (this macro doesn't 
> > > take into account alignment, so is only valid for architectures like 
> > > x86, with no strict alignment requirements (as long as none of the 
> > > ivars are vectors), and would break on SPARC and similar archs.  The 
> > > next ivar you add would need an even more complex macro to account 
> > > for the alignment of a1.  This ivar would not be exposed via any 
> > > introspection, so you couldn't use it with KVC/KVO, bindings, or 
> > > EtoileUI without writing even more code.
> > > In short, this adds a lot more problems than it solves.  The extra 
> > > storage is not at self+1 unless self is cast to the correct subclass 
> > > first.  Pointer arithmetic like that only works when the size of the 
> > > pointee is known at compile time, which is not the case for 
> > > Objective-C objects, except in the trivial case of classes with no 
> > > subclasses, which do not encounter this problem to start with.
> > > David
> >
> > The extra bytes are already used in some class in GNUstep (GSString, 
> > GSArray...) and AKAIK there is no problem with system arch.
>
> GSArray is a private GNUstep class.  It is not exposed in headers 
> anywhere and can not be subclassed by anything external to GNUstep.
>
> > In GSArray.m:417 :
> >
> >  _content_array = (id*)&self[1];
> >
> > So my 'self+1' was not so naive ;)
>
> See above.  This only works in the special case where the class has no 
> subclasses.  Create a subclass of GSArray, add an instance variable, and 
> watch everything go badly wrong.
>
> This discussion is only relevant to classes which will be subclassed in 
> third-party apps and frameworks.  Private classes are not part of the 
> public ABI and so are completely irrelevant.  You can do whatever you 
> want with a class that won't be subclassed, or for which you control all 
> of the subclasses.  You can't do any of these tricks with a class 
> intended for subclassing, and these classes are the topic of the 
> discussion.
>
> > I have no idea of the macro that would be written for this task. Yours 
> > is very ugly ;) But it'd have to be writen only once if the extra 
> > bytes are defined as a structure. And then it would not be so error 
> > prone.
>
> Then the structure becomes part of the ABI and we're back where we 
> started, only without introspection and reimplementing parts of the 
> language in an inefficient and error-prone way.
>
> > // In NSObject
> > #define OBJECT_EXTRA_PTR ((void*)(((char*)self) +isa->instance_size))
> >
> > // In FooClass
> > struct _foo_extra_st
> > {
> >  int a1;
> > } _foo_extra;
> >
> > #define FOO_EXTRA (*(_foo_extra*)(OBJECT_EXTRA_PTR))
> >
> > - (int) a1
> > {
> >  return FOO_EXTRA.a1;
> > }
>
> Right, and now try having two classes using this mechanism.  The 
> subclass will need to know the offset from the parent.
>
> It quickly becomes horribly unmaintainable, it adds an extra layer of 
> indirection, and provides no real benefits.  It also breaks any classes 
> that use the extra bytes mechanism themselves.  And, as I said, it also 
> breaks introspection, which will make all sorts of things that use the 
> more advanced features of Objective-C fail silently and in a manner that 
> is almost impossible to debug.
>
> > You may find two other examples in GSString.m: 633, 3087
>
> GSString is another private class.  It is completely irrelevant.
>
> > Anyway that's only an idea, not a solution.
>
> Correct.
>
> > I'm convinced that the solution would be more 'political' than technical.
>
> Totally agree.  In many cases, avoiding changing the ABI can be avoided 
> or postponed, but there needs to be an attitude that it is something 
> that we want to avoid, rather than something to do because it's easy.  
> Until we have non-fragile ivars in the runtime, anyway, at which point 
> the whole discussion becomes academic.
>
> David

I'm sorry, i still can't see any problem.

NSAllocateObject is implemented as this : (NSObject.m:767)

  size = aClass->instance_size + extraBytes + sizeof(struct obj_layout);
  new = NSZoneMalloc(zone, size);

Then the extra bytes are allocated always _after_ the class ivar, 
whatever the class is : the parent class or a subclass. If you get 
isa->instance_size (as *you* did) you find the extra bytes.

---- parent class ayout:
{
  { // the instance ivar
    int a;
  }
  { // the extra ivar
    int a_ext;
  }
}

---- subclass layout:
{
  { // the instance ivars
    int a;
    int b;
  }
  { // the extra ivars
    int a_ext;
  }
}

This is the same behavior than 'classic' external ivars, but the memory 
is allocated along with the instance itself (remember: the question is 
'no extra malloc call'). The pointer to external ivars is not stored as 
is, but is computed with self and instance_size. Still an 'extra load', 
but not more than with a compiler-side solution. I even think this is a 
'hand made' non-fragile ivar system, with no need for compiler support.
And the alignment issue seems to be solved with padding in 
NSObject.m:334-371

> It quickly becomes horribly unmaintainable, it adds an extra layer of
> indirection, and provides no real benefits.  It also breaks any classes
> that use the extra bytes mechanism themselves.  And, as I said, it also
> breaks introspection, which will make all sorts of things that use the
> more advanced features of Objective-C fail silently and in a manner that
> is almost impossible to debug.

I dont agree with some of these points, but don't forget the class that 
would be modified are GNUstep ones, and these ivars are private. Only 
GNUstep developpers would have to debug them, and they would know where 
and how to find them, as they do today.
And if we actually need introspection or other features, we only have to 
add them to the runtime. KVC on external ivars would be quite easy to 
implement. And 'Standard' external ivars would always be better than 
today 'wild' external ivars ;)

Finally I dont think all this is 'academic'.
Someone (you?) said : external ivars are bad because of the extra malloc 
call, then I try to find an external ivar system with no extra malloc. 
If I'm right then external ivars are still an option.

IMHO the question is : does a future GNUstep release need a future 
compiler release ? Here is the political problem.
I think it should not. So I try to find arguments against it :)



I dont know who wrote it, but accordingly to the Wikipedia article 
'Fragile binary interface problem', there is no problem with 
Objective-C. Then I wonder why we are looking for solution ? ;o)

-- Xavier