time for libio?

[Roland McGrath]

Someone reported some small bug in stdio again recently, and this reminded
me to poke Marcus again about when we can do the incompatible libc ABI
change to switch to libio.  

The current state of the libc code is good AFAIK, but hasn't gotten as much
testing as it should.  That is, you can presently compile libc with
--enable-libio and get a libc.so.0.3 that works fine last time anyone
tried.  If you compile hurd against that, it all works fine when last tried.

The only issues needing dealt with are binary compatibility and
packaging/upgrade questions.

The motivations for doing this sooner rather than later are a) why the hell
not already, b) no more GNU stdio bug reports to ignore, and the recent
addition of c) glibc 2.3 is in the offing before too long now, and wouldn't
it be grand if I could nuke the old stdio source code entirely and have
2.2.n be the final version that can produce libc.so.0.2 at all?

To be clear, what we have now is a glibc-2.2.x source tree that can build
either the current libc.so.0.2 with GNU stdio or build libc.so.0.3 with
libio.  I don't expect to change that much at all, so we can continue to
build libc.so.0.2 binaries from the stable 2.2.x branch as long as we want
to.  If we are deploying libc.so.0.3 soon, we can also build that from the
stable 2.2.x branch.  In the not too distant future, the 2.3.x branch will
only be able to build libc.so.0.3; but for the near to medium-term future,
that branch will be substantially unstable anyway and not what we want to
deploy in Debian GNU/Hurd.

> On Thu, Nov 29, 2001 at 04:32:41PM -0500, Roland McGrath wrote:
> > Can we do the libio switch soon?
> 
> I have never tried libio, but Jeff has.  As Jeff is also the one doing
> binary packages now, he has more to say in the matter than me.
> 
> The binary incompatibility problem is well understood by you and me at
> least.  I am not sure I still have the mails related to this, though (lost a
> bunch of mail folders recently), but I can get Jeff up to date on this any
> time.  I guess I am in favor of updating bunches of related packages at a
> time.  Initially we might break a slew of normal applications, but if we can
> keep the base and development system working at all times, we will have no
> serious problems, I guess.

I have lots of details about the plain binary compatibility issues with
libc, the dynamic linker, other libraries, hurd libraries, etc.  Complete
compatibility is doable from that perspective, depending how much you want
to bother with.  But there is much hair to keep in mind.

The last discussion I recall with Marcus concluded that basically all the
libc.so.0.2 -> libc.so.0.3 binary compatibility questions in the code
itself would be basically moot because it's much more trouble than it's
worth in the Debian packaging world to make compatibility packages and old
and new packages all fit together.  

So if we are really happy with a giant flag day where libc0.3 conflicts
with libc0.2 and there is no libc0.2 compatibility package you can get, so
everybody has to upgrade every single package they have when they want the
new libc and hurd packages, then what's stopping us?

Can we have an autobuilder that collects and rebuilds all the existing
packages based on the new libc environment, and doesn't upload any of them
to debian until they are all built?  Then maybe use a temporary public
location that people can do giant apt-get dist-upgrade's from if they are
brave enough to test the new world.  If that all goes well, we just upload
them en masse.

If this is the plan, the only kind of binary compatibility I am concerned
about is from flag day to the future.  I still think my plan of the new
libc.so.0.3 ABI as used by single-threaded programs being supported by a
future libc.so.0.3 that works with a future pthreads library could be
viable.  But that can only be plausible with someone with much libc
know-how seriously working on the question before and through the first
deployment of libc.so.0.3 to make sure that the ABI dependencies resulting
for newly-built packages are things that could be met by the future libc.
So realistically that means either me getting a test machine and spending a
good bit of time on it, or Mark spending a good bit of time on it.
Neither seems too likely in the imminent future.

Anyway, it certainly seems more than possible that when a new pthreads
reality does eventually happen, there will be sufficient internal
reorganization of the libc hurd code that some ABIs would get broken that
aren't easy to predict now.  So perhaps we are just in for another big flag
day in the future too.