Re: [GNUnet-developers] Proposal: Make GNUnet Great Again?

[ng0]

Schanzenbach, Martin transcribed 9.7K bytes:
> 
> 
> > On 10. Feb 2019, at 13:56, Christian Grothoff <address@hidden> wrote:
> > 
> > On 2/10/19 11:59 AM, Schanzenbach, Martin wrote:
> >>> --disable-FEATURE flats for configure where then src/Makefile.am simply
> >>> doesn't enter certain subdirectories would certainly have my approval 
> >>> here.
> >> 
> >> See above, yay more flags. But this is actually a problem I have not 
> >> thought through tbh. I do not know how GNUnet is supposed to "work" in 
> >> this regard.
> >> Example:
> >> I release reclaim. It is supposed to do X.
> >> How am I supposed to release this? As a plugin for GNUnet? As _part_ of 
> >> GNUnet (i.e. I need to tell my users to install this completely unrelated 
> >> beast which includes a social network and peer-to-peer file sharing)?
> > 
> > I like this way of looking at it, chiefly what is the release plan. It
> > depends a bit on whether the reclaim components are from a separarate
> > repository or not. I would follow the general plan we have for GNU Taler:
> > 
> > * The major version (i.e. 0.11 for GNUnet) should indicate API and
> >  protocol compatibility. So reclaim (or components) would say they
> >  need GNUnet 0.11.  This would be what you would consider the 'stable'
> >  basis.  CI tests should run against it, as well as 'master'.
> 
> Yes.
> 
> > * Other repositories (gnunet-gtk, reclaim components, secushare UIs,
> >  gnunet-fuse) can then make *independent* point releases, but they
> >  should be numbered based on the GNUnet major version they require,
> >  i.e. gnunet-gtk 0.11.1, gnunet-gtk 0.11.2, etc.
> 
> 
> Great.
> 
> > * For updates to parts that are part of the GNUnet "core" (say
> >  API&protocol compatible improvements to transport/cadet/fs), we would
> >  similarly make gnunet 0.11.{1,2,3}, but this should "guarantee" that
> >  say gnunet-gtk 0.11.1 works with gnunet (core) 0.11.3.
> > 
> > Which brings up a good point: if for some application (say reclaimID)
> > the release schedule is completely different (usually: way more
> > frequent), it might make sense to split it off --- or increase the
> > 'core' release frequency.
> > 
> > Note that I am talking about *source* releases here, as developers
> > should IMO release *source*, not binaries or VM images.
> > 
> 
> Yes. I agreed.
> 
> 
> >> I mean, currently I provide an out-of-the-box client with a docker image 
> >> that runs GNUnet in a configuration that is most sensible for reclaim.
> > 
> > No, that's horrible. The main release should be a source release,
> > packaging into particular containers, VMs or operating systems should
> > never be the _primary_ release mechanism. That's almost worse than the
> > idea of us primarily offering DEBs for download --- and thus effectively
> > not supporting RPM or Guix. Sure, providing binary packages or images
> > *in addition* is OK, but IMO not as the primary release mechanism.
> 
> Wat.
> No, you are completely missing the point I am making. I am saying that I 
> cannot conceivably provide adequate installation instructions for _one_ 
> service that is built on GNUnet.
> 
> Hence, for our PoC (reclaim client), I took the easy route (for the user): A 
> docker image pre-built with what is needed.
> I fully agree that this way of doing it is completely incompatible with other 
> GNUnet installations and is not the way it should be done. But currently, if 
> I want users to try it out, it is the _only_ reasonable way.
> The reason why it is the only reasonable way is the bloatiness and complexity 
> and missing packages for gnunet that provide a minimum viable system.
> 
> > 
> >> But the architecture of gnunet actually calls more for a GNUnet "app" 
> >> store for a platform (GNUnet core,transport,gns,cadet et al) with a bunch 
> >> of apps (fs, social, reclaim).
> >> Now, ideally I would work with packages in a reclaim installer, i.e.
> >> 
> >> - check if gnunet-core is installed, if not install
> >> - check if gnunet-reclaim is installed, if not install
> >> - profit.
> > 
> > Well, Guix, RPM and DEB all offer this very easily, you just specify the
> > dependency. Now, how to build a single gnunet-reclaim package from your
> > 5-6 separate Git repositories and (presumably) source TGZ releases is
> > beyond me, but I'm not the one who atomized the reclaim sources ;-).
> 
> You are mixing things up again. gnunet-reclaim would only be the reclaim 
> service that currently also is in gnunet.git.
> And _no_ it is not "very easily" offered by a package maintainer, because the 
> whole build is a bloody mess of a billion binaries from file-sharing to chat 
> systems.
> 
> 
> > 
> >> I mean, what exactly do you expect a package maintainer to do? Have 
> >> separate packages for fs/social/reclaim? One huge package that depends on 
> >> everything?
> > 
> > I expect *good* package maintainers to split up our monolith into
> > separate packages with proper dependencies (which we might document
> > better for them, but that's another story).  More importantly, there
> > wouldn't be simply gnunet-core, but more like:
> > 
> > gnunet-core requires gnunet-db
> >  gnunet-db is provided by
> >    gnunet-mysql
> >    gnunet-sqlite
> >    gnunet-postgres
> > gnunet-fs requires gnunet-core
> > gnunet-fs-gtk requires gnunet-fs and gnunet-gtk-core
> > gnunet-reclaim requires gnunet-core
> > gnunet-secushare might require gnunet-experimental (multicast, psyc) OR
> >        that might be part of gnunet-core
> > gnunet-namestore-gtk requires gnunet-gtk-core
> > ...
> 
> 
> Hahah you expect package maintainers to do that from what results from a 
> "make" in gnunet.git? Are we living on the same planet?
> 
> > 
> > Furthermore, I expect that there will be distributions that do not break
> > it up so much, maybe even to the point of shipping one big monolith
> > which drags in 1 GB of dependencies. But that's the *distributions*
> > choice, how many packages they want, what audience they target (our
> > users have huge disks and do not want 100000+ packages vs. our
> > distribution targets embedded systems and we try to atomize vs. our
> > distribution separates resource files that are architecture-independent
> > into a -noarch dependency) --- there are so many trade-offs here, I
> > would not even want to be involved in that discussion.
> > 
> > And yes, IMO in extreme cases some distribution might ship every app as
> > a Docker image (openoffice-docker, gnome-docker, firefox-docker,
> > emacs-docker, reclaim-docker).  IIRC QubesOS is going in that direction ;-)
> 
> No. I do not think any maintainer would agree. You do not give the maintainer 
> this monster and say "there you go, have fun breaking it down if you need to".
> You provide proper, functionally separated units to package. Ideally source 
> separated for automated builds.
> (see ng0's comments)

on top of this, a practical example:
packaging is a hard, unthankful craft. You can document as much as
you want, people will still manage to get it wrong because of whatever
factors in their life or QA in general.
Before I started here, I had to piece together the actual dependencies
for gnunet, bit by bit, for an existing gentoo ebuild someone had
given up on and lynX had adopted. Same for Guix later on, and i think
the operating systems which haven't given up on gnunet or put it on
hold in a 'wip' repository will still have their fair share of
mistakes in there.
This is all due to the fact that gnunet is simply big. and for
packagers that's okay. look at the 3 years it took from a simple
wip package I posted to guix-devel for the inox webbrowser and
now people have created a chromium variant out of it, swapped out
all the code.
As written in my previous email, you can only hand out tools to break
it down and recommendations which will be understood as rules to follow.
The rest is outside of your control if you don't speak to the people,
which we often don't do. I try to approach people when I see mistakes,
even when it's a badly written Dockerfile for gnunet someone made
for gnunet. The recommendations have to be understood, the way it
has to be split up has to be possible in some documented, programmatic
way and that's it.
 
> 
> > 
> >> If yes, then how would the build look like? Do they have to "fully" build 
> >> gnunet 3 times and cherry pick binaries? Do they have separate the build 
> >> by hand because we build everything or nothing?
> > 
> > It *should* suffice to build "everything" once, and then cherry pick
> > binaries and/or resource files, simply because we have linked things
> > _properly_ and use plugins where appropriate, so you can separate
> > binaries related to sqlite from those related to postgres easily, while
> > building both at the same time. The same holds for virtually everything
> > else, including internal dependencies. So yes, distributors should just
> > turn on everything, build everything once, but then have to cherry pick
> > binaries by hand. But while we can provide guidance on the cherry
> > picking, I don't think we can do away with it at the exact grouping
> > depends on the distro's philosophy.
> 
> 
> See above. This is "F** Magic", not "Actual Machine" thinking; 
> https://fritzfreiheit.com/wiki/AM/FM
> There are only two ways this ends:
> 
> 1. gentoo-like source-based distros: provide installation-time flags (for 
> which gnunet is actually ok to install then)
> 2. DEB-like distros that will provide "gnunet" packages that pull: Gtk+ (if 
> we include it in gnunet.git), postgres, mysql, sqlite, libgabe, ... (you get 
> it)
> 
> The reason why (2) will happen is because people don't have time for this 
> shit. Some goes for 1, btw. It is just coincidentally a good approach.
> 
> 
> > 
> > 
> 



> _______________________________________________
> GNUnet-developers mailing list
> address@hidden
> https://lists.gnu.org/mailman/listinfo/gnunet-developers