This whole debate saddens me

Posted Dec 10, 2014 17:55 UTC (Wed) by nix (subscriber, #2304)
In reply to: This whole debate saddens me by cesarb
Parent article: The "Devuan" Debian fork

Of course it has. But, like the Linux kernel internal API, it's fluid and can change from one commit to the other.

To me that's a sign of developer laziness: they can't be bothered to design APIs right the first time, and they're making everyone else pay for that. (It *can* be done, though it is a lot harder than just guessing and constantly chopping and changing them.)

This whole debate saddens me

Posted Dec 10, 2014 18:41 UTC (Wed) by peter-b (subscriber, #66996) [Link]

>> Of course it has. But, like the Linux kernel internal API, it's fluid and can change from one commit to the other.

> To me that's a sign of developer laziness: they can't be bothered to design APIs right the first time, and they're making everyone else pay for that. (It *can* be done, though it is a lot harder than just guessing and constantly chopping and changing them.)

Designing good, future-proof APIs is extremely difficult, as you acknowledge. Maintaining APIs in a bug-compatible manner is much more difficult still. It's totally reasonable for developers to define a small, stable, external API, while maintaining flexibility within their software to modify internal implementation details. It enables improvements to the efficiency, reliability, or features of the implementation. In fact, this approach is evidence of a robust development process and a sustainable project.

By contrast, it is completely *unreasonable* to expect people to only ever design, write and release perfect, flawless, future-proof code, and accuse people who have realistic expectations of what is achievable of "laziness". It is, in fact, arrogant and insulting.

What's going on here is that the systemd developers defined some public APIs that they wanted to provide, and went ahead and implemented a system that provided those APIs. You're now nitpicking their implementation. systemd's public APIs are fully documented, with stability guarantees, and that makes the project a heck of a lot more professionally developed than 99.9% of FOSS software projects.

I invite you to: 1. "improve" systemd to match your expectations (or pay someone else to), 2. fork systemd and "fix" it to your satisfaction, or 3. produce a plumbing suite that solves the same key problems in a "better" way. Until then, perhaps you should consider *not* wantonly slinging dismissive insults around. They're not going to get you whatever it is that you think that you want.

This whole debate saddens me

Posted Dec 11, 2014 0:02 UTC (Thu) by mchapman (subscriber, #66589) [Link] (14 responses)

> To me that's a sign of developer laziness: they can't be bothered to design APIs right the first time, and they're making everyone else pay for that.

Did you completely miss the word "internal"?

These APIs are *not* public APIs. They are *not* used by other software. They are merely the internal function calls from one part of a binary to another. They aren't even visible to or accessible by end-users.

This whole debate saddens me

Posted Dec 11, 2014 0:41 UTC (Thu) by dlang (guest, #313) [Link] (13 responses)

but the systemd folks keep claiming that systemd isn't monolithic, that anyone is free to replace any component, they just need to implement the same functions.

but if the API of those functions is "internal" then it is monolithic in design (even if implemented in multiple binaries)

This whole debate saddens me

Posted Dec 11, 2014 0:49 UTC (Thu) by pizza (subscriber, #46) [Link]

> but if the API of those functions is "internal" then it is monolithic in design (even if implemented in multiple binaries)

The API *between* components is a published, stable DBUS (and/or command-line) interface.

The implementation details, ie the C functions, are internal, private. You know, not unlike Linux's published userspace API, vs the internal implementation details that are changing all the time?

This whole debate saddens me

Posted Dec 11, 2014 3:11 UTC (Thu) by mchapman (subscriber, #66589) [Link] (11 responses)

> but the systemd folks keep claiming that systemd isn't monolithic, that anyone is free to replace any component, they just need to implement the same functions.

Oh criminy. Let's take this from the top again.

You asked why the different systemd components are in the one tree. I said that one likely reason is so that they can share code. nix wondered why they didn't just use a shared library for that. I said that it might be because the shared code isn't part of the *public* interface of systemd, so there is no compelling reason to make it a dynamically-linkable library.

Got it yet? We're talking about code that is statically linked into multiple, separate, independently-reimplementable binaries. This has *absolutely nothing* to do with their public APIs (i.e. a set of D-Bus interfaces) provided by those binaries.

This whole debate saddens me

Posted Dec 11, 2014 17:22 UTC (Thu) by nix (subscriber, #2304) [Link] (10 responses)

Quite. I happen to think this is shoddy design, unless (like, say, gnulib) there are very good reasons for it (in gnulib's case, that a lot of the horrifically evil dark magic it does happens before compile time, at the configure stage, and that it's only trying to fix up holes in already-public APIs anyway). It was a bad idea when GNOME did it with its 'cut-n-paste-code' directories, it was a bad idea when people embedded dozens of copies of libpng and zlib, and it's a bad idea now. Yes, doing it right is more work. Doing it right often is.

And no, I'm not going to go away and fork or reimplement systemd or shut up as peter-b so charmingly suggested. I don't have infinite time. I am, in fact, permitted to say 'this is bad design, other projects don't do this, it has been done this way before and every time turned out to be a bad idea' without simultaneously being obligated to fix every single instance of bad design I see. Nobody could ever have enough time to do that.

This is a classic systemd proponents' attack: if you criticise the holy project in any way you should shut up unless you have personally implemented something better. Sorry, it doesn't work that way, and repeatedly attempting to shut down any and all criticism this way does *not* make me feel better about the project as a whole.

This whole debate saddens me

Posted Dec 11, 2014 19:30 UTC (Thu) by peter-b (subscriber, #66996) [Link] (4 responses)

Well, you can think it's shoddy design all you like. But that's your opinion. It's completely unreasonable to expect any software project to go to the trouble of maintaining bug-compatible stability of internal interfaces that are explicitly labeled as not for public use.

Show me a free software project that has such a ridiculous policy.

This whole debate saddens me

Posted Dec 13, 2014 2:49 UTC (Sat) by flussence (guest, #85566) [Link] (2 responses)

Does Wine count?

This whole debate saddens me

Posted Dec 13, 2014 3:30 UTC (Sat) by rahulsundaram (subscriber, #21946) [Link]

Not really. They had no choice given the purpose of their project.

This whole debate saddens me

Posted Dec 13, 2014 12:39 UTC (Sat) by cesarb (subscriber, #6266) [Link]

> > But that's your opinion. It's completely unreasonable to expect any software project to go to the trouble of maintaining bug-compatible stability of internal interfaces that are explicitly labeled as not for public use.
> >
> > Show me a free software project that has such a ridiculous policy.

> Does Wine count?

Wine's main internal, not-for-public-use interface is the server protocol, which is the RPC between the lower-level Wine DLLs and the shared wineserver process.

They don't maintain bug-compatible stability of that interface. Every time they feel the need, they add, remove, or change its methods and structures. The only concession to compatibility is a single version number, which is increased ever time they make a change, and used to detect version mismatches between the server and the DLL. That version number is already over 450, meaning there were over four hundred times that interface was changed in an incompatible manner.

So no, Wine doesn't count.

This whole debate saddens me

Posted Dec 14, 2014 17:21 UTC (Sun) by nix (subscriber, #2304) [Link]

Projects at the same layer: glibc. libgcc. libstdc++. Projects at a different layer: kdelibs (yes, really, right down to internal functions). libX11.

There is a certain degree of conservatism that should come with writing software at the kernel/userspace boundary, or at the 'everyone has lots of dependencies on me' level. glibc has it. systemd (like libpng) doesn't.

This whole debate saddens me

Posted Dec 11, 2014 23:49 UTC (Thu) by mchapman (subscriber, #66589) [Link] (3 responses)

> I happen to think this is shoddy design

This the first time I've heard of "calling a function in a different compilation unit" shoddy design. If that's what you really think, then all software is shoddy.

This whole debate saddens me

Posted Dec 14, 2014 17:24 UTC (Sun) by nix (subscriber, #2304) [Link] (2 responses)

It's not shoddy design unless you then turn around and say that things that use said functions in those compilation units are not really part of that software at all, they're just bundled into the same source tree so they can use that library interface! That position implies very strongly that that interface is actually an external interface, since things can call it that are not 'really' part of the project.

As far as can see, there are two possibilities here:

- Either those interfaces are external interfaces, and they should be a stable shared library API, and the systemd developers are doing shoddy work by not making them stable, or

- those interfaces are internal interfaces, subject to change as and when needed, and those things in the systemd source tree that use them are *part of systemd* and claims that oh no they just share a source tree and are not really part of it at all are, not to put too fine a point on it, lies.

I don't see a third option, and neither of these are particularly pleasant.

This whole debate saddens me

Posted Dec 14, 2014 20:15 UTC (Sun) by anselm (subscriber, #2796) [Link] (1 responses)

I don't know whether explaining this to you once more will make any difference or if you're deliberately trying to be obtuse, but here goes:

systemd as a whole consists of a number of modules that communicate via documented external interfaces. There is a stability promise that covers most of these interfaces. It is possible to replace most of those modules by independent implementations that are otherwise unrelated to the systemd code base as long as one sticks to the documented interfaces.

In the interest of saving work, various modules avail themselves of common subroutines in a separate directory in the source tree. Since all of systemd is released together and nothing outside of systemd uses these subroutines, there is little point in making these into a shared library (especially since then invariably somebody outside the project would want to to use that, and that would impact the systemd developers' freedom to change things around as needed to suit the implementation), and systemd uses linker features to ensure that every systemd modules contains only the code it actually uses.

Therefore systemd's "library API" is an internal implementation detail similar to the internal APIs of the Linux kernel. It only exists to make the systemd developers' life easier. Whether systemd consists of separate modules (it does) is of no consequence to this, because it is still possible to reimplement many of these separate systemd modules independently based on the documented external interfaces and the API stability promise.

This whole debate saddens me

Posted Dec 14, 2014 21:05 UTC (Sun) by deepfire (guest, #26138) [Link]

It should be a FAQ entry somewhere. You've written it up quite well..

This whole debate saddens me

Posted Dec 13, 2014 0:29 UTC (Sat) by njs (guest, #40338) [Link]

I don't see the comparison to the cut-and-paste code or libpng/zlib issues. The problems there are that you end up with multiple copies of the same source code, which then diverge.

In the systemd case, there's exactly one copy of the source code, and it always gets updated in sync with its reverse-dependencies. You do end up with multiple copies of the *compiled* code, but I don't see how this can cause problems beyond a miniscule waste of space. It's not really any different from how macros or inline functions or C++ templates work. Plus it almost certainly increases reliability to avoid adding another .so dependency to your early boot stack, esp. once you take into the account the possibility of version skew etc.