DNF and Yum in Fedora
Did you know...?LWN.net is a subscriber-supported publication; we rely on subscribers to keep the entire operation going. Please help out by buying a subscription and keeping LWN on the net.
The 0.4.10 release of the DNF package manager—slated to eventually replace Yum—was the proximate cause for a sizable fedora-devel mailing list thread, but it would seem a bit premature to get too concerned about the switch. The switch is targeted for Fedora 22, which is unlikely to be delivered before 2015. But the discussion does provide an opportunity to look in on DNF to see where things stand since we last looked at the package manager in June 2012.
Yum is, of course, the current package manager for Fedora (and RHEL). It started out as the Yellowdog Updater (YUP for Yellow Dog Linux), but was rewritten (and renamed) to "Yellowdog Updater, Modified" or Yum in the early 2000s. Yum manages packages in the RPM format, so it became the package manager of choice for several RPM-based distributions. But Yum has fairly slow performance and high memory usage; code "cruft" has increased over time as well. According to Martin Langhoff, much of that is due to backward compatibility requirements that Yum has—a rewrite could dispense with many of those.
There is another reason that Yum is being targeted for replacement: it uses its own, iterative dependency-resolution mechanism. More recent (and better performing) schemes for doing dependency resolution exist, and one, in the form of the satisfiability solving library libsolv, has been adopted by several other projects (including, of course, libsolv's origin: openSUSE's zypper package manager).
DNF does not use libsolv directly, but via the hawkey library. The move to hawkey has a number of benefits, including providing an API for languages other than Python. The existing Yum dependency-resolution code can be called from elsewhere, but only easily from Python. Programs like PackageKit that are written in C or other languages must either run Yum to do that work, or come up with their own dependency resolution. So, PackageKit, the Anaconda installer, and other programs that need to determine the dependencies for packages can use hawkey (and thus libsolv) for that.
Unlike Yum, DNF will explicitly define its API. The API for Yum has grown in an ad hoc fashion over the years, so revamping Yum will allow removing some parts of the API moving forward.
Eventually, DNF (which, contrary to many jokes, doesn't stand for anything) will be renamed to Yum, even though it is not a drop-in replacement. There are commands and features of Yum that will not be supported by DNF. Therein lies some of the unhappiness seen in the mailing list thread.
DNF complaints
Harald Reindl noted an announcement from DNF project lead Aleš Kozumplík that invited Fedora users to start trying out DNF in place of Yum. Reindl is concerned that various important Yum features are getting left by the wayside and that DNF will replace Yum without being ready. In another post, he likened it to the GRUB2 and systemd transitions, which were done before those components were ready, he said.
In particular, Reindl believes that the kernel package should be treated specially, as it is in Yum, rather than treated like any other package, as DNF does. A "yum erase kernel" command will not erase (aka remove) the package for the currently running kernel, while the DNF equivalent (replace "yum" with "dnf") will remove all kernel packages, including the currently running kernel. That will likely leave the system in an unbootable state, which is probably not what the user intended.
Kozumplík does not seem worried about the problem, though. He added an entry in a page describing differences between DNF and Yum indicating that there are no plans to treat the kernel package differently. He closed a bug filed about the problem as well. That didn't sit well with Reindl, but others were not particularly upset. Chris Adams explained that the Yum behavior was actually rather surprising to him:
But, as Rahul Sundaram pointed out, it is not just the kernel package that needs protection. The Yum protected_packages setting is ignored by DNF, but Sundaram thinks that may need to be reconsidered:
Matthew Miller noted that
protected_packages was originally put into Yum as a plugin, "but then
it proved itself in real use and became built-in
". So suggesting
that the feature could be added to DNF via plugin, as the DNF documentation
does, brings it all full circle—except that there is no
protected_packages-like plugin for DNF, at
least yet.
Another issue is the keepcache option, which has been dropped
from DNF. It would keep a copy of downloaded RPM files in the cache,
rather than deleting them after installation. The DNF documentation
indicates that there are "no real use cases
", but several in
the thread disagreed. Kevin Kofler said
that he changes the Fedora Yum default (keepcache=0) on every machine he
installs. A bug filed on
the problem lists additional use cases. Kozumplík's response indicated that
keepcache may return.
While some of the missing features are clearly important (and Sundaram has collected a list of problems he believes need to be addressed before any switch could be made), there is no huge rush. Fedora 21 is slated for August at the earliest, which would push Fedora 22 well into 2015—February or beyond. That is part of what caused the irritation displayed by some of the thread participants; it is just too early (at least in their minds) to be alarmed about the state of DNF. If it were three months into the Fedora 22 cycle, one would guess that things would be different.
Barring any serious bugs being found, DNF would seem to be in pretty good shape overall, as Sundaram's list (which includes all of the features that Reindl and others were complaining about) shows. Before it replaces Yum, there will still be opportunities to slow down that process and delay it into Fedora 23 or beyond. But a year is a long time and one suspects that the DNF team will have dealt with any problems by then. Filing bugs and trying to ensure they get addressed is quite reasonable at this point—in fact quite helpful to the team—but it is rather premature to get up in arms about the transition.
DNF and Yum in Fedora
Posted Jan 16, 2014 10:51 UTC (Thu)
by NRArnot (subscriber, #3033)
[Link] (4 responses)
Posted Jan 16, 2014 10:51 UTC (Thu) by NRArnot (subscriber, #3033) [Link] (4 responses)
Why does anybody think that it's undesirable for the package manager to balk and warn you if you try to do something known to be dangerous? All that's needed is a --force flag to override the safety catch.
DNF and Yum in Fedora
Posted Jan 17, 2014 2:45 UTC (Fri)
by HelloWorld (guest, #56129)
[Link] (1 responses)
Posted Jan 17, 2014 2:45 UTC (Fri) by HelloWorld (guest, #56129) [Link] (1 responses)
There is a mechanism for that. dnf asks every time you uninstall a package and you can say yes or no. I don't think more than that is needed.
DNF and Yum in Fedora
Posted Jan 17, 2014 3:15 UTC (Fri)
by rahulsundaram (subscriber, #21946)
[Link]
Posted Jan 17, 2014 3:15 UTC (Fri) by rahulsundaram (subscriber, #21946) [Link]
DNF and Yum in Fedora
Posted Jan 17, 2014 11:51 UTC (Fri)
by jezuch (subscriber, #52988)
[Link]
Posted Jan 17, 2014 11:51 UTC (Fri) by jezuch (subscriber, #52988) [Link]
If you're doing things like that, then you have some weird habits. It's not hard to cause a disaster this way and after two or three of them you start to rely on autocompletion to make absolutely sure that you didn't mistype anything.
Well, at least I did :)
(I also double-check any command that has wildcards in it. ANY command.)
> Why does anybody think that it's undesirable for the package manager to balk and warn you if you try to do something known to be dangerous?
"Balk and warn" is not the same as "do not allow". The Debian package manager does warn if you try to remove a package from a "core" section (don't remember the exact naming) but it won't prevent you from that if you know what you are doing (and express this clearly enough). It will also warn you if you try to remove a running kernel, but that's done in the maintainer scripts in the package itself, not in the package manager.
DNF and Yum in Fedora
Posted Jan 17, 2014 12:38 UTC (Fri)
by hummassa (guest, #307)
[Link]
Posted Jan 17, 2014 12:38 UTC (Fri) by hummassa (guest, #307) [Link]
Why write a new package manager anyway?
Posted Jan 16, 2014 12:16 UTC (Thu)
by niner (subscriber, #26151)
[Link] (25 responses)
Posted Jan 16, 2014 12:16 UTC (Thu) by niner (subscriber, #26151) [Link] (25 responses)
Why write a new package manager anyway?
Posted Jan 16, 2014 12:54 UTC (Thu)
by johannbg (guest, #65743)
[Link] (9 responses)
Posted Jan 16, 2014 12:54 UTC (Thu) by johannbg (guest, #65743) [Link] (9 responses)
It's along time since this stopped being funny...
APT - aptitude - dselect - YUM - APT-RPM - poldek - up2date - urpmi - ZYpp -slapt-get - slackpkg - zendo - netpkg - swaret - appbrowser - Conary - Equo - pkgutils - pacman - PETget - PISI - Portage - Smart Package Manager - Steam - Tazpkg - Upkg - dpkg - synaptic - rpm - zif, dnf - PackageKit -Gnome Software - Ubuntu Software Center
And probably bunch of other stuff I forgot.
Why write a new package manager anyway?
Posted Jan 17, 2014 16:16 UTC (Fri)
by etienne (guest, #25256)
[Link] (6 responses)
Posted Jan 17, 2014 16:16 UTC (Fri) by etienne (guest, #25256) [Link] (6 responses)
most package manager lack something...
Even the list of thing to do is complex to describe:
- capture the full list of dependencies automatically, at compile time (by looking which include files are used, and which library are linked-in, and what versions) - depending on the configuration selected for the package itself.
- install all dependent packages when the user install an application
- remember which packages have been installed automatically and remove them automatically when they are no more needed
- when a package is uninstalled, magically decide if the dependencies which have been upgraded by the install should be downgraded
- detect which package has never been used (newest access time of any files in the package) when a user ask for it
- detect/disallow two package using the same (configuration) file, even if this file is the obvious place to change that parameter
- manage any "plugin" which have to be upgraded after a main application has been upgraded
- manage interpreted languages: upgrade python/java/... libraries without creating a package version containing the whole thing
- display the dependency graphically so that a user can understand where a problem comes from (dselect is not that appealing)
- display some information about a package without having to install, and classify packages in a way a user can find what he needs
I have probably missed some requirements.
Why write a new package manager anyway?
Posted Jan 17, 2014 21:36 UTC (Fri)
by mathstuf (subscriber, #69389)
[Link] (5 responses)
Posted Jan 17, 2014 21:36 UTC (Fri) by mathstuf (subscriber, #69389) [Link] (5 responses)
Do you have an example of this? It doesn't seem obvious to me.
Why write a new package manager anyway?
Posted Jan 20, 2014 10:29 UTC (Mon)
by etienne (guest, #25256)
[Link] (4 responses)
Posted Jan 20, 2014 10:29 UTC (Mon) by etienne (guest, #25256) [Link] (4 responses)
> Do you have an example of this? It doesn't seem obvious to me.
If the user uninstall a package because he does not need it, then the dependent libraries shall be kept (it is better to use up-to-date software), but if a user uninstall a package because there is a regression - then the package manager shall guess if the regression is in the package itself or in one of the dependant libraries pulled by the main package, that is the automagic part.
Why write a new package manager anyway?
Posted Jan 20, 2014 12:42 UTC (Mon)
by mpr22 (subscriber, #60784)
[Link] (3 responses)
Making that work halfway reliably sounds halting-complete.
Posted Jan 20, 2014 12:42 UTC (Mon) by mpr22 (subscriber, #60784) [Link] (3 responses)
Why write a new package manager anyway?
Posted Jan 20, 2014 18:50 UTC (Mon)
by vonbrand (subscriber, #4458)
[Link] (2 responses)
Posted Jan 20, 2014 18:50 UTC (Mon) by vonbrand (subscriber, #4458) [Link] (2 responses)
Worse. In the halting problem you know what behaviour to look for; here you haven't got the faintest clue (Is it slower? Now crashes with <random file>? The screen shows strange colors?).
Why write a new package manager anyway?
Posted Jan 20, 2014 23:50 UTC (Mon)
by mathstuf (subscriber, #69389)
[Link]
Posted Jan 20, 2014 23:50 UTC (Mon) by mathstuf (subscriber, #69389) [Link]
Why write a new package manager anyway?
Posted Jan 21, 2014 17:45 UTC (Tue)
by nix (subscriber, #2304)
[Link]
Posted Jan 21, 2014 17:45 UTC (Tue) by nix (subscriber, #2304) [Link]
Why write a new package manager anyway?
Posted Jan 23, 2014 22:59 UTC (Thu)
by etrusco (guest, #4227)
[Link] (1 responses)
Posted Jan 23, 2014 22:59 UTC (Thu) by etrusco (guest, #4227) [Link] (1 responses)
I used to test it from time in Ubuntu and Debian, but it's been a few years since I last used it. Time to play :)
Why write a new package manager anyway?
Posted Jan 24, 2014 8:43 UTC (Fri)
by BlueLightning (subscriber, #38978)
[Link]
Posted Jan 24, 2014 8:43 UTC (Fri) by BlueLightning (subscriber, #38978) [Link]
Why write a new package manager anyway?
Posted Jan 16, 2014 17:19 UTC (Thu)
by smoogen (subscriber, #97)
[Link] (13 responses)
Posted Jan 16, 2014 17:19 UTC (Thu) by smoogen (subscriber, #97) [Link] (13 responses)
Zypper solves the problems that OpenSuSE developers and users are looking to be solved. Hopefully yum/dnf look at the problems that Fedora developers and user are looking to be solved. Sometimes they overlap (hawkeye) and sometimes they don't.
Why write a new package manager anyway?
Posted Jan 16, 2014 17:29 UTC (Thu)
by jhoblitt (subscriber, #77733)
[Link] (5 responses)
Posted Jan 16, 2014 17:29 UTC (Thu) by jhoblitt (subscriber, #77733) [Link] (5 responses)
Could you explain how package management on those distros have mutually incompatible requirements?
Why write a new package manager anyway?
Posted Jan 16, 2014 18:42 UTC (Thu)
by dowdle (subscriber, #659)
[Link] (4 responses)
Posted Jan 16, 2014 18:42 UTC (Thu) by dowdle (subscriber, #659) [Link] (4 responses)
So dnf to me is really just a streamlining of yum and trying to make it better rather than being a completely different package manager.
Why write a new package manager anyway?
Posted Jan 16, 2014 19:15 UTC (Thu)
by johannbg (guest, #65743)
[Link] (3 responses)
Posted Jan 16, 2014 19:15 UTC (Thu) by johannbg (guest, #65743) [Link] (3 responses)
Red Hat just has to swallow that pill.
Why write a new package manager anyway?
Posted Jan 16, 2014 19:50 UTC (Thu)
by rahulsundaram (subscriber, #21946)
[Link] (2 responses)
Posted Jan 16, 2014 19:50 UTC (Thu) by rahulsundaram (subscriber, #21946) [Link] (2 responses)
However if you really want to talk about unification, I don't see Red Hat switching to DEB format or Debian switching to RPM format anytime soon but either of these formats are just tarballs with some additional metadata.
The interesting problems are the large amount of differences in other layers higher up (packaging guidelines, choice of components, home grown solutions like Debian menu etc) and if you unify them all, you will have a single Linux distribution. Not going to happen though but the differences are getting somewhat nullified via other deployment models (language specific ones are prominent now) and also emerging sandbox (https://docs.google.com/document/d/1QTgxakyUVFMkvr-xFY2Xg...) and container based models(ex: Docker).
GNOME Software's answer appears to be abstract it out and have plugins support more than just the native packaging format of the distributions. It is a changing landscape and we will see what such efforts evolve into.
Why write a new package manager anyway?
Posted Jan 16, 2014 21:23 UTC (Thu)
by johannbg (guest, #65743)
[Link] (1 responses)
Posted Jan 16, 2014 21:23 UTC (Thu) by johannbg (guest, #65743) [Link] (1 responses)
"if you unify them all, you will have a single Linux distribution."
For quite sometime now ( couple of years ) the plumbing layer has been slowly moving to the direction of becoming whole which is an move which I see significantly strengthen the GNU/Linux ecosystem but I disagree that we wind up with a single Linux distribution since I philosophize and say what deviates distributions now is not so much the bits but more their philosophy and the communities surrounding each philosophy and those will always exist which also is good.
I agree it's going to be interesting to see if an to what extent GNOME Software changes the landscape for distributions shipping Gnome and distributions in general and I would not be surprised if it became it's own distribution in that process as well.
Why write a new package manager anyway?
Posted Jan 18, 2014 10:09 UTC (Sat)
by rahulsundaram (subscriber, #21946)
[Link]
Posted Jan 18, 2014 10:09 UTC (Sat) by rahulsundaram (subscriber, #21946) [Link]
Finish your sentence if you want others to follow your train of thought.
" For quite sometime now ( couple of years ) the plumbing layer has been slowly moving to the direction of becoming whole which is an move which I see significantly strengthen the GNU/Linux ecosystem"
The plumbing layer is consolidating however your idea of consolidating packaging formats is certainly not happening. There is now ever more than before.
" I agree it's going to be interesting to see if an to what extent GNOME Software changes the landscape for distributions shipping Gnome and distributions in general and I would not be surprised if it became it's own distribution in that process as well."
It would have a software center that would act as a distribution point but there is no real reason it would morph into a Linux distribution on its own. I don't see the logical reason or the commercial incentive to do that.
Why write a new package manager anyway?
Posted Jan 16, 2014 17:40 UTC (Thu)
by niner (subscriber, #26151)
[Link] (6 responses)
Posted Jan 16, 2014 17:40 UTC (Thu) by niner (subscriber, #26151) [Link] (6 responses)
* dnf looks for a middle ground between a sane API and some backwards compatibility.
* dnf and hawkey are first steps toward using the same resolver accross the entire stack.
* Libsolv is a well tested and proven code base, currently the most sophisticated and optimized dependency solving implementation.
But zypper uses libsolv (indeed, libsolv was developed for zypper), so the third point would be fulfilled by default. Since zypper uses libsolv, hawkey can still be used to provide the same resolver for the entire stack fulfilling the second point.
This leaves just the "middle ground between a sane API and some backwards compatibility". As hawkey could still be used the "sane API" argument does not count as a reason leaving only "some backwards compatibility". I'm quite sure that writing a wrapper around zypper that provides backwards compatibility with yum would be an easy task.
This leaves just the openSUSE specific things in zypper you mentioned that I can't find anything about. Could you please elaborate?
Why write a new package manager anyway?
Posted Jan 16, 2014 18:16 UTC (Thu)
by smoogen (subscriber, #97)
[Link] (5 responses)
Posted Jan 16, 2014 18:16 UTC (Thu) by smoogen (subscriber, #97) [Link] (5 responses)
In the end, I am going off from what the various people who have tried to get zypper into Fedora in the past. They ran into issues that are just basic design choices between the OS's.
Why write a new package manager anyway?
Posted Jan 16, 2014 18:26 UTC (Thu)
by johannbg (guest, #65743)
[Link] (4 responses)
Posted Jan 16, 2014 18:26 UTC (Thu) by johannbg (guest, #65743) [Link] (4 responses)
Why write a new package manager anyway?
Posted Jan 16, 2014 21:06 UTC (Thu)
by RedDwarf (guest, #89472)
[Link] (3 responses)
Posted Jan 16, 2014 21:06 UTC (Thu) by RedDwarf (guest, #89472) [Link] (3 responses)
a) "Clean"
Uninstall unused packages. You know, you install pacA, which requires pacB. The you uninstall pacA and pacB stays installed.
To do this properly you need to know if a package was installed automatically, or directly per the user instruction.
libzypp uses an history file (O(n), with n being the number of RPM/zypp operations you have made since you installed the system).
This information should be stored in the RPM database itself, so if you install a package without using libzypp you still have the info.
b) Soft dependencies
"Recommends", "Suggests", etc.
Why upstream RPM still doesn't include them is a big mystery to me.
c) Poorly documented dependency types
AND dependencies, which are included in the repository metadata but not in the RPMs (WTF!) because RPM doesn't support AND dependencies.
Dependencies on installed hardware (think drivers), system language, etc... basically extra features that you can ignore.
d) Something similar to file triggers
It's not even really used. But libzypp supports something to run a script at the end of the whole transaction. This shouldn't be needed, RPM should support file triggers.
openSUSE's RPM is here: https://build.opensuse.org/package/show/openSUSE:Factory/rpm
Yes, a lot of patches, but they don't really change things so much.
Why write a new package manager anyway?
Posted Jan 16, 2014 22:19 UTC (Thu)
by mathstuf (subscriber, #69389)
[Link]
Posted Jan 16, 2014 22:19 UTC (Thu) by mathstuf (subscriber, #69389) [Link]
Yum stores this info in the yumdb. Since RPM doesn't actually fetch, it doesn't have a reason why it is doing installs and removals. For yum, there's the yum-remove-leaves package (and yum history undo $transaction). My understanding is that dnf acts with remove-leaves behavior by default.
> "Recommends", "Suggests", etc.
Ask RPM devs. Something about it being hairy to implement? Or the semantics associated with it.
> AND dependencies
While nice, this isn't included anywhere. Instead, you Require a virtual package (like web-server) which is Provide'd by httpd, nginx, etc. But yes, dependencies on things outside of the RPM world would be nice, but is also likely a Hard Problem (what if I remove my joystick; should RPM refuse to do anything until I uninstall a joystick-requiring package?).
> Something similar to file triggers
This would be very nice. No more need to look up which scriptlet my package may need.
Why write a new package manager anyway?
Posted Jan 17, 2014 2:52 UTC (Fri)
by HelloWorld (guest, #56129)
[Link] (1 responses)
Posted Jan 17, 2014 2:52 UTC (Fri) by HelloWorld (guest, #56129) [Link] (1 responses)
Why write a new package manager anyway?
Posted Jan 17, 2014 3:22 UTC (Fri)
by rahulsundaram (subscriber, #21946)
[Link]
Posted Jan 17, 2014 3:22 UTC (Fri) by rahulsundaram (subscriber, #21946) [Link]
https://www.youtube.com/watch?v=FNwNF19oFqM
Unfortunately the slides are not accessible anymore but if anyone has a non video reference, feel free to post it.
Why write a new package manager anyway?
Posted Aug 18, 2023 20:50 UTC (Fri)
by wtarreau (subscriber, #51152)
[Link]
Posted Aug 18, 2023 20:50 UTC (Fri) by wtarreau (subscriber, #51152) [Link]
Didn't you know that NIH is good healthy food for distros in general ? Sadly that's often true :-(
DNF and Yum in Fedora
Posted Jan 17, 2014 7:23 UTC (Fri)
by xorbe (guest, #3165)
[Link] (2 responses)
Posted Jan 17, 2014 7:23 UTC (Fri) by xorbe (guest, #3165) [Link] (2 responses)
DNF and Yum in Fedora
Posted Jan 26, 2014 23:14 UTC (Sun)
by gdt (subscriber, #6284)
[Link]
Posted Jan 26, 2014 23:14 UTC (Sun) by gdt (subscriber, #6284) [Link]