Get Gentoo!
gentoo.org sites
gentoo.org Wiki Bugs Forums Packages
Planet Archives Sources
Infra Status
aboutsummaryrefslogtreecommitdiff
path: root/lib/_emerge/depgraph.py
Commit message (Collapse)AuthorAgeFilesLines
* check_reverse_dependencies: dereference virtual expansions (bug 764764)Zac Medico2021-01-101-0/+22
| | | | | | | | | | | | | If an atom is the result of virtual expansion, then derefrence it to _orig_atom in check_reverse_dependencies so that it will be correctly handled as a built slot operator dependency when appropriate. This solves a case triggered in bug 764764 where a virtual expansion from virtual/dist-kernel:0/5.10.5= to =virtual/dist-kernel-5.10.5 prevented the atom from being handled as a built slot operator dependency, which prevented rebuilds from being triggered. Bug: https://bugs.gentoo.org/764764 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* depgraph._add_pkg: fix AttributeError when myparent is not a PackageZac Medico2020-12-031-1/+1
| | | | | | Fixes: b991f23ad915 ("Allow a package to replace its own buildtime dependency") Bug: https://bugs.gentoo.org/757741 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* find_smallest_cycle: don't merge satisfied PDEPEND too earlyZac Medico2020-12-021-4/+4
| | | | | | | | | | | | | | After PDEPENDs have been neglected by the find_smallest_cycle function, do not try to merge them too early if they are already satisfied by an installed package. This fixes incorrect merge order for PDEPEND cycles involving xorg-server and xorg-drivers, which was triggered by commit 5095c2023595a75e2848f1ad3dbe25b5fb451a44 because it gave PDEPEND higher priority than satisfied buildtime dependencies. Fixes: 5095c2023595 ("find_smallest_cycle: enhance search prioritization") Reported-by: josef64 in #gentoo-portage Bug: https://bugs.gentoo.org/754903 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Allow a package to replace its own buildtime dependencyZac Medico2020-11-281-13/+55
| | | | | | | | | | | | | | | | | If a package has a buildtime dependency on a previous version that it will replace, then do not treat it as a slot conflict. This solves inappropriate behavior for dev-lang/rust[system-bootstrap]. This requires adjustments to package selection logic in several locations, in order to ensure that an installed package instance will be selected to satisfy a buildtime dependency when appropriate. Dependencies of the installed package will be entirely ignored, but that has already been the case when using installed package to break cycles, as discussed in bug 199856. Bug: https://bugs.gentoo.org/756793 Bug: https://bugs.gentoo.org/756961 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* find_smallest_cycle: enhance search prioritizationZac Medico2020-11-211-17/+26
| | | | | | | | | | | | | | | | | | | | | | | | | Enhance the find_smallest_cycle function to prioritize its search so that it will minimize the use of installed packages to break cycles. When installed packages must be used to break cycles, it will now prefer to do this for runtime dependencies over buildtime dependencies, since it's preferable to build against latest versions of buildtime dependencies whenever possible. This should solve some cases of bug 199856 which have been triggered by unsafe reliance on installed packages to break cycles. The included unit test case demonstrates correct merge order for a dependency calculation involving 6 independent cycles. This test case fails with the master branch, due to a buildtime dependency cycle of 3 packages being merged earlier than cycles of 2 packages. We can generalize this to say that the master branch may use an installed package to break an arbitrarily sized cycle in a somewhat random location, even though that cycle may be composed of smaller independent cycles which would be safer to break individually. Bug: https://bugs.gentoo.org/754903 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Typofix: Use just one definite articlePetr Šabata2020-11-211-1/+1
| | | | | | | | | I first noticed this in make.conf(5) but it turned out there were many more occurences of these; fixed with find & sed. Closes: https://github.com/gentoo/portage/pull/634 Signed-off-by: Petr Šabata <contyk@redhat.com> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emerge: add --quickpkg-direct-root optionZac Medico2020-11-011-2/+9
| | | | | | | | | Specify the root to use as the --quickpkg-direct package source. This root is assumed to be immutable during the entire emerge operation. The default is set to "/". Bug: https://bugs.gentoo.org/752066 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* _slot_confict_backtrack: group similar missed updates (bug 743115)Zac Medico2020-09-201-4/+21
| | | | | | | | | | | | When a slot conflict occurs due to a missed update, and some other similar update(s) are available, add the similar update(s) to the runtime package mask for the same backtracking choice. This reduces minimum number of backtrack tries required to solve the test case for bug 743115 from 7 to 4, where the difference of 3 corresponds to the number of other similar setuptools updates available. Bug: https://bugs.gentoo.org/743115 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* _slot_confict_backtrack: minimize conflict atoms (bug 743631)Zac Medico2020-09-201-0/+6
| | | | | | | | | | | Prefer choices that minimize conflict atoms, so that choices which satisfy all parents are preferred. This reduces the minimum necessary backtrack tries from 21 to 7 for the unit test related to bug 743115. Bug: https://bugs.gentoo.org/743115 Bug: https://bugs.gentoo.org/743631 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* dephgraph: Allow elimination of highest version after slot conflict (bug 439688)Zac Medico2020-08-301-9/+0
| | | | | | | | | | | | | | | | | After a slot conflict occurs, allow the highest version to be eliminated from the graph when appropriate. This is needed for correct behavior in cases the highest version cannot be installed because an older version is required by some package. This reverts a change related to bug 531656 from commit a9064d08ef4c92a5d0d1bfb3dc8a01b7850812b0, and that change no longer appears to be necessary, since the unit tests related to bug 531656 now pass without it. Due to this change in slot conflict handling, the --changed-slot test case related to bug 456208 will now fail unless we use an @world update to trigger rebuilds, therefore fix it to do so. Bug: https://bugs.gentoo.org/439688 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* *: drop all import __future__ statementsAaron Bauman2020-08-061-3/+1
| | | | | | | * We only support Py3 now. Signed-off-by: Aaron Bauman <bman@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* lib/_emerge/depgraph.py: drop unused-importAaron Bauman2020-08-031-2/+0
| | | | | Signed-off-by: Aaron Bauman <bman@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Fix R1705.Alec Warner2020-07-291-67/+64
| | | | | | | | | | | | | | | | | | | | | | The codebase appears to have this pattern like: if foo == bar: return a elif foo == baz: return b else: return c This can often be rewritten as: if foo == bar: return a if foo == baz: return b return c Closes: https://github.com/gentoo/portage/pull/592 Signed-off-by: Alec Warner <antarus@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Fix redefined-builtin errors for pylint.Alec Warner2020-07-231-1/+1
| | | | | | | | | | | Some of these are simple variable renames. A few of the uses were refactored and some were simply disabled; often due to redefinition as part of a function signature. I did not do any research in terms of API changes so these were typically left as-is. Closes: https://github.com/gentoo/portage/pull/587 Signed-off-by: Alec Warner <antarus@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Fix 'unused-import' for stdlib imports.Alec Warner2020-07-221-1/+0
| | | | | | | | | | | This is part of a cleanup to enable pylint. This is only stdlib modules; there are hundreds of unused imports of other modules but its less clear how safe those are to remove due to side effects. Signed-off-by: Alec Warner <antarus@gentoo.org> Change-Id: If9bee83bbbe1620b34a0f8d153637909af2b8512 Closes: https://github.com/gentoo/portage/pull/585 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Fix R0205 in _emerge.Alec Warner2020-07-221-8/+8
| | | | | | | | Inheriting from object is not required in py3. Signed-off-by: Alec Warner <antarus@gentoo.org> Change-Id: Ia12500d590ef3b7f5852b6af03801d40a8af8fa2 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Eliminate basestring/long/_unicode py3 compatMichał Górny2020-07-161-11/+4
| | | | | | | | Replace basestring and _unicode hacks with str, and long with int. Reviewed-by: Zac Medico <zmedico@gentoo.org> Closes: https://github.com/gentoo/portage/pull/570 Signed-off-by: Michał Górny <mgorny@gentoo.org>
* Remove from __future__ import unicode_literalsZac Medico2020-07-151-1/+1
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* depgraph: respect <foo-version:= for slot operator rebuild (bug 717140)Zac Medico2020-04-111-3/+9
| | | | | | | | | | | | | | When searching for slot operator rebuilds, respect non slot-operator components of parent dependencies, so that a <foo-version:= dependency like the <dev-libs/libgit2-1:0=[ssh?] dependency from bug 717140 will not be completely ignored. This will prevent erroneous attempts to trigger slot operator rebuilds for upgrades that would break <foo-version:= dependencies (which triggered upgrade/downgrade loops when backtracking tried to resolve the breakage). Fixes: d569a2d7275c ("_slot_operator_update_probe: fix bug #508762") Bug: https://bugs.gentoo.org/717140 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* depgraph: respect --deep=<depth> with --update (bug 712298)Zac Medico2020-03-141-2/+6
| | | | | | | | | | | Fix the _wrapped_select_pkg_highest_available_imp method to select an installed package when appropriate for the current --deep=<depth> setting, even with --update enabled. This prevents violation of the current --deep=<depth> setting in cases where an installed package which satisfies a dependency is masked for any reason. Bug: https://bugs.gentoo.org/712298 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* _expand_set_args: un-reverse nested set traversalZac Medico2020-02-141-4/+2
| | | | | | | Un-reverse traversal order for @profile @selected @system. Bug: https://bugs.gentoo.org/649622 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* depclean: ensure consistency with update actions (bug 649622)Zac Medico2020-02-141-6/+15
| | | | | | | | | | | | | | Make depclean traverse dependencies in the same order as update actions, in order to ensure consistency in decisions which are dependent on the order of dependency evaluation due to inconsistent use of || preferences in different packages. In unit tests, update test_virtual_w3m_realistic to assert that the order of graph traversal is deterministic and consistent between update and removal (depclean) actions. Bug: https://bugs.gentoo.org/649622 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* depgraph: sort nested package set names for consistent resultsZac Medico2020-02-111-2/+4
| | | | | | | | | Traverse the unordered set returned from PackageSet.getNonAtoms() in sorted order, for more consistent results. Use a reverse sort since we pop these sets from a stack. Bug: https://bugs.gentoo.org/649622 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* depgraph: sort package set atoms for consistent resultsZac Medico2020-02-091-3/+3
| | | | | | | | Traverse the unordered set returned from PackageSet.getAtoms() in sorted order, for more consistent results. Bug: https://bugs.gentoo.org/649622 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* cpv_expand: treat GLEP 81 acct-* categories like virtual (bug 691798)Zac Medico2020-02-011-1/+1
| | | | | | | | | | This solves common name collisions introduced by GLEP 81 user and group management packages. TODO: Provide user configuration so that users an ignore name collisions with selected categories like app-emacs or dev-haskell. Signed-off-by: Zac Medico <zmedico@gentoo.org>
* dep_zapdeps: adjust || preference for slot upgrades (bug 706278)Zac Medico2020-01-261-2/+14
| | | | | | | | | | | | | | | | | | | | | Prefer choices that include a slot upgrade when appropriate, like for the || ( llvm:10 ... llvm:7 ) case reported in bug 706278. In order to avoid pulling in inappropriate slot upgrades, like those which should only be pulled in with --update and --deep, add a want_update flag to each choice which is True for choices that pull in a new slot for which an update is desirable. Mark the test case for bug 480736 as todo, since the "undesirable" slot upgrade which triggers a blocker conflict in this test case is practically indistinguishable from a desirable slot upgrade. This particular blocker conflict is no longer relevant, since current versions of media-libs/libpostproc are no longer compatible with any available media-video/ffmpeg slot. In order to solve this test case, some fancy backtracking (like for bug 382421) will be required. Bug: https://bugs.gentoo.org/706278 Bug: https://bugs.gentoo.org/480736 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* _serialize_tasks: limit scope of dropped circular dependenciesZac Medico2019-12-261-38/+56
| | | | | | | | | | | | | | | | | | | | | | | | | Ensure that all members of a buildtime dependency cycle are merged as a group, such that packages which depend on one or more members of the group will only be merged *after* the entire group has been merged. This extends runtime cycle handling to also handle buildtime cycles in cases where the buildtime dependencies happen to be satisfied by installed packages. In situations when this is necessary, it is desirable to rely on the old installed instances of these packages as little as possible, since they might have been broken by the upgrade of a package that is a member of the dependency cycle. Upgrading members of the cycle as a group effectively minimizes reliance on the old installed package instances, avoiding some cases of bug 199856. For example, it should avoid bug 703676, where libspectre reportedly failed to build against an old installed instance of ghostscript-gpl. Bug: https://bugs.gentoo.org/199856 Bug: https://bugs.gentoo.org/689644 Bug: https://bugs.gentoo.org/690436 Bug: https://bugs.gentoo.org/703676 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emerge --with-test-deps: use _queue_disjunctive_depsZac Medico2019-12-261-5/+9
| | | | | | | | This delays evaluation of || deps, avoiding possible redundant packages and conflicts. Fixes: 85f0dd173ab7 ("emerge --with-test-deps: allow circular deps") Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Clean up remnants of eapi5-hdepend, HDEPEND and targetrootMichał Górny2019-12-261-11/+1
| | | | | | Reviewed-by: Zac Medico <zmedico@gentoo.org> Closes: https://github.com/gentoo/portage/pull/490 Signed-off-by: Michał Górny <mgorny@gentoo.org>
* backtracking: adjust || preference to break dependency cyclesZac Medico2019-12-231-2/+40
| | | | | | | | | | | | | | | | | | | Store dependency cycle edges as backtracking parameters, and use them to adjust || preferences in order to break dependency cycles. This extends direct cycle breaking to handle indirect dependency cycles, which solves the cmake-bootstrap test case for bug 703440. If any cycle(s) remain unsolved by the next backtracking run, then backtracking aborts and the cycle(s) are reported as usual. Note that backtracking is necessary in order to avoid bugs of the form "emerge installs packages only to have them removed by depclean", since this sort of behavior is desirable only when it eliminates a dependency cycle. Bug: https://bugs.gentoo.org/382421 Bug: https://bugs.gentoo.org/384107 Bug: https://bugs.gentoo.org/703440 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* _queue_disjunctive_deps: group disjunctions recursively (bug 701996)Zac Medico2019-12-231-4/+4
| | | | | | | | | | When disjunctive dependencies are queued, recursively group together disjunctions from the same dependency string so that any overlap between them will trigger expansion to DNF. Fixes: fa7b6ea6ecdc ("_queue_disjunctive_deps: group disjunctions (bug 701996)") Bug: https://bugs.gentoo.org/701996 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emerge --with-test-deps: allow circular depsZac Medico2019-12-231-4/+15
| | | | | | | | | | When USE=test is not enabled, allow circular test dependencies by treating them like PDEPEND. When USE=test is enabled, circular dependencies are still not allowed, as shown in unit tests. Suggested-by: Michał Górny <mgorny@gentoo.org> Bug: https://bugs.gentoo.org/703348 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emerge: Show package USE in conflict messagesZac Medico2019-12-141-5/+21
| | | | | Bug: https://bugs.gentoo.org/310009 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emerge: add --implicit-system-deps <y|n> option (bug 681312)Zac Medico2019-12-051-3/+4
| | | | | | | | | | | | Assume that packages may have implicit dependencies on packages which belong to the @system set. This option is enabled by default. One of the effects of disabling this option is to allow the --jobs option to spawn jobs without accounting for the possiblity of implicit dependencies on packages that belong to the @system set (this causes the @system set to behave more like the @profile set). Bug: https://bugs.gentoo.org/681312 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* _queue_disjunctive_deps: group disjunctions (bug 701996)Zac Medico2019-12-051-2/+6
| | | | | | | | | When disjunctive dependencies are queued, group together disjunctions from the same dependency string so that any overlap between them will trigger expansion to DNF. Bug: https://bugs.gentoo.org/701996 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* depgraph: fix buildtime_blockers logicZac Medico2019-11-271-3/+1
| | | | | | Fixes: 1b3131db0e22 ("emerge --buildpkgonly: respect buildtime hard blockers") Bug: https://bugs.gentoo.org/689226 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emerge: add --quickpkg-direct optionZac Medico2019-11-261-0/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Enable use of installed packages directly as binary packages. This is similar to using binary packages produced by quickpkg(1), but installed packages are used directly as though they are binary packages. This option only works in combination with the --root=DIR option, and it comes with the caveat that packages are only allowed to be installed into the root that is specified by the --root=DIR option. The other root which serves as a source of packages is assumed to be immutable during the entire operation (similar to --buildpkgonly mode). Default behavior for handling of protected configuration files is controlled by the QUICKPKG_DEFAULT_OPTS variable. When a configuration file is not included because it is protected, an ewarn message is logged. Suggested use cases: * Install packages from a buildtime container into an empty root, in order to create a minimal runtime container (which need not include a package manager). In a multi-stage Dockerfile, install runtime files to an empty directory in the build stage, and in the final stage use COPY to populate a container with the contents of that directory. For greater efficiency, use buildah to install directly into a mounted container, avoiding the COPY step. Use the emerge --usepkgonly and --ignore-soname-deps=n options to account for soname dependencies, allowing implicit system dependencies such as glibc to be automatically pulled into the runtime image. * Enable a live usb, iso, or pxe image to act as a binary installer that uses packages installed in the live image as a source of binary packages. Bug: https://bugs.gentoo.org/699986 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emerge --buildpkgonly: respect buildtime hard blockersZac Medico2019-11-261-8/+16
| | | | | Bug: https://bugs.gentoo.org/689226 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Improve runtime performance of depgraph module functions.Justin Turner Arthur2019-10-231-156/+169
| | | | | | | | Use set literals, compile-time str concats, reduce function calls, especially those on built-in types that can be replaced by operators. Signed-off-by: Justin Turner Arthur <justinarthur@gmail.com> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Default disable autounmask package.accept_keywords/mask changes (bug 658648)Zac Medico2019-10-141-8/+12
| | | | | | | | | | | | Add emerge --autounmask-license and --autounmask-use options which are enabled by default, and disable package.accept_keywords/mask changes by default. For backward compatibility, previous behavior of --autounmask=y and --autounmask=n is entirely preserved. Users can get the old behavior simply by adding --autounmask to the make.conf EMERGE_DEFAULT_OPTS variable. Bug: https://bugs.gentoo.org/658648 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* _add_dep: less aggressive backtracking (bug 693836)Zac Medico2019-09-111-0/+13
| | | | | | | | | | | | | | | | In order to suppress the sort of aggressive backtracking that can trigger undesirable downgrades as in bug 693836, do not backtrack for an unsatisfied dependency if there's an available package in the runtime package mask which was involved in a slot conflict and satisfied all involved parent atoms. Instead, discard the current depgraph in favor of other backtracking configurations that may exist. This case would not have been encountered prior to the fix for bug 692746 which enabled backtracking for the type of slot conflict that is detected here. Fixes: 994ac00aa764 ("_slot_confict_backtrack: consider masking a package matched by all parent atoms (bug 692746)") Bug: https://bugs.gentoo.org/693836 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* _slot_confict_backtrack: consider masking a package matched by all parent ↵Zac Medico2019-09-011-0/+5
| | | | | | | | | | | | | | atoms (bug 692746) When a slot conflict occurs involving a package that is matched by all involved parent atoms, consider masking the package in order to avoid a possible missed update. The included unit test demonstrates the case fixed by this patch. There are 2 previously existing unit tests that require larger backtracking values in order to succeed with this patch, since more possible solutions are now considered. Bug: https://bugs.gentoo.org/692746 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* _backtrack_depgraph: fix premature backtracking termination (bug 693242)Zac Medico2019-08-311-2/+2
| | | | | | | | | | Make backtracking continue as long as the backtracker has remaining nodes to explore. This fixes a case where it would terminate prematurely when the depgraph.need_restart() method returned False, even though the backtracker had remaining nodes to explore. Bug: https://bugs.gentoo.org/693242 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Use _safe_loop where applicableZac Medico2019-05-181-2/+2
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Import OrderedDict directly from collectionsZac Medico2019-04-211-3/+3
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Refer to "ebuild repository" rather than "portage tree"Zac Medico2018-08-061-1/+1
|
* Rename pym→lib, for better distutils-r1 interoperabilityMichał Górny2018-07-181-0/+10049
Closes: https://github.com/gentoo/portage/pull/343