Commit message (Collapse)AuthorAgeFilesLines
* _serialize_tasks: limit scope of dropped circular dependenciesZac Medico2019-12-261-38/+56
| | | | | | | | | | | | | | | | | | | | | | | | | Ensure that all members of a buildtime dependency cycle are merged as a group, such that packages which depend on one or more members of the group will only be merged *after* the entire group has been merged. This extends runtime cycle handling to also handle buildtime cycles in cases where the buildtime dependencies happen to be satisfied by installed packages. In situations when this is necessary, it is desirable to rely on the old installed instances of these packages as little as possible, since they might have been broken by the upgrade of a package that is a member of the dependency cycle. Upgrading members of the cycle as a group effectively minimizes reliance on the old installed package instances, avoiding some cases of bug 199856. For example, it should avoid bug 703676, where libspectre reportedly failed to build against an old installed instance of ghostscript-gpl. Bug: https://bugs.gentoo.org/199856 Bug: https://bugs.gentoo.org/689644 Bug: https://bugs.gentoo.org/690436 Bug: https://bugs.gentoo.org/703676 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emerge --with-test-deps: use _queue_disjunctive_depsZac Medico2019-12-261-5/+9
| | | | | | | | This delays evaluation of || deps, avoiding possible redundant packages and conflicts. Fixes: 85f0dd173ab7 ("emerge --with-test-deps: allow circular deps") Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Clean up remnants of eapi5-hdepend, HDEPEND and targetrootMichał Górny2019-12-263-15/+4
| | | | | | Reviewed-by: Zac Medico <zmedico@gentoo.org> Closes: https://github.com/gentoo/portage/pull/490 Signed-off-by: Michał Górny <mgorny@gentoo.org>
* backtracking: adjust || preference to break dependency cyclesZac Medico2019-12-232-4/+49
| | | | | | | | | | | | | | | | | | | Store dependency cycle edges as backtracking parameters, and use them to adjust || preferences in order to break dependency cycles. This extends direct cycle breaking to handle indirect dependency cycles, which solves the cmake-bootstrap test case for bug 703440. If any cycle(s) remain unsolved by the next backtracking run, then backtracking aborts and the cycle(s) are reported as usual. Note that backtracking is necessary in order to avoid bugs of the form "emerge installs packages only to have them removed by depclean", since this sort of behavior is desirable only when it eliminates a dependency cycle. Bug: https://bugs.gentoo.org/382421 Bug: https://bugs.gentoo.org/384107 Bug: https://bugs.gentoo.org/703440 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* _queue_disjunctive_deps: group disjunctions recursively (bug 701996)Zac Medico2019-12-231-4/+4
| | | | | | | | | | When disjunctive dependencies are queued, recursively group together disjunctions from the same dependency string so that any overlap between them will trigger expansion to DNF. Fixes: fa7b6ea6ecdc ("_queue_disjunctive_deps: group disjunctions (bug 701996)") Bug: https://bugs.gentoo.org/701996 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emerge --with-test-deps: allow circular depsZac Medico2019-12-231-4/+15
| | | | | | | | | | When USE=test is not enabled, allow circular test dependencies by treating them like PDEPEND. When USE=test is enabled, circular dependencies are still not allowed, as shown in unit tests. Suggested-by: Michał Górny <mgorny@gentoo.org> Bug: https://bugs.gentoo.org/703348 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emerge: Show package USE in conflict messagesZac Medico2019-12-142-9/+34
| | | | | Bug: https://bugs.gentoo.org/310009 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emerge: drop FEATURES=distcc-pump support, bug #702146Sergei Trofimovich2019-12-081-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | 'distcc' distributes code generation for preprocessed files. 'pump' distributes preprocessing and code generation of files and imposes very strict requirement: """ Note that distcc's pump-mode assumes that sources files will not be modified during the lifetime of the include server, so modifying source files during a build may cause inconsistent results. """ `src_configure()` (where we used to start include server before this change) almost always violates that requirement. It is not uncommon to generate more intermediate source files as a package builds (`bison`, `flex`, child `./configure` calls from `make`) and thus quite unsafe to use `pump`. This change drops `FEATURES=distcc-pump` and leaves only FEATURES=distcc. This way all the proprocessing happens as expected and only code generation is offloaded. Bug: https://bugs.gentoo.org/702146 Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emerge: add --implicit-system-deps <y|n> option (bug 681312)Zac Medico2019-12-054-3/+18
| | | | | | | | | | | | Assume that packages may have implicit dependencies on packages which belong to the @system set. This option is enabled by default. One of the effects of disabling this option is to allow the --jobs option to spawn jobs without accounting for the possiblity of implicit dependencies on packages that belong to the @system set (this causes the @system set to behave more like the @profile set). Bug: https://bugs.gentoo.org/681312 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* _queue_disjunctive_deps: group disjunctions (bug 701996)Zac Medico2019-12-051-2/+6
| | | | | | | | | When disjunctive dependencies are queued, group together disjunctions from the same dependency string so that any overlap between them will trigger expansion to DNF. Bug: https://bugs.gentoo.org/701996 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* depgraph: fix buildtime_blockers logicZac Medico2019-11-271-3/+1
| | | | | | Fixes: 1b3131db0e22 ("emerge --buildpkgonly: respect buildtime hard blockers") Bug: https://bugs.gentoo.org/689226 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emerge: add --quickpkg-direct optionZac Medico2019-11-265-32/+97
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Enable use of installed packages directly as binary packages. This is similar to using binary packages produced by quickpkg(1), but installed packages are used directly as though they are binary packages. This option only works in combination with the --root=DIR option, and it comes with the caveat that packages are only allowed to be installed into the root that is specified by the --root=DIR option. The other root which serves as a source of packages is assumed to be immutable during the entire operation (similar to --buildpkgonly mode). Default behavior for handling of protected configuration files is controlled by the QUICKPKG_DEFAULT_OPTS variable. When a configuration file is not included because it is protected, an ewarn message is logged. Suggested use cases: * Install packages from a buildtime container into an empty root, in order to create a minimal runtime container (which need not include a package manager). In a multi-stage Dockerfile, install runtime files to an empty directory in the build stage, and in the final stage use COPY to populate a container with the contents of that directory. For greater efficiency, use buildah to install directly into a mounted container, avoiding the COPY step. Use the emerge --usepkgonly and --ignore-soname-deps=n options to account for soname dependencies, allowing implicit system dependencies such as glibc to be automatically pulled into the runtime image. * Enable a live usb, iso, or pxe image to act as a binary installer that uses packages installed in the live image as a source of binary packages. Bug: https://bugs.gentoo.org/699986 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emerge --buildpkgonly: respect buildtime hard blockersZac Medico2019-11-261-8/+16
| | | | | Bug: https://bugs.gentoo.org/689226 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Scheduler._run_pkg_pretend: don't use local event loopZac Medico2019-11-241-5/+1
| | | | | | Relevant hooks should be added/removed in the _merge method. Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Binpkg: fix _chpathtool_exit rmtree ENOENT after renameZac Medico2019-11-171-2/+3
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Binpkg: use cached BINPKGMD5 if availableZac Medico2019-11-171-8/+8
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Binpkg: use aux_get for check_missing_metadataZac Medico2019-11-161-11/+7
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emerge: fix error message for unknown options (bug 673400)Zac Medico2019-10-271-5/+4
| | | | | | | | Do not use parse_known_args to parse positional arguments, since that causes unknown options to be handled like unknown positional arguments. Bug: https://bugs.gentoo.org/673400 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Improve runtime performance of depgraph module functions.Justin Turner Arthur2019-10-231-156/+169
| | | | | | | | Use set literals, compile-time str concats, reduce function calls, especially those on built-in types that can be replaced by operators. Signed-off-by: Justin Turner Arthur <justinarthur@gmail.com> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Default disable autounmask package.accept_keywords/mask changes (bug 658648)Zac Medico2019-10-143-8/+57
| | | | | | | | | | | | Add emerge --autounmask-license and --autounmask-use options which are enabled by default, and disable package.accept_keywords/mask changes by default. For backward compatibility, previous behavior of --autounmask=y and --autounmask=n is entirely preserved. Users can get the old behavior simply by adding --autounmask to the make.conf EMERGE_DEFAULT_OPTS variable. Bug: https://bugs.gentoo.org/658648 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* _add_dep: less aggressive backtracking (bug 693836)Zac Medico2019-09-111-0/+13
| | | | | | | | | | | | | | | | In order to suppress the sort of aggressive backtracking that can trigger undesirable downgrades as in bug 693836, do not backtrack for an unsatisfied dependency if there's an available package in the runtime package mask which was involved in a slot conflict and satisfied all involved parent atoms. Instead, discard the current depgraph in favor of other backtracking configurations that may exist. This case would not have been encountered prior to the fix for bug 692746 which enabled backtracking for the type of slot conflict that is detected here. Fixes: 994ac00aa764 ("_slot_confict_backtrack: consider masking a package matched by all parent atoms (bug 692746)") Bug: https://bugs.gentoo.org/693836 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* _slot_confict_backtrack: consider masking a package matched by all parent ↵Zac Medico2019-09-012-0/+14
| | | | | | | | | | | | | | atoms (bug 692746) When a slot conflict occurs involving a package that is matched by all involved parent atoms, consider masking the package in order to avoid a possible missed update. The included unit test demonstrates the case fixed by this patch. There are 2 previously existing unit tests that require larger backtracking values in order to succeed with this patch, since more possible solutions are now considered. Bug: https://bugs.gentoo.org/692746 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* _backtrack_depgraph: fix premature backtracking termination (bug 693242)Zac Medico2019-08-311-2/+2
| | | | | | | | | | Make backtracking continue as long as the backtracker has remaining nodes to explore. This fixes a case where it would terminate prematurely when the depgraph.need_restart() method returned False, even though the backtracker had remaining nodes to explore. Bug: https://bugs.gentoo.org/693242 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emerge: deprecate --changelog option (bug 693096)Zac Medico2019-08-301-0/+5
| | | | | | | | The emerge --changelog option is not very useful since the gentoo repository no longer includes ChangeLog files. Bug: https://bugs.gentoo.org/693096 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* EbuildExecuter: remove unused _live_eclasses attributeZac Medico2019-08-051-2/+0
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Support PROPERTIES="live".Ulrich Müller2019-07-271-2/+2
| | | | | Bug: https://bugs.gentoo.org/233589 Signed-off-by: Ulrich Müller <ulm@gentoo.org>
* emerge: Don't treat empty EPREFIX or PORTAGE_CONFIGROOT as unsetJames Le Cuirot2019-07-071-1/+1
| | | | | | | | | | | | | | | If a prefix user wanted to build within a ROOT but without a prefix, they previously had to set EPREFIX=/ rather than EPREFIX= as the latter was simply treated as unset. Also applies to ROOT and SYSROOT but probably makes no difference to these as they are blank by default. This should be safe to do as all these variables get normalised anyway. Bug: https://bugs.gentoo.org/642604 Closes: https://github.com/gentoo/portage/pull/434 Signed-off-by: James Le Cuirot <chewi@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Use _safe_loop where applicableZac Medico2019-05-182-5/+5
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* fetch: atomic downloads (bug 175612)Zac Medico2019-04-291-2/+2
| | | | | | | | | | | Direct FETCHCOMMAND/RESUMECOMMAND output to a temporary file with a constant .__download__ suffix, and atomically rename the file to remove the suffix only after the download has completed successfully (includes digest verification when applicable). Also add unit tests to cover most fetch cases. Bug: https://bugs.gentoo.org/175612 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emergelog: handle /dev/null symlink (bug 684232)Zac Medico2019-04-241-1/+1
| | | | | | | | | | If the log path is a symlink to /dev/null, os.path.isfile returns False, so use os.path.exists in order to avoid chown/chmod in this case. Reported-by: Harald Becker <hb-xxl@web.de> Bug: https://bugs.gentoo.org/684232 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Import OrderedDict directly from collectionsZac Medico2019-04-211-3/+3
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emerge --info: show per-package FEATURES (bug 553224)Zac Medico2019-04-201-1/+1
| | | | | Bug: https://bugs.gentoo.org/553224 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* SpawnProcess: allow unshare_mount and unshare_pidZac Medico2019-01-211-1/+1
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emerge: compare new slot USE to installed slotArfrever Frehtes Taifersar Arahesis2019-01-191-4/+9
| | | | | | | | | For new-slot package installation, show USE flags added/deleted in comparison with highest installed other-slot package instance Bug: https://bugs.gentoo.org/675756 Signed-off-by: Arfrever Frehtes Taifersar Arahesis <Arfrever@Apache.Org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* pid-sandbox: whitelist selected pkg_* phases (bug 673794)Zac Medico2019-01-081-3/+6
| | | | | | | | | | Whitelist the same phases that are whitelisted for FEATURES=cgroup, since pid-sandbox is less valuable and is likely to have unintended consenquences during these phases. Bug: https://bugs.gentoo.org/673794 Reviewed-by: Michał Górny <mgorny@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Renamed PORT_LOGDIR[_CLEAN] to PORTAGE_LOGDIR[_CLEAN]M. J. Everitt2018-12-191-1/+1
| | | | | | | | | To match similar existing PORTAGE_* variables Suggested-by: Zac Medico <zmedico@gentoo.org> Acked-by: Michael Everitt <m.j.everitt@iee.org> Bug: https://bugs.gentoo.org/668538 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emerge --help: add -U option short for --changed-useGreg Kubaryk2018-11-251-1/+1
| | | | | Fixes: d55690aa3a1a ("emerge: add -U option short for --changed-use") Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emerge: don't ignore binpkgs with buildpkgonlyAndrew Jeddeloh2018-10-051-7/+0
| | | | | | | | | Do not ignore binpkgs when using --buildpkgonly. If binpackages exist that can be used, they should be. This prevents rebuilding binpkgs when using --emptytree with --buildpkgonly. Signed-off-by: Andrew Jeddeloh <andrew.jeddeloh@coreos.com> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Refer to "ebuild repository" rather than "portage tree"Zac Medico2018-08-064-4/+4
* Scheduler: cancel job delay timeout after main loop (bug 662668)Zac Medico2018-08-021-0/+3
| | | | | | | | | | | | | | | | | | | This fixes the following error during emerge --keep-going dependency calculation: Exception in callback PollScheduler._schedule() handle: <TimerHandle when=44692.1015056609 PollScheduler._schedule()> Traceback (most recent call last): File "/usr/lib64/python3.6/asyncio/events.py", line 145, in _run self._callback(*self._args) File "/usr/lib64/python3.6/site-packages/_emerge/PollScheduler.py", line 154, in _schedule self._schedule_tasks() File "/usr/lib64/python3.6/site-packages/_emerge/Scheduler.py", line 1605, in _schedule_tasks self._keep_scheduling() or self._main_exit.done()): AttributeError: 'NoneType' object has no attribute 'done' Reported-by: Rick Farina <zerochaos@gentoo.org> Bug: https://bugs.gentoo.org/662668
* Rename pym→lib, for better distutils-r1 interoperabilityMichał Górny2018-07-1889-0/+31245
Closes: https://github.com/gentoo/portage/pull/343