Get Gentoo!
gentoo.org sites
gentoo.org Wiki Bugs Forums Packages
Planet Archives Sources
Infra Status
aboutsummaryrefslogtreecommitdiff
path: root/pym
Commit message (Collapse)AuthorAgeFilesLines
* Rename pym→lib, for better distutils-r1 interoperabilityMichał Górny2018-07-18591-117710/+0
| | | | Closes: https://github.com/gentoo/portage/pull/343
* EventLoop: raise TypeError for unexpected call_* keyword argsZac Medico2018-07-171-0/+24
|
* EventLoop: use python2.7 compatible **kwargs for call_* context argZac Medico2018-07-171-4/+4
| | | | | | | Since python2.7 does not allow positional default arguments after *args, use **kwargs instead. Fixes: ae8cc32ccd81 ("EventLoop: add call_* context arg for python3.7 compat")
* EventLoop: add call_* context arg for python3.7 compatZac Medico2018-07-171-4/+22
| | | | | The context argument currently does nothing, but exists for minimal interoperability with Future instances that require it for PEP 567.
* repoman: populate implicit IUSE for empty profile (bug 660982)Zac Medico2018-07-151-7/+3
| | | | | | | | | | | | For the empty profile that's used to check dependencies of ebuilds that have empty KEYWORDS, populate implicit IUSE from all of the make.defaults files found in the relevant repositories (this should take less than 1 second on most hardware). Since the IUSE.missing check cannot be performed without implicit IUSE settings, this makes the IUSE.missing check work for ebuilds with empty KEYWORDS. Bug: https://bugs.gentoo.org/660982
* RepoConfig: fix make.conf PORTDIR override (bug 661276)Zac Medico2018-07-151-1/+2
| | | | | | | | | | Pass the main-repo name from repos.conf as the repository name for the PORTDIR RepoConfig constructor, so that it can override the main repo location even if the profiles/repo_name file does not exist yet (like in a stage3 tarball). Bug: https://bugs.gentoo.org/661276 Reported-by: Jorge Manuel B. S. Vicetto <jmbsvicetto@gentoo.org>
* portdbapi.cp_list: honor porttrees modifications (bug 650814)Zac Medico2018-07-151-2/+23
| | | | | | | | | Consumers, such as repoman and emirrordist, may modify the porttrees attribute in order to modify the effective set of repositories for all portdbapi operations. Fixes: 27eeeb6e4fc8 ("portdbapi.cp_list: cache repo associations (bug 650814)") Bug: https://bugs.gentoo.org/650814
* config: cache profile.bashrc stat results (bug 649806)Zac Medico2018-07-151-5/+8
| | | | | | | | Optimize config to stat profile.bashrc files once in the constructor, in order to avoid repeated stat calls in the setcpv method. Bug: https://bugs.gentoo.org/649806 Reviewed-by: Brian Dolbec <dolsen@gentoo.org>
* portdbapi.xmatch: return separate items for each repoZac Medico2018-07-151-16/+3
| | | | | | | | | Since xmatch match-all shares cache with the cp_list method, it needs to return separate items when the same package version is found in multiple repos. Fixes: 27eeeb6e4fc8 ("portdbapi.cp_list: cache repo associations (bug 650814)") Bug: https://bugs.gentoo.org/650814
* portdbapi.cp_list: cache repo associations (bug 650814)Zac Medico2018-07-142-21/+32
| | | | | | | | | | | | | | | Make portdbapi.cp_list return _pkg_str instances that have a 'repo' attribute (bindbapi.cp_list already does this), with results in ascending order by (pkg.version, repo.priority). Optimize portdbapi.findname2 to use the 'repo' attribute to enable cached results for files previously found by the portdbapi.cp_list method, avoiding filesystem access when possible. Optimize the depgraph._iter_match_pkgs_atom method by elimination of the repo loop, since portdbapi.cp_list now returns separate items when the same package version is found in multiple repos. Bug: https://bugs.gentoo.org/650814 Reviewed-by: Brian Dolbec <dolsen@gentoo.org>
* _unmerge_dirs: revisit parents of removed symlinks (bug 640058)Zac Medico2018-07-141-2/+23
| | | | | | | | When removal of a symlink is triggered by removal of the directory that it points to, revisit the parent directories of the symlink. Bug: https://bugs.gentoo.org/640058 Reviewed-by: Brian Dolbec <dolsen@gentoo.org>
* dbapi: fix repoman implicit IUSE (bug 660982)Zac Medico2018-07-142-1/+20
| | | | | | | | | Account for repoman modifications of the portdbapi self.settings reference, and treat all flags as valid for the empty profile because it does not have any implicit IUSE settings. Bug: https://bugs.gentoo.org/660982 Reviewed-by: Brian Dolbec <dolsen@gentoo.org>
* Add python2 compatible coroutine support (bug 660426)Zac Medico2018-07-112-0/+271
| | | | | | | | | | | | | | For readability, it's desirable to make asynchronous code use coroutines to avoid callbacks when possible. For python2 compatibility, generators that yield Futures can be used to implement coroutines. Add a compat_coroutine module which provides a @coroutine decorator and a coroutine_return function that can be used to return a value from a generator. The decorated function returns a Future which is done when the generator is exhausted. Usage is very similar to asyncio coroutine usage in python3.4 (see unit tests). Bug: https://bugs.gentoo.org/660426
* GitSync: support sync-depth (bug 552814)Zac Medico2018-07-102-5/+25
| | | | | | | | | | | | | | | | Support sync-depth for shallow sync, using git reset --merge just like in the earlier implementation that was reverted in commit ab840ac982d3c8b676b89f6bedd14e85dd06870f. Also, run git gc --auto in the foreground, in order to trigger periodic housekeeping and hopefully avoid errors from automatic git gc calls as reported in bug 599008. The default sync-depth is unlimited, which means that default behavior remains unchanged (unlike the previous implementation that was reverted). Bug: https://bugs.gentoo.org/552814 Bug: https://bugs.gentoo.org/599008
* rsync: quarantine data prior to verification (bug 660410)Zac Medico2018-07-092-9/+85
| | | | | | | | | | | | | | | | | | | | | | | | | Sync into a quarantine subdirectory, using the rsync --link-dest option to create hardlinks to identical files in the previous snapshot of the repository. If hardlinks are not supported, then show a warning message and sync directly to the normal repository location. If verification succeeds, then the quarantine subdirectory is synced to the normal repository location, and the quarantine subdirectory is deleted. If verification fails, then the quarantine directory is preserved for purposes of analysis. Even if verification happens to be disabled, the quarantine directory is still useful for making the repository update more atomic, so that it is less likely that normal repository location will be observed in a partially synced state. The new behavior may conflict with configurations that restrict the use of hardlinks, such as overlay filesystems. Therefore, users will have to set "sync-allow-hardlinks = no" in repos.conf if they have a configuration that prevents the use of hardlinks, but this should not be very common. Bug: https://bugs.gentoo.org/660410
* GitSync: add key refresh retry (bug 660732)Zac Medico2018-07-091-4/+3
| | | | Bug: https://bugs.gentoo.org/660732
* SyncBase: split out _refresh_keys method (bug 660732)Zac Medico2018-07-092-32/+42
| | | | | | | Split out a _refresh_keys method from the RsyncSync class, so GitSync can use it for retry support. Bug: https://bugs.gentoo.org/660732
* GitSync: fix verification of remote branchZac Medico2018-07-081-2/+2
| | | | Fixes: 3cd8cf93abb6 ("GitSync: abort checkout for signature problem (bug 660372)")
* GitSync: abort checkout for signature problem (bug 660372)Zac Medico2018-07-081-5/+34
| | | | | | | | | Fetch the upstream remote and use git merge to update the checkout only after successful verification of the upstream head. Suggested-by: Richard Freeman <rich0@gentoo.org> Reviewed-by: Arfrever Frehtes Taifersar Arahesis <Arfrever@Apache.Org> Bug: https://bugs.gentoo.org/660372
* test_sync_local: fix GitSync coverageZac Medico2018-07-051-6/+9
| | | | Fixes: 0655b4a26e37 ("test_sync_local: add test for auto-sync set to 'no'")
* depgraph._show_unsatisfied_blockers: AttributeError unevaluated_atom (bug ↵Zac Medico2018-06-281-1/+1
| | | | | | | | | 659564) Only attempt to access the unevaluated_atom attribute for package atoms, since soname atoms do not have this attribute. Bug: https://bugs.gentoo.org/659564
* {,PKG_}INSTALL_MASK: support trailing slash (bug 658322)Zac Medico2018-06-262-2/+134
| | | | | | | | | Fix the python INSTALL_MASK implementation so that a trailing slash matches a directory, for compatibility with the previous bash implementation. Fixes: 3416876c0ee7 ("{,PKG_}INSTALL_MASK: python implementation") Bug: https://bugs.gentoo.org/658322
* AsyncioEventLoop: exit after unhandled exception (bug 658684)Zac Medico2018-06-261-0/+31
| | | | | | | | | | | | Fix portage commands to exit immediately for any unhandled exceptions that are raised while the asyncio event loop is running without a tty. If we have a tty then start the debugger, since in might aid in diagnosis of the problem. In order to avoid potential interference with API consumers, do not call set_exception_handler unless portage._internal_caller is True. Bug: https://bugs.gentoo.org/658684
* binarytree._merge_pkgindex_header: deduplicate values (bug 657422)Zac Medico2018-06-261-1/+3
| | | | | | | | | Deduplicate values of implicit IUSE variables, in order to prevent them from growing without bound, triggering "[Errno 7] Argument list too long" errors as reported in bug 657422. Fixes: cab78dba98c4 ("emerge --usepkgonly: propagate implicit IUSE and USE_EXPAND (bug 640318)") Bug: https://bugs.gentoo.org/657422
* SpawnProcess._unregister: cancel PipeLogger (bug 658806)Zac Medico2018-06-231-0/+3
| | | | | | | | | | If the PipeLogger has not closed itself before the _unregister method has been called, then it must be cancelled so that it will close the log immediately. This fixes the case reported in bug 658806, where an unclosed PipeLogger instance left the log file with a temporarily incomplete gzip stream. Bug: https://bugs.gentoo.org/658806
* Revert "_post_src_install_uid_fix: allow files with portage group ↵Ulrich Müller2018-06-231-6/+3
| | | | | | | | | permissions (bug 600804)" This reverts commit f479a4cdcac5db92231f489f232f10eb934c6f12. Acked by zmedico in #gentoo-portage. Bug: https://bugs.gentoo.org/600804
* AbstractEbuildProcess: handle SIGINT during dir lock (bug 657436)Zac Medico2018-06-051-0/+7
| | | | | | | When the build dir lock is interrupted by SIGINT/TERM, avoid raising an unwanted CancelledError. Bug: https://bugs.gentoo.org/657436
* _unlock_builddir_exit: fix cancel/returncode interactionZac Medico2018-06-053-6/+10
| | | | | | | | | | When the _unlock_builddir_exit returncode parameter has not been specified, do not attempt to cancel the current task, since the task has already completed. This mirrors logic added to the Binpkg _unlock_builddir_exit method in the previous commit. Fixes: 937d0156aa06 ("CompositeTask: handle SIGINT/TERM cancelled futures (bug 657436)") Bug: https://bugs.gentoo.org/657436
* CompositeTask: handle SIGINT/TERM cancelled futures (bug 657436)Zac Medico2018-06-058-1/+68
| | | | | | | | | | | In order to avoid raising an unwanted CancelledError, make CompositeTask check for cancelled futures before attempting to call future.result(). The intention is only to handle failures triggered by SIGINT/TERM, since other types of expected failures should always be handled by catching the exception raised from future.result(). Bug: https://bugs.gentoo.org/657436
* emerge --depclean: 'str' has no attribute 'soname' (bug 657420)Zac Medico2018-06-051-1/+16
| | | | | | | | Convert str to Atom, in order to avoid an AttributeError in the DbapiProvidesIndex.match method. Also, add comment explaining the reason for _unicode(atom) usage here, since it's not obvious. Bug: https://bugs.gentoo.org/657420
* emerge -pf: spawn pkg_nofetch asynchronously (bug 657360)Zac Medico2018-06-043-11/+25
| | | | | | | | For pretend mode, fix doebuild to skip the spawn_nofetch call that would trigger event loop recursion, and spawn pkg_nofetch asynchronously. Bug: https://bugs.gentoo.org/657360
* emerge: make --nodeps disable --dynamic-deps (bug 656942)Zac Medico2018-05-311-1/+1
| | | | | | | | | | Since --nodeps disable's the depgraph's _dynamic_deps_preload code, calls to BlockerDB can trigger inappropriate calls to the FakeVartree _aux_get_wrapper method, triggering event loop recursion. Therefore, make --nodeps disable --dynamic-deps, in order to eliminate the inappropriate _aux_get_wrapper calls. Bug: https://bugs.gentoo.org/656942
* PiperReader._start: pass file to _array_output_handlerZac Medico2018-05-261-11/+6
|
* PiperReader._start: handle case where fd is 0Zac Medico2018-05-261-1/+1
|
* AbstractPollTask._read_array: document performanceZac Medico2018-05-261-9/+3
| | | | | | | | A benchmark that copies bytes from /dev/zero to /dev/null shows that arrays give a 15% performance improvement for Python 2.7.14. However, arrays significantly *decrease* performance for Python 3. Also, remove obsolete documentation about the POLLIN bit.
* PipeReaderTestCase: wait for consumerZac Medico2018-05-261-0/+1
|
* EventLoop.close: fix 'AttributeError: close' for python2Zac Medico2018-05-261-1/+1
| | | | | | | | For python2 without epoll, fix handling of missing 'close' attribute on self._poll_obj. Fixes: 4095be74985c ("Add ForkExecutor (bug 649588)") Reported-by: Brian Evans <grknight@gentoo.org>
* PipeReaderTestCase: cover sockets and named pipesZac Medico2018-05-261-25/+49
|
* prepare_build_dirs: convert PermissionError to PermissionDenied (bug 656542)Zac Medico2018-05-251-1/+4
| | | | | | | | | | This supresses display of a big PermissionError traceback for bin/ebuild, like in https://bugs.gentoo.org/656542#c0, and also suppresses additional messages that may occur when the unhandled PermissionError prevents bin/ebuild from cleanly closing the asyncio event loop. Bug: https://bugs.gentoo.org/656542
* SonameDepsProcessor: fix 'set' object has no attribute 'items' (bug 656492)Zac Medico2018-05-251-4/+5
| | | | | | | | Use collections.defaultdict to ensure that requires_map contains correct defaults. Fixes: 1364cd44e7a6 ("SonameDepsProcessor: handle internal libs without DT_SONAME (bug 646190)") Bug: https://bugs.gentoo.org/656492
* AsyncioEventLoop: remove redundant set_wakeup_fd callZac Medico2018-05-241-2/+0
|
* AsyncioEventLoop: suppress BlockingIOError warning (bug 655656)Zac Medico2018-05-242-7/+106
| | | | | | | | | | Override AbstractEventLoop.run_until_complete() to prevent BlockingIOError from occurring when the event loop is not running, by using signal.set_wakeup_fd(-1) to temporarily disable the wakeup fd. In order to avoid potential interference with API consumers, only modify wakeup fd when portage._interal_caller is True. Bug: https://bugs.gentoo.org/655656
* SonameDepsProcessor: handle internal libs without DT_SONAME (bug 646190)Zac Medico2018-05-244-3/+66
| | | | | | | | | | | | | | | | | | | | | | | | | Packages like ebtables have internal libraries that lack a DT_SONAME field in their ELF header. Consumers of these internal libraries have DT_RUNPATH entries that refer to the directory containing the internal libraries. For library dependencies that are satisfied by internal libraries like this, it is inappropriate for SonameDepsProcessor to include these depenedencies in the REQUIRES metadata, therefore fix SonameDepsProcessor to automatically detect this case and exclude these dependencies from the REQUIRES metadata. This solves incorrect reporting of broken soname dependencies like the following: $ emerge -p --depclean --ignore-soname-deps=n Calculating dependencies... done! * Broken soname dependencies found: * * x86_64: libebt_redirect.so required by: * net-firewall/ebtables-2.0.10.4 * * x86_64: libebt_log.so required by: * net-firewall/ebtables-2.0.10.4 Bug: https://bugs.gentoo.org/646190
* doebuild: eliminate redundant aux_get calls (bug 656394)Zac Medico2018-05-241-6/+13
| | | | | | | These redundant aux_get calls triggered event loop recursion when called via EbuildFetchonly as reported in bug 656394. Bug: https://bugs.gentoo.org/656394
* preinst_selinux_labels: disable LD_PRELOAD sandbox (bug 655996)Zac Medico2018-05-213-9/+55
| | | | | | | | Since SELinux does not allow LD_PRELOAD across domain transitions, disable the LD_PRELOAD sandbox for preinst_selinux_labels. Bug: https://bugs.gentoo.org/655996 Tested-by: Jason Zaman <perfinion@gentoo.org>
* MergeProcess,spawn: unregister SIGCHLD and wakeup_fd (bug 655656)Zac Medico2018-05-172-0/+20
| | | | | | | | | In order to prevent forked processes from invoking the parent process's SIGCHLD handler and writing to wakeup_fd (triggering BlockingIOError), unregister the SIGCHLD and wakeup_fd. Bug: https://bugs.gentoo.org/655656 Reported-by: Helmut Jarausch <jarausch@igpm.rwth-aachen.de>
* ForkProcess: unregister SIGCHLD and wakeup_fd (bug 655656)Zac Medico2018-05-141-0/+10
| | | | | | | | | In order to prevent forked processes from invoking the parent process's SIGCHLD handler and writing to wakeup_fd (triggering BlockingIOError), unregister the SIGCHLD and wakeup_fd. Bug: https://bugs.gentoo.org/655656 Reported-by: Helmut Jarausch <jarausch@igpm.rwth-aachen.de>
* AsyncioEventLoop: suppress BlockingIOError warning during loop close (bug ↵Zac Medico2018-05-131-1/+9
| | | | | | | | | | | 655656) Disable the asyncio event loop's SIGCHLD handler before attempting to close it, in order to suppress a harmless BlockingIOError warning during loop close. Closes: https://bugs.gentoo.org/655656 Reported-by: Helmut Jarausch <jarausch@igpm.rwth-aachen.de>
* _EbuildFetcherProcess: fix event loop recursion (bug 655378)Zac Medico2018-05-121-1/+15
| | | | | | | Get SRC_URI metadata asynchronously in order to avoid event loop recursion in doebuild_environment. Bug: https://bugs.gentoo.org/655378
* DefaultEventLoopPolicy: test NotImplementedError due to recursionZac Medico2018-05-091-0/+29
|