aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
...
* lib/_emerge/Package.py: drop unused-importAaron Bauman2020-08-031-1/+0
| | | | | Signed-off-by: Aaron Bauman <bman@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* lib/_emerge/resolver/output.py: fix unused-importAaron Bauman2020-08-031-2/+0
| | | | | Signed-off-by: Aaron Bauman <bman@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* lib/_emerge/DependencyArg.py: fix unused-importAaron Bauman2020-08-031-3/+0
| | | | | Signed-off-by: Aaron Bauman <bman@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* lib/_emerge/Scheduler.py: fix unused-importAaron Bauman2020-08-031-2/+0
| | | | | Signed-off-by: Aaron Bauman <bman@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* lib/_emerge/resolver/slot_collision.py: fix redefined-builtin W0622Aaron Bauman2020-08-021-37/+37
| | | | | | | | | * This fixes the referenced warning by renaming the 'id' variable to 'name' and the 'type' variable to 'atype' * Additional cosmetic/style changes found along the way. Signed-off-by: Aaron Bauman <bman@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Fix R1705.Alec Warner2020-07-2910-129/+119
| | | | | | | | | | | | | | | | | | | | | | The codebase appears to have this pattern like: if foo == bar: return a elif foo == baz: return b else: return c This can often be rewritten as: if foo == bar: return a if foo == baz: return b return c Closes: https://github.com/gentoo/portage/pull/592 Signed-off-by: Alec Warner <antarus@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Fix redefined-builtin errors for pylint.Alec Warner2020-07-237-20/+20
| | | | | | | | | | | Some of these are simple variable renames. A few of the uses were refactored and some were simply disabled; often due to redefinition as part of a function signature. I did not do any research in terms of API changes so these were typically left as-is. Closes: https://github.com/gentoo/portage/pull/587 Signed-off-by: Alec Warner <antarus@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Fix C0305, trailing newlines.Alec Warner2020-07-2218-18/+0
| | | | | | | | | | | pylint --disable=all --enable=C0305 | awk -F: '{print $1}' | xargs sed -i -e '$d' Removes the trailing newlines. Signed-off-by: Alec Warner <antarus@gentoo.org> Change-Id: I18db6ec682192ab44e07bad00c14f316caf43c0e Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Fix 'unused-import' for stdlib imports.Alec Warner2020-07-2220-24/+0
| | | | | | | | | | | This is part of a cleanup to enable pylint. This is only stdlib modules; there are hundreds of unused imports of other modules but its less clear how safe those are to remove due to side effects. Signed-off-by: Alec Warner <antarus@gentoo.org> Change-Id: If9bee83bbbe1620b34a0f8d153637909af2b8512 Closes: https://github.com/gentoo/portage/pull/585 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* _EbuildFetcherProcess: emit eerror for fetch failure in _proc_join_doneZac Medico2020-07-221-8/+4
| | | | | | | | | | | Since ForkProcess now receives process exit status in the _proc_join_done method instead of the _async_waitpid_cb method, _EbuildFetcherProcess needs to emit eerror messages there instead. Fixes: bde44b75407d ("ForkProcess: replace os.fork with multiprocessing.Process (bug 730192)") Bug: https://bugs.gentoo.org/730192 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Fix R0205 in _emerge.Alec Warner2020-07-2226-45/+45
| | | | | | | | Inheriting from object is not required in py3. Signed-off-by: Alec Warner <antarus@gentoo.org> Change-Id: Ia12500d590ef3b7f5852b6af03801d40a8af8fa2 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* SubProcess._async_waitpid: document override usageZac Medico2020-07-181-3/+6
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Remove py<3.4 import compatibilityMichał Górny2020-07-182-9/+2
| | | | | | Closes: https://github.com/gentoo/portage/pull/577 Reviewed-by: Zac Medico <zmedico@gentoo.org> Signed-off-by: Michał Górny <mgorny@gentoo.org>
* Remove support code for Python < 3.4Michał Górny2020-07-175-78/+4
| | | | | | Reviewed-by: Zac Medico <zmedico@gentoo.org> Closes: https://github.com/gentoo/portage/pull/576 Signed-off-by: Michał Górny <mgorny@gentoo.org>
* Update copyright headers after previous changeMichał Górny2020-07-166-6/+6
| | | | Signed-off-by: Michał Górny <mgorny@gentoo.org>
* Eliminate the most of explicit py3 conditionalsMichał Górny2020-07-1612-78/+7
| | | | | | | | | Eliminate the most of py2/py3 conditions in the code. Leave a few where the relevant code is unclear, they will be addressed later. Reviewed-by: Zac Medico <zmedico@gentoo.org> Closes: https://github.com/gentoo/portage/pull/574 Signed-off-by: Michał Górny <mgorny@gentoo.org>
* Eliminate basestring/long/_unicode py3 compatMichał Górny2020-07-1614-112/+52
| | | | | | | | Replace basestring and _unicode hacks with str, and long with int. Reviewed-by: Zac Medico <zmedico@gentoo.org> Closes: https://github.com/gentoo/portage/pull/570 Signed-off-by: Michał Górny <mgorny@gentoo.org>
* Remove from __future__ import unicode_literalsZac Medico2020-07-1517-28/+6
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* SpawnProcess._async_wait: allow _main_task to exit normallyZac Medico2020-06-231-0/+11
| | | | | | | | | | Unnecessary cancellation of _main_task triggered this ResourceWarning for the ebuild(1) fetch command, since it prevented cleanup of BuildLogger's internal PipeLogger instance: sys:1: ResourceWarning: unclosed file <_io.FileIO name=10 mode='rb' closefd=True> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* EbuildPhase._elog: _set_nonblocking(build_logger.stdin.fileno())Zac Medico2020-06-231-1/+3
| | | | | Fixes: dd69ce742c62 ("Support PORTAGE_LOG_FILTER_FILE_CMD (bug 709746)") Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Support PORTAGE_LOG_FILTER_FILE_CMD (bug 709746)Zac Medico2020-06-225-24/+90
| | | | | | | | | | | | | | | | | | | | This variable specifies a command that filters build log output to a log file. The plan is to extend this to support a separate filter for tty output in the future. In order to enable the EbuildPhase class to write elog messages to the build log with PORTAGE_LOG_FILTER_FILE_CMD support, convert its _elog method to a coroutine, and add a SchedulerInterface async_output method for it to use. Use a new BuildLogger class to manage log output (with or without a filter command), with compression support provided by PipeLogger. BuildLogger has a stdin property which provides access to a writable binary file stream (refers to a pipe) that log content is written to. Bug: https://bugs.gentoo.org/709746 Reviewed-by: Brian Dolbec <dolsen@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Revert "Support PORTAGE_LOG_FILTER_FILE_CMD (bug 709746)"Zac Medico2020-06-155-90/+24
| | | | | | | | This reverts commit ab7556a5c0df418d61148a05347f4c73115e5747. We've had reports of emerge hangs, so reverting this for now. Bug: https://bugs.gentoo.org/709746 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Support PORTAGE_LOG_FILTER_FILE_CMD (bug 709746)Zac Medico2020-06-145-24/+90
| | | | | | | | | | | | | | | | | | | This variable specifies a command that filters build log output to a log file. The plan is to extend this to support a separate filter for tty output in the future. In order to enable the EbuildPhase class to write elog messages to the build log with PORTAGE_LOG_FILTER_FILE_CMD support, convert its _elog method to a coroutine, and add a SchedulerInterface async_output method for it to use. Use a new BuildLogger class to manage log output (with or without a filter command), with compression support provided by PipeLogger. BuildLogger has a stdin property which provides access to a writable binary file stream (refers to a pipe) that log content is written to. Bug: https://bugs.gentoo.org/709746 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* depgraph: respect <foo-version:= for slot operator rebuild (bug 717140)Zac Medico2020-04-111-3/+9
| | | | | | | | | | | | | | When searching for slot operator rebuilds, respect non slot-operator components of parent dependencies, so that a <foo-version:= dependency like the <dev-libs/libgit2-1:0=[ssh?] dependency from bug 717140 will not be completely ignored. This will prevent erroneous attempts to trigger slot operator rebuilds for upgrades that would break <foo-version:= dependencies (which triggered upgrade/downgrade loops when backtracking tried to resolve the breakage). Fixes: d569a2d7275c ("_slot_operator_update_probe: fix bug #508762") Bug: https://bugs.gentoo.org/717140 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* _schedule_merge_wakeup: handle main loop exitZac Medico2020-04-091-1/+2
| | | | | | | | | | Detect main loop exit and do not attempt to schedule in this case. Fixes: 389429d798a1 ("Scheduler: wakeup for empty merge queue (bug 711322)") Reported-by: Rick Farina <zerochaos@gentoo.org> Bug: https://bugs.gentoo.org/711322 Bug: https://bugs.gentoo.org/716636 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Scheduler: wakeup for empty merge queue (bug 711322)Zac Medico2020-04-082-10/+35
| | | | | | | | | | | | Add a wakeup callback to schedule a new merge when the merge queue becomes empty. This prevents the scheduler from hanging in cases where the order of _merge_exit callback invocation may cause the the merge queue to appear non-empty when it is about to become empty. Bug: https://bugs.gentoo.org/711322 Bug: https://bugs.gentoo.org/716636 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Revert "AsynchronousTask: add coroutine async_start method"Zac Medico2020-04-073-37/+9
| | | | | | | This reverts commit d66e9ec0b10522528d62e18b83e012c1ec121787. Bug: https://bugs.gentoo.org/716636 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Revert "AbstractEbuildProcess: add _async_start coroutine"Zac Medico2020-04-072-14/+27
| | | | | | | This reverts commit a287c49f84ad3af7c8e20bebd116ea972f318e04. Bug: https://bugs.gentoo.org/716636 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Revert "EbuildPhase: add _async_start coroutine"Zac Medico2020-04-072-22/+13
| | | | | | | This reverts commit c7e52d0466211907d20cdbc04f1e90e7da626694. Bug: https://bugs.gentoo.org/716636 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Revert "AsynchronousTask: Call _start_hook after _start_async"Zac Medico2020-04-071-2/+2
| | | | | | | This reverts commit 036c644712c36b2d67e818e6f60aa753ebd6acaf. Bug: https://bugs.gentoo.org/716636 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Revert "EbuildBuildDir: use async_start method"Zac Medico2020-04-071-44/+78
| | | | | | | This reverts commit 2c596f49bab63c6c81dd4d68789823d45341264d. Bug: https://bugs.gentoo.org/716636 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Revert "ForkExecutor: use async_start method"Zac Medico2020-04-071-13/+2
| | | | | | | This reverts commit 4df7a0a0c16c5ded65ad601d39840797b7704770. Bug: https://bugs.gentoo.org/716636 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Revert "SpawnProcess: use async_start method (bug 709746)"Zac Medico2020-04-076-40/+12
| | | | | | | This reverts commit 5c40c3e7ec180c9c7d1eea521d69487177c7f519. Bug: https://bugs.gentoo.org/716636 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Revert "SpawnProcess: add _main coroutine"Zac Medico2020-04-071-22/+11
| | | | | | | This reverts commit 8074127bbc213fde75d51309c8fb4ee33ad278aa. Bug: https://bugs.gentoo.org/716636 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Revert "SpawnProcess: only cancel self._main_task if it's not done"Zac Medico2020-04-071-1/+1
| | | | | | | This reverts commit 10cc6febb701cde8f7c099f5ecdf177486e94b3c. Bug: https://bugs.gentoo.org/716636 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Revert "SpawnProcess: make _main cancel pipe_logger when appropriate"Zac Medico2020-04-071-7/+2
| | | | | | | This reverts commit 6eb6500435c80837f106d3f9f9f6bb246ff00c16. Bug: https://bugs.gentoo.org/716636 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Revert "SpawnProcess: cancel pipe_logger after async_start CancelledError"Zac Medico2020-04-071-6/+1
| | | | | | | This reverts commit d331f03ab25422e9afc2e752ea8e39c93a4b0bec. Bug: https://bugs.gentoo.org/716636 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Revert "CompositeTask: call self.cancel() for _start_task CancelledError"Zac Medico2020-04-071-1/+1
| | | | | | | This reverts commit 1c681e0cc783ca241b1f2e1c782fccd8f1aac769. Bug: https://bugs.gentoo.org/716636 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Revert "EbuildBuildDir: cancel current tasks for CancelledError"Zac Medico2020-04-071-30/+16
| | | | | | | This reverts commit 899d5e61010a85702e7c83506d2214ede179c964. Bug: https://bugs.gentoo.org/716636 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Revert "Support PORTAGE_LOG_FILTER_FILE (bug 709746)"Zac Medico2020-04-073-55/+15
| | | | | | | This reverts commit 73f72f526a66b9953a46868cc1390fde2820997f. Bug: https://bugs.gentoo.org/716636 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Revert "_BinpkgFetcherProcess: fix async_lock event loop recursion (bug 711178)"Zac Medico2020-04-071-17/+15
| | | | | | | This reverts commit 1681309f252a4e91d7256b895a9af26ef82a9b30. Bug: https://bugs.gentoo.org/716636 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Revert "SpawnProcess: cancel _main_task in _unregister (bug 711174)"Zac Medico2020-04-071-2/+0
| | | | | | | This reverts commit 97587a47fcd00d070c081ad3933e4036cf0f8e81. Bug: https://bugs.gentoo.org/716636 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Revert "Rename PORTAGE_LOG_FILTER_FILE_CMD from PORTAGE_LOG_FILTER_FILE"Zac Medico2020-04-072-2/+2
| | | | | | | This reverts commit e24859eaa03ec86e10d842296f5570dd98bed4b7. Bug: https://bugs.gentoo.org/716636 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emerge --info: Filter variables for credentialsSam James (sam_c)2020-03-221-0/+4
| | | | | | | Bug: https://bugs.gentoo.org/713726 Closes: https://github.com/gentoo/portage/pull/536 Signed-off-by: Sam James (sam_c) <sam@cmpct.info> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* AsynchronousTask: in _async_wait, call self.wait() immediatelyZac Medico2020-03-221-6/+6
| | | | | | | | | | In _async_wait, call self.wait() immediately so that the stack trace for an InvalidStateError will contain useful debugging information. Since commit 46903f3e5622, exit listeners are invoked via call_soon, so it's no longer unecessary for _async_wait to invoke self.wait() via call_soon. Signed-off-by: Zac Medico <zmedico@gentoo.org>
* AsynchronousTask: include self in InvalidStateError messageZac Medico2020-03-222-2/+2
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* depgraph: respect --deep=<depth> with --update (bug 712298)Zac Medico2020-03-141-2/+6
| | | | | | | | | | | Fix the _wrapped_select_pkg_highest_available_imp method to select an installed package when appropriate for the current --deep=<depth> setting, even with --update enabled. This prevents violation of the current --deep=<depth> setting in cases where an installed package which satisfies a dependency is masked for any reason. Bug: https://bugs.gentoo.org/712298 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* SequentialTaskQueue: cancel unstarted tasks when appropriate (bug 711322)Zac Medico2020-03-072-0/+7
| | | | | | | | | | | | When the clear method is called, cancel any tasks which have not started yet, in order to ensure that their start/exit listeners are called. This fixes a case where emerge would hang after SIGINT. Also fix the CompositeTask _cancel method to react appropriately to the cancel event when the task has not started yet. Bug: https://bugs.gentoo.org/711322 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* AsynchronousTask: handle addStartListener after exit (bug 711322)Zac Medico2020-03-071-0/+8
| | | | | | | | | When addStartListener is called after the task has already exited with a returncode, immediately schedule the listener to be invoked via call_soon. This behavior is similar to the Future add_done_callback method (and addExitListener since commit 5d476c4e5002). Signed-off-by: Zac Medico <zmedico@gentoo.org>
* AsynchronousTask: simplify isAlive (bug 711688)Zac Medico2020-03-074-14/+1
| | | | | | | | | | | | | | Simplify all AsynchronousTask subclasses to use the default isAlive implementation, which returns True if self.returncode is not None. This fixes cases where the method would erroneously return False, leading to issues like bug 711688, where the CompositeTask isAlive implementation returned False for a BinpkgPrefetcher instance that was still in the process of starting via its async_start method. Fixes: d66e9ec0b105 ("AsynchronousTask: add coroutine async_start method") Bug: https://bugs.gentoo.org/711688 Signed-off-by: Zac Medico <zmedico@gentoo.org>