Get Gentoo!
gentoo.org sites
gentoo.org Wiki Bugs Forums Packages
Planet Archives Sources
Infra Status
aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Fix 'unused-import' for stdlib imports.Alec Warner2020-07-2286-94/+4
| | | | | | | | | | | This is part of a cleanup to enable pylint. This is only stdlib modules; there are hundreds of unused imports of other modules but its less clear how safe those are to remove due to side effects. Signed-off-by: Alec Warner <antarus@gentoo.org> Change-Id: If9bee83bbbe1620b34a0f8d153637909af2b8512 Closes: https://github.com/gentoo/portage/pull/585 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* _EbuildFetcherProcess: emit eerror for fetch failure in _proc_join_doneZac Medico2020-07-221-8/+4
| | | | | | | | | | | Since ForkProcess now receives process exit status in the _proc_join_done method instead of the _async_waitpid_cb method, _EbuildFetcherProcess needs to emit eerror messages there instead. Fixes: bde44b75407d ("ForkProcess: replace os.fork with multiprocessing.Process (bug 730192)") Bug: https://bugs.gentoo.org/730192 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* MergeProcess: handle cancelled future in _proc_join_doneZac Medico2020-07-221-1/+1
| | | | | | | | If the future was cancelled, then avoid a possible ValueError when accessing proc.exitcode. Fixes: f587ebf3d492 ("MergeProcess: handle RETURNCODE_POSTINST_FAILURE in _proc_join_done") Signed-off-by: Zac Medico <zmedico@gentoo.org>
* MergeProcess: handle RETURNCODE_POSTINST_FAILURE in _proc_join_doneZac Medico2020-07-221-5/+4
| | | | | | | | | | Since ForkProcess now receives process exit status in the _proc_join_done method instead of the _async_waitpid_cb method, MergeProcess needs to handle RETURNCODE_POSTINST_FAILURE there instead. Fixes: 3561071e07ad ("MergeProcess: replace os.fork with multiprocessing.Process (bug 730192)") Signed-off-by: Zac Medico <zmedico@gentoo.org>
* MergeProcess: replace os.fork with multiprocessing.Process (bug 730192)Zac Medico2020-07-221-76/+30
| | | | | | | | | Fix the MergeProcess _spawn method to call the superclass _spawn method, in order to replace os.fork with multiprocessing.Process, promoting a healthy state for the forked interpreter. Bug: https://bugs.gentoo.org/730192 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Fix R0205 in repoman.Alec Warner2020-07-2222-22/+22
| | | | | | | | Do not inherit from object in py3. Signed-off-by: Alec Warner <antarus@gentoo.org> Change-Id: I52c8f46e16fa1dc9c2b90a6c9a5cdf4a417050a0 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Remove 2.7 from runtests.Alec Warner2020-07-221-1/+0
| | | | | | | | | With this branch we will unable to pass 2.7 as the old style classes are gone. Signed-off-by: Alec Warner <antarus@gentoo.org> Change-Id: I498bd6f4d7b674a0b19d56b70940d9e7cbfc719c Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Fix R0205 in _emerge.Alec Warner2020-07-2226-45/+45
| | | | | | | | Inheriting from object is not required in py3. Signed-off-by: Alec Warner <antarus@gentoo.org> Change-Id: Ia12500d590ef3b7f5852b6af03801d40a8af8fa2 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Fix pylint R0205.Alec Warner2020-07-22100-160/+160
| | | | | | | | In python3 inheriting from object is no longer required. Signed-off-by: Alec Warner <antarus@gentoo.org> Change-Id: Id489d4103d975a751be148205694cb6a20d33b31 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* syncbase: Add back missing functools importZac Medico2020-07-201-0/+1
| | | | | | Fixes: 788c0e8bb07b ("Remove from __future__ import unicode_literals") Bug: https://bugs.gentoo.org/733370 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* EbuildFetchTestCase: fix PORTAGE_DEPCACHEDIR leakage from calling envZac Medico2020-07-191-2/+2
| | | | | Reported-by: Sam James <sam@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Updates for portage-3.0.0 releaseportage-3.0.0Zac Medico2020-07-183-1/+18
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* ForkProcess: replace os.fork with multiprocessing.Process (bug 730192)Zac Medico2020-07-183-35/+119
| | | | | | | | | | | | | | | | Replace os.fork with multiprocessing.Process, in order to leverage any pre-fork and post-fork interpreter housekeeping that it provides, promoting a healthy state for the forked interpreter. Since multiprocessing.Process closes sys.__stdin__, fix relevant code to use the portage._get_stdin() compatibility function. In case there's a legitimate need to inherit stdin for things like PROPERTIES=interactive support, create a temporary duplicate of fd_pipes[0] when appropriate, and restore sys.stdin and sys.__stdin__ in the subprocess. Bug: https://bugs.gentoo.org/730192 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* AsyncFunction: stdin supportZac Medico2020-07-182-2/+40
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* SubProcess._async_waitpid: document override usageZac Medico2020-07-181-3/+6
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* validate_desktop_entry: update _trivial_warnings regex (bug 733154)Zac Medico2020-07-181-1/+6
| | | | | | | | The message has changed in desktop-file-utils-0.25. See: https://gitlab.freedesktop.org/xdg/desktop-file-utils/-/commit/e935a1b0c300d8fc97661e01a8200af14876e627 Bug: https://bugs.gentoo.org/733154 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Remove py<3.4 import compatibilityMichał Górny2020-07-1825-224/+61
| | | | | | Closes: https://github.com/gentoo/portage/pull/577 Reviewed-by: Zac Medico <zmedico@gentoo.org> Signed-off-by: Michał Górny <mgorny@gentoo.org>
* Correct inverted py3 json indentationMichał Górny2020-07-172-2/+2
| | | | Signed-off-by: Michał Górny <mgorny@gentoo.org>
* Remove support code for Python < 3.4Michał Górny2020-07-1713-163/+20
| | | | | | Reviewed-by: Zac Medico <zmedico@gentoo.org> Closes: https://github.com/gentoo/portage/pull/576 Signed-off-by: Michał Górny <mgorny@gentoo.org>
* Remove support code for Python < 3.3Michał Górny2020-07-178-117/+39
| | | | Signed-off-by: Michał Górny <mgorny@gentoo.org>
* Remove support code for Python < 3.2Michał Górny2020-07-173-21/+8
| | | | Signed-off-by: Michał Górny <mgorny@gentoo.org>
* Clean up more py2 conditional codeMichał Górny2020-07-1716-148/+52
| | | | | | Closes: https://github.com/gentoo/portage/pull/575 Reviewed-by: Zac Medico <zmedico@gentoo.org> Signed-off-by: Michał Górny <mgorny@gentoo.org>
* Update copyright headers after previous changeMichał Górny2020-07-169-9/+9
| | | | Signed-off-by: Michał Górny <mgorny@gentoo.org>
* Eliminate the most of explicit py3 conditionalsMichał Górny2020-07-1639-453/+102
| | | | | | | | | Eliminate the most of py2/py3 conditions in the code. Leave a few where the relevant code is unclear, they will be addressed later. Reviewed-by: Zac Medico <zmedico@gentoo.org> Closes: https://github.com/gentoo/portage/pull/574 Signed-off-by: Michał Górny <mgorny@gentoo.org>
* BinTestCase: export PORTAGE_PYTHONZac Medico2020-07-151-0/+2
| | | | | | | This solves doins test failures triggered in travis-ci when doins executed with /usr/bin/python instead of PORTAGE_PYTHON. Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Eliminate basestring/long/_unicode py3 compatMichał Górny2020-07-1669-490/+227
| | | | | | | | Replace basestring and _unicode hacks with str, and long with int. Reviewed-by: Zac Medico <zmedico@gentoo.org> Closes: https://github.com/gentoo/portage/pull/570 Signed-off-by: Michał Górny <mgorny@gentoo.org>
* Remove from __future__ import unicode_literalsZac Medico2020-07-1593-156/+30
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Remove unnecessary time.monotonic() compatMichał Górny2020-07-144-45/+10
| | | | | | | | | time.monotonic() is available since py3.3, so there's no need for the compat anymore. Reviewed-by: Zac Medico <zmedico@gentoo.org> Closes: https://github.com/gentoo/portage/pull/569 Signed-off-by: Michał Górny <mgorny@gentoo.org>
* Add caching to use_reduce, vercmp, and catpkgsplitChun-Yu Shei2020-07-132-43/+67
| | | | | | | | | | | | | | | | Each of these functions is called repeatedly with the same arguments many times. Cache sizes were selected to minimize memory use increase, while still providing about the same speedup compared to a cache with unbounded size. "emerge -uDvpU --with-bdeps=y @world" runtime decreases from 44.32s -> 29.94s -- a 48% speedup, while the maximum value of the RES column in htop increases from 280 MB -> 290 MB. "emerge -ep @world" time slightly decreases from 18.77s -> 17.93, while max observed RES value actually decreases from 228 MB -> 214 MB (similar values observed across a few before/after runs). Bug: https://bugs.gentoo.org/732378 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* quickpkg: split tokens in BINPKG_COMPRESS_FLAGSZac Medico2020-07-121-1/+1
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* man/ebuild.5: Fix $P and $PN exampleFlorian Schmaus2020-07-071-2/+2
| | | | | | | | | | | | | | | | | | | | | Both variables do *not* include the package's category. So $PN, instead of x11-base/xorg-server returns xorg-server Same goes for $P. This commit fixes the error which was introduced by 84b9b5c2a ("man/ebuild.5: Improvements to the ebuild(5) man page"). Bug: https://bugs.gentoo.org/731246 Closes: https://github.com/gentoo/portage/pull/567 Signed-off-by: Florian Schmaus <flo@geekplace.eu> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* travis.yml: drop python 2.7 (bug 731114)Zac Medico2020-07-064-7/+4
| | | | | | | | | It should be pretty safe to drop support for python2.7 at this point. Bug: https://bugs.gentoo.org/731114 Reviewed-by: Brian Dolbec <dolsen@gentoo.org> Reviewed-by: Michał Górny <mgorny@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* git: Verify boolean values passed to sync-git-verify-commit-signatureWynn Wolf Arbor2020-07-021-1/+12
| | | | | | | | | | | | | Currently, if 'sync-git-verify-commit-signature' is set to anything other than 'yes', 'no', 'true', or 'false', its value is ignored silently and nothing is verified because the option defaults to 'false'. Introduce a check to CheckGitConfig that warns the user if their input is invalid. Bug: https://bugs.gentoo.org/703698 Signed-off-by: Wynn Wolf Arbor <wolf@oriole.systems> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* repos.conf: Use consistent list of values for boolean optionsWynn Wolf Arbor2020-07-024-32/+29
| | | | | | | | | | | | | | | | | | Valid values for boolean options in repos.conf are currently not managed in a consistent manner. Some options only support 'true' and 'false', whilst others additionally support 'yes' and 'no'. Using the latter forms on options that do not support them will lead to unexpected behaviour. For example, an option checking for 'true' will be disabled when 'yes' is used. This is counter-intuitive and adds additional burden: the user has to look up in the manual which form is accepted by which option. Have all boolean options consistently accept 'yes', 'no', 'true', and 'false' and make sure to document this in the portage(5) manual. Additionally, document the default value for each. Signed-off-by: Wynn Wolf Arbor <wolf@oriole.systems> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* ecompress: optimize docompress -x precompressed comparisonZac Medico2020-07-022-13/+15
| | | | | | | | | | | Use sort and comm with temporary files in order to compare lists of docompress -x and precompressed files, since the file lists can be extremely large. Also strip ${D%/} from paths in order to reduce length. Bug: https://bugs.gentoo.org/721516 Suggested-by: Robin H. Johnson <robbat2@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* man: fix syntax for page refsMike Frysinger2020-06-302-6/+6
| | | | Signed-off-by: Mike Frysinger <vapier@gentoo.org>
* xtermTitle: support konsoleZac Medico2020-06-271-1/+1
| | | | | | Bug: https://bugs.gentoo.org/729852 Reported-by: rnddim@gmail.com Signed-off-by: Zac Medico <zmedico@gentoo.org>
* ecompress: fix precompressed false positive for docompress -x <directory>Zac Medico2020-06-241-1/+1
| | | | | | | | Use ${REPLY%.ecompress} to correctly strip the .ecompress suffix. Fixes: 5508bf7a6db5 ("ecompress: ignore docompress -x files in precompressed QA check (bug 721516)") Bug: https://bugs.gentoo.org/721516 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Updates for portage-2.3.103 releaseportage-2.3.103Zac Medico2020-06-232-2/+9
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* SpawnProcess._async_wait: allow _main_task to exit normallyZac Medico2020-06-231-0/+11
| | | | | | | | | | Unnecessary cancellation of _main_task triggered this ResourceWarning for the ebuild(1) fetch command, since it prevented cleanup of BuildLogger's internal PipeLogger instance: sys:1: ResourceWarning: unclosed file <_io.FileIO name=10 mode='rb' closefd=True> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* PipeLogger._unregister: sanity check for closed file or loopZac Medico2020-06-231-5/+6
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* ecompress: fix "Argument list too long" for sed (bug 727522)Patrick McLean2020-06-231-2/+3
| | | | | | | | | Use sed -f to feed commands to sed via stdin, in order to avoid the "Argument list too long" error reported in bug 727522. Fixes: 5508bf7a6db5 ("ecompress: ignore docompress -x files in precompressed QA check (bug 721516)") Bug: https://bugs.gentoo.org/727522 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* EbuildPhase._elog: _set_nonblocking(build_logger.stdin.fileno())Zac Medico2020-06-231-1/+3
| | | | | Fixes: dd69ce742c62 ("Support PORTAGE_LOG_FILTER_FILE_CMD (bug 709746)") Signed-off-by: Zac Medico <zmedico@gentoo.org>
* repoman: Update for a repoman-2.3.23 releaserepoman-2.3.23Zac Medico2020-06-222-1/+6
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Updates for portage-2.3.102 releaseportage-2.3.102Zac Medico2020-06-222-1/+7
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Support PORTAGE_LOG_FILTER_FILE_CMD (bug 709746)Zac Medico2020-06-2210-30/+243
| | | | | | | | | | | | | | | | | | | | This variable specifies a command that filters build log output to a log file. The plan is to extend this to support a separate filter for tty output in the future. In order to enable the EbuildPhase class to write elog messages to the build log with PORTAGE_LOG_FILTER_FILE_CMD support, convert its _elog method to a coroutine, and add a SchedulerInterface async_output method for it to use. Use a new BuildLogger class to manage log output (with or without a filter command), with compression support provided by PipeLogger. BuildLogger has a stdin property which provides access to a writable binary file stream (refers to a pipe) that log content is written to. Bug: https://bugs.gentoo.org/709746 Reviewed-by: Brian Dolbec <dolsen@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* PipeLogger: non-blocking write to pipe (bug 709746)Zac Medico2020-06-222-17/+116
| | | | | | | | | | | | Add support to write to a non-blocking pipe instead of a log file. This is needed for the purposes of bug 709746, where PipeLogger will write to a pipe that is drained by anoher PipeLogger instance which is running in the same process. Bug: https://bugs.gentoo.org/709746 Reviewed-by: Brian Dolbec <dolsen@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* _writer: fix unsafe finally clause (bug 728580)Zac Medico2020-06-182-28/+33
| | | | | | | | | | | | | | In the coroutine finally clause, do not call remove_writer in cases where fd has been closed and then re-allocated to a concurrent coroutine as in bug 716636. Also, assume that the caller will put the file in non-blocking mode and close the file when done, so that this function is suitable for use within a loop. Bug: https://bugs.gentoo.org/728580 Reviewed-by: Brian Dolbec <dolsen@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* repoman: deprecate netsurf.eclass.Michael Orlitzky2020-06-161-0/+1
| | | | | | | | | | | | | | | | | | | | While investigating bug 489542, it became clear that the netsurf eclass is deprecated in spirit if not in fact. All of the newer netsurf ebuilds use a custom function loaded from a script that is shipped in netsurf-buildsystem's $FILESDIR to do (some of) what the eclass used to do. That function probably does belong in an eclass, but at this point, we should throw this thing out and start from scratch. By deprecating the eclass, we make sure that no one else inherits it during the time it takes to purge the two remaining consumers. Then, once those are gone, the build system script magic can be put back into an eclass, and its consumers updated one-at-a-time. Bug: https://bugs.gentoo.org/489542 Bug: https://bugs.gentoo.org/637824 Signed-off-by: Michael Orlitzky <mjo@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Revert "PipeLogger: non-blocking write to pipe (bug 709746)"Zac Medico2020-06-152-114/+13
| | | | | | | | This reverts commit 3e46825a047067a96ed997fe394f85e042e542a8. We've had reports of emerge hangs, so reverting this for now. Bug: https://bugs.gentoo.org/709746 Signed-off-by: Zac Medico <zmedico@gentoo.org>