aboutsummaryrefslogtreecommitdiff
path: root/lib
Commit message (Collapse)AuthorAgeFilesLines
...
* Remove unused _PortageEventLoop and _PortageChildWatcherZac Medico2021-03-061-112/+0
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Removed unused portage.util.futures.eventsZac Medico2021-03-062-209/+3
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Remove unused _EventLoopFuture classZac Medico2021-03-061-155/+1
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Removed unused portage.util.futures.transportsZac Medico2021-03-062-577/+2
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Removed unused portage.util.futures._asyncio.tasksZac Medico2021-03-062-123/+19
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Remove unused PollSelectAdapter and PollConstants classesZac Medico2021-03-062-91/+0
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* bin_entry_point: handle versioned python shebangsZac Medico2021-03-061-1/+1
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Remove unused EventLoop classZac Medico2021-03-061-1153/+0
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Remove unit tests for deprecated EAPI-progressZac Medico2021-03-062-196/+0
| | | | | Fixes: 67cf9c2b0504 ("Mark EAPIs 4-python and 5-progress as deprecated") Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Mark EAPIs "4-python" and "5-progress" as deprecatedMatt Turner2021-03-061-4/+4
| | | | | Signed-off-by: Matt Turner <mattst88@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Use asyncio.subprocess.Process directlyMatt Turner2021-03-062-120/+4
| | | | | | | | With no need to support Python 2, we can remove our private implementation. Signed-off-by: Matt Turner <mattst88@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* lib: Remove outdated Python 2 commentsMatt Turner2021-03-067-30/+0
| | | | | | Fixes: 788c0e8bb ("Remove from __future__ import unicode_literals") Signed-off-by: Matt Turner <mattst88@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Remove outdated mention of Python 2 from commentMatt Turner2021-03-061-5/+5
| | | | | | Fixes: 5e9fe0f2a ("Eliminate basestring/long/_unicode py3 compat") Signed-off-by: Matt Turner <mattst88@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Remove Python 2 workaroundMatt Turner2021-03-061-5/+1
| | | | | Signed-off-by: Matt Turner <mattst88@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* _test_lock_fn: extend lock_fn to return unlock functionZac Medico2021-03-061-7/+12
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* _get_lock_fn: split out _test_lock_fnZac Medico2021-03-051-10/+17
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* bin_entry_point: rewrite python shebangs for venvZac Medico2021-03-031-1/+11
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emerge: make --binpkg-respect-use=y imply --autounmask-use=nZac Medico2021-03-022-17/+25
| | | | | | | | | If --binpkg-respect-use=y is given explicitly, then it implies --autounmask-use=n, because these options naturally oppose eachother. Bug: https://bugs.gentoo.org/773469 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Include --autounmask-backtrack in unit test for bug 773469Zac Medico2021-03-011-4/+6
| | | | | | | | | The --autounmask-backtrack option is needed in order to trigger the --binpkg-respect-use=y behavior that appears confusingly similar to --binpkg-respect-use=n behavior. Bug: https://bugs.gentoo.org/773469 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Add unit test demonstrating --autounmask-use / --binpkg-respect-use=y conflictZac Medico2021-02-281-1/+17
| | | | | Bug: https://bugs.gentoo.org/773469 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* setup.py: add entry_points when installed with pipZac Medico2021-02-281-0/+24
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emirrordist: add --content-db option required for content-hash layout (bug ↵Zac Medico2021-02-268-9/+402
| | | | | | | | | | | | | | | | | | | | | | | | | | | 756778) Add a --content-db option which is required for the content-hash layout because its file listings return content digests instead of distfile names. The content db serves to translate content digests to distfiles names, and distfiles names to content digests. All keys have one or more prefixes separated by colons. For a digest key, the first prefix is "digest" and the second prefix is the hash algorithm name. For a filename key, the prefix is "filename". The value associated with a digest key is a set of file names. The value associated with a distfile key is a set of content revisions. Each content revision is expressed as a dictionary of digests which is suitable for construction of a DistfileName instance. A given content digest will translate to multiple distfile names if multiple associations have been created via the content db add method. The relationship between a content digest and a distfile name is similar to the relationship between an inode and a hardlink. Bug: https://bugs.gentoo.org/756778 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emirrordist: support minimal object as options for use in unit testsZac Medico2021-02-261-16/+17
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* RepoConfig.config_string: don't override repos with aliases (bug 749333)Zac Medico2021-02-261-0/+2
| | | | | Bug: https://bugs.gentoo.org/749333 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* make.globals: make FEATURES=-binpkg-multi-instance sticky for existing installsZac Medico2021-02-261-0/+33
| | | | | | | | | | Add a _compat_upgrade.binpkg_multi_instance script that the ebuild can call in pkg_preinst in order to maintain a backward-compatible FEATURES=-binpkg-multi-instance default on existing installs where enabling binpkg-multi-instance could cause disruption. Bug: https://bugs.gentoo.org/772785 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* iter_parents: prevent infinite loop after empty os.path.dirname resultZac Medico2021-02-251-1/+3
| | | | | | Reported-by: dkjii@cock.li Bug: https://bugs.gentoo.org/772806 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* fetch.py: fix UnboundLocalError: local variable 'myfile'Zac Medico2021-02-251-2/+2
| | | | | | | | Reported-by: Ionen Wolkens <sudinave@gmail.com> Bug: https://bugs.gentoo.org/772386 Fixes: 3e5ef19d5d6b ("fetch.py: fix UnboundLocalError: local variable 'myfile'") Fixes: b9ef191c7498 ("MirrorLayoutConfig: content digest support (bug 756778)") Signed-off-by: Zac Medico <zmedico@gentoo.org>
* FetchIterator: pass DistfileName type as FetchTask filenameZac Medico2021-02-242-10/+11
| | | | | Bug: https://bugs.gentoo.org/756778 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Add emirrordist shelve dump/restore (bug 721680)Zac Medico2021-02-242-0/+118
| | | | | Bug: https://bugs.gentoo.org/721680 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* fetch.py: fix UnboundLocalError: local variable 'myfile'Zac Medico2021-02-241-2/+2
| | | | | | | Reported-by: Ionen Wolkens <sudinave@gmail.com> Bug: https://bugs.gentoo.org/772386 Fixes: b9ef191c7498 ("MirrorLayoutConfig: content digest support (bug 756778)") Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Add USE conflict unit test for bug 615824Zac Medico2021-02-231-0/+51
| | | | | | | | | | | | | | | | | | | | Test bug 615824, where an automask USE change results in a conflict which is not reported. In order to install L, foo must be disabled for both K and M, but autounmask disables foo for K and leaves it enabled for M: [ebuild N ] sci-libs/K-1 USE="-foo" [ebuild N ] sci-libs/L-1 [ebuild N ] sci-libs/M-1 USE="foo" The following USE changes are necessary to proceed: (see "package.use" in the portage(5) man page for more details) # required by sci-libs/L-1::test_repo # required by sci-libs/L (argument) >=sci-libs/K-1 -foo Bug: https://bugs.gentoo.org/615824 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* binarytree.inject: create PKGDIRZac Medico2021-02-231-0/+1
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* make.globals: default enable FEATURES=binpkg-multi-instanceZac Medico2021-02-221-2/+2
| | | | | Bug: https://bugs.gentoo.org/715112 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* Add content-hash distfiles layout (bug 756778)Daniel Robbins2021-02-222-0/+133
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The content-hash layout is identical to the filename-hash layout, except for these three differences: 1) A content digest is used instead of a filename digest. 2) The final element of the path returned from the get_path method corresponds to the complete content digest. The path is a function of the content digest alone. 3) Because the path is a function of content digest alone, the get_filenames implementation cannot derive distfiles names from paths, so it instead yields DistfileName instances whose names are equal to content digest values. The DistfileName documentation discusses resulting implications. Motivations to use the content-hash layout instead of the filename-hash layout may include: 1) Since the file path is independent of the file name, file name collisions cannot occur. This makes the content-hash layout suitable for storage of multiple types of files (not only gentoo distfiles). For example, it can be used to store distfiles for multiple linux distros within the same tree, with automatic deduplication based on content digest. This layout can be used to store and distribute practically anything (including binary packages for example). 2) Allows multiple revisions for the same distfiles name. An existing distfile can be updated, and if a user still has an older copy of an ebuild repository (or an overlay), then a user can successfully fetch a desired revision of the distfile as long as it has not been purged from the mirror. 3) File integrity data is integrated into the layout itself, making it very simple to verify the integrity of any file that it contains. The only tool required is an implementation of the chosen hash algorithm. Bug: https://bugs.gentoo.org/756778 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* MirrorLayoutConfig: content digest support (bug 756778)Zac Medico2021-02-222-17/+114
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In order to support mirror layouts that use content digests, extend MirrorLayoutConfig validate_structure and get_best_supported_layout methods to support an optional filename parameter of type DistfileName which includes a digests attribute. Use the new parameter to account for availablility of specific distfile content digests when validating and selecting mirror layouts which require those digests. The DistfileName type represents a distfile name and associated content digests, used by MirrorLayoutConfig and related layout implementations. The path of a distfile within a layout must be dependent on nothing more than the distfile name and its associated content digests. For filename-hash layout, path is dependent on distfile name alone, and the get_filenames implementation yields strings corresponding to distfile names. For content-hash layout, path is dependent on content digest alone, and the get_filenames implementation yields DistfileName instances whose names are equal to content digest values. The content-hash layout simply lacks the filename-hash layout's innate ability to translate a distfile path to a distfile name, and instead caries an innate ability to translate a distfile path to a content digest. In order to prepare for a migration from filename-hash to content-hash layout, all consumers of the layout get_filenames method need to be updated to work with content digests as a substitute for distfile names. For example, in order to prepare emirrordist for content-hash, a key-value store needs to be added as a means to associate distfile names with content digest values yielded by the content-hash get_filenames implementation. Bug: https://bugs.gentoo.org/756778 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* make.defaults: prevent USE="${USE} ..." misbehaviorZac Medico2021-02-211-3/+14
| | | | | | | | | | | | | | | | Discard parent profile USE from the expand_map variable before it is used to evaluate a child profile. This prevents accidents triggered by USE="${USE} ..." settings at the top of make.defaults which caused parent profile USE to override parent profile package.use settings. Note that would be nice to guard the USE_EXPAND variables like this too, but unfortunately USE_EXPAND is not known until after make.defaults has been evaluated, so that will require some form of make.defaults preprocessing. Bug: https://bugs.gentoo.org/771549 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* testEbuildFetch: refactor to test multiple layout preferencesZac Medico2021-02-211-32/+83
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* PopenProcess: use call_soon for _async_waipid in _startZac Medico2021-02-141-2/+2
| | | | | | | | | Use call_soon to delay the add_child_handler call that _async_waipid will trigger, so that it will occur after the event loop is running. Bug: https://bugs.gentoo.org/770712 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* portage.getpid: call os.getpid() lazilyZac Medico2021-02-071-1/+3
| | | | | | | | | Call os.getpid() lazily, which eliminates getpid calls when possible after os.fork() in the portage.process module. Bug: https://bugs.gentoo.org/767913 Reviewed-by: Brian Dolbec <dolsen@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emerge: disable --autounmask-license by defaultZac Medico2021-02-072-6/+27
| | | | | | | | | | | | | | | | | | | | | | | | Disable --autounmask-license by default, in order to limit user exposure to risks associated with package.license changes. The changes that this option suggests are only intended to be accepted when a user has made a conscious decision to accept the corresponding license(s). Creation of package.license changes introduces a risk that users may erroneously accept the changes due to some kind of accident or misunderstanding, rather than due to conscious decisions about licenses. These risks provide motivation to disable --autounmask-license by default. The --autounmask-use option will remain as the only autounmask option that is still enabled by default. The unit tests demonstrate interactions between --autounmask and --autounmask-license options. The --autounmask option enables --autounmask-license unless --autounmask-license=n has been specified. If --autounmask=n is used to disable autounmask, then --autounmask-license=y has no effect. Bug: https://bugs.gentoo.org/766773 Reviewed-by: Brian Dolbec <dolsen@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emaint --fix merges: add -y, --yes optionZac Medico2021-02-072-6/+19
| | | | | | | | | Since the emaint --fix merges uses emerge --ask, add a -y, --yes option to use --ask=n instead. Bug: https://bugs.gentoo.org/766767 Reviewed-by: Brian Dolbec <dolsen@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org>
* emirrordist: prevent distfiles_db _pkg_str pickle problemsZac Medico2021-01-211-1/+2
| | | | | | | | When storing a cpv as a shelve value, convert _pkg_str to str in order to prevent pickle problems. Bug: https://bugs.gentoo.org/766459 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* binarytree.move_ent: copy on write for package moveZac Medico2021-01-193-16/+48
| | | | | | | | | | | | | Copy on write when applying package moves, and silently skip package moves when the same move has already been applied to the same build of the package. Since the old package instance is preserved, it avoids the problem of having entries for deleted packages remain in the package index. We can simply assume that the package will be deleted by eclean-pkg when its time comes. Bug: https://bugs.gentoo.org/766012 Signed-off-by: Zac Medico <zmedico@gentoo.org>
* bin: misc pylint fixesZac Medico2021-01-181-1/+2
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* ProxyManager: remove unused loop parameterZac Medico2021-01-181-1/+1
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* ProxyManager: Use async and await syntaxZac Medico2021-01-182-8/+5
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* RepoStorageInterface: Use async and await syntaxZac Medico2021-01-185-96/+56
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* SubprocessExecTestCase: Use async and await syntaxZac Medico2021-01-181-18/+14
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* ChildWatcherTestCase: Use async and await syntaxZac Medico2021-01-181-6/+4
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>
* AsyncFunctionTestCase: Use async and await syntaxZac Medico2021-01-181-6/+4
| | | | Signed-off-by: Zac Medico <zmedico@gentoo.org>