Get Gentoo!
gentoo.org sites
gentoo.org Wiki Bugs Forums Packages
Planet Archives Sources
Infra Status
aboutsummaryrefslogtreecommitdiff
path: root/slave
Commit message (Collapse)AuthorAgeFilesLines
* Explicitely use tuples for except:HEADmasterNirbheek Chauhan2009-02-242-2/+2
| | | | | This behaviour will change in Python 3.0, so we should make sure it's explicit right now.
* Fix Gentoo Foundation Copyright datesNirbheek Chauhan2009-02-241-1/+1
| | | | * For files edited in 2009 as well
* Minor changes to follow Pythonesque idiomsNirbheek Chauhan2009-02-242-21/+30
| | | | | * try..finally instead of osp.exists() * Explicit relative imports
* Fix versioning -- 0.0.1.95Nirbheek Chauhan2009-02-182-2/+3
| | | | | | Also fix Makefile to do dist in dist/ directory (similar to setup.py) Yes, this means there will be a 0.0.2 coming soon :p
* Add autotua.crypt to setup.pyNirbheek Chauhan2009-01-181-2/+2
| | | | | It won't get installed unless it's added to the list of modules to install.
* [bugfix] Don't check gpghome while init_gpghome()Nirbheek Chauhan2008-10-141-0/+6
| | | | | | | | | This is the kind of bug that is easy to miss without proper testing :) init_gpghome() would never work because it would check if self.gpghome was a valid gpghome before creating it :p Reported by Anielkis Herrera González (p0w3r3d)
* Update TODONirbheek Chauhan2008-10-131-1/+5
|
* Basic slave-master stateful encrypted interactionNirbheek Chauhan2008-10-104-12/+207
| | | | | | | | | | | | | | | | | | | | | | | * Slave can "take" jobs from the master now - Asymmetrical encryption is used via GPG - models.Slave stores the gpg fingerprint in models.GPGFingerprintField - Slave imports the master's GPG key (/slave_api/autotua_master.asc) * Currently, Slave registration is manual (./manage.py shell) - Slave does fancy encrypted pickle talking (autotua.talk()) :) * "Take" jobs via autotua.Jobs().takejob(maintainer, job_name) - slave/autotua/crypt/__init__.py: * Implements the glue with `gpg` (maybe pygnupg later?) * Crypto() object. Has encrypt() and decrypt() - Also see autotua.decrypt_if_required() - GNUPGHOME for the slave is /var/tmp/autotua * => Job().fetch() requires root access (userpriv/sandbox later) * Phases store state to allow pausing/stopping and resuming of jobs - Future feature, not really used ATM - Job().everything() has prelim support for "resume" * Various small bug fixes and tweaks - Yes, I know I need to make this stuff more atomic :p
* Extra Whitespace.Nirbheek Chauhan2008-10-081-1/+0
|
* Errr. Use hard links for README-AUTOTUA.Nirbheek Chauhan2008-09-181-1/+51
|
* Slave is licensed under GPL-3Nirbheek Chauhan2008-09-1812-12/+686
| | | | | - For compatibility with AGPL-3 - Include COPYING
* Symlink the root README into master/ and slave/Nirbheek Chauhan2008-09-172-0/+2
| | | | Also, add it to MANIFEST.in so that setup.py puts it in the sdist tarball
* Fix setup.py to include all required filesNirbheek Chauhan2008-09-173-0/+4
| | | | | | - Need a MANIFEST.in to include package_data and data_files - Add .gitignore to ignore temporary setup.py files - chmod +x setup.py
* Notify the user if autotua_master is unsetNirbheek Chauhan2008-09-171-1/+3
|
* Bugfixes to master server.Nirbheek Chauhan2008-09-033-19/+24
| | | | | | | | | - Allow the master to run with a non-installed autotua-slave by not failing if /etc/autotua/slave.cfg doesn't exist - Use os.makedirs instead of os.mkdir in config.py for creating the autotua dirs - The user needs to manually enter the URL to the master server (temporary change)
* setup-master.py now does db initializationNirbheek Chauhan2008-09-021-0/+1
| | | | | | | | | | | | - After the previous models.py mega-change, the database needed the info that was in master.process.const initialised before it was usable. - setup-master.py + custom.db_defaults can now do all this - If you do not already have autotua-slave installed as a module, you will have to export the PYTHONPATH variable to allow setup-master.py and manage.py to import autotua - Update TODOs - Fix default to copy in setup-master.py
* Update README and TODONirbheek Chauhan2008-09-012-5/+5
|
* Use + not __add__ for adding tuples.Nirbheek Chauhan2008-08-311-1/+1
|
* Allow unprivileged users to parse jobuildsNirbheek Chauhan2008-08-301-3/+5
| | | | If we can't write to the logfile, don't :p
* Update TODOs.Nirbheek Chauhan2008-08-301-3/+4
|
* Numerous small bugfixes for commit 5dc25dNirbheek Chauhan2008-08-304-18/+19
| | | | | | | | | | * Compatibility with git-1.6 (git-* commands are gone) * atexit.register() needs a function *reference* you dope :) * job_data['atoms'] is no longer an array, but is a space-separated string * jobuild.name -> jobuild.atom (a while ago) * Replace string exceptions with Exception(); for compatibility with unicode objects sent by django-1.0 * jobtagedir is different when using a chroot and not (duh)
* Make the slave system-installable and configurableNirbheek Chauhan2008-08-3012-112/+202
| | | | | | | | | | | | | | | * Move configurable constants from const.py to config.py - These values are now overidden by /etc/autotua/slave.cfg * config.VERBOSE is overidden in slave.cfg - Fix all packages to import config.py instead - Change USE_PROXY to IGNORE_PROXY * bai bai test_modules.py. Testing is now done by executing modules * Install git-proxy-cmd.sh into /usr/bin Advantages: * python setup.py sdist * python setup.py install --root ~/autotua-slave * python setup.py help # ;-)
* Mount PORTAGE_DIR and DISTFILES_DIR only if they are directories. Also,Nirbheek Chauhan2008-08-301-3/+5
| | | | don't mount DISTFILES_DIR if it's PORTAGE_DIR+'/distfiles'
* Use "atexit" for cleaning up instead of the test_modules.sh kludge.Nirbheek Chauhan2008-08-304-35/+2
|
* Implement basic dependency resolution:Nirbheek Chauhan2008-08-177-34/+96
| | | | | | | | | | | | | | | | | | | | | | | | slave/autotua/jobuild/__init__.py:Resolver() * Basic dependency resolution; gives back a list of the job order * Does not deal with circular dependencies * Does not parse SIDEPEND slave/autotua/__init__.py slave/autotua/bin/jobuild.sh slave/autotua/bin/jobuild-functions.sh slave/autotua/jobuild/__init__.py * Pass jobtagedir around; don't assume it's always /tmp/autotua/jobtage inside a chroot slave/autotua/daemon/__init__.py slave/autotua/jobuild/__init__.py * Change things so we don't always have to chroot for parsing. Major speedup for dep resolution. * jobtagedir needs to be passed since we might not be chrooting for parsing slave/autotua/chroot/__init__.py * const.CHAUTOTUA_DIR could change from /tmp/autotua; use that instead of hardcoding the path slave/autotua/const.py * Use abspath for AUTOTUA_DIR
* Link the master and the slave in a simplistic (for now) way:Nirbheek Chauhan2008-08-162-67/+9
| | | | | | | | | * stage url processing is shifted to the master now * Communication is via protocol 2 (binary) pickles * slave API can be accessed via /slave_api/jobs /slave_api/jobs/<username> /slave_api/jobs/<username>/<job_name>
* Reorder slave/autotua/const.pyNirbheek Chauhan2008-08-151-29/+28
|
* Update READMENirbheek Chauhan2008-08-091-1/+2
|
* Typos in jobuild-functions.shNirbheek Chauhan2008-08-091-3/+3
|
* Add http proxy support for git and git+sshNirbheek Chauhan2008-08-091-0/+9
| | | | git-proxy-cmd.sh uses the http_proxy variable to tunnel the git connections
* - Implement post() and emerge()Nirbheek Chauhan2008-08-071-0/+42
| | | | - Document a few functions in jobuild-functions.sh
* Initialise some basic variables: ATOMS, P, PN, PVNirbheek Chauhan2008-08-061-1/+21
|
* Just spent the better part of an hour tracing a regression that turned out ↵Nirbheek Chauhan2008-08-062-6/+8
| | | | | | to be a missing '/' Tell me. What did I do to deserve this?
* Move old autotua tmpdirs inside the chroot to /tmp/autotua-old instead of /tmpNirbheek Chauhan2008-08-061-1/+3
| | | | (reduce clutter)
* Fix syncing of the jobtage tree (master:master not HEAD:HEAD)Nirbheek Chauhan2008-08-051-1/+1
|
* * Fix unmounting when we die prematurelyNirbheek Chauhan2008-08-052-3/+4
|
* Define a default module ('job') for which_tests.py . Document it too.Nirbheek Chauhan2008-07-142-1/+7
| | | | Document http_proxy too
* Whoops, jobtagedir should be relative to TMPDIRNirbheek Chauhan2008-07-111-1/+1
|
* - Message to indicate status (Preparing WorkChroot)Nirbheek Chauhan2008-07-092-1/+3
| | | | | - Another bug! (armin76 you... :P) RMTREE_INIT was incomplete and incorrect
* Cleanup after premature dying. Temporary till we haz nice exception handling.Nirbheek Chauhan2008-07-093-0/+32
| | | | (based on threats by armin76)
* Fix major logic problems in autotua.sync which led to *no* syncing happening ↵Nirbheek Chauhan2008-07-081-22/+44
| | | | | | | | | | in most cases. (Found by armin76) - Use names for the return values of is_repo() (I should've implemented Ford_Prefect's suggestion earlier) - Renovate git cloning/fetching; inspired from git.eclass (Thanks for git.eclass dberkholz!) - Fix detection of repository for bzr - rm -rf properly when {git,bzr}-export-ing - Add a few comments explaining what's going on
* And it works! =)Nirbheek Chauhan2008-07-078-22/+157
| | | | | | | | | | | | | | | | | | | | | | | | | | | | slave/autotua/__init__.py: - _fetch_src -> _setup_jobfiles * Fetch jobfiles and hardlink them inside the chroot * Copy them if hardlinking fails - Job().run() slave/autotua/bin/jobuild-functions.sh: - Flick unpack() from pkgcore ;p - Skeleton src_unpack() - run_all_phases() - Fix indentation in get_param() slave/autotua/bin/jobuild.sh: - run_phase() -- implement "all" -> run_all_phases() slave/autotua/chroot/__init__.py: - mkdir -p $chrootdir/{bin,jobfiles,jobtage,src} * jobfiles will be linked into jobfiles/ * Extracted into src/ - Don't need to import sys slave/autotua/jobuild/__init__.py: - Document run_phase() - Check for returncode in _msg(msg) slave/autotua/const.py: - New 2008.0 release! :) slave/autotua/test_modules.py: - Update to run jobs - Update TODO
* A revolution in how jobuilds are parsed. All parsing is now done inside the ↵Nirbheek Chauhan2008-07-066-17/+47
| | | | | | | | | | | | | | | | | | | | chroot :P Why? Because everything has to be done in that enviroment stupid! slave/autotua/jobuild/__init__.py: - self.jobuild is now relative to the jobtagedir slave/autotua/bin/jobuild.sh: - Get AUTOTUA_DIR from autotua.daemon.Spawn() - Prepend AUTOTUA_DIR (inside the chroot) to the jobuild path slave/autotua/daemon/__init__.py: - Chroot all spawned processes - Send const.CHAUTOTUA_DIR (AUTOTUA_DIR inside the chroot) to the process slave/autotua/__init__.py: - Prepare chroot before parsing
* autotua.chroot.WorkChroot():Nirbheek Chauhan2008-07-051-3/+7
| | | | | - Bind-mount read-only by default - Everything except ${DISTDIR} is bind,ro
* autotua.chroot.WorkChroot():Nirbheek Chauhan2008-07-052-24/+37
| | | | | | | | | | | | | - Revamp _clean_mounts() * Reads /proc/mounts to find all mounts inside chrootdir - Move out bind mount code in _setup_mounts() to _bind() - Bind mount ${workdir}/jobtage and autotua/bin into ${chrootdir}/tmp/autotua - Tidy up in setup() before preparing chroot - Move ${chrootdir}/tmp/autotua to ${chrootdir}/tmp/aututua-${timestamp} - Replace useless osp.joins with direct path separators autotua.Job(): - Tidy before cleaning in clean()
* WorkChroot rsync is now 'rsync-nc' (no-clobber)Nirbheek Chauhan2008-07-042-6/+6
|
* Move jobuild.sh 'ping-pong' code to autotua.daemonNirbheek Chauhan2008-07-042-6/+10
|
* - Move running of commands to autotua.daemonNirbheek Chauhan2008-07-044-12/+36
| | | | | | * Makes the jobuild processor cleaner - Fix a bug in autotua.Job()._fetch_src() -- processing is no longer handled via the Jobuild() object - Update TODO to reflect this commit
* - Make PORTDIR and DISTDIR chroot bind mounting optionalNirbheek Chauhan2008-07-032-3/+11
| | | | default will be to fetch portage snapshot
* - Implement a jobuild processor: autotua.jobuild.Processor()Nirbheek Chauhan2008-07-032-6/+45
| | | | | - Insert placeholder for running phases inside a chroot (autotua/bin/jobuild.sh) - Remove a stray debug print