[Extras-cauldron-commits] r2 - in trunk: . buildme buildme/buildlib buildme/debian buildme/tools
mishas at garage.maemo.org
mishas at garage.maemo.org
Sat Mar 29 18:11:40 EET 2008
Author: mishas
Date: 2008-03-29 18:11:40 +0200 (Sat, 29 Mar 2008)
New Revision: 2
Added:
trunk/buildme/
trunk/buildme/HOWTO
trunk/buildme/TODO
trunk/buildme/buildlib/
trunk/buildme/buildlib/__init__.py
trunk/buildme/buildlib/app.py
trunk/buildme/buildlib/dest.py
trunk/buildme/buildlib/dsc.py
trunk/buildme/buildlib/exceptions.py
trunk/buildme/buildlib/fsm.py
trunk/buildme/buildlib/helpers.py
trunk/buildme/buildlib/lock.py
trunk/buildme/buildlib/logger.py
trunk/buildme/buildlib/sbdarch.py
trunk/buildme/debian/
trunk/buildme/debian/changelog
trunk/buildme/debian/compat
trunk/buildme/debian/control
trunk/buildme/debian/copyright
trunk/buildme/debian/dirs
trunk/buildme/debian/rules
trunk/buildme/setup.py
trunk/buildme/tools/
trunk/buildme/tools/buildme
trunk/buildme/tools/buildme.conf
Log:
importing buildme sources
Added: trunk/buildme/HOWTO
===================================================================
--- trunk/buildme/HOWTO (rev 0)
+++ trunk/buildme/HOWTO 2008-03-29 16:11:40 UTC (rev 2)
@@ -0,0 +1,88 @@
+Builder for Maemo Extras (buildME).
+------------------------------------
+
+1. What is it?
+
+It's automatic package building system for Maemo Extras repository based on
+sbdmock package builder[1].
+
+Typical usage scenario of builder is the following:
+ * build Debian source package
+ * upload it to builder incoming directory
+ * wait for build results and logs
+ * if build is OK builder will put result packages into extras-devel repository
+ * if build is failed uploader should fix packages and re-upload them for the
+ new build
+ * builder sends a short report about the build to a dedicated mailing list
+ * detailed logs can be found in results directory
+
+2. How to prepare and upload source packages for build?
+
+Source package can be produced using this command[2]:
+ dpkg-buildpackage -rfakeroot -sa -S
+
+and signed with[3]:
+ debsign -k<key_id> <package>.changes
+
+Uploading source packages for builder is similar to uploading to extras and
+extras-devel repositories[4]. To be able to upload source packages for
+building uploader must have upload rights to extras repository.
+
+To upload sources to for build uploader should have the following section in
+/etc/dput.cf[5]
+
+ [chinook-extras-builder]
+ login = <your_garage_login_name>
+ fqdn = garage
+ method = scp
+ hash = md5
+ allow_unsigned_uploads = 0
+ incoming = /var/www/extras-devel/incoming-builder/chinook
+
+Uploading is done by:
+ dput chinook-extras-builder *.changes
+
+3. What happens when packages are uploaded?
+
+The builder picks up packages from the builder queue and produces binaries for
+all required architectures. Architectures are determined by 'Architecture:'
+field of the package's debian/control file. For architecture 'any' builds are
+performed for all supported architectures. For architecture 'all' only one
+build for fastest architecture is performed. For specific architecture ('i386'
+and 'armel' at the moment) only one build for specified architecture is
+performed. Build is done in a clean environment. It means that scratchbox
+target created from scratch for each build. Only essential build packages and
+build dependencies specified in package's control file will be installed into
+target before build. Sbdmock uses packages from SDK and extras-devel repositories
+to satisfy build dependencies. Build is done using dpkg-buildpackage. Builder
+produces 2 logs:
+ root.log -- information about setting up build target
+ build.log -- information about the actual build operation
+
+4. What to expect as a result
+
+When build is complete, status information is sent to the 'builds' mailing list[6].
+
+Build results are available in results directory:
+ https://garage.maemo.org/builder/chinook/<package name and version>/
+
+Content of this directory depends on the build status. For successful builds
+you will find only log files. For failed build you will find the sources that
+triggered the build (they will be available in 'sources' subdirectory) and any
+produced binary packages (these will be available in 'results' subdirectory).
+Short build report is available from summary.log
+
+NOTE: result directory will be rewriten by the builder if package with the same
+name and version is re-uploaded for the build.
+
+If build is successful packages will be signed by builder and automatically
+uploaded to extras-devel repository.
+
+5. References
+
+[1] Scratchbox Debian package builder tool: http://bifh.org/wiki/sbdmock
+[2] dpkg-buildpackage man page: http://www.penguin-soft.com/penguin/man/1/dpkg-source.html
+[3] debsign man page: http://www.penguin-soft.com/penguin/man/1/debsign.html
+[4] Maemo Extras Repository: http://maemo.org/community/application-catalog/extras_repository.html
+[5] dput man page: http://www.penguin-soft.com/penguin/man/1/dput.html
+[6] extras-cauldron-builds: https://garage.maemo.org/mailman/listinfo/extras-cauldron-builds
Added: trunk/buildme/TODO
===================================================================
--- trunk/buildme/TODO (rev 0)
+++ trunk/buildme/TODO 2008-03-29 16:11:40 UTC (rev 2)
@@ -0,0 +1,4 @@
+1. Make this kind of warnings go to summary.log, not to stdout/err:
+[2008-02-24 00:55:02] Warning: Unknown architecture: 'arm'
+
+2. Add a link to the build results into the mail message
Added: trunk/buildme/buildlib/__init__.py
===================================================================
--- trunk/buildme/buildlib/__init__.py (rev 0)
+++ trunk/buildme/buildlib/__init__.py 2008-03-29 16:11:40 UTC (rev 2)
@@ -0,0 +1,2 @@
+#!/usr/bin/python -tt
+""" Dummy file to make this a package """
Added: trunk/buildme/buildlib/app.py
===================================================================
--- trunk/buildme/buildlib/app.py (rev 0)
+++ trunk/buildme/buildlib/app.py 2008-03-29 16:11:40 UTC (rev 2)
@@ -0,0 +1,108 @@
+#!/usr/bin/python -tt
+# vim: sw=4 ts=4 expandtab ai
+#
+# Application
+#
+# Copyright (C) 2008 Ed Bartosh <bartosh at gmail.com>
+#
+# This program is free software; you can redistribute it and/or modify it under
+# the terms of the GNU General Public License version 2 as published by the
+# Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
+# details.
+#
+# You should have received a copy of the GNU General Public License along with
+# this program; if not, write to the Free Software Foundation, Inc., 51
+# Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# $Id$
+
+"""
+Application. Wrapper class for main() with exception handling and logging
+"""
+
+__revision__ = "r"+"$Revision$".split(' ')[1]
+
+import sys, os, logging
+
+from ConfigParser import ConfigParser
+
+from minideblib.LoggableObject import LoggableObject
+from buildlib.exceptions import Error
+from buildlib.logger import CustomLogger
+
+class App(LoggableObject):
+ """ Application """
+
+ RC_OK = 0
+ RC_RUNTIME_ERROR = 1
+ RC_UNKNOWN_ERROR = 2
+
+ def __init__(self):
+ """ Init """
+
+ logging.setLoggerClass(CustomLogger)
+
+ # Set up logger to prevent LoggableObject to call basicConfig()
+ self.handler = logging.StreamHandler(sys.stderr)
+ self.handler.setFormatter(CustomLogger.formatter)
+ logger = logging.getLogger()
+ logger.addHandler(self.handler)
+
+ self.appname = os.path.splitext(os.path.basename(sys.argv[0]))[0]
+
+ self.conf = None
+ self.confname = os.path.join('/etc', self.appname) + '.conf'
+
+ if os.path.exists(self.confname):
+ self.conf = ConfigParser()
+ self.conf.read(self.confname)
+
+ def run(self, argv, cmdline_parser, mainfunc):
+ """ Run main function. Catch exceptions """
+
+ try:
+ options, argv = cmdline_parser(argv)
+
+ # set up loglevel
+ loglevel = logging.WARNING
+ if options.debug:
+ loglevel = logging.DEBUG
+ elif options.verbose:
+ loglevel = logging.INFO
+
+ self.handler.setLevel(loglevel)
+
+ mainfunc(argv, options, self._logger, self.conf)
+
+ # known exceptions
+ except Error, exobj:
+ print str(exobj)
+ return self.RC_RUNTIME_ERROR
+
+ # unknown exceptions
+ except:
+ # some modules(optparse) use sys.exit()
+ if sys.exc_info()[0] != SystemExit:
+ self._logger.exception("Unexpected error:\n %s: %s" % \
+ (str(sys.exc_info()[0]).split('.')[1], sys.exc_info()[1]))
+ #logger.exception(traceback.format_exception(sys.exc_type, sys.exc_value, sys.exc_traceback))
+ return self.RC_UNKNOWN_ERROR
+ else:
+ return sys.exc_info()[1]
+
+ return self.RC_OK
+
+if __name__ == '__main__':
+ raise NotImplemented
+
+# Local Variables:
+# mode: python
+# py-indent-offset: 4
+# indent-tabs-mode nil
+# tab-width 4
+# End:
+
Property changes on: trunk/buildme/buildlib/app.py
___________________________________________________________________
Name: svn:keywords
+ Id Author HeadURL LastChangedDate LastChangedRevision LastChangedBy Rev Revision
Added: trunk/buildme/buildlib/dest.py
===================================================================
--- trunk/buildme/buildlib/dest.py (rev 0)
+++ trunk/buildme/buildlib/dest.py 2008-03-29 16:11:40 UTC (rev 2)
@@ -0,0 +1,154 @@
+#!/usr/bin/python -tt
+# vim: sw=4 ts=4 expandtab ai
+#
+# Destinations
+#
+# Copyright (C) 2008 Ed Bartosh <bartosh at gmail.com>
+#
+# This program is free software; you can redistribute it and/or modify it under
+# the terms of the GNU General Public License version 2 as published by the
+# Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
+# details.
+#
+# You should have received a copy of the GNU General Public License along with
+# this program; if not, write to the Free Software Foundation, Inc., 51
+# Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# $Id$
+
+"""
+Destinations. Only SSh and local destinations implemented so far.
+"""
+
+__revision__ = "r"+"$Revision$".split(' ')[1]
+
+import os, pwd, re, shutil
+
+from glob import glob
+from commands import getstatusoutput
+
+from buildlib.exceptions import Error
+
+class SshDest:
+ """ Ssh destination """
+
+ def __init__(self, hostinfo):
+
+ self.command = None
+ self.hostinfo = hostinfo
+ if hostinfo['user']:
+ self.user = hostinfo['user']
+ else:
+ self.user = pwd.getpwuid(os.geteuid())[0]
+
+ self.dest = "%s@%s" % (self.user, self.hostinfo['host'])
+
+ def copyto(self, srcs):
+ """ scp files to dest dir """
+
+ dest = self.hostinfo['dest']
+ for src in srcs:
+ (rcode, out) = getstatusoutput("scp %s %s" % (src, dest))
+ if rcode:
+ raise Error("Error: can't copy '%s' to '%s'. Output: %s" % (src, self.dest, out))
+
+ def copyfrom(self, destdir):
+ """ scp them back """
+
+ for resultdir in self.command["results"]:
+ srcdir = "%s:%s" % (self.hostinfo['host'], resultdir)
+ if self.hostinfo['user']:
+ srcdir = "%s@%s" % (self.hostinfo['user'], srcdir)
+ (rcode, out) = getstatusoutput("scp %s/* %s/" % (srcdir, destdir))
+ if rcode:
+ raise Error("Error: can't copy '%s/*' to '%s/'. Output: %s" % (srcdir, destdir, out))
+
+ def remove(self, srcs):
+ """ Remove files """
+
+ sfiles = ' '.join([os.path.join(self.hostinfo['dir'], os.path.basename(fname)) for fname in srcs])
+ (rcode, out) = getstatusoutput("ssh %s rm %s" % (self.dest, sfiles))
+ if rcode:
+ raise Error("Error: can't remove '%s' from '%s'. Output: %s" % (sfiles, self.dest, out))
+
+ def run(self, dscfn):
+ """ Run remote command """
+
+ return getstatusoutput("ssh %s %s %s" % (self.dest, self.command['command'],
+ os.path.join(self.hostinfo['dir'], os.path.basename(dscfn))))
+
+class LocalDest:
+ """ Local destination """
+
+ def __init__(self, ddir):
+ """ Constructor. Store command and destination dir """
+
+ self.command = None
+ self.ddir = ddir
+ self.user = pwd.getpwuid(os.geteuid())[0]
+
+ def copyto(self, srcs):
+ """ Copy sources to destination dir """
+
+ for src in srcs:
+ try:
+ shutil.copy(src, self.ddir)
+ except IOError:
+ raise Error("Error: can't copy '%s' to '%s'" % (src, self.ddir))
+
+ def copyfrom(self, destdir):
+ """ Copy result directory to output directory """
+
+ for resultdir in self.command["results"]:
+ try:
+ for fname in glob(os.path.join(resultdir,'*')):
+ shutil.copy(fname, destdir)
+ except Exception, exobj:
+ raise Error(str(exobj))
+
+ def remove(self, srcs):
+ """ """
+
+ for fname in [os.path.join(self.ddir, os.path.basename(fpath)) for fpath in srcs]:
+ try:
+ unlink(fname)
+ except OSError, exobj:
+ raise Error("Error: can't remove file '%s'. %s" % (fname, exobj))
+
+ def run(self, dscfn):
+ """ Run command """
+
+ return getstatusoutput(self.command['command'] + ' ' +
+ os.path.join(self.ddir, os.path.basename(dscfn)))
+
+def destfactory(destination):
+ """ Makes destination objects
+ ssh and local objects supported so far"""
+
+ if not destination:
+ raise Error("Error: empty destination")
+
+ if os.path.isdir(destination):
+ return LocalDest(destination)
+ else:
+ parsed = re.match("((?P<user>.*)@)?((?P<host>.*):)?((?P<dir>.+))?", destination).groupdict()
+ if parsed['host'] and parsed['dir']:
+ parsed['dest'] = destination
+ return SshDest(parsed)
+
+ raise Error("Error: unknown destination '%s'" % destination)
+
+if __name__ == '__main__':
+ raise NotImplemented
+
+# Local Variables:
+# mode: python
+# py-indent-offset: 4
+# indent-tabs-mode nil
+# tab-width 4
+# End:
+
Property changes on: trunk/buildme/buildlib/dest.py
___________________________________________________________________
Name: svn:keywords
+ Id Author HeadURL LastChangedDate LastChangedRevision LastChangedBy Rev Revision
Added: trunk/buildme/buildlib/dsc.py
===================================================================
--- trunk/buildme/buildlib/dsc.py (rev 0)
+++ trunk/buildme/buildlib/dsc.py 2008-03-29 16:11:40 UTC (rev 2)
@@ -0,0 +1,119 @@
+#!/usr/bin/python -tt
+# vim: sw=4 ts=4 expandtab ai
+#
+# Dsc API
+#
+# Copyright (C) 2008 Ed Bartosh <bartosh at gmail.com>
+#
+# This program is free software; you can redistribute it and/or modify it under
+# the terms of the GNU General Public License version 2 as published by the
+# Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
+# details.
+#
+# You should have received a copy of the GNU General Public License along with
+# this program; if not, write to the Free Software Foundation, Inc., 51
+# Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# $Id$
+
+"""
+Dsc extention of minideblib's ChangeFile
+"""
+
+__revision__ = "r"+"$Revision$".split(' ')[1]
+
+import os, shutil
+
+from commands import getstatusoutput
+
+from buildlib.lock import Lockf
+from minideblib.ChangeFile import ChangeFile, ChangeFileException
+from minideblib.SignedFile import SignedFile
+
+class Dsc(ChangeFile):
+ """ .dsc file with locking and some extra functionality """
+
+ _lockdir = "/var/tmp/"
+
+ def __init__(self, fname, exclude=None, mustbesigned=True):
+ """ Init """
+
+ name = os.path.basename(fname)
+
+ # try to lock the file
+ self.lock = Lockf(os.path.join(self._lockdir, name + '.lock'))
+ if not self.lock.lock():
+ raise ChangeFileException("%s is locked" % name)
+
+ ChangeFile.__init__(self)
+
+ try:
+ self.load_from_file(fname)
+ except IOError:
+ raise ChangeFileException("Error: Can't read from dsc file '%s'" % name)
+
+ fptr = open(fname)
+ self.signed = SignedFile(fptr).getSigned()
+ fptr.close()
+
+ if mustbesigned and not self.signed:
+ raise ChangeFileException("Error: file %s isn't signed" % name)
+
+ if not exclude:
+ exclude = []
+
+ self.exclude = exclude
+ self.dir = os.path.dirname(fname)
+ self.fname = fname
+ self.name = os.path.basename(os.path.splitext(fname)[0])
+
+ def verify(self, directory = None):
+ """ the same as parent's method, but doesn't require directory """
+
+ if not directory:
+ directory = self.dir
+
+ ChangeFile.verify(self, directory)
+
+ # Check gpg signature if file is signed
+ if self.signed:
+ self._logger.debug("'%s' is signed, checking" % self.fname)
+ (rcode, out) = getstatusoutput("gpg --verify %s" % self.fname)
+ if rcode:
+ raise ChangeFileException("GPG verification of '%s' failed. gpg returned %s, output: %s" \
+ % (os.path.basename(self.fname), rcode, out))
+
+ def getFiles(self):
+ """ Wrapper around parent's method. Understands excludes """
+
+ return [finfo for finfo in ChangeFile.getFiles(self) if os.path.splitext(finfo[4])[1] not in self.exclude]
+
+ def getpaths(self):
+ """ Return list of file paths including .changes file itself """
+
+ return [self.fname] + [os.path.join(self.dir, finfo[4]) for finfo in self.getFiles()]
+
+ def movefiles(self, dest):
+ """ Move files to dest dir """
+
+ for fname in self.getpaths():
+ shutil.move(fname, dest)
+
+ self.dir = dest
+ self.fname = os.path.join(dest, os.path.basename(self.fname))
+
+
+if __name__ == '__main__':
+ raise NotImplemented
+
+# Local Variables:
+# mode: python
+# py-indent-offset: 4
+# indent-tabs-mode nil
+# tab-width 4
+# End:
+
Property changes on: trunk/buildme/buildlib/dsc.py
___________________________________________________________________
Name: svn:keywords
+ Id Author HeadURL LastChangedDate LastChangedRevision LastChangedBy Rev Revision
Added: trunk/buildme/buildlib/exceptions.py
===================================================================
--- trunk/buildme/buildlib/exceptions.py (rev 0)
+++ trunk/buildme/buildlib/exceptions.py 2008-03-29 16:11:40 UTC (rev 2)
@@ -0,0 +1,42 @@
+#!/usr/bin/python -tt
+# vim: sw=4 ts=4 expandtab ai
+#
+# Exceptions for using extras builder classes
+#
+# Copyright (C) 2008 Ed Bartosh <bartosh at gmail.com>
+#
+# This program is free software; you can redistribute it and/or modify it under
+# the terms of the GNU General Public License version 2 as published by the
+# Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
+# details.
+#
+# You should have received a copy of the GNU General Public License along with
+# this program; if not, write to the Free Software Foundation, Inc., 51
+# Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# $Id$
+
+"""
+exeptions for maemo extras builder
+"""
+
+__revision__ = "r"+"$Revision$".split(' ')[1]
+
+class Error(Exception):
+ """ Error exception for raising in maemo extras builder """
+ pass
+
+if __name__ == '__main__':
+ raise NotImplemented
+
+# Local Variables:
+# mode: python
+# py-indent-offset: 4
+# indent-tabs-mode nil
+# tab-width 4
+# End:
+
Property changes on: trunk/buildme/buildlib/exceptions.py
___________________________________________________________________
Name: svn:keywords
+ Id Author HeadURL LastChangedDate LastChangedRevision LastChangedBy Rev Revision
Added: trunk/buildme/buildlib/fsm.py
===================================================================
--- trunk/buildme/buildlib/fsm.py (rev 0)
+++ trunk/buildme/buildlib/fsm.py 2008-03-29 16:11:40 UTC (rev 2)
@@ -0,0 +1,89 @@
+#!/usr/bin/python -tt
+# vim: sw=4 ts=4 expandtab ai
+#
+# Finite State Machine
+#
+# Copyright (C) 2008 Ed Bartosh <bartosh at gmail.com>
+#
+# This program is free software; you can redistribute it and/or modify it under
+# the terms of the GNU General Public License version 2 as published by the
+# Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
+# details.
+#
+# You should have received a copy of the GNU General Public License along with
+# this program; if not, write to the Free Software Foundation, Inc., 51
+# Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# $Id$
+
+"""
+Finite State Machine
+"""
+
+from buildlib.exceptions import Error
+
+__revision__ = "r"+"$Revision$".split(' ')[1]
+
+class FSM:
+ """ Finite State Machine """
+
+ # helper return codes for handlers, not mandatory
+ STOP = 'STOP'
+ OK = 'OK'
+ NEXT = 'NEXT'
+
+ def __init__(self, init_handler, table, params=None):
+ """ Init FSM table, initial handler and parameters """
+
+ self.table, self.handler, self.params = ({}, None, {})
+
+ self.reset(init_handler, table, params)
+
+ def reset(self, init_handler, table, params=None):
+ """ Reset state machine """
+
+ self.handler = init_handler
+ self.table = table
+ if params:
+ self.params = params
+ else:
+ self.params = {}
+
+ def run(self):
+ """ Run state machine """
+
+ handler = self.handler
+ table = self.table
+
+ while handler:
+
+ # check table
+ if handler not in table:
+ raise Error('handler %s not found in the FSM table' % handler)
+
+ # run it
+ try:
+ code = handler(self)
+ except StopIteration: # workaround StopIteration for iterators
+ code = self.STOP
+
+ # get next handler from the table using return code
+ try:
+ handler = table[handler][code]
+ except KeyError:
+ raise Error('Unknown return code %s for handler %s' % (code, handler))
+
+if __name__ == '__main__':
+ raise NotImplemented
+
+# Local Variables:
+# mode: python
+# py-indent-offset: 4
+# indent-tabs-mode nil
+# tab-width 4
+# End:
+
Property changes on: trunk/buildme/buildlib/fsm.py
___________________________________________________________________
Name: svn:keywords
+ Id Author HeadURL LastChangedDate LastChangedRevision LastChangedBy Rev Revision
Added: trunk/buildme/buildlib/helpers.py
===================================================================
--- trunk/buildme/buildlib/helpers.py (rev 0)
+++ trunk/buildme/buildlib/helpers.py 2008-03-29 16:11:40 UTC (rev 2)
@@ -0,0 +1,72 @@
+#!/usr/bin/python -tt
+# vim: sw=4 ts=4 expandtab ai
+#
+# Helpers
+#
+# Copyright (C) 2008 Ed Bartosh <bartosh at gmail.com>
+#
+# This program is free software; you can redistribute it and/or modify it under
+# the terms of the GNU General Public License version 2 as published by the
+# Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
+# details.
+#
+# You should have received a copy of the GNU General Public License along with
+# this program; if not, write to the Free Software Foundation, Inc., 51
+# Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# $Id$
+
+"""
+Helper functions
+"""
+
+__revision__ = "r"+"$Revision$".split(' ')[1]
+
+import smtplib
+
+from email.MIMEText import MIMEText
+from buildlib.exceptions import Error
+
+def sendemail(xfrom, xto, xsubj, text):
+ """ Sends email. Parameters are taken from msg. """
+ msg = MIMEText(text)
+ msg['Subject'] = xsubj
+ msg['From'] = xfrom
+ msg['To'] = xto
+ soket = smtplib.SMTP()
+ soket.connect()
+ soket.sendmail(xfrom, [xto], msg.as_string())
+ soket.close()
+
+def writefile(fname, content):
+ """ Write content to the file """
+
+ fptr = open(fname, "w")
+ fptr.write(content)
+ fptr.close()
+
+def readfile(fname):
+ """ return file content """
+ try:
+ fptr = open(fname)
+ except IOError:
+ raise Error("Can't open file: %s" % fname)
+ content = fptr.read()
+ fptr.close()
+
+ return content
+
+if __name__ == '__main__':
+ raise NotImplemented
+
+# Local Variables:
+# mode: python
+# py-indent-offset: 4
+# indent-tabs-mode nil
+# tab-width 4
+# End:
+
Property changes on: trunk/buildme/buildlib/helpers.py
___________________________________________________________________
Name: svn:keywords
+ Author LastChangedBy Date LastChangedDate Rev Revision LastChangedRevision Id
Added: trunk/buildme/buildlib/lock.py
===================================================================
--- trunk/buildme/buildlib/lock.py (rev 0)
+++ trunk/buildme/buildlib/lock.py 2008-03-29 16:11:40 UTC (rev 2)
@@ -0,0 +1,82 @@
+#!/usr/bin/python -tt
+# vim: sw=4 ts=4 expandtab ai
+#
+# Locking APIs
+#
+# Copyright (C) 2008 Ed Bartosh <bartosh at gmail.com>
+#
+# This program is free software; you can redistribute it and/or modify it under
+# the terms of the GNU General Public License version 2 as published by the
+# Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA
+# 02110-1301 USA
+#
+# $Id$
+
+"""
+Locking APIs
+"""
+
+__revision__ = "r"+"$Revision$".split(' ')[1]
+
+import os, fcntl
+
+class Lockf:
+ """ Make a lock using fcntl calls """
+
+ def __init__(self, lockfn):
+ """ Constructor. opens file """
+
+ self.lockfn = lockfn
+ self.locked = False
+
+ self.lockfile = open(lockfn, 'w')
+
+ def __del__(self):
+ if self.locked:
+ self.unlock()
+ if os.path.exists(self.lockfn):
+ os.unlink(self.lockfn)
+
+ def lock(self):
+ """ Lock it """
+
+ if self.locked:
+ return True
+
+ try:
+ fcntl.lockf(self.lockfile, fcntl.LOCK_EX | fcntl.LOCK_NB)
+ self.locked = True
+ except IOError:
+ self.locked = False
+ return False
+
+ return True
+
+ def unlock(self):
+ """ Unlock """
+
+ if not self.locked:
+ return True
+
+ fcntl.flock(self.lockfile.fileno(), fcntl.LOCK_UN)
+ self.locked = False
+
+if __name__ == '__main__':
+ raise NotImplemented
+
+# Local Variables:
+# mode: python
+# py-indent-offset: 4
+# indent-tabs-mode nil
+# tab-width 4
+# End:
+
Property changes on: trunk/buildme/buildlib/lock.py
___________________________________________________________________
Name: svn:keywords
+ Id Author HeadURL LastChangedDate LastChangedRevision LastChangedBy Rev Revision
Added: trunk/buildme/buildlib/logger.py
===================================================================
--- trunk/buildme/buildlib/logger.py (rev 0)
+++ trunk/buildme/buildlib/logger.py 2008-03-29 16:11:40 UTC (rev 2)
@@ -0,0 +1,64 @@
+#!/usr/bin/python -tt
+# vim: sw=4 ts=4 expandtab ai
+#
+# Logging APIs
+#
+# Copyright (C) 2008 Ed Bartosh <bartosh at gmail.com>
+#
+# This program is free software; you can redistribute it and/or modify it under
+# the terms of the GNU General Public License version 2 as published by the
+# Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
+# details.
+#
+# You should have received a copy of the GNU General Public License along with
+# this program; if not, write to the Free Software Foundation, Inc., 51
+# Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# $Id$
+
+"""
+Logging APIs
+"""
+
+__revision__ = "r"+"$Revision$".split(' ')[1]
+
+import logging
+
+class CustomLogger(logging.Logger):
+ """ Custom Logger. Redefines formatter """
+
+ formatter = logging.Formatter('[%(asctime)s] %(message)s', "%F %X")
+ SUMMARY = logging.INFO + 1
+
+ def __init__(self, domain = None, level = logging.DEBUG):
+ """ Initializes logger """
+
+ logging.Logger.__init__(self, domain, level)
+ logging.addLevelName(self.SUMMARY, 'SUMMARY')
+
+ def addHandler(self, handler):
+ """ Redefine default format """
+
+ handler.setFormatter(self.formatter)
+ logging.Logger.addHandler(self, handler)
+
+ def summary(self, msg, *args, **kwargs):
+ """ Log 'msg % args' with severity 'SUMMARY'. """
+
+ if self.manager.disable < self.SUMMARY and self.SUMMARY >= self.getEffectiveLevel():
+ apply(self._log, (self.SUMMARY, msg, args), kwargs)
+
+if __name__ == '__main__':
+ raise NotImplemented
+
+# Local Variables:
+# mode: python
+# py-indent-offset: 4
+# indent-tabs-mode nil
+# tab-width 4
+# End:
+
Property changes on: trunk/buildme/buildlib/logger.py
___________________________________________________________________
Name: svn:keywords
+ Id Author HeadURL LastChangedDate LastChangedRevision LastChangedBy Rev Revision
Added: trunk/buildme/buildlib/sbdarch.py
===================================================================
--- trunk/buildme/buildlib/sbdarch.py (rev 0)
+++ trunk/buildme/buildlib/sbdarch.py 2008-03-29 16:11:40 UTC (rev 2)
@@ -0,0 +1,108 @@
+#!/usr/bin/python -tt
+# vim: sw=4 ts=4 expandtab ai
+#
+# sbdarch
+#
+# Copyright (C) 2008 Ed Bartosh <bartosh at gmail.com>
+#
+# This program is free software; you can redistribute it and/or modify it under
+# the terms of the GNU General Public License version 2 as published by the
+# Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
+# details.
+#
+# You should have received a copy of the GNU General Public License along with
+# this program; if not, write to the Free Software Foundation, Inc., 51
+# Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# $Id$
+
+"""
+sbdarch. Determine sbdmock target architecture.
+"""
+
+__revision__ = "r"+"$Revision$".split(' ')[1]
+
+import os, re, pprint
+
+from buildlib.exceptions import Error
+from minideblib.LoggableObject import LoggableObject
+
+TARGETS = ("armel:maemo-chinook-armel-extras-devel", "i386:maemo-chinook-i386-extras-devel")
+
+class SbdArch(LoggableObject):
+ """ Determine sbdmock target[s] from dsc architecture """
+
+ # supported_arches sorted by the preference to use for 'all' source architecture
+ _supported_arches = ['i386', 'armel', 'arm', 'ui386', 'uarm']
+ _debian_arches = [ 'alpha', 'amd64', 'arm', 'armeb', 'armel', 'hppa', 'i386', 'ia64', 'm32r', 'm68k', \
+ 'mips', 'mipsel', 'powerpc', 'ppc64', 's390', 's390x', 'sh3', 'sh3eb', 'sh4', \
+ 'sh4eb', 'sparc' ]
+
+ def __init__(self, atmaps):
+ """ Init. Process arch-target mapping """
+
+ if not atmaps:
+ raise Error("No arch<->target mappings defined")
+
+ self.arches = {'any':[], 'all':[]}
+ self.tamaps = {}
+
+ for (arch, target) in [atmap.split(':') for atmap in atmaps]:
+
+ self.tamaps[target] = arch
+
+ if arch not in self._supported_arches:
+ raise Error("Unsupported arch: %s" % arch)
+
+ if arch not in self.arches:
+ self.arches[arch] = []
+ self.arches[arch].append(target)
+ if arch in self._debian_arches:
+ self.arches['any'].append(target)
+
+ # special case for packages which produce only arch independed binaries.
+ for arch in self._supported_arches:
+ if arch in self.arches:
+ self.arches['all'].append(self.arches[arch][0])
+ break
+ # nothing found, pick first from 'any'
+ if not self.arches['all'] and self.arches['any']:
+ self.arches['all'].append(self.arches['any'][0])
+
+ def gettargets(self, dsc):
+ """ Get list of target for Dsc object """
+
+ arches = self.arches
+ result = []
+ for archname in ('scratchbox-architecture', 'architecture'):
+ if dsc.has_key(archname):
+ for targ in [targ.strip() for targ in re.split(",? +", dsc[archname])]:
+ if arches.has_key(targ):
+ for target in arches[targ]:
+ result.append((self.tamaps[target], target))
+ else:
+ self._logger.warning("Warning: Unknown %s: '%s'" % (archname, targ))
+
+ if not result:
+ raise Error("Can't find 'architecture' field in dsc file")
+
+ self._logger.debug('Targets: %s' % pprint.pformat(result))
+
+ if not result:
+ raise Error("Can't find any suitable target for %s" % os.path.basename(dsc.fname))
+
+ return result
+
+if __name__ == '__main__':
+ raise NotImplemented
+
+# Local Variables:
+# mode: python
+# py-indent-offset: 4
+# indent-tabs-mode nil
+# tab-width 4
+# End:
Property changes on: trunk/buildme/buildlib/sbdarch.py
___________________________________________________________________
Name: svn:keywords
+ Id Author HeadURL LastChangedDate LastChangedRevision LastChangedBy Rev Revision
Added: trunk/buildme/debian/changelog
===================================================================
--- trunk/buildme/debian/changelog (rev 0)
+++ trunk/buildme/debian/changelog 2008-03-29 16:11:40 UTC (rev 2)
@@ -0,0 +1,132 @@
+buildme (1.3.1) unstable; urgency=low
+
+ * added --product commadline argument
+ * getting info from config file works now
+
+ -- Ed Bartosh <bartosh at gmail.com> Sun, 24 Feb 2008 23:07:24 +0200
+
+buildme (1.3.0) unstable; urgency=low
+
+ * buildme: get parameters from config file
+ * app: parse application config file if exists
+ * buldme: fix: set proper extention for root.log when build is failed
+
+ -- Ed Bartosh <bartosh at gmail.com> Sun, 24 Feb 2008 16:18:02 +0200
+
+buildme (1.2.2) unstable; urgency=low
+
+ * use template for email subject
+ * fix: set proper extention for root.log when build is failed
+
+ -- Ed Bartosh <bartosh at gmail.com> Wed, 20 Feb 2008 17:52:57 +0200
+
+buildme (1.2.1) unstable; urgency=low
+
+ * howto added
+ * dsc: don't call unlock, it leads to not removing lock file
+ * set svn:keywords for fsm.py
+ * fixed wrong FSM naming
+ * fsm: fixed missed import
+
+ -- Ed Bartosh <bartosh at gmail.com> Tue, 19 Feb 2008 22:51:36 +0200
+
+buildme (1.2.0) unstable; urgency=low
+
+ * buildme,helpers: send report by email
+ * buildme: remove sources from the destination after the build
+ * buildme: logging improvements
+ * lock: fix: remove file only if it's locked
+
+ -- Ed Bartosh <bartosh at gmail.com> Tue, 19 Feb 2008 01:10:18 +0200
+
+buildme (1.1.1) unstable; urgency=low
+
+ * build: generators and other minor bugs fixed
+ * fsm: properly check FSM table.
+ * fsm: workaround for StopIteration added
+
+ -- Ed Bartosh <bartosh at gmail.com> Sun, 17 Feb 2008 20:43:07 +0200
+
+buildme (1.1.0) unstable; urgency=low
+
+ * Use FSM instead of huge main()
+
+ -- Ed Bartosh <bartoshgmail.com> Sun, 17 Feb 2008 17:26:05 +0200
+
+buildme (1.0.1) unstable; urgency=low
+
+ * move locking from command_factory to separate class
+ * warning message replaced to info to avoid cronspamming
+
+ -- Ed Bartosh <bartosh at gmail.com> Sat, 16 Feb 2008 18:35:17 +0200
+
+buildme (1.0.0) unstable; urgency=low
+
+ * Project renamed
+ * APIs moved to buildlib/
+ * python-buildlib package added
+
+ -- Ed Bartosh <bartosh at gmail.com> Sat, 16 Feb 2008 15:03:54 +0200
+
+sbdmock-runner (0.2.1) unstable; urgency=low
+
+ * remove empty logs and result dir.
+ * Don't raise exception when build failed
+ * Improved logging.
+ * Improved output directory structure
+
+ -- Ed Bartosh <bartosh at gmail.com> Sat, 16 Feb 2008 13:09:46 +0200
+
+sbdmock-runner (0.2.0) unstable; urgency=lo w
+
+ * --preserve comandline option added
+ (preserve working directories/files)
+ * put only root.log and build.log into logs dir
+ * infrastructure simplified ('target' level removed)
+
+ -- Ed Bartosh <bartosh at gmail.com> Wed, 13 Feb 2008 00:35:39 +0200
+
+sbdmock-runner (0.1.1) unstable; urgency=low
+
+ * fixed bug in cleaning output dirs
+ * logging improved
+ * Dsc: implemented gpg signature check
+ * control: added dependencies
+
+ -- Ed Bartosh <bartosh at gmail.com> Mon, 11 Feb 2008 23:30:29 +0200
+
+sbdmock-runner (0.1.0) unstable; urgency=low
+
+ * infrasfactory: remove directories by default
+ * LockF: remove lock file when object deleted
+ * get username from destination if specified
+ * --user option removed, get user from destination
+ * --queue option added (copy results to incoming queue)
+ * --sign option added (debsign results)
+
+ -- Ed Bartosh <bartosh at gmail.com> Fri, 8 Feb 2008 23:43:33 +0200
+
+sbdmock-runner (0.0.3) unstable; urgency=low
+
+ * SbdArch class added
+ functionality borrowed from sbdarchitecture utility,
+ thanks to Alexander Kanevskiy <kad at bifh.org>
+ * Use SbdArch to determine sbdmock targets
+ * Build sources and arch-independent binaries only on the first run
+
+ -- Ed Bartosh <bartosh at gmail.com> Thu, 7 Feb 2008 21:41:49 +0200
+
+sbdmock-runner (0.0.2) unstable; urgency=low
+
+ * LockF: make sure the lock is released if the LockF object is deleted
+ * fixed the name of dsc file to check
+ * SshDest/copyfrom: fixed src directory
+
+ -- Ed Bartosh <bartosh at gmail.com> Thu, 7 Feb 2008 19:54:51 +0200
+
+sbdmock-runner (0.0.1) unstable; urgency=low
+
+ * Initial revison
+
+ -- Ed Bartosh <bartosh at gmail.com> Sun, 20 Jan 2008 23:06:00 +0200
+
Added: trunk/buildme/debian/compat
===================================================================
--- trunk/buildme/debian/compat (rev 0)
+++ trunk/buildme/debian/compat 2008-03-29 16:11:40 UTC (rev 2)
@@ -0,0 +1 @@
+4
\ No newline at end of file
Added: trunk/buildme/debian/control
===================================================================
--- trunk/buildme/debian/control (rev 0)
+++ trunk/buildme/debian/control 2008-03-29 16:11:40 UTC (rev 2)
@@ -0,0 +1,22 @@
+Source: buildme
+Section: python
+Priority: optional
+Maintainer: Ed Bartosh <bartosh at gmail.com>
+Build-Depends-Indep: debhelper (>= 4.1.25), python-central
+Standards-Version: 3.6.2
+XS-Python-Version: all
+
+Package: buildme
+Architecture: all
+Depends: ${python:Depends}, devscripts (>= 2.8.14), python-buildlib
+Description: Builder for Maemo Extras
+ This package contains buildme commandline builder tool for Maemo Extras
+XB-Python-Version: all
+
+Package: python-buildlib
+Architecture: all
+Depends: ${python:Depends}, ssh, gnupg, python-minideblib (>= 0.6.21.27)
+Description: Library for buildme
+ This package contains APIs for Maemo Extras builder tool buildme
+XB-Python-Version: all
+
Added: trunk/buildme/debian/copyright
===================================================================
--- trunk/buildme/debian/copyright (rev 0)
+++ trunk/buildme/debian/copyright 2008-03-29 16:11:40 UTC (rev 2)
@@ -0,0 +1,18 @@
+Copyright (C) 2008 Ed Bartosh <bartosh at gmail.com>
+
+This program is free software; you can redistribute it and/or modify it under
+the terms of the GNU General Public License version 2 as published by the Free
+Software Foundation.
+
+This program is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
+PARTICULAR PURPOSE. See the GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License with the
+Debian GNU/Linux distribution in file /usr/share/common-licenses/GPL; if not,
+write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330,
+Boston, MA 02111-1307 USA
+
+A copy of the GNU General Public License is available as
+/usr/share/common-licenses/GPL-2 in the Debian GNU/Linux distribution.
+
Added: trunk/buildme/debian/dirs
===================================================================
--- trunk/buildme/debian/dirs (rev 0)
+++ trunk/buildme/debian/dirs 2008-03-29 16:11:40 UTC (rev 2)
@@ -0,0 +1,3 @@
+usr/bin
+etc
+
Added: trunk/buildme/debian/rules
===================================================================
--- trunk/buildme/debian/rules (rev 0)
+++ trunk/buildme/debian/rules 2008-03-29 16:11:40 UTC (rev 2)
@@ -0,0 +1,58 @@
+#!/usr/bin/make -f
+# -*- makefile -*-
+
+# Uncomment this to turn on verbose mode.
+#export DH_VERBOSE=1
+
+PACKAGE := $(shell head -1 $(CURDIR)/debian/changelog | sed 's/^\([^ ]\+\) .*/\1/')
+BINARY := tools/$(PACKAGE)
+DESTDIR := $(CURDIR)/debian/$(PACKAGE)
+
+LIB := python-buildlib
+LDESTDIR := $(CURDIR)/debian/$(LIB)
+
+build: build-stamp
+
+build-stamp:
+ dh_testdir
+ touch build-stamp
+
+clean:
+ dh_testdir
+ dh_testroot
+ rm -rf build-stamp build
+ -find . -name '*.py[co]' -exec rm {} \;
+ -find . -name '*~' -exec rm {} \;
+ dh_clean
+
+install: build
+ dh_testdir
+ dh_testroot
+ dh_clean -k
+ dh_installdirs
+ python setup.py install --root=$(LDESTDIR) --no-compile
+ install -m 755 $(BINARY) $(DESTDIR)/usr/bin/
+ install -m 644 $(BINARY).conf $(DESTDIR)/etc/
+
+# Build architecture-independent files here.
+binary-indep: build install
+ dh_testdir
+ dh_testroot
+ dh_installchangelogs
+ dh_installdocs
+ dh_installman
+ dh_link
+ dh_pycentral
+ dh_compress
+ dh_fixperms
+ dh_installdeb
+ dh_gencontrol
+ dh_md5sums
+ dh_builddeb
+
+# Build architecture-dependent files here.
+binary-arch: build install
+# We have nothing to do by default.
+
+binary: binary-indep binary-arch
+.PHONY: build clean binary-indep binary-arch binary install configure
Property changes on: trunk/buildme/debian/rules
___________________________________________________________________
Name: svn:executable
+ *
Added: trunk/buildme/setup.py
===================================================================
--- trunk/buildme/setup.py (rev 0)
+++ trunk/buildme/setup.py 2008-03-29 16:11:40 UTC (rev 2)
@@ -0,0 +1,36 @@
+#!/usr/bin/python -tt
+# vim: sw=4 ts=4 expandtab ai
+#
+# buildME - Builder for Maemo Extras
+#
+# Copyright (C) 2008 Ed Bartosh <bartosh at gmail.com>
+#
+# This program is free software; you can redistribute it and/or modify it under
+# the terms of the GNU General Public License version 2 as published by the
+# Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
+# details.
+#
+# You should have received a copy of the GNU General Public License along with
+# this program; if not, write to the Free Software Foundation, Inc., 51
+# Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# $Id$
+
+
+from distutils.core import setup
+
+def debpkgver(changelog = "debian/changelog"):
+ return open(changelog).readline().split(' ')[1][1:-1]
+
+setup (name = "buildme",
+ description="Builder for Maemo Extras",
+ version=debpkgver(),
+ author="Ed Bartosh",
+ author_email="bartosh at gmail.com",
+ packages=['buildlib'],
+ license="Python",
+)
Property changes on: trunk/buildme/setup.py
___________________________________________________________________
Name: svn:keywords
+ Id Author HeadURL LastChangedDate LastChangedRevision LastChangedBy Rev Revision
Added: trunk/buildme/tools/buildme
===================================================================
--- trunk/buildme/tools/buildme (rev 0)
+++ trunk/buildme/tools/buildme 2008-03-29 16:11:40 UTC (rev 2)
@@ -0,0 +1,482 @@
+#!/usr/bin/python -tt
+# vim: sw=4 ts=4 expandtab ai
+#
+# buildme - builder for Maemo Extras
+#
+# Copyright (C) 2008 Ed Bartosh <bartosh at gmail.com>
+#
+# This program is free software; you can redistribute it and/or modify it under
+# the terms of the GNU General Public License version 2 as published by the
+# Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
+# details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA
+# 02110-1301 USA
+#
+# $Id$
+
+"""
+buildme. Builds source packages for Maemo Extras-devel repo
+"""
+
+__revision__ = "r"+"$Revision$".split(' ')[1]
+
+import sys, os, logging, pwd, shutil
+
+from glob import glob
+from optparse import OptionParser
+from commands import getstatusoutput
+
+from minideblib.ChangeFile import ChangeFileException
+
+from buildlib.fsm import FSM
+from buildlib.exceptions import Error
+from buildlib.sbdarch import SbdArch
+from buildlib.lock import Lockf
+from buildlib.dest import destfactory
+from buildlib.dsc import Dsc
+from buildlib.app import App
+from buildlib.logger import CustomLogger
+from buildlib.helpers import sendemail, readfile
+
+class UserLock(Lockf):
+ """ Lock for user """
+
+ _locktpl = "/var/tmp/buildme-%s.lock"
+
+ def __init__(self, username):
+ """ Lock the user. If not lockable - raise exception """
+
+ Lockf.__init__(self, self._locktpl % username)
+ if not self.lock():
+ raise Error("File %s is already locked" % self.lockfn)
+
+def infrasfactory(outdir, subdir, clean=True):
+ """ Creates output directory structure """
+
+ outd = os.path.join(outdir, subdir)
+
+ if clean and os.path.exists(outd):
+ shutil.rmtree(outd)
+
+ infras = {'logs':outd}
+
+ for dname in ('sources', 'results'):
+ dpath = os.path.join(outd, dname)
+ os.makedirs(dpath)
+ infras[dname] = dpath
+
+ return infras
+
+def parse_commandline(argv):
+ """ Parse commandline, check options """
+
+ parser = OptionParser(usage = "%prog [options]")
+
+ parser.add_option("--product", type="string", dest="product", help="specify one product for the build")
+ parser.add_option("--preserve", action='store_true', dest='preserve', help='preserve working files')
+ parser.add_option('--debug', action='store_true', dest='debug', help='enable debug output')
+ parser.add_option('--verbose', action='store_true', dest='verbose', help='verbose output')
+
+ options, argv = parser.parse_args(argv)
+
+ return (options, argv)
+
+# helpers
+def iter_items(env, item_name):
+ """ Create iterator for items container if doesn't exist.
+ Items container should exist in env.params and its name
+ should be equal item_name + 's', for example for 'product' item
+ container env.params['products'] should exist
+ This function returns next item from existent or created iterator"""
+
+ iterator_name = item_name + '_iterator'
+ container_name = item_name + 's'
+
+ if iterator_name not in env.params:
+ def gen():
+ """ generate iterator for container """
+ for item in env.params[container_name]:
+ env.params[item_name] = item
+ yield FSM.OK
+
+ env.params[iterator_name] = gen()
+
+ return env.params[iterator_name].next()
+
+# FSM handlers
+
+def init(env):
+ """ Check if there is anything to process. Lock .dsc if found """
+
+ product = env.params['options'].product
+
+ conf = env.params['conf']
+ if conf:
+ # build for only one product if specified in commandline
+ if product:
+ env.params['products'] = (product,)
+ # build for all products specified in config
+ else:
+ env.params['products'] = conf.sections()
+ else:
+ raise Error('Configuration file not found')
+
+ return FSM.OK
+
+def iter_products(env):
+ """ Iterate through products """
+
+ return iter_items(env, 'product')
+
+def init_product(env):
+ """ Initialize product build environment """
+
+ conf = env.params['conf']
+ product = env.params['product']
+ logger = env.params['logger']
+
+ incoming = conf.get(product, 'incoming_dir')
+
+ fnames = [fname for (_, fname) in sorted([(os.stat(fname).st_mtime, fname) for fname in \
+ glob(os.path.join(incoming, '*.dsc'))])]
+ if not fnames:
+ logger.info("No .dsc files found in '%s'" % incoming)
+ return FSM.NEXT
+ else:
+ env.params['fnames'] = fnames
+
+ # get destination names from config
+ env.params['dest_names'] = eval(conf.get(product, 'destinations'))
+
+ return FSM.OK
+
+def iter_destnames(env):
+ """ Iterate through destinations """
+
+ return iter_items(env, 'dest_name')
+
+def init_dest(env):
+ """ Initialize destination """
+
+ conf = env.params['conf']
+ product = env.params['product']
+ logger = env.params['logger']
+
+ # create destination
+ logger.debug("destination: '%s'" % env.params['dest_name'])
+ destination = destfactory(env.params['dest_name'])
+
+ # lock remote user
+ try:
+ env.params['lock'] = UserLock(destination.user)
+ except Error, exobj:
+ logger.debug(exobj)
+ return FSM.NEXT
+
+ # get target names tuple from config
+ tnames = eval(conf.get(product, 'targets'))
+
+ # set global variables for use in next steps
+ env.params['sbdtargets'] = SbdArch(tnames)
+ env.params['destination'] = destination
+
+ return FSM.OK
+
+def iter_fnames(env):
+ """ Iterate through filenames """
+
+ return iter_items(env, 'fname')
+
+def setup_build(env):
+ """ Setup build environment.
+ Verify .dsc file
+ setup summary log """
+
+ conf = env.params['conf']
+ product = env.params['product']
+ logger = env.params['logger']
+ fname = env.params['fname']
+ destination = env.params['destination']
+ result_dir = conf.get(product, 'result_dir')
+
+ uploader = pwd.getpwuid(os.stat(fname).st_uid).pw_name
+ info = "Processing package %s. Uploader: %s, builder: %s" % \
+ (os.path.splitext(os.path.basename(fname))[0].replace('_', ' '), uploader, destination.user)
+ logger.info(info)
+
+ env.params['uploader'] = uploader
+
+ try:
+ dsc = Dsc(fname, exclude = ('.deb',))
+ dsc.verify()
+ except ChangeFileException, exobj:
+ logger.info(exobj)
+ logger.info("skipped")
+ return FSM.NEXT
+
+ env.params['extraopts'] = ""
+ env.params['targets'] = env.params['sbdtargets'].gettargets(dsc)
+ env.params['dsc'] = dsc
+ env.params['firstbuild'] = True
+
+ infras = infrasfactory(result_dir, dsc.name)
+
+ # set up summary.log
+ handler = logging.FileHandler(os.path.join(infras['logs'], 'summary.log'))
+ handler.setLevel(CustomLogger.SUMMARY)
+ logger.addHandler(handler)
+
+ env.params['infras'] = infras
+
+ # output initial info into summary.log
+ logger.summary(info)
+
+ return FSM.OK
+
+def iter_targets(env):
+ """ Iterate through targets """
+
+ if 'iter_targets' not in env.params:
+ def gen():
+ """ generate targets iterator """
+ for (arch, target) in env.params['targets']:
+ env.params['arch'] = arch
+ env.params['target'] = target
+ yield FSM.OK
+
+ env.params['iter_targets'] = gen()
+
+ return env.params['iter_targets'].next()
+
+def do_build(env):
+ """ Copy files to destination. Run build command. """
+
+ logger = env.params['logger']
+ dsc = env.params['dsc']
+ destination = env.params['destination']
+ target = env.params['target']
+ extraopts = env.params['extraopts']
+ firstbuild = env.params['firstbuild']
+ infras = env.params['infras']
+
+ # get command from config
+ conf = env.params['conf']
+ product = env.params['product']
+ build_command = conf.get(product, 'build_command', False, {'target' : target, 'extraopts' : extraopts})
+ build_results = eval(conf.get(product, 'build_results', False, {'user' : destination.user, 'target' : target}))
+
+ logger.summary("Building %s for target '%s'" % (dsc.name.replace('_', ' '), target))
+
+ # set up command
+ destination.command = {'command' : build_command, 'results' : build_results}
+
+ # prepare sources only once:
+ if firstbuild:
+ logger.debug("Moving files to '%s'" % infras['sources'])
+ dsc.movefiles(dest = infras['sources'])
+ logger.debug("copy files to destination")
+ destination.copyto(dsc.getpaths())
+ env.params['firstbuild'] = False
+
+ # Run sbdmock
+ logger.debug('run build command')
+ (env.params['rcode'], _) = destination.run(dsc.fname)
+
+ return FSM.OK
+
+def copy_results(env):
+ """ Copy build results from the destination. Organize log files """
+
+ logger = env.params['logger']
+ destination = env.params['destination']
+ resultdir = env.params['infras']['results']
+ logsdir = env.params['infras']['logs']
+
+ logger.debug("copy results to '%s'" % resultdir)
+ destination.copyfrom(destdir = resultdir)
+
+ logger.debug("move logs")
+ buildlog = ""
+ for logname in ('build.log', 'root.log'):
+ fname = os.path.join(resultdir, logname)
+ suffix = 'OK'
+ if env.params['rcode']:
+ suffix = 'FAILED'
+ # if buildlog exists root.log is OK
+ if logname == 'root.log' and os.path.exists(buildlog):
+ suffix = 'OK'
+ if os.path.exists(fname):
+ if os.stat(fname).st_size:
+ buildlog = os.path.join(logsdir, '.'.join((env.params['arch'], logname, suffix, 'txt')))
+ shutil.move(fname, buildlog)
+ else:
+ os.unlink(fname)
+
+ logname = os.path.join(resultdir, 'sbdmockconfig.log')
+ if os.path.exists(logname):
+ os.unlink(logname)
+
+ return FSM.OK
+
+def check_build(env):
+ """ Check build. Clean srcs and results if needed """
+
+ logger = env.params['logger']
+ preserve = env.params['options'].preserve
+ resultdir = env.params['infras']['results']
+ sourcedir = env.params['infras']['sources']
+
+ # exit if build failed
+ if env.params['rcode']:
+ # remove results dir if empty
+ dcontent = os.walk(resultdir).next()
+ if not (dcontent[1] or dcontent[2]):
+ os.rmdir(resultdir)
+
+ logger.summary('FAILED')
+ return FSM.STOP
+
+ logger.summary("OK")
+
+ if not preserve and os.path.exists(sourcedir):
+ # delete sources if at least one build is OK
+ shutil.rmtree(sourcedir)
+
+ # build only architecture-dependent binaries for the following builds
+ # sources and 'all' binaries have been built in first build
+ env.params['extraopts'] = '-B'
+
+ return FSM.OK
+
+def send_email(env):
+ """ Send report email to the mailing list if specified """
+
+ conf = env.params['conf']
+ product = env.params['product']
+
+ if conf.has_option(product, 'mail_list'):
+
+ mail_list = conf.get(product, 'mail_list')
+ logdir = env.params['infras']['logs']
+ dsc = env.params['dsc']
+ builder_email = conf.get(product, 'builder_email')
+
+ # prepare subject variables to fill the template
+ (package, version) = dsc.name.split('_')
+ if env.params['rcode']:
+ build_status = "FAILED"
+ else:
+ build_status = "OK"
+
+ subject = conf.get(product, 'subject_template', False,
+ {'package' : package, 'version' : version, 'build_status' : build_status})
+
+ sendemail(xfrom = builder_email, xto = mail_list, xsubj = subject,
+ text = readfile(os.path.join(logdir, "summary.log")))
+
+ return FSM.OK
+
+def remove_sources(env):
+ """ Send email """
+
+ env.params['destination'].remove(env.params['dsc'].getpaths())
+
+ if env.params['rcode']:
+ return FSM.STOP
+
+ return FSM.OK
+
+def sign_results(env):
+ """ Sign results if needed """
+
+ logger = env.params['logger']
+
+ conf = env.params['conf']
+ product = env.params['product']
+
+ if conf.has_option(product, 'gpg_key'):
+
+ gpg_key = conf.get(product, 'gpg_key')
+
+ env.params['logger'].summary("Signing build results")
+ for changes in glob(os.path.join(env.params['infras']['results'], '*.changes')):
+ logger.debug("signing %s", changes)
+ (rcode, out) = getstatusoutput("debsign -k%s %s" % (gpg_key, changes))
+ if rcode:
+ raise Error("Can's sign build results. Debsing output: %s" % out)
+
+ return FSM.OK
+
+def copy_to_queue(env):
+ """ Copy results to incoming queue if needed """
+
+ conf = env.params['conf']
+ product = env.params['product']
+
+ if conf.has_option(product, 'repo_queue'):
+
+ repo_queue = conf.get(product, 'repo_queue')
+
+ logger = env.params['logger']
+ resultdir = env.params['infras']['results']
+ preserve = env.params['options'].preserve
+
+ logger.summary("Moving results to incoming queue")
+ for fname in glob(os.path.join(resultdir, '*')):
+ try:
+ if preserve:
+ shutil.copy(fname, repo_queue)
+ else:
+ shutil.move(fname, repo_queue)
+ except IOError:
+ raise Error("Error: can't move '%s' to '%s'" % (fname, repo_queue))
+
+ if not preserve and os.path.exists(resultdir):
+ shutil.rmtree(resultdir)
+
+ return FSM.OK
+
+def main(argv, options, logger, conf):
+ """ Main """
+
+ FSM(init_handler = init,
+ table = {init : {FSM.OK : iter_products,},
+ iter_products : {FSM.OK : init_product, FSM.STOP : None},
+ init_product : {FSM.OK : iter_destnames, FSM.NEXT : iter_products},
+ iter_destnames: {FSM.OK : init_dest, FSM.STOP : iter_products},
+ init_dest : {FSM.OK : iter_fnames, FSM.NEXT : iter_destnames},
+ iter_fnames : {FSM.OK : setup_build, FSM.STOP : None},
+ setup_build : {FSM.OK : iter_targets, FSM.NEXT : iter_fnames},
+ iter_targets : {FSM.OK : do_build, FSM.STOP : remove_sources},
+ do_build : {FSM.OK : copy_results},
+ copy_results : {FSM.OK : check_build},
+ check_build : {FSM.OK : iter_targets, FSM.STOP : remove_sources},
+ remove_sources: {FSM.OK : sign_results, FSM.STOP : send_email},
+ sign_results : {FSM.OK : copy_to_queue},
+ copy_to_queue : {FSM.OK : send_email},
+ send_email : {FSM.OK : None}
+ },
+ params = {
+ 'argv' : argv,
+ 'options' : options,
+ 'logger' : logger,
+ 'conf' : conf
+ }).run()
+
+if __name__ == '__main__':
+ logging.setLoggerClass(CustomLogger)
+ sys.exit(App().run(sys.argv, parse_commandline, main))
+
+# Local Variables:
+# mode: python
+# py-indent-offset: 4
+# indent-tabs-mode nil
+# tab-width 4
+# End:
Property changes on: trunk/buildme/tools/buildme
___________________________________________________________________
Name: svn:executable
+ *
Name: svn:keywords
+ Author LastChangedBy Id Rev Revision Date LastChangedDate
Added: trunk/buildme/tools/buildme.conf
===================================================================
--- trunk/buildme/tools/buildme.conf (rev 0)
+++ trunk/buildme/tools/buildme.conf 2008-03-29 16:11:40 UTC (rev 2)
@@ -0,0 +1,17 @@
+[DEFAULT]
+subject_template = [%(product)s]: %(package)s %(version)s %(build_status)s
+builder_email = Maemo Extras Builder <extras-cauldron-builds at garage.maemo.org>
+mail_list = Extras-cauldron builds <extras-cauldron-builds at garage.maemo.org>
+incoming_dir = /var/www/extras-devel/incoming-builder/%(product)s
+result_dir = /var/www/extras-devel/builder/%(product)s
+targets = ("armel:maemo-%(product)s-armel-extras-devel", "i386:maemo-%(product)s-i386-extras-devel")
+#repo_queue = /var/www/extras-devel/incoming/%(product)s/
+repo_queue = /var/www/extras-devel/builder/%(product)s/_extras-devel-incoming.debug/
+gpg_key = 0xb11fff2f
+build_command = sbdmock -r %(target)s -u %(extraopts)s
+build_results = ("/scratchbox/users/%(user)s/home/%(user)s/%(target)s/result",)
+
+[chinook]
+product = chinook
+destinations = ('builder1 at corsola.dmz:/home/builder1/', 'builder2 at corsola.dmz:/home/builder2/', 'builder3 at corsola.dmz:/home/builder3')
+
More information about the Extras-cauldron-commits
mailing list