summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
...
* cooker: no cached in progressbar and add ETAChris Larson2011-01-043-20/+12
| | | | | | | | | | | | Rather than updating the progress bar based on the recipe being processed (whether cached or parsed), consider only parsed recipes. This reduces the instability in progress rate introduced by the cached entries, and allows the ETA to be resurrected and be a bit more useful. (Bitbake rev: 618480f7739f6ae846f67a57bee5a78efb37839d) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cache: change to more incremental formatChris Larson2011-01-041-22/+33
| | | | | | | (Bitbake rev: 4fe4ffbef3885887c97eebe021edc3f23feab9ea) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cooker: pass back child exceptions to the serverChris Larson2011-01-041-6/+12
| | | | | | | (Bitbake rev: 0f68f8bcd0e0aa944f76f88a4a85c9bcc1e42bee) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* vercmp: don't choke on empty first version componentsChris Larson2011-01-041-1/+1
| | | | | | | (Bitbake rev: e75aa94e9477933c5a40021b2a8e844db54f29da) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Use __file__, not sys.argv[0]Chris Larson2011-01-042-2/+2
| | | | | | | (Bitbake rev: 97e92abe49663eee189c89c1dc91fe69891faf73) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cache: ensure 'pn' is included in the pkgvarsChris Larson2011-01-041-12/+12
| | | | | | | (Bitbake rev: cefc28a2aa1e5703119dacfd885c8c159a1e47a3) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* knotty: drop the ETA from the progressbar for nowChris Larson2011-01-041-1/+1
| | | | | | | | | | | | | | Currently, the progress bar is an indication of the processing of our recipes, which includes loading the cache file, then for each recipe, either adding the existing cached information to the CacheData or parsing the recipe from disk. These tasks clearly take different amounts of time, so the ETA is unreliable today. We'll resurrect this functionality after we revamp the progress handling, fully incorporating the load of the cache file. (Bitbake rev: 80867372dcbef91ebaf7d77a77ca871741dd3f74) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cooker: ensure that the cache sync completesChris Larson2011-01-041-2/+7
| | | | | | | | | | | Without explicitly joining the thread, it's possible for the process to end (e.g. after a bitbake -p) and kill off the thread without waiting for it to exit cleanly. So, register the thread join with atexit. (Bitbake rev: 97ce57e6f860d3e6f34cc7a603ed1eeac4f423d3) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cooker: get number of threads in constructorChris Larson2011-01-041-3/+3
| | | | | | | (Bitbake rev: e7fd259d1528f48b0812fb7b0a54dd6d5d22f4a9) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cooker: don't add info for skipped recipesChris Larson2011-01-041-2/+3
| | | | | | | (Bitbake rev: ac596d55de4ad86e1cdc84dabfea1350dd54efd0) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cooker: show progress bar before initializing the cacheChris Larson2011-01-041-13/+16
| | | | | | | | | | | | | This ensures that the time spent loading the cache from disk occurs with the progress bar up. Though the progress bar stays at 0% during this period, I think this is an improvement over the multi-second stall which occurred previously before the progress bar came up. Ideally, we'd integrate cache loading from disk into the progress display, but this is a first step. (Bitbake rev: f6d0a5c219f9deb84f702450d30d868ba6271f77) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cooker: save progress chunk value (total/100)Chris Larson2011-01-041-1/+2
| | | | | | | (Bitbake rev: 09333737cbeeb9875d938521ddcd519fc808bcc3) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cooker: stop loading the cache for -bChris Larson2011-01-044-32/+33
| | | | | | | | | | | Previously, the cache was actually being loaded from disk twice whenever using -b or -e -b. This also moves the bb_cache instance into the CookerParser, as it's not needed by the cooker itself at all. (Bitbake rev: dd0ec2f7b18e2a9ab06c499b775670516bd06ac8) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cooker: don't fire unnecessary parse progress eventsBob Foerster2011-01-041-1/+3
| | | | | | | (Bitbake rev: 7cf7d5f14405681496fced3640a50a20ef1acac1) Signed-off-by: Bob Foerster <robert@erafx.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cache: don't add info to cache if cache is disabledChris Larson2011-01-041-0/+3
| | | | | | | (Bitbake rev: f12bb303f3d86a68d0b3dda1112dd654b9251704) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cache: sync the cache file to disk in the backgroundChris Larson2011-01-044-43/+60
| | | | | | | | | | | | | | | | This version uses a thread rather than a process, to avoid problems with waitpid handling. This gives slightly less overall build time reduction than the separate process for it did (this reduces a -c compile coreutils-native by about 3 seconds, while the process reduced it by 7 seconds), however this time is quite insignificant relative to a typical build. The biggest issue with non-backgrounded syncing is the perceived delay before work begins, and this resolves that without breaking anything, or so it seems. (Bitbake rev: 5ab6c5c7b007b8c77c751582141afc07c183d672) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Simplify cache syncingChris Larson2011-01-041-22/+13
| | | | | | | | | | | Rather than adding nocache items to the cache, then copying the cache and removing them to sync it, don't add them in the first place. Also use 'with' for the cachefile. (Bitbake rev: 343b6f6255ad020c39e30742175a241f0859a5a6) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Experimental usage of the 'progressbar' moduleChris Larson2011-01-042-8/+396
| | | | | | | (Bitbake rev: 64feb03bc2accecb49033df65e0a939ef5ab5986) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Implement parallel parsing supportChris Larson2011-01-043-77/+157
| | | | | | | | | | | This utilizes python's multiprocessing module. The default number of threads to be used is the same as the number of available processor cores, however, you can manually set this with the BB_NUMBER_PARSE_THREADS variable. (Bitbake rev: c7b3ec819549e51e438d293969e205883fee725f) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cache: don't write out the cache unnecessarilyChris Larson2011-01-041-1/+5
| | | | | | | | | | | If the only recipes's we reparsed this run were those flagged as not to be cached, there's no point in re-saving the cache, as those items won't be included anyway. (Bitbake rev: 1e0c4dbcbec886a30b89f8b4bb365c3c927ef609) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cache: create and use a RecipeInfo classChris Larson2011-01-042-175/+197
| | | | | | | | | | | | | | | | | | This class holds the particular pieces of information about a recipe which are needed for runqueue to do its job. By using it, I think we improve code clarity, reduce method sizes, reduce overuse of primitive types, and prepare for parallel parsing. In addition, this ditches the leaky abstraction whereby bb.cache attempted to hide the difference between cached data and a full recipe parse. This was a remnant from the way things used to be done, and the code using it had to know the difference anyway. If we choose to reimplement caching of the full recipes, we can do it in bb.parse, in a completely transparent way. (Bitbake rev: 992cc252452221f5f23575e50eb67528b2838fdb) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Prefer xrange over range for small performance gain.Bob Foerster2011-01-044-37/+37
| | | | | | | | | | | | | | range() allocates an actual list when called. xrange() is just an iterator and creates the next range item on demand. This provides a slight performance increase. In python 3, range will do what xrange does currently, but the upgrade will be handled by the 2to3 tool. (Bitbake rev: 73b40f06444cb877a5960b2aa66abf7dacbd88f0) Signed-off-by: Bob Foerster <robert@erafx.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Revert "svn fetcher: warn people to switch to SRCREV"Chris Larson2011-01-041-2/+1
| | | | | | | | | 1) too spammy 2) can be implemented in the metadata instead This reverts commit 8da9744fcdf856abebcfbe9e3bc1b8cf07bc317b. Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cache: make loadDataFull a classmethodChris Larson2011-01-041-4/+5
| | | | | | | (Bitbake rev: 16a9d689e61c35cfca94bbecd9772eab3c5072ba) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* svn fetcher: warn people to switch to SRCREVBernhard Reutner-Fischer2011-01-041-1/+2
| | | | | | | | | as noted by rp in ac00ca89a4e43cd4f38ba86455079d31be78e644 (Bitbake rev: 8da9744fcdf856abebcfbe9e3bc1b8cf07bc317b) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* fetch: be more pythonicBernhard Reutner-Fischer2011-01-049-64/+21
| | | | | | | | | no functional changes (Bitbake rev: e88834fb7c6821cc29c12d296f2edd51f6eb3746) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* fetch: use os.path.joinBernhard Reutner-Fischer2011-01-041-1/+1
| | | | | | | (Bitbake rev: c360b01df18d90a513a3d61d395f905102e7568e) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* fetch: add common helper _strip_leading_slashes()Bernhard Reutner-Fischer2011-01-046-22/+13
| | | | | | | | | | Several fetcher need a way to strip leading slashes off a local path. This helper-function consolidates all such occurances. (Bitbake rev: 823a02185ed109054c6c1ae366221aaed0353f24) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* fetch: rename suppports_srcrev to supports_srcrevBernhard Reutner-Fischer2011-01-046-8/+8
| | | | | | | | | osc had it already spelled correctly?! (Bitbake rev: b8bb4433de7a981c6826173e926ca34705c4ac70) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cache: Add debug msg for a nonexistant dep fileChris Larson2011-01-041-0/+2
| | | | | | | | | | If a recipe depends on a file, and that file is out of date, we show a message, but if that file was removed, we do not, until now. (Bitbake rev: 67984ba0ac2db79874541bc031f2e3e9ff7a6c32) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cache: use new style classesChris Larson2011-01-041-6/+3
| | | | | | | (Bitbake rev: e084430446be2544dd1a6b627088f888c37cc7f0) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cache: pyflakes/pep8/pylint cleanupChris Larson2011-01-041-29/+40
| | | | | | | (Bitbake rev: 06420ff839ffc37de3e42474b8b0e47c4608a985) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cache: use set for clean, checkedChris Larson2011-01-041-11/+9
| | | | | | | (Bitbake rev: 3a776ef947e23b1a04bdd9e78a3a0c4616249b0e) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cache: make use of defaultdictChris Larson2011-01-041-34/+10
| | | | | | | (Bitbake rev: 7171019b11ad656d0edb979564941fcf92ab0a02) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Pre-explode rundeps/runrecs in CacheDataChris Larson2011-01-043-8/+10
| | | | | | | (Bitbake rev: 0c8e0f3191252ccedb83c431c48e5c7d905e2717) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* persist_data: handle locked db for SELECTChris Larson2011-01-041-3/+1
| | | | | | | | | | Parallel processes interacting with the persist_data db can quite easily explode without this. (Bitbake rev: b3d5432cff0ff28f4c8a5bcf10efa3e383b4fd4d) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* BBHandler: use os.path in inherit()Bernhard Reutner-Fischer2011-01-041-1/+1
| | | | | | | (Bitbake rev: 5b85de2c71973ba490b95a5d9ab634635f395142) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* BBHandler: simplify supports()Bernhard Reutner-Fischer2011-01-041-1/+2
| | | | | | | (Bitbake rev: 858d704d713d15bf97053eb1374758c74b0d8874) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bb.cache: only log if the respective action was takenBernhard Reutner-Fischer2011-01-041-5/+8
| | | | | | | | | | This avoids alot of misleading log-messages like "Removing FOO from cache" if FOO was not in the cache and as such is not a removal candidate. (Bitbake rev: de34a403e206867e09410ad4925c7b9cff04fee6) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bb.pysh: add Case support to format_commandsChris Larson2011-01-041-0/+3
| | | | | | | (Bitbake rev: 6e24f573a0e95068eb9237c1d264ad1148b2f690) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bb.pysh: fix writing pyshtables all over the placeChris Larson2011-01-041-1/+4
| | | | | | | (Bitbake rev: ff4753e362714a3c4c759c2fad8a9e5b8fe5bef5) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Move the pysh package into the bb packageChris Larson2011-01-0410-1/+1
| | | | | | | | | | The pysh we're using is modified, and we don't want to risk it conflicting with one from elsewhere. (Bitbake rev: 1cbf8a9403b4b60d59bfd90a51c3e4246ab834d6) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Allow mercurial fetcher to follow tipPeter Chubb2011-01-041-1/+8
| | | | | | | | | | | | | | | | | There are occasions when developing when I want a package always to grab the latest copy of a package. Witht eh CVS fetcher you can do this by setting the `date' tag to `now'. This patch adds similar functionality to the mercurial fetcher: if the revision to fetch is `tip' then always grab from the server, and don't use the cached tarball. Oh, and I fixed a typo in the Class comment. (Bitbake rev: 01b85608d8a37f8af66dfd80133e950120679079) Signed-off-by: Peter Chubb <peter.chubb@nicta.com.au> Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Fix GraphViz .dot output for rdepends and rrecsChris Larson2011-01-041-2/+2
| | | | | | | | | | | Use bb.utils.explode_deps to break up the rdepends and rrecommends strings. This fixes the same issue which was fixed by a number of patches floating around, but uses explode_deps rather than regular expressions. (Bitbake rev: 83cdb23f8b89453a3527a276bd0b4deb85d63deb) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Make 'cache is clean' message debugChris Larson2011-01-041-1/+1
| | | | | | | (Bitbake rev: 92637a355d55cb66de91b4314bc0e7cf1ac64ade) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake: lib/bb/fetch/hg: fix fetching from a mercurial repositoryEric BENARD2011-01-041-0/+1
| | | | | | | | | | | | | | * without this fix, we get : updating working directory 74 files updated, 0 files merged, 0 files removed, 0 files unresolved abort: There is no Mercurial repository here (.hg not found)! (Bitbake rev: 75ea005ac8fc05b2b3afca803d77a6b5f558efee) Signed-off-by: Eric Bénard <eric@eukrea.com> Tested-by: Frans Meulenbroeks <fransmeulenbroeks@gmail.com> Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Resurrect display of failed filesChris Larson2011-01-041-0/+4
| | | | | | | | | | This was inadvertantly removed when trying to reduce the amount of duplicated information the user sees when a failure occurs. (Bitbake rev: 850d6158ea9daa58e896fd6b258d586df797dcf4) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Access metadata vars as locals in python snippetsChris Larson2011-01-041-1/+14
| | | | | | | | | | | | | Example: FOO = "bar" BAR = "${@FOO + '/baz'}" ${BAR} == "bar/baz" (Bitbake rev: 606fa1fd97cbd47a6a7ebdc7a2e6aa93a8f65cf5) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Fix __getitem__ for DataSmartChris Larson2011-01-041-2/+6
| | | | | | | | | | Ensure it raises KeyError for a missing key, this is required to use this as a mapping in various places, e.g. as locals in an eval. (Bitbake rev: 8d661ce0c303e8d69f17c1d095545d5ed086d1d5) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* fetchers: Use tar --exclude pattern to remove SCM filesKhem Raj2011-01-045-5/+5
| | | | | | | | | | | | This option will exclude the SCM metadata from tar files. Tested with gcc where svn tar which used to be 156M for gcc 4.5 is now 77M (Bitbake rev: f264cb6d43472525ad787b0887764ea696ec52ba) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
generated by cgit v1.2.3 (git 2.46.0) at 2024-11-09 01:38:07 +0000