Commit Graph

66 Commits

Author SHA1 Message Date
Chris Jones
4c7a3bf79f Improve source-repositories git caching.
We now unconditionally cache the repositories being fetched with
source-repositories.

Additionally, by improving the way we handle repository refs, we are now
able to build images with code taken directly from gerrit reviews.

Change-Id: Ifd4c5691f6761eb5551663e6d0aa1c0e42afced3
2013-12-20 14:48:30 +00:00
Mark McLoughlin
10eac21043 Log the repository fetches in source-repositories
Make the source-repositories element log which repo it's cloning from
so it's easier to tell what's going on.

Change-Id: I224f7bd98bc3d0ba94313dae9063d4ba4e25bdaa
2013-12-05 06:28:13 +00:00
Mark McLoughlin
4f97b087d5 Fix typo in source-repositories README
Change-Id: Iaf5367099544fc5d4699140af642dfd392c7090f
2013-11-25 10:54:18 +00:00
Victor Lowther
43b70ce224 /bin/bash all the things.
Since we are using bash syntax in some of the element fragments,
we should make sure we use bash for all of them, so that things don't
break on systems where /bin/sh != /bin/bash.

Change-Id: If2f043c57aa4e1492b7f9839213ef6123f683612
2013-11-17 16:59:31 -06:00
Monty Taylor
5e3b5320e4 Remove github references
OpenStack runs git.openstack.org which is more reliable and responsive
when projects operate within OpenStack Infra. Replace all of the
references to github with referneces to git.openstack.org.

Change-Id: Ib3ece85aba6451801487b0bdbd83147e39d9e155
2013-09-18 08:17:10 -05:00
Steve Baker
9e5bc37265 Add support for file to source-repositories
Adds a file type to source-repositories to allow individual files
to make use of source-repositories caching.

REPO_DIRECTORY has been rename to REPO_DEST since it is now sometimes
a file.

This change also fixes the incorrect path to the cache-url in the
tar type.

Change-Id: Ib79e41969c982a02f0235318d9f254b39c3c6d93
2013-09-04 10:16:49 +12:00
Robert Collins
c7c4ec19a3 Capture the repositories from source-repositories.
We try to expose metadata to users about what dib included in the
image and source-repositories was not doing that, so this fixes it. It
also permits easy introspection from other elements (such as an
upcoming PyPI cache layer).

Change-Id: I1e48e65f69ebe1f4556cc3bf35ccf195e99bcc76
2013-08-05 09:52:28 +12:00
Monty Taylor
4535c5323a Ignore emacs autosave files in source-repositories
source-repositories uses find to discover things it should import. If
someone happens to use emacs, and happens to have had an autosave file
in a directory with a wildly broken version of a source-repositories-
file, then dib will attempt to execute the contents of the autosave file
and the user will be sad. Adding a filter to the find command for files
ending in ~ will prevent pain and suffering.

Change-Id: I27c8a4b53d3c9b913ca926db16b4271941f6fb5d
2013-07-27 14:53:29 -04:00
Jenkins
5f52562570 Merge "Support repo names with multiple '-'s" 2013-07-19 19:45:42 +00:00
Derek Higgins
a4a0523f2b Support repo names with multiple '-'s
The search and replace that converted dashes in REPONAME only converted
the first dash. They should all be converted.

Change-Id: I76a48cf5bdfa8664f0a65d0238fa4da7d80305db
2013-07-19 17:45:01 +01:00
Derek Higgins
51216dbd2d Set work-dir to cached repository
source-repositories does a git reset on the .git directory of cached
repositories. But doesn't specify the directory to reset. A working
directory needs to be specified so that the $PWD isn't used.

Without this change $PWD is polluted with the contents of repositories
being cached.

Change-Id: Ic37b702ac579bf766bb2204a988fa9468d308abf
2013-07-19 17:40:16 +01:00
Robert Collins
e187a402ae Move the getsources hook earlier.
Caching PyPI objects will require inspecting the content downloaded by
the source repositories, so needs to run after it.

Change-Id: I551c075e0bb6853b2ef79268b7f6455a4f06743d
2013-07-18 23:06:30 +12:00
Robert Collins
79ca4d901c Cache repository-sources data.
Cloning large repositories over the internet can take considerable
time.  Caching them locally makes repeated image builds significantly
faster, so lets do that.

When users override the element source they will often be using a
local repository, so in those cases I don't cache - but we could
easily change our minds on that in the future.

Change-Id: I9822362cf722b904d9806dbbb4bb07cfe2b33437
2013-07-18 22:40:23 +12:00
Derek Higgins
195053108b Adding docs for the source-repositories element
Change-Id: I57244a5cc2f563b2e69eb49388c3c6b49202c9eb
2013-07-10 13:04:39 +01:00
Derek Higgins
f527b024b9 Install git with source-repositories element.
Git isn't a dependency of this element, but is a dependency of pbr
if you are doing pip install on a git cloned directoy, because this
looks to be a fairly common pattern it makes sense to have git on the
target image.

Change-Id: I49ee3fa4eac493b9dfa379a674ab9b3390cfc89a
2013-06-20 14:47:08 +01:00
Derek Higgins
ef2aa6e8a1 Adding element to get source for elements.
Fixes bug 1184943

Adding an element that allows other elements to register
repositories it would like to be retrieved for it.

Doing this outside of the chroot allows locally cached
repositories to be used. It also gives the ability to d-i-b
to specify what revision to use if an alternative to the
most recent is required.

Effectively allowing a CI system to test d-i-b and elements
without being effected by unavailable git repositories or
breakages in actively developed source code being used by
the elements.

Change-Id: I1527facebaad256a357af680e017b34b1788575d
2013-06-14 18:46:59 +01:00