Commit Graph

3930 Commits

Author SHA1 Message Date
Clint Byrum
f6cc6bb185 Use ccache to cache all compiles between builds.
Many python extensions are built from source each time an image is
built. Repeating these is wasteful, so ccache is employed to eliminate
that waste. A cache dir outside the chroot is used to speed up
subsequent image builds.

Change-Id: Ib73563ddbe5f3be7454bfc54ab91cedb559a1304
2013-07-09 18:18:19 -07:00
Clint Byrum
3464064a87 Add environment.d hook to setup environment.
Before each phase dib-run-parts will pick these up to setup the
environment for the commands it runs.

Change-Id: I56697d7bc370918d4063eee67a1c35d4988a5359
2013-07-09 15:01:33 -07:00
Lucas Alvares Gomes
9f93451710 Curl to redo the request (Found 302)
The URL to download the Fedora 19 images is not persistent anymore,
they are now download from http://download.fedoraproject.org where the
request will be redirected to a different location. Passing the -L option
to the curl command will make it redo the request on the new place and
download the image correctly.

Related to bug #1199237

Change-Id: Ib9b9d65734992e2bf1a6888c5f2eb9de366f8204
2013-07-09 11:22:51 +01:00
Richard Su
9a81644d65 Fix loop0p2 does not exist on F19
On Fedora 19, there is a delay between the time loop0p2
is created to the time it is ready to be mounted.

This patches waits for 5 seconds for /dev/mapper/loop0p2
to be accessible. If it is not ready then the script
exits with status 1.

Fixes: bug #1197340

Change-Id: Iff0cfd0566673799a77500e0e24954f5f3743a17
2013-07-05 12:38:24 -07:00
Lucas Alvares Gomes
57a1304df9 Add arping to the fedora map-packages
Needed for the quantum-network-node element

Change-Id: I9a96476667c33a8260a60e7ad95c0f15166bc868
2013-07-05 10:21:33 +01:00
Richard Su
ad0fe876df Fix 50-firmware.rules no such file on Fedora 19
systemd on Fedora 19 has made /lib/udev/rules.d/50-firmware.rules
optional:
http://lists.freedesktop.org/archives/systemd-devel/2013-March/009708.html

Firmware is now handled by the in-kernel loader.

Added a file exist check for Fedora 19 and any other distros that
may have dropped the rules file. Backwards compatible with existing
distros.

Fixes: bug #1196409

Change-Id: I1d2acf3f88543736d75eb2e8766e83a3da194638
2013-07-03 14:10:33 -07:00
Lucas Alvares Gomes
095b0b3b61 Add mysql-devel to fedora map-packages
Needed for Ifc3580d4b59a5f1e97de77c0c4a875b123208100

Change-Id: I984a5c63a20bc79747d3d09affd9b45371c46702
2013-06-28 16:22:56 +01:00
Vipul Sabhaya
c0206aa52c Fix cache-url to use single '=' in test expression
When downloading the base image for the first time, the response
test expression fails with '[: 200: unexpected operator'.  This is
caused by using '==' in the test expression, which is not supported
in sh scripts.

Fixes bug 1195030

Change-Id: I66260814cb591371dc5c10f8436f90c2f18d78cf
2013-06-26 14:51:30 -07:00
Jenkins
860090bd1a Merge "Re-use cache_url() in fedora element." 2013-06-26 13:04:24 +00:00
Jenkins
2ced195d20 Merge "Fix pyOpenSSL on Fedora." 2013-06-26 04:25:34 +00:00
Jenkins
59a3adf734 Merge "Move functions to common-functions for reuse." 2013-06-26 03:57:45 +00:00
Richard Su
44df18af75 Fix cloud-init routing issue on Fedora
Add "NONZEROCONF=yes" to /etc/sysconfig/network to prevent
169.254.0.0 route from being created. This route interferes
with fedora baremetal nodes' access to the nova metadata
server at 169.254.169.254.

Change-Id: Iad77abfc97c0eba944680772f233389481a270ff
2013-06-25 14:49:18 -07:00
Robert Collins
a969be49a2 Move functions to common-functions for reuse.
Change-Id: I12a5687d94df678d7f7e8c0b075ee4aba1082156
2013-06-25 17:36:37 +01:00
Robert Collins
0b1ff7d4b0 Factor out element processing.
This reduces code duplication for hygiene, no functional changes.

Change-Id: I10c96da4f07bcd1e5aabfcfbd115ecaef7bb65a1
2013-06-24 16:30:25 +01:00
Robert Collins
9b01f2b129 Consolidate more ramdisk and disk-image code.
RAMDISK_ELEMENT is a spurious difference, to merge the code we need
only semantic difference to exist.

Change-Id: I7d38b2457b6fc838d1a536406bb3c34c00f2b1f9
2013-06-24 20:53:19 +12:00
Lucas Alvares Gomes
1ee499729b Fix pyOpenSSL on Fedora.
On fedora 18 the package pyOpenSSL is conflicting with one of the
packages needed to build the bootstack image. The problem gets sorted
if you install pyOpenSSL at the beginning of the image building process
and this is what this patch is doing.

Change-Id: I681683b81ad638dfeb690fde1bc7d08a2116aaed
2013-06-21 18:08:44 +01:00
Jenkins
539e8e2e69 Merge "Improve first time download of ubuntu images." 2013-06-21 13:51:54 +00:00
Jenkins
a37f7ca2a4 Merge "Enable Flake8 F*** checks." 2013-06-21 13:44:53 +00:00
Jenkins
aa8afcf47d Merge "Switch from pep8 to flake8/hacking." 2013-06-21 13:33:58 +00:00
Jenkins
e6655bde8c Merge "Use the source-repository interface." 2013-06-21 13:18:07 +00:00
Jenkins
1da4e4dbe4 Merge "Install git with source-repositories element." 2013-06-21 10:59:20 +00:00
Jenkins
1a681d3a4f Merge "Improve debugging of missing elements." 2013-06-21 08:07:49 +00:00
Joe Gordon
a98c0fe282 Improve debugging of missing elements.
Print the checked directory when cannot find an element.

Change-Id: I2c1f74e4975e66b6ec9d4122c02067bc2c473620
2013-06-20 14:26:58 -07:00
Joe Gordon
f95644470a Fix unit tests to have accurate return code.
Before 'tox -epy27' would not fail even if a unit test failed.

Change-Id: I3ed43fdd78d0d877b93b59e2eaaf5de289a17259
2013-06-20 14:21:40 -07:00
Derek Higgins
674a6c172b Use the source-repository interface.
Use the new source-repository interface to git clone
source repository.

Also there was a few thing in this element that needed to be fixed.
o url to the git repo wasn't curren't.
o pip and git are also dependencies that needed to be installed.

Change-Id: Ieb7be6576fc909aa4bba0233b7b81cab03395d76
2013-06-20 14:47:27 +01:00
Derek Higgins
f527b024b9 Install git with source-repositories element.
Git isn't a dependency of this element, but is a dependency of pbr
if you are doing pip install on a git cloned directoy, because this
looks to be a fairly common pattern it makes sense to have git on the
target image.

Change-Id: I49ee3fa4eac493b9dfa379a674ab9b3390cfc89a
2013-06-20 14:47:08 +01:00
Derek Higgins
866c55dced Use full path to dib-run-parts during firstboot.
Commit cc5ecb4e9c switched from using
run-parts in favor of dib-run-parts.

/usr/local/bin isn't part of PATH on Ubuntu causing
"/usr/sbin/dib-first-boot: line 8: dib-run-parts: command not found".

Change-Id: Ia7e401221add197edf880af82e0148f3ba7d2b9c
2013-06-20 08:13:30 +01:00
Joe Gordon
fcd1a2e7fd Enable Flake8 F*** checks.
Fix and enable F*** Checks by removing unused imports.

Change-Id: I39863a4664abff1349911546f16bfea055104e63
2013-06-19 22:42:13 -07:00
Joe Gordon
e39e3385b5 Switch from pep8 to flake8/hacking.
Use flake8/hacking, as is standard in other OpenStack projects, but
disable new checks -- they will be enabled in later patches.

Flake8 catches many useful things
http://flake8.readthedocs.org/en/2.0/warnings.html, and hacking enforces
the OpenStack Style guidelines, including things like one import per
line and alphabetical imports in order to make merging easier.

Change-Id: Ib43f7a6790bc17c0b4efdda0a5f891b5a90303c4
2013-06-19 22:32:55 -07:00
Jenkins
92094ecdc5 Merge "Add mechanism to send error messages to helper." 2013-06-19 23:09:22 +00:00
Mark McLoughlin
6c997fda97 Re-use cache_url() in fedora element.
The fedora element downloads images too, so we should re-use the caching
code from the ubuntu element.

There doesn't seem to be other examples of code shared between root.d
scripts. In the fedora and dpkg elements we copy install-packages into
the chroot, but that model doesn't apply when we're running scripts
outside of the chroot. Seems sane to just run it directly from the bin/
dir in the temporary hooks directory.

Change-Id: Iaa6aca660042fea323cab4271633a4bdbbc271b8
2013-06-20 00:04:59 +01:00
Mark McLoughlin
efb1f435d4 Improve first time download of ubuntu images.
If a cached copy of the file doesn't exist, cache_url() passes a
non-existent path to -z/--time-cond and you see this warning:

 Warning: Illegal date format for -z, --timecond (and not a file name).
 Warning: Disabling time condition. See curl_getdate(3) for valid date syntax.

It works just fine, but the warning is ugly.

Change-Id: Ic6f13a2c596b988308d7fca9cd1745e5d48ae5fb
2013-06-20 00:04:53 +01:00
Jenkins
ded16a0eb3 Merge "Use dib-run-parts on dib-first-boot." 2013-06-19 22:27:27 +00:00
Jenkins
35c4a0a8fa Merge "Adding element to get source for elements." 2013-06-19 22:23:19 +00:00
Jenkins
0e4b0556ed Merge "Make bash troubleshooting configurable" 2013-06-19 22:22:45 +00:00
Jenkins
e5e5845335 Merge "Reduce duplication between ramdisk and disk image codepaths." 2013-06-19 22:20:14 +00:00
Jenkins
a3fc33dc77 Merge "Make the finalise_base function less Ubuntu-ish." 2013-06-19 22:17:45 +00:00
Lucas Alvares Gomes
cc5ecb4e9c Use dib-run-parts on dib-first-boot.
At least on RedHat the run-parts command is very precarious and doesn't
even check for the return code of the scripts it runs causing the
dib-first-boot to delete itself even when something went wrong with the
configuration of the first boot of the image. Using the dib-run-parts
instead will make the whole first boot routine to be more robust and
less platform dependent.

Change-Id: I0073fd52fc1fc8848ca9277a84603124215cdb6e
2013-06-19 17:22:18 +01:00
Robert Collins
2c47b700be Reduce duplication between ramdisk and disk image codepaths.
Change-Id: Ie9e2d60ffe94f971953d52b7b5bce9c1a86fb1b5
2013-06-19 21:26:29 +12:00
Dan Prince
04bfd491ad Add mechanism to send error messages to helper.
Adds a new err_msg function which is used to:

-echo feedback to the deploy ramdisk console
-keep track of the first error message we hit so
 that we can send it along to the baremetal-deploy-helper.

Also, updates our wget request back to baremetal-deploy-helper
to include the first the first error message (if any) as
the 'e' parameter.

The err_msg uses a new simple safe_url_encode function to ensure
we don't send invalid characters in our HTTP post requests.

Change-Id: I5a623a6f66cde8d81ff1e75800dc2953ca2703a8
2013-06-18 14:51:04 -04:00
Dan Prince
0ff82aff46 Make bash troubleshooting configurable
Adds a new 'troubleshooting' function which encapsulates
the things we might want to do when something bad happens.
Typically this is echo'ing a simple message and launching a
bash shell.

The new troubleshooting function also makes use of a new
'troubleshoot' kernel param which if specific will allow you to
hang the deployment process and interactively debug the deploy_ramdisk
via a bash shell on a console.

Troubleshoot is disabled by default.

Fixes LP Bug #1191043.

Change-Id: I2cba8a9674075ba7e420027d40ef8cbe756cf07e
2013-06-18 14:28:52 -04:00
Lucas Alvares Gomes
1435f6e48d Make the finalise_base function less Ubuntu-ish.
The finalise_base function was creating a broken link at /etc/resolv.conf
when building a non ubuntu image.

Change-Id: I974d43fd6db365dbff9c85195e3b72950f3409da
2013-06-18 12:51:39 +01:00
Jenkins
207aeced03 Merge "deploy element: Call find_disk in loop w/ timeout." 2013-06-16 22:08:29 +00:00
Jenkins
13858d499c Merge "Add disable-selinux element." 2013-06-16 22:07:51 +00:00
Derek Higgins
ef2aa6e8a1 Adding element to get source for elements.
Fixes bug 1184943

Adding an element that allows other elements to register
repositories it would like to be retrieved for it.

Doing this outside of the chroot allows locally cached
repositories to be used. It also gives the ability to d-i-b
to specify what revision to use if an alternative to the
most recent is required.

Effectively allowing a CI system to test d-i-b and elements
without being effected by unavailable git repositories or
breakages in actively developed source code being used by
the elements.

Change-Id: I1527facebaad256a357af680e017b34b1788575d
2013-06-14 18:46:59 +01:00
Dan Prince
9fe9afb159 deploy element: Call find_disk in loop w/ timeout.
Updates our deploy element's init script so that it calls
find_disk in a loop until it returns true (meaning we have a disk).

Previously on some bare metal machines find_disk would return ""
which causes the subsequent iscsi target commands to fail due to
a bad backing store.

This commit also updates the error message when start_iscsi_target
fails to be a bit more accurate as well.

Fixes LP Bug #1190984.

Change-Id: I3cd535d6672c197c1c3c539c83bba36be7a14e18
2013-06-14 09:36:15 -04:00
Lucas Alvares Gomes
50e0512a28 Add disable-selinux element.
Having all security enabled is not always interesting, e.g. when
developing or testing new elements.

Change-Id: I828ecedc805ce4f89d60d185994eaa9c651d436a
2013-06-14 09:39:07 +01:00
Lucas Alvares Gomes
4e428c6844 dib-run-parts was failing with empty targets.
When the target directory is empty, the for loop printing our profile
data was failing because of a glob problem.

Change-Id: I34e612021c70d6b8f75423de0d8be969c52e58e2
2013-06-13 11:29:02 +01:00
Lucas Alvares Gomes
f17e169f69 Add /usr/local/bin to the secure_path variable.
This patch will append /usr/local/bin to the sudoers secure_path variable,
by doing so commands prefixed with sudo will then find some binaries
delivered by other elements in that path (e.g nova-rootwrap).

Change-Id: I1ebad8b1932620559a3d90aa39298f9b12e636fd
2013-06-12 11:01:01 +01:00
Robert Collins
03b437db0e Fix cloud-init-nocloud to actually work.
Bug 1124593 reported that ssh host keys were not being created, and the
lack of seed metadata files was the cause. Add those files and remove
the workaround.

Change-Id: I6fb55be7f8c7efd1f75aeba9123f495e312ff695
2013-06-12 13:28:27 +12:00