Commit Graph

2308 Commits

Author SHA1 Message Date
Andreas Florath
b365d3c833 Refactor: remove unused functions
There were a couple of functions which were unused:
ensure_nbd, map_nbd, unmount_qcow_image, mount_qcow_image, ensure_sudo
Because some of them use 'trap ... EXIT' this hinders introducing a
separate exit phase - therefore they are removed.

(It would also be impossible to use them in the current setup, because
they overwrite the 'trap ... EXIT' of the disk-image-creates 'main'.)

Change-Id: If932a557dca9aea4864154ad6c4f286373d6dd7c
Signed-off-by: Andreas Florath <andreas@florath.net>
2016-06-02 21:06:12 +02:00
Jenkins
38dcce0fc4 Merge "Fix variable unbound error while REPOREF="*"" 2016-06-02 05:20:37 +00:00
Jenkins
419568884c Merge "Rework yum-minimal locale cleanup" 2016-05-31 06:42:20 +00:00
Ian Wienand
f5dff9c52a Rework yum-minimal locale cleanup
It turns out our manual locale cleanup is causing issues (see
I54490b17a7f8b2f977369044fcc6bb49cc13768e).  Upon further
investigation, I think this is a better approach than manually
deleting repos.

glibc on Fedora obeys the %_install_langs macro for reducing the
installed locales (as mentioned in the comments, F24 has moved to
having different packages, but worry about that later).

So our existing clear-out is really only required for CentOS, whose
glibc does not have any way to indicate to build less locales.
However, %_install_langs is still correct there, as it restricts some
of the translation files and other things installed with the %lang
macro in spec files.

This is complicated by us having to set this at glibc-common install
time, which happens with the "yum" from outside the chroot (i.e. on
trusty).  Since this is too old to have flags to pass this, we need to
fiddle with rpmmacros.

I've tested this with fedora-minimal builds and the locales file is
about 2MiB, which is what it was after the cleanups, and the listed
locales are only those we expect (i.e. it appears to be working).

Change-Id: I528a68beeb7b2ceec25ccbec1900670501608158
2016-05-31 15:14:24 +10:00
Andreas Florath
9fbc462e00 Remove Fedora 21 from test-build
Because Fedora 21 is EOL, the appropriate cloud images were
removed from the mirrors during the last days.
Because of the removal, currently all CI tests are failing.

This patch removes the Fedora 21 CI tests.

(Adding and supporting Fedora 23 is done with another patch -
because some additional changes are needed here.)

Change-Id: Ib85bb6fafd4f56ecc55dd420048f4d9e6e6969f3
Signed-off-by: Andreas Florath <andreas@florath.net>
2016-05-30 22:54:58 +02:00
Jenkins
5f4cac3303 Merge "Add a best-effort sudo safety check" 2016-05-26 17:31:00 +00:00
Liu Qing
d3255835d1 Fix variable unbound error while REPOREF="*"
If REPOREF="*", HAS_REF will be used without initialization. As -u is set
the script will terminate with error.

Change-Id: Ic1d88415adfef66dfc6c1d92610a45a9eb6359f3
Signed-off-by: Liu Qing <liuqing@chinac.com>
2016-05-26 09:24:59 +08:00
Jiri Stransky
1e9cf3a1c8 Remove deprecated overriding of cloud-init defaults
The "set to localhost by default" behavior for manage_etc_hosts has been
deprecated for more than a year now by change
Ia8582883f737548e2911d3f36a1943e5b236281b.

Setting that value to "localhost" is still possible, but it won't be the
default anymore. If the previous behavior of assigning the hostname and
FQDN to 127.0.0.1 is still desired in some environments, it can be
achieved by setting the DIB_CLOUD_INIT_ETC_HOSTS environment variable,
as the deprecation warning message suggested.

Change-Id: I5a19d46e2f305769a0c89c9d25d2e6be02910221
2016-05-23 17:44:32 +02:00
Jenkins
f0b57d5efd Merge "Fix apt-sources configuration for debian-minimal" 2016-05-21 20:10:18 +00:00
Jenkins
95c874abb9 Merge "Add dhcp-all-interfaces.target for syncing units" 2016-05-18 19:39:10 +00:00
Ben Kero
9a00bf144f Revert "Revert "Properly fail/trap in eval_run_d"" and fix PIPESTATUS
This reverts commit f07e33a2e9.

This change reverts the revert while fixing the underlying issue --
$PIPESTATUS needed to be encapsulated in ${}s

Change-Id: I1df06ffa7aecf4ea4b8e187dc756e9fc779786bc
2016-05-16 16:25:45 -07:00
Jenkins
99ddd58a15 Merge "Revert "Properly fail/trap in eval_run_d"" 2016-05-16 22:46:31 +00:00
Jenkins
2fffc9f892 Merge "Fix path issue for locale-archive.tmpl" 2016-05-16 22:01:47 +00:00
Jenkins
02822dc6b9 Merge "Add centos7 test" 2016-05-16 21:43:28 +00:00
Paul Belanger
0478fb15db
Fix path issue for locale-archive.tmpl
Change-Id: Id589c16aab46d447b3c21f00f3acfd06890e43d2
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2016-05-16 09:23:58 -04:00
Derek Higgins
f07e33a2e9 Revert "Properly fail/trap in eval_run_d"
This reverts commit 0d1d6bec7c.
This patch breaks tripleo-ci (the instack.qcow2 images is failing
to build) and was merged without passing CI.

Closes-Bug: #1582115
Change-Id: Ic4725ad0689c937fb4c8c792e1eaff5f4ea9ada9
2016-05-16 13:16:12 +01:00
Sam Betts
eb99fe7144 Add dhcp-all-interfaces.target for syncing units
Currently there is no way for a service to become aware that
dhcp-all-interfaces is finished configuring all the interfaces at
boot time. This causes problems for applications like the
ironic-python-agent which scans the interfaces when it first starts as
part of the inspection stage and can race against dhcp-all-interfaces
bringing up the interfaces, leading to inconsistent results.

This patch ensures that the dhcp-all-interfaces script runs before any
network interface is configured and brought up by the rest of the
system, and also ensures that the ironic-agent element also waits for
the network to be online before starting. This is done by using the
network targets provided by systemd.

Change-Id: Id9583b7f54361aa603a6229da598ad6a0f0f7938
2016-05-16 10:15:53 +01:00
Ian Wienand
e142014dbd Small doc update, add link to relnotes
I noticed we have no way easy way to get to the release notes.  I
updated the front-page text to be a simpler introduction/overview and
provided links to the relnotes and source directly.

Change-Id: I5e339baf2921752ca3d409d82e0cbfc856ead1f8
2016-05-16 15:13:39 +10:00
Ian Wienand
5d23abb338 centos-minimal: can be used with base
I'm not sure why this comment is here.  base works with centos-minimal
(we changed it in I2956aaa49ba3137a799f97e0983ab4b7c93a0a0c) and we've
got images deployed with such a configuration.

Update the comment

Change-Id: I3207f87dc29280183c0960ea863533a4d441081c
2016-05-16 13:36:55 +10:00
Jenkins
ae10241f3e Merge "Add 1.16.0 releasenotes" 2016-05-16 03:05:03 +00:00
Jenkins
e7625214f6 Merge "Properly fail/trap in eval_run_d" 2016-05-16 03:04:56 +00:00
Jenkins
d253a94187 Merge "Do not remove sudo in ironic-agent" 2016-05-16 01:41:08 +00:00
Gregory Haynes
0d1d6bec7c Properly fail/trap in eval_run_d
In phases which are called from eval_run_d (block-device.d) we do not
listen to exit 1's nor do we allow break=after-error. This is because
the run_d function is called in a subshell in order to grab its output.

This also turns on pipefail in the main disk-image-create script.

Change-Id: I88ab2e7104148437eabfe6880e3a1e5ebbb2c15d
2016-05-16 01:37:02 +00:00
Ian Wienand
8bda23a1d8 Add 1.16.0 releasenotes
Add some releasenotes for prior changes since 1.15.0

Change-Id: Ie2e6f7672df2076c2936ef4d0bbcc477abb0fa76
2016-05-16 11:09:39 +10:00
vmud213
45628993e8 Do not remove sudo in ironic-agent
"ironic-agent" element is currently removing sudo, which breaks other
elements such as devuser.  There appears to be no security or other
reason to do this, it's just the way it has always been.  Leave sudo
in as it is considered part of the base cloud images.

Change-Id: Ida9b1885f745146071e4b2d85ae59341ac85d5c8
Closes-Bug: #1572486
2016-05-16 10:39:04 +10:00
Andreas Florath
92355004d9 Fix apt-sources configuration for debian-minimal
The apt-sources element did not work with debian-minimal, because
the later one overwrote the /etc/apt/sources.list file created by
the apt-sources element.

Two changes were made:
o the debian-minimal uses now files inside the /etc/apt/sources.list.d
  directory.  Therefore there is no possibilty for clashes between those
  two elements any more.
o instead of only adding backports, also the updates and the security
  repository is added by default which gives perfect initial
  configuration for a stable system.
  If you want to use local mirrors with other naming schemas or an
  unstable tree, there is the possibility to fully specify the
  repositories.

Change-Id: I69dbaa34be3db3d667e6bd8450ef4ce04a751c70
Signed-off-by: Andreas Florath <andreas@florath.net>
2016-05-13 09:26:49 +02:00
Jenkins
77d03556df Merge "Add EPEL as requirement of centos-minimal" 2016-05-13 03:18:58 +00:00
Jenkins
4ad40f5387 Merge "Add test dependency installation on Gentoo" 2016-05-13 03:18:55 +00:00
Jenkins
0a3d9999d6 Merge "Add install-bin element" 2016-05-13 03:00:25 +00:00
Jenkins
1a9d088d44 Merge "tests/elements/fake-os: add '/tmp' as top level dir" 2016-05-13 02:17:38 +00:00
Jenkins
4e97654bf6 Merge "Updated from global requirements" 2016-05-13 02:11:14 +00:00
Jenkins
36a86afdad Merge "Install proliantutils in IPA's virtualenv" 2016-05-13 01:54:06 +00:00
Gregory Haynes
642f906476 Add install-bin element
The various distro elements include a copy of this script which installs
all files in the bin/ dir of the copied element hooks in to
/usr/local/bin. Lets share code rather than repeating ourselves.

Change-Id: I354382f8c42ede2e9b8c548b7df8367c03e6836e
2016-05-13 11:52:23 +10:00
Andreas Florath
0aa7f38a7f tests/elements/fake-os: add '/tmp' as top level dir
lib/img-functions:finalise_base() does a recursive unmount of /tmp to
cleanup any old mounts that might be in there (we do things like bind
mount caches in there).  unmount_dir fails if /tmp isn't a
directory.

The 'run_output_format_test.sh' uses the 'fake-os' element for
testing.  Because 'fake-os' did not create the '/tmp' directory, the
test script fails.

This patch fixes the problem: it adds '/tmp' to the list of
directories that are created.

It was implemented replacing the existing five 'mkdir'
command with a for loop over now six directories.

Change-Id: Ic7aca155492967b9819f129d3843cf95e136410e
Signed-off-by: Andreas Florath <andreas@florath.net>
2016-05-13 11:17:42 +10:00
Jenkins
8cb8806227 Merge "yum-minimal : better cleanup of initial yum failure" 2016-05-13 01:04:33 +00:00
Jenkins
88a9365816 Merge "yum-minimal: strip locale archive" 2016-05-13 01:04:27 +00:00
Jenkins
5657852b22 Merge "Fix up EPEL element" 2016-05-13 01:04:24 +00:00
Jenkins
c3cb375771 Merge "Split YAML & JSON parsing" 2016-05-13 00:42:45 +00:00
OpenStack Proposal Bot
4bb5a5f4f3 Updated from global requirements
Change-Id: I109295e1d88e672235a7d5fc470f9f1fa12d3170
2016-05-13 00:39:26 +00:00
Jenkins
d45d4d46c1 Merge "Add some output to dib-lint" 2016-05-13 00:38:09 +00:00
Jenkins
7c6f91fe37 Merge "Fix OpenSUSE support" 2016-05-12 00:36:23 +00:00
Ian Wienand
672705831f Add a best-effort sudo safety check
As motivation for this; we have had two breakouts of dib in recent
memory.  One was a failure to unmount through symlinks in the core
code (I335316019ef948758392b03e91f9869102a472b9) and the other was
removing host keys on the build-system
(Ib01d71ff9415a0ae04d963f6e380aab9ac2260ce).

For the most part, dib runs unprivileged.  Bits of the core code are
hopefully well tested (modulo bugs like the first one!).  We give free
reign inside the chroot (although there is still some potential there
for adverse external affects via bind mounts).  Where we could be a
bit safer (and could have prevented at least the second of these
breakouts) is with some better checking that the "sudo" calls
*outside* the chroot at least looked sane.

This adds a basic check that we're using chroot or image paths when
calling sudo in those parts of elements that run *outside* the chroot.
Various files are updated to accomodate this check; mostly by just
ignoring it for existing code (I have not audited these calls).

Nobody is pretending this type of checking makes dib magically safe,
or removes the issues with it needing to do things as root during the
build.  But this can help find egregious errors like the key removal.

Change-Id: I161a5aea1d29dcdc7236f70d372c53246ec73749
2016-05-09 15:41:38 +10:00
Jenkins
6c57795056 Merge "Add documentation for dib-lint" 2016-05-05 06:43:21 +00:00
Colleen Murphy
b5f51322a3 Fix OpenSUSE support
The dhcp-all-interfaces and simple-init elements did not have the ISC
DHCP Client package mapped for OpenSUSE, which caused DIB to fail with
"'isc-dhcp-client' not found in package names. Trying capabilities."

Similarly, the bootloader element did not have the grub-pc package
properly mapped for OpenSuse, which caused DIB to fail with "Package
'grub-pc' not found.".

This patch adds the package mappings for these elements so that the
opensuse element can be created and booted successfully.

Change-Id: Ife478158fec3a95de73a9206b38dcc6511d56cc8
2016-05-03 22:23:51 -07:00
Jenkins
83b607557e Merge "Remove cloud-initramfs-growroot package" 2016-05-03 13:24:53 +00:00
Nisha Agarwal
9d397d2568 Install proliantutils in IPA's virtualenv
The proliant-tools element helps to do RAID
configuration in ironic for HPE servers.
This fix proposes to install the proliantutils
in ironic-python-agent's virtualenv created
using ironic-agent element.

Closes-Bug: 1563648
Change-Id: If63c725a42740ab244a2b4004797cba09d0f154e
2016-05-02 01:56:26 -07:00
Ben Nemec
c6b6f269cc Add documentation for dib-lint
Prior to this, no user documentation of dib-lint existed, which
meant users had to read the dib-lint code itself to figure out
how it worked.  This changes adds documentation on using dib-lint
and the checks it currently supports.

Change-Id: I285c5cc680dd9fbd9bd3f667ef102be14e248114
2016-05-02 01:29:17 -05:00
Matthew Thode
d1b0fc16aa
Add test dependency installation on Gentoo
Here I add test deps for Gentoo, the main ones to call out are as follows.

* pyyaml: not installed by default and needed for the package element
* parted: not installed by default and needed for the VM element
* multipath-tools: not installed by default and provides kpartx

Change-Id: I07ce871fb3e684bfd9d06268d5c5bd118314f321
2016-04-30 20:56:01 -05:00
Ian Wienand
11128b0673 Use generic "dhcp-client" name
Every platform has a different name for their DHCP client, so use a
generic name "dhcp-client" in the package name and let everyone choose
their sub-name.  This also brings some consistency across simple-init
& dhcp-all-interfaces

Change-Id: I797aa7aacb13dfb7f35700463dc11d55552eb108
2016-04-22 11:31:54 +10:00
Ian Wienand
8b4a5e9919 Split YAML & JSON parsing
It turns out that invalid JSON can be valid YAML ... thus if you mess
up a pkg-map file that still works as a YAML file dib-lint will let it
pass, but when pkg-map later tries to open it as a JSON file, it
fails.

Parse each type separately to catch these problems.

Change-Id: Ib3985e7d1599ed6bf3b7a73b786a53177b71fae0
2016-04-22 11:20:57 +10:00