Commit Graph

1711 Commits

Author SHA1 Message Date
Jenkins
7e0e13106d Merge "Explain difference between two envvars" 2016-09-01 04:38:38 +00:00
Xiang Chen
ff234b65ea Explain difference between two envvars
Explain difference between 'DIB_OFFLINE' and
 'DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE'

Those variables are not redundant,they hava different effective ranges.
However,some people may be confused about this and reported a bug.
So,this difference should be writen in the README file.

Closes-Bug: #1506275

Change-Id: Ie5316de41d129bf98781708954f09ef0b2592b53
2016-08-25 10:07:29 +08:00
Jenkins
0ca20753ce Merge "Add "audit"package to yum-minimal" 2016-08-22 17:00:27 +00:00
Jenkins
c20a42051a Merge "Change DIB_IPA_CERT resulting file name" 2016-08-18 13:35:12 +00:00
Juan Antonio Osorio Robles
1787754d57 Change DIB_IPA_CERT resulting file name
Even though this file ends up in the /tmp directory, for readability
it's good to point out that server.pem is not necessarily the
certificate for a server, but can be a CA certificate which is
trusted if this option is used.

Change-Id: Iea27a702a844456e4472957438f75ed3819d62ca
2016-08-15 15:19:04 +03:00
Yolanda Robla Mota
c1bac651cb Allow to skip kernel cleanup
For some use cases, it can be useful to keep all the kernels
and not just keep the latest one. Add a parameter that allows
it, and continue cleaning up kernels by default.

Change-Id: Ia6e6c1fa18e3724c1eb89226151d81e9e748b793
2016-08-12 12:49:19 +02:00
Jenkins
cf2030c2f6 Merge "secure_path in sudoers: deal with possible quotes" 2016-08-11 00:21:14 +00:00
Ian Wienand
944b4fea0f Add "audit"package to yum-minimal
Kernels are built with auditing support, and without the audit deamon
logs bubble up to spam the console and /var/log/messages.  This
package contains the audit daemon that catches these messages.

Change-Id: Ie3e216bab33b27f2d67a9379ddc3e89d66449251
2016-08-08 17:54:20 +10:00
Jenkins
46ce2ac527 Merge "Optionally remove portage files" 2016-08-04 05:20:54 +00:00
Gabriele Cerami
5e957f6350 secure_path in sudoers: deal with possible quotes
Sometimes the secure_path option value in /etc/sudoers is surrounded by
quotes, in this case the current command creates an invalid entry and
it's not possible to sudo anymore.
This fix adjust the sed command to deal with possible quotes

Change-Id: Ifd6f9e29b3c0d04d6f65d3f55524ad202fb3294e
2016-08-03 15:24:07 +02:00
Matthew Thode
ffb8efda74
Optionally remove portage files
Optionally remove portage files, so that we can cache package and
keep the portage directory around, specifically for nodepool.

This also adds a section to the Gentoo readme about the variable
and renames the 00-gentoo-distro-name environment file to a more
appropriate name of 00-gentoo-envars.

Also brought up was the location of the gentoo-releng.gpg file,
this has been moved and the refrencing paths updated.

Change-Id: I20c91b36082828faa1ca481585acc5f9933211e1
2016-08-02 16:42:29 -05:00
Ben Nemec
e49d347ba9 Generalize logic for skipping final image generation
Since the ironic-agent element builds the ramdisk and extracts the
kernel itself, there's no need to actually generate an image at the
end of the process.  Previously the unnecessary image was being
deleted, but this wastes a bunch of time compressing and converting
the image.  It's better to just not create the image at all.

This change adds a noop element called no-final-image that
disk-image-create looks for in the element list and, if found, will
cause it to skip the final image generation.  This is more flexible
than the previous ironic-agent-specific method that would have
required changes to disk-image-create for every element that wanted
to behave similarly.

Note that this cannot be done using an environment variable, because
element environments.d entries do not propagate out to
disk-image-create.  It also doesn't make sense as a user option
because it should be set by the element author, not the user.

Change-Id: I168feb18f0d578b3babbe4784d3ef75e755e1ebd
2016-07-28 13:14:36 -05:00
Jenkins
ed116d60a9 Merge "Fix proliant-tools dependencies" 2016-07-27 21:19:32 +00:00
Jenkins
e884309272 Merge "yum-minimal: set locale.conf and tz in chroot" 2016-07-27 08:52:39 +00:00
stephane
b7d391f25d Fix proliant-tools dependencies
The proliant-tools element was missing a few
dependencies which were stopping it from
building correctly.

Change-Id: Ib7159a0baa7932d1571272cefffaf01d60e9debc
Closes-Bug: #1590176
2016-07-20 14:50:21 -07:00
Jenkins
71c68bf18d Merge "Fix packaging problems for Debian" 2016-07-20 21:21:08 +00:00
Andreas Florath
a8c8c61711 Fix packaging problems for Debian
This patch solves three issues with Debian packaging / apt:

o When building 'testing' only default apt sources is
  included - backports, updates and security are skipped because they
  do not exists.

o The default release for Debian was `unstable`: this is now fixed to
  `stable`.

o Starting a Debian Stretch VM that was build with diskimage-builder
  does not work, because some mandatory packages are missing.
  This patch fixes this problem: it adds the mandatory packages and
  the test case.

Change-Id: If49b5b162c4da1e074e9b19324839bc59d87dc57
Signed-off-by: Andreas Florath <andreas@florath.net>
2016-07-19 07:11:57 +02:00
Jeffrey Zhang
cf667e5dc3 Remove the escape in the centos7 README file
No need to add escape character '\' in the README file

Change-Id: I475b3e3d12c093a396bdcc81acbceeba03aed6df
2016-07-19 11:32:49 +08:00
Jenkins
b8d10afd8f Merge "package-installs: add list to arch and "not-arch" list" 2016-07-15 06:57:59 +00:00
Ian Wienand
b745821353 yum-minimal: set locale.conf and tz in chroot
We should be doing more to ensure initial configuration during
configuration.  Taken from the steps done by [1], here we set
locale.conf and a general timezone.

The only reliable UTF8 locale is en_US.UTF-8; we don't want to use C
locale as it causes havoc with things like python3 and unicode.  We
set locale.conf to this.

For Fedora 24 ensure we install the en_* locales too (this is really a
bug separate to this -- when you log in, by default ssh tries to copy
over your locale env variables, so logging into a F24 system would
result in using invalid locales for the most common en_* cases).

While we are here, setup a timezone link.  It turns out infra puppet
overwrites this later, but at least we have a sane default.

[1] https://www.freedesktop.org/software/systemd/man/systemd-firstboot.html#

Change-Id: Ib8951a97f1772bc5228c682e88628ff53400a923
2016-07-15 13:35:10 +10:00
Jenkins
96ed39aa35 Merge "Make Fedora 24 the default" 2016-07-14 05:26:04 +00:00
stephane
09317fccca Make Fedora 24 the default
Fedora 24 is the current release, so use it in the
fedora and fedora-minimal elements.

Change-Id: I0ac0d0767d4de9d28ae86d1344fd7c9d8876e5a1
2016-07-08 09:15:08 -07:00
James Slagle
1d629ccd46 Check sudoers file after editing
"visudo -c" should be run after the sudoers file has been edited. This
will ensure that the file is still syntactically correct, and exit 1 if
it isn't.  Otherwise, obscure errors can occur later on, and it is
difficult to track them back to this script as the source of the error.

Change-Id: Id0e5114d72c0779952a0c2c2c06696929c6c8b17
2016-07-07 09:51:43 -04:00
Jenkins
8b00250547 Merge "Make ubuntu-core support releases" 2016-07-06 23:49:31 +00:00
Jenkins
15e6d51dfe Merge "Revert "Pre-install pip/virtualenv packages"" 2016-07-05 21:42:48 +00:00
Ricardo Carrillo Cruz
a645fa4ffb Revert "Pre-install pip/virtualenv packages"
This reverts commit a85ce75d6b.

The reason for this revert is because it breaks CentOS dsvm jobs:

http://logs.openstack.org/35/332435/6/check/gate-ansible-role-cloud-launcher-dsvm-ansible-func-centos-7/13b3b66/logs/devstacklog.txt.gz

It is due to DevStack removing python-virtualenv by default, causing issues
as CentOS images have virtualenv installed via system-package and pip
by DIB.

Closes-Bug: 1599277
Change-Id: Ib0c01a7b3be32159e7c864a465fe9455e8d651ef
2016-07-05 20:09:39 +00:00
Jenkins
1b1a6a91eb Merge "Add new posix element." 2016-07-05 19:13:06 +00:00
Ian Wienand
8a1c8370a1 package-installs: add list to arch and "not-arch" list
Icf8a075224833fcfbbe2128e8802ff41c39f3c09 looked rather ugly, and it's
easy for us to expand the processing done in the arch list.

Change "arch" to a comma-separated list of architectures that should
match for install.

Add a "not-arch" list which will exclude the package from installation
on those architectures.  (An aside -- I considered making it just he
one list with foo,!bar,moo but ! has special meaning in YAML, so it's
easier to have two lists).

$ ARCH=ppc64 package-installs-squash --elements ironic-agent --path=./elements/ /dev/stdout | grep dmidecode
$ ARCH=ppc64 package-installs-squash --elements ironic-agent --path=./elements/ /dev/stdout | grep lshw
    "lshw",
$ ARCH=amd64 package-installs-squash --elements ironic-agent --path=./elements/ /dev/stdout | grep lshw
$ ARCH=amd64 package-installs-squash --elements ironic-agent --path=./elements/ /dev/stdout | grep dmidecode
    "dmidecode",

Change-Id: Ic69dd02a09e6f3ba9078a2377d8df29871a20db2
2016-07-01 21:31:59 +02:00
Bernard Cafarelli
097240c226 Export YUM variable in centos bases
Other fedora/centos elements can use the YUM variable, already set in
some base elements (fedora, centos-minimal). This commit also exports it
for centos/centos7.

Set a fallback value in pip-and-virtualenv element.

Change-Id: I681d77b924be035c81043bb34c72ec5f859e7108
Closes-Bug: 1598087
2016-07-01 12:14:38 +02:00
Jenkins
908ca1d079 Merge "dmidecode does not exist for ppc64/ppc64el" 2016-06-30 05:00:02 +00:00
Mark Hamzy
76bf793a39 dmidecode does not exist for ppc64/ppc64el
Adding three separate lines because the arch tag does not support a list.

Change-Id: Icf8a075224833fcfbbe2128e8802ff41c39f3c09
2016-06-29 16:51:46 -05:00
Jenkins
14152b1657 Merge "Run RHEL system unregister element earlier" 2016-06-28 09:32:53 +00:00
Jenkins
4493aadbd1 Merge "Clean more from ironic-agent ramdisk image" 2016-06-27 15:13:55 +00:00
Ben Nemec
4e5dcdd9cf Clean more from ironic-agent ramdisk image
While we already clean a number of things off the ironic-agent
ramdisk, there are a few more significant ones that we should add
to the list.

First is the kernel source.  If you're rebuilding your kernel on
the agent ramdisk after the initial image build, then you need to
re-examine your life choices. ;-)

Second is /var/cache.  On yum-based distros, this contains a large
number of yum cache files that take up significant space.  We don't
really want to be copying around caches when booting a ramdisk
anyway, so cleaning this is the right thing to do regardless.

Third is all *.pyc or *.pyo files.  There are a lot of these, so
they eat up significant space and bloat the number of files in the
ramdisk, which makes it take longer to build.  the only purpose for
the files is to slightly speed up Python app startup, and we
probably lose more time transferring the files over the network
than we would gain in quicker start times.  Note that we were
already trying to remove these, but for some reason I was still
seeing them show up in my final images.  It makes more sense to
put them in the same pruning command as all the others anyway.

Fourth is /usr/include.  These are files only needed for
compilation.  See above for my thoughts on compiling in a ramdisk.

These changes have reduced the agent ramdisk from 391 MB to 333 MB
in my local centos 7 builds, and have reduced the number of files
in the ramdisk by over 18000.

Change-Id: I550f9904b9afd12d48da9ba24559acb23133d076
2016-06-24 13:21:32 -05:00
Ian Wienand
1b4e6eb5f7 Handle locales install on Fedora 24
Fedora 24 has split locales into separate packages.  Testing revealed
what is possibly a bug in the choosing of default packages, so add a
small work-around to ensure the minimal locale pack is installed.

This appears to be the only change required for fedora-minimal with
Fedora 24; at least to build with the project-config infra elements.

Change-Id: I64438c34c572ed96211384ae1bfb45b2949e4318
2016-06-24 10:28:06 +10:00
Bernard Cafarelli
87379da56a Run RHEL system unregister element earlier
This does not need to be the last finalise step, and some late finalise
steps can disable the network (for example, Octavia amphora DIB cleans
resolv.conf at 99) Moving it to 60 also aligns it with rhsm-unregister
rhel6 element, and still allows to run subscription-manager steps
before.

Also fix an unbound variable error that appeared when both
BASE_IMAGE_FILE and DIB_CLOUD_IMAGES are unset.

Change-Id: Icb0e20b01479fea345e01309fc4bf3f7f639900c
2016-06-22 17:10:04 +02:00
Jenkins
f9d3bf0b1c Merge "Pre-install pip/virtualenv packages" 2016-06-17 05:05:11 +00:00
Ian Wienand
a85ce75d6b Pre-install pip/virtualenv packages
If we're installing pip/virtualenv from source, we need to make sure
we pre-install the packaged versions before the upstream
versions. Otherwise, CI jobs later on that depend on packaged versions
of pip/virtualenv can bring them in and overwrite the upstream
versions we have installed, which leads to a heck of a mess and
usually very confusing failures.

I have also moved in a small hack from system-config:install_puppet.sh
that we found was necessary when using pip versions from upstream.

Note this is not as much of an issue on Debian/Ubuntu, as they keep
their pip packages in a separate place to the system packages, so you
don't have these overwite conflicts as much.

Change-Id: Ib40708c07b939b84661c44df88a5a308fd0c7216
2016-06-17 09:20:57 +10:00
stephane
7eb72c0874 Ironic agent kernel should be owned by user building image
The initramfs file created by the ironic-agent element is
owned by the user running disk-image-create; ensure that
the other files created by the element are also owned by
the user.

Change-Id: I829db5b8e8bf1fc68face9cd2bda52d2a5ccdd4f
Closes-Bug: 1593010
2016-06-16 14:28:23 -07:00
Jenkins
d9525cfee9 Merge "Add cinder-backup mappings" 2016-06-10 01:20:45 +00:00
Ian Wienand
b9fdc70e32 Move pkg-map to dib-python
The latest Fedora/Ubuntu images don't ship python2 by default, so we
need to use our dib-python wrapper for this so we work in python3 only
environments.

Closes-bug: 1577105
Change-Id: I1048ceef35f269960216066924986eec6117ca00
2016-06-08 21:16:50 +00:00
Erno Kuvaja
6fd6b3bb25 Add cinder-backup mappings
'cinder-backup' to 'openstack-cinder-backup' mappings where necessary

Change-Id: I5f57401295d655112f0ad1d6442268df149daa7e
2016-06-07 14:15:47 +02:00
Jenkins
fb92e61082 Merge "Add cloud-initramfs-growroot for Precise" 2016-06-07 01:14:10 +00:00
Jenkins
dc149ec9fd Merge "Add release to pkg-map" 2016-06-07 01:01:09 +00:00
Jenkins
fb6fa48f85 Merge "Export FS_TYPE and remove hardcoded ext4 values" 2016-06-07 01:01:04 +00:00
Jenkins
13323c9495 Merge "Export die() function" 2016-06-07 01:00:50 +00:00
Jenkins
536e8a2dd2 Merge "Cleanup source-repositories output" 2016-06-07 00:59:02 +00:00
Ian Wienand
74d0185dde Cleanup source-repositories output
This element takes up a *huge* part of the logs of openstack-infra
builds as we go and cache every git tree.

This silences most of the noise which will reduce the mess
in the logs considerably.

Note that we've had logging on this turned down since
I91c5e55814ba9840769357261d203f4850e2eba6 but it has been
ineffective in stopping the log-spew output, see the
dependency change.

Change-Id: I60f06f84d57087c82b3907575bff125015d35171
Depends-On: I1e39822f218dc0322e2490a770f3dc867a55802c
2016-06-06 15:57:39 +10:00
Jenkins
38dcce0fc4 Merge "Fix variable unbound error while REPOREF="*"" 2016-06-02 05:20:37 +00:00
Jenkins
419568884c Merge "Rework yum-minimal locale cleanup" 2016-05-31 06:42:20 +00:00