Commit graph

48 commits

Author SHA1 Message Date
Matthew Thode
97e4627d87
update various gentoo bits
Update an rc-update call to only be made if running openrc instead of
all gentoo profiles (systemd does not have rc-update).

Add python3-pyyaml package mappings.

Update serial console to support multiple arches.

Update open-iscsi and open-isns keywords (looks like upstream merged
some musl fixes).

Update the kernel and initramfs file name globs for the
gentoo-kernel-bin usage.

Change-Id: I259bffed3a3e3f92be2210ead6bdfa383917d457
Signed-off-by: Matthew Thode <mthode@mthode.org>
2020-08-24 10:21:35 -05:00
Matthew Thode
bea81bd234
update gentoo to allow building arm64 images
Adds:

1. grub-efi package mappings
2. efi-64 support
3. default (openrc) arm64 profile
4. systemd arm64 profile

Cleans up the keywords and use flags in 02-gentoo-02-flags.  Most stuff
was stablized.  Also cleaned up some formatting for the if statements.

Enables less trusted overlays (up to the end user to verify).

in 10-gentoo-image I cleaned up some bash lint things as well.
using && instead of -a and avoiding $?

Change-Id: I3dffe1aab4bbdc4946a9bf2269bf0cde49529a4e
2020-08-17 23:50:39 -05:00
Your Name
e384da1a98
update gentoo-releng gpg key
Simplify gpg checking by caching a keyring instead of keys to import.

Change-Id: I5ed74ec0e12732aec40ef31377e72d7ddc347f95
Signed-off-by: Matthew Thode <mthode@mthode.org>
2020-07-12 17:50:00 -05:00
Zuul
9f7c35b238 Merge "add musl profile to gentoo" 2020-07-06 18:52:39 +00:00
Matthew Thode
64787d8ec2
add musl profile to gentoo
Change-Id: Id954ffe4d9b019c7e7bb648725ff7f976c929b4d
2020-06-22 18:37:30 -05:00
Matthew Thode
c7b6f0d6a1
add more python selections to gentoo
Change-Id: If4420e9ae7870739aac8a673ff290f9ed1391acf
2020-06-14 01:16:18 -05:00
Matthew Thode
041bdd331a
use stage3 instead of stage4 for gentoo builds
The main reason for using the stage4 is now gone (kernel compile).
Install and use the distro provided binary kernel package.

In addition to this, set the locale and timezone, beyond that very
little was done in the gentoo stage4.

Change-Id: I541b7d9b807e2357398ae1c249b1978958dd1137
Signed-off-by: Matthew Thode <mthode@mthode.org>
2020-05-27 18:50:16 -05:00
Matthew Thode
87e7f7b869
allow building of Gentoo images for non-systemd profiles
Change-Id: I61e3ba391eaaf9b300f88b082c03116388401ba3
2019-12-20 17:51:55 -06:00
Andreas Florath
3636b40f74 Introduce manual setting of DIB_INIT_SYSTEM
The current implementation evauates the dib-init-system
script too early.  Also it looks that there is no simple
way of getting the info about the init system automatically:
another element can install (later on) a different
init system.  Therefore the only reliable way of setting
this is manual.

Change-Id: I6e9ffa1bdb3154f488f4fd335b197699b86aacd4
Signed-off-by: Andreas Florath <andreas@florath.net>
2019-11-21 12:38:15 +11:00
Matthew Thode
9755c4f9a2
update gentoo systemd profile to 17.1 from 17.0
Upstream is now publishing 17.1 profile systemd stages

Also updates the docs that were forgotten in the last patch

Change-Id: I0f2e7976845b1d3c55ffe8869eec0bc04a191252
2019-08-19 15:13:09 -05:00
Zuul
f59ffcd9f3 Merge "fix comments / spelling errors in gentoo element" 2019-07-30 07:22:26 +00:00
Zuul
3d995645e6 Merge "update version of open-iscsi that is installed on musl" 2019-07-30 02:37:18 +00:00
Matthew Thode
559d2bcf3b
fix comments / spelling errors in gentoo element
Change-Id: I41d906a99470599da3cfac7aaa0350232ce79316
2019-07-29 08:54:16 -05:00
Matthew Thode
f1d7e902e3
support alternate portage directories
The 17.1 profile changed the defaults used in portage for where we store
our repo, distfiles and binpkgs.  Some portage related variables need to
be set deterministically.  17.1 is no enabled for Systemd's profile.

Change-Id: Ib55f6875c5cb461c3c530b51d7420ce3dc8da360
2019-07-26 19:30:01 -05:00
Matthew Thode
ed95b4eba1
update version of open-iscsi that is installed on musl
Newer versions of open-iscsi seem to compile on Gentoo / musl.  Use them
if we can.  This also removes the cap on open-iscsi.

Change-Id: I596cb61494e459a419bce6a63deff89f9e78fe23
2019-07-22 14:30:32 -05:00
Matthew Thode
de94e07a86
disable autounmask for emerge
autounmask=y (default) changes portage depsolving, causing errors
(mostly often seen in perl and binpkg related issues).

Disabling this functionality for DIB builds is OK as the enviroment is
not passed on post build and the build process is not interactive
anyway.

Change-Id: Ife9ace246bec16864ee4982bc456763af5dff2e8
Signed-off-by: Matthew Thode <mthode@mthode.org>
2019-07-05 12:41:51 -05:00
Dirk Mueller
421a0fa541 fail early when lates build information can not be fetched
When the mirror returns a error, it was trying to interpret the error
message (e.g. <html><title>Internal server error..) as a download link.
By using -f on curl we get an empty reply and an exit code, which, as
we run in set -e mode, aborts.

Change-Id: Ibaa39aedb7db286f859c4b090114c6a233b150c7
2019-05-31 16:09:25 +10:00
Matthew Thode
afcac1922c
allow the use of non-bzip compressed stages for building gentoo
Upstream is switching to xz so we need to be able to support it.

Change-Id: I382cc3e8038e2e552c553c526a990a01e51aeb12
2019-05-24 09:32:57 -05:00
Zuul
0f8d340c6c Merge "Update gentoo-releng gpg key" 2019-02-28 00:33:06 +00:00
Matthew Thode
b2cc91d276
Update gentoo-releng gpg key
A new signing subkey is used.

Change-Id: Idc9aceba7ee144fd0307737c24991acfacf68985
2019-02-27 15:18:25 -06:00
Zuul
703549412d Merge "update spelling errors" 2019-02-25 08:49:56 +00:00
Matthew Thode
f4a1c7f89f
fix systemd import-tar for gentoo
use a newer version of systemd
Fixes https://review.openstack.org/#/c/608102

Change-Id: I23fd671adb893f3abd9fbc65382f2aec5a317c24
2019-01-27 18:16:51 -06:00
Matthew Thode
1bde2591ae
support cracklib in pam for Gentoo's musl profile
Needed for any musl build, is default in other profiles

Change-Id: Ib7cae9124f5846d33c05f26befd8f13646a08610
2019-01-15 09:47:46 -06:00
Matthew Thode
cf786bb175
change to python36 for gentoo
fixes build issues since dev-util/glib-utils was updated

Change-Id: Ie8b5c425f846619ab4fc07f5bd1902dc83172a59
2019-01-08 04:59:08 -06:00
weiyj
64a8fc7c58 update spelling errors
Change-Id: Ic206b8247acce1975409329faa29deccd4f886de
2019-01-08 14:31:06 +08:00
Matthew Thode
3ad8235180 simplify overlay logic for Gentoo
openssl/cryptography versions are updated/stable
musl profiles need newer versions of open-iscsi masked as upstream
doesn't want to work with multiple libcs

Change-Id: If5baf339516390ae332015928557c6bb734486c2
2018-10-18 00:21:31 +00:00
Matthew Thode
1e257f59cc simplify python3.6 selection on gentoo
python 3.6 is now stable in gentoo

Change-Id: Ide96c039b03817c216ab58930ff5c897556e6c8a
2018-10-18 00:21:16 +00:00
Matthew Thode
89f95a5351
enable caching for gentoo builds
To facilitate this I've created two new environment variables to set
the environment and default options for package actions.

eval is needed for the export as it preserves quotes.

Change-Id: Ib03651ee8dacd48cd1c135afd57cd31101356056
Signed-off-by: Matthew Thode <mthode@mthode.org>
2018-10-15 23:17:37 -05:00
Zuul
b29aee3383 Merge "Install sudo on Gentoo images by deault" 2018-09-11 00:09:33 +00:00
Matthew Thode
b9f1c7a22f
better handle existing keywords files/directories
The existing directories are needed for stage building (a part of the
Gentoo build process).  Normally these directories are empty, but there
are times where overrides need to be defined.  This commit handles
existing overrides for keywords.  For historical reasons the overrides
were able to be put in different files and directories, this
centralizes them.

This also updates the version of openssl/cryptography that works with
or without bindist.

Change-Id: I62c934ed305a711a4a9a3ef01fa55ad142aebb78
2018-06-25 09:20:52 -05:00
Matthew Thode
f7818e4a57
Install sudo on Gentoo images by deault
Change-Id: I30640017e3c652a467be421169029045a8b675d2
Close-Bug: #1766484
2018-04-24 11:44:39 -05:00
Matthew Thode
6a6d78e63c
Change the GENTOO_PORTAGE_CLEANUP variable default
Default the GENTOO_PORTAGE_CLEANUP to True.  By default we should not
ship package info, this bloats the image and is usually outdated by the
time it'd be consumed.

Change-Id: I14c2530d91807cbc6a3806e01c7e4f6f472b190d
2018-03-26 23:26:49 -05:00
Matthew Thode
cfa7935e43
enable systemd profile for Gentoo
Change-Id: Id3ac1d97b280f10f9938a60c4871d08f59b85002
2018-03-22 15:12:59 -05:00
Matthew Thode
5e2f3646ad
remove portage git directory
this shrinks thinks by ~50%, from 722M to 352M

Change-Id: I1267cc05700ee28c45a331de7f571b9ee075c6b5
2018-03-16 18:29:40 -05:00
Matthew Thode
6b3c22dd75
update Gentoo Hardened profiles (now stable)
Change-Id: I6d89f27bfd62fc0e86fec1a0bc6cb37f9ec6c82f
Signed-off-by: Matthew Thode <mthode@mthode.org>
2018-02-21 11:06:16 -06:00
Matthew Thode
bfee8bf094
update gentoo vars for new profile and python
Change-Id: I810b09f98c479e6ebdbf8de6fae31166a3e87667
2018-02-13 17:02:11 -06:00
Matthew Thode
7223f2ce20 Add the groundwork for musl profile support
At the moment all musl needs in addition to an official stage4 file is a
few keywords and use flag changes.

Change-Id: Ibf4a6d616aca1aef876967e2aa34170c96ac9ef8
2017-11-30 18:17:21 +11:00
Matthew Thode
c4f83f2311 Enable support for Gentoo overlays
This is intended to eventually support building musl-libc based images,
which need the musl overlay.

Change-Id: I8f5429ffa64e74c860772d9a00ff0b7eebb7721a
2017-11-30 05:14:00 +00:00
Matthew Thode
c886c4cbe1 Make preinstall.d more deterministic
Reorders the preinstall, making the order more explicit.  Also dedupes
some folder setup.

Change-Id: I423dcba169558ff6037a3382b997675722e77405
2017-11-30 13:48:18 +11:00
Matthew Thode
ade82904a1
Make python changes more reliable
We oneshot emerge without calculating dependencies a few things to solve
for possible dependency loops.

Python 3.5 also became stable, so don't need to do special things for
it.

Matched the uninstall with the install lines (no need for a full if
statement).

Change-Id: I7c5e546612ac47d659e73a46a52e34d39ca81949
2017-11-20 23:19:46 -06:00
Matthew Thode
e29f031bec
Update Gentoo element for element changes
There have been a few changes over the past few months, here we make the
following changes.

* change from backtrack=99 to complete-graph as a more correct flag
* make python version selection more in line with what gentoo supports
* set up python before stuff gets pip installed
* ensure we have the proper pip so we can install pip packages as root
* ensure we have the proper use flags for the disk formatting changes
* set DIB_RELEASE like other distros
* fix openssh-server element for gentoo

Change-Id: I17202de3016616ce34c8cbead7d0fb047a64e96b
2017-10-08 12:02:46 -05:00
Matthew Thode
ce7ea9d34c
allow uninstalls to fail on gentoo
The cleanup of packages should be more opertunistic, if it's not there
then fail quietly.

Change-Id: I207a1162abc9ca5e9636b8de192f21424db0f569
2017-05-29 23:46:42 -05:00
Matthew Thode
6f51fbe355
Make Gentoo package updates work more often
Some package updates are more complex and require things like --backtrack=99 to
be passed to emerge.  We also try harder to ensure the system is in a consistent
state as a last step.

Change-Id: Ia5d3514e8b2a6cb2d656ade997cebb798d9c0a47
2017-04-22 19:34:11 -05:00
Ian Wienand
6802cf7100 Run dib-run-parts out of /tmp
The dib-run-parts element was copying our internal version of
dib-run-parts into /usr/local/bin to be used running scripts inside
the target chroot.  However, it never cleaned up after itself.  This
means all images were left with an unmanaged local install of
dib-run-parts.

This copies dib-run-parts into the hooks directory of the chroot and
runs it from there.  It is cleaned up automatically on the exit path.

The dib-run-parts element is no longer required and it has been
removed from all dependencies.  It is left with a deprecation notice
in the README.  For compatability we convert it to simply install
dib-utils.

Codesearch shows no users depending on this unintentional implicit
install.  Note os-refresh-config depends on dib-utils and thus will
have an explicitly installed version.

Partial-Bug: #1673144
Change-Id: Ia2e96c00a4246c04beb96c17f83b8aefb69219ca
2017-04-05 13:11:22 +10:00
Ian Wienand
f1776d72aa Merge branch 'master' into merge-branch
Change-Id: Ib3c4b0030c94459a8c2eca836a16e42edf1accd4
2017-03-02 13:49:29 +11:00
Ian Wienand
3f8800832a Release 1.27.0
-----BEGIN PGP SIGNATURE-----
 
 iQEcBAABAgAGBQJYk8+9AAoJEBty/58O8cX8LdIH+wU/VrEVs0XYohiL6DUgabzs
 112U3UUihH5xMc/ca9Tarx+XwEvfMZkwYN2Qr0JoRJjmSt2AL6AezUhGSV+98vaY
 iQEccaFDFYlyDHm4V2r7N1xwS0B3mx87FPqVQQSUKlc3JsQxCy4o9RtD9aM8Gvqy
 +gAxMxL3p3O131K0Rvb0U5lC1FLgft9SuljCV8i5nU4/HdoryD6hedz2/ss8a9KG
 KKEdBKvPBKn73+nb8peQD/VXpej9C31r87q5VEjUsZkJ7gduY/qYLlGGgoBQqAXN
 WQ/ef1RkQKW5ba2jsjnk7fdOrA0+wYENxorR2WecuZbe2ieXw6fP3lYiD6VeWsM=
 =IUuh
 -----END PGP SIGNATURE-----

Merge tag '1.27.0' into merge-branch

Release 1.27.0

Change-Id: I9f6948636cae6d375d1d8315976504021f5a3bbb
2017-02-03 11:49:45 +11:00
Ian Wienand
bfca36c772 Release 1.25.2
-----BEGIN PGP SIGNATURE-----
 
 iQEcBAABAgAGBQJYV1yqAAoJEBty/58O8cX8hLwIAKP66w6MdPN8PDgUOteui/Sx
 N0UFKJ9yR4GQOAP0NffPLjch5/g0iJLs3eFKOhtGC1LjbDjpVgjX8vW18ib8wBZK
 GemOZPF3uxg8FROrZF1vpoDy/cHgL1YV10hCnwdjN/r9rb8zOuSabqjW+Dennj2n
 fZ0SJfa8Owfudn3YxGuOymVb/wMtEloDmVGBEI1Y+h7osELCCDi3OXmwsA8qMsdl
 cTwbeugBs4PlOVbZUK/JKGuwIHKgPnDYzYu5KpXw77/MdjGT0fo5Tlq5AOBDI2sC
 9JOFEBDli4Ro05VwvI58ADMpvvOax+9EvOhLbB1dRPdZl21Iyb6gOdy2PUbFO0c=
 =aKxq
 -----END PGP SIGNATURE-----

Merge tag '1.25.2' into merge-branch

Release 1.25.2

Change-Id: I698bcf2e82117bd81649cd065a7af5cac85990c7
2017-02-02 11:20:00 +11:00
Ian Wienand
97c01e48ed Move elements & lib relative to diskimage_builder package
Currently we have all our elements and library files in a top-level
directory and install them into
<root>/share/diskimage-builder/[elements|lib] (where root is either /
or the root of a virtualenv).

The problem with this is that editable/development installs (pip -e)
do *not* install data_files.  Thus we have no canonical location to
look for elements -- leading to the various odd things we do such as a
whole bunch of guessing at the top of disk-image-create and having a
special test-loader in tests/test_elements.py so we can run python
unit tests on those elements that have it.

data_files is really the wrong thing to use for what are essentially
assets of the program.  data_files install works well for things like
config-files, init.d files or dropping documentation files.

By moving the elements under the diskimage_builder package, we always
know where they are relative to where we import from.  In fact,
pkg_resources has an api for this which we wrap in the new
diskimage_builder/paths.py helper [1].

We use this helper to find the correct path in the couple of places we
need to find the base-elements dir, and for the paths to import the
library shell functions.

Elements such as svc-map and pkg-map include python unit-tests, which
we do not need tests/test_elements.py to special-case load any more.
They just get found automatically by the normal subunit loader.

I have a follow-on change (I69ca3d26fede0506a6353c077c69f735c8d84d28)
to move disk-image-create to a regular python entry-point.

Unfortunately, this has to move to work with setuptools.  You'd think
a symlink under diskimage_builder/[elements|lib] would work, but it
doesn't.

[1] this API handles stuff like getting files out of .zip archive
modules, which we don't do.  Essentially for us it's returning
__file__.

Change-Id: I5e3e3c97f385b1a4ff2031a161a55b231895df5b
2016-11-01 17:27:41 -07:00