Commit graph

182 commits

Author SHA1 Message Date
Steve Baker
833c5b8ceb Support LVM thin provisioning
This change extends the block device lvs attributes to allow creating
a volume which represents a thin pool, and to create volumes which are
allocated from this pool.

Change-Id: Ic58f55c36236cc8c6279fbcb708e27dc2982f2d5
2022-08-24 10:34:42 +12:00
wangxiyuan
934a65bc34 Upgrade openEuler to 22.03 LTS
openEuler 20.03-LTS-SP2 was out of date in May 2022. 22.03 LTS
is the newest LTS version. It was release in March 2022 and
will be maintained for 2 years. This patch upgrades the LTS
version. It'll be used in Devstack, Kolla-ansible and so on
in CI jobs.

This patch also enables the YUM mirror to speed up the package
download.

Change-Id: Iba38570d96374226b924db3aca305f7571643823
2022-08-01 19:22:35 +08:00
Ian Wienand
f44644f20f Update platform support to describe stable testing
The reality is that "stable" is what is tested.  This tries to give
enough info that users can ascertain what tests are running at any
given time and hence what elements are known to be working.

Additional, clarify the Fedora position in the README as now described
by above.

Closes: #1653561
Change-Id: Ifb91b9089790897861bd7e671c3dba59adac239d
2022-02-16 10:40:09 +11:00
Ian Wienand
79ea63f525 Futher bootloader cleanups
GRUB_OPTS has never been documented as externally available, and is
not used.  Assume it's value to simplify the code.

Move the grub version check separately, as we only support grub2

Remove references to buliding i386 images.  I don't image it works in
any way.

Remove ci.md, which is no longer relevant.

Refactor the test for "building BIOS image on EFI system" consiberably
after these changes.

Change-Id: Ia99687815667c3cf5e82cf21d841d3b1008b8fa9
2022-02-10 15:08:56 +11:00
Eduardo Santos
0f430664a2 Fix openSUSE images and bump them to 15.3
SUSE dropped OpenStack Cloud in 2019 [1], and as a result, some
OpenStack-related repositories were removed from openSUSE Download and
root filesystem images stopped being provided. This change deprecates
Leap releases before 15.3 and employs the extract-image script. It also
moves the extract-image script to the sysprep element, since now it's
also used by openSUSE-related elements.

Additionally, revert the "Remove opensuse related funtests" change [2]
so that the opensuse element is tested again and set the default Leap
release to 15.3.

[1] https://www.zdnet.com/article/suse-drops-openstacks/
[2] https://review.opendev.org/c/openstack/diskimage-builder/+/824002

Change-Id: I73d6323aa65cee69a55e54bc53ed682f096dfc89
2022-01-28 02:18:47 -03:00
Xinliang Liu
a6ee4d0c21 Introduce openEuler distro
Add openeuler-minimal element and add CI functional tests for both
x86_64 and arm64.

OpenEuler is an open source community driven YUM/DNF distro like
Fedora. It references Fedora and CentOS a lot for the rpm packages
building. So somewhat it can be treated as a redhat family distro
and reuse the YUM/DNF related elements to help build openEuler images.

For more info about openEuler, see: https://openeuler.org/en

Depends-On: https://review.opendev.org/c/zuul/zuul-jobs/+/803413
Change-Id: I3e06e49b524364c3a4edeba8bce7a8c06b9c7b76
2021-08-04 03:06:55 +00:00
Takashi Kajinami
4107dc3bc4 Update IRC networks
The following two channels were migrated to OFTC.
 #tripleo
 #openstack-dib

Also, the following channel was migrated to Libera Chat[1].
 #opensuse-cloud

[1] https://en.opensuse.org/openSUSE:IRC_list

Change-Id: Ia4c729a8d284bbfcbdb3b8621ae29d9be57886f5
2021-07-22 19:17:51 +09:00
Dmitry Tantsur
fd1848282f Remove the deprecated ironic-agent element
Despite having several issues (like missing firmware), it is still
used by people. It seems that the only way to stop that is to remove it.

Change-Id: I4baed8e8ab663c624dcc8d06ff0293d57b082abb
2021-01-21 14:06:08 +01:00
Christian Berendt
0255291609 docs: fix wrong key in the lvm sample configuration
Change-Id: Ia1dcbaf4f510a8ed275f1bf9987f3ff5a0ca6d23
2020-12-16 20:19:11 +01:00
Ian Wienand
7b6819213e Fedora 32 support
Update for Fedora 32 support.

Change-Id: I51c5645856a76e2877c013d72e9849a758ba12ff
2020-08-17 19:40:02 +10:00
melissaml
abba0b5574 Switch to newer openstackdocstheme and reno versions
Switch to openstackdocstheme 2.2.1 and reno 3.1.0 versions. Using
these versions will allow especially:
* Linking from HTML to PDF document
* Allow parallel building of documents
* Fix some rendering problems

Update Sphinx version as well.

Disable openstackdocs_auto_name to use 'project' variable as name.

Change pygments_style to 'native' since old theme version always used
'native' and the theme now respects the setting and using 'sphinx' can
lead to some strange rendering.

openstackdocstheme renames some variables, so follow the renames
before the next release removes them. A couple of variables are also
not needed anymore, remove them.

See also
http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014971.html

Change-Id: If3e8d8ac29a728aa41562b31976ebe9bfa5df66f
2020-06-09 10:23:37 +02:00
Andreas Jaeger
e7885499dd Cleanup py27 and docs support
This repo is now testing only with Python 3, so let's make
a few cleanups:
- Remove python 2.7 stanza from setup.py
- Remove obsolete sections from setup.cfg
- Update classifiers
- Update requirements, no need for python_version anymore
- Cleanup doc/source/conf.py to remove now obsolete content,
  use sphinxcontrib.apidoc to build module index again (this
  seems to have been broken when switching to sphinx-build).
- Remove install_command from tox.ini, the default is fine
- Use TOX_CONSTRAINTS_FILE instead of obsolete UPPER_CONSTRAINTS_FILE.
- Update tox.ini for python3 only support.

Change-Id: Id8738ecfb0f578d2a7953c63ffe10779f835bcaf
2020-06-05 12:04:30 +02:00
Ian Wienand
262281e5f7 Remove Trusty testing
Opendev has dropped trusty from the mirrors.  With no testing the
distribution is effectively unsupported, add a release note.  Update a
few other random doc bits (that are not really that up to date
anyway).

Change-Id: I5bd0d0a94477cf8d84cef72f5d4b2e9e15ab9fd2
2020-04-23 10:00:13 +10:00
Lee Yarwood
ee9ad32b6f Add Fedora 31 support and test jobs
Change-Id: Iad0261ac8db001ffa4d3a4dd6df05fe923402a69
2020-03-30 08:47:00 +11:00
Ian Wienand
6758849b66 vhd-util : note on Xen/RAX images
Add a note covering what we've discovered about creating RAX/Xenserver
compatible images over the last few days.

Change-Id: Iffd9dab1ca54f27390ec7850093e828ca7576e98
2019-10-31 08:36:07 +11:00
pengyuesheng
a871726834 Blacklist sphinx 2.1.0 (autodoc bug)
See https://github.com/sphinx-doc/sphinx/issues/6440 for upstream details

Change-Id: Ic5149826f443a29801237a250804660a3bb337b8
2019-10-09 16:15:11 +08:00
jacky06
9e3400fa9c Move doc related modules to doc/requirements.txt
Change-Id: I5106dc365856932f00a2fbdf4e7a4d58e48cca8a
2019-09-09 14:13:36 +08:00
Zuul
48edd472e7 Merge "Allow configurable gzip binary name" 2019-09-06 09:47:06 +00:00
Logan V
d9e85efd7c Allow configurable gzip binary name
Add a new environment variable $DIB_GZIP_BIN allowing builders to
specify a different gzip (such as pigz) to be used when compressing
tgz images.

Change-Id: Ifb617568140a149e2fda241e07ff8a59429e6697
2019-08-30 17:46:20 +02:00
Ian Wienand
a5a6482ac1 Fedora 30 functional and boot tests
Update testing for Fedora 30

Change-Id: If60eb0b87e45efc0e71db2ddcd814223539f07b7
2019-08-28 11:21:46 +10:00
inspurericzhang
d1661e922f Mailing lists change openstack-dev to openstack-discuss
Change-Id: I9774488a307d384565331441e5e31cfc96a251a7
2019-07-02 07:23:06 +00:00
Zuul
fe618002ff Merge "Update test coverage for openSUSE/-minimal to 15.1" 2019-06-18 06:21:21 +00:00
Dirk Mueller
a81cf9e231 Update test coverage for openSUSE/-minimal to 15.1
Use openSUSE 15.1 as default, which is the latest released stable
openSUSE release.

Remove leftovers for unmaintained openSUSE 42.2 images.

Depends-On: https://review.opendev.org/#/c/660126/
Change-Id: I0b204b7b3d7ae74b6749320b3bfe1ca89d154ebb
2019-06-13 09:20:40 +02:00
Zuul
c47a9d5001 Merge "Replace git.openstack.org URLs with opendev.org URLs" 2019-05-24 08:03:47 +00:00
melissaml
a6322c6ed0 Replace git.openstack.org URLs with opendev.org URLs
Change-Id: I03e9162d5a59a2aa1631a9ecf6f6833bb7ac6050
2019-05-16 14:45:52 +08:00
Zuul
3d3ba26edd Merge "Use megabyte granularity for image extra space" 2019-05-15 06:51:10 +00:00
Logan V
87a18f51e3 Use megabyte granularity for image extra space
I want to use the new --image-extra-size flag[1] but my use-case
calls for megabyte granularity of this value. Rather than adding
60% to an 800MB image, maybe I only want to add 100 or 200MB, etc.

[1] https://review.opendev.org/#/c/655127/

Change-Id: I8fb9685d60ebb1260d5efcf03c5c23c561c24384
2019-05-15 13:38:25 +10:00
Dirk Mueller
c7ac6ee0cb Update test coverage for openSUSE/-minimal to 15.0
Use openSUSE 15.0 as default, which is the latest released stable
openSUSE release. Switch to https for accessing download.o.org
as encrypted transfers should be used by default.

Remove leftovers for definitely unmaintained openSUSE 13.x images
and split into old/new leap style versioning scheme for clarity.

Change-Id: Iab129eeee2b1a2563f0f0d2cb17bbad57c068e38
2019-05-08 14:59:51 +00:00
Zuul
fa34eb7fe4 Merge "Support defining the free space in the image" 2019-05-07 10:14:01 +00:00
Zuul
8bf37a064e Merge "Allow specification of filesystem journal size" 2019-05-07 10:14:00 +00:00
Zuul
1575a0cf2f Merge "Document the various global filesystem options" 2019-05-07 09:03:44 +00:00
Zuul
5f9024affa Merge "Replace git.openstack.org URLs with opendev.org URLs" 2019-04-29 12:18:34 +00:00
Tobias Henkel
778d007150 Support defining the free space in the image
Currently diskimage-builder supports two ways to specify the image
size. One is defining a fixed image size using DIB_IMAGE_SIZE, the
other one is auto-detection while adding a security margin of 60% as
free space. This means when building larger images (e.g. >100GB) with
unknown size upfront we end up with much wasted space, IO and network
traffic when uploading the images to several cloud providers. This can
be optimized by adding a third way by defining DIB_IMAGE_EXTRA_SIZE to
specify the free space in GB. This makes it possible to easily build
images of varying sizes while still minimizing the overhead by keeping
the free space constant to e.g. 1GB.

Change-Id: I114c739d11d0cfe3b8d8abc6df5ff989edfb67f2
2019-04-29 20:18:43 +10:00
caoyuan
0329a6de5e Replace git.openstack.org URLs with opendev.org URLs
Change-Id: Iac5a9da62db84365a769ea07146281866215a9c5
2019-04-29 20:15:25 +10:00
Logan V
11142f75b4 Allow specification of filesystem journal size
In many cases, the statically sized 64MB journal is far below the
e2fstools default calculation[0] which calls for a 64MB journal only
on filesystems smaller than 16GB. On bare metal in particular, the
correct default journal size will often be in the 512MB-1GB range.

Since we cannot know what the target system is, this should be a
tunable parameter that the user can set depending on the intended
image usage.

Add a DIB_JOURNAL_SIZE envvar and --mkfs-journal-size parameter
to the image creation so users can override the default journal
size.

[0] https://git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git/tree/lib/ext2fs/mkjournal.c#n333

Change-Id: I65fa13a088eecdfe61636678578577ea2cfb3c0c
2019-04-29 17:00:30 +10:00
Ian Wienand
d18e73147e Document the various global filesystem options
These options have always been a bit of a mess since v2 when we added
all the block-device flexibility.  Add some explicit documentation to
try and help explain the relationship between these options and the
block-device config.

Change-Id: I49affcbef868d644f673b833bef8310cf25cfd0f
2019-04-29 17:00:04 +10:00
Dirk Mueller
08b9f7ebb8 Update to https:// version of *openstack.org urls
The non-encrypted urls are just redirecting to https, so we
can save one roundtrip by linking the encrypted versions directly.

Change-Id: I88fe8b8c8ccfb5471f59f7898a69bf62cb6cfcaf
2019-04-28 03:03:52 +02:00
Ian Wienand
5085f23fdf Update to Fedora 29
Update builds to Fedora 29.  Remove the openstack gate CI mirror
workaround for pre-28 versions as they're not building in the gate any
more.

Change-Id: Ia6a8ae8d66d69f6add39e571043328e7274ba26c
2018-11-16 09:05:08 +00:00
Ian Wienand
c48a40d479 Move several packages to bindep.txt
Move several parts of the "install_test_deps.sh" script into the more
standard bindep.txt.  This list is intentionally restricted as a first
step.

Developer documentation is updated to use bindep and clarified
slightly.

Change-Id: I7520902dc324d920a0c7c44a2d35fe49f9b05614
2018-10-29 09:52:13 +11:00
Tobias Henkel
eff5b2312b
Add a post-root.d phase
While trying to get docker image pre-caching to work we couldn't get a
docker daeomon to run within the chrooted environment. However we got
docker running with the help of bwrap outside of the chrooted
environment. The only option so far for this is the block-device.d
phase. But this has the problem that it runs after the image size has
been calculated. This leads to broken builds if the docker images
being pulled are big.

This can be solved by adding a post-root.d phase that runs outside the
chroot but before the image size calculation.

Change-Id: I36c2a81e2d9f5069f18ce5b0d52c5f1c7212c3ae
2018-10-19 10:33:56 +02:00
Zuul
46ec4992ff Merge "Add support for Fedora 28, remove EOL Fedora 26" 2018-10-18 23:44:51 +00:00
Paul Belanger
7173658df2 Add support for Fedora 28, remove EOL Fedora 26
This updates diskimage-builder to support current Fedora releases (27
and 28) and removes support for Fedora 26 which is EOL as of June
2018.

Change-Id: I602b22ed4d5397b39dc1eef67964f6fbdcd93060
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-10-18 19:43:12 +11:00
Ian Wienand
fadf99af05 Add a pre-finalise.d phase
In exploring Gentoo caching, it was realised that we have no way to
bind mount the cache into the finalised image for the finalise.d
phases.

By adding a pre-finalise.d phase that runs outside the chroot, we can
mount outside things into the hierarchy at $TMP_BUILD_DIR/mnt which
are then seen by the in-chroot finalise.d phase.

This is similar to the pre-install phase

Change-Id: I9d782994843383ddf90f62c40498af9925fd9558
2018-10-15 12:45:23 +11:00
Ian Wienand
ee156e8232 Minor documentation updates
Some minor things after looking at these parts.

The dib-run-parts element doesn't do any of the copying any more, so
these comments are wrong.

The reason for the multiple mounts in the bind mount was non-obvious
to modern eyes (as util-linux has handled this for some time).

Formatting fix for the rst

Change-Id: Idb4c9ff32c49aced2c68a5c905bf7a8b2832a5a2
2018-10-12 11:51:56 +11:00
kaiokassiano
87c3d28206 Fix typo in installation.rst
Trivial fix.

Change-Id: I7d4fa9e4007954d5d847e659a17ac602f337e78f
2018-07-31 18:19:10 -03:00
Arx Cruz
33756305fc Update developing_elements
Make clear to where in the chroot the contents of
`$TMP_HOOKS_PATH` will be available.

Change-Id: I4b9d20f7ec1c317eb61da44bfd05242dd45263c4
Co-Authored-By: Elyezer Rezende <erezende@redhat.com>
2018-07-06 13:45:05 -03:00
Ian Wienand
e443700b5c Formalise saving of /etc/resolv.conf
systemd-resolved has a new behaviour in bionic, in that if there is no
/etc/resolv.conf file when it installs, it assumes it is a fresh
system and makes /etc/resolf.conf a symlink into its compatability
files.

dib ends up saving & restoring whatever /etc/resolv.conf we have after
the inital chroot creation, which may not be what we want -- in the
above case it restores the system-resolved symlink.  For
openstack-infra, we use unbound and want simply "127.0.0.1" in a
/etc/resolv.conf file [1].

Formalise the ability to save specific contents into the final image.
Add documentation, and a note in the code that it's an external
interface.

I would have preferred to namespace the .ORIG file with DIB_ or
similar, but this unofficial interface has already escaped into the
wild.  Leave it as is for simplicity.

[1] Note that systemd-resolved will obey /etc/resolv.conf as you would
expect, if file exists.

Change-Id: Ie0e97d8072e2b21a54b053fa6fb07b62960c686d
2018-04-04 15:17:45 +10:00
Zuul
96af247400 Merge "Add block-device defaults" 2018-03-16 08:31:10 +00:00
Zuul
c60a20b59d Merge "GPT partitioning support" 2018-03-16 08:14:27 +00:00
Thomas Goirand
d2d6c3a6a7 Make the build reproducible
Whilst working on the Reproducible Builds effort [0], the team noticed
that diskimage-builder could not be built reproducibly as it iterates
over XML elements in a non-deterministic order when generating the
documentation. This patch sorts the elements, and is a forward of the
patch sent to the Debian BTS [1].

 [0] https://reproducible-builds.org/
 [1] https://bugs.debian.org/892020

Change-Id: I40011514a135bc50383b072ab83c7f91689bad5a
2018-03-04 23:24:10 +01:00