Commit graph

2773 commits

Author SHA1 Message Date
Pino Toscano
aeab45ad7f dib-lint: validate json/yaml files
Look for files .yaml and pkg-map configurations, and try to load them
either as json or yaml. This way, invalid ones can be detected before
they are committed unnoticed.

Also, exclude .yaml files from being searched while checking bash and
python scripts.

Change-Id: I2478837cfe66929ae1b0d7dd96e049773a35e11c
2015-09-15 12:48:23 +02:00
Jenkins
fe23b262fd Merge "Add documentation of output formats for users" 2015-09-11 13:06:42 +00:00
Dmitry Tantsur
b544e91494 Make sure dnf won't autoremove packages that we explicitly installed
As dnf 1.1.1 is pretty recent, make sure we update it as early as possible.

Change-Id: I30039090cfe652b864d847713ab2986c5659023d
2015-09-10 10:43:02 +02:00
Ian Wienand
ab5ed610e4 Handle install with pip -e
As described in the comments, inspect the installation to see if we
have been installed with "pip -e" and, if so, make sure we reference
the scripts from the source location rather than the
system-installations.

Update the documentation with a terse but helpful quick-start to show
an easy way to start developing a change using this.

Closes-Bug: #1491035
Change-Id: I0460061b834a2b854175f8c9be2be8d38c540c9d
2015-09-10 16:55:39 +10:00
Antoine Musso
f22a5a8b91 doc: migrate from README.rst to Sphinx
The README.rst has a lot of information that has been duplicated in the
Sphinx maintained documentation (3600330).

Remove dupes from README.rst
Point to http://docs.openstack.org/developer/diskimage-builder/

Change summary:

=====================+======================================
README.md            | Sphinx document
section              |
=====================+======================================
Installation         | installation.rst
---------------------+--------------------------------------
Invocation           | invocation.rst
---------------------+--------------------------------------
Requirements         | installation.rst Speedups
---------------------+--------------------------------------
Caches/offline       | caches.rst + changes from 849e9cb2
                     | fix some markup
---------------------+--------------------------------------
Install Types        | install_types.rst
---------------------+--------------------------------------
Writing an element   | developing_elements.rst + fe7823a2
                     | `Testing element` from b9b6640f
                     | `3rd party elements' from f1e7bf3a
---------------------+--------------------------------------
Existing elements    | elements.rst
---------------------+--------------------------------------
What tools are there | components.rst
---------------------+--------------------------------------
Design               | design.rst
---------------------+--------------------------------------

Change-Id: I578daa8e3a8d876b3ee3c9a748d7c8aa2bf7a0b7
2015-09-10 16:55:39 +10:00
Augustina Ragwitz
63b63e9533 Update fedora elements README from free text to table formatting
Change-Id: Iafb7f1b2f225dbe96e2e24bb7f405bcec74f8621
2015-09-09 15:35:37 -07:00
Augustina Ragwitz
099bdc3d34 Update redhat-common elements README from free text to table formatting
Change-Id: Ie87cf9937a54110719814046c4f46829e307ff3d
2015-09-09 15:08:29 -07:00
Pino Toscano
430ea923e0 svc-map: do not specify user/group for install
Since install is run as root already (using sudo), the resulting file
will be already root:root.

Change-Id: I6023fd277ea77fbfa4f7ad42b04d58c8cd6bd4fe
2015-09-07 17:04:03 +02:00
THOMAS J. COCOZZELLO
d8a8f9b732 Activate pep8 check that _ is imported
Remove the specification in tox.ini that _ is a builtin so that
it will no longer assume that _ does not need to be imported.
This helps ensure that the _ from i18n is used.

Activating this check did not flag any violations.

Change-Id: Iac73937b60a6f8e7520123fbf59e748d296d4c7d
2015-09-03 15:41:31 -05:00
Jenkins
bf4a60a9e3 Merge "Update dpkg elements README from free text to table formatting" 2015-09-03 18:23:28 +00:00
Jenkins
b2fd0f3ad7 Merge "[ironic-agent] Use svc-map for enabling agent" 2015-09-03 15:05:24 +00:00
Jenkins
08da5f4bac Merge "ironic-agent: ensure dmidecode and ipmitool are installed" 2015-09-03 13:47:45 +00:00
Evgeny Antyshev
8349f0350e Remove grub2 in redhat-common/pre-install.d/15-remove-grub
In Id1e430e7d050a0b99ac449e2ea435e06cda1c4e6 I made the mistake of not
actually removing grub in 15-remove-grub.

This restores the removal phase and adds a bunch of comments.  It
seems the centos7 and centos (6) images have grub2 installed, but F22
does not; hence the check.

For anyone interested in the history; it seems the whole idea of
removing grub and re-installing it in the finalise stage is to do with
Ubuntu grub scripts failing in the chroot.  It is not clear this does,
or has ever, affected rpm based systems; but that's how it is, so
leave well enough alone.

The whole reasoning behind the rpm download & re-install is actually
explained in If095adc4abb52a19a3aa0b1caebfb3e4d8f605ef, but over time
the comments got lost as code moved around.  I've restored in here
some detailed explaination of why we don't just re-install the package
"normally".  I've also added a note to the pre-install of various
things that are related to this step.  Again I think there are some
questions around this that we can investigate in another change.

Change-Id: I1acd19da8567ab93b5003caf67673cc70efea5fa
2015-09-02 15:15:13 +10:00
Ian Wienand
b5ae6bf39c Remove old entries from MANIFEST.in
I noticed these don't exist in the tree any more; you do get warnings
out of setup.py about it

Change-Id: I0b133ee4492167600dfb631bf3738ad69fa0e879
2015-09-02 15:15:13 +10:00
Dmitry Tantsur
5f68750b4b ironic-agent: ensure dmidecode and ipmitool are installed
Currently they are used for inspection, but may be also used for
other purposes, as they're accessed from IPA generic hardware layer.

Change-Id: I32c6a711d466131b9445023812a2a260ed2e01f3
2015-09-01 17:39:51 +02:00
John Trowbridge
109e02b1ce [ironic-agent] Use svc-map for enabling agent
Switch to using svc-map element for systemd based agent.

This allows both .deb and .rpm installs to share the
element for systemd based installs. There are not any
plans to package a .rpm package for upstart or sysv, so
these are left as is.

Change-Id: Idca7ad97355cae785162989774a7e6dea6fdc5b5
Closes-Bug: #1490584
2015-09-01 07:12:52 -04:00
Augustina Ragwitz
2cffdcda30 Update dpkg elements README from free text to table formatting
Change-Id: Iaf890c85b72f01999d3ae31398bdb65a356218e7
2015-08-31 13:43:04 -07:00
Jenkins
12d71c6447 Merge "Fix ironic-image pkg-map" 2015-08-28 17:17:00 +00:00
Jenkins
ad9fc6d4e7 Merge "Fix a typo in README.rst" 2015-08-28 02:44:28 +00:00
Jenkins
37ca006996 Merge "Updated from global requirements" 2015-08-27 21:14:53 +00:00
Julia Kreger
51816f7dba Fix ironic-image pkg-map
Fixing the ironic-agent pkg-map by adding missing commas. Validated
updated form passes json linting. Also includes a listing for curl.

Change-Id: I1983f7a581be3a5aaa771b19c6609cf12b61a7bb
Closes-Bug: #1488969
2015-08-27 15:40:19 +00:00
Clark Boylan
ea79655a08 Run growroot after all filesystems are mounted
Appears that growroot was running before /dev is mounted so the script
is unable to introspect the filesystem partition info. Run this after
all local filesystems are mounted to fix this issue.

Change-Id: Ia7c41ba6ef79788fdbf198998622eeaa20dd4245
2015-08-25 17:04:38 -07:00
OpenStack Proposal Bot
21c2559c70 Updated from global requirements
Change-Id: I45ed19e4333c37c8ea4943c344320e8a0a73c507
2015-08-25 07:06:43 +00:00
Jenkins
9618974f92 Merge "create growroot element" 2015-08-24 18:35:57 +00:00
Jenkins
0035b1440e Merge "Reduce the size of the ironic-agent ramdisk" 2015-08-24 16:52:14 +00:00
Junya Akahira
4211822978 Fix a typo in README.rst
Change-Id: If38d147a9521de0e599f10a3e336000373ad1efa
2015-08-24 17:33:26 +09:00
Jenkins
f372425970 Merge "Add ability to build ironic-python-agent ramdisk from packages" 2015-08-22 15:04:13 +00:00
Jenkins
243fc75d06 Merge "Add explicit f21 test" 2015-08-21 10:22:18 +00:00
Jenkins
cb3799f0e5 Merge "Add test for centos 6" 2015-08-21 10:22:02 +00:00
Jenkins
3b770adf4c Merge "Install-static depends on rsync" 2015-08-21 10:21:59 +00:00
Jenkins
1356885447 Merge "Fix init-scripts element path munging and deps" 2015-08-21 10:21:25 +00:00
Gregory Haynes
16fc1f9012 create growroot element
We can resize the rootfs without the initrd based approach. Create a
growroot element which performs rootfs resizing as part of system init.

Change-Id: Ibeb846b0170d141fb72323a441d14b65b93ae0a1
2015-08-19 21:12:56 +00:00
Gregory Haynes
7f6dbe534f Install-static depends on rsync
We just dont express it.

Change-Id: Id4235d262ec9db9797ece76fce94541998fbf4fc
2015-08-17 16:21:09 +00:00
Gregory Haynes
ca594ef20f Fix init-scripts element path munging and deps
There is a bug where the init scripts element incorrectly munges the
install path making it useless. Also removing the dep on rsync since
this occurs from inside the chroot.

Change-Id: I8f2717d36d7d2ff4b195ec21e91afeaf30a1d803
2015-08-17 16:16:43 +00:00
Jenkins
86bcb6ec22 Merge "Remove docker doc from docs" 2015-08-14 07:43:19 +00:00
Jenkins
988d9f4699 Merge "Pin the Fedora mirror for testing" 2015-08-14 05:09:07 +00:00
Gregory Haynes
e354dcdc94 Pin the Fedora mirror for testing
We are getting mirror failures for Fedora, lets pin to a known stable
one.

Change-Id: I22ea750136f68d707f45ececb97bd564f13eeff3
2015-08-11 22:14:18 +00:00
Jenkins
caf402fd04 Merge "Handle modern sfdisk and correctly align image partition" 2015-08-07 04:55:08 +00:00
Jenkins
9e42c69c80 Merge "Set and export DIB_RELEASE for centos7" 2015-08-07 04:54:46 +00:00
Lucas Alvares Gomes
1181fb8543 Reduce the size of the ironic-agent ramdisk
This patch is reducing the size of the ramdisk image generated by the
ironic-agent element. It does remove extra packages (graphical stuff,
dev stuff, miscs, docs, etc...) and purges directories that are not
needed for a ramdisk (like /boot since it boots using an external
kernel)

Currently it was tested generating a Fedora 22 image and reduced the
size of the final image from 464 MB to 211MB compacted (54% decrease).

I was able to boot a VM with 1.3 GiB of ram instead of the previous 3 GiB
needed.

Change-Id: Id6333ca5d99716ccad75ea1964896acf371fa72a
2015-08-06 16:34:30 +01:00
Jenkins
c2b96259a4 Merge "Add element to disable cloud-init resizefs" 2015-08-06 02:01:44 +00:00
Jenkins
a9fd90b8ea Merge "Correct URL in ironic-agent README" 2015-08-05 05:00:27 +00:00
Jenkins
2f4fde2294 Merge "debian: properly configure interfaces" 2015-08-05 02:41:30 +00:00
Abel Lopez
570ecf16a5 Set and export DIB_RELEASE for centos7
The default value was set in the centos7 element, but not
exported, which caused issues in rpm-distro. Also changed
a test in rpm-distro to only check for DIB_RELEASE > 22
if it's fedora.

Closes-Bug: #1477172

Change-Id: Ib6f4227411c2e8f1965c3b78bc318512c59a7876
2015-08-04 08:35:20 -07:00
Julia Kreger
00d4e3faba Correct URL in ironic-agent README
URL was pointing to https://github.com, when it should be pointing
to https://git.openstack.org/cgit/

Change-Id: If86e45f3e8eb57f9dfb53b5a80db7dbac8bed93e
2015-08-04 07:58:03 -04:00
Julia Kreger
a71e8d7279 Add curl to ironic-agent package installs
The script for ironic-agent utilizes curl, however an extremely
minimal system may not have it, and as such we should list it as
a package that must be installed to support the element.

Change-Id: Id118f84e2d5e6adf0ae3d653864565368b0d76bf
2015-08-04 07:44:50 -04:00
Jenkins
eda377e5bf Merge "Adds Ubuntu and Debian to ironic-python-agent Support-list" 2015-08-04 07:27:30 +00:00
Jenkins
7725b21ff2 Merge "Fix link in installation doc" 2015-08-04 05:59:05 +00:00
Jenkins
7e5c1c812f Merge "Only warn about missing map files with debug" 2015-08-04 05:30:18 +00:00
Ian Wienand
2c32c7ae1a Handle modern sfdisk and correctly align image partition
As described in the comments, sfdisk was rewritten for util-linux 2.26
(as shipped in F22) and now interprets arguments a sectors, rather
than cylinders.

The current partitioning line is "1 - - *" (start/size/type/bootable)
which means you start getting:

---
/usr/sbin/grub2-install: warning: this msdos-style partition label has
no post-MBR gap; embedding won't be possible.

/usr/sbin/grub2-install: warning: Embedding is not possible.  GRUB can
only be installed in this setup by using blocklists.  However,
blocklists are UNRELIABLE and their use is discoura ged..

/usr/sbin/grub2-install: error: will not proceed with blocklists.
---

when building images, because the start is interpreted by the new
sfdisk as sector 1 and it crams the partition right next to the MBR.

Specifying "-" for the size is undefined in the man page; even reading
the source it's not totally clear what "-" for the size does [2].  In
any case, the alignment is wrong in sectors or cylinders; we want to
be a multiple of 4KiB for best performance.

The intent here is to create one single, Linux, bootable, partition
taking up the whole disk starting at 1MiB, so "2048 + L *" makes this
clear.

We use the -uS argument to ensure both versions treat this start-value
as a sector offset (newer sfdisk essentially ignores the argument).
As described in the comments, bugs in the older sfdisk necessitate
usage of "--force".

Although we could choose more or less, it seems most common to align
to a 1MiB boundary (i.e. starting at sector 2048).  libguestfs has
some disucssion around --alignment and where it sets it's default to
this [3].  The 2.26-era sfdisk also defaults to putting partitions
here.  1MiB should be enough for GPT schemes in the future as well.

[1] https://github.com/karelzak/util-linux/blob/master/libfdisk/src/script.c#L1050
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1249893
[3] http://libguestfs.org/virt-resize.1.html

Change-Id: I2c2966f98d1d5ad4ebb433ea148b3b26c65dc1b5
2015-08-04 15:26:13 +10:00