Commit Graph

143 Commits

Author SHA1 Message Date
Tristan Cacqueray
9d13084c41 Add squashfs output image format
The squashfs format brings a couple of advantages over the other
formats. Image is often an order of magnitude smaller and it can
be used natively, either as an initrd, either with loop mount.

Change-Id: If72940b0c4dafb2504c52dd0429a8eb3f8305751
2016-12-19 07:21:39 +00:00
Paul Belanger
da41ee6012 Add output image format tgz support
We now support tgz (tar.gz) as an output format.

Change-Id: Iadec92f2f96c3f904f28bd49f87ffc7d48ef7bd7
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2016-12-17 16:41:42 -05:00
Ian Wienand
5e2330d89c Put MKFS_OPTS after filesystem type
mkfs's arguments are

 mkfs [options] [-t type] [fs-options] device [size]

So it seems our MKFS_OPTS are really supposed to be fs-options, rather
than options to mkfs itself.

Why didn't we notice?  It's quite a trap -- mkfs.ext2 has a "-t"
option, so when we're calling

 $ mkfs -i 4096 ... -t ext4 ...

We actually just fall-back to the default from the mkfs wrapper which
is mkfs.ext2 which works!  But when you make that, say, xfs, we're not
calling the right wrapper at all.

Also update documentation

Closes-Bug: #1648287
Change-Id: I3ea5807088ab361bd9c235c07fb1553fbaf9178b
2016-12-09 07:22:38 +11:00
Ian Wienand
7d5afecfd9 Merge remote-tracking branch 'origin/master' into merge-branch
Change-Id: Ibab1bb95521292ae818bd91f7073c3749a2cc0cb
2016-11-18 13:53:56 +11:00
Ian Wienand
d65678678e Move dib-run-parts into diskimage-builder
Move dib-run-parts from dib-utils into diskimage-builder directly.

For calling outside the chroot, we provide a standard entry-point
script.  However, as noted in the warning comment, the underlying
script is still copied directly into the chroot by the dib-run-parts
element.  I believe this to be the KISS approach.

This removes the dependency on dib-utils.  We have discussed this
previously and nobody seemed to think retiring dib-utils was going to
be an issue.

This also updates the documentation to not mention dib-utils, or using
disk-image-create via $PATH setup, but rather gives instructions on
installing from pip with a virtualenv.

Change-Id: Ic1e22ba498d2c368da7d72e2e2b70ff34324feb8
2016-11-04 17:07:37 +11:00
Ian Wienand
97c01e48ed Move elements & lib relative to diskimage_builder package
Currently we have all our elements and library files in a top-level
directory and install them into
<root>/share/diskimage-builder/[elements|lib] (where root is either /
or the root of a virtualenv).

The problem with this is that editable/development installs (pip -e)
do *not* install data_files.  Thus we have no canonical location to
look for elements -- leading to the various odd things we do such as a
whole bunch of guessing at the top of disk-image-create and having a
special test-loader in tests/test_elements.py so we can run python
unit tests on those elements that have it.

data_files is really the wrong thing to use for what are essentially
assets of the program.  data_files install works well for things like
config-files, init.d files or dropping documentation files.

By moving the elements under the diskimage_builder package, we always
know where they are relative to where we import from.  In fact,
pkg_resources has an api for this which we wrap in the new
diskimage_builder/paths.py helper [1].

We use this helper to find the correct path in the couple of places we
need to find the base-elements dir, and for the paths to import the
library shell functions.

Elements such as svc-map and pkg-map include python unit-tests, which
we do not need tests/test_elements.py to special-case load any more.
They just get found automatically by the normal subunit loader.

I have a follow-on change (I69ca3d26fede0506a6353c077c69f735c8d84d28)
to move disk-image-create to a regular python entry-point.

Unfortunately, this has to move to work with setuptools.  You'd think
a symlink under diskimage_builder/[elements|lib] would work, but it
doesn't.

[1] this API handles stuff like getting files out of .zip archive
modules, which we don't do.  Essentially for us it's returning
__file__.

Change-Id: I5e3e3c97f385b1a4ff2031a161a55b231895df5b
2016-11-01 17:27:41 -07:00
Jenkins
cc1b37e731 Merge "Don't set tracing in environment files" 2016-11-01 01:42:59 +00:00
Gregory Haynes
7cc65d3cbd Merge branch 'master' into feature/v2
Change-Id: I0747a27560161ef869868967ab140b5d0be9d4da
2016-10-31 11:21:38 -07:00
Ian Wienand
9e392f56b0 Don't set tracing in environment files
Because environment files are sourced into the current environment,
they shouldn't be setting global settings like tracing else they
affect every preceeding import.  This is quite confusing when only
half your imports are traced in the logs, because it was either turned
on, or off, by a preceeding environment import.

There is a corresponding dib-run-parts change in
I29f7df1514aeb988222d1094e8269eddb485c2a0 that will greatly increase
debugability for environment files by deliberately logging what files
are sourced and consistently turning on tracing around their import.

This isn't strictly necessary (since dib-run-parts with the prior
change will just turn tracing off after import anyway) but it's a
decent cleanup for consistency.  A bare-minimum dib-lint check is
added.  Documentation is updated.

Change-Id: I10f68be0642835a04af7e5a2bc101502f61e5357
2016-10-20 13:58:00 +11:00
Jenkins
2e0f812efa Merge "Add option to be able to run_functests.sh in parallel" 2016-10-18 21:03:05 +00:00
Matthew Thode
bb29082910
Document install of diskimage-builder on Gentoo
Change-Id: I27bf3499f998dcbdeb5cfebddde306e5200cc0b5
2016-10-11 20:17:49 -05:00
Jenkins
fd6e1475a1 Merge "Rename devloper documentation to developer guide" 2016-10-07 05:01:59 +00:00
Jenkins
1daae21508 Merge "Remove copyright from docs / toc" 2016-10-07 05:01:53 +00:00
Jenkins
a30a72a3b2 Merge "Use globaltoc in docs for sidebar" 2016-10-07 05:00:52 +00:00
Jenkins
ce461668a0 Merge "Fix developer guide TOC" 2016-10-07 05:00:50 +00:00
Gregory Haynes
7f9506962a Fix formatting for supported distros in docs
The supported distros table is not formatted consistently.

Change-Id: Ic9e32786419243c118088f933d86329480f3345b
2016-10-07 15:19:47 +11:00
Gregory Haynes
5758176a42 Rename devloper documentation to developer guide
We currently have 'user guide' and 'developer documentation'. Lets
rename to 'developer guide' for consistency.

Change-Id: I834ea313bc34275ef33e8c49a1689dff41892015
2016-10-07 15:15:30 +11:00
Gregory Haynes
9f3bb9652a Remove copyright from docs / toc
This is redundant (we have these copyright notices elsewhere) and
provide no value to the user.

Change-Id: I3699c01f0d4126cdffb3e3edf87ce62325f9f67a
2016-10-07 15:15:30 +11:00
Gregory Haynes
14b9d7ff3e Use globaltoc in docs for sidebar
By default sphinx uses localtoc which means 'show TOC for this page'.
Using a global table of contents on the sidebar is much more user
friendly wih our docs structure.

Change-Id: I215732d3848b4b75d9171bdbaaf2ff2e4dcc01f0
2016-10-07 15:15:30 +11:00
Gregory Haynes
c80cf9ec58 Fix developer guide TOC
The table of contents for our developer guide does not show due to the
fact that it is past the first sub-header.

Change-Id: I8459a4949e3e4822b0a3cd4f163475d2c60b0f2e
2016-10-07 15:15:28 +11:00
Jenkins
9a4f9e541e Merge "Add low-hanging-fruit bug tag to docs" 2016-10-07 04:07:36 +00:00
Gregory Haynes
3e4889cd6a Add low-hanging-fruit bug tag to docs
This is useful in the developer quickstart guide.

Change-Id: I54e7166bace055ae583f5dc84b648bba5f4f9cf4
2016-10-06 14:51:17 -07:00
Gregory Haynes
9a8d67e36f Add bugs link to docs
Change-Id: I09dcfe5abbc76e1383851282f063f4c338f6d93a
2016-10-06 14:49:30 -07:00
Hironori Shiina
30b3fc8dcc Fix a command in Developer Documentation
Fix a command for creating a new virtualenv.

Change-Id: Ia4981af390bf5218f22ea753db86a5edfbb602f2
2016-10-04 22:22:26 +09:00
Ian Wienand
18a4109713 Merge remote-tracking branch 'origin/master' into feature/v2
Change-Id: I796546f3873a0c9072215e63703aaa64753c5afd
2016-09-20 20:23:11 +10:00
Jenkins
61123b0286 Merge "Spec for changing the block device handling" 2016-09-12 21:44:20 +00:00
Jenkins
668561d156 Merge "Add specs dir" 2016-09-12 21:44:13 +00:00
Ian Wienand
91b431ce78 Move element-info to a standard entry-point
Move element-info from a wrapper script to a standard entry-point
console_script.

Update the documentation to explain how to run it for development.  I
don't think we should support the idea that you can check-out the code
and run ./bin/disk-image-create -- it has dependencies (dib-utils,
etc) and needs to be run from a virtualenv (this is what CI in the
gate does).  A follow-up can clean-up some of the path munging stuff
we have for this in disk-image-create.

Change-Id: Ic0c03995667f320a27ac30441279f3e6abb6bca8
2016-09-08 15:29:56 +10:00
Andreas Florath
3d48a528c1 Refactor: block-device handling (local loop)
Block device handling can be somewhat complex - especially
when taking things like md, lvm or encryption into account.

This patch factors out the creation and deletion of the local
loop image device handling into a python library.

The main propose of this patch is to implement the needed
infrastructure.  Based on this, more advanced functions can be added.
Example: (advanced) partitioning, LVM, handling different boot
scenarios (BIOS, UEFI, ...), possibility of handling multiple images
(local loop image, iSCSI, physical hard disk, ...), handling of
different filesystems for different partitions / LVs.

Change-Id: Ib626b36a00f8a5dc3dbde8df3e2619a2438eaaf1
Signed-off-by: Andreas Florath <andreas@florath.net>
2016-09-08 04:31:01 +00:00
Ian Wienand
37a53354ec Add IMAGE_ELEMENT_YAML and get_image_element_array
These new variables are a list of elements chosen for the build along
with their full paths.  For Python elements, IMAGE_ELEMENT_YAML is a
YAML formatted list that can be easily parsed.  For bash elements,
"get_image_element_array" will produce an associative-array of the
same (working around lack of array export in Bash).

This list is intended for consumption of elements who need to copy
files from other elements, such as pkg-map and svc-map.  As discussed
in I2a29861c67de2d25c595cb35d850e92807d26ac6, this list has already
been pruned and had overrides processed, so it is safe to simply walk
over this list with no further processing.

Since we're presenting the element list in a couple of different ways,
we combine it all into the element-info script.  It will output an
eval-able string that declares the appropriate variables.

I've added some inline documentation so they still appear in grep.
The documentation is updated with examples, and moved to a more
appropriate location as a sub-section of the element sytle guide.

To test this out, use the associative-array in generate_hooks, where
we can now find the element's directory without searching.

Change-Id: Ibbd07d082ec827441def2d3f6240df3efdc6eae3
2016-09-08 11:08:07 +10:00
Ian Wienand
274be6de55 Making element overriding explicit
This is a re-factor of element_dependencies to achieve two things --
centralising override policy and storing path names.

Firstly we want to make the override policy for elements completely
explicit.  Currently, elements that wish to copy parts of other
elements walk ELEMENTS_PATH themselves and look for elements in
IMAGE_ELEMENT.  How they handle duplicate elements can differ, leading
to inconsistent behaviour.

We introduce logic in element-info to find elements in each of the
directories in ELEMENT_PATHS in *reverse* order -- that is to say,
earlier entries in the paths will overwrite later ones.

For example

 ELEMENT_PATHS=foo:bar:baz

will mean that "foo/element" will override "baz/element", since "foo"
is first.  This should be sane to anyone familiar with $PATH.
Documentation is clarified around this point and a test-case is added.

The second thing is that we want to keep the complete path of the
elements we have chosen.  We want the aforementioned elements that
walk the element list to use these canonical paths to pickup files;
this way they don't need to make local decisions about element
overrides, but can simply iterate a list and copy/merge files if they
exist.

A follow-on change (I7092e1845942f249175933d67ab121188f3511fd) will
expose this data in a separate variable that can be parsed by elements
(a further follow-on I0a64b45e9f2cfa28e84b2859d76b065a6c4590f0
modifies the elements to use this information).  Thus this does not
change the status-quo -- elements that are walking ELEMENTS_PATH
themselves and can/will continue doing that.

Change-Id: I2a29861c67de2d25c595cb35d850e92807d26ac6
2016-09-08 10:58:19 +10:00
Andreas Florath
20bb6a0255 Add option to be able to run_functests.sh in parallel
Running the functional tests is time consuming.  This patch adds the
option `-j <job count>` to the tests/run_functests.sh: when given the
test run in parallel up the <job count> jobs.
When using this, be sure to have enough resources (CPUs, RAM and HD
space) on the host.

In addition there was the need to change two things:
o Global /tmp/dib-test-should-fail was move to temporary build
  directory of each execution.
o Because the logs might now interleave, each log line has now a
  prefix of the name of the testcase.

[In my environment running functests sequential takes 15+ minutes,
running them parallel takes less than 6 minutes.]

Change-Id: Id9ea5131f0026c292ca6453ba2c80fe12c47f808
Signed-off-by: Andreas Florath <andreas@florath.net>
2016-09-03 16:51:55 +02:00
Jenkins
8e214d598c Merge "Add blurb about communication to docs landing page" 2016-09-01 06:29:27 +00:00
Andreas Florath
6a5da7e157 Spec for changing the block device handling
During the creation of a disk image (e.g. for a VM), there is the need
to create, setup, configure and afterwards detach some kind of storage
where the newly installed OS can be copied to or directly installed
in.

Change-Id: I0a43e247fb9e258e3983db35362f627416983773
Depends-On: I7bd7e9fa94635621590f72702107e218155fef2a
Signed-off-by: Andreas Florath <andreas@florath.net>
2016-08-22 16:16:00 +00:00
Gregory Haynes
62efc03732 Clarify OVERWRITE_OLD_IMAGE docs
There has been some confusion about what this environment variable
controls, and it isnt very clear in the docs.

Change-Id: Id21b3c5ce361c4d1121eb7015020235b4c0a2f36
2016-08-15 12:15:22 -07:00
Gregory Haynes
16bad71e8b Add blurb about communication to docs landing page
We should really have a summary of how to communicate with us on the
documentation landing page.

Change-Id: I24a14826cbcd19a1eedc48a62220caa60ce07b43
2016-08-15 08:42:56 -07:00
Gregory Haynes
b59ae02431 Add specs dir
Currently we do not have a dib-specific specs repository. Technically,
we are part of the tripleo-specs repository but dib-core does not imply
tripleo-specs core. To fix this and to encourage the use of specs lets
create a specs process that lives right in tree.

Change-Id: I7bd7e9fa94635621590f72702107e218155fef2a
2016-08-10 22:05:00 +00:00
Andreas Florath
828f9279c0 Spec for changing the block device handling: partitioning
During the creation of a disk image (e.g. for a VM), there is the need
to create, setup, configure and afterwards detach some kind of storage
where the newly installed OS can be copied to or directly installed
in.

This document describes the details of partitioning.

Change-Id: I72d8459e81d7da7a8feaebbf295d6c436a686a07
Signed-off-by: Andreas Florath <andreas@florath.net>
2016-07-16 21:45:05 +02:00
Jenkins
3d85df1518 Merge "Fix sphinx-build to not depend on diskimage-builder" 2016-06-28 08:34:05 +00:00
Gregory Haynes
74fe5e3d4e Fix copyright in docs
We don't do copyright assignment to the foundation.

Change-Id: I95557a3038953755466b7b775491c396cc3e9129
2016-06-23 02:19:03 +00:00
Gregory Haynes
8b3419b0a9 Fix sphinx-build to not depend on diskimage-builder
Currently, running sphinx_build fails for us because we depend on
diskimage-builder in our sphinx conf.py. This causes doc generation
on sites like rtfd to fail unless they install the diskimage-builder
module beforehand. We can, alternatively, import pbr directly and not
require the module as part of doc generation.

Change-Id: I41f222ff9c67950fc30841935a6a603f5718395e
2016-06-20 18:36:42 -07:00
Jenkins
5f4cac3303 Merge "Add a best-effort sudo safety check" 2016-05-26 17:31:00 +00:00
Ian Wienand
e142014dbd Small doc update, add link to relnotes
I noticed we have no way easy way to get to the release notes.  I
updated the front-page text to be a simpler introduction/overview and
provided links to the relnotes and source directly.

Change-Id: I5e339baf2921752ca3d409d82e0cbfc856ead1f8
2016-05-16 15:13:39 +10:00
Ian Wienand
672705831f Add a best-effort sudo safety check
As motivation for this; we have had two breakouts of dib in recent
memory.  One was a failure to unmount through symlinks in the core
code (I335316019ef948758392b03e91f9869102a472b9) and the other was
removing host keys on the build-system
(Ib01d71ff9415a0ae04d963f6e380aab9ac2260ce).

For the most part, dib runs unprivileged.  Bits of the core code are
hopefully well tested (modulo bugs like the first one!).  We give free
reign inside the chroot (although there is still some potential there
for adverse external affects via bind mounts).  Where we could be a
bit safer (and could have prevented at least the second of these
breakouts) is with some better checking that the "sudo" calls
*outside* the chroot at least looked sane.

This adds a basic check that we're using chroot or image paths when
calling sudo in those parts of elements that run *outside* the chroot.
Various files are updated to accomodate this check; mostly by just
ignoring it for existing code (I have not audited these calls).

Nobody is pretending this type of checking makes dib magically safe,
or removes the issues with it needing to do things as root during the
build.  But this can help find egregious errors like the key removal.

Change-Id: I161a5aea1d29dcdc7236f70d372c53246ec73749
2016-05-09 15:41:38 +10:00
Ben Nemec
c6b6f269cc Add documentation for dib-lint
Prior to this, no user documentation of dib-lint existed, which
meant users had to read the dib-lint code itself to figure out
how it worked.  This changes adds documentation on using dib-lint
and the checks it currently supports.

Change-Id: I285c5cc680dd9fbd9bd3f667ef102be14e248114
2016-05-02 01:29:17 -05:00
Gregory Haynes
9a3f31df98 Document upstream executable numbering convention
Add documentation to our developer guide about not creating executables
before or after 10/90 in the upstream element's phase directories.

Change-Id: I93ab70f37da0d81f8683a76fd3b341b761ea04e9
2016-04-20 04:09:39 +00:00
Jenkins
404ca1b944 Merge "Add image size report" 2016-04-02 15:45:42 +00:00
Jenkins
db50f8f8e1 Merge "Reorder developer quickstart docs" 2016-03-17 22:11:41 +00:00
SamYaple
5b6716cee8 Use fstrim to prep the block device
This cuts the image size down alot, esspecially if there were lots of
small file deletes.

The fstrim utility is in the util-linux package and should be on
most all systems. fstrim also works with XFS, ext4, btrfs, etc
prodiving the kernel is new enough.

A reduction of 25% or more in size is common.

Change-Id: I269b4416be450369616f9b8e030f84c30e329804
2016-03-13 16:24:59 +00:00
SamYaple
0c32784663 Revert "Zerofree the image if possible"
This reverts commit 5184d02a7c.

The decision was made to go with fstrim because it is faster and more
universal that zerofree. The related-id has the patchset that implements
fstrim.

Related-Id: I269b4416be450369616f9b8e030f84c30e329804
Change-Id: If40cf2fc0ecd8686768cbfeac9ecee90907674e7
2016-03-10 20:38:01 +00:00
Ian Wienand
fa3c5e3056 Add image size report
In the common case of not specifying a size, we are already running
"du" over the image to figure out how big it is.  Leverage that by
saving it's output and displaying a pruned list of big files when
requested.

We add a flag to show a summarised option (files >10MiB) and another
to show full output, should you wish that level of detail.

"Invocation" documentation is updated (and formatted a little better
while we're here).

Change-Id: I255800790a62fed1c82fcd311f1cc29c9867766d
2016-03-08 13:58:40 +11:00
Clark Boylan
5184d02a7c Zerofree the image if possible
If the image has an ext filesystem and the zerofree utility is present
on the build system then run zerofree. This should make images as
compressable as possible which is a nice feature when building
compressed qcow2 images.

Change-Id: Ia6062c291f7a3f58b85a4f408ecb3d0574c65d53
2016-03-07 17:45:14 -08:00
Gregory Haynes
3eeefefa85 Reorder developer quickstart docs
The quickstart should be the first bit of developer documentation, not
the last. Also add in a short blurb for the developer docs so we dont
have two doc titles back to back.

Change-Id: Icb5683b8eb22e759fefb1cb2252ed445dea5f7dd
2016-03-01 21:44:18 -08:00
Matthew Thode
175e292ec1 Add Gentoo to the list of supported distributions
We also document any additional package needed.

Change-Id: I7b9193fa5711138287f587935fc864fee4f95856
2016-02-29 12:20:50 +00:00
Ian Wienand
b18f71f781 Rework functional test runner
This simplifies and enhances the functional-test runner script for
much better interactive behaviour and to give us the ability to better
choose what is running in CI.

Firstly, I have split the image-output testing into a separate script.
This is not actually part of the functional testing of elements and is
both logically and functionally different.  It currently does not run
in upstream CI because we don't have docker in the images.  I have
nothing against it, but it can be it's own thing.

run_functests.sh is overhauled to have a useful interactive interface,
e.g.

---
 $ ./run_functests.sh -h
 run_functests.sh [-h] [-l] <test> <test> ...
   -h : show this help
   -l : list available tests
   <test> : functional test to run
            Special test 'all' will run all tests

 $ ./run_functests.sh -l
 The available functional tests are:

  apt-sources/test-sources
  debian/build-succeeds
  fedora/build-succeeds
  fedora/build-succeeds-f21
  ironic-agent/build-succeeds-fedora
---

As described there, you can run a single test, a number of tests, the
default tests (as CI will do) or all tests.  Running all tests is too
much for regular CI, but currently the only way to stop a low priority
test running, or temporarily pause is to remove it completely --
clearly sub-optimal (see I93c2990472e88ab3e5ff14db56b4ff1b4dd965ef).

There is nothing complicated about this, and to further simplify I
have merged the runner functions back into run_functests.sh which
remains a very modest ~150 lines, with most of that being argument
sanity.  With that and the image-format cleanup, we can remove the
indirection of the 3 small library files.

For consistency, I have renamed the "dib_functions_test" (that tests
things from the dib functions library) with a run_* prefix.

Because the default list is the same as the current functional tests
run, this does not modify the status-quo.  I plan to modify this,
however, to run fedora-minimal & centos-minimal tests in a future
change, as these are required to be stable for openstack ci.

Documentation is updated, and a README.rst is added in the tests
directory for discoverability.

Change-Id: I86d208bd34ff09a29fdb916a4e7ef740c7f65af8
2016-02-19 13:50:09 +11:00
Jenkins
6f6a096d94 Merge "Handle install with pip -e" 2016-02-12 20:28:51 +00:00
Isaac Beckman
24d1527111 Move speedup section to image building guide
speedup section explains  the user how to sppedup image build by using
tmpfs. The  correct user guide to have this section, is the user guide
about image building rather than the installation user guide.

Change-Id: I96b90bd79df53db4f926a928ae3c86b888315230
2016-01-18 07:02:00 +00:00
Abel Lopez
5e9c451d5f Document byte-to-inode ratio
End user docs would benefit from a section about the byte-to-inode
ratio, and why it's set the way it is. This update explains why
and how to manipulate the ratio depending on the intended use.

Change-Id: Iffb5ef6f4c7c74f4aa6e25912d4991d7a611c8fe
Closes-bug: 1512841
2015-12-16 13:58:17 -08:00
Gregory Haynes
930b597220 Move install-types doc to user guide
Install-types are a user facing feature, not just for developers. Lets
move the docs on them in to the user guide.

Change-Id: I6ee8f657c270cf90da9c0729494740bb23aa47c5
2015-11-13 18:45:28 +00:00
Jenkins
45928ab5d1 Merge "Add a tox target to run functional tests locally" 2015-11-03 20:56:02 +00:00
Dmitry Tantsur
0e122e8e35 Add a tox target to run functional tests locally
Now 'tox -efunc' can be invoked to run all functional tests in
the 'venv' tox environment. Also `tox -efunc element-name` can be
used to run function tests for one element (e.g. ironic-agent).

Change-Id: Ia685d1b2a7deef2f8b98876ac09792134dd30f2f
2015-10-23 12:05:23 +02:00
Antoine Musso
bcce4842cf Prettify 'Caches and offline mode' documentation
Use inline code syntax for env variables.
Link to elements README files.
Point 'redhat' to 'redhat-common' instead.

Change-Id: Ied1150aaa631c7c6d7f2f55314f9aa3529fd4ba0
2015-09-17 12:14:54 +02:00
Antoine Musso
e8cecab45c Prettify 'Developing Elements' documentation
Augment the developing_elements.rst by taking advantage of Sphinx
markup. Most of the doc used to be in /README.md and thus did not
leverage on Sphinx.

Use inline codeblock to denote variables, files, command: ``foo``
Phase Subdirectories:
  List phase names in the preliminary introduction
  Get rid of lists in favor of definitions
  Highlight whether the phase runs in or outside the chroot
  Input parameters are now lists

Use definition lists in Dependencies and Ramdisk sections.

Link to elements README when they exist.

Testing Element: split into two subsections: 'shell' and 'python'.

Use "sourcecode:: sh" for the couple examples at the very top and very
bottom of the document.

Change-Id: I2421f76ec452cac243ccb2208f88c7d320ffedd3
2015-09-16 15:49:23 +02:00
Jenkins
79a0df6199 Merge "doc: migrate from README.rst to Sphinx" 2015-09-16 01:24:07 +00:00
Jenkins
fe23b262fd Merge "Add documentation of output formats for users" 2015-09-11 13:06:42 +00:00
Ian Wienand
ab5ed610e4 Handle install with pip -e
As described in the comments, inspect the installation to see if we
have been installed with "pip -e" and, if so, make sure we reference
the scripts from the source location rather than the
system-installations.

Update the documentation with a terse but helpful quick-start to show
an easy way to start developing a change using this.

Closes-Bug: #1491035
Change-Id: I0460061b834a2b854175f8c9be2be8d38c540c9d
2015-09-10 16:55:39 +10:00
Antoine Musso
f22a5a8b91 doc: migrate from README.rst to Sphinx
The README.rst has a lot of information that has been duplicated in the
Sphinx maintained documentation (3600330).

Remove dupes from README.rst
Point to http://docs.openstack.org/developer/diskimage-builder/

Change summary:

=====================+======================================
README.md            | Sphinx document
section              |
=====================+======================================
Installation         | installation.rst
---------------------+--------------------------------------
Invocation           | invocation.rst
---------------------+--------------------------------------
Requirements         | installation.rst Speedups
---------------------+--------------------------------------
Caches/offline       | caches.rst + changes from 849e9cb2
                     | fix some markup
---------------------+--------------------------------------
Install Types        | install_types.rst
---------------------+--------------------------------------
Writing an element   | developing_elements.rst + fe7823a2
                     | `Testing element` from b9b6640f
                     | `3rd party elements' from f1e7bf3a
---------------------+--------------------------------------
Existing elements    | elements.rst
---------------------+--------------------------------------
What tools are there | components.rst
---------------------+--------------------------------------
Design               | design.rst
---------------------+--------------------------------------

Change-Id: I578daa8e3a8d876b3ee3c9a748d7c8aa2bf7a0b7
2015-09-10 16:55:39 +10:00
Jenkins
86bcb6ec22 Merge "Remove docker doc from docs" 2015-08-14 07:43:19 +00:00
Jenkins
7725b21ff2 Merge "Fix link in installation doc" 2015-08-04 05:59:05 +00:00
Jenkins
0bd4567261 Merge "typos on the document" 2015-08-04 01:52:51 +00:00
Gregory Haynes
486f3c01eb Add documentation of output formats for users
A blurb on output formats is useful for our users.

Change-Id: Iffa5036a84c1500ccb38cd3edb258ddbf5148a3e
2015-08-02 23:19:39 -07:00
Gregory Haynes
33ce0d3f6e Remove docker doc from docs
We now support docker as a native output format, so this doc is out of
date and not useful.

Change-Id: Ib13cfc815a9acb7178bce02a858262d8f3b17c87
2015-08-02 23:17:56 -07:00
Gregory Haynes
837bac1070 Fix link in installation doc
The link to building an image is not being properly parsed by sphinx in
the install user guide.

Change-Id: I1e73a4722b603ba11805031c522b107cae8c86b4
2015-07-24 23:47:30 +00:00
Jenkins
8fe44fba7e Merge "Document our supported distributions" 2015-07-24 00:49:23 +00:00
Jenkins
2a0b32418b Merge "Document what our stable interfaces are" 2015-07-23 14:18:00 +00:00
Atsushi SAKAI
f24e78a48d typos on the document
jenkin => jenkins
 on ci.md
documention => documentation
 on developer/caches.html
typicallly => typically
 on developer/developing_elements.html
Closes-Bug: #1476993

Change-Id: Ie40205debad5dbc6074e65672e0f3ebeaee5b08e
2015-07-23 11:56:30 +00:00
Gregory Haynes
c922640d3f Document what our stable interfaces are
Before we do a 1.0 release, we should really document what we are
commiting to.

Change-Id: I44c62f9b1a35ace5dc7d963ab5bddc83fba4bce3
2015-07-22 09:17:02 +00:00
Gregory Haynes
89d0896b97 Document our supported distributions
Before we commit to supporting a set of distributions, we should
document what they are.

Change-Id: I076c326ec0eb4d1640a4229c0dfebd3d2cc880b4
2015-07-13 00:36:44 +00:00
Jenkins
7aab6c63c2 Merge "Remove deprecated disk-image-get-kernel" 2015-07-07 01:37:51 +00:00
Gregory Haynes
8cf37064a5 Remove deprecated disk-image-get-kernel
This script has been deprecated in favor of the baremetal element for
some time. Lets remove it.

Change-Id: Idd7db766a039ba8e74f0e98e74eb27246e753f4b
2015-07-06 16:44:07 +00:00
Gregory Haynes
9a166b9df6 Document only exectuables in phase dirs policy
Having data files in the phase subdirs is an easy source of confusion
in reviews (especially when the data file is a script) and theres really
no reason to be putting data files there at this point. Lets make a
convention out of not doing this.

Change-Id: I99571a2a49e14e8c709af20f6d13d662ac745eb4
2015-06-16 17:03:17 +00:00
Jenkins
eb732d3d65 Merge "Initial element tests" 2015-05-21 20:19:20 +00:00
Jenkins
75a0b91fe2 Merge "doc: small snippet about operating system elements" 2015-05-20 19:09:46 +00:00
Jenkins
2835199ace Merge "Update install docs to be more user friendly" 2015-05-19 22:31:19 +00:00
Gregory Haynes
b9b6640fa7 Initial element tests
Adding a test function which allows us to use elements to perform
element-specific tests. In order for this to work sanely, also adding
some configuration to our break system so we can assert on negative
tests.

Also adding a test for apt-sources to verify this code actually works.

Change-Id: I378a74255010eca192f5766b653f8a42404be5ea
2015-05-17 02:07:40 +00:00
Pino Toscano
2a7052e8ff doc: small snippet about operating system elements
Add a small documentation paragraph about the operating system elements,
what they are required to provide (and thus what other elements can rely
on).

This makes DISTRO_NAME a prime-class variable, which can now be assumed
to always exists (it was de-facto required so far).

Change-Id: Iffbc69de0516b58bfde48e87cd73073428d66b05
2015-05-15 19:12:41 +02:00
Antoine Musso
8d052a54a7 Turn docs warnings into errors and fix issues
Set the pbr option 'warnerrors' to make build_sphinx turns warnings into
error. Fix all warnings.

`tox -edocs` will thus abort whenever someone introduce a new error.

Change-Id: Id6d09768a241866e1fdc1a1e2bf90336f5c5087d
2015-04-27 15:03:19 +00:00
Gregory Haynes
d98e6dcff9 Update install docs to be more user friendly
Our install docs are out of date and not very user friendly. Lets fix
that.

Change-Id: Idaff33096bf32865020b85ee776abd6691ac45ad
2015-04-12 16:08:47 +00:00
Gregory Haynes
360033027f Create a user guide
Our docs are very developer focused. Lets create a separate user guide
to help new users get started.

Change-Id: I8a03920e6d3306dd0405177875ea55ccb4b40fea
2015-04-01 19:51:08 +00:00
James Polley
27d03777dd Cleanup/restify components.rst
Rework the list as a definition list.

Mark literal code snippets as literal code snippets.

Fix heading typo.

Change-Id: Ie3d393a49914779f12f3075ff2ac2df5ecf78321
2015-03-18 02:12:32 +00:00
Gregory Haynes
4f9131e692 Split out README into separate files
This makes the docs site a lot more manageable and begins moving us in
the direction of separate user and developer docs.

Change-Id: I1650ef9d5be1733b8bc118d99090143cb5b06102
2015-02-24 15:32:02 -08:00
Gregory Haynes
a35d7f88cf Use oslosphinx for docs site
This gets us the openstack theme

Change-Id: I679bbc36d58294ef14852a7938435eacf71321ed
2015-02-11 23:29:21 -08:00
Gregory Haynes
c4bbb6f3bc Create docs site containing element READMEs
We currently do not have the ability to create a docs site which
outlines all the elements.

Change-Id: I77ccf61e0c4b1509b3e7ce9b8f15ea5ccfd50d9b
2015-02-10 11:45:35 -08:00