diskimage-builder/doc/source/developer/caches.rst

Caches and offline mode
=======================

Since retrieving and transforming operating system image files, git
repositories, Python or Ruby packages, and so on can be a significant overhead,
we cache many of the inputs to the build process.

The cache location is read from ``DIB_IMAGE_CACHE``. :ref:`developing-elements`
describes the interface within disk-image-builder for caching.

When invoking disk-image-builder, the ``--offline`` option will instruct
disk-image-builder to not refresh cached resources. Alternatively you can set
``DIB_OFFLINE=1``.

Note that we don't maintain operating system package caches, instead depending
on your local infrastructure (e.g. Squid cache, or an APT or Yum proxy) to
facilitate caching of that layer, so you need to arrange independently for
offline mode. For more information about setting up a squid proxy, consult the
`TripleO documentation
<https://docs.openstack.org/tripleo-incubator/latest/devtest_setup.html>`_.

Base images
-----------

These are cached by the standard elements - :doc:`../elements/fedora/README`,
:doc:`../elements/redhat-common/README`, :doc:`../elements/ubuntu/README`,
:doc:`../elements/debian/README` and :doc:`../elements/opensuse/README`.

source-repositories
-------------------

Git repositories and tarballs obtained via the
:doc:`../elements/source-repositories/README` element will be cached.

PyPI
----

The :doc:`../elements/pypi/README` element will bind mount a PyPI mirror from
the cache dir and configure pip and easy-install to use it.
Split out README into separate files This makes the docs site a lot more manageable and begins moving us in the direction of separate user and developer docs. Change-Id: I1650ef9d5be1733b8bc118d99090143cb5b06102 2015-02-12 07:07:47 +00:00			`Caches and offline mode`
			`=======================`

			`Since retrieving and transforming operating system image files, git`
			`repositories, Python or Ruby packages, and so on can be a significant overhead,`
doc: migrate from README.rst to Sphinx The README.rst has a lot of information that has been duplicated in the Sphinx maintained documentation (3600330). Remove dupes from README.rst Point to http://docs.openstack.org/developer/diskimage-builder/ Change summary: =====================+====================================== README.md \| Sphinx document section \| =====================+====================================== Installation \| installation.rst ---------------------+-------------------------------------- Invocation \| invocation.rst ---------------------+-------------------------------------- Requirements \| installation.rst Speedups ---------------------+-------------------------------------- Caches/offline \| caches.rst + changes from 849e9cb2 \| fix some markup ---------------------+-------------------------------------- Install Types \| install_types.rst ---------------------+-------------------------------------- Writing an element \| developing_elements.rst + fe7823a2 \| `Testing element` from b9b6640f \| `3rd party elements' from f1e7bf3a ---------------------+-------------------------------------- Existing elements \| elements.rst ---------------------+-------------------------------------- What tools are there \| components.rst ---------------------+-------------------------------------- Design \| design.rst ---------------------+-------------------------------------- Change-Id: I578daa8e3a8d876b3ee3c9a748d7c8aa2bf7a0b7 2015-06-18 09:24:13 +00:00			`we cache many of the inputs to the build process.`

Prettify 'Caches and offline mode' documentation Use inline code syntax for env variables. Link to elements README files. Point 'redhat' to 'redhat-common' instead. Change-Id: Ied1150aaa631c7c6d7f2f55314f9aa3529fd4ba0 2015-09-17 10:14:24 +00:00			The cache location is read from ``DIB_IMAGE_CACHE``. :ref:`developing-elements`
doc: migrate from README.rst to Sphinx The README.rst has a lot of information that has been duplicated in the Sphinx maintained documentation (3600330). Remove dupes from README.rst Point to http://docs.openstack.org/developer/diskimage-builder/ Change summary: =====================+====================================== README.md \| Sphinx document section \| =====================+====================================== Installation \| installation.rst ---------------------+-------------------------------------- Invocation \| invocation.rst ---------------------+-------------------------------------- Requirements \| installation.rst Speedups ---------------------+-------------------------------------- Caches/offline \| caches.rst + changes from 849e9cb2 \| fix some markup ---------------------+-------------------------------------- Install Types \| install_types.rst ---------------------+-------------------------------------- Writing an element \| developing_elements.rst + fe7823a2 \| `Testing element` from b9b6640f \| `3rd party elements' from f1e7bf3a ---------------------+-------------------------------------- Existing elements \| elements.rst ---------------------+-------------------------------------- What tools are there \| components.rst ---------------------+-------------------------------------- Design \| design.rst ---------------------+-------------------------------------- Change-Id: I578daa8e3a8d876b3ee3c9a748d7c8aa2bf7a0b7 2015-06-18 09:24:13 +00:00			`describes the interface within disk-image-builder for caching.`

			When invoking disk-image-builder, the ``--offline`` option will instruct
Prettify 'Caches and offline mode' documentation Use inline code syntax for env variables. Link to elements README files. Point 'redhat' to 'redhat-common' instead. Change-Id: Ied1150aaa631c7c6d7f2f55314f9aa3529fd4ba0 2015-09-17 10:14:24 +00:00			`disk-image-builder to not refresh cached resources. Alternatively you can set`
			``DIB_OFFLINE=1``.
Split out README into separate files This makes the docs site a lot more manageable and begins moving us in the direction of separate user and developer docs. Change-Id: I1650ef9d5be1733b8bc118d99090143cb5b06102 2015-02-12 07:07:47 +00:00
			`Note that we don't maintain operating system package caches, instead depending`
			`on your local infrastructure (e.g. Squid cache, or an APT or Yum proxy) to`
			`facilitate caching of that layer, so you need to arrange independently for`
doc: migrate from README.rst to Sphinx The README.rst has a lot of information that has been duplicated in the Sphinx maintained documentation (3600330). Remove dupes from README.rst Point to http://docs.openstack.org/developer/diskimage-builder/ Change summary: =====================+====================================== README.md \| Sphinx document section \| =====================+====================================== Installation \| installation.rst ---------------------+-------------------------------------- Invocation \| invocation.rst ---------------------+-------------------------------------- Requirements \| installation.rst Speedups ---------------------+-------------------------------------- Caches/offline \| caches.rst + changes from 849e9cb2 \| fix some markup ---------------------+-------------------------------------- Install Types \| install_types.rst ---------------------+-------------------------------------- Writing an element \| developing_elements.rst + fe7823a2 \| `Testing element` from b9b6640f \| `3rd party elements' from f1e7bf3a ---------------------+-------------------------------------- Existing elements \| elements.rst ---------------------+-------------------------------------- What tools are there \| components.rst ---------------------+-------------------------------------- Design \| design.rst ---------------------+-------------------------------------- Change-Id: I578daa8e3a8d876b3ee3c9a748d7c8aa2bf7a0b7 2015-06-18 09:24:13 +00:00			`offline mode. For more information about setting up a squid proxy, consult the`
			`TripleO documentation
Correct link address Change-Id: Ic1680f845f3342cf14c2667589fb6f28b7c67e9d 2018-01-18 07:20:37 +00:00			<https://docs.openstack.org/tripleo-incubator/latest/devtest_setup.html>`_.
Split out README into separate files This makes the docs site a lot more manageable and begins moving us in the direction of separate user and developer docs. Change-Id: I1650ef9d5be1733b8bc118d99090143cb5b06102 2015-02-12 07:07:47 +00:00
			`Base images`
			`-----------`

Prettify 'Caches and offline mode' documentation Use inline code syntax for env variables. Link to elements README files. Point 'redhat' to 'redhat-common' instead. Change-Id: Ied1150aaa631c7c6d7f2f55314f9aa3529fd4ba0 2015-09-17 10:14:24 +00:00			These are cached by the standard elements - :doc:`../elements/fedora/README`,
			:doc:`../elements/redhat-common/README`, :doc:`../elements/ubuntu/README`,
			:doc:`../elements/debian/README` and :doc:`../elements/opensuse/README`.
Split out README into separate files This makes the docs site a lot more manageable and begins moving us in the direction of separate user and developer docs. Change-Id: I1650ef9d5be1733b8bc118d99090143cb5b06102 2015-02-12 07:07:47 +00:00
			`source-repositories`
			`-------------------`

Prettify 'Caches and offline mode' documentation Use inline code syntax for env variables. Link to elements README files. Point 'redhat' to 'redhat-common' instead. Change-Id: Ied1150aaa631c7c6d7f2f55314f9aa3529fd4ba0 2015-09-17 10:14:24 +00:00			`Git repositories and tarballs obtained via the`
			:doc:`../elements/source-repositories/README` element will be cached.
Split out README into separate files This makes the docs site a lot more manageable and begins moving us in the direction of separate user and developer docs. Change-Id: I1650ef9d5be1733b8bc118d99090143cb5b06102 2015-02-12 07:07:47 +00:00
			`PyPI`
			`----`

Prettify 'Caches and offline mode' documentation Use inline code syntax for env variables. Link to elements README files. Point 'redhat' to 'redhat-common' instead. Change-Id: Ied1150aaa631c7c6d7f2f55314f9aa3529fd4ba0 2015-09-17 10:14:24 +00:00			The :doc:`../elements/pypi/README` element will bind mount a PyPI mirror from
			`the cache dir and configure pip and easy-install to use it.`