From 94e88d6ee7ddcdcdd868f4d9dc02acaa5a7ecf7b Mon Sep 17 00:00:00 2001 From: Louis Abel Date: Fri, 21 Oct 2022 20:26:02 -0700 Subject: [PATCH] add reference examples --- docs/documentation/.pages | 3 +- docs/documentation/compose.md | 7 - docs/documentation/compose/index.md | 5 + .../references/empanadas_common.md | 22 + .../references/empanadas_config.md | 622 ++++++++++++++++++ .../references/empanadas_sig_config.md | 16 + docs/sop/sop_release.md | 66 +- 7 files changed, 732 insertions(+), 9 deletions(-) delete mode 100644 docs/documentation/compose.md create mode 100644 docs/documentation/compose/index.md create mode 100644 docs/documentation/references/empanadas_config.md create mode 100644 docs/documentation/references/empanadas_sig_config.md diff --git a/docs/documentation/.pages b/docs/documentation/.pages index eb7b5b2..f28e2d1 100644 --- a/docs/documentation/.pages +++ b/docs/documentation/.pages @@ -1,7 +1,8 @@ --- nav: - ... | index.md - - Composing Releases: compose.md + - Composing Releases: compose - Empanadas: empanadas.md + - Peridot: peridot.md - References: references ... diff --git a/docs/documentation/compose.md b/docs/documentation/compose.md deleted file mode 100644 index 2d8c886..0000000 --- a/docs/documentation/compose.md +++ /dev/null @@ -1,7 +0,0 @@ ---- -title: Composing Releases ---- - -This page goes over the process of composing a release. - -More to come. diff --git a/docs/documentation/compose/index.md b/docs/documentation/compose/index.md new file mode 100644 index 0000000..2dc4925 --- /dev/null +++ b/docs/documentation/compose/index.md @@ -0,0 +1,5 @@ +--- +title: Composing Releases +--- + +This section goes over the process of composing a release. diff --git a/docs/documentation/references/empanadas_common.md b/docs/documentation/references/empanadas_common.md index 9fc255f..30bbfa6 100644 --- a/docs/documentation/references/empanadas_common.md +++ b/docs/documentation/references/empanadas_common.md @@ -2,8 +2,13 @@ title: Empanadas common.py Configuration --- +The `common.py` configuration contains dictionaries and classes that dictate +most of the functionality of empanadas. + ## Config Items +type: Dictionary + ### config.rlmacro type: String @@ -183,6 +188,13 @@ required: False description: URL of the S3-compatible bucket that is used to pull images from. +## allowed_type_variants items + +type: Dictionary + +description: Key value pairs of cloud or image variants. The value is either +`None` or a list type. + ## Reference Example ``` @@ -212,4 +224,14 @@ config = { "bucket": "resf-empanadas", "bucket_url": "https://resf-empanadas.s3.us-east-2.amazonaws.com" } + +ALLOWED_TYPE_VARIANTS = { + "Azure": None, + "Container": ["Base", "Minimal", "UBI"], + "EC2": None, + "GenericCloud": None, + "Vagrant": ["Libvirt", "Vbox"], + "OCP": None + +} ``` diff --git a/docs/documentation/references/empanadas_config.md b/docs/documentation/references/empanadas_config.md new file mode 100644 index 0000000..a7a3d81 --- /dev/null +++ b/docs/documentation/references/empanadas_config.md @@ -0,0 +1,622 @@ +--- +title: Empanadas config yaml Configuration +--- + +Each file in `empanads/config/` is a yaml file that contains configuration +items for the distribution release version. The configuration can heavily +dictate the functionality and what features are directly supported by empanadas +when ran. + +See the items below to see which options are mandatory and optional. + +# Config Items + +## Top Level + +The Top Level is the name of the profile and starts the YAML dictionary for the +release. It is alphanumeric and accepts punctuation within reason. Common +examples: + +* `9` +* `9-beta` +* `8-lookahead` + +### fullname + +type: String + +required: True + +description: Needed for treeinfo and discinfo generation. + +### revision + +type: String + +required: True + +description: Full version of a release + +### rclvl + +type: String + +required: True + +description: Release Candidate or Beta descriptor. Sets names and versions with +this descriptor if enabled. + +### major + +type: String + +required: True + +description: Major version of a release + +### minor + +type: String + +required: True + +description: Minor version of a release + +### profile + +type: String + +required: True + +description: Matches the top level of the release. This should not differ from +the top level assignment. + +### disttag + +type: String + +required: True + +description: Sets the dist tag for mock configs. + +### bugurl + +type: String + +required: True + +description: A URL to the bug tracker for this release or distribution. + +### checksum + +type: String + +required: True + +description: Checksum type. Used when generating checksum information for +images. + +### fedora_major + +type: String + +required: False + +description: For future use with icicle. + +### allowed_arches + +type: list + +required: True + +description: List of supported architectures for this release. + +### provide_multilib + +type: boolean + +required: True + +description: Sets if architecture x86_64 will be multilib. It is recommended +that this is set to `True`. + +### project_id + +type: String + +required: True + +description: Appended to the base repo URL in common.py. For peridot, it is the +project id that is generated for the project you are pulling from. It can be set +to anything else if need be for non-peridot use. + +### repo_symlinks + +type: dict + +required: False + +description: For future use. Sets symlinks to repositories for backwards +compatibility. Key value pairs only. + +### renames + +type: dict + +required: False + +description: Renames a repository to the value set. For example, renaming `all` +to `devel`. Set to `{}` if no renames are goign to occur. + +### all_repos + +type: list + +required: True + +description: List of repositories that will be synced/managed by empanadas. + +### structure + +type: dict + +required: True + +description: Key value pairs of `packages` and `repodata`. These are appended +appropriately during syncing and ISO actions. Setting these are mandatory. + +### iso_map + +type: dictionary + +required: True if building ISO's and operating with lorax. + +description: Controls how lorax and extra ISO's are built. + +If are you not building images, set to `{}` + +#### xorrisofs + +type: boolean + +required: True + +description: Dictates of xorrisofs is used to build images. Setting to false +uses genisoimage. It is recommended that xorrisofs is used. + +#### iso_level + +type: boolean + +required: True + +description: Set to false if you are using xorrisofs. Can be set to true when +using genisoimage. + +#### images + +type: dict + +required: True + +description: Dictates the ISO images that will be made or the treeinfo that will +be generated. + +**Note**: The primary repository (for example, BaseOS) will need to be listed to +ensure the treeinfo data is correctly generated. `disc` should be set to `False` +and `isoskip` should be set to `True`. See the example section for an example. + +##### name.disc + +type: boolean + +required: True + +description: This tells the iso builder if this will be a generated ISO. + +##### name.isoskip + +type: boolean + +required: False + +description: This tells the iso builder if this will be skipped, even if `disc` +is set to `True`. Default is `False`. + +##### name.variant + +type: string + +required: True + +description: Names the primary variant repository for the image. This is set in +.treeinfo. + +##### name.repos + +type: list + +required: True + +description: Names of the repositories included in the image. This is added to +.treeinfo. + +##### name.volname + +type: string + +required: True + +required value: `dvd` + +description: This is required if building more than the DVD image. By default, +the the name `dvd` is harcoded in the buildImage template. + +#### lorax + +type: dict + +required: True if building lorax images. + +description: Sets up lorax images and which repositories to use when building +lorax images. + +##### lorax.repos + +type: list + +required: True + +description: List of repos that are used to pull packages to build the lorax +images. + +##### lorax.variant + +type: string + +required: True + +description: Base repository for the release + +##### lorax.lorax_removes + +type: list + +required: False + +description: Excludes packages that are not needed when lorax is running. + +##### lorax.required_pkgs + +type: list + +required: True + +description: Required list of installed packages needed to build lorax images. + +#### livemap + +type: dict + +required: False + +description: Dictates what live images are built and how they are built. + +##### livemap.git_repo + +type: string + +required: True + +description: The git repository URL where the kickstarts live + +##### livemap.branch + +type: string + +required: True + +description: The branch being used for the kickstarts + +##### livemap.ksentry + +type: dict + +required: True + +description: Key value pairs of the live images being created. Key being the +name of the live image, value being the kickstart name/path. + +##### livemap.allowed_arches + +type: list + +required: True + +description: List of allowed architectures that will build for the live images. + +##### livemap.required_pkgs + +type: list + +required: True + +description: Required list of packages needed to build the live images. + +#### cloudimages + +type: dict + +required: False + +description: Cloud related settings. + +Set to `{}` if not needed. + +##### cloudimages.images + +type: dict + +required: True + +description: Cloud images that will be generated and in a bucket to be pulled, +and their format. + +###### cloudimages.images.name + +type: dict + +required: True + +description: Name of the cloud image being pulled. + +Accepted key value option `format`, which is `raw` or `qcow2` + +#### repoclosure_map + +type: dict + +required: True + +description: Repoclosure settings. These settings are absolutely required when +doing full syncs and need to check repositories for consistency. + +##### repoclosure_map.arches + +type: dict + +required: True + +description: For each architecture (key), dnf switches/settings that dictate how +repoclosure will check for consistency (value, string). + +example: `x86_64: '--forcearch=x86_64 --arch=x86_64 --arch=athlon --arch=i686 --arch=i586 --arch=i486 --arch=i386 --arch=noarch'` + +##### repoclosure_map.repos + +type: dict + +required: True + +description: For each repository that is pulled for a given release(key), +repositories that will be included in the repoclosure check. A repository that +only checks against itself must have a value of `[]`. + +#### extra_files + +type: dict + +required: True + +description: Extra files settings and where they come from. Git repositories are +the only supported method. + +##### extra_files.git_repo + +type: string + +required: True + +description: URL to the git repository with the extra files. + +##### extra_files.git_raw_path + +type: string + +required: True + +description: URL to the git repository with the extra files, but the "raw" url +form. + +example: `git_raw_path: 'https://git.rockylinux.org/staging/src/rocky-release/-/raw/r9/'` + +##### extra_files.branch + +type: string + +required: True + +description: Branch where the extra files are pulled from. + +##### extra_files.gpg + +type: dict + +required: True + +description: For each gpg key type (key), the relative path to the key in the +git repository (value). + +These keys help set up the repository configuration when doing syncs. + +By default, the RepoSync class sets `stable` as the gpgkey that is used. + +##### extra_files.list + +type: list + +required: True + +description: List of files from the git repository that will be used as "extra" +files and placed in the repositories and available to mirrors and will appear on +ISO images if applicable. + +# Reference Example + +``` +--- +'9': + fullname: 'Rocky Linux 9.0' + revision: '9.0' + rclvl: 'RC2' + major: '9' + minor: '0' + profile: '9' + disttag: 'el9' + bugurl: 'https://bugs.rockylinux.org' + checksum: 'sha256' + fedora_major: '20' + allowed_arches: + - x86_64 + - aarch64 + - ppc64le + - s390x + provide_multilib: True + project_id: '55b17281-bc54-4929-8aca-a8a11d628738' + repo_symlinks: + NFV: 'nfv' + renames: + all: 'devel' + all_repos: + - 'all' + - 'BaseOS' + - 'AppStream' + - 'CRB' + - 'HighAvailability' + - 'ResilientStorage' + - 'RT' + - 'NFV' + - 'SAP' + - 'SAPHANA' + - 'extras' + - 'plus' + structure: + packages: 'os/Packages' + repodata: 'os/repodata' + iso_map: + xorrisofs: True + iso_level: False + images: + dvd: + disc: True + variant: 'AppStream' + repos: + - 'BaseOS' + - 'AppStream' + minimal: + disc: True + isoskip: True + repos: + - 'minimal' + - 'BaseOS' + variant: 'minimal' + volname: 'dvd' + BaseOS: + disc: False + isoskip: True + variant: 'BaseOS' + repos: + - 'BaseOS' + - 'AppStream' + lorax: + repos: + - 'BaseOS' + - 'AppStream' + variant: 'BaseOS' + lorax_removes: + - 'libreport-rhel-anaconda-bugzilla' + required_pkgs: + - 'lorax' + - 'genisoimage' + - 'isomd5sum' + - 'lorax-templates-rhel' + - 'lorax-templates-generic' + - 'xorriso' + cloudimages: + images: + EC2: + format: raw + GenericCloud: + format: qcow2 + livemap: + git_repo: 'https://git.resf.org/sig_core/kickstarts.git' + branch: 'r9' + ksentry: + Workstation: rocky-live-workstation.ks + Workstation-Lite: rocky-live-workstation-lite.ks + XFCE: rocky-live-xfce.ks + KDE: rocky-live-kde.ks + MATE: rocky-live-mate.ks + allowed_arches: + - x86_64 + - aarch64 + required_pkgs: + - 'lorax-lmc-novirt' + - 'vim-minimal' + - 'pykickstart' + - 'git' + variantmap: + git_repo: 'https://git.rockylinux.org/rocky/pungi-rocky.git' + branch: 'r9' + git_raw_path: 'https://git.rockylinux.org/rocky/pungi-rocky/-/raw/r9/' + repoclosure_map: + arches: + x86_64: '--forcearch=x86_64 --arch=x86_64 --arch=athlon --arch=i686 --arch=i586 --arch=i486 --arch=i386 --arch=noarch' + aarch64: '--forcearch=aarch64 --arch=aarch64 --arch=noarch' + ppc64le: '--forcearch=ppc64le --arch=ppc64le --arch=noarch' + s390x: '--forcearch=s390x --arch=s390x --arch=noarch' + repos: + devel: [] + BaseOS: [] + AppStream: + - BaseOS + CRB: + - BaseOS + - AppStream + HighAvailability: + - BaseOS + - AppStream + ResilientStorage: + - BaseOS + - AppStream + RT: + - BaseOS + - AppStream + NFV: + - BaseOS + - AppStream + SAP: + - BaseOS + - AppStream + - HighAvailability + SAPHANA: + - BaseOS + - AppStream + - HighAvailability + extra_files: + git_repo: 'https://git.rockylinux.org/staging/src/rocky-release.git' + git_raw_path: 'https://git.rockylinux.org/staging/src/rocky-release/-/raw/r9/' + branch: 'r9' + gpg: + stable: 'SOURCES/RPM-GPG-KEY-Rocky-9' + testing: 'SOURCES/RPM-GPG-KEY-Rocky-9-Testing' + list: + - 'SOURCES/Contributors' + - 'SOURCES/COMMUNITY-CHARTER' + - 'SOURCES/EULA' + - 'SOURCES/LICENSE' + - 'SOURCES/RPM-GPG-KEY-Rocky-9' + - 'SOURCES/RPM-GPG-KEY-Rocky-9-Testing' +... +``` diff --git a/docs/documentation/references/empanadas_sig_config.md b/docs/documentation/references/empanadas_sig_config.md new file mode 100644 index 0000000..8a7c462 --- /dev/null +++ b/docs/documentation/references/empanadas_sig_config.md @@ -0,0 +1,16 @@ +--- +title: Empanadas SIG yaml Configuration +--- + +Each file in `empanads/sig/` is a yaml file that contains configuration +items for the distribution release version. The configuration determines the +structure of the SIG repositories synced from Peridot or a given repo. + +Note that a release profile (for a major version) is still required for this +sync to work. + +See the items below to see which options are mandatory and optional. + +## Config Items + +## Reference Example diff --git a/docs/sop/sop_release.md b/docs/sop/sop_release.md index e311845..b492c6f 100644 --- a/docs/sop/sop_release.md +++ b/docs/sop/sop_release.md @@ -37,11 +37,20 @@ be queried to verify if mirrors are or appear to be in sync. ## Notes about Patch Days +Within a minimum of one (1) to two (2) days, the following should be true: + +1. Updates should be completed in the build system, and verified in staging. + +2. Updates should be sent to production and file lists updated to allow mirrors +to sync. + ## Prior to Release Day notes +Ensure the SIG/Core Checklist is read thoroughly and executed as listed. + ## Release Day -## Priorities +### Priorities During release day, these should be verified/completed in order: @@ -68,8 +77,63 @@ this should be documented. ### Beta +* Compose Completed +* Repoclosure must be checked and pass +* Lorax Run +* ISO's are built +* Cloud Images built +* Live Images built +* Compose Synced to Staging +* AWS/Azure Images in Marketplace +* Vagrant Images +* Container Images +* Mirror Manager + + * Ready to Migrate from previous beta release (rltype=beta) + * Boot image install migration from previous beta release + +* Pass image to Testing Team for final validation + ### Release Candidate +* Compose Completed +* Repoclosure must be checked and pass +* Lorax Run +* ISO's are built +* Cloud Images built +* Live Images built +* Compose Synced to Staging +* AWS/Azure Images in Marketplace +* Vagrant Images +* Container Images +* Mirror Manager + + * Ready to Migrate from previous release + * Boot image install migration from previous release + +* Pass image to Testing Team for validation + ### Final +* Compose Completed +* Repoclosure must be checked and pass +* Lorax Run +* ISO's are built +* Cloud Images built +* Live Images built +* Compose Synced to Staging +* AWS/Azure Images in Marketplace +* Vagrant Images +* Container Images +* Mirror Manager + + * Ready to Migrate from previous release + * Boot image install migration from previous release + +* Pass image to Testing Team for final validation +* Sync to Production +* Sync to Europe Mirror if applicable +* Hardlink Run +* Bitflip after 24-48 Hours + {% include "resources_bottom.md" %}