From 787bc6d0e9d172f6ae1df7a436e04b445ac0c85b Mon Sep 17 00:00:00 2001 From: Louis Abel Date: Wed, 5 Apr 2023 01:10:02 -0700 Subject: [PATCH] update compose info --- docs/documentation/compose/.pages | 5 ++ docs/documentation/compose/koji.md | 5 ++ docs/sop/.pages | 1 + docs/sop/sop_compose.md | 4 +- docs/sop/sop_compose_sig.md | 73 ++++++++++++++++++++++++++++++ 5 files changed, 86 insertions(+), 2 deletions(-) create mode 100644 docs/documentation/compose/.pages create mode 100644 docs/documentation/compose/koji.md create mode 100644 docs/sop/sop_compose_sig.md diff --git a/docs/documentation/compose/.pages b/docs/documentation/compose/.pages new file mode 100644 index 0000000..0918373 --- /dev/null +++ b/docs/documentation/compose/.pages @@ -0,0 +1,5 @@ +--- +nav: + - ... | index.md + - Manual Updates in Koji: koji.md +... diff --git a/docs/documentation/compose/koji.md b/docs/documentation/compose/koji.md new file mode 100644 index 0000000..a0c4d12 --- /dev/null +++ b/docs/documentation/compose/koji.md @@ -0,0 +1,5 @@ +--- +title: Updates and Management in Koji, A Manual +--- + +More to come. diff --git a/docs/sop/.pages b/docs/sop/.pages index 9139ca0..9e0df47 100644 --- a/docs/sop/.pages +++ b/docs/sop/.pages @@ -4,6 +4,7 @@ nav: - Release Procedures: sop_release.md - Compose and Repo Sync for Rocky Linux 8: sop_compose_8.md - Compose and Repo Sync for Rocky Linux and Peridot: sop_compose.md + - Compose and Repo Sync for Rocky Linux Special Interest Groups: sop_compose_sig.md - Generalized Prep Checklist for Upcoming Releases: sop_upstream_prep_checklist.md - Mirror Manager: sop_mirrormanager2.md ... diff --git a/docs/sop/sop_compose.md b/docs/sop/sop_compose.md index 86ad048..07da957 100644 --- a/docs/sop/sop_compose.md +++ b/docs/sop/sop_compose.md @@ -81,7 +81,7 @@ poetry run finalize_compose --release X Syncing utilizes the sync scripts provided in the release engineering toolkit. -When the scripts are being ran, they are usually ran with a specific purpose or a reason. As such, the correct scripts should be ran. +When the scripts are being ran, they are usually ran with a specific purpose, as each major version may be different. The below are common vars files. common_X will override what's in common. Typically these set what repositories exist and how they are named or look at the top level. These also set the current major.minor release as necessary. @@ -108,7 +108,7 @@ These are for the releases in general. What they do is noted below. ├── sync-to-staging-sig.sh -> Syncs a sig provided compose to staging ``` -Generally, you will only run `sync-to-staging.sh` or `sync-to-staging.delete.sh` to sync. The former is for older releases, the latter is for newer releases. +Generally, you will only run `sync-to-staging.sh` or `sync-to-staging.delete.sh` to sync. The former is for older releases, the latter is for newer releases. Optionally, if you are syncing a "beta" or "lookahead" release, you will need to also provide the `RLREL` variable as `beta` or `lookahead`. ``` # The below syncs to staging for Rocky Linux 8 diff --git a/docs/sop/sop_compose_sig.md b/docs/sop/sop_compose_sig.md new file mode 100644 index 0000000..42dadc0 --- /dev/null +++ b/docs/sop/sop_compose_sig.md @@ -0,0 +1,73 @@ +--- +title: 'SOP: Compose and Repo Sync for Rocky Linux Special Interest Groups' +--- + +This SOP covers how the Rocky Linux Release Engineering Team handles composes and repository syncs for Special Interest Groups. + +## Contact Information +| | | +| - | - | +| **Owner** | Release Engineering Team | +| **Email Contact** | releng@rockylinux.org | +| **Email Contact** | infrastructure@rockylinux.org | +| **Mattermost Contacts** | `@label` `@mustafa` `@neil` `@tgo` | +| **Mattermost Channels** | `~Development` | + +## Composing Repositories + +### Mount Structure + +There is a designated system that takes care of composing repositories. These systems contain the necessary EFS/NFS mounts for the staging and production repositories as well as composes. + +* `/mnt/compose` -> Compose data +* `/mnt/repos-staging` -> Staging +* `/mnt/repos-production` -> Production + +### Empanadas + +Each repository or set of repositories are controlled by various comps and pungi configurations that are translated into peridot. Empanadas is used to run a reposync from peridot's yumrepofs repositories, generate ISO's, and create a pungi compose look-a-like. Because of this, the comps and pungi-rocky configuration is not referenced with empanadas. + +### Running a Compose + +First, the toolkit must be cloned. In the `iso/empanadas` directory, run `poetry install`. You'll then have access to the various commands needed: + +* `sync_sig` + +To perform a compose of a SIG, it must be defined in the configuration. As an example, here is composing the `core` sig. + +``` +# This creates a brand new directory under /mnt/compose/X and symlinks it to latest-SIG-Y-X +~/.local/bin/poetry run sync_sig --release 9 --sig core --hashed --clean-old-packages --full-run + +# This assumes the directories already exist and will update in place. +~/.local/bin/poetry run sync_sig --release 9 --sig core --hashed --clean-old-packages +``` + +## Syncing Composes + +Syncing utilizes the sync scripts provided in the release engineering toolkit. + +When the scripts are being ran, they are usually ran with a specific purpose, as each major version may be different. + +For SIG's, the only files you'll need to know of are `sync-to-staging-sig.sh` and `sync-to-prod-sig.sh`. The former syncs a specific sig over to staging. As of this writing, the latter syncs everything in staging to production. **Both scripts will delete packages and data that are no longer in the compose.** + +``` +# The below syncs the core 8 repos to staging +RLVER=8 bash sync-to-staging-sig.sh core +# The below syncs the core 9 repos to staging +RLVER=9 bash sync-to-staging-sig.sh core + +# The below syncs everything in staging for 8 to prod +RLVER=8 bash sync-to-prod-sig.sh + +# The below syncs everything in staging for 9 to prod +RLVER=9 bash sync-to-prod-sig.sh +``` + +Once staging is completed and reviewed, it is synced to production. + +``` +bash sync-file-list-parallel.sh +``` + +During this phase, staging is rsynced with production, the file list is updated, and the full time list is also updated to allow mirrors to know that the repositories have been updated and that they can sync.