[RFE] Setup vault syncing and/or X.Y version schemes for SIG content #14

New Issue

label · 2023-11-22T03:12:01Z

label commented

2023-11-22 03:12:01 +00:00

Special Interest Groups are traditionally setup to be rolling and were never meant to take into account prior versions of Rocky Linux. However, there are very few edge cases where having a "transition period" such as with EPEL 10 may help some SIG's in this regard. This would involve the currently available content to be in the vault upon a new minor release or the existence of X.Y directories with a symlink of X to the latest. Both of their benefits and their drawbacks.

Regardless of the option, this could allow the release package(s) of the SIG's to provide an easy-to-use script to immediately pin versions for them. Either option does not require changes to Peridot.

What currently exists

The toolkit contains various bash scripts that handle the rsync from compose, to staging, to production. These scripts source in common variables that dictate how variables are set and how they are used. All SIG content is separated by major version.

What can be done

Option 1: Vault Only

A vault section can be made. A full rsync from prod can be synced at release time to a new X.Y directory in the vault, thus not disturbing the current "X" only structure.

What are the benefits?

Reduces the need to make changes to mirror manager every six (6) months
Prevents overcrowding in /pub/sig (we have it already in /pub/rocky)
Allows users to modify their repo files to point directly to the vault as some do already with Rocky Linux itself
- This is typically done by modifying the baseurl.
- SIG's can potentially provide a quick script to pin versions quickly
Based on SIG's discretion, when they have updated content that matches/works with the newly released minor version of Rocky Linux X, it can be synced and the vaulted content will not be disturbed. Or there is a potential for a SIG's new content to be released at the same time as the minor release.
The essence of "rolling" is still there in /pub/sig, and /vault/sig/X.Y is considered a snapshot that will never update or change

What would need to be done?

This would just need to be added to the release playbook in mattermost for SIG/Core
A script created to handle this in the meantime
Operation maintained in AWX when available

What are the drawbacks?

Please comment or discuss in mattermost of what would be the drawbacks to this method.

Other information

See this commit for the current example script. This effectively makes a "snapshot" by doing an rsync from /pub/sig/X -> /vault/sig/X.Y and a user would simply modify the repo file to point to the desired location.

Option 2: X.Y directories + Symlink

X.Y directories can be maintained/managed like they are with Rocky Linux, with X symlinking to the latest. This would almost match what happens in /pub/rocky for the core distribution and the identity of "rolling" for the SIGs can be maintained.

The primary caveats to this method:

Older Rocky Linux content is typically vaulted within 24 hours of a release. The same policy would likely apply here.
Mirror manager may not be friendly during its daily scans, which can affect how it answers queries from dnf. This can be very user-unfriendly
Additional overhead: a fully copy (rsync) or redeployment of all SIG repos would be required, similar to Rocky Linux itself. 24-hour vaulting would not make this worth the cycles.

Special Interest Groups are traditionally setup to be rolling and were never meant to take into account prior versions of Rocky Linux. However, there are very few edge cases where having a "transition period" such as with EPEL 10 may help some SIG's in this regard. This would involve the currently available content to be in the vault upon a new minor release or the existence of X.Y directories with a symlink of X to the latest. Both of their benefits and their drawbacks. Regardless of the option, this could allow the release package(s) of the SIG's to provide an easy-to-use script to immediately pin versions for them. Either option does *not* require changes to Peridot. ## What currently exists The toolkit contains various bash scripts that handle the rsync from compose, to staging, to production. These scripts source in common variables that dictate how variables are set and how they are used. All SIG content is separated by major version. ## What can be done ### Option 1: Vault Only A vault section can be made. A full rsync from prod can be synced at release time to a new X.Y directory in the vault, thus not disturbing the current "X" only structure. #### What are the benefits? * Reduces the need to make changes to mirror manager every six (6) months * Prevents overcrowding in `/pub/sig` (we have it already in `/pub/rocky`) * Allows users to modify their repo files to point directly to the vault as some do already with Rocky Linux itself * This is typically done by modifying the baseurl. * SIG's can potentially provide a quick script to pin versions quickly * Based on SIG's discretion, when they have updated content that matches/works with the newly released minor version of Rocky Linux X, it can be synced and the vaulted content will not be disturbed. Or there is a potential for a SIG's new content to be released at the same time as the minor release. * The essence of "rolling" is still there in `/pub/sig`, and `/vault/sig/X.Y` is considered a snapshot that will never update or change #### What would need to be done? * This would just need to be added to the release playbook in mattermost for SIG/Core * A [script created](https://git.resf.org/sig_core/toolkit/commit/b394a4d74ee4a5220cf901f97b775eadb1303bde) to handle this in the meantime * Operation maintained in AWX when available #### What are the drawbacks? Please comment or discuss in mattermost of what would be the drawbacks to this method. #### Other information See [this commit](https://git.resf.org/sig_core/toolkit/commit/b394a4d74ee4a5220cf901f97b775eadb1303bde) for the current example script. This effectively makes a "snapshot" by doing an rsync from `/pub/sig/X -> /vault/sig/X.Y` and a user would simply modify the repo file to point to the desired location. ### Option 2: X.Y directories + Symlink X.Y directories can be maintained/managed like they are with Rocky Linux, with X symlinking to the latest. This would *almost* match what happens in `/pub/rocky` for the core distribution and the identity of "rolling" for the SIGs can be maintained. The primary caveats to this method: * Older Rocky Linux content is typically vaulted within 24 hours of a release. The same policy would likely apply here. * Mirror manager may not be friendly during its daily scans, which can affect how it answers queries from dnf. This can be very user-unfriendly * Additional overhead: a fully copy (rsync) or redeployment of all SIG repos would be required, similar to Rocky Linux itself. 24-hour vaulting would not make this worth the cycles.

label added the due date 2024-05-01

2023-11-22 08:09:26 +00:00

Sign in to join this conversation.