Atom feed of PRs adding/removing translations to/from po/*.po

(This is experimental, and our concrete need to automate this is not
super high, but I want to have a solution for this problem in general.)

There are various automated ways to notify people of changes to a
component, without them having to set up complicated filtering.
For example
1. using GitHub's codeowners feature, perhaps creating a GitHub team
   per component
2. by requiring all changes to the component to go through a fork
3. Using [public-inbox](https://public-inbox.org/)'s "lei" tool,
   which makes it easy to create a filter for all patches touching a given file
   (consumable as pull-based mbox or RSS feed), see
   https://people.kernel.org/monsieuricon/lore-lei-part-1-getting-started

Option 3 seems ideal because it allows anyone to subscribe/unsubscribe,
and it remains obvious where to send PRs to.

It would be fairly easy to write a client-side tool (fetch all commits
and/or PRs and look for interesting changes) but people are more likely
to use something that works OOTB with newsreaders / mail clients.

Expose atom feeds of PRs proposing translation additions/removals.
The one with removals is higher traffic because it can be triggered
when rewording a source string.

Feeds are already online. For example, use:

	https://raw.githubusercontent.com/fish-shell/fish-shell/fish-shell-events/po/zh_CN-additions.atom
	https://raw.githubusercontent.com/fish-shell/fish-shell/fish-shell-events/po/zh_CN-removals.atom

Ref: https://github.com/fish-shell/fish-shell/pull/11833#discussion_r2394161766

.github/workflows/update-event-feeds.yml

@@ -0,0 +1,56 @@
+on:
+  pull_request_target:
+    types: [opened, synchronize]
+  workflow_dispatch:
+    inputs:
+      pr_number:
+        description: Pull request number
+        required: true
+        type: number
+
+permissions:
+  contents: write
+
+jobs:
+  update-event-feeds:
+    runs-on: ubuntu-latest
+    steps:
+    - name: Install dependencies
+      run: |
+        sudo apt install python3-git python3-polib
+        sudo pip3 install --break-system-packages feedgen==1.0.0
+    - name: Update event feeds
+      env:
+        GITHUB_TOKEN: ${{ github.token }}
+      run: |
+        set -x
+        if [ "${{ github.event_name }}" = pull_request ]; then
+          pr_number=${{ github.event.pull_request.number }}
+        else
+          pr_number=${{ inputs.pr_number }}
+        fi
+        # E.g. https://github.com/fish-shell/fish-shell
+        repo_url=${{ github.server_url }}/${{ github.repository }}
+        workflow_sha=${{ github.workflow_sha }}
+        run_id=${{ github.run_id }}
+        events_ref=refs/fish-shell-events
+        git clone "$repo_url" --revision="$events_ref" worktree --depth=1
+        cd worktree
+        git fetch origin "$workflow_sha" --depth=1
+        set -o pipefail
+        git show "$workflow_sha":build_tools/event-feeds.py |
+          python - "$pr_number"
+        git add .
+        if git diff --cached --exit-code; then
+          exit
+        fi
+        git config user.name fish-shell-bot
+        git config user.email fish-shell-bot
+        git commit -m "update via github run ID $run_id"
+        cat >>.git/config <<'EOF'
+        [credential]
+          username = fish-shell-bot
+          helper = !sh -c 'echo "password=$GITHUB_TOKEN"'
+        EOF
+        # Might fail due to a race; that's usually fine to ignore...
+        git push origin HEAD:"$events_ref" ||:

build_tools/event-feeds.py

@@ -0,0 +1,155 @@
+from datetime import datetime
+from pathlib import Path
+from typing import Tuple
+import dateutil.tz
+import json
+import re
+import os
+import subprocess
+import sys
+import xml.etree.ElementTree as ET
+
+# TODO maybe use https://github.com/getpelican/feedgenerator instead?
+from feedgen.feed import FeedGenerator
+import git
+import polib
+
+
+def message_dictionary(repo, ref, file: Path):
+    po_contents = repo.git().show(f"{ref}:{file}")
+    messages = polib.pofile(po_contents)
+    return {e.msgid: e.msgstr for e in messages}
+
+
+def write_feed(
+    feed_path: Path,
+    base_ref_oid: str,
+    title: str,
+    subtitle: str,
+    this_pr: Tuple[str, str] | None,
+):
+    fg = FeedGenerator()
+    tree_url = (
+        "https://raw.githubusercontent.com/fish-shell/fish-shell/fish-shell-events"
+    )
+    feed_url = f"{tree_url}/{feed_path}"
+    fg.id(feed_url)
+    fg.title(title)
+    fg.subtitle(subtitle)
+    fg.link(href=feed_url, rel="self")
+    generator_url = f"https://github.com/fish-shell/fish-shell/blob/{base_ref_oid}/.github/workflows/update-event-feeds.yml"
+    fg.link(href=generator_url, rel="alternate")
+
+    entries = {}
+    if feed_path.exists():
+        tree = ET.parse(feed_path)
+        root = tree.getroot()
+        xmlns = "{http://www.w3.org/2005/Atom}"
+        for child in root.findall(f"{xmlns}entry"):
+            pr_url = child.find(f"{xmlns}id").text
+            pr_title = child.find(f"{xmlns}title").text
+            updated = child.find(f"{xmlns}updated").text
+            updated = datetime.fromisoformat(updated)
+            entries[pr_url] = (pr_title, updated)
+    if this_pr is not None:
+        (this_pr_title, this_pr_url) = this_pr
+        if this_pr_url not in entries:
+            entries[this_pr_url] = (this_pr_title, datetime.now(dateutil.tz.tzutc()))
+    for pr_url, (pr_title, updated) in entries.items():
+        fe = fg.add_entry()
+        fe.id(pr_url)
+        fe.title(pr_title)
+        fe.updated(updated)
+        fe.link(href=pr_url)
+    atomfeed = fg.atom_str(pretty=True).decode()
+    feed_path.write_text(atomfeed)
+
+
+def pull_request_data(pr_number: str):
+    data = json.loads(
+        subprocess.check_output(
+            (
+                "gh",
+                "pr",
+                "view",
+                "--json",
+                "baseRefOid,headRefOid,title,url",
+                str(pr_number),
+            ),
+            text=True,
+        )
+    )
+    base_ref_oid = data["baseRefOid"]
+    head_ref_oid = data["headRefOid"]
+    this_pr_title = data["title"]
+    this_pr_url = data["url"]
+    return (base_ref_oid, head_ref_oid, this_pr_title, this_pr_url)
+
+
+def main():
+    pr_number = sys.argv[1]
+
+    (base_ref_oid, head_ref_oid, this_pr_title, this_pr_url) = pull_request_data(
+        pr_number
+    )
+
+    repo = git.Repo()
+    repo.git().execute(("git", "fetch", "origin", base_ref_oid, head_ref_oid))
+    po_files = (
+        Path(file)
+        for file in repo.git()
+        .execute(("git", "ls-tree", "--name-only", base_ref_oid, ":/po/"))
+        .splitlines()
+    )
+    changed_files = {
+        Path(file)
+        for file in subprocess.check_output(
+            ("git", "diff", "--name-only", base_ref_oid, head_ref_oid), text=True
+        ).splitlines()
+        if re.match(r"^po/\w+\.po$", file)
+    }
+    Path("po").mkdir(exist_ok=True)
+    for po_file in po_files:
+        file_without_extension = str(po_file).removesuffix(".po")
+        ll_CC = os.path.basename(file_without_extension)
+        feed_additions_path = Path(f"{file_without_extension}-additions.atom")
+        feed_removals_path = Path(f"{file_without_extension}-removals.atom")
+
+        def title(verb: str):
+            return f"fish-shell PRs {verb} {ll_CC} translations"
+
+        def write_additions(this_pr):
+            subtitle = "PRs that add new translations"
+            write_feed(
+                feed_additions_path, base_ref_oid, title("adding"), subtitle, this_pr
+            )
+
+        def write_removals(this_pr):
+            subtitle = "PRs that remove translations, often due to a change in the messages source"
+            write_feed(
+                feed_removals_path, base_ref_oid, title("removing"), subtitle, this_pr
+            )
+
+        if not feed_additions_path.exists():
+            write_additions(None)
+        if not feed_removals_path.exists():
+            write_removals(None)
+
+        if po_file not in changed_files:
+            continue
+        old = message_dictionary(repo, base_ref_oid, po_file)
+        new = message_dictionary(repo, head_ref_oid, po_file)
+        if any(
+            new_msgstr != old.get(new_msgid) and new_msgstr != ""
+            for (new_msgid, new_msgstr) in new.items()
+        ):
+            write_additions((this_pr_title, this_pr_url))
+        elif any(
+            old_msgstr != "" and new.get(old_msgid, "") == ""
+            for (old_msgid, old_msgstr) in old.items()
+        ):
+            write_removals((this_pr_title, this_pr_url))
+
+
+if __name__ == "__main__":
+    main()

build_tools/update_translations.fish

@@ -101,10 +101,14 @@ end
 # This is used to identify lines which should be set here via $header_lines.
 # Make sure that this prefix does not appear elsewhere in the file and only contains characters
 # without special meaning in a sed pattern.
-set -g header_prefix "# fish-note-sections: "
+set -g header_prefix "# fish-note: "
 
-function print_header
+function print_header -a po_file
+    set -l ll_CC (basename $po_file .po)
     set -l header_lines \
+        "To subscribe to proposed updates, point your feed reader at:" \
+        https://raw.githubusercontent.com/fish-shell/fish-shell/fish-shell-events/po/$ll_CC-additions.atom \
+        https://raw.githubusercontent.com/fish-shell/fish-shell/fish-shell-events/po/$ll_CC-removals.atom \
         "Translations are divided into sections, each starting with a fish-section-* pseudo-message." \
         "The first few sections are more important." \
         "Ignore the tier3 sections unless you have a lot of time."
@@ -123,7 +127,7 @@ function merge_po_files --argument-names template_file po_file
     or cleanup_exit
 
     begin
-        print_header
+        print_header $po_file
         # Paste PO file without old header lines.
         sed '/^'$header_prefix'/d' $new_po_file
     end >$po_file
@@ -139,7 +143,7 @@ for po_file in $po_files
             merge_po_files $template_file $po_file
         else
             begin
-                print_header
+                print_header $po_file
                 cat $template_file
             end >$po_file
         end

po/de.po

@@ -1,6 +1,9 @@
-# fish-note-sections: Translations are divided into sections, each starting with a fish-section-* pseudo-message.
-# fish-note-sections: The first few sections are more important.
-# fish-note-sections: Ignore the tier3 sections unless you have a lot of time.
+# fish-note: To subscribe to proposed updates, point your feed reader at:
+# fish-note: https://raw.githubusercontent.com/fish-shell/fish-shell/fish-shell-events/po/de-additions.atom
+# fish-note: https://raw.githubusercontent.com/fish-shell/fish-shell/fish-shell-events/po/de-removals.atom
+# fish-note: Translations are divided into sections, each starting with a fish-section-* pseudo-message.
+# fish-note: The first few sections are more important.
+# fish-note: Ignore the tier3 sections unless you have a lot of time.
 # translation of de.po to deutsch
 # This file is distributed under the same license as the PACKAGE package.
 # Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER.

po/en.po

@@ -1,6 +1,9 @@
-# fish-note-sections: Translations are divided into sections, each starting with a fish-section-* pseudo-message.
-# fish-note-sections: The first few sections are more important.
-# fish-note-sections: Ignore the tier3 sections unless you have a lot of time.
+# fish-note: To subscribe to proposed updates, point your feed reader at:
+# fish-note: https://raw.githubusercontent.com/fish-shell/fish-shell/fish-shell-events/po/en-additions.atom
+# fish-note: https://raw.githubusercontent.com/fish-shell/fish-shell/fish-shell-events/po/en-removals.atom
+# fish-note: Translations are divided into sections, each starting with a fish-section-* pseudo-message.
+# fish-note: The first few sections are more important.
+# fish-note: Ignore the tier3 sections unless you have a lot of time.
 # SOME DESCRIPTIVE TITLE.
 # Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
 # This file is distributed under the same license as the PACKAGE package.

po/fr.po

@@ -1,6 +1,9 @@
-# fish-note-sections: Translations are divided into sections, each starting with a fish-section-* pseudo-message.
-# fish-note-sections: The first few sections are more important.
-# fish-note-sections: Ignore the tier3 sections unless you have a lot of time.
+# fish-note: To subscribe to proposed updates, point your feed reader at:
+# fish-note: https://raw.githubusercontent.com/fish-shell/fish-shell/fish-shell-events/po/fr-additions.atom
+# fish-note: https://raw.githubusercontent.com/fish-shell/fish-shell/fish-shell-events/po/fr-removals.atom
+# fish-note: Translations are divided into sections, each starting with a fish-section-* pseudo-message.
+# fish-note: The first few sections are more important.
+# fish-note: Ignore the tier3 sections unless you have a lot of time.
 # translation of fr.po to Français
 # FRENCH TRANSLATION FOR FISH
 # Copyright (C) 2006 Xavier Douville

po/pl.po

@@ -1,6 +1,9 @@
-# fish-note-sections: Translations are divided into sections, each starting with a fish-section-* pseudo-message.
-# fish-note-sections: The first few sections are more important.
-# fish-note-sections: Ignore the tier3 sections unless you have a lot of time.
+# fish-note: To subscribe to proposed updates, point your feed reader at:
+# fish-note: https://raw.githubusercontent.com/fish-shell/fish-shell/fish-shell-events/po/pl-additions.atom
+# fish-note: https://raw.githubusercontent.com/fish-shell/fish-shell/fish-shell-events/po/pl-removals.atom
+# fish-note: Translations are divided into sections, each starting with a fish-section-* pseudo-message.
+# fish-note: The first few sections are more important.
+# fish-note: Ignore the tier3 sections unless you have a lot of time.
 # Translation of fish in Polish
 # This file is distributed under the same license as the fish package.
 # Author: m4sk1n <m4sk1n@vivaldi.net>

po/pt_BR.po

@@ -1,6 +1,9 @@
-# fish-note-sections: Translations are divided into sections, each starting with a fish-section-* pseudo-message.
-# fish-note-sections: The first few sections are more important.
-# fish-note-sections: Ignore the tier3 sections unless you have a lot of time.
+# fish-note: To subscribe to proposed updates, point your feed reader at:
+# fish-note: https://raw.githubusercontent.com/fish-shell/fish-shell/fish-shell-events/po/pt_BR-additions.atom
+# fish-note: https://raw.githubusercontent.com/fish-shell/fish-shell/fish-shell-events/po/pt_BR-removals.atom
+# fish-note: Translations are divided into sections, each starting with a fish-section-* pseudo-message.
+# fish-note: The first few sections are more important.
+# fish-note: Ignore the tier3 sections unless you have a lot of time.
 # Portuguese translations for fish package.
 # Copyright (C) 2014 THE fish'S COPYRIGHT HOLDER
 # This file is distributed under the same license as the fish package.

po/sv.po

@@ -1,6 +1,9 @@
-# fish-note-sections: Translations are divided into sections, each starting with a fish-section-* pseudo-message.
-# fish-note-sections: The first few sections are more important.
-# fish-note-sections: Ignore the tier3 sections unless you have a lot of time.
+# fish-note: To subscribe to proposed updates, point your feed reader at:
+# fish-note: https://raw.githubusercontent.com/fish-shell/fish-shell/fish-shell-events/po/sv-additions.atom
+# fish-note: https://raw.githubusercontent.com/fish-shell/fish-shell/fish-shell-events/po/sv-removals.atom
+# fish-note: Translations are divided into sections, each starting with a fish-section-* pseudo-message.
+# fish-note: The first few sections are more important.
+# fish-note: Ignore the tier3 sections unless you have a lot of time.
 # Copyright © 2006
 # This file is distributed under the same license as the fish package.
 # Axel Liljencrantz <liljencrantz@gmail.com>, 2006

po/zh_CN.po

@@ -1,6 +1,9 @@
-# fish-note-sections: Translations are divided into sections, each starting with a fish-section-* pseudo-message.
-# fish-note-sections: The first few sections are more important.
-# fish-note-sections: Ignore the tier3 sections unless you have a lot of time.
+# fish-note: To subscribe to proposed updates, point your feed reader at:
+# fish-note: https://raw.githubusercontent.com/fish-shell/fish-shell/fish-shell-events/po/zh_CN-additions.atom
+# fish-note: https://raw.githubusercontent.com/fish-shell/fish-shell/fish-shell-events/po/zh_CN-removals.atom
+# fish-note: Translations are divided into sections, each starting with a fish-section-* pseudo-message.
+# fish-note: The first few sections are more important.
+# fish-note: Ignore the tier3 sections unless you have a lot of time.
 #
 # General Notices 凡例
 # abort: 中止