Version 20191212

lib: support arbitrary compressed package files
We need to modify the matching of valid package files to support formats like zstd. Let's try to use an eager approach instead of a simple whitelist in order to be functional for arbitrary formats that may be introduced in the future without the need to adjust any code. Allow any single fragment word as compression type but filter out known non-package content like signature files. Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2025-09-13 09:56:18 +02:00 · 2019-12-12 01:01:44 +01:00 · 2019-12-08 20:59:11 +01:00 · 2019-12-05 23:34:23 +01:00 · 2019-12-05 22:50:11 +01:00 · 2019-11-30 13:44:02 +01:00
4 changed files with 19 additions and 6 deletions
--- a/2
+++ b/2
@@ -1,4 +1,4 @@
-V=20191130
+V=20191212

 PREFIX = /usr/local
 MANDIR = $(PREFIX)/share/man
--- a/lib/common.sh
+++ b/lib/common.sh
@@ -142,7 +142,7 @@ find_cached_package() {
 	for dir in "${searchdirs[@]}"; do
 		[[ -d $dir ]] || continue

-		for pkg in "$dir"/*.pkg.tar?(.?z); do
+		for pkg in "$dir"/*.pkg.tar?(.!(sig|*.*)); do
 			[[ -f $pkg ]] || continue

 			# avoid adding duplicates of the same inode
@@ -152,7 +152,7 @@ find_cached_package() {

 			# split apart package filename into parts
 			pkgbasename=${pkg##*/}
-			pkgbasename=${pkgbasename%.pkg.tar?(.?z)}
+			pkgbasename=${pkgbasename%.pkg.tar*}

 			arch=${pkgbasename##*-}
 			pkgbasename=${pkgbasename%-"$arch"}
--- a/makechrootpkg.in
+++ b/makechrootpkg.in
@@ -387,17 +387,28 @@ if (( ret != 0 )); then
 else
 	if (( run_checkpkg )); then
 		msg "Running checkpkg"
-		remotepkgs=($(pacman -Syddp --logfile /dev/null "${pkgnames[@]}"|grep '://'))
+
+		# sync off-site databases for up-to-date queries
+		trap 'rm -rf $dbpath; cleanup' EXIT INT TERM QUIT
+		dbpath=$(mktemp -d --tmpdir makechrootpkg-database.XXXXXXXXXX)
+		mkdir -p "$dbpath"
+		pacman -Sy --dbpath "$dbpath" --logfile /dev/null
+
+		# query current package locations
+		remotepkgs=($(pacman -Sddp --dbpath "$dbpath" --logfile /dev/null "${pkgnames[@]}"))
 		if (( $? )); then
 			warning "Skipped checkpkg due to missing repo packages"
 			exit 0
 		fi
-		for remotepkg in ${remotepkgs[@]}; do
+
+		# download package files if any non-local location exists
+		for remotepkg in "${remotepkgs[@]}"; do
 			[[ $remotepkg == file://* ]] && continue
 			msg2 "Downloading current versions"
-			pacman --noconfirm -Swdd --logfile /dev/null "${pkgnames[@]}"
+			pacman --noconfirm -Swdd --dbpath "$dbpath" --logfile /dev/null "${pkgnames[@]}"
 			break
 		done
+
 		msg2 "Checking packages"
 		sudo -u "$makepkg_user" checkpkg --rmdir --warn
 	fi
--- a/2
+++ b/2
@@ -116,4 +116,6 @@ if (( ${#files[@]} )); then
    printf '%s\n' '' '-> copying files...'
    load_makepkg_config
    scp "${files[@]/#/$server:}" "${PKGDEST:-${PWD}}/"
+else
+    exit 1
 fi
Author	SHA1	Message	Date
Levente Polyak	d4546c59ce	Version `20191212`	2019-12-12 01:01:44 +01:00
Levente Polyak	d6866e0544	lib: support arbitrary compressed package files We need to modify the matching of valid package files to support formats like zstd. Let's try to use an eager approach instead of a simple whitelist in order to be functional for arbitrary formats that may be introduced in the future without the need to adjust any code. Allow any single fragment word as compression type but filter out known non-package content like signature files. Signed-off-by: Levente Polyak <anthraxx@archlinux.org>	2019-12-08 20:59:11 +01:00
Eli Schwartz	8edad22616	offload-build: return failure if no package files were found This means that the remote command died at some stage earlier than the printing of created files. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>	2019-12-05 23:34:23 +01:00
Levente Polyak	f20435643f	makechrootpkg: sync databases for checkpkg off-site Use pacman's --dbpath feature to sync fresh databases inside an isolated location and split up the database sync and package location calls to remove the need of weird grep calls. It isn't nice of makechrootpkg to modify the host database state just by building packages. No foreign program shall automatically modify the host database other than by the explicit will of a system maintainer, which is the major reason this changes get incorporated. However, there is certain indoctrinated believe that using -Sy is the prime evil. In fact it has been declared as a social rule to a technical problem of not getting into potential partial upgrade states. This is not a proper loophole less solution as there are multiple ways and use cases that lead to such a state, like aborting a -Syu on the prompt for whatever reason, what really matters is that it is not a technically bullet proof solution to solve the problem. Databases shall have the freedom to be as up to date as databases or their owner wishes, allowing querying on latest database state without fear. The only loophole-less contract that _really_ is from importance is always using -Su instead of plain -S to install packages. Installing packages is what actually brings one into a potential partial upgrade state and by using -Su an outstanding upgrade is forced when installing a new package. This properly solves all edge cases in a technical manner instead of declaring people who abort the prompt of -Syu to be the problem. In fact, using this simple contract allows whatever system maintenance workflow a host owner wants to follow, which may still be to always use -Syu and deal with system upgrades explicitly instead of the time when installing new packages, but the -Su contract is the real safe guard to guarantee no edge case can ever slip in. This magically also opens up the freedom to people who wish to use -Sy to simply query on up to date data as the currently indoctrinated "never do -Sy" stone plates not only are not rock solid in technical terms but also make certain use cases simply impossible and hence cripple the functionality without at the very least being fully loophole free. Signed-off-by: Levente Polyak <anthraxx@archlinux.org>	2019-12-05 22:50:11 +01:00
Levente Polyak	8d99df602d	makechrootpkg: double quote array expansions for checkpkg locations Avoid re-splitting remotepkg elements used for checkpkg conditions.	2019-11-30 13:44:02 +01:00