fix(build): check the submodules of parallel builds one at a time (#29178)

* fix(build): check the submodules of parallel builds one at a time

Each configure runs Tools/check_submodules.sh, which fetches the missing submodules. When two builds configured at the same time in one checkout, from two terminals, an IDE and a terminal, or a script building several targets, both fetched the same submodules and git failed in one of them, for example with "could not lock config file .git/config: File exists" and "Failed to register url for submodule path", which stopped that build.

The check now holds a lock in the git directory, which the worktrees of a repository share, as they share its submodule configuration. A build that configures while another one checks the submodules waits for it, then finds them fetched.

Assisted-by: Claude:opus-5.5
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>

* fix(build): check the submodules without the lock when it can't be taken

A configure failed when the submodule lock couldn't be taken: when the git directory is read-only, such as an out-of-tree build from a read-only mount, or after the 30 minutes of the timeout, where it succeeded before the lock. It now warns, naming the lock file, and checks the submodules without the lock. As file(LOCK) fails the configure when it can't create its file, whatever its RESULT_VARIABLE, the lock file is created first with cmake -E touch. With GIT_SUBMODULES_ARE_EVIL set, where check_submodules.sh exits at once, as in every slot of build_all, no lock is taken.

Assisted-by: Claude:opus-5.5
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>

---------

Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
This commit is contained in:
Benjamin Cabé authored and GitHub committed 2026-10-29 18:25:14 -06:00
1 parent 64f85effd2
commit 1bdbb4fbf0
1 file changed
+34 -1
+34 -1
View File
@@ -38,12 +38,45 @@ endif(px4_git_included)
set(px4_git_included true)
# Check all submodules once per configure, whether the build started from
# make, an IDE or CMake directly.
# make, an IDE or CMake directly. Builds that configure at the same time check
# them one at a time: git can't fetch submodules twice at once.
execute_process(
COMMAND git rev-parse --git-common-dir
WORKING_DIRECTORY ${PX4_SOURCE_DIR}
OUTPUT_VARIABLE px4_git_dir
OUTPUT_STRIP_TRAILING_WHITESPACE
ERROR_QUIET
)
# No lock where check_submodules.sh exits at once (GIT_SUBMODULES_ARE_EVIL). A
# lock that can't be taken, such as in a read-only git directory, or after the
# timeout, only warns: the check then runs without it, as before the lock.
set(px4_submodules_lock)
if(px4_git_dir AND "$ENV{GIT_SUBMODULES_ARE_EVIL}" STREQUAL "")
get_filename_component(px4_git_dir ${px4_git_dir} ABSOLUTE BASE_DIR ${PX4_SOURCE_DIR})
set(px4_submodules_lock ${px4_git_dir}/px4_submodules.lock)
# file(LOCK) fails the configure when it can't create the lock file, whatever its RESULT_VARIABLE
execute_process(
COMMAND ${CMAKE_COMMAND} -E touch ${px4_submodules_lock}
RESULT_VARIABLE px4_submodules_lock_result
OUTPUT_QUIET
ERROR_QUIET
)
if(px4_submodules_lock_result EQUAL 0)
file(LOCK ${px4_submodules_lock} GUARD FILE TIMEOUT 1800 RESULT_VARIABLE px4_submodules_lock_result)
endif()
if(NOT px4_submodules_lock_result STREQUAL "0")
message(WARNING "Could not lock ${px4_submodules_lock} (${px4_submodules_lock_result}): checking the submodules without the lock")
set(px4_submodules_lock)
endif()
endif()
execute_process(
COMMAND Tools/check_submodules.sh ${PX4_GIT_SUBMODULES}
WORKING_DIRECTORY ${PX4_SOURCE_DIR}
RESULT_VARIABLE check_submodules_result
)
if(px4_submodules_lock)
file(LOCK ${px4_submodules_lock} RELEASE)
endif()
if(NOT check_submodules_result EQUAL 0)
message(FATAL_ERROR "Tools/check_submodules.sh failed")
endif()