$ git clone --depth=1 https://chromium.googlesource.com/infra/infra chrome-infra $ cd chrome-infra/go/src/infra/build/siso $ go build .
A Failed Experiment with Siso
To lead: the post URL was overly optimistic, and the current post title is far more accurate. I do not have Siso fully working, and specifically not the remote execution part that’d make it a distcc replacement as I had hoped. This is being posted so I can share the notes on progress I made. It will hopefully get edited into something with a fully working setup in the future, and then re-published.
Chrome Infra team sent out a PSA that they’re switching from Ninja to Siso. So what’s Siso? It bills itself as a ninja-compatible drop-in replacement which can leverage the Bazel Remote Execution and Remote Caching APIs. All the remote compilation and caching goodness of bazel, but in the easy to use formfactor of a ninja-compatible build tool? Sounds great!
So, let’s take it for a spin.
Building Siso
The setup that one needs to build siso is just:
I ran into[1]: [1]: Filed bug 428926700.
verifying github.com/jdxcode/netrc@v1.0.0/go.mod: checksum mismatch downloaded: h1:Gh9eFQJnoTNIRHXl2j5bJXA1u84hQWJWgGh569zF3v8= go.sum: h1:Zi/ZFkEqFHTm7qkjyNJjaWH4LQA9LQhGJyF0lTYGpxw=
So I just removed the lines for jdxcode/netrc from go.sum. ¯\_(ツ)_/¯
$ go build . $ ls siso siso $ sudo cp $SOMEWHERE_ON_PATH
Success!
On a different build attempt, I ended up with a few more chrome repos which I had to clone into their directories. You’re supposed to acquire all these repos by using depot_tools, but it’s not terrible to do by hand either.
Local Builds
I have a little CMake project that benchmarks a number of different cryptography libraries, and so I’m using that as my test project. No details about what the code or CMakeLists.txt is has mattered yet, so I’m just leaving it as opaque.
Siso can be used via siso ninja <target>. Our first problem is the complaint of a missing config/siso/main.star in the build directory:
$ cmake -S . -B build -G Ninja $ cd build $ siso ninja cryptobench 0.00s Error: can not detect exec_root: config/siso not found $ mkdir -p config/siso $ siso ninja cryptobench 0.01s Error: failed to load @config//main.star: open main.star: no such file or directory
And so we have to write a minimal main.star. You can basically stub out the entire thing[2] and it appears to work:
[2]: Filed bug 428883432.
build/config/siso/main.star
load("@builtin//struct.star", "module")
def init(ctx):
return module(
"config", step_config = '{"rules": []}',
filegroups={}, handlers={})Note that that’s from the source directory. Siso doesn’t care either way if that build is the same as your cmake build output directory. And now we can successfully attempt a local build:
$ siso ninja cryptobench
load build.ninja... panic: runtime error: index out of range [1] with length 1
goroutine 85 [running]:
go.chromium.org/infra/build/siso/toolsupport/ninjautil.(*fileParser).setup(0xc000141808, {0x17f8858, 0xc00088c0c0})
$HOME/ws/siso/build/siso/toolsupport/ninjautil/file_parser.go:285 +0x2b0
go.chromium.org/infra/build/siso/toolsupport/ninjautil.(*fileParser).parseContent(0xc000141808, {0x17f8858, 0xc00088c0c0})
$HOME/ws/siso/build/siso/toolsupport/ninjautil/file_parser.go:165 +0x499
go.chromium.org/infra/build/siso/toolsupport/ninjautil.(*fileParser).parseFile(0xc000141808, {0x17f8890, 0xc0000fc410}, {0x1340e99, 0xb})
$HOME/ws/siso/build/siso/toolsupport/ninjautil/file_parser.go:82 +0x391
go.chromium.org/infra/build/siso/toolsupport/ninjautil.(*ManifestParser).loadFile(0xc000882000, {0x17f8890, 0xc0000fc410}, {0x1340e99, 0xb})
$HOME/ws/siso/build/siso/toolsupport/ninjautil/manifest_parser.go:88 +0x10b
go.chromium.org/infra/build/siso/toolsupport/ninjautil.(*ManifestParser).Load.func1()
$HOME/ws/siso/build/siso/toolsupport/ninjautil/manifest_parser.go:64 +0x27
golang.org/x/sync/errgroup.(*Group).Go.func1()
$HOME/go/pkg/mod/golang.org/x/sync@v0.13.0/errgroup/errgroup.go:79 +0x50
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 1
$HOME/go/pkg/mod/golang.org/x/sync@v0.13.0/errgroup/errgroup.go:76 +0x93
Err… almost.[3] [3]: Filed bug 428883433.
Peeking at file_parser.go:285 it’s code related to include handling, and build.ninja includes CMakeFiles/rules.ninja. So lets work around this by just doing the include ourself by copy-pasting the entire rules.ninja file where the include is in build.ninja.
$ siso ninja cryptobench
build finished
local:10 remote:0 cache:0 fallback:0 retry:0 skip:1
fs: ops: 1686(err:786) / r:785(err:0) 145.15MiB / w:0(err:0) 0B
E0630 14:36:12.557508 1188343 state.go:762] dir:/usr/lib64/llvm20/bin name:/usr/lib64/llvm20 entries:&directory{m:0xc00052bdc8}
E0630 14:36:12.557698 1188343 state.go:762] dir:/usr/lib/gcc/x86_64-redhat-linux/15 name:/usr/lib/gcc/x86_64-redhat-linux entries:&directory{m:0xc010fe5288}
6.28s Build Succeeded: 10 steps - 1.59/s
$ ls cryptobench
cryptobench
It worked!
Remote Builds
I signed up for BuildBuddy, as they were the first hosted remote execution service I found offering a free tier. (EngFlow forces you to go through one intake form, then another intake form, then talk to a human over video chat, so don’t go there.)
BuildBuddy then gives you the configuration for bazel to set up remote execution:
build --bes_results_url=https://app.buildbuddy.io/invocation/ build --bes_backend=grpcs://remote.buildbuddy.io build --remote_timeout=10m build --remote_executor=grpcs://remote.buildbuddy.io build --remote_header=x-buildbuddy-api-key=API_KEY_REDACTED
I think how one is supposed to hook this together is to follow the pattern from a bazel github issue and a siso bug report and take the API key and place it into a "credential helper" script:
siso-credential-helper.sh
#!/bin/bash
# https://www.buildbuddy.io/docs/guide-auth/
apikey="XKnFdxJHJetfq1aKExDh"
# Requires an RFC3339 date.
expires=`date --date='now +6 hours' -Iseconds`
cat <<EOF
{
"headers": {
"x-buildbuddy-api-key": ["${apikey}"]
},
"expires": "${expires}"
}
EOF
And then the invocation looks like:
Fixed by bug 428883434
$ SISO_CREDENTIAL_HELPER=siso-credential-helper.sh siso ninja \
-reapi_address=remote.buildbuddy.io:443 \
-reapi_instance=default \
-strict_remote \
reapi instance: default
0.02s Error: failed to dial remote.buildbuddy.io:443: google: could not find default credentials. See https://cloud.google.com/docs/authentication/external/set-up-adc for more information
So, now we get to set up gcloud credentials for some reason?
$ gcloud init $ gcloud auth application-default login
And new we can try again:
$ SISO_CREDENTIAL_HELPER=siso-credential-helper.sh siso ninja \
-reapi_address=remote.buildbuddy.io:443 \
-reapi_instance=default \
-strict_remote \
build finished
local:10 remote:0 cache:0 fallback:0 retry:0 skip:1
reapi: ops: 2(err:2) / r:0(err:0) 0B / w:0(err:0) 0B
fs: ops: 953(err:424) / r:411(err:0) 43.62MiB / w:0(err:0) 0B
E0630 18:59:09.582292 1210745 state.go:762] dir:/usr/lib64/llvm20/bin name:/usr/lib64/llvm20 entries:&directory{m:0xc011b480b8}
E0630 18:59:09.582423 1210745 state.go:762] dir:/usr/lib/gcc/x86_64-redhat-linux/15 name:/usr/lib/gcc/x86_64-redhat-linux entries:&directory{m:0xc011b0bb98}
0.91s Build Succeeded: 10 steps - 10.94/s
But local:10 remote:0 makes it look like it all executed locally? I had filed bug 428946258, but the problem was actually me not understanding that remote execution has to be opted into, and all rules default to local only. (Sorry siso devs!)
I’m using DuckDB as my large CMake project for testing at this point, so there’s a chance this might be specialized to that accidentally. I finally found the starlark config docs, and what we need to do is define a StepRule which rewrites the ninja CXX rules into remotely executable ones. I’ve managed to get that working via:
main.star
load("@builtin//encoding.star", "json")
load("@builtin//struct.star", "module")
step_config = {
"rules": [
{
"name": "clang++",
"action": "CXX_COMPILER.*",
"remote_command": "clang++",
"input_root_absolute_path": True,
"remote": True,
}
]
}
def init(ctx):
return module("config", step_config = json.encode(step_config),
filegroups={}, handlers={})Trying the siso build again, I instead get an error like:
err: remote-exec failed no-fallback: disable remote: failed to get gcc deps: inputs are not under exec root 1 ["/usr"]...: platform=map["InputRootAbsolutePath":"$HOME/ws/duckdb"]
Which looking into the starlark docs for siso, I see there’s a:
use_system_input: ok to use input outside of exec root
And thus adding
"rules": [
{
"name": "clang++",
"action": "CXX_COMPILER.*",
"remote_command": "clang++",
"input_root_absolute_path": True,
+ "use_system_input": True,
"remote": True,
}Now gives us:
err: remote-exec 8f3bd458a4ce933cc7788422d95e136b0ca5fc4ce20374d602b0944bf59e830c/198 failed: exit=1 ... build step: CXX_COMPILER__duckdb_core_functions_algebraic_unscanned_Release "./extension/core_functions/aggregate/algebraic/CMakeFiles/duckdb_core_functions_algebraic.dir/ub_duckdb_core_functions_algebraic.cpp.o" siso_rule: clang++ stderr: clang-11: error: no such file or directory: '$HOME/ws/duckdb/build/release/extension/core_functions/aggregate/algebraic/ub_duckdb_core_functions_algebraic.cpp' clang-11: error: no input files
And… I’m just going to give up here, as this seems like one would need to do a bunch of path rewriting, and that seems like more of a siso dev problem than I’m willing to take on.