Re: [PATCH 3/4] fdtfactor: Support compressed DTB inputs and outputs

[Simon Glass]

Hi Doug,

On 2026-10-15T18:21:33, Douglas Anderson <dianders@xxxxxxxxxxxx> wrote:
> fdtfactor: Support compressed DTB inputs and outputs
>
> Firmware and FIT image build pipelines frequently store device tree
> blobs compressed with LZ4, LZMA/XZ, or gzip (e.g., `.dtb.lz4`).
>
> Extend `fdtfactor` to detect compressed input blobs by magic bytes or
> file extension, transparently decompress them before factoring, and
> compress the resulting `.dtbo` and `base.dtb` outputs when appropriate:
>
> - Support `.lz4` (via the `lz4` CLI), `.lzma`/`.xz` (via Python's
>   `lzma` module), and `.gz` (via Python's `gzip` module).
> - Preserve the input compression format on generated `.dtbo` files and
>   compress `base.dtb` when its output path carries a compression suffix
>   (e.g., `-b base.dtb.lz4`).
> - Report compressed size savings alongside uncompressed savings when
>   `-v`/`--verbose` is specified.
>
> On the 47 Qualcomm SC7180 Trogdor Chromebook `.dtb.lz4` files,
> `fdtfactor -S` reduces total LZ4-compressed size from 1.71 MB to 213 KB
> (87.5% savings).
> [...]
>
>  README.md            |  16 +--
>  fdtfactor            | 267 +++++++++++++++++++++++++++++++++++++++++++++------
>  tests/Makefile.tests |   5 +-
>  tests/run_tests.sh   |  34 ++++++-
>  4 files changed, 284 insertions(+), 38 deletions(-)

David is leaving this patch until patch 2 has been reworked, so these
are just two things to carry into the next version.

> diff --git a/fdtfactor b/fdtfactor
> @@ -11,31 +11,195 @@ original device tree.
> +LZ4_MAGIC = b'\x04\x22\x4d\x18'
> +LZ4_LEGACY_MAGIC = b'\x02\x21\x4c\x18'

Legacy lz4 input is detected here, but compress_blob() always runs
'lz4 -12 -z', which writes the frame format. So a legacy-format
board.dtb.lz4 comes out as a frame-format board.dtbo.lz4. I checked:
'lz4 -l -9' output starts with 02 21 4c 18, while the compress_blob()
command gives 04 22 4d 18. The kernel's cmd_lz4 uses 'lz4 -l', and
lib/decompress_unlz4.c only accepts the legacy format. So a consumer
that handles only one of the two formats cannot read the result. This
doesn't match 'Preserve the input compression format' in the commit
message. Please can you track legacy as its own compression type (and
pass -l when compressing), or else reject legacy input?

> +def decompress_blob(raw_bytes, filename='', comp=None):
...
> +    if comp == 'lz4':
> +        try:
> +            res = subprocess.run(
> +                ['lz4', '-d', '-q', '-c'],
> +                input=raw_bytes,
> +                capture_output=True,
> +                check=False,
> +            )
> +        except FileNotFoundError as exc:
> +            raise RuntimeError(
> +                'lz4 command not found; install lz4 to process .lz4 files'
> +            ) from exc

compress_blob() repeats this block almost exactly, so it should be in
a helper. If the helper takes the filename, compression errors can
also say which file failed.

Regards,
Simon