CUB3D

CLADE, SLAYED

Intro

CLADE is a hardware-backed compression algorithm introduced by Qualcomm to replace their older Q6ZIP and DELTA compression formats, it is somewhat misunderstood and totally undocumented. In modern Qualcomm modem images, this is the primary method of compressing both code and data and extracting it is essential to obtaining a fully examinable firmware.

We'll cover what parameters you need to decompress CLADE sections, how to find them in a firmware, and provide a full reimplementation of the CLADE decompression algorithm that can be used for decompression without any Hexagon SDK access. I'll not be coving the second version of this format "CLADE2".

CLADE parameters

To work with CLADE compressed data you need the following: As it is a dictionary-based compression format you need to find the three dictionaries, which are specific to the data being decompressed. These are each 0x2000 bytes long (or 0x800 words if you fancy), with a total size of 0x6000 bytes. Each "line" in the CLADE stream specifies the dictionary that should be used to decompress that line.

Data compressed with CLADE is broken up into a series of "Program Directories", which are sub-divided into four regions:

The compressed region
The exception_high region
The exception_low region
The exception_low_small region

When a line is decompressed, if the LSB is set then that region is decompressed from the "exception" region. Either "exception_high" if the line is in the "high" region or "exception_low" if the region is located in the "low" region. A line is either a "high" or "low" line, based on its address. A "high" line is one read from in the first 0x4000000 bytes (64M) of the compressed region, and a "low" line is any line read from an address greater than this.

How this works in practise, is the CLADE hardware unit is configured very early during QuRT startup. The unit is provided with the physical address of these dictionaries and regions. When a CLADE compressed address is accessed, the TLB handles translation of the requested virtual address. If the resulting physical address is the physical address of the CLADE output region, the CLADE engine will decompress the requested data on the fly to serve the read request. The CLADE engine uses the requested address to determine weather a "high" or "low" line is requested.

Finding CLADE params in modem images

Some of the above CLADE parameters are located within a TLV structure that is parsed during early QuRT init. To find that structure, I usually recommend looking for the function that turns on the CLADE engine:

uint clade_on(void) {
  puVar2 = clade_tag_5_body;
  if (QURTK_clade_cfg_base == NULL) {
    panic("Kernel Error: invalid QURTK_clade_cfg_base !!!\n");
  }
  *(uint *)QURTK_clade_cfg_base = (uint)clade_tag_5_body[4] >> 0x1d;
  if ((int *)(QURTK_clade_cfg_base + 0x10) == NULL) {
    panic("Invalid Clade address!!!\n");
  }
  *(uint *)(QURTK_clade_cfg_base + 0x10) = ((uint)*puVar2 >> 0x10) << 0xc;
  if ((int *)(QURTK_clade_cfg_base + 0x14) == NULL) {
    panic("Invalid Clade address!!!\n");
  }
  *(uint *)(QURTK_clade_cfg_base + 0x14) = ((uint)puVar2[1] >> 0xc) << 8;
  if ((uint *)(QURTK_clade_cfg_base + 0x18) == NULL) {
    panic("Invalid Clade address!!!\n");
  }
  *(uint *)(QURTK_clade_cfg_base + 0x18) = puVar2[2] - puVar2[3] & 0xffffff;
  if ((int *)(QURTK_clade_cfg_base + 0x1c) == NULL) {
    panic("Invalid Clade address!!!\n");
  }
  *(uint *)(QURTK_clade_cfg_base + 0x1c) = ((uint)puVar2[3] >> 0xc) << 8;
  uVar1 = SYSCFG;
  SYSCFG = uVar1 | 0x100000;
  instruction_sync();
  return SYSCFG;
}

You can typically find this by looking for some of the strings within. This function first validates the values located in the TLV with TAG=5 (more on that later) then sets bit 20 in SYSCFG which I believe is the chicken bit to enable the CLADE engine. Looking up the call stack from this, you should find this snippit in one of the callers:

  sz = 0x10;
  tlv = &clade_tlvs[0].val;
  do {
    tlv_ = tlv;
    if (ADJ(tlv)->tag == 5) break;
    tlv = (clade_tlv_ptr_4_dword)((int)&ADJ(tlv)->val + (sz & 0xffff));
    sz = (uint)(short)ADJ(tlv).size;
    tlv_ = clade_tag_5_body;
  } while (sz != 0);
    clade_tag_5_body = tlv_;

This walks the TLVs till it finds the TLV with TAG=5 and saves it in the global used in clade_on. The data in clade_tlvs is a sequence of:

struct TLV {
    /// Size of TLV, including size field and tag
    size: u16,
    /// Tag value
    tag: u16,
    /// data
    data: [u8; size-4]
}

The end of the list is indicated with a TLV with size == 0 && tag == 0. The tags I'm aware of are:

0x03 -> Unknown:

data: [u32; 3]

0x04 -> CLADE dictionary:

dict_address: u32
unknown: u32,
dict_size: u32

0x05 -> CLADE base/config address:

data: [u32; 5]

0x10 -> Firmware Name / Id:

data: [u8; size] # string

0x11 -> Unknown:

data: [u32; 4]

0x12 -> Unknown:

data: [u32; 3]

0x13 -> CLADE exc_hi address:

address: u32

My ghidra plugin contains a script mark_clade_tlvs.py that can annotate these for you.

You might find that there are more or less than three dictionaries, I'm not sure what the cause of this is yet. But I've found that the addresses can always be rearranged to give linear 0x2000 byte chunks. e.g. from the pixel 5 - March 2023 firmware, the dictionary address are:

CF5E8000
CF5E9000
CF5EA000
CF5EB000
CF5EC000
CF5ED000 Each of these have a size of 0x1000 so can be combined to give:
0xCF5E8000
0xCF5EA000
0xCF5EC000

For the exception_high value, you can either find it in the TLV with tag=0x13, but this is not always present. Another place it can be found is via a structure used by the "memdump" subsystem. You can find the function of interest, by looking for relevant strings:

void memdump(void) {
  // <snip>
  if (comp_sec_c859fd50.field2_0x8 != NULL) {
    print(0,1,"Memdump overlay Start VA=0x%08x");
    comp_sec_c859fd50.low =
         (void *)((int)comp_sec_c859fd50.low + (int)comp_sec_c859fd50.field4_0x10);
    print(0,1,"Memdump overlay modified Start VA=0x%08x");
    print(0,1,"Memdump overlay End VA=0x%08x");
    comp_sec_c859fd50.hi = (void *)((uint)comp_sec_c859fd50.hi & 0xfff00000);
    print(0,1,"Memdump overlay modified End VA=0x%08x");
    **(int **)(unaff_GP + 0x6d00) = (int)comp_sec_c859fd50.hi - (int)comp_sec_c859fd50.low;
    print(0,1,"Memdump overlay size=0x%08x");
    print(0,2,"Memdump memory map VA start: 0x%08X  end_va_map_fw_dump_overlay end: 0x%08X");
    uVar1 = FUN_c059d750(comp_sec_c859fd50.low);
    **(undefined4 **)(unaff_GP + 0xb8f8) = uVar1;
    print(0,1,"Memdump address lookup PA=0x%08x",uVar1);
    if (**(int **)(unaff_GP + 0xb8f8) == 0) {
      print(0,0,"Error: Could not get the start PA for overlay section");
      return;
    }
    **(uint **)(unaff_GP + 0xb8f8) = **(int **)(unaff_GP + 0xb8f8) + 0xfffffU & 0xfff00000;
    print(0,1,"Memdump overlay start PA=0x%08x");
    print(0,2,"Memdump creating memory map PA 0x%08X, size = 0x%08X");
    iVar2 = FUN_c0777140(comp_sec_c859fd50.field2_0x8,**(undefined4 **)(unaff_GP + 0xb8f8),
                         **(undefined4 **)(unaff_GP + 0x6d00),7,3);
    if (iVar2 != 0) {
      print(0,1,"Error: memory map fail :%x");
      return;
    }
    print(0,2,"Memdump overwrite address start : 0x%08X end: 0x%08X",
          **(undefined4 **)(unaff_GP + 0x6d00));
  }
}

The comp_sec_c859fd50 struct has the format:

struct comp_sec {
    clade_comp_address: u32,
    clade_high_address: u32,
    // <snip>
}

Which gives both the "compressed" and "clade_exception_high" addresses.

To find the size of the low section, find the following function via its string:

void integrity_check(void) {
    int iVar1;
    brkpt();
    brkpt();
    if (verify_data != NULL) {
        FUN_c07f831c(&DAT_ca1b851c,"Integrity Check",0x80,0x10301);
        iVar1 = thunk_FUN_c05ea034(0xc05ea740);
        if (iVar1 == 0) {
            print(0,0,"unable to register with crash handler");
            return;
        }
    }
    return;
}

The structure of verify_data is:

struct verify_data {
    unknown0: u32, # == 1
    unknown1: u32, # == 0
    address_of_clade_low_region: u32,
    unknown2: [u32; 7], # all == 0
    length: u32,
    // <snip>
}

This gives us the length.

Decompressing

Prior work has been done on decompressing these sections, namely this research. However, this code requires the proprietary Hexagon SDK, which also requires running under Linux x86-64 due to libclade.so only being available for that architecture. Also, this code only decompresses one of the two regions, which is missing the code for e.g. QMI handler functions. I'd like to avoid that, so I've created a reimplementation of the CLADE algorithm that lets it run on any platform and avoids libclade.so. To start, we need to create a config file with our parameters:

dictionaries = [
    0xCF5E8000,
    0xCF5EA000,
    0xCF5Ec000,
]

program_directory = [
    { comp = 0xCD100000, exc_hi = 0xcf2e0000, exc_lo = 0xcf2e0000, exc_lo_small = 0 }
]

size = 0x21DF2C0
size_lo = 0x10f0000

This is the config for the March 2023 Pixel 5 modem as mentioned above.

You will need to concatenate the modem sections into a 4GiB flat file, e.g using this code:

let f = std::fs::read("/path/to/modem/elf.elf")?;
let x = goblin::Object::parse(&f)?;
let x = match x {
    Object::Elf(e) => e,
    _ => unreachable!(),
};

let mut data = vec![0u8; 0xFFFF_FFFF];
for s in x.program_headers {
    if s.p_filesz > 0 {
        let file_data = &f[s.p_offset as usize..][..s.p_filesz as usize];
        data[s.p_vaddr as usize..][..s.p_filesz as usize].copy_from_slice(file_data);
    }
}
std::fs::write("./flat.bin", data)?;

Finally, to decompress the sections using CLADE_rs:

cargo r --release -- /path/to/flat/binary.bin /path/to/config.toml

You can find CLADE_rs here.

CLADE, SLAYED

Contents

Intro

CLADE parameters

Finding CLADE params in modem images

Decompressing