In this post, we will be going to talk all about the Android OTA Updates. We’ll understand what is an Android OTA Update? How these updates are carried out?, What are the contents of an update package?, How Android OTA update package is created? etc etc etc. So read this post till the end.
First of all lets understand what is an Android OTA update?
Our Android phones are currently running on some Android versions like Android 10, or Pie or Oreo. The OEM(Original equipment manufacturer like Samsung, Nokia, Google, etc) doesn’t just stop there after releasing a phone. They keep on updating their Android OS with some new features or if there are any existing bugs found they fix them as well. Now to roll out these new features and a bug fix to all the users, they release an OTA(Over-the-Air) update.
OTA updates are the popular way of updating android devices over the internet. The Android devices can receive and install this over-the-air (OTA) updates to the system and application software. But how these updates are actually applied on Android devices? Before we dive in, let’s see what are the different types of Android OTA update mechanism?
At the moment, Android supports two different update mechanisms:
- Non-A/B system updates
- A/B (Seamless) system updates
Non-A/B system updates
This update mechanism was basically used in older devices and is also used in some of the devices which have limited flash size. In Non-A/B system, the device has only one copy of each partition. So How this update happens?
First the device checks for new updates by regularly checking in with the OTA server. Once the OTA update is available, the update package is downloaded to /cache or /data partition. Then its cryptographic signature is verified against the certificates which are already present in the device at location /system/etc/security/otacerts.zip. This check is necessary because no user wants to download an untrusted OTA. This is for security purposes.
Once the download and verification are completed the update is initiated using the RecoverySystem API. This is a system API and requires the caller to have android.permission.RECOVERY permission which is not granted to any regular application.
RecoverySystem.installPackage() API is called for applying the update. If the update package is in /cache, it setups the BCB(Bootloader control block) command with some parameters such as update package file location, locale info(for displaying the text in the appropriate language), and security(if its a security update). After that, the device reboots into recovery.
If the update package is under /data partition, it needs to be uncrypted. Uncrypt is a binary that takes a file name and creates a block map of it. This block map can then be used to read the content of the file without mounting the file system. This is done so that recovery can access the update package without mounting the /data partition, since recovery partition is not supposed to change the content of it.
Once the device boots into recovery, the recovery also verifies the cryptographic signature of the package against the public key /res/keys (part of the RAM disk contained in the recovery partition). After verification the /boot, /system and/or /vendor partitions are updated.
But how about updating the recovery partition? You may ask.
Well, it is not updated at this point.
What exactly happens is, when the system partition is updated, the new /system partition contains the content of the new Recovery partition as well. So when the device boots up using the new /boot and /system partitions, Android looks for that file in the new /system partition and update the recovery partition using that patch.
Now at this point system update is complete and update logs can be found at /cache/recovery/last_log.
Types of Non-A/B update
1. File-based Updates :
Update each file on a filesystem level. The problem with that approach is that depending on when the update is actually applied, system partition will have files with inconsistent last modified date information. Because of this, file-based OTA updates cannot be used when dm-verity is enabled, since dm-verity attempts to calculate SHA256 of each block and compares them to the expected values. Due to inconsistent last modified dates, this check will fail and the system will not be able to boot after the update.
2. Block-based Updates :
In block OTA the entire partition is considered as a single file and this results in the generation of a single patch for an entire partition. As a result, the device will have a far more consistent system partition.
Block-based OTA uses the following two mechanisms:
- Full update: The update will have a full copy of the system image, boot image, and other images.
- Incremental update: The update will have the diff of the current image and the new image.
Android OTA update package size comparision between File-based and Block-based
A/B (Seamless) system updates
It is the advanced version of the Non-A/B update. The problem with the Non-A/B update was it has the potential to brick the system. As updates are directly applied to each partition, the device may be left with an unbootable system if anything goes wrong.
So how do you ensure you can still have a working system even if something goes terribly wrong?
It’s simple, by storing two copies of each partition.
A/B updates are introduced from the Android N. A/B system keeps a duplicate copy of /boot, /system, /vendor, and other vendor-specific partitions. /recovery partition is no longer used since it was very similar to /boot partition in the first place, /boot partition now contains recovery ramdisk. Keeping a duplicate copy of each partition will definitely take up the storage memory by 2x, but it worth it.
A/B system contains A/B partition called slots(normally slot A and slot B) such as system_a/system_b. Your current slot is the active and bootable slot and the update is applied to the inactive slot. So if anything goes wrong during the update, You can still continue to run the old OS.
OTA updates can be applied while the system is still running, as the update will be applied on the inactive slot. A/B system uses updated_engine to apply updates. Once the update is applied, the device will be rebooted and the inactive slot will be marked as active and bootable. If the device boots successfully, the slot will be marked as successful. And from then, the device will boot from that slot only.
If the device is not able to boot after some attempts, the bootloader will mark that slot as unbootable and marks your old, untouched slot as active/bootable slot again.
update_engine! What the hell it is?
update_engine is a background daemon used by A/B system for the updates process. It reads the update package from the current slot(either A or B) and writes it to the other slot. Once the update is applied, it marks the other slot as active and bootable, so that from next time the device will boot from that slot only. OTA update can instruct the update_engine to run a post_install program. We will talk about the post_install in a later section.
You can see the update_engine source from system/update_engine.
Life of an A/B update
Once the update package(refered as payload in code) is available to download, the update process begins. This process can be hampered with policies such as battery level, user activity, charging state etc. As the update happens in background it can be interrupted with these policies or unexpected reboot or user action.
|1||Marks the current slot(let say slot A) or source slot as successful with markBootSuccessful() API.|
|2||Marks the target slot(slot B) as unbootable with setSlotAsUnbootable().|
|4||Update payload consists of metadata and extra data. The metadata consists of the list of operations. In this step, all these operations are applied and its associated memory is then discarded.|
|5||Re-read the whole partition and then verified against an expected cryptographic signature (SHA256).|
|6||Runs the post_install step (if any). If anything goes wrong during the update process, the update will fail and it is re-attempted with different payload.|
|7||Marks the target slot or unused slot as active by calling setActiveBootSlot(). This is done so that system will boot from that slot onwards.|
|8||If Post-installation is defined in the OTA package, then it has to be executed and the program must return with exit code 0; otherwise, the update fails.|
|9||Marks the target slot as the current slot with markBootSuccessful() API, after the system successfully boots with a new slot.|
Note: Steps 3 and 4 take most of the update time as they involve writing and downloading large amounts of data, and are likely to be interrupted for reasons of policy or reboot.
Every partition has a post_install step defined. Update engine has the responsibility to mounts the new partition into a specific location and executes the program specified in the OTA relative to the mounted partition.
For example, if the post-install program is defined as usr/bin/postinstall in the system partition, this partition from the unused slot will be mounted in a fixed location (such as /postinstall_mount) and the /postinstall_mount/usr/bin/postinstall command is executed.
Streaming A/B Updates
A/B systems also support streaming updates which can allow your android system to apply the updates as they are downloaded. Since full package doesn’t need to be downloaded, you don’t need the extra space that is allocated for a /cache partition, hence no cache partition is needed at all.
Full OTA update and Incremental OTA update
Full OTA update:
As the name suggests, it contains the full-size partitions which simply rewrite the old one.
Incremental update packages are generated by using two target_files (previous version and new version). The delta of those target files creates a patch. These patches are then applied during the update mechanism. Since incremental updates are specifically generated for certain versions, they can only be used to update from one specific version to another. Because of that, even a slight change in the current partitions on the device would result in incremental updates to fail since the partition hash is calculated and compared with the expected value during updates.(If you somehow root your device and edit your system partition by remounting it, you can say bye-bye to incremental updates).
Generating Android OTA Update Packages
OTA packages are all generated from target_files .zip files. These packages contain all partitions in them. ota_from_target_files.py is used to generate an OTA package from a target file. –block option is specified in order to generate a block-based OTA update, default is file-based. You can generate incremental updates by specifying -i option and providing the script two target_files zips.
OTA package Content
OTA package or update package is a .zip file(ota_update.zip, incremental_ota_update.zip) that contains each partition (or patches of them for incremental updates). It also contains a file called otacerts which is used for package verification (against /system/etc/security/otacerts.zip).
Other than that, there are two more files called updater-script and updater-binary.These file are located at META-INF/com/google/android/.
Updater binary simply executes commands defined in updater-script in order to perform the update.
The updater-script uses an extensible scripting language called edify(which is a single expression language).
Updater-script may vary depending on how the OTA package is generated, but generally it contains the following:
- Build date comparison (For full OTA updates, so an older update cannot be applied over a newer system).
- Extra property checks such as ro.product.device, ro.build.fingerprint, etc.
- Cryptographic hash comparisons (For Incremental Updates).
- Commands for extracting files, applying the update chunks.
- Prints for debugging (usually logged to serial debug lines).
Android OTA updates are designed to upgrade your underlying operating system, the read-only apps installed on the system partition, and/or time zone rules; these updates do not affect applications installed by the user from Google Play.