UBI 烧写器如何工作

The following is a list of what the UBI flasher program has to do when erasing the flash or when flashing UBI images.

• First of all, scan the flash and collect the erase counters. Namely, it read the EC header from each PEB, check the CRC-32 checksum of the header, and save the erase counter in a RAM. It is not necessary to read VID headers. Bad PEBs should be skipped.
• Calculate average erase counter. It should be used for PEBs with corrupted or missing EC headers. Such PEBs may be there because of unclean reboots, but there shouldn't be too many of them.
• If the intention is to just erase the flash, then each PEB has to be erased and proper EC header has to be written at the beginning of the PEB. The EC header should contain incremented erase counter. Bad PEBs should be just skipped. For NAND flashes, in case of I/O errors while erasing or writing, the PEB should be marked as bad (see here for more information how UBI marks PEBs as bad).
• If the intention is to flash an UBI image, then the flasher should do the following for each non-bad PEB.
  
  • Read the contents of this PEB from the UBI image (PEB size bytes) into a buffer.
  • Stripe min. I/O units full of 0xFF bytes from the end of the buffer (the details are given below in this section).
  • Erase the PEB.
  • Change the EC header in the buffer - put the new erase counter value there and re-calculate the CRC-32 checksum.
  • Write the buffer to the physical eraseblock.
  As usually, bad PEBs should be just skipped. And for NAND flashes, in case I/O errors while erasing or writing, the PEB should be marked as bad.

下面是 UBI 烧写器在擦除flash或烧写 UBI 映像时要做的工作列表：

• 首先， 扫描 flash 设备以收集擦除计数； 换句话说， 就是读取 EC header， 检查头的 CRC-32， 把擦除计数存储到 RAM 中。 不需要读取 VID header， 坏块也要跳过。
• 计算平均擦除计数。 该值会用于损坏的 PEB 或丢失 EC header的PEB。 
• 如果目的是要烧写 UBI 映像， 烧写器必须对好 PEB做如下工作：
  • 从映像读取本PEB的内容到buffer
  • 把不满最小I/O单元单元的部分用 0xFF 填充
  • 擦除 PEB
  • 改变 Buffer 中的 EC header - 增加擦除计数和重新计算 CRC-32
  • 把buffer中的内容写到 PEB

一般， 坏块只需跳过， 对 NAND flash来说， 为了避免 在擦除或写数据时出现I/O错误， PEB 应该被标记为坏的。

In practice the input UBI image is usually shorter than the flash, so the flasher has to flash the used PEBs properly, and erase the unused PEBs properly.

Note, when writing an UBI image, it does not matter where eraseblocks from the input UBI image will be written. For example, the first input eraseblock may be written to the first PEB, or to the second one, or to the last one.

Also note, if you implement a flasher which writes UBI images at the production line, i.e., only once, then the flasher does not have to change EC headers of the input UBI image, because this is new flash and each PEB has zero erase counter anyway. This means the production line flasher may be simpler.

If your UBI image contains UBIFS file system, and your flash is NAND, you may have to drop 0xFF bytes the end of input PEB data. This is very important, although not required for all NAND flashes. Sometimes a failure to do this may result in very unpleasant problems which might be difficult to debug later. So we recommend to always do this.

The reason for this is that UBIFS treats NAND pages which contain only 0xFF bytes (let's refer them to as empty NAND pages) as free. For example, suppose the first NAND page of a PEB has some data, the second one is empty, the third one also has some data, the fourth one and the rest of NAND pages are empty as well. In this case UBIFS will treat all NAND pages starting from the fourth one as free, and will write data there. However, if the flasher program has already written 0xFF's to these pages, so they will be written to twice! However, many NAND flashes require NAND pages to be written only once, even if the data contains only 0xFF bytes.

实际上， UBI 映像通常比flash小， 烧写器必须恰当的烧写使用的PEB， 擦除未使用的PEB。

注意， 当写 UBI 映像时， 映像的块写到哪个擦除块并无关系； 例如，第一个输入块可能写入第一个 PEB， 也可能第二个， 或最后一个。

还需要注意一点， 如果写了产线上用的烧写器， 那么烧写器不需要修改 EC header， 因为它是新设备， 每个 PEB 的擦除计数都是0， 这意味着产线上的烧写器更简单。

如果你的 UBI 映像 含有 UBIFS 文件系统， 你的 flash 是NAND，  你必须丢掉输入映像中末尾的0xFF填充字节。 这很重要， 尽管不是所有的 NAND flash 都要求这样。 有时， 不这样做会导致非常难以调试的问题， 因此我推荐总是这样做。

原因是 UBIFS 把仅含有 0xFF 的页当作空页， 未使用的空闲页。 例如， 假设PEB的第一个 NAND page 含有一些数据， 而第二个是空的， 第三个也是含有一些数据， 第四个和剩余的也都是空的。 在这种情况下， UBIFS 认为从第四个 page 开始都是空的， 会把新数据写到那。 然而， 如果烧写器已经向这些地方写入了 0xFF， 这就会导致写两次， 但很多 NAND 仅支持写一次， 即使这些数据是0xFF. 

To put it differently, writing 0xFF bytes may have side-effects. What the flasher has to do is to drop all empty NAND pages from the end of the PEB buffer before writing it. It is not necessary to drop all empty NAND pages, just the last ones. This means that the flasher does not have to scan whole buffer for 0xFF's. It is enough to scan the buffer from the end and stop on the first non-0xFF byte. This is much faster. Here is the code from UBI which does the right thing.

写 0xFF 可能含有副作用， 烧写器要做的就是写之前丢掉映像中的空页， 但并不是所有的， 只需丢掉后面的， 这意味着烧写器不必扫描整个buffer中的 0xFF, 有可能那本来就是有用的。

/**
 * calc_data_len - calculate how much real data are stored in a buffer.
 * @ubi: UBI device description object
 * @buf: a buffer with the contents of the physical eraseblock
 * @length: the buffer length
 *
 * This function calculates how much "real data" is stored in @buf and returns
 * the length. Continuous 0xFF bytes at the end of the buffer are not
 * considered as "real data".
 */
int ubi_calc_data_len(const struct ubi_device *ubi, const void *buf,
                      int length)
{
        int i;

        for (i = length - 1; i >= 0; i--)
                if (((const uint8_t *)buf)[i] != 0xFF)
                        break;

        /* The resulting length must be aligned to the minimum flash I/O size */
        length = ALIGN(i + 1, ubi->min_io_size);
        return length;
}

This function is called before writing the buf buffer to the PEB. The purpose of this function is to drop 0xFF's from the end and prevent the situation described above. The ubi->min_io_size is the minimal input/output unit size which is equivalent to NAND page size.

本函数在写buf中的内容到 PEB之前调用， 目的是丢掉末尾的 0xFF， 以避免前述情形的发生。 ubi->min_io_size 是最小I/O单元， 它等价于 NAND page size。