Notes about Porting suspend-to-disk function to android

porting suspend-to-disk

email: niedao516@126.com

1. suspend-to-disk introduce

It is exciting to have the oppotunity  for me to port suspend-to-disk function to Android system. Suspend-to-disk, also called hibernation, is Linux kernel's important feature in PC world,X86 platform. But in embeded Linux/ARM system,it is not formal released yet because of the stability.

After less than 2 weeks of studying and debuging,I get a draft version. Yes, it's only a draft version. I mean the basic hibernation functionality is ready,such as:

1).  android processes freeze

2).  device suspend,

3).  create and store the image to swap partition in disk, then power off fully.

4).  read image from swap and restore android system

5).  operate it normally from both UI and console

Unfortunately, it is unstable. Kernel will reboot somtimes inexplicably, without logs printed.

So, I need to do some notes to sort out my thinking.

2. target platform

cpu: omap4460 blaze tablet

memory: 1-GB DRAM

OS: android 4.0.3, kernel 3.0.31

3. steps to port suspend-to-disk

1). apply  the patch from website for omap4 : http://lists.tuxonice.net/pipermail/tuxonice-devel/2011-April/006740.html

It is the latest patch for omap4460 i can search from website.As Frank Hofmann's mail, you may need adjust the line numbers in arch/arm/mach-omap2/sleep44xx.S.It is the linux kernel mail list,not formal release. Actually, hibernation for ARM is in the development phase. So it is not supported by TI omap4460 blaze talbet bsp.After applying the patch to the kernel source code, you should compile it successfully.

2). format a partition from EMMC/NAND for swap, at least two fifths of your DRAM in size. Of course, the larger, the better. For example, I allocate 1-G byte as swap partition for 1-G RAM.

3). execute "make menuconfig" to add some configurations:

  CONFIG_SWAP  --> add swap partition support for storing image

  CONFIG_HIBERNATION  

             --> enable hibernation configuratinon,it will also other configuration,such as CONFIG_PM_STD_PARTITION,CONFIG_HIBERNATE_CALLBACKS,CONFIG_LZO_COMPRESS,CONFIG_LZO_DECOMPRESS

  CONFIG_PM_STD_PARTITION="/dev/block/mmcblk0p11"   --> specify the swap partition you use

  CONFIG_ARCH_HIBERNATION_POSSIBLE  ---> select automatic by the patch

Attaching the patch: 

commit c3699dea0313e461111b381ea2229024250a1255
Date:   Sun Jun 9 23:06:30 2013 +0800

    [hiber] add configuration,make menuconfig

diff --git a/arch/arm/configs/defconfig b/arch/arm/configs/defconfig
index 11e6d96..0536b19 100644
--- a/arch/arm/configs/defconfig
+++ b/arch/arm/configs/defconfig
@@ -46,7 +46,7 @@ CONFIG_KERNEL_GZIP=y
 # CONFIG_KERNEL_LZMA is not set
 # CONFIG_KERNEL_LZO is not set
 CONFIG_DEFAULT_HOSTNAME="(none)"
-# CONFIG_SWAP is not set
+CONFIG_SWAP=y
 CONFIG_SYSVIPC=y
 CONFIG_SYSVIPC_SYSCTL=y
 # CONFIG_POSIX_MQUEUE is not set
@@ -532,6 +532,9 @@ CONFIG_USER_WAKELOCK=y
 CONFIG_EARLYSUSPEND=y
 # CONFIG_NO_USER_SPACE_SCREEN_ACCESS_CONTROL is not set
 CONFIG_FB_EARLYSUSPEND=y
+CONFIG_HIBERNATE_CALLBACKS=y
+CONFIG_HIBERNATION=y
+CONFIG_PM_STD_PARTITION="/dev/block/mmcblk0p11"
 CONFIG_PM_SLEEP=y
 CONFIG_PM_SLEEP_SMP=y
 CONFIG_PM_RUNTIME=y
@@ -996,6 +999,7 @@ CONFIG_MTD_BLOCK=y
 # CONFIG_SSFDC is not set
 # CONFIG_SM_FTL is not set
 # CONFIG_MTD_OOPS is not set
+# CONFIG_MTD_SWAP is not set
 
 #
 # RAM/ROM/Flash chip drivers
@@ -2914,7 +2918,6 @@ CONFIG_RCU_CPU_STALL_VERBOSE=y
 # CONFIG_CPU_NOTIFIER_ERROR_INJECT is not set
 # CONFIG_FAULT_INJECTION is not set
 CONFIG_SYSCTL_SYSCALL_CHECK=y
-# CONFIG_DEBUG_PAGEALLOC is not set
 CONFIG_HAVE_FUNCTION_TRACER=y
 CONFIG_HAVE_FUNCTION_GRAPH_TRACER=y
 CONFIG_HAVE_DYNAMIC_FTRACE=y
@@ -3079,6 +3082,8 @@ CONFIG_CRC7=m
 CONFIG_LIBCRC32C=y
 CONFIG_ZLIB_INFLATE=y
 CONFIG_ZLIB_DEFLATE=y
+CONFIG_LZO_COMPRESS=y
+CONFIG_LZO_DECOMPRESS=y
 # CONFIG_XZ_DEC is not set
 # CONFIG_XZ_DEC_BCJ is not set
 CONFIG_DECOMPRESS_GZIP=y

4). After you flash the zImage to your target system, input commands as bellow in console to settup a swap partition:

#busybox mkswap /dev/block/mmcblk0p11       (set up a Linux swap area)

Setting up swbapspace version 1, size = 1068478464 bytes

UUID=deaafa20-b54a-4fc1-9c10-2b47c2c98a7

#busybox swapon  /dev/block/mmcblk0p11      (instruct the operating system to use the newly created swap partition)

[   39.259979] Adding 1043436k swap on /dev/block/mmcblk0p11.  Priority:-1 extents:1 across:1043436k SS

5). hibernate system in console

#echo "disk" > /sys/power/state

it will start execute kernel hibernation and the log show as bellow:

[  287.894348] PM: Basic memory bitmaps created
[  287.899230] PM: Syncing filesystems ... done.
[  287.911102] Freezing user space processes ... 
[  287.921295] active wake lock main
[  287.925506] 
[  287.927307] Freezing of user space  aborted
[  287.932708] 
[  287.934570] Restarting tasks ... done.
[  287.944396] PM: Basic memory bitmaps freed

Unfortutory, error happans, system exit hibernation and recovery. The root cause is that Android has wakelock locked when system try to freeze the processes. Wakelock and early suspend are newly imported to android by Google. They are nonexistent in original linux kernel.In Google's design, system can't enter suspend mode (suspend-to-RAM) when there is activate wakelock. So you can need to add early suspend functions to do some operations, for example, to turn off LCM,to disable touch panel, to stop kernel work queue, the most importantly, to release "MAIN" wakelock which created by kernel. Then system will enter really kernel suspend as long as there is no other activate wakelock in the system.

After tracing the kernel source code, you will find the code which report the error:

kernel/power/process.c, in function try_to_freeze_tasks()

read_unlock(&tasklist_lock);

if (!sig_only) {
	wq_busy = freeze_workqueues_busy();
	todo += wq_busy;
}

if (todo && has_wake_lock(WAKE_LOCK_SUSPEND)) {
	wakeup = 1;
	break;
}
		
if (!todo || time_after(jiffies, end_time))
	break;

if (pm_wakeup_pending()) {
	wakeup = true;
	break;
}

In order to verify the hibernation patch immediately, we can just comment the code in red color as a temporary solution. I will show the better solution later in the text.

6). not to suspend console when hibernation, referecne the patch.

diff --git a/kernel/printk.c b/kernel/printk.c
index 2414614..21a5a9a 100644
--- a/kernel/printk.c
+++ b/kernel/printk.c
@@ -1152,7 +1152,7 @@ int update_console_cmdline(char *name, int idx, char *name_new, int idx_new, cha
        return -1;
 }
 
-int console_suspend_enabled = 1;
+int console_suspend_enabled = 0;
 EXPORT_SYMBOL(console_suspend_enabled);
 
 static int __init console_suspend_disable(char *str)

After re-flashing the zImage, by now we can create the hibernation image and save it to swap partition, then power off system.

Great ! 

Hibernation log:

[   49.141235] Freezing user space processes ... (elapsed 0.02 seconds) done.
[   49.170776] Freezing remaining freezable tasks ... (elapsed 0.02 seconds) done.
[   49.205322] PM: Preallocating image memory... done (allocated 101077 pages)
[   49.672821] PM: Allocated 404308 kbytes in 0.45 seconds (898.46 MB/s)
[   49.685211] PVR: PVRSRVDriverSuspend(pDevice=c78a4600)
[   49.690856] PVR: SysSystemPrePowerState: Entering state D3
[   49.696838] PVR: Uninstalling device LISR on IRQ 53 with cookie c704bc00
[   49.704193] PVR: DisableSystemClocks: Disabling System Clocks
[   49.864044] sharpls043: sdp4430_disable_Sharpls043
[   49.874664] sharpls043: sdp4430_disable_Sharpls043 done
[   49.880676] Invalid Device Structure
[   49.884948] omapdss HDMI: Enter hdmi_display_disable
[   49.908905] omapdss HDMI: Release c-state constraint for HDMI
[   49.908905] 
[   49.916961] Invalid Device Structure
[   49.921020] PM: freeze of devices complete after 241.180 msecs
[   49.927825] PM: late freeze of devices complete after 0.427 msecs
[   49.934509] Disabling non-boot CPUs ...
[   49.947662] CPU1: shutdown
[   49.951171] PM: Creating hibernation image:
[   49.951171] PM: Need to copy 65955 pages
[   49.951171] PM: Normal pages needed: 11428 + 1024, available pages: 21322
[   49.951171] PM: Hibernation image created (65955 pages copied)
[   49.951171] rex,hibernation.c create_image,in_suspend=1,error=0
[   49.951354] Enabling non-boot CPUs ...
[   49.973297] CPU1 is up
[   49.976318] PM: early thaw of devices complete after 0.427 msecs
[   49.979919] Switched to NOHz mode on CPU #1
[   49.988006] sharpls043: sharpls043_power_on
[   49.995849] sharpls043: sdp4430_enable_Sharpls043
[   50.011047] sharpls043: sharpls043_hw_reset
[   50.035308] sharpls043: sharpls043_hw_reset done
[   50.324737] sharpls043: sharpls043_power_on done
[   50.330383] PVR: PVRSRVDriverResume(pDevice=c78a4600)
[   50.336151] PVR: SysSystemPostPowerState: Entering state D0
[   50.342620] PVR: EnableSystemClocks: Enabling System Clocks
[   50.348968] PVR: Installing device LISR SGX ISR on IRQ 53 with cookie c704bc00
[   50.371246] in hpd work 1, state=0
[   50.375061] omapdss HDMI: ENTER hdmi_display_enable
[   50.380432] omapdss HDMI: Set c-state constraint for HDMI
[   50.380432] 
[   50.387969] HDCP: invalid AKSV
[   50.391326] omapdss HDMI error: *** INVALID AKSV: 0 Do not perform HDCP AUTHENTICATION
[   50.402984] PM: thaw of devices complete after 415.344 msecs
[   50.409667] PM: writing image.
[   50.413024] PM: Free swap pages: 260858
[   50.424560] PM: Compressing and saving image data (66020 pages) ...   0%
[   50.495605] omapdss DISPC error: timeout waiting for EVSYNC
[   50.5412  1%
[   50.565917] in hpd work 2, state=1
[   50.6703  7%
[   50.940887] in hpd work 3, state=1
[   50.9849 12%
[   51.315887] in hpd work 4, state=1
[   51.3457 18%
[   51.690917] in hpd work 5, state=1
[   51.7684 23%
[   52.065917] in hpd work 6, state=1
[   52.1195 27%
[   52.378692] Failed to read EDID after 5 times. Givingdone
[   57.667938] PM: Wrote 264080 kbytes in 7.23 seconds (36.52 MB/s)
[   57.675445] PM: S|
[   57.800689] PVR: PVRSRVDriverShutdown(pDevice=c78a4600)
[   57.806365] PVR: SysSystemPrePowerState: Entering state D3
[   57.812377] PVR: Uninstalling device LISR on IRQ 53 with cookie c704bc00
[   57.819671] PVR: DisableSystemClocks: Disabling System Clocks
[   57.995574] sharpls043: sdp4430_disable_Sharpls043
[   58.005706] sharpls043: sdp4430_disable_Sharpls043 done
[   58.041809] omapdss HDMI: Enter hdmi_display_disable
[   58.075012] omapdss HDMI: Release c-state constraint for HDMI
[   58.075012] 
[   58.082885] hdmi: clearing EDID info
[   58.086761] Disabling non-boot CPUs ...
[   58.099182] CPU1: shutdown
[   58.102386] Power down.

Press power key to power on system.

restore system from swap file log:

[    5.603759] synaptics_rmi4_i2c 2-0070: Scan PDT
[    5.614349] synaptics_rmi4_i2c 2-0070: fwu_read_f34_queries perm:0, bl0, display:0
[    5.627807] clock: disabling unused clocks to save power
[    5.633697] sr_class1p5_calib_work: core: Calibration complete: Voltage Nominal=1127000Calib=1038160 margin=13000
[    5.645843] rex,hibernate.c software_resume,resume_file=/dev/block/mmcblk0p11
[    5.653778] PM: Checking hibernation image partition /dev/block/mmcblk0p11
[    5.661407] rex,swsusp_resume_device=271581187
[    5.666290] PM: Hibernation image partition 259:3 present
[    5.672271] PM: Looking for hibernation image.
[    5.684326] PM: Image signature found, resuming
[    5.689422] PM: Basic memory bitmaps created
[    5.694274] PM: Preparing processes for restore.
[    5.700592] Freezing user space processes ... (elapsed 0.00 seconds) done.
[    5.708953] Freezing remaining freezable tasks ... 
[    5.722137] sr_class1p5_calib_work: iva Stop sampling: Voltage Nominal=950000 samples=7
[    5.731719] sr_class1p5_calib_work: iva: Calibration complete: Voltage Nominal=950000Calib=860920 margin=13000
[    5.743103] (elapsed 0.02 seconds) done.
[    5.747955] PM: Loading hibernation image.
[    5.753021] sr_class1p5_calib_work: core: Calibration complete: Voltage Nominal=962000Calib=886240 margin=38000
[    5.765258] PM: Loading and decompressing image data (66020 pages) ...   0%
[    5.815917] sr_class1p5_calib_work: mpu Stop sampling: Voltage Nominal=1025000 samples=11
[    5.839477] sr_class1p5_calib_work: mpu: Calibration complete: Voltage Nominal=1025000Calib=848260 margin=10000
[    6.0727  4%
[    6.284576] (stk) :ldisc installation timeout
[    6.289337] (stk) :ldisc_install  5%
[    6.308074] sr_class1p5_calib_work: mpu: Calibration complete: Voltage Nominal=1390000Calib=1329340 margin=0
[    6.3416 15%
[    6.831542] sr_class1p5_calib_work: mpu: Calibration complete: Voltage Nominal=1317000Calib=1114120 margin=0
[    6.8573 26%
[    7.448669] (stk) : timed out waiting for ldisc to be un-insta 29%
[    7.574737] (stk) :ldisc_install 51%
[    8.729919] (stk) :ldisc installation timeout
[    8.735198] (stk) :ldisc_install 71%
[    9.893981] (stk) : timed out waiting for ldisc to be un-insta 73%
[   10.013427] (stk) :ldisc_install 93%
[   11.167419] (stk) :ldisc installation timeout
[   11.172180] (stk) :ldisc_installdone
[   11.620605] PM: Read 264080 kbytes in 5.84 seconds (45.21 MB/s)
[   11.627166] PM: Image successfully loaded
[   11.632598] Invalid Device Structure
[   11.639770] omapdss HDMI: Enter hdmi_display_disable
[   11.675720] omapdss HDMI: Release c-state constraint for HDMI
[   11.675720] 
[   11.683898] PM: quiesce of devices complete after 46.508 msecs
[   11.690704] PM: late quiesce of devices complete after 0.396 msecs
[   11.697448] Disabling non-boot CPUs ...
[   11.702056] CPU1: shutdown
[   49.951171] rex,hibernation.c create_image,in_suspend=1,error=0
[   49.951843] Enabling non-boot CPUs ...
[   49.973754] CPU1 is up
[   49.976776] PM: early thaw of devices complete after 0.427 msecs
[   49.980468] Switched to NOHz mode on CPU #1
[   49.988311] sharpls043: sharpls043_power_on
[   49.996154] sharpls043: sdp4430_enable_Sharpls043
[   50.011383] sharpls043: sharpls043_hw_reset
[   50.035797] sharpls043: sharpls043_hw_reset done
[   50.325225] sharpls043: sharpls043_power_on done
[   50.330718] PVR: PVRSRVDriverResume(pDevice=c78a4600)
[   50.336486] PVR: SysSystemPostPowerState: Entering state D0
[   50.342956] PVR: EnableSystemClocks: Enabling System Clocks
[   50.349304] PVR: Installing device LISR SGX ISR on IRQ 53 with cookie c704bc00
[   50.371765] in hpd work 1, state=0
[   50.375579] omapdss HDMI: ENTER hdmi_display_enable
[   50.380981] omapdss HDMI: Set c-state constraint for HDMI
[   50.381011] 
[   50.388549] HDCP: invalid AKSV
[   50.391937] omapdss HDMI error: *** INVALID AKSV: 0 Do not perform HDCP AUTHENTICATION
[   50.403503] PM: thaw of devices complete after 415.466 msecs
[   50.410217] PM: writing image.
[   50.413574] PM: Free swap pages: 260858
[   50.419677] PM: Compressing and saving image data (0 pages) ... done
[   50.427459] PM: Wrote 0 kbytes in 0.01 seconds (0.00 MB/s)
[   50.434387] PM: S|
[   50.496093] omapdss DISPC error: timeout waiting for EVSYNC
[   50.558624] in hpd work 2, state=1
[   50.562744] PVR: PVRSRVDriverShutdown(pDevice=c78a4600)
[   50.568450] PVR: SysSystemPrePowerState: Entering state D3
[   50.574432] PVR: Uninstalling device LISR on IRQ 53 with cookie c704bc00
[   50.581756] PVR: DisableSystemClocks: Disabling System Clocks
[   50.769531] sharpls043: sdp4430_disable_Sharpls043
[   50.779663] sharpls043: sdp4430_disable_Sharpls043 done
[   50.879211] omapdss HDMI: Enter hdmi_display_disable
[   50.902191] omapdss HDMI: Release c-state constraint for HDMI
[   50.902191] 
[   50.910064] hdmi: clearing EDID info
[   50.913970] Disabling non-boot CPUs ...
[   50.926361] CPU1: shutdown
[   50.929595] Power down.

Now we can restore the image from swap partition, but there is something wrong: System power down,no familiar android UI !!

What's happan?

7).  The variable "in_suspend" should be in "no_savedata" section of the zImage's ELF format.

You can check the System.map, "in_suspend" is out of "no_savedata" section.

c0899b20 D __start___jump_table
c0899b20 D __start___verbose
c0899b20 D __stop___jump_table
c0899b20 D __stop___verbose
c0899b20 D _edata
c0899b20 A _edata_loc
c0899b20 D in_suspend
c089a000 A __nosave_begin
c089a000 A __nosave_end
c089a000 T __start_notes
c089a024 A __bss_start
c089a024 T __stop_notes

patch file:

diff --git a/include/linux/init.h b/include/linux/init.h
index 9146f39..3654327 100644
--- a/include/linux/init.h
+++ b/include/linux/init.h
@@ -308,7 +308,7 @@ void __init parse_early_options(char *cmdline);
 #endif
 
 /* Data marked not to be saved by software suspend */
-#define __nosavedata __section(.data..nosave)
+#define __nosavedata __section(.data.nosave)
 
 /* This means "can be init if no module support, otherwise module load
    may call it." */

Ohh,goodness, Android UI appear !!!

8). debug for stablitiy 

   continued....

4. reference

1. android wakelock: http://blog.csdn.net/lhf_tiger/article/details/7325301

2. suspend-to-disk patch for omap3 and omap4: http://lists.tuxonice.net/pipermail/tuxonice-devel/2011-April/006740.htm

3. android kernel debug hibernation and suspend: http://www.verydemo.com/demo_c146_i22714.html

4. suspend to disk for arm: http://elinux.org/Suspend_To_Disk_For_ARM

5. android suspend/resume function analysis: http://hi.baidu.com/embeded/item/f5d60c1bc87778f787ad4ea1

6. suspend-to-disk; http://en.opensuse.org/SDB:Suspend_to_disk

7. supend-to-disk for arm patch based-on kernel 3.7.0 omap3: https://patchwork.kernel.org/patch/1743731/

8. suspend-to-disk for arm patch from huffman based-on 2.6.37 omap3430 and smdk6450: http://lists.linux-foundation.org/pipermail/linux-pm/2011-February/030207.html