Kernel panic on RSB3730 adv_wdt

Hello,
I’m trying to modify the linux kernel for the RSB3730 board. For now, I am starting with a “stock” kernel, on which I made NO MODIFICATIONS.

I am starting from branch adv_5.4.70_2.3.0_rm02 , and compiling with the main defconfig imx_v8_adv_defconfig.

The kernel builds just fine and an Image and dtb are made. However, I get the following kernel panic:

[    4.317297] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: adv_wdt_i2c_probe+0x254/0x340
[    4.327866] CPU: 1 PID: 33 Comm: kworker/1:1 Not tainted 5.4.70 #2
[    4.334057] Hardware name: Advantech RISC Single Board (DT)
[    4.339698] Workqueue: events deferred_probe_work_func
[    4.344858] Call trace:
[    4.347334]  dump_backtrace+0x0/0x150
[    4.351020]  show_stack+0x14/0x20
[    4.354365]  dump_stack+0xb4/0x11c
[    4.357793]  panic+0x158/0x334
[    4.360854]  __stack_chk_fail+0x1c/0x20
[    4.364700]  adv_wdt_i2c_probe+0x254/0x340
[    4.368825]  i2c_device_probe+0x284/0x310
[    4.372847]  really_probe+0xdc/0x420
[    4.376430]  driver_probe_device+0x54/0xf0
[    4.380531]  __device_attach_driver+0xa4/0x100
[    4.385006]  bus_for_each_drv+0x74/0xd0
[    4.388849]  __device_attach+0xc4/0x170
[    4.392700]  device_initial_probe+0x10/0x20
[    4.396888]  bus_probe_device+0x90/0xa0
[    4.400744]  device_add+0x318/0x660
[    4.404246]  device_register+0x1c/0x30
[    4.408005]  i2c_new_client_device+0x14c/0x2b0
[    4.412461]  i2c_new_device+0xc/0x20
[    4.416063]  of_i2c_register_devices+0xf4/0x188
[    4.420601]  i2c_register_adapter+0x14c/0x3f0
[    4.424968]  __i2c_add_numbered_adapter+0x58/0xa0
[    4.429684]  i2c_add_adapter+0x9c/0xd0
[    4.433444]  i2c_add_numbered_adapter+0x24/0x30
[    4.437991]  i2c_imx_probe+0x378/0x880
[    4.441757]  platform_drv_probe+0x50/0xb0
[    4.445780]  really_probe+0xdc/0x420
[    4.449362]  driver_probe_device+0x54/0xf0
[    4.453472]  __device_attach_driver+0xa4/0x100
[    4.457922]  bus_for_each_drv+0x74/0xd0
[    4.461771]  __device_attach+0xc4/0x170
[    4.465619]  device_initial_probe+0x10/0x20
[    4.469813]  bus_probe_device+0x90/0xa0
[    4.473654]  deferred_probe_work_func+0x64/0xa0
[    4.478217]  process_one_work+0x1c0/0x350
[    4.482233]  worker_thread+0x200/0x450
[    4.486006]  kthread+0x148/0x150
[    4.489261]  ret_from_fork+0x10/0x24
[    4.492890] SMP: stopping secondary CPUs
[    4.497491] Kernel Offset: disabled
[    4.501000] CPU features: 0x0002,2000200c
[    4.505022] Memory Limit: none
[    4.508123] ---[ end Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: adv_wdt_i2c_probe+0x254/0x340 ]---

Using gdb, I can see that the instruction pointer mentioned in the panic can be found at drivers/watchdog/watchdog_advantech.c:496:

        ret = adv_wdt_i2c_read_version(client, &tmp_version);

        if (ret == 0 )
        {
     adv_wdt.version[0]= (tmp_version & 0xFF00) >> 8; // HERE it panics
     adv_wdt.version[1]= tmp_version & 0xFF;
     adv_wdt_info.firmware_version = (unsigned int)(adv_wdt.version[1] - '0') * 10 + (unsigned int)(adv_wdt.version[0] - '0');
        } else {
                pr_err("Read watchdog version err=%d\n", ret);
                goto fail;
        }

Am I doing something wrong? I haven’t touched neither the source code, nor the configuration file.

Thank you,
Rick

Hi~

Could you provide which the Yocto BSP version do you used? And steps how to build the image?

For build the image, it can reference OS released tag version and the Yocto BSP user guide below:
OS release note:
RSB-3730 Release Note - ESS-WIKI (advantech.com.tw)
Yocto 3.0 BSP user guide
Yocto Linux BSP Ver.A User Guide for iMX8 series -Yocto 3.0 - ESS-WIKI (advantech.com.tw)

Hi Jems,
I’m using the pre-built image that I found here: https://www.dropbox.com/scl/fo/gts9kel7q39ahgww6v88h/h?dl=0&e=1&preview=3730A2AIM30LIVA1045_iMX8MM_4G_2023-05-06.img.gz&rlkey=qp1umab5whthnmv9232n16vu1

I’m trying to build bootloader and kernel manually, just to add a few things and see if they work, so I can send feedback to the client without having to spend the time and effort to build the Yocto image.

I’m using these source files:

Kernel : https://github.com/ADVANTECH-Corp/linux-imx/commit/f76ed69da02bc698685996105089ef77aa753817

The .config file that I’m using: .config

U-Boot : https://github.com/ADVANTECH-Corp/uboot-imx/commit/501984c9bad3bb78e59473a79e177a53fc7a93b8

And the .config file: .config

I compile without errors, and U-Boot works fine and loads the kernel, again, with no errors.

Then the kernel just panics for the reason shown above.

Do you see something wrong with my config files?

Thank you,
Rick

Hi Rick,

Please follow the Yocto BSP user guide to build the whole image for customer if you are not familiar the uboot-imx and linux-imx versions.
Yocto BSP tag version: VA1045

meta-advantech2 with aim30-zeus2 branch list in imx8LBVA1045.xml file:

The uboot-imx and linux-imx branch and commit version in meta-advantech2 are:
[Official Release] imx8LBVA1045 · ADVANTECH-Corp/meta-advantech2@90b21c2 (github.com)


uboot-imx: adv_v2020.04_5.4.70_2.3.0(03967623201a0e41d4adb8b400c8fa909feceebf)

Kernel linux-imx: adv_5.4.70_2.3.0(16ac00f91950fadef0340930a9ee5b5ec49dd00f)

The uboot-imx defconfig is:
uboot-imx/configs at adv_v2020.04_5.4.70_2.3.0 · ADVANTECH-Corp/uboot-imx (github.com)

The linux-imx defconfig:
Yocto Linux BSP Ver.A User Guide for iMX8 series -Yocto 3.0 - ESS-WIKI (advantech.com.tw)


linux-imx/arch/arm64/configs/imx_v8_adv_defconfig at adv_5.4.70_2.3.0 · ADVANTECH-Corp/linux-imx (github.com)

Kernel linux-imx 3730 device tree:
linux-imx/arch/arm64/boot/dts/freescale at adv_5.4.70_2.3.0 · ADVANTECH-Corp/linux-imx (github.com)

Best Regards,
Jems

Hello Jems,
sorry for the delay, I’ve only just gotten back on this project.

I tried the Linux kernel at the checkout that you gave me, 16ac00f91950fadef0340930a9ee5b5ec49dd00f.
That still gives me kernel panic with my configuration file.

If, instead, I use the branch adv_5.4.70_2.3.0 (your second link), i get a compile time error:

With that said, I can compile just fine the kernel from the tag imx8LBVA1045, no compile time error and no kernel panic.

But then I see the Lontium peripheral is disabled in the .dts file. Enabling it seems to make no difference, as the device is not responsive on the i2c bus (tried with i2cdetect).

What do I need to do to enable video output on the board?

Thank you,
Rick

Hi Rick,

I know don’t why you will compile error. But I can compile the kernel and dtb file successfully.

I found your error when build the imx8mm-rom5721 but you report the issue is on RSB3730. COuld you help to check it?
list below is my command history for build image and dtb file on my system.
542 git clone GitHub - ADVANTECH-Corp/linux-imx: Advantech i.MX series Kernel source code -b adv_5.4.70_2.3.0
543 git log
544 cd linux-imx/
545 git log
546 git log
547 git branch
548 git checkout v.A1047 16ac00f9
549 git checkout 16ac00f9 VA1047
550 git checkout v.A1047 imx8LBVA1047
551 git checkout v_A1047 imx8LBVA1047
552 git checkout -b v_A1047 imx8LBVA1047
553 git log
554 source /opt/fsl-imx-xwayland/5.4-zeus/environment-setup-aarch64-poky-linux
555 make distclean
556 make imx_v8_adv_defconfig
557 make menuconfig
558 make -j4 Image
559 unset LDFLAGS
560 make -j4 modules
561*
562 vim arch/arm64/boot/dts/freescale/Makefile
563 make -j4 Image
564 vim arch/arm64/boot/dts/freescale/imx8mm-rsb3730-a2.dtb
565 make -j4 Image
566 make arch/arm64/boot/dts/freescale/imx8mm-rsb3730-a2.dtb
567 make freescale/imx8mm-rsb3730-a2.dtb
568 git log
569 history

HI Jems,
I can’t run your commands verbatim, as I’m not using Yocto and don’t have your toolchain. I assumed it’s nothing special, since it isn’t mentioned in the “Building & updating kernel/modules/dtb manually” section in your Wiki.

Unfortunately, I need to make these changes to the .config file, otherwise my compiler throws an error due to those modules requiring access to some x86-64 libraries:

CONFIG_MARVELL_PHY=n
CONFIG_WLAN_VENDOR_MARVELL=n
CONFIG_MBT_SDIO_8997=n
CONFIG_MWIFI_SDIO_8997=m
CONFIG_NXP_PCIE_9098=n

However, I still get kernel panic.

Also, for some extra context on the kernel panic, I noticed these kernel prints right before the panic message:

[    2.291266] gpio-142 (scl): enforced open drain please flag it properly in DT/ACPI DSDT/board file
[    2.383192] ldo6: supplied by regulator-dummy
[    2.392020] i2c i2c-0: IMX I2C adapter registered
[    2.397183] gpio-144 (scl): enforced open drain please flag it properly in DT/ACPI DSDT/board file
[    3.230844] ddrc freq set to low bus mode
[    3.982612] adv-wdt-i2c 1-0029: Advantech Watchdog Timer enabled. timeout=60s (nowayout=0), Ver.17

My command history:

   15  source tc
   16  make imx_v8_adv_defconfig
   17  vim .config
   18  make Image
   19  make Image -j6
   20  vim .config
   21  unset LDFLAGS
   22  make -j6 modules
   23  make freescale/imx8mm-rsb3730-a2.dtb
   24  history

And the toolchain I’m using, which is just these three exports:

export ARCH=arm64
export CROSS_COMPILE=/home/path-to-my/bin/aarch64-buildroot-linux-gnu-
export LDFLAGS="-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed -fstack-protector-strong -Wl,-z,relro,-z,now"

Hi Rick,

Please use the SDK toolchian we provided.

After source SDK to setup the cross-compile environment:

Hi Rick,

From the link you provide the WIKI BSP user guide:
The section 11.2 and 11.3 are guide you how to set up SDK and cross compiling environment.