Hello:
We have update some GW with a kernel 5.10.100 extracted from DSGW-210-DB12-HA-RV1.2.2.img and we are having some problems with the WiFi kernel module.
First of all, we have expermented a hangout, aparently casused by the proccess that manages the coexistence betwen wifi and bluetooth:
root@DSAE7BE38AEE:/media/userdata/roca/log# [324273.305980] INFO: task RTW_CMD_T READ:289 blocked for more than 122 seconds.
[324273.313182] Not tainted 5.10.100-g304c70ad206a-dirty #81
[324273.319197] “echo 0 > /proc/sys/kernel/hung_task_timeout_secs” disables this message.
[324273.327205] task:RTW_CMD_THREAD state:D stack: 0 pid: 289 ppid: 2 f ags:0x00000028
[324273.335720] Call trace:
[324273.338339] __switch_to+0xb8/0x11c
[324273.342018] __schedule+0x3ec/0x5cc
[324273.345683] schedule+0x74/0xbc
[324273.349014] __mmc_claim_host+0xe8/0x1d4
[324273.353085] sdio_claim_host+0x40/0x44
[324273.357014] rtw_sdio_raw_write+0x294/0x2a8
[324273.361372] sdio_io+0xc0/0x360
[324273.364659] rtw_sdio_write_cmd53+0x44/0x54
[324273.369008] _halmac_sdio_reg_write_32+0x78/0xe8
[324273.373823] r_indir_cmd53_88xx+0xa8/0x148
[324273.378061] r_indir_sdio_88xx+0xd8/0x328
[324273.382256] reg_r8_sdio_8821c+0xf8/0x12c
[324273.386445] rtw_halmac_read8+0x34/0x40
[324273.390451] _rtw_read8+0x30/0x3c
[324273.393948] halbtcoutsrc_BitMaskWrite1Byte+0x40/0xac
[324273.399178] ex_halbtc8821c1ant_media_status_notify+0xac/0x2d4
[324273.405190] EXhalbtcoutsrc_media_status_notify+0xf0/0x100
[324273.410827] hal_btcoex_MediaStatusNotify+0x2c/0x38
[324273.415899] rtw_btcoex_MediaStatusNotify+0x40/0x80
[324273.420957] lps_ctrl_wk_hdl+0x200/0x2b8
[324273.425034] rtw_drvextra_cmd_hdl+0xe0/0x4f0
[324273.429486] rtw_cmd_thread+0x3c0/0x740
[324273.433546] kthread+0xf0/0x100
[324273.436850] ret_from_fork+0x10/0x18
[324273.440803] Kernel panic - not syncing: hung_task: blocked tasks
[324273.446924] CPU: 1 PID: 34 Comm: khungtaskd Not tainted 5.10.100-g304c70ad20 a-dirty #81
[324273.455100] Hardware name: Rockchip RK3328 EVB (DT)
[324273.460071] Call trace:
[324273.462621] dump_backtrace+0x0/0x1a8
[324273.466378] show_stack+0x2c/0x38
[324273.469790] dump_stack+0xd4/0x110
[324273.473294] panic+0x174/0x36c
[324273.476454] watchdog+0x158/0x378
[324273.479867] kthread+0xf0/0x100
[324273.483107] ret_from_fork+0x10/0x18
[324273.486784] SMP: stopping secondary CPUs
[324273.490817] PMU:
[324273.492770] 00000000: 00008000 00000000 00000000 00001000 00001555 00005555 0000005 00000000
[324273.501392] 00000020: 00000a05 00000200 00000000 00000000 00000000 0000aa05 0004124 00000020
[324273.510012] 00000040: 00000003 00000000 00002aaa 00000000 00000021 00000000 0000000 00000000
[324273.518633] 00000060: 00000000 00000000 00000000 00000000 00000000 00000000 0000000 00000000
[324273.527254] 00000080: 00000000 00000000 00000000 00000000 00000000 00000000 0000000 00000000
[324273.535875] 000000a0: 00000000 00000000 00000000 00000000 00000000 00000000 0000000 00000000
[324273.544495] 000000c0: 00000000 00000000 00000000 00000000 00000000 00000000 0000000 00000000
[324273.553115] 000000e0: 00000000 00000000 00000000 00000000 00000000 00000000 0000000 00000000
[324273.561725] Kernel Offset: disabled
[324273.565310] CPU features: 0x0040002,20002004
[324273.569675] Memory Limit: none
[324273.572844] —[ end Kernel panic - not syncing: hung_task: blocked tasks ]- -
As the wifi module have not the blueetooth interface connected we have dissabled the coexistance by using the modules parameters:
Deactivate wifi bluetooth coexistance and low power check
echo 0 > /sys/module/8821cs/parameters/rtw_lps_chk_by_tp
echo 0 > /sys/module/8821cs/parameters/rtw_btcoex_enable
With this seems this problem is solved, but we have find another one. After 2 weeks running continously the GW wifi stops running, it cannot connect to networks, cannot scan for networks. Although it appears among the network devices, it has a randomized MAC and the dmesg anounces some error related to leaving power save mode:
root@DSAE7BE38AEE:/etc/NetworkManager/conf.d# hwinfo --wlan
04: None 00.0: 0282 WLAN controller
[Created at pci.1030]
Unique ID: yukn.Vu98QzaObUF
SysFS ID: /devices/platform/ff510000.dwmmc/mmc_host/mmc2/mmc2:0001/mmc2:0001:1
SysFS BusID: mmc2:0001:1
Hardware Class: network
Model: “ARM Ethernet controller”
Device: “ARM Ethernet controller”
Driver: “rtl8821cs”
Device File: wlan0
Features: WLAN
HW Address: f0:b0:40:bf:05:b5
Permanent HW Address: f0:b0:40:bf:05:b5
Link detected: no
WLAN channels: 1 2 3 4 5 6 7 8 9 10 11 12 13 36 40 44 48 52 56 60 64 100 104 10
WLAN frequencies: 2.412 2.417 2.422 2.427 2.432 2.437 2.442 2.447 2.452 2.457 2 5.64 5.66 5.68 5.7
WLAN bitrates: 1 2 5.5 11
WLAN encryption modes: TKIP CCMP
WLAN authentication modes: open wpa-psk wpa-eap
Module Alias: "of:NdwmmcT(null)Crockchip,rk3328-dw-mshcCrockchip,rk3288-dw-mshc
Config Status: cfg=new, avail=yes, need=no, active=unknown
05: None 00.0: 0282 WLAN controller
[Created at pci.1030]
Unique ID: yukn.Vu98QzaObUF
SysFS ID: /devices/platform/ff510000.dwmmc/mmc_host/mmc2/mmc2:0001/mmc2:0001:1
SysFS BusID: mmc2:0001:1
Hardware Class: network
Model: “ARM Ethernet controller”
Device: “ARM Ethernet controller”
Driver: “rtl8821cs”
Device File: wlan1
Features: WLAN
HW Address: f2:b0:40:bf:05:b5
Permanent HW Address: f2:b0:40:bf:05:b5
Link detected: no
WLAN channels: 1 2 3 4 5 6 7 8 9 10 11 12 13 36 40 44 48 52 56 60 64 100 104 10
WLAN frequencies: 2.412 2.417 2.422 2.427 2.432 2.437 2.442 2.447 2.452 2.457 2 5.64 5.66 5.68 5.7
WLAN bitrates: 1 2 5.5 11
WLAN encryption modes: TKIP CCMP
WLAN authentication modes: open wpa-psk wpa-eap
Module Alias: "of:NdwmmcT(null)Crockchip,rk3328-dw-mshcCrockchip,rk3288-dw-mshc
Config Status: cfg=new, avail=yes, need=no, active=unknown
Here is a trace of our dmesg:
[849163.786955] RTW: rtw_net_set_mac_address(wlan0): Set Mac Addr to e2:ef:fb:b8:ec:7c Successfully
[849163.787126] RTW: rtw_ndev_notifier_call(wlan0) state:8
[849163.790405] RTW: rtw_ndev_notifier_call(wlan0) state:14
[849163.790464] RTW: _netdev_open(wlan0) , bup=1
[849163.790530] RTW: -871x_drv - drv_open, bup=1
[849163.790830] RTW: cfg80211_rtw_set_power_mgmt(wlan0) enabled:1, timeout:-1
[849163.790857] RTW: rtw_ndev_notifier_call(wlan0) state:1
[849163.835792] RTW: cfg80211_rtw_get_txpower(wlan0) total max: -10000 mbm
[849163.847318] RTW: cfg80211_rtw_get_txpower(wlan1) total max: -10000 mbm
[849163.857831] RTW: cfg80211_rtw_scan(wlan0)
[849163.857876] RTW: LeaveAllPowerSaveMode(wlan0): bSurpriseRemoved=_TRUE Skip!
[849164.551301] RTW: cfg80211_rtw_scan(wlan0)
[849164.551346] RTW: LeaveAllPowerSaveMode(wlan0): bSurpriseRemoved=_TRUE Skip!
[849168.775428] RTW: cfg80211_rtw_scan(wlan0)
[849168.775477] RTW: LeaveAllPowerSaveMode(wlan0): bSurpriseRemoved=_TRUE Skip!
[849173.773780] RTW: cfg80211_rtw_scan(wlan0)
[849173.773829] RTW: LeaveAllPowerSaveMode(wlan0): bSurpriseRemoved=_TRUE Skip!
For the moment the only way we have found to solve this is to reboot the GW. As far as we know this reboot is no powering down the GPIO for the wifi power supply (GPIO50 acording to kernel device tree), so we suspect that the module that controls the WiFi is no working correctly. As the kernel is monolitic we can not unload and reload the kernel module to restart them, so the only way to restart them is to make a reboot.
Does anybody experience something similar? Have any body find a better solution?
Aditional information:
We are using a debian 12 based rootfs.
Tanks for the help