http://lwip.100.n7.nabble.com/TCPIP-and-MAC-thread-task-priorities-td28800.html
I was looking through my codebase recently (a mix of code from LPC Open, and also a project I inherited) and I saw a comment which I want to fact check:
/* TCPIP thread must run at higher priority than MAC threads! */
My software is using FreeRTOS. The Ethernet interrupt only signals a semaphore which wakes up the "MAC" thread. This thread handles incoming and outgoing packets on the ethernet hardware.
The comment implies that this MAC thread must run at a lower priority than the TCPIP thread. I don't know who wrote the comment, or if it is correct.
This posting
indicates that there may not be such a stringent requirement, and it might even be advantageous to have the Ethernet task run at a higher priority than the TCPIP thread, but I'm not sure if the same applies to a MAC thread.
Thanks
- Wayne
_______________________________________________ lwip-users mailing list [hidden email] https://lists.nongnu.org/mailman/listinfo/lwip-users |
Feb 27, 2017; 7:02pm
Re: TCPIP and MAC thread task priorities
497 posts
|
Hi Wayne,
On Mon, Feb 27, 2017 at 01:56:24PM +1000, Wayne Uroda wrote: > I was looking through my codebase recently (a mix of code from LPC Open, > and also a project I inherited) and I saw a comment which I want to fact > check: > > /* TCPIP thread must run at higher priority than MAC threads! */ > > My software is using FreeRTOS. The Ethernet interrupt only signals a > semaphore which wakes up the "MAC" thread. This thread handles incoming and > outgoing packets on the ethernet hardware. > > The comment implies that this MAC thread must run at a lower priority than > the TCPIP thread. I don't know who wrote the comment, or if it is correct. > This posting > http://lists.nongnu.org/archive/html/lwip-users/2010-03/msg00148.html > > indicates that there may not be such a stringent requirement, and it might > even be advantageous to have the Ethernet task run at a higher priority > than the TCPIP thread, but I'm not sure if the same applies to a MAC thread. At first sight, if your TCPIP thread is busy handling packets from the message box there is little value of pushing more packets into it. But (there is always a but), if you do otherwise, if the TCPIP message box is full, you will enter a priority inversion condition and your RX thread will wait for a little while to queue the message or drop the packet. It means having your RX thread with a higher priority helps your system to fill the TCPIP thread message box, this is not the case if your RX thread have a lower priority than the TCPIP thread. So, I'm with Simon on that, your RX thread *should* have a higher priority than the TCPIP thread. Anyway, this is always the case for RX interrupt based systems. Sylvain _______________________________________________ lwip-users mailing list [hidden email] https://lists.nongnu.org/mailman/listinfo/lwip-users |
300 posts
|
Hi All,
This comes from my own experience with STM micro that has Cortex-M core Running with FreeRTOS. It does apply to STM9 etc... TCP main task priority should be high priority. It should not be the highest priority but it should not be lower than other tasks that consume lots of time share. The most misunderstood issue is interrupt priority levels in Cortext-M and FreeRTOS. First of all: A lower number in Coretex means a higher priority while in FreeRTOS a higher number Means a higher priority. Secondly Cortex use a nesting interrupt and a smart level... Assume you set an interrupt to level 2 and try to run some critical section code from RS232 but that RS232 has an interrupt level set at 5. What happens is that the interrupt 2 (higher) will never be masked because the issuer of the Critical section runs at a lower interrupt level. FreeRTOS has lots of sections like this one: portENTER_CRITICAL(); { // some code } portEXIT_CRITICAL(); If you set your interrupt level to be higher priority than the FreeRTOS timer tick that part will never Be masked and your critical section will not work as expected. I had strange problems and only after understanding the above I stabilized my software. I hope this is clear now. BR, Noam. -----Original Message----- From: lwip-users [mailto:lwip-users-bounces+noam= [hidden email]] On Behalf Of Sylvain Rochet Sent: Monday, February 27, 2017 1:02 PM To: Mailing list for lwIP users Subject: Re: [lwip-users] TCPIP and MAC thread task priorities Hi Wayne, On Mon, Feb 27, 2017 at 01:56:24PM +1000, Wayne Uroda wrote: > I was looking through my codebase recently (a mix of code from LPC > Open, and also a project I inherited) and I saw a comment which I want > to fact > check: > > /* TCPIP thread must run at higher priority than MAC threads! */ > > My software is using FreeRTOS. The Ethernet interrupt only signals a > semaphore which wakes up the "MAC" thread. This thread handles > incoming and outgoing packets on the ethernet hardware. > > The comment implies that this MAC thread must run at a lower priority > than the TCPIP thread. I don't know who wrote the comment, or if it is correct. > This posting "must" is wrong, there is no such requirement. > http://lists.nongnu.org/archive/html/lwip-users/2010-03/msg00148.html > > indicates that there may not be such a stringent requirement, and it > might even be advantageous to have the Ethernet task run at a higher > priority than the TCPIP thread, but I'm not sure if the same applies to a MAC thread. At first sight, if your TCPIP thread is busy handling packets from the message box there is little value of pushing more packets into it. But (there is always a but), if you do otherwise, if the TCPIP message box is full, you will enter a priority inversion condition and your RX thread will wait for a little while to queue the message or drop the packet. It means having your RX thread with a higher priority helps your system to fill the TCPIP thread message box, this is not the case if your RX thread have a lower priority than the TCPIP thread. So, I'm with Simon on that, your RX thread *should* have a higher priority than the TCPIP thread. Anyway, this is always the case for RX interrupt based systems. Sylvain _______________________________________________ lwip-users mailing list [hidden email] https://lists.nongnu.org/mailman/listinfo/lwip-users |
Feb 27, 2017; 8:24pm
Re: TCPIP and MAC thread task priorities
497 posts
|
Hi,
On Mon, Feb 27, 2017 at 11:50:26AM +0000, Noam Weissman wrote: > Hi All, > > This comes from my own experience with STM micro that has Cortex-M core > Running with FreeRTOS. It does apply to STM9 etc... > > TCP main task priority should be high priority. It should not be the > highest priority but it should not be lower than other tasks that > consume lots of time share. > > The most misunderstood issue is interrupt priority levels in Cortext-M > and FreeRTOS. > > First of all: A lower number in Coretex means a higher priority while > in FreeRTOS a higher number Means a higher priority. > > Secondly Cortex use a nesting interrupt and a smart level... Assume > you set an interrupt to level 2 and try to run some critical section > code from RS232 but that RS232 has an interrupt level set at 5. > > What happens is that the interrupt 2 (higher) will never be masked > because the issuer of the Critical section runs at a lower interrupt > level. > > FreeRTOS has lots of sections like this one: > > portENTER_CRITICAL(); > { > // some code > } > portEXIT_CRITICAL(); > > > If you set your interrupt level to be higher priority than the > FreeRTOS timer tick that part will never Be masked and your critical > section will not work as expected.
... [
show rest of quote]
... [
show rest of quote]
I don't understand any of this, portENTER_CRITICAL/portEXIT_CRITICAL
proper implementation is to disable *ALL* interrupts, therefore having a critical section which is not enforced for whatever reason can't happen at all. By the way, FreeRTOS is not supposed to run with nested interrupts, you have to put all OS calls from interrupts (i.e. *FromISR() functions) in a critical section. Sylvain _______________________________________________ lwip-users mailing list [hidden email] https://lists.nongnu.org/mailman/listinfo/lwip-users |
Feb 27, 2017; 9:40pm
Re: TCPIP and MAC thread task priorities
497 posts
|
Hi,
On Mon, Feb 27, 2017 at 01:24:39PM +0100, Sylvain Rochet wrote: > On Mon, Feb 27, 2017 at 11:50:26AM +0000, Noam Weissman wrote: > > > > If you set your interrupt level to be higher priority than the > > FreeRTOS timer tick that part will never Be masked and your critical > > section will not work as expected. > > I don't understand any of this, portENTER_CRITICAL/portEXIT_CRITICAL > proper implementation is to disable *ALL* interrupts, therefore having a > critical section which is not enforced for whatever reason can't happen > at all. API, you indeed don't care for interrupts that are outside the FreeRTOS scope, which don't even need to be naked at all, and therefore can't wake up tasks. > By the way, FreeRTOS is not supposed to run with nested interrupts, you > have to put all OS calls from interrupts (i.e. *FromISR() functions) in > a critical section. Except if your FreeRTOS port support it, looks like some of them do :-) Anyway, this is a bit outside the initial subject. The RX thread (if any) should have a higher priority than the TCPIP thread, both priorities should obviously be within the allowed priorities window which only depend on your FreeRTOS port. Sylvain _______________________________________________ lwip-users mailing list [hidden email] https://lists.nongnu.org/mailman/listinfo/lwip-users |
300 posts
|
In reply to
this post by Sylvain Rochet
Dear Sylvain,
If you issue portDISABLE_INTERRUPTS() it will mask all interrupts If you issue portENTER_CRITICAL() it will only mask interrupts that have a priority Lower than the FreeRTOS one interrupt. See these: http://www.freertos.org/RTOS-Cortex-M3-M4.html In this link check the paragraph that starts with Attention please!: http://www.freertos.org/portcortexiar.html https://mcuoneclipse.com/2016/08/14/arm-cortex-m-interrupts-and-freertos-part-1/ BR, Noam. -----Original Message----- From: lwip-users [mailto:lwip-users-bounces+noam= [hidden email]] On Behalf Of Sylvain Rochet Sent: Monday, February 27, 2017 2:25 PM To: Mailing list for lwIP users Subject: Re: [lwip-users] TCPIP and MAC thread task priorities Hi, On Mon, Feb 27, 2017 at 11:50:26AM +0000, Noam Weissman wrote: > Hi All, > > This comes from my own experience with STM micro that has Cortex-M > core Running with FreeRTOS. It does apply to STM9 etc... > > TCP main task priority should be high priority. It should not be the > highest priority but it should not be lower than other tasks that > consume lots of time share. > > The most misunderstood issue is interrupt priority levels in Cortext-M > and FreeRTOS. > > First of all: A lower number in Coretex means a higher priority while > in FreeRTOS a higher number Means a higher priority. > > Secondly Cortex use a nesting interrupt and a smart level... Assume > you set an interrupt to level 2 and try to run some critical section > code from RS232 but that RS232 has an interrupt level set at 5. > > What happens is that the interrupt 2 (higher) will never be masked > because the issuer of the Critical section runs at a lower interrupt > level. > > FreeRTOS has lots of sections like this one: > > portENTER_CRITICAL(); > { > // some code > } > portEXIT_CRITICAL(); > > > If you set your interrupt level to be higher priority than the > FreeRTOS timer tick that part will never Be masked and your critical > section will not work as expected.
... [
show rest of quote]
I don't understand any of this, portENTER_CRITICAL/portEXIT_CRITICAL proper implementation is to disable *ALL* interrupts, therefore having a critical section which is not enforced for whatever reason can't happen at all. By the way, FreeRTOS is not supposed to run with nested interrupts, you have to put all OS calls from interrupts (i.e. *FromISR() functions) in a critical section. Sylvain _______________________________________________ lwip-users mailing list [hidden email] https://lists.nongnu.org/mailman/listinfo/lwip-users |
Feb 27, 2017; 10:09pm
Re: TCPIP and MAC thread task priorities
2733 posts
|
In reply to
this post by Sylvain Rochet
Sylvain Rochet wrote:
>> If you set your interrupt level to be higher priority than the >> FreeRTOS timer tick that part will never Be masked and your critical >> section will not work as expected. > > I don't understand any of this, portENTER_CRITICAL/portEXIT_CRITICAL > proper implementation is to disable *ALL* interrupts To add some confusion: that's not entirely true: at least on the Cortex-M ARMs, you can have "fast-interrupts" that are NOT masked by ENTER/EXIT_CRICITAL. If I remember correctly, all (Cortex-M) ports I used have been setting up the lowest/highest interrupt priority in a way that at least some interrupts were such fast-interrupts, so there's definitively a possibility that this happens if you configure things wrong! Keeping that aside and answering to the OP: There is *no* limitation in lwIP itself that imposes the requirement of specific task priorities. The lwIP port to your OS/system can however impose this, so there's no way for us knowing if the comment you saw is true or not. In the end, you'll have to know which thread is more important than others. Given the tcpip_thread/eth_thread combination, you probably have to weight TX against RX, but that can largely depend on your application and/or driver. Simon _______________________________________________ lwip-users mailing list [hidden email] https://lists.nongnu.org/mailman/listinfo/lwip-users |
Feb 28, 2017; 3:03am
Re: TCPIP and MAC thread task priorities
2733 posts
|
In reply to
this post by Sylvain Rochet
Oops, didn't see that before pushing the 'send' button :-)
Sylvain Rochet wrote: > The RX thread (if any) should have a higher priority than the TCPIP thread, That's probably true. Having it the other way round doesn't actually break it I guess, but you should have a strong cause to do so (e.g. RX throttling, giving priority to TX). Simon _______________________________________________ lwip-users mailing list [hidden email] https://lists.nongnu.org/mailman/listinfo/lwip-users |
Feb 28, 2017; 4:24am
Re: TCPIP and MAC thread task priorities
497 posts
|
Hi,
On Mon, Feb 27, 2017 at 08:03:09PM +0100, [hidden email] wrote: > Oops, didn't see that before pushing the 'send' button :-) > > Sylvain Rochet wrote: > >The RX thread (if any) should have a higher priority than the TCPIP thread, > > That's probably true. Having it the other way round doesn't actually > break it I guess, but you should have a strong cause to do so (e.g. RX > throttling, giving priority to TX). It doesn't matter much anyway, we all know that with mutexes, semaphores, critical sections, and queues, priority inversion is the norm rather than the exception. If the system can't process RX fast enough, whichever task have precedence, incoming packets will be lost, either if DMA rx ring is full or if tcpip message box is full, then it won't work properly. Sylvain _______________________________________________ lwip-users mailing list [hidden email] https://lists.nongnu.org/mailman/listinfo/lwip-users |
3 posts
|
In reply to
this post by Simon Goldschmidt
Thank you Simon, and everyone else who contributed to the discussion.
I have had both tasks set at the same priority (RTOS task priority, nothing to do with the Cortex interrupt priority). I have not noticed any issues with this so I will leave it as is for now.
I don't think that the comment is accurate in my source code so I will remove it (perhaps replacing it with a link to this conversation).
Thanks again for the great work on supporting lwip.
- Wayne
On 28 February 2017 at 00:09, Simon Goldschmidt
<[hidden email]> wrote:
Sylvain Rochet wrote:
... [
show rest of quote]
_______________________________________________ lwip-users mailing list [hidden email] https://lists.nongnu.org/mailman/listinfo/lwip-users |