System panics in drain_squeue() TCP/IP function under heavy stress.

The following Sun Alerts describe TCP/IP issues for the new SPARC systems which use CMT processors.

• Sun Alert ID: 102551
• Sun Alert ID: 102553
• Sun Alert ID: 102554

This issue can occur in the following releases:

SPARC Platform

• Solaris 10 without patch 119998-01

x86 Platform

• Solaris 10 without patch 119999-01

Notes: Solaris 8 and 9 are not impacted by this issue.

Extremely heavy TCP/IP loads with many tens of thousands of rapidly opening/closing TCP connections.

CMT processors (Chip Multi-Threading technology) are more likely to sustain the throughput required, while still being able to context switch rapidly enough to reproduce this. It is highly unlikely that this issue will be seen on other systems.

There are two classes of system using CMT processors. These can be identified using the following commands:

a) Systems with Sun4v kernel architecture use CMT processors. These are identified by the command:

   % uname -m

which will return "sun4v".

b) UltraSPARC IV and UltraSPARC IV+ systems use CMT processors. These are identified by the command:

   % prtconf -pv | grep "SUNW,UltraSPARC-IV"

which will return a non-NULL result something similar to:

   compatible: 'SUNW,UltraSPARC-IV'

The following panic string and stack trace is representative of this issue:

   BAD TRAP: type=31 rp=2a17492b380 addr=1c mmu_fsr=0
   occurred in module "ip" due to a NULL pointer dereference

   <trap>ip:squeue_drain+0x114(0x6000192d300, 0x10)
   ip:squeue_enter+0x34c()
   ip:tcp_wput(0x30011b6cbe0, 0x60023c83780) - frame recycled
   unix:putnext+0x218(0x30011b6ce70, 0x60023c83780?)
   genunix:strput+0x1b4(0x300143f46b8, 0x60023c83780, 0x0, 0x2a17492b918,
   0x0, 0x0)
   genunix:kstrputmsg+0x328(0x60008790780, , 0x0, 0x0, 0x0, 0x2c4, 0x0)
   sockfs:sotpi_setsockopt+0x58c(0x3001445ebf0, 0x121c2113, 0x6c3e4fdd,
   0x2a17492bac8, 0xc)
   sockfs:setsockopt+0xc0()
   unix:syscall_trap+0xac()
   -- switch to user thread's user stack --

Please be aware that athough this stacktrace does not match the stacktrace in the bug report for bug 6266950, it is the same issue.

The stacktrace in the bug report illustrated this same problem when running an IP module compiled with DEBUG turned on.

There is no workaround. Please see Resolution section below.

This issue is addressed in the following releases:

SPARC Platform

• Solaris 10 with patch 119998-01 or later

x86 Platform

• Solaris 10 with patch 119999-02 or later

• Updated Relief/Workaround with temporary patches

• Updated Contributing Factors, Relief/Workaround and Resolution sections
• State: Resolved

This solution has no attachment