Received: by oss.sgi.com id ; Tue, 20 Jun 2000 14:50:19 -0700 Received: from ppp0.ocs.com.au ([203.34.97.3]:34576 "HELO mail.ocs.com.au") by oss.sgi.com with SMTP id ; Tue, 20 Jun 2000 14:49:50 -0700 Received: (qmail 15774 invoked by uid 502); 20 Jun 2000 21:49:45 -0000 Received: (qmail 15750 invoked from network); 20 Jun 2000 21:49:40 -0000 Received: from ocs3.ocs-net (192.168.255.3) by mail.ocs.com.au with SMTP; 20 Jun 2000 21:49:40 -0000 X-Mailer: exmh version 2.1.1 10/15/1999 From: Keith Owens To: Rusty Russell cc: "netdev@oss.sgi.com" , Donald Becker , Andrew Morton Subject: Re: modular net drivers In-reply-to: Your message of "Tue, 20 Jun 2000 17:01:56 +1000." <20000620070156.D030D817F@halfway> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 21 Jun 2000 07:49:40 +1000 Message-ID: <4450.961537780@ocs3.ocs-net> Sender: owner-netdev@oss.sgi.com Precedence: bulk Return-Path: X-Orcpt: rfc822;netdev-outgoing On Tue, 20 Jun 2000 17:01:56 +1000, Rusty Russell wrote: >Keith Owens wrote: >> It is also an important bug fix. The module code has suffered from >> unload races ever since the kernel locking became fine grained, users >> can crash the kernel. > >Races which can be largely solved at the moment by having the module >page removal code sync all bh's and softirqs after calling cleanup(). >Hell, we could even poll all CPUs and check they're not executing in >the about-to-be-freed pages. Speed is completely unimportant here. This race is not obvious but IMHO it exists. The original theory was Kernel load and unload code runs under the big kernel lock. open() and similar code runs under the big kernel lock. If the code does MOD_INC_USE_COUNT before sleeping then we are safe. But consider this race, even on UP. Module has been used, nothing is currently using it, use_count == 0. rmmod runs, either manual or autoclean. The module is marked as being deleted. module_cleanup() is entered, does I/O, sleeps, loses big kernel lock. open() is entered, calls the module code, does MOD_INC_USE_COUNT. open() code in module sleeps, loosing big kernel lock. module_cleanup() resumes, gets big kernel lock, unloads the module. open() code in module resumes - the code pages have gone. Checking in module_cleanup() to see if the use count has changed is not a solution. module_cleanup() may already have destroyed structures that the open() code expects to use, either immediately or later. Polling bh and softirq is not the answer either. While the code sleeps it is in schedule() - the only thing that says a process is in a module is some return address 4+ layers up the stack! Linus suggested only allowing module unload when all processors where idle but that has the same problem - schedule is idle. AFAICT the only safe mechanism is one that checks the module state *before* entering the module. Once you enter the module and sleep all bets are off. And that means exporting the module information to the open() layer, which is what Al Viro has been doing.