From srompf@isg.de Sun Dec 1 14:07:24 2002 Received: with ECARTIS (v1.0.0; list netdev); Sun, 01 Dec 2002 14:07:27 -0800 (PST) Received: from mail.isg.de (rzfoobar.is-asp.com [217.11.194.155]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB1M7DuR008331 for ; Sun, 1 Dec 2002 14:07:13 -0800 Received: from isg.de (Ae153.pppool.de [213.6.225.83]) by mail.isg.de (Postfix) with ESMTP id C201988BEC; Sun, 1 Dec 2002 22:30:37 +0100 (CET) Message-ID: <3DEA0452.B1F15BFD@isg.de> Date: Sun, 01 Dec 2002 13:45:06 +0100 From: Stefan Rompf X-Mailer: Mozilla 4.79 [en] (X11; U; Linux 2.4.20-rc2-lw-apic i686) X-Accept-Language: en MIME-Version: 1.0 To: David Brownell Cc: netdev@oss.sgi.com Subject: Re: 2.5.50 BUG_TRAP on !dev->deadbeaf, and oopses References: <3DE9290A.7070502@pacbell.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-archive-position: 1275 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: srompf@isg.de Precedence: bulk X-list: netdev Hi, David Brownell wrote: > KERNEL: assertion (!dev->deadbeaf) failed at net/core/dev.c(2544) > > I think there's another bug, beyond the obvious speling erorz. Namely, > that "deadbeaf" is only set after that BUG_TRAP, or on one error path. > The assertion prevents hotpluggable network drivers from unregistering > when the hardware goes away ... which is a regression. actually, the assertion is triggered when someone tries to unregister a netdevice twice, and that's also why you get > unregister_netdevice: device /dfd74058 never was registered >From a short browsing through usb.c I don't see a similiar bug catcher in usb_device_remove(), so have a look if the USB subsystem itself removes a unplugged device twice for some reason. Stefan From david-b@pacbell.net Mon Dec 2 11:09:43 2002 Received: with ECARTIS (v1.0.0; list netdev); Mon, 02 Dec 2002 11:09:51 -0800 (PST) Received: from mta7.pltn13.pbi.net (mta7.pltn13.pbi.net [64.164.98.8]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB2J9huR013179 for ; Mon, 2 Dec 2002 11:09:43 -0800 Received: from pacbell.net ([67.118.246.114]) by mta7.pltn13.pbi.net (iPlanet Messaging Server 5.1 (built May 7 2001)) with ESMTP id <0H6I007YCAP740@mta7.pltn13.pbi.net> for netdev@oss.sgi.com; Mon, 02 Dec 2002 11:12:45 -0800 (PST) Date: Mon, 02 Dec 2002 10:44:05 -0800 From: David Brownell Subject: Re: 2.5.50 BUG_TRAP on !dev->deadbeaf, and oopses To: Stefan Rompf Cc: netdev@oss.sgi.com Message-id: <3DEBA9F5.6000606@pacbell.net> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=us-ascii Content-transfer-encoding: 7bit X-Accept-Language: en-us, en, fr User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.9) Gecko/20020513 References: <3DE9290A.7070502@pacbell.net> <3DEA0452.B1F15BFD@isg.de> X-archive-position: 1277 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: david-b@pacbell.net Precedence: bulk X-list: netdev Content-Length: 1608 Lines: 43 Hi Stefan, >> KERNEL: assertion (!dev->deadbeaf) failed at net/core/dev.c(2544) >> >>I think there's another bug, beyond the obvious speling erorz. Namely, >>that "deadbeaf" is only set after that BUG_TRAP, or on one error path. >>The assertion prevents hotpluggable network drivers from unregistering >>when the hardware goes away ... which is a regression. > > > actually, the assertion is triggered when someone tries to unregister a > netdevice twice, and that's also why you get Then why will grep of all kernel files not turn up other places where 'deadbeaf' gets set? There's strange stuff going on here regardless (as well as speling issue), which looks pretty buglike. Plus: this kind of bugcatch should use magic numbers, or maybe zero. Assuming "any nonzero value is valid", like this assertion does, is clearly going to fail for any of the class of bugs highlighted by slab poisoning. (0xa5a5a5a5 gets accepted as valid...) >> unregister_netdevice: device /dfd74058 never was registered > > > From a short browsing through usb.c I don't see a similiar bug catcher > in usb_device_remove(), so have a look if the USB subsystem itself > removes a unplugged device twice for some reason. At least one failure path also involves "rmmod" of the network drivers, where the device hardware is still around; so that code would not always be called. I wouldn't rule out problems in the relevant usbcore/sysfs bits, even now that they seem to have stabilized again (and yes, I was wondering about multiple disconnects too), but all that deadbeaf logic still looks fishy to me. - Dave From jgarzik@pobox.com Mon Dec 2 19:30:55 2002 Received: with ECARTIS (v1.0.0; list netdev); Mon, 02 Dec 2002 19:30:57 -0800 (PST) Received: from www.linux.org.uk (parcelfarce.linux.theplanet.co.uk [195.92.249.252]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB33UruR021173 for ; Mon, 2 Dec 2002 19:30:54 -0800 Received: from rdu74-162-040.nc.rr.com ([24.74.162.40] helo=pobox.com) by www.linux.org.uk with esmtp (Exim 3.33 #5) id 18J3on-0001o4-00; Tue, 03 Dec 2002 03:33:54 +0000 Message-ID: <3DEC2600.2070306@pobox.com> Date: Mon, 02 Dec 2002 22:33:20 -0500 From: Jeff Garzik User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2) Gecko/20021126 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Linux Kernel Mailing List CC: netdev@oss.sgi.com, akpm@zip.com.au Subject: Network interface synchronization docs Content-Type: multipart/mixed; boundary="------------050300030504070804060609" X-archive-position: 1278 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jgarzik@pobox.com Precedence: bulk X-list: netdev Content-Length: 2122 Lines: 81 This is a multi-part message in MIME format. --------------050300030504070804060609 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit I just updated Documentation/networking/netdevices.txt locally to the attached file, which describes the synchronization rules for net drivers in the 2.4.x and 2.5.x kernels. (Astute readers will notice a striking similarity to 2.2.x locking rules as well) Comments and corrections to the attached doc requested. I realize it's not much right now, but this is core info I want to make sure is correct, and that everyone agrees on. Thanks to Robert Olsson for a pointer about dev->poll(). Future notes: this doc will hopefully expand in time to describe not only the netdevice API but also standard ethernet and net driver practices. Patches welcome! Jeff --------------050300030504070804060609 Content-Type: text/plain; name="netdevices.txt" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="netdevices.txt" Network Devices, the Kernel, and You! Introduction ============ The following is a random collection of documentation regarding network devices. struct net_device synchronization rules ======================================= dev->open: Synchronization: rtnl_lock() semaphore. Context: process dev->stop: Synchronization: rtnl_lock() semaphore. Context: process Notes: netif_running() is guaranteed false when this is called dev->do_ioctl: Synchronization: rtnl_lock() semaphore. Context: process dev->get_stats: Synchronization: dev_base_lock rwlock. Context: nominally process, but don't sleep inside an rwlock dev->hard_start_xmit: Synchronization: dev->xmit_lock spinlock. Context: BHs disabled dev->tx_timeout: Synchronization: dev->xmit_lock spinlock. Context: BHs disabled dev->set_multicast_list: Synchronization: dev->xmit_lock spinlock. Context: BHs disabled dev->poll: Synchronization: __LINK_STATE_RX_SCHED bit in dev->state. See dev_close code and comments in net/core/dev.c for more info. Context: softirq --------------050300030504070804060609-- From rz5b@cs.virginia.edu Mon Dec 2 21:35:42 2002 Received: with ECARTIS (v1.0.0; list netdev); Mon, 02 Dec 2002 21:35:46 -0800 (PST) Received: from ares.cs.Virginia.EDU (mail.cs.Virginia.EDU [128.143.137.19]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB35ZeuR023915 for ; Mon, 2 Dec 2002 21:35:41 -0800 Received: from bobbidi.cs.Virginia.EDU (bobbidi.cs.Virginia.EDU [128.143.67.48]) by ares.cs.Virginia.EDU (8.9.3+Sun/8.9.2/UVACS-2000040300) with ESMTP id AAA19945 for ; Tue, 3 Dec 2002 00:38:43 -0500 (EST) Date: Tue, 3 Dec 2002 00:39:02 -0500 (EST) From: Ronghua Zhang To: netdev@oss.sgi.com Subject: synchronization between bottom half and user context Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 1279 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: rz5b@cs.virginia.edu Precedence: bulk X-list: netdev Content-Length: 354 Lines: 11 Can someone tell me when lock should be used to synchronize bh and user context? For examle: in tcp_accept(), lock_sock() is called to avoid any change made by net_bh during its operation. But if tcp_accept() is called, it's impossilbe that net_bh is also executing. (i think this is true for UP, what about SMP?), then why need lock_sock()? ronghua From yan@intruvert.com Tue Dec 3 12:27:32 2002 Received: with ECARTIS (v1.0.0; list netdev); Tue, 03 Dec 2002 12:27:37 -0800 (PST) Received: from ivexs1.intruvert.com (mx2.intruvert.com [65.209.235.21]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB3KRWuR014414 for ; Tue, 3 Dec 2002 12:27:32 -0800 Received: by ivexs1.intruvert.com with Internet Mail Service (5.5.2653.19) id ; Tue, 3 Dec 2002 12:30:33 -0800 Message-ID: From: Yan-Fa Li To: netdev@oss.sgi.com Subject: TG3 Performance Question Date: Tue, 3 Dec 2002 12:30:33 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Disposition: inline Content-Type: text/plain Content-Transfer-Encoding: 7bit X-archive-position: 1280 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: yan@intruvert.com Precedence: bulk X-list: netdev Content-Length: 2070 Lines: 56 Hello I'm trying to figure out some performance issues I have with the tg3 driver. I'm trying to increase the performance of an existing test app, a simple http simulator, by improving the hardware. My original setup uses dual 1.1GHz PIII systems on 64bit-66MHz PCI slots and Serverworks LE chipsets. The new setup is on AMD MPX2 systems with a single Athlon 2000MP. Because of packet reordering issues, I have restricted my app to using only a single CPU using the cpu affinity APIs. On the serverworks systems I have DL2K network cards and am able to sustain 400-450mbits of traffic per system pair using a NAPIzed driver and a hacked up 2.4.19 kernel. On the new AMD system I'm using a fresh 2.4.20 with a TG3 card. The DL2K cards I have, have a chipset level incompatibility with the AMD chipset which I believe has been fixed in the TG3 driver and has only been fixed in hardware for the DL2K so I am unable to test them in the AMD box. Basically I am only able to get around the same performance with a 1.6GHz Athlon than I can with a single 1.1GHz PIII. This perplexes me since I thought originally I was CPU bound. Is this: a) network card issue (i.e. the tg3 isn't as a well documented/ tuned as the dl2k yet) ? b) a PCI chipset issue, i.e. the AMD MPX is kinda lame, I've seen comments from Alan Cox that this may be so, especially vs the server works chipset. c) 400-450mbits is the best you'll ever get with a single gigE card in a 64bit/66MHz PCI slot ? I'm assuming that the tg3 and 2.4.20 kernels are both NAPI, looking through the source code seems to indicate they are. With the DL2K driver I am able to specify tx and rx interrupt mitigation values. Is there an equivalent for the tg3 ? I can't see any useful parameters using modinfo -p. Any suggestions you might have to increase the performance would be appreciated. Right now I thinking about trying the following, a) getting a new rev of DL2K that works with the AMD or b) switching motherboards to Serverworks again. Thanks Yan [[HTML alternate version deleted]] From greearb@candelatech.com Tue Dec 3 12:41:56 2002 Received: with ECARTIS (v1.0.0; list netdev); Tue, 03 Dec 2002 12:41:57 -0800 (PST) Received: from grok.yi.org (IDENT:q4+2c0tAdCTHII6UwBhaYKZ8k2TPAH5L@dhcp101-dsl-usw4.w-link.net [208.161.125.101]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB3KfsuR015868 for ; Tue, 3 Dec 2002 12:41:55 -0800 Received: from candelatech.com (IDENT:QXFqn6vJudJIqZOFtRyGnUjEF9gGrKAA@localhost.localdomain [127.0.0.1]) by grok.yi.org (8.11.6/8.11.2) with ESMTP id gB3Kimq21488; Tue, 3 Dec 2002 12:44:48 -0800 Message-ID: <3DED17C0.2010404@candelatech.com> Date: Tue, 03 Dec 2002 12:44:48 -0800 From: Ben Greear Organization: Candela Technologies User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2a) Gecko/20020910 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Yan-Fa Li CC: netdev@oss.sgi.com Subject: Re: TG3 Performance Question References: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 1281 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: greearb@candelatech.com Precedence: bulk X-list: netdev Content-Length: 474 Lines: 17 Yan-Fa Li wrote: > Hello > > I'm trying to figure out some performance issues I have with > the tg3 driver. Are you sure your tg3 NIC is 66/64 PCI? For 66/64, try an e1000 NIC, you should be able to get 800Mbps or better out of it on such a system. Ben -- Ben Greear President of Candela Technologies Inc http://www.candelatech.com ScryMUD: http://scry.wanfear.com http://scry.wanfear.com/~greear From srompf@isg.de Tue Dec 3 15:43:54 2002 Received: with ECARTIS (v1.0.0; list netdev); Tue, 03 Dec 2002 15:44:05 -0800 (PST) Received: from mail.isg.de (rzfoobar.is-asp.com [217.11.194.155]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB3NhhuR019661 for ; Tue, 3 Dec 2002 15:43:43 -0800 Received: from isg.de (B1ac8.pppool.de [213.7.26.200]) by mail.isg.de (Postfix) with ESMTP id 5A5D4EA6E88; Wed, 4 Dec 2002 00:08:39 +0100 (CET) Message-ID: <3DED302A.E3DEB585@isg.de> Date: Tue, 03 Dec 2002 23:28:58 +0100 From: Stefan Rompf X-Mailer: Mozilla 4.79 [en] (X11; U; Linux 2.4.20 i686) X-Accept-Language: en MIME-Version: 1.0 To: davem@redhat.com Cc: netdev@oss.sgi.com Subject: Re: Patch: Backport of link state notification to 2.4.20 References: <3DD2D204.C9B7FA2D@isg.de> Content-Type: multipart/mixed; boundary="------------8ABD0F6A5B9C2D9D4F3AC90F" X-archive-position: 1284 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: srompf@isg.de Precedence: bulk X-list: netdev Content-Length: 9338 Lines: 332 This is a multi-part message in MIME format. --------------8ABD0F6A5B9C2D9D4F3AC90F Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Hi David, attached is the recent backport of link state notification for the stable kernel series. The feature is optional and can be configured away to nothing, and tested on several systems. I'd really like to have it in 2.4.21, and as your the one to decide before Marcelo gets it, can you have a look? Stefan --------------8ABD0F6A5B9C2D9D4F3AC90F Content-Type: text/plain; charset=us-ascii; name="patch-linkwatch-2.4.20" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="patch-linkwatch-2.4.20" diff -uNr linux-2.4.20rc1/Documentation/Configure.help linux/Documentation/Configure.help --- linux-2.4.20rc1/Documentation/Configure.help 2002-11-05 00:31:35.000000000 +0100 +++ linux/Documentation/Configure.help 2002-11-13 22:32:46.000000000 +0100 @@ -26011,6 +26011,18 @@ would like kernel messages to be formatted into GDB $O packets so that GDB prints them as program output, say 'Y'. +Device link state notification +CONFIG_LINKWATCH + When this option is enabled, the kernel will forward changes in the + operative ("RUNNING") state of an interface via the netlink socket. + This is most useful when running linux as a router. + + Note that currently not many drivers support this, compliant ones + can be found by watching the RUNNING flag in ifconfig output that + should follow operative state. + + If unsure, say 'N'. + 802.1Q VLAN Support CONFIG_VLAN_8021Q Select this and you will be able to create 802.1Q VLAN interfaces on your diff -uNr linux-2.4.20rc1/include/linux/netdevice.h linux/include/linux/netdevice.h --- linux-2.4.20rc1/include/linux/netdevice.h 2002-11-05 00:31:42.000000000 +0100 +++ linux/include/linux/netdevice.h 2002-11-13 22:32:46.000000000 +0100 @@ -207,7 +207,8 @@ __LINK_STATE_PRESENT, __LINK_STATE_SCHED, __LINK_STATE_NOCARRIER, - __LINK_STATE_RX_SCHED + __LINK_STATE_RX_SCHED, + __LINK_STATE_LINKWATCH_PENDING }; @@ -630,6 +631,10 @@ * who is responsible for serialization of these calls. */ +#ifdef CONFIG_LINKWATCH +extern void linkwatch_fire_event(struct net_device *dev); +#endif + static inline int netif_carrier_ok(struct net_device *dev) { return !test_bit(__LINK_STATE_NOCARRIER, &dev->state); @@ -639,14 +644,24 @@ static inline void netif_carrier_on(struct net_device *dev) { +#ifdef CONFIG_LINKWATCH + if (test_and_clear_bit(__LINK_STATE_NOCARRIER, &dev->state)) + linkwatch_fire_event(dev); +#else clear_bit(__LINK_STATE_NOCARRIER, &dev->state); +#endif if (netif_running(dev)) __netdev_watchdog_up(dev); } static inline void netif_carrier_off(struct net_device *dev) { +#ifdef CONFIG_LINKWATCH + if (!test_and_set_bit(__LINK_STATE_NOCARRIER, &dev->state)) + linkwatch_fire_event(dev); +#else set_bit(__LINK_STATE_NOCARRIER, &dev->state); +#endif } /* Hot-plugging. */ diff -uNr linux-2.4.20rc1/net/Config.in linux/net/Config.in --- linux-2.4.20rc1/net/Config.in 2002-08-03 02:39:46.000000000 +0200 +++ linux/net/Config.in 2002-11-13 22:32:46.000000000 +0100 @@ -48,6 +48,7 @@ bool ' Per-VC IP filter kludge' CONFIG_ATM_BR2684_IPFILTER fi fi + bool 'Device link state notification (EXPERIMENTAL)' CONFIG_LINKWATCH fi tristate '802.1Q VLAN Support' CONFIG_VLAN_8021Q diff -uNr linux-2.4.20rc1/net/core/Makefile linux/net/core/Makefile --- linux-2.4.20rc1/net/core/Makefile 2002-08-03 02:39:46.000000000 +0200 +++ linux/net/core/Makefile 2002-11-13 22:33:32.000000000 +0100 @@ -31,4 +31,6 @@ # Ugly. I wish all wireless drivers were moved in drivers/net/wireless obj-$(CONFIG_NET_PCMCIA_RADIO) += wireless.o +obj-$(CONFIG_LINKWATCH) += link_watch.o + include $(TOPDIR)/Rules.make diff -uNr linux-2.4.20rc1/net/core/dev.c linux/net/core/dev.c --- linux-2.4.20rc1/net/core/dev.c 2002-11-05 00:31:42.000000000 +0100 +++ linux/net/core/dev.c 2002-11-13 22:32:46.000000000 +0100 @@ -194,6 +194,12 @@ #endif +#ifdef CONFIG_LINKWATCH +extern void linkwatch_init(void); +extern void linkwatch_run_queue(void); +#endif + + /****************************************************************************************** Protocol management and registration routines @@ -2628,6 +2634,18 @@ /* Rebroadcast unregister notification */ notifier_call_chain(&netdev_chain, NETDEV_UNREGISTER, dev); } + +#ifdef CONFIG_LINKWATCH + if (test_bit(__LINK_STATE_LINKWATCH_PENDING, &dev->state)) { + /* We must not have linkwatch events pending + * on unregister. If this happens, we simply + * run the queue unscheduled, resulting in a + * noop for this device + */ + linkwatch_run_queue(); + } +#endif + current->state = TASK_INTERRUPTIBLE; schedule_timeout(HZ/4); current->state = TASK_RUNNING; @@ -2675,6 +2693,10 @@ dv_init(); #endif /* CONFIG_NET_DIVERT */ +#ifdef CONFIG_LINKWATCH + linkwatch_init(); +#endif + /* * Initialise the packet receive queues. */ diff -uNr linux-2.4.20rc1/net/core/link_watch.c linux/net/core/link_watch.c --- linux-2.4.20rc1/net/core/link_watch.c 1970-01-01 01:00:00.000000000 +0100 +++ linux/net/core/link_watch.c 2002-11-13 22:36:44.000000000 +0100 @@ -0,0 +1,149 @@ +/* + * Linux network device link state notifaction + * + * Author: + * Stefan Rompf + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License + * as published by the Free Software Foundation; either version + * 2 of the License, or (at your option) any later version. + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + + +enum lw_bits { + LW_RUNNING = 0, + LW_SE_USED +}; + +static unsigned long linkwatch_flags = 0; +static unsigned long linkwatch_nextevent = 0; + +static void linkwatch_event(void *dummy); +static void linkwatch_timer(unsigned long dummy); + +static struct tq_struct linkwatch_tq; +static struct timer_list linkwatch_ti; + +static LIST_HEAD(lweventlist); +static spinlock_t lweventlist_lock = SPIN_LOCK_UNLOCKED; + +struct lw_event { + struct list_head list; + struct net_device *dev; +}; + +/* Avoid kmalloc() for most systems */ +static struct lw_event singleevent; + +/* Must be called with the rtnl semaphore held */ +void linkwatch_run_queue(void) { + LIST_HEAD(head); + struct list_head *n, *next; + + spin_lock_irq(&lweventlist_lock); + list_splice_init(&lweventlist, &head); + spin_unlock_irq(&lweventlist_lock); + + list_for_each_safe(n, next, &head) { + struct lw_event *event = list_entry(n, struct lw_event, list); + struct net_device *dev = event->dev; + + if (event == &singleevent) { + clear_bit(LW_SE_USED, &linkwatch_flags); + } else { + kfree(event); + } + + /* We are about to handle this device, + * so new events can be accepted + */ + clear_bit(__LINK_STATE_LINKWATCH_PENDING, &dev->state); + + if (dev->flags & IFF_UP) { + netdev_state_change(dev); + } + + dev_put(dev); + } +} + + +static void linkwatch_event(void *dummy) +{ + /* Limit the number of linkwatch events to one + * per second so that a runaway driver does not + * cause a storm of messages on the netlink + * socket + */ + linkwatch_nextevent = jiffies + HZ; + clear_bit(LW_RUNNING, &linkwatch_flags); + + rtnl_lock(); + linkwatch_run_queue(); + rtnl_unlock(); +} + + +static void linkwatch_timer(unsigned long dummy) { + (void)schedule_task(&linkwatch_tq); +} + + +void linkwatch_fire_event(struct net_device *dev) +{ + if (!test_and_set_bit(__LINK_STATE_LINKWATCH_PENDING, &dev->state)) { + unsigned long flags; + struct lw_event *event; + + if (test_and_set_bit(LW_SE_USED, &linkwatch_flags)) { + event = kmalloc(sizeof(struct lw_event), GFP_ATOMIC); + + if (unlikely(event == NULL)) { + clear_bit(__LINK_STATE_LINKWATCH_PENDING, &dev->state); + return; + } + } else { + event = &singleevent; + } + + dev_hold(dev); + event->dev = dev; + + spin_lock_irqsave(&lweventlist_lock, flags); + list_add_tail(&event->list, &lweventlist); + spin_unlock_irqrestore(&lweventlist_lock, flags); + + if (!test_and_set_bit(LW_RUNNING, &linkwatch_flags)) { + unsigned long thisevent = jiffies; + + if (thisevent >= linkwatch_nextevent) { + (void)schedule_task(&linkwatch_tq); + } else { + linkwatch_ti.expires = linkwatch_nextevent; + add_timer(&linkwatch_ti); + } + } + } +} + + +void __init linkwatch_init(void) { + linkwatch_ti.function = linkwatch_timer; + init_timer(&linkwatch_ti); + INIT_TQUEUE(&linkwatch_tq, linkwatch_event, NULL); +} + diff -uNr linux-2.4.20rc1/net/netsyms.c linux/net/netsyms.c --- linux-2.4.20rc1/net/netsyms.c 2002-11-05 00:31:42.000000000 +0100 +++ linux/net/netsyms.c 2002-11-13 22:33:32.000000000 +0100 @@ -591,6 +591,10 @@ EXPORT_SYMBOL(softnet_data); +#ifdef CONFIG_LINKWATCH +EXPORT_SYMBOL(linkwatch_fire_event); +#endif + #if defined(CONFIG_NET_RADIO) || defined(CONFIG_NET_PCMCIA_RADIO) #include EXPORT_SYMBOL(wireless_send_event); --------------8ABD0F6A5B9C2D9D4F3AC90F-- From srompf@isg.de Tue Dec 3 15:43:54 2002 Received: with ECARTIS (v1.0.0; list netdev); Tue, 03 Dec 2002 15:43:56 -0800 (PST) Received: from mail.isg.de (rzfoobar.is-asp.com [217.11.194.155]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB3NhhuR019660 for ; Tue, 3 Dec 2002 15:43:43 -0800 Received: from isg.de (B1ac8.pppool.de [213.7.26.200]) by mail.isg.de (Postfix) with ESMTP id F2C82EA7414; Wed, 4 Dec 2002 00:08:44 +0100 (CET) Message-ID: <3DED3869.5E47D7A2@isg.de> Date: Wed, 04 Dec 2002 00:04:09 +0100 From: Stefan Rompf X-Mailer: Mozilla 4.79 [en] (X11; U; Linux 2.4.20 i686) X-Accept-Language: en MIME-Version: 1.0 To: jamal Cc: davem@redhat.com, netdev@oss.sgi.com Subject: Re: Patch resubmission: RFC2863 operstatus for 2.5.49 References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-archive-position: 1282 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: srompf@isg.de Precedence: bulk X-list: netdev Content-Length: 729 Lines: 18 Hi Jamal, > Just thought of something, it may be a little tricky but valuable and > i am not quiet sure if it should part of your patch: We probably need to > flush the qdiscs software queues; maybe even the DMA ring i.e simulate > admin down followed by admin up. dev_close() is doing quite a lot of stuff, so we should do nothing more than flush the qdiscs when the link comes up. But is it really useful? Normally, the queues are short anyway to keep latencies low, and fifty additional packets don't hurt. I rather think about clearing neighbor tables and the route cache whenever the operstatus goes down. Anyway, implementation and usage of the notification should not mix up, so it let it be a separate patch. Stefan From srompf@isg.de Tue Dec 3 15:43:54 2002 Received: with ECARTIS (v1.0.0; list netdev); Tue, 03 Dec 2002 15:43:58 -0800 (PST) Received: from mail.isg.de (rzfoobar.is-asp.com [217.11.194.155]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB3NhhuR019659 for ; Tue, 3 Dec 2002 15:43:43 -0800 Received: from isg.de (B1ac8.pppool.de [213.7.26.200]) by mail.isg.de (Postfix) with ESMTP id 13E36EA7373; Wed, 4 Dec 2002 00:08:29 +0100 (CET) Message-ID: <3DED2EA9.D812C881@isg.de> Date: Tue, 03 Dec 2002 23:22:33 +0100 From: Stefan Rompf X-Mailer: Mozilla 4.79 [en] (X11; U; Linux 2.4.20 i686) X-Accept-Language: en MIME-Version: 1.0 To: "David S. Miller" Cc: netdev@oss.sgi.com Subject: Re: Patch resubmission: RFC2863 operstatus for 2.5.50 References: <3DE33D6D.25B9C9B4@isg.de> <20021126.021546.91313706.davem@redhat.com> Content-Type: multipart/mixed; boundary="------------DD5A4FC259A764B48D9E75EC" X-archive-position: 1283 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: srompf@isg.de Precedence: bulk X-list: netdev Content-Length: 13942 Lines: 481 This is a multi-part message in MIME format. --------------DD5A4FC259A764B48D9E75EC Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Hi David, here is an updated version with the following changes: -split the patch: First adds the userspace notification, the other (depends on the first) RFC2863 semantics. I am aware that we are well beyond feature freeze, and it makes the code more readable. -simplified locking as suggested -made the notification unconditional. We want it ;-) -fixed a bug in RFC2863 netif_carrier_ok() emulation (was !=, should be ==) Can you have a look? Stefan "David S. Miller" wrote: > > This locking below achieves nothing. > > + read_lock_irqsave(&dev->operstate_lock, flags); > + state = dev->operstate; > + read_unlock_irqrestore(&dev->operstate_lock, flags); > > In fact, the other side, locking when setting this value, can > be done with a simple spinlock. Probably something else in > the device struct can be reused. > > I also don't think this should be conditional, either we want > it or we don't. --------------DD5A4FC259A764B48D9E75EC Content-Type: text/plain; charset=us-ascii; name="patch-lw-2.5.50" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="patch-lw-2.5.50" diff -urN linux-2.5.50/include/linux/netdevice.h linux-2.5.50-lw/include/linux/netdevice.h --- linux-2.5.50/include/linux/netdevice.h 2002-11-22 22:41:08.000000000 +0100 +++ linux-2.5.50-lw/include/linux/netdevice.h 2002-12-02 22:56:34.000000000 +0100 @@ -207,7 +207,8 @@ __LINK_STATE_PRESENT, __LINK_STATE_SCHED, __LINK_STATE_NOCARRIER, - __LINK_STATE_RX_SCHED + __LINK_STATE_RX_SCHED, + __LINK_STATE_LINKWATCH_PENDING }; @@ -631,6 +632,8 @@ * who is responsible for serialization of these calls. */ +extern void linkwatch_fire_event(struct net_device *dev); + static inline int netif_carrier_ok(struct net_device *dev) { return !test_bit(__LINK_STATE_NOCARRIER, &dev->state); @@ -640,14 +643,16 @@ static inline void netif_carrier_on(struct net_device *dev) { - clear_bit(__LINK_STATE_NOCARRIER, &dev->state); + if (test_and_clear_bit(__LINK_STATE_NOCARRIER, &dev->state)) + linkwatch_fire_event(dev); if (netif_running(dev)) __netdev_watchdog_up(dev); } static inline void netif_carrier_off(struct net_device *dev) { - set_bit(__LINK_STATE_NOCARRIER, &dev->state); + if (!test_and_set_bit(__LINK_STATE_NOCARRIER, &dev->state)) + linkwatch_fire_event(dev); } /* Hot-plugging. */ diff -urN linux-2.5.50/net/core/Makefile linux-2.5.50-lw/net/core/Makefile --- linux-2.5.50/net/core/Makefile 2002-12-02 22:37:15.000000000 +0100 +++ linux-2.5.50-lw/net/core/Makefile 2002-12-02 23:16:18.000000000 +0100 @@ -12,7 +12,7 @@ obj-$(CONFIG_FILTER) += filter.o -obj-$(CONFIG_NET) += dev.o dev_mcast.o dst.o neighbour.o rtnetlink.o utils.o +obj-$(CONFIG_NET) += dev.o dev_mcast.o dst.o neighbour.o rtnetlink.o utils.o link_watch.o obj-$(CONFIG_NETFILTER) += netfilter.o obj-$(CONFIG_NET_DIVERT) += dv.o diff -urN linux-2.5.50/net/core/dev.c linux-2.5.50-lw/net/core/dev.c --- linux-2.5.50/net/core/dev.c 2002-11-22 22:40:43.000000000 +0100 +++ linux-2.5.50-lw/net/core/dev.c 2002-12-02 23:08:47.000000000 +0100 @@ -262,6 +262,7 @@ br_write_unlock_bh(BR_NETPROTO_LOCK); } +void linkwatch_run_queue(void); /** * dev_remove_pack - remove packet handler @@ -2642,6 +2643,15 @@ /* Rebroadcast unregister notification */ notifier_call_chain(&netdev_chain, NETDEV_UNREGISTER, dev); + + if (test_bit(__LINK_STATE_LINKWATCH_PENDING, &dev->state)) { + /* We must not have linkwatch events pending + * on unregister. If this happens, we simply + * run the queue unscheduled, resulting in a + * noop for this device + */ + linkwatch_run_queue(); + } } current->state = TASK_INTERRUPTIBLE; schedule_timeout(HZ / 4); diff -urN linux-2.5.50/net/core/link_watch.c linux-2.5.50-lw/net/core/link_watch.c --- linux-2.5.50/net/core/link_watch.c 1970-01-01 01:00:00.000000000 +0100 +++ linux-2.5.50-lw/net/core/link_watch.c 2002-12-02 22:35:00.000000000 +0100 @@ -0,0 +1,134 @@ +/* + * Linux network device link state notifaction + * + * Author: + * Stefan Rompf + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License + * as published by the Free Software Foundation; either version + * 2 of the License, or (at your option) any later version. + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + + +enum lw_bits { + LW_RUNNING = 0, + LW_SE_USED +}; + +static unsigned long linkwatch_flags = 0; +static unsigned long linkwatch_nextevent = 0; + +static void linkwatch_event(void *dummy); +static DECLARE_WORK(linkwatch_work, linkwatch_event, NULL); + +static LIST_HEAD(lweventlist); +static spinlock_t lweventlist_lock = SPIN_LOCK_UNLOCKED; + +struct lw_event { + struct list_head list; + struct net_device *dev; +}; + +/* Avoid kmalloc() for most systems */ +static struct lw_event singleevent; + +/* Must be called with the rtnl semaphore held */ +void linkwatch_run_queue(void) { + LIST_HEAD(head); + struct list_head *n, *next; + + spin_lock_irq(&lweventlist_lock); + list_splice_init(&lweventlist, &head); + spin_unlock_irq(&lweventlist_lock); + + list_for_each_safe(n, next, &head) { + struct lw_event *event = list_entry(n, struct lw_event, list); + struct net_device *dev = event->dev; + + if (event == &singleevent) { + clear_bit(LW_SE_USED, &linkwatch_flags); + } else { + kfree(event); + } + + /* We are about to handle this device, + * so new events can be accepted + */ + clear_bit(__LINK_STATE_LINKWATCH_PENDING, &dev->state); + + if (dev->flags & IFF_UP) { + netdev_state_change(dev); + } + + dev_put(dev); + } +} + + +static void linkwatch_event(void *dummy) +{ + /* Limit the number of linkwatch events to one + * per second so that a runaway driver does not + * cause a storm of messages on the netlink + * socket + */ + linkwatch_nextevent = jiffies + HZ; + clear_bit(LW_RUNNING, &linkwatch_flags); + + rtnl_lock(); + linkwatch_run_queue(); + rtnl_unlock(); +} + + +void linkwatch_fire_event(struct net_device *dev) +{ + if (!test_and_set_bit(__LINK_STATE_LINKWATCH_PENDING, &dev->state)) { + unsigned long flags; + struct lw_event *event; + + if (test_and_set_bit(LW_SE_USED, &linkwatch_flags)) { + event = kmalloc(sizeof(struct lw_event), GFP_ATOMIC); + + if (unlikely(event == NULL)) { + clear_bit(__LINK_STATE_LINKWATCH_PENDING, &dev->state); + return; + } + } else { + event = &singleevent; + } + + dev_hold(dev); + event->dev = dev; + + spin_lock_irqsave(&lweventlist_lock, flags); + list_add_tail(&event->list, &lweventlist); + spin_unlock_irqrestore(&lweventlist_lock, flags); + + if (!test_and_set_bit(LW_RUNNING, &linkwatch_flags)) { + unsigned long thisevent = jiffies; + + if (thisevent >= linkwatch_nextevent) { + schedule_work(&linkwatch_work); + } else { + schedule_delayed_work(&linkwatch_work, linkwatch_nextevent - thisevent); + } + } + } +} + diff -urN linux-2.5.50/net/netsyms.c linux-2.5.50-lw/net/netsyms.c --- linux-2.5.50/net/netsyms.c 2002-11-22 22:40:39.000000000 +0100 +++ linux-2.5.50-lw/net/netsyms.c 2002-12-02 23:09:51.000000000 +0100 @@ -632,4 +632,6 @@ EXPORT_SYMBOL(wireless_send_event); #endif /* CONFIG_NET_RADIO || CONFIG_NET_PCMCIA_RADIO */ +EXPORT_SYMBOL(linkwatch_fire_event); + #endif /* CONFIG_NET */ --------------DD5A4FC259A764B48D9E75EC Content-Type: text/plain; charset=us-ascii; name="patch-lw-2.5.50-rfc2863" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="patch-lw-2.5.50-rfc2863" diff -urN linux-2.5.50-lw/include/linux/netdevice.h linux-2.5.50-lw-rfc2863/include/linux/netdevice.h --- linux-2.5.50-lw/include/linux/netdevice.h 2002-12-02 22:56:34.000000000 +0100 +++ linux-2.5.50-lw-rfc2863/include/linux/netdevice.h 2002-12-02 22:45:14.000000000 +0100 @@ -204,14 +204,26 @@ { __LINK_STATE_XOFF=0, __LINK_STATE_START, - __LINK_STATE_PRESENT, + __LINK_STATE_PRESENT_OBSOLETE, __LINK_STATE_SCHED, - __LINK_STATE_NOCARRIER, + __LINK_STATE_NOCARRIER_OBSOLETE, __LINK_STATE_RX_SCHED, __LINK_STATE_LINKWATCH_PENDING }; +/* Device operative state as per RFC2863 */ +enum netdev_operstate_t { + NETDEV_OPER_UP = 1, + NETDEV_OPER_DOWN, /* Obsoletes LINK_STATE_NOCARRIER */ + NETDEV_OPER_TESTING, + NETDEV_OPER_UNKNOWN, + NETDEV_OPER_DORMANT, + NETDEV_OPER_NOTPRESENT, /* Obsoletes !LINK_STATE_PRESENT */ + NETDEV_OPER_LOWERDOWN +}; + + /* * This structure holds at boot time configured netdevice settings. They * are then used in the device probing. @@ -309,6 +321,10 @@ * which this device is member of. */ + /* Operative state, access semaphore */ + spinlock_t operstate_lock; + unsigned char operstate; + /* Interface address info. */ unsigned char broadcast[MAX_ADDR_LEN]; /* hw bcast add */ unsigned char dev_addr[MAX_ADDR_LEN]; /* hw address */ @@ -634,36 +650,63 @@ extern void linkwatch_fire_event(struct net_device *dev); +static inline unsigned char netif_set_operstate(struct net_device *dev, unsigned char newstate) +{ + unsigned long flags; + unsigned char oldstate; + + spin_lock_irqsave(&dev->operstate_lock, flags); + oldstate = dev->operstate; + dev->operstate = newstate; + spin_unlock_irqrestore(&dev->operstate_lock, flags); + + if (oldstate != newstate) linkwatch_fire_event(dev); + + return oldstate; +} + +static inline unsigned char netif_get_operstate(struct net_device *dev) +{ + return(dev->operstate); +} + static inline int netif_carrier_ok(struct net_device *dev) { - return !test_bit(__LINK_STATE_NOCARRIER, &dev->state); + return netif_get_operstate(dev) == NETDEV_OPER_UP; +} + +static inline int netif_operstate_to_iff_running(struct net_device *dev) +{ + unsigned char state = netif_get_operstate(dev); + + return((1 << state) & + (1 << NETDEV_OPER_UP | 1 << NETDEV_OPER_UNKNOWN)); } extern void __netdev_watchdog_up(struct net_device *dev); + static inline void netif_carrier_on(struct net_device *dev) { - if (test_and_clear_bit(__LINK_STATE_NOCARRIER, &dev->state)) - linkwatch_fire_event(dev); + netif_set_operstate(dev, NETDEV_OPER_UP); if (netif_running(dev)) __netdev_watchdog_up(dev); } static inline void netif_carrier_off(struct net_device *dev) { - if (!test_and_set_bit(__LINK_STATE_NOCARRIER, &dev->state)) - linkwatch_fire_event(dev); + netif_set_operstate(dev, NETDEV_OPER_DOWN); } /* Hot-plugging. */ static inline int netif_device_present(struct net_device *dev) { - return test_bit(__LINK_STATE_PRESENT, &dev->state); + return netif_get_operstate(dev) != NETDEV_OPER_NOTPRESENT; } static inline void netif_device_detach(struct net_device *dev) { - if (test_and_clear_bit(__LINK_STATE_PRESENT, &dev->state) && + if (netif_set_operstate(dev, NETDEV_OPER_NOTPRESENT) != NETDEV_OPER_NOTPRESENT && netif_running(dev)) { netif_stop_queue(dev); } @@ -671,7 +714,7 @@ static inline void netif_device_attach(struct net_device *dev) { - if (!test_and_set_bit(__LINK_STATE_PRESENT, &dev->state) && + if (netif_set_operstate(dev, NETDEV_OPER_UNKNOWN) == NETDEV_OPER_NOTPRESENT && netif_running(dev)) { netif_wake_queue(dev); __netdev_watchdog_up(dev); diff -urN linux-2.5.50-lw/net/core/dev.c linux-2.5.50-lw-rfc2863/net/core/dev.c --- linux-2.5.50-lw/net/core/dev.c 2002-12-02 23:08:47.000000000 +0100 +++ linux-2.5.50-lw-rfc2863/net/core/dev.c 2002-12-02 23:22:03.000000000 +0100 @@ -199,7 +199,6 @@ int netdev_fastroute_obstacles; #endif - /******************************************************************************* Protocol management and registration routines @@ -2019,7 +2018,7 @@ IFF_RUNNING)) | (dev->gflags & (IFF_PROMISC | IFF_ALLMULTI)); - if (netif_running(dev) && netif_carrier_ok(dev)) + if (netif_running(dev) && netif_operstate_to_iff_running(dev)) ifr->ifr_flags |= IFF_RUNNING; return 0; @@ -2434,6 +2433,10 @@ goto out; #endif /* CONFIG_NET_DIVERT */ + /* Initial operstate */ + spin_lock_init(&dev->operstate_lock); + dev->operstate = NETDEV_OPER_UNKNOWN; + dev->iflink = -1; /* Init, if this function is available */ @@ -2459,13 +2462,6 @@ if (!dev->rebuild_header) dev->rebuild_header = default_rebuild_header; - /* - * Default initial state at registry is that the - * device is present. - */ - - set_bit(__LINK_STATE_PRESENT, &dev->state); - dev->next = NULL; dev_init_scheduler(dev); write_lock_bh(&dev_base_lock); @@ -2746,6 +2742,8 @@ #ifdef CONFIG_NET_FASTROUTE dev->fastpath_lock = RW_LOCK_UNLOCKED; #endif + spin_lock_init(&dev->operstate_lock); + dev->operstate = NETDEV_OPER_UNKNOWN; dev->xmit_lock_owner = -1; dev->iflink = -1; dev_hold(dev); @@ -2778,7 +2776,6 @@ if (!dev->rebuild_header) dev->rebuild_header = default_rebuild_header; dev_init_scheduler(dev); - set_bit(__LINK_STATE_PRESENT, &dev->state); } } diff -urN linux-2.5.50-lw/net/core/rtnetlink.c linux-2.5.50-lw-rfc2863/net/core/rtnetlink.c --- linux-2.5.50-lw/net/core/rtnetlink.c 2002-11-22 22:41:13.000000000 +0100 +++ linux-2.5.50-lw-rfc2863/net/core/rtnetlink.c 2002-12-02 22:35:00.000000000 +0100 @@ -165,7 +165,7 @@ r->ifi_flags = dev->flags; r->ifi_change = change; - if (!netif_running(dev) || !netif_carrier_ok(dev)) + if (!netif_running(dev) || !netif_operstate_to_iff_running(dev)) r->ifi_flags &= ~IFF_RUNNING; else r->ifi_flags |= IFF_RUNNING; --------------DD5A4FC259A764B48D9E75EC-- From jgarzik@pobox.com Tue Dec 3 16:40:59 2002 Received: with ECARTIS (v1.0.0; list netdev); Tue, 03 Dec 2002 16:41:03 -0800 (PST) Received: from www.linux.org.uk (parcelfarce.linux.theplanet.co.uk [195.92.249.252]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB40ewuR025683 for ; Tue, 3 Dec 2002 16:40:58 -0800 Received: from rdu74-162-040.nc.rr.com ([24.74.162.40] helo=pobox.com) by www.linux.org.uk with esmtp (Exim 3.33 #5) id 18JNdz-0004rj-00; Wed, 04 Dec 2002 00:44:04 +0000 Message-ID: <3DED4FB4.5070700@pobox.com> Date: Tue, 03 Dec 2002 19:43:32 -0500 From: Jeff Garzik User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2) Gecko/20021126 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Stefan Rompf CC: davem@redhat.com, netdev@oss.sgi.com Subject: Re: Patch: Backport of link state notification to 2.4.20 References: <3DD2D204.C9B7FA2D@isg.de> <3DED302A.E3DEB585@isg.de> In-Reply-To: <3DED302A.E3DEB585@isg.de> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 1285 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jgarzik@pobox.com Precedence: bulk X-list: netdev Content-Length: 5946 Lines: 222 Stefan Rompf wrote: > Hi David, > > attached is the recent backport of link state notification for the > stable kernel series. The feature is optional and can be configured away > to nothing, and tested on several systems. > It might be a backport, but I haven't seen a 2.5.x version posted in a while. If this is going into the stable series, it should already be merged in 2.5.x at the same time, if not first. > I'd really like to have it in 2.4.21, and as your the one to decide > before Marcelo gets it, can you have a look? s/before/if/ :) comments follow. note these are all minor objections: I think the patch needs revising, but I also think it is ok overall and support its eventual inclusion [after tidying]. Not that my opinion matters much here compared to DaveM's (and Jamal's and Alexey's and...)... overall, I have one question: how does netdev_state_change(dev) propagate link status to userspace? Via the IFF_RUNNING flag's presence/absence when IFF_UP is true? > diff -uNr linux-2.4.20rc1/include/linux/netdevice.h linux/include/linux/netdevice.h > --- linux-2.4.20rc1/include/linux/netdevice.h 2002-11-05 00:31:42.000000000 +0100 > +++ linux/include/linux/netdevice.h 2002-11-13 22:32:46.000000000 +0100 > @@ -630,6 +631,10 @@ > * who is responsible for serialization of these calls. > */ > > +#ifdef CONFIG_LINKWATCH > +extern void linkwatch_fire_event(struct net_device *dev); > +#endif remove the ifdef, not needed and ifdefs are discouraged in all Linux code. > diff -uNr linux-2.4.20rc1/net/Config.in linux/net/Config.in > --- linux-2.4.20rc1/net/Config.in 2002-08-03 02:39:46.000000000 +0200 > +++ linux/net/Config.in 2002-11-13 22:32:46.000000000 +0100 > @@ -48,6 +48,7 @@ > bool ' Per-VC IP filter kludge' CONFIG_ATM_BR2684_IPFILTER > fi > fi > + bool 'Device link state notification (EXPERIMENTAL)' CONFIG_LINKWATCH ifyou mark it experimental, then it should be dep_bool ... CONFIG_LINKWATCH $CONFIG_EXPERIMENTAL > diff -uNr linux-2.4.20rc1/net/core/Makefile linux/net/core/Makefile > --- linux-2.4.20rc1/net/core/Makefile 2002-08-03 02:39:46.000000000 +0200 > +++ linux/net/core/Makefile 2002-11-13 22:33:32.000000000 +0100 > @@ -31,4 +31,6 @@ > # Ugly. I wish all wireless drivers were moved in drivers/net/wireless > obj-$(CONFIG_NET_PCMCIA_RADIO) += wireless.o > > +obj-$(CONFIG_LINKWATCH) += link_watch.o > + > include $(TOPDIR)/Rules.make add it to export-objs too (because of below change) > diff -uNr linux-2.4.20rc1/net/core/dev.c linux/net/core/dev.c > --- linux-2.4.20rc1/net/core/dev.c 2002-11-05 00:31:42.000000000 +0100 > +++ linux/net/core/dev.c 2002-11-13 22:32:46.000000000 +0100 > @@ -194,6 +194,12 @@ > #endif > > > +#ifdef CONFIG_LINKWATCH > +extern void linkwatch_init(void); > +extern void linkwatch_run_queue(void); > +#endif ug :) the ifdef can be killed, and the prototypes should be in netdevice.h (or any other header), not in mainline code. > @@ -2675,6 +2693,10 @@ > dv_init(); > #endif /* CONFIG_NET_DIVERT */ > > +#ifdef CONFIG_LINKWATCH > + linkwatch_init(); > +#endif no need for an ifdef, make linkwatch_init() a no-op static inline for the !CONFIG_LINKWATCH case. > +static unsigned long linkwatch_flags = 0; > +static unsigned long linkwatch_nextevent = 0; statics are automatically initialized to zero, don't do it explicitly. you waste a couple bytes in the kernel image storing a zero value. > + > +static void linkwatch_event(void *dummy); > +static void linkwatch_timer(unsigned long dummy); > + > +static struct tq_struct linkwatch_tq; > +static struct timer_list linkwatch_ti; > + > +static LIST_HEAD(lweventlist); > +static spinlock_t lweventlist_lock = SPIN_LOCK_UNLOCKED; why initialize some complex structs programmatically and some explicitly? IMO for consistency you should initialize lweventlist_lock in linkwatch_init(). (or vice versa) > +/* Must be called with the rtnl semaphore held */ > +void linkwatch_run_queue(void) { > + LIST_HEAD(head); > + struct list_head *n, *next; > + > + spin_lock_irq(&lweventlist_lock); > + list_splice_init(&lweventlist, &head); > + spin_unlock_irq(&lweventlist_lock); spin_lock_irq{save,restore} would probably better serve you here... > + > + list_for_each_safe(n, next, &head) { > + struct lw_event *event = list_entry(n, struct lw_event, list); > + struct net_device *dev = event->dev; > + > + if (event == &singleevent) { > + clear_bit(LW_SE_USED, &linkwatch_flags); > + } else { > + kfree(event); > + } style: braces not needed > + > + /* We are about to handle this device, > + * so new events can be accepted > + */ > + clear_bit(__LINK_STATE_LINKWATCH_PENDING, &dev->state); > + > + if (dev->flags & IFF_UP) { > + netdev_state_change(dev); > + } likewise. (pointlessly makes code longer) > + > + dev_put(dev); > + } > +} > + > + > +static void linkwatch_event(void *dummy) > +{ > + /* Limit the number of linkwatch events to one > + * per second so that a runaway driver does not > + * cause a storm of messages on the netlink > + * socket > + */ > + linkwatch_nextevent = jiffies + HZ; > + clear_bit(LW_RUNNING, &linkwatch_flags); the high availability folks would prefer a half-second (HZ >> 1). > +void __init linkwatch_init(void) { > + linkwatch_ti.function = linkwatch_timer; > + init_timer(&linkwatch_ti); > + INIT_TQUEUE(&linkwatch_tq, linkwatch_event, NULL); > +} set the timer function after the initialize the timer :) > + > diff -uNr linux-2.4.20rc1/net/netsyms.c linux/net/netsyms.c > --- linux-2.4.20rc1/net/netsyms.c 2002-11-05 00:31:42.000000000 +0100 > +++ linux/net/netsyms.c 2002-11-13 22:33:32.000000000 +0100 > @@ -591,6 +591,10 @@ > > EXPORT_SYMBOL(softnet_data); > > +#ifdef CONFIG_LINKWATCH > +EXPORT_SYMBOL(linkwatch_fire_event); > +#endif move this to link_watch.c so that the featureset is fully encapsulated. Jeff From jgarzik@pobox.com Tue Dec 3 16:42:54 2002 Received: with ECARTIS (v1.0.0; list netdev); Tue, 03 Dec 2002 16:42:55 -0800 (PST) Received: from www.linux.org.uk (parcelfarce.linux.theplanet.co.uk [195.92.249.252]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB40gquR026073 for ; Tue, 3 Dec 2002 16:42:53 -0800 Received: from rdu74-162-040.nc.rr.com ([24.74.162.40] helo=pobox.com) by www.linux.org.uk with esmtp (Exim 3.33 #5) id 18JNfr-0004tk-00; Wed, 04 Dec 2002 00:45:59 +0000 Message-ID: <3DED5027.7070807@pobox.com> Date: Tue, 03 Dec 2002 19:45:27 -0500 From: Jeff Garzik User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2) Gecko/20021126 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Stefan Rompf CC: davem@redhat.com, netdev@oss.sgi.com Subject: Re: Patch: Backport of link state notification to 2.4.20 References: <3DD2D204.C9B7FA2D@isg.de> <3DED302A.E3DEB585@isg.de> <3DED4FB4.5070700@pobox.com> In-Reply-To: <3DED4FB4.5070700@pobox.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 1286 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jgarzik@pobox.com Precedence: bulk X-list: netdev Content-Length: 670 Lines: 22 Jeff Garzik wrote: > Stefan Rompf wrote: > >> Hi David, >> >> attached is the recent backport of link state notification for the >> stable kernel series. The feature is optional and can be configured away >> to nothing, and tested on several systems. >> > > It might be a backport, but I haven't seen a 2.5.x version posted in a > while. If this is going into the stable series, it should already be > merged in 2.5.x at the same time, if not first. ...and of course this email arrives immediately in my inbox ;-) Which leads to the question: why CONFIG_LINKWATCH in 2.4 but not 2.5? Let's do the same in both kernels unless there is a good reason not to... From jgarzik@pobox.com Tue Dec 3 16:50:23 2002 Received: with ECARTIS (v1.0.0; list netdev); Tue, 03 Dec 2002 16:50:25 -0800 (PST) Received: from www.linux.org.uk (parcelfarce.linux.theplanet.co.uk [195.92.249.252]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB40oMuR026648 for ; Tue, 3 Dec 2002 16:50:23 -0800 Received: from rdu74-162-040.nc.rr.com ([24.74.162.40] helo=pobox.com) by www.linux.org.uk with esmtp (Exim 3.33 #5) id 18JNn6-000501-00; Wed, 04 Dec 2002 00:53:29 +0000 Message-ID: <3DED51E9.1080408@pobox.com> Date: Tue, 03 Dec 2002 19:52:57 -0500 From: Jeff Garzik User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2) Gecko/20021126 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Stefan Rompf CC: "David S. Miller" , netdev@oss.sgi.com Subject: Re: Patch resubmission: RFC2863 operstatus for 2.5.50 References: <3DE33D6D.25B9C9B4@isg.de> <20021126.021546.91313706.davem@redhat.com> <3DED2EA9.D812C881@isg.de> In-Reply-To: <3DED2EA9.D812C881@isg.de> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 1287 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jgarzik@pobox.com Precedence: bulk X-list: netdev Content-Length: 287 Lines: 12 Pardon my dumb question, but what parts of RFC2863 require kernel additions over and above your link state patch? Your second patch I am less enthusiastic about than the first... :( I wonder if userspace cannot figure out ifOperStatus from existing data we make available? Jeff From anand@eis.iisc.ernet.in Tue Dec 3 23:35:31 2002 Received: with ECARTIS (v1.0.0; list netdev); Tue, 03 Dec 2002 23:35:33 -0800 (PST) Received: from iisc.ernet.in (www-cache.iisc.ernet.in [144.16.64.3] (may be forged)) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB47ZQuR031859 for ; Tue, 3 Dec 2002 23:35:29 -0800 Received: from eis.iisc.ernet.in (eis.iisc.ernet.in [144.16.64.5]) by iisc.ernet.in (8.9.2/8.9.0) with SMTP id NAA59458 for ; Wed, 4 Dec 2002 13:09:45 +0530 (IST) Received: by eis.iisc.ernet.in (SMI-8.6/SMI-4.1) id NAA02825; Wed, 4 Dec 2002 13:08:31 +0530 From: anand@eis.iisc.ernet.in (SVR Anand) Message-Id: <200212040738.NAA02825@eis.iisc.ernet.in> Subject: tcp flows hash function To: netdev@oss.sgi.com Date: Wed, 4 Dec 2002 13:08:31 +0530 (GMT+05:30) X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-archive-position: 1288 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: anand@eis.iisc.ernet.in Precedence: bulk X-list: netdev Content-Length: 1004 Lines: 33 Hi, I am writing a program that requires maintanence of per-connection statistics of many tcp connections based on multi-field classification. The problem is that of fast search and update when there are 1000s of connections. Couple of questions. 1. In many places of network code, I am seeing hashing being used. Can I also use the same ? From the recently published literature on packet classification, people are talking about some sort of trie variants. I would like to know if someone has done performance comparison of the algorithm used in Linux against other schemes. 2. While going through tcp_ipv4.c, I happened to see tcp_hashfn. What is the algorithm behind this ? static __inline__ int tcp_hashfn(__u32 laddr, __u16 lport, __u32 faddr, __u16 fport) { int h = ((laddr ^ lport) ^ (faddr ^ fport)); h ^= h>>16; h ^= h>>8; return h & (tcp_ehash_size - 1); } Any inputs from you will help me proceed further. Anand From srompf@isg.de Wed Dec 4 02:17:10 2002 Received: with ECARTIS (v1.0.0; list netdev); Wed, 04 Dec 2002 02:17:14 -0800 (PST) Received: from mail.isg.de (rzfoobar.is-asp.com [217.11.194.155]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB4AGxuR018902 for ; Wed, 4 Dec 2002 02:16:59 -0800 Received: from barkeeper.isg.de (barkeeper.is-asp.com [192.168.5.46]) by mail.isg.de (Postfix) with ESMTP id D2BAEEA90A8; Wed, 4 Dec 2002 10:44:58 +0100 (CET) Received: from isg.de (localhost.localdomain [127.0.0.1]) by barkeeper.isg.de (8.9.3/8.9.3) with ESMTP id KAA01050; Wed, 4 Dec 2002 10:44:58 +0100 Message-ID: <3DEDCE9A.61FDEBBC@isg.de> Date: Wed, 04 Dec 2002 10:44:58 +0100 From: Stefan Rompf X-Mailer: Mozilla 4.8 [en] (X11; U; Linux 2.4.20-rc4-sr i686) X-Accept-Language: en MIME-Version: 1.0 To: Jeff Garzik Cc: "David S. Miller" , netdev@oss.sgi.com Subject: Re: Patch resubmission: RFC2863 operstatus for 2.5.50 References: <3DE33D6D.25B9C9B4@isg.de> <20021126.021546.91313706.davem@redhat.com> <3DED2EA9.D812C881@isg.de> <3DED51E9.1080408@pobox.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-archive-position: 1289 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: srompf@isg.de Precedence: bulk X-list: netdev Content-Length: 529 Lines: 20 Hi, Jeff Garzik wrote: > Pardon my dumb question, but what parts of RFC2863 require kernel > additions over and above your link state patch? the kernel does not know LOWERLAYERDOWN, TESTING, DORMANT, UNKNOWN. They can be useful when drivers adopt to this scheme. > Your second patch I am less enthusiastic about than the first... :( Well, with your opinion I count two against two: I want it, Jamal has proposed the semantics, and Alexey doesn't want to waste a single bit of a netlink message for this. What now? Stefan From kiran@in.ibm.com Wed Dec 4 04:35:50 2002 Received: with ECARTIS (v1.0.0; list netdev); Wed, 04 Dec 2002 04:35:53 -0800 (PST) Received: from e3.ny.us.ibm.com (e3.ny.us.ibm.com [32.97.182.103]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB4CZnuR021984 for ; Wed, 4 Dec 2002 04:35:50 -0800 Received: from northrelay04.pok.ibm.com (northrelay04.pok.ibm.com [9.56.224.206]) by e3.ny.us.ibm.com (8.12.2/8.12.2) with ESMTP id gB4Ccrfw117300; Wed, 4 Dec 2002 07:38:53 -0500 Received: from asterix.in.ibm.com (asterix.in.ibm.com [9.182.12.249]) by northrelay04.pok.ibm.com (8.12.3/NCO/VER6.4) with ESMTP id gB4CclpG200538; Wed, 4 Dec 2002 07:38:49 -0500 Received: (from kirant@localhost) by asterix.in.ibm.com (8.11.2/8.11.2) id gB4CbXj21415; Wed, 4 Dec 2002 18:07:33 +0530 Date: Wed, 4 Dec 2002 18:07:32 +0530 From: Ravikiran G Thirumalai To: linux-kernel@vger.kernel.org Cc: netdev , "David S. Miller " , Andrew Morton , dipankar@in.ibm.com Subject: Re: [patch] Change Networking mibs to use kmalloc_percpu -- 2/3 Message-ID: <20021204180732.D17375@in.ibm.com> References: <20021204180510.C17375@in.ibm.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20021204180510.C17375@in.ibm.com>; from kiran@in.ibm.com on Wed, Dec 04, 2002 at 06:05:10PM +0530 X-archive-position: 1290 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kiran@in.ibm.com Precedence: bulk X-list: netdev Content-Length: 9989 Lines: 302 This patch is 2 of 3 D: Name: ipv6mibs-2.5.50-1.patch D: Author: Ravikiran Thirumalai D: Description: Changes ipv6 stats to use kmalloc_percpu from the traditional D: padded NR_CPUS arrays include/net/ipv6.h | 11 ++++-- net/ipv6/af_inet6.c | 83 +++++++++++++++++++++++++++++++++++++++++++++++ net/ipv6/icmp.c | 8 ++-- net/ipv6/ipv6_sockglue.c | 2 - net/ipv6/proc.c | 38 ++++++++++++--------- net/ipv6/udp.c | 2 - 6 files changed, 119 insertions(+), 25 deletions(-) diff -ruN -X dontdiff linux-2.5.50/include/net/ipv6.h mibstats-2.5.50/include/net/ipv6.h --- linux-2.5.50/include/net/ipv6.h Thu Nov 28 04:06:17 2002 +++ mibstats-2.5.50/include/net/ipv6.h Wed Dec 4 12:13:33 2002 @@ -19,6 +19,7 @@ #include #include #include +#include #define SIN6_LEN_RFC2133 24 @@ -105,15 +106,19 @@ /* sysctls */ extern int sysctl_ipv6_bindv6only; -extern struct ipv6_mib ipv6_statistics[NR_CPUS*2]; +DECLARE_SNMP_STAT(struct ipv6_mib, ipv6_statistics); #define IP6_INC_STATS(field) SNMP_INC_STATS(ipv6_statistics, field) #define IP6_INC_STATS_BH(field) SNMP_INC_STATS_BH(ipv6_statistics, field) #define IP6_INC_STATS_USER(field) SNMP_INC_STATS_USER(ipv6_statistics, field) -extern struct icmpv6_mib icmpv6_statistics[NR_CPUS*2]; +DECLARE_SNMP_STAT(struct icmpv6_mib, icmpv6_statistics); #define ICMP6_INC_STATS(field) SNMP_INC_STATS(icmpv6_statistics, field) #define ICMP6_INC_STATS_BH(field) SNMP_INC_STATS_BH(icmpv6_statistics, field) #define ICMP6_INC_STATS_USER(field) SNMP_INC_STATS_USER(icmpv6_statistics, field) -extern struct udp_mib udp_stats_in6[NR_CPUS*2]; +#define ICMP6_STATS_PTR_BH(field) \ + (& \ + ((per_cpu_ptr(icmpv6_statistics[0], smp_processor_id()))-> \ + field)) +DECLARE_SNMP_STAT(struct udp_mib, udp_stats_in6); #define UDP6_INC_STATS(field) SNMP_INC_STATS(udp_stats_in6, field) #define UDP6_INC_STATS_BH(field) SNMP_INC_STATS_BH(udp_stats_in6, field) #define UDP6_INC_STATS_USER(field) SNMP_INC_STATS_USER(udp_stats_in6, field) diff -ruN -X dontdiff linux-2.5.50/net/ipv6/af_inet6.c mibstats-2.5.50/net/ipv6/af_inet6.c --- linux-2.5.50/net/ipv6/af_inet6.c Thu Nov 28 04:06:02 2002 +++ mibstats-2.5.50/net/ipv6/af_inet6.c Wed Dec 4 12:13:33 2002 @@ -619,6 +619,81 @@ inet_unregister_protosw(p); } +static int __init init_ipv6_mibs(void) +{ + int i; + + ipv6_statistics[0] = kmalloc_percpu(sizeof (struct ipv6_mib), + GFP_KERNEL); + if (!ipv6_statistics[0]) + goto err_ip_mib0; + ipv6_statistics[1] = kmalloc_percpu(sizeof (struct ipv6_mib), + GFP_KERNEL); + if (!ipv6_statistics[1]) + goto err_ip_mib1; + + icmpv6_statistics[0] = kmalloc_percpu(sizeof (struct icmpv6_mib), + GFP_KERNEL); + if (!icmpv6_statistics[0]) + goto err_icmp_mib0; + icmpv6_statistics[1] = kmalloc_percpu(sizeof (struct icmpv6_mib), + GFP_KERNEL); + if (!icmpv6_statistics[1]) + goto err_icmp_mib1; + + udp_stats_in6[0] = kmalloc_percpu(sizeof (struct udp_mib), + GFP_KERNEL); + if (!udp_stats_in6[0]) + goto err_udp_mib0; + udp_stats_in6[1] = kmalloc_percpu(sizeof (struct udp_mib), + GFP_KERNEL); + if (!udp_stats_in6[1]) + goto err_udp_mib1; + + /* Zero all percpu versions of the mibs */ + for (i = 0; i < NR_CPUS; i++) { + if (cpu_possible(i)) { + memset(per_cpu_ptr(ipv6_statistics[0], i), 0, + sizeof (struct ipv6_mib)); + memset(per_cpu_ptr(ipv6_statistics[1], i), 0, + sizeof (struct ipv6_mib)); + memset(per_cpu_ptr(icmpv6_statistics[0], i), 0, + sizeof (struct icmpv6_mib)); + memset(per_cpu_ptr(icmpv6_statistics[1], i), 0, + sizeof (struct icmpv6_mib)); + memset(per_cpu_ptr(udp_stats_in6[0], i), 0, + sizeof (struct udp_mib)); + memset(per_cpu_ptr(udp_stats_in6[1], i), 0, + sizeof (struct udp_mib)); + } + } + return 0; + +err_udp_mib1: + kfree_percpu(udp_stats_in6[0]); +err_udp_mib0: + kfree_percpu(icmpv6_statistics[1]); +err_icmp_mib1: + kfree_percpu(icmpv6_statistics[0]); +err_icmp_mib0: + kfree_percpu(ipv6_statistics[1]); +err_ip_mib1: + kfree_percpu(ipv6_statistics[0]); +err_ip_mib0: + return -ENOMEM; + +} + +static void __exit cleanup_ipv6_mibs(void) +{ + kfree_percpu(ipv6_statistics[0]); + kfree_percpu(ipv6_statistics[1]); + kfree_percpu(icmpv6_statistics[0]); + kfree_percpu(icmpv6_statistics[1]); + kfree_percpu(udp_stats_in6[0]); + kfree_percpu(udp_stats_in6[1]); +} + static int __init inet6_init(void) { struct sk_buff *dummy_skb; @@ -668,6 +743,11 @@ * be able to create sockets. (?? is this dangerous ??) */ (void) sock_register(&inet6_family_ops); + + /* Initialise ipv6 mibs */ + err = init_ipv6_mibs(); + if (err) + goto init_mib_fail; /* * ipngwg API draft makes clear that the correct semantics @@ -735,6 +815,8 @@ #if defined(MODULE) && defined(CONFIG_SYSCTL) ipv6_sysctl_unregister(); #endif + cleanup_ipv6_mibs(); +init_mib_fail: return err; } module_init(inet6_init); @@ -765,6 +847,7 @@ #ifdef CONFIG_SYSCTL ipv6_sysctl_unregister(); #endif + cleanup_ipv6_mibs(); } module_exit(inet6_exit); #endif /* MODULE */ diff -ruN -X dontdiff linux-2.5.50/net/ipv6/icmp.c mibstats-2.5.50/net/ipv6/icmp.c --- linux-2.5.50/net/ipv6/icmp.c Thu Nov 28 04:05:55 2002 +++ mibstats-2.5.50/net/ipv6/icmp.c Wed Dec 4 12:13:33 2002 @@ -65,7 +65,7 @@ #include #include -struct icmpv6_mib icmpv6_statistics[NR_CPUS*2]; +DEFINE_SNMP_STAT(struct icmpv6_mib, icmpv6_statistics); /* * ICMP socket for flow control. @@ -377,7 +377,7 @@ ip6_build_xmit(sk, icmpv6_getfrag, &msg, &fl, len, NULL, -1, MSG_DONTWAIT); if (type >= ICMPV6_DEST_UNREACH && type <= ICMPV6_PARAMPROB) - (&(icmpv6_statistics[smp_processor_id()*2].Icmp6OutDestUnreachs))[type-1]++; + ICMP6_STATS_PTR_BH(Icmp6OutDestUnreachs) [type-1]++; ICMP6_INC_STATS_BH(Icmp6OutMsgs); out: icmpv6_xmit_unlock(); @@ -539,9 +539,9 @@ type = hdr->icmp6_type; if (type >= ICMPV6_DEST_UNREACH && type <= ICMPV6_PARAMPROB) - (&icmpv6_statistics[smp_processor_id()*2].Icmp6InDestUnreachs)[type-ICMPV6_DEST_UNREACH]++; + ICMP6_STATS_PTR_BH(Icmp6InDestUnreachs)[type-ICMPV6_DEST_UNREACH]++; else if (type >= ICMPV6_ECHO_REQUEST && type <= NDISC_REDIRECT) - (&icmpv6_statistics[smp_processor_id()*2].Icmp6InEchos)[type-ICMPV6_ECHO_REQUEST]++; + ICMP6_STATS_PTR_BH(Icmp6InEchos)[type-ICMPV6_ECHO_REQUEST]++; switch (type) { case ICMPV6_ECHO_REQUEST: diff -ruN -X dontdiff linux-2.5.50/net/ipv6/ipv6_sockglue.c mibstats-2.5.50/net/ipv6/ipv6_sockglue.c --- linux-2.5.50/net/ipv6/ipv6_sockglue.c Thu Nov 28 04:05:49 2002 +++ mibstats-2.5.50/net/ipv6/ipv6_sockglue.c Wed Dec 4 12:13:33 2002 @@ -51,7 +51,7 @@ #include -struct ipv6_mib ipv6_statistics[NR_CPUS*2]; +DEFINE_SNMP_STAT(struct ipv6_mib, ipv6_statistics); static struct packet_type ipv6_packet_type = { diff -ruN -X dontdiff linux-2.5.50/net/ipv6/proc.c mibstats-2.5.50/net/ipv6/proc.c --- linux-2.5.50/net/ipv6/proc.c Thu Nov 28 04:05:58 2002 +++ mibstats-2.5.50/net/ipv6/proc.c Wed Dec 4 12:13:33 2002 @@ -59,11 +59,11 @@ static struct snmp6_item { char *name; - unsigned long *ptr; - int mibsize; + void **mib; + int offset; } snmp6_list[] = { /* ipv6 mib according to draft-ietf-ipngwg-ipv6-mib-04 */ -#define SNMP6_GEN(x) { #x , &ipv6_statistics[0].x, sizeof(struct ipv6_mib)/sizeof(unsigned long) } +#define SNMP6_GEN(x) { #x , (void **)ipv6_statistics, offsetof(struct ipv6_mib, x) } SNMP6_GEN(Ip6InReceives), SNMP6_GEN(Ip6InHdrErrors), SNMP6_GEN(Ip6InTooBigErrors), @@ -97,7 +97,7 @@ OutRouterAdvertisements too. OutGroupMembQueries too. */ -#define SNMP6_GEN(x) { #x , &icmpv6_statistics[0].x, sizeof(struct icmpv6_mib)/sizeof(unsigned long) } +#define SNMP6_GEN(x) { #x , (void **)icmpv6_statistics, offsetof(struct icmpv6_mib, x) } SNMP6_GEN(Icmp6InMsgs), SNMP6_GEN(Icmp6InErrors), SNMP6_GEN(Icmp6InDestUnreachs), @@ -127,7 +127,7 @@ SNMP6_GEN(Icmp6OutGroupMembResponses), SNMP6_GEN(Icmp6OutGroupMembReductions), #undef SNMP6_GEN -#define SNMP6_GEN(x) { "Udp6" #x , &udp_stats_in6[0].Udp##x, sizeof(struct udp_mib)/sizeof(unsigned long) } +#define SNMP6_GEN(x) { "Udp6" #x , (void **)udp_stats_in6, offsetof(struct udp_mib, Udp##x) } SNMP6_GEN(InDatagrams), SNMP6_GEN(NoPorts), SNMP6_GEN(InErrors), @@ -135,17 +135,23 @@ #undef SNMP6_GEN }; -static unsigned long fold_field(unsigned long *ptr, int size) +static unsigned long +fold_field(void *mib[], int offt) { - unsigned long res = 0; - int i; - - for (i=0; i -struct udp_mib udp_stats_in6[NR_CPUS*2]; +DEFINE_SNMP_STAT(struct udp_mib, udp_stats_in6); /* XXX This is identical to tcp_ipv6.c:ipv6_rcv_saddr_equal, put * XXX it somewhere common. -DaveM From kiran@in.ibm.com Wed Dec 4 04:36:45 2002 Received: with ECARTIS (v1.0.0; list netdev); Wed, 04 Dec 2002 04:36:48 -0800 (PST) Received: from e6.ny.us.ibm.com (e6.ny.us.ibm.com [32.97.182.106]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB4CaiuR022197 for ; Wed, 4 Dec 2002 04:36:44 -0800 Received: from northrelay04.pok.ibm.com (northrelay04.pok.ibm.com [9.56.224.206]) by e6.ny.us.ibm.com (8.12.2/8.12.2) with ESMTP id gB4CdlvY150586; Wed, 4 Dec 2002 07:39:47 -0500 Received: from asterix.in.ibm.com (asterix.in.ibm.com [9.182.12.249]) by northrelay04.pok.ibm.com (8.12.3/NCO/VER6.4) with ESMTP id gB4CZrpG013754; Wed, 4 Dec 2002 07:35:54 -0500 Received: (from kirant@localhost) by asterix.in.ibm.com (8.11.2/8.11.2) id gB4CZA621397; Wed, 4 Dec 2002 18:05:10 +0530 Date: Wed, 4 Dec 2002 18:05:10 +0530 From: Ravikiran G Thirumalai To: linux-kernel@vger.kernel.org Cc: netdev , "David S. Miller " , Andrew Morton , dipankar@in.ibm.com Subject: [patch] Change Networking mibs to use kmalloc_percpu -- 1/3 Message-ID: <20021204180510.C17375@in.ibm.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i X-archive-position: 1291 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kiran@in.ibm.com Precedence: bulk X-list: netdev Content-Length: 26171 Lines: 758 Here's a patchset to enable networking mibs to use kmalloc_percpu instead of the traditional padded NR_CPUS arrays. Advantages: 1. Removes NR_CPUS bloat due to static definition 2. Can support node local allocation 3. Will work with modules The changes have been split for ipv4, ipv6 and sctp. We can later do away with the __pad and associated logic duing proc reporting (in the mibs) if this patchset is acceptable. This patch is 1 of 3 D: Name: ipv4mibs-2.5.50-1.patch D: Author: Ravikiran Thirumalai D: Description: Changes ipv4 stats to use kmalloc_percpu from the traditional D: padded NR_CPUS arrays include/net/icmp.h | 15 ++++++- include/net/ip.h | 6 +- include/net/snmp.h | 39 +++++++++++++------ include/net/tcp.h | 6 +- include/net/udp.h | 2 net/ipv4/af_inet.c | 61 +++++++++++++++++++++++++++++ net/ipv4/icmp.c | 96 ++++++++++++++++++++++------------------------- net/ipv4/ip_input.c | 2 net/ipv4/proc.c | 37 +++++++++--------- net/ipv4/tcp.c | 13 +++--- net/ipv4/tcp_input.c | 4 - net/ipv4/tcp_minisocks.c | 4 - net/ipv4/tcp_timer.c | 3 - net/ipv4/udp.c | 2 14 files changed, 190 insertions(+), 100 deletions(-) diff -ruN -X dontdiff linux-2.5.50/include/net/icmp.h mibstats-2.5.50/include/net/icmp.h --- linux-2.5.50/include/net/icmp.h Thu Nov 28 04:06:18 2002 +++ mibstats-2.5.50/include/net/icmp.h Wed Dec 4 12:13:33 2002 @@ -23,6 +23,7 @@ #include #include +#include #include struct icmp_err { @@ -31,10 +32,22 @@ }; extern struct icmp_err icmp_err_convert[]; -extern struct icmp_mib icmp_statistics[NR_CPUS*2]; +DECLARE_SNMP_STAT(struct icmp_mib, icmp_statistics); #define ICMP_INC_STATS(field) SNMP_INC_STATS(icmp_statistics, field) #define ICMP_INC_STATS_BH(field) SNMP_INC_STATS_BH(icmp_statistics, field) #define ICMP_INC_STATS_USER(field) SNMP_INC_STATS_USER(icmp_statistics, field) +#define ICMP_INC_STATS_FIELD(offt) \ + (*((unsigned long *) ((void *) \ + per_cpu_ptr(icmp_statistics[!in_softirq()],\ + smp_processor_id())) + offt))++; +#define ICMP_INC_STATS_BH_FIELD(offt) \ + (*((unsigned long *) ((void *) \ + per_cpu_ptr(icmp_statistics[0], \ + smp_processor_id())) + offt))++; +#define ICMP_INC_STATS_USER_FIELD(offt) \ + (*((unsigned long *) ((void *) \ + per_cpu_ptr(icmp_statistics[1], \ + smp_processor_id())) + offt))++; extern void icmp_send(struct sk_buff *skb_in, int type, int code, u32 info); extern int icmp_rcv(struct sk_buff *skb); diff -ruN -X dontdiff linux-2.5.50/include/net/ip.h mibstats-2.5.50/include/net/ip.h --- linux-2.5.50/include/net/ip.h Thu Nov 28 04:06:15 2002 +++ mibstats-2.5.50/include/net/ip.h Wed Dec 4 12:13:33 2002 @@ -149,14 +149,16 @@ }; extern struct ipv4_config ipv4_config; -extern struct ip_mib ip_statistics[NR_CPUS*2]; +DECLARE_SNMP_STAT(struct ip_mib, ip_statistics); #define IP_INC_STATS(field) SNMP_INC_STATS(ip_statistics, field) #define IP_INC_STATS_BH(field) SNMP_INC_STATS_BH(ip_statistics, field) #define IP_INC_STATS_USER(field) SNMP_INC_STATS_USER(ip_statistics, field) -extern struct linux_mib net_statistics[NR_CPUS*2]; +DECLARE_SNMP_STAT(struct linux_mib, net_statistics); #define NET_INC_STATS(field) SNMP_INC_STATS(net_statistics, field) #define NET_INC_STATS_BH(field) SNMP_INC_STATS_BH(net_statistics, field) #define NET_INC_STATS_USER(field) SNMP_INC_STATS_USER(net_statistics, field) +#define NET_ADD_STATS_BH(field, adnd) SNMP_ADD_STATS_BH(net_statistics, field, adnd) +#define NET_ADD_STATS_USER(field, adnd) SNMP_ADD_STATS_USER(net_statistics, field, adnd) extern int sysctl_local_port_range[2]; extern int sysctl_ip_default_ttl; diff -ruN -X dontdiff linux-2.5.50/include/net/snmp.h mibstats-2.5.50/include/net/snmp.h --- linux-2.5.50/include/net/snmp.h Thu Nov 28 04:05:57 2002 +++ mibstats-2.5.50/include/net/snmp.h Wed Dec 4 12:13:33 2002 @@ -62,7 +62,7 @@ unsigned long IpFragFails; unsigned long IpFragCreates; unsigned long __pad[0]; -} ____cacheline_aligned; +}; struct ipv6_mib { @@ -89,7 +89,7 @@ unsigned long Ip6InMcastPkts; unsigned long Ip6OutMcastPkts; unsigned long __pad[0]; -} ____cacheline_aligned; +}; struct icmp_mib { @@ -121,7 +121,7 @@ unsigned long IcmpOutAddrMaskReps; unsigned long dummy; unsigned long __pad[0]; -} ____cacheline_aligned; +}; struct icmpv6_mib { @@ -159,7 +159,7 @@ unsigned long Icmp6OutGroupMembResponses; unsigned long Icmp6OutGroupMembReductions; unsigned long __pad[0]; -} ____cacheline_aligned; +}; struct tcp_mib { @@ -178,7 +178,7 @@ unsigned long TcpInErrs; unsigned long TcpOutRsts; unsigned long __pad[0]; -} ____cacheline_aligned; +}; struct udp_mib { @@ -187,7 +187,7 @@ unsigned long UdpInErrors; unsigned long UdpOutDatagrams; unsigned long __pad[0]; -} ____cacheline_aligned; +}; /* draft-ietf-sigtran-sctp-mib-07.txt */ struct sctp_mib @@ -216,7 +216,7 @@ unsigned long SctpValCookieLife; unsigned long SctpMaxInitRetr; unsigned long __pad[0]; -} ____cacheline_aligned; +}; struct linux_mib { @@ -286,7 +286,7 @@ unsigned long TCPAbortFailed; unsigned long TCPMemoryPressures; unsigned long __pad[0]; -} ____cacheline_aligned; +}; /* @@ -294,8 +294,25 @@ * addl $1,memory is atomic against interrupts (but atomic_inc would be overkill because of the lock * cycles). Wants new nonlocked_atomic_inc() primitives -AK */ -#define SNMP_INC_STATS(mib, field) ((mib)[2*smp_processor_id()+!in_softirq()].field++) -#define SNMP_INC_STATS_BH(mib, field) ((mib)[2*smp_processor_id()].field++) -#define SNMP_INC_STATS_USER(mib, field) ((mib)[2*smp_processor_id()+1].field++) +#define DEFINE_SNMP_STAT(type, name) \ + __typeof__(type) *name[2] +#define DECLARE_SNMP_STAT(type, name) \ + extern __typeof__(type) *name[2] + +#define SNMP_STAT_USRPTR(name) (name[0]) +#define SNMP_STAT_BHPTR(name) (name[1]) + +#define SNMP_INC_STATS_BH(mib, field) \ + (per_cpu_ptr(mib[0], smp_processor_id())->field++) +#define SNMP_INC_STATS_USER(mib, field) \ + (per_cpu_ptr(mib[1], smp_processor_id())->field++) +#define SNMP_INC_STATS(mib, field) \ + (per_cpu_ptr(mib[!in_softirq()], smp_processor_id())->field++) +#define SNMP_DEC_STATS(mib, field) \ + (per_cpu_ptr(mib[!in_softirq()], smp_processor_id())->field--) +#define SNMP_ADD_STATS_BH(mib, field, addend) \ + (per_cpu_ptr(mib[0], smp_processor_id())->field += addend) +#define SNMP_ADD_STATS_USER(mib, field, addend) \ + (per_cpu_ptr(mib[1], smp_processor_id())->field += addend) #endif diff -ruN -X dontdiff linux-2.5.50/include/net/tcp.h mibstats-2.5.50/include/net/tcp.h --- linux-2.5.50/include/net/tcp.h Thu Nov 28 04:05:50 2002 +++ mibstats-2.5.50/include/net/tcp.h Wed Dec 4 12:13:33 2002 @@ -28,6 +28,7 @@ #include #include #include +#include #include #include #if defined(CONFIG_IPV6) || defined (CONFIG_IPV6_MODULE) @@ -630,10 +631,11 @@ extern struct proto tcp_prot; -extern struct tcp_mib tcp_statistics[NR_CPUS*2]; +DECLARE_SNMP_STAT(struct tcp_mib, tcp_statistics); #define TCP_INC_STATS(field) SNMP_INC_STATS(tcp_statistics, field) #define TCP_INC_STATS_BH(field) SNMP_INC_STATS_BH(tcp_statistics, field) #define TCP_INC_STATS_USER(field) SNMP_INC_STATS_USER(tcp_statistics, field) +#define TCP_DEC_STATS(field) SNMP_DEC_STATS(tcp_statistics, field) extern void tcp_put_port(struct sock *sk); extern void __tcp_put_port(struct sock *sk); @@ -1399,7 +1401,7 @@ /* fall through */ default: if (oldstate==TCP_ESTABLISHED) - tcp_statistics[smp_processor_id()*2+!in_softirq()].TcpCurrEstab--; + TCP_DEC_STATS(TcpCurrEstab); } /* Change state AFTER socket is unhashed to avoid closed diff -ruN -X dontdiff linux-2.5.50/include/net/udp.h mibstats-2.5.50/include/net/udp.h --- linux-2.5.50/include/net/udp.h Thu Nov 28 04:06:17 2002 +++ mibstats-2.5.50/include/net/udp.h Wed Dec 4 12:13:33 2002 @@ -71,7 +71,7 @@ extern int udp_ioctl(struct sock *sk, int cmd, unsigned long arg); extern int udp_disconnect(struct sock *sk, int flags); -extern struct udp_mib udp_statistics[NR_CPUS*2]; +DECLARE_SNMP_STAT(struct udp_mib, udp_statistics); #define UDP_INC_STATS(field) SNMP_INC_STATS(udp_statistics, field) #define UDP_INC_STATS_BH(field) SNMP_INC_STATS_BH(udp_statistics, field) #define UDP_INC_STATS_USER(field) SNMP_INC_STATS_USER(udp_statistics, field) diff -ruN -X dontdiff linux-2.5.50/net/ipv4/af_inet.c mibstats-2.5.50/net/ipv4/af_inet.c --- linux-2.5.50/net/ipv4/af_inet.c Thu Nov 28 04:05:50 2002 +++ mibstats-2.5.50/net/ipv4/af_inet.c Wed Dec 4 12:13:33 2002 @@ -115,7 +115,7 @@ #include #endif -struct linux_mib net_statistics[NR_CPUS * 2]; +DEFINE_SNMP_STAT(struct linux_mib, net_statistics); #ifdef INET_REFCNT_DEBUG atomic_t inet_sock_nr; @@ -1055,6 +1055,59 @@ .handler = icmp_rcv, }; +static int __init init_ipv4_mibs(void) +{ + int i; + + net_statistics[0] = + kmalloc_percpu(sizeof (struct linux_mib), GFP_KERNEL); + net_statistics[1] = + kmalloc_percpu(sizeof (struct linux_mib), GFP_KERNEL); + ip_statistics[0] = kmalloc_percpu(sizeof (struct ip_mib), GFP_KERNEL); + ip_statistics[1] = kmalloc_percpu(sizeof (struct ip_mib), GFP_KERNEL); + icmp_statistics[0] = + kmalloc_percpu(sizeof (struct icmp_mib), GFP_KERNEL); + icmp_statistics[1] = + kmalloc_percpu(sizeof (struct icmp_mib), GFP_KERNEL); + tcp_statistics[0] = kmalloc_percpu(sizeof (struct tcp_mib), GFP_KERNEL); + tcp_statistics[1] = kmalloc_percpu(sizeof (struct tcp_mib), GFP_KERNEL); + udp_statistics[0] = kmalloc_percpu(sizeof (struct udp_mib), GFP_KERNEL); + udp_statistics[1] = kmalloc_percpu(sizeof (struct udp_mib), GFP_KERNEL); + if (! + (net_statistics[0] && net_statistics[1] && ip_statistics[0] + && ip_statistics[1] && tcp_statistics[0] && tcp_statistics[1] + && udp_statistics[0] && udp_statistics[1])) + return -ENOMEM; + + /* Set all the per cpu copies of the mibs to zero */ + for (i = 0; i < NR_CPUS; i++) { + if (cpu_possible(i)) { + memset(per_cpu_ptr(net_statistics[0], i), 0, + sizeof (struct linux_mib)); + memset(per_cpu_ptr(net_statistics[1], i), 0, + sizeof (struct linux_mib)); + memset(per_cpu_ptr(ip_statistics[0], i), 0, + sizeof (struct ip_mib)); + memset(per_cpu_ptr(ip_statistics[1], i), 0, + sizeof (struct ip_mib)); + memset(per_cpu_ptr(icmp_statistics[0], i), 0, + sizeof (struct icmp_mib)); + memset(per_cpu_ptr(icmp_statistics[1], i), 0, + sizeof (struct icmp_mib)); + memset(per_cpu_ptr(tcp_statistics[0], i), 0, + sizeof (struct tcp_mib)); + memset(per_cpu_ptr(tcp_statistics[1], i), 0, + sizeof (struct tcp_mib)); + memset(per_cpu_ptr(udp_statistics[0], i), 0, + sizeof (struct udp_mib)); + memset(per_cpu_ptr(udp_statistics[1], i), 0, + sizeof (struct udp_mib)); + } + } + + return 0; +} + int ipv4_proc_init(void); static int __init inet_init(void) @@ -1150,7 +1203,13 @@ #if defined(CONFIG_IP_MROUTE) ip_mr_init(); #endif + /* + * Initialise per-cpu ipv4 mibs + */ + if(init_ipv4_mibs()) + printk(KERN_CRIT "inet_init: Cannot init ipv4 mibs\n"); ; + ipv4_proc_init(); return 0; } diff -ruN -X dontdiff linux-2.5.50/net/ipv4/icmp.c mibstats-2.5.50/net/ipv4/icmp.c --- linux-2.5.50/net/ipv4/icmp.c Thu Nov 28 04:06:20 2002 +++ mibstats-2.5.50/net/ipv4/icmp.c Wed Dec 4 12:13:33 2002 @@ -113,7 +113,7 @@ /* * Statistics */ -struct icmp_mib icmp_statistics[NR_CPUS * 2]; +DEFINE_SNMP_STAT(struct icmp_mib, icmp_statistics); /* An array of errno for error messages from dest unreach. */ /* RFC 1122: 3.2.2.1 States that NET_UNREACH, HOS_UNREACH and SR_FAIELD MUST be considered 'transient errs'. */ @@ -213,12 +213,11 @@ * ICMP control array. This specifies what to do with each ICMP. */ -struct icmp_control -{ - unsigned long *output; /* Address to increment on output */ - unsigned long *input; /* Address to increment on input */ +struct icmp_control { + int output_off; /* Field offset for increment on output */ + int input_off; /* Field offset for increment on input */ void (*handler)(struct sk_buff *skb); - short error; /* This ICMP is classed as an error message */ + short error; /* This ICMP is classed as an error message */ }; static struct icmp_control icmp_pointers[NR_ICMP_TYPES+1]; @@ -343,10 +342,7 @@ static void icmp_out_count(int type) { if (type <= NR_ICMP_TYPES) { - (icmp_pointers[type].output)[(smp_processor_id() * 2 + - !in_softirq()) * - sizeof(struct icmp_mib) / - sizeof(unsigned long)]++; + ICMP_INC_STATS_FIELD(icmp_pointers[type].output_off); ICMP_INC_STATS(IcmpOutMsgs); } } @@ -1005,9 +1001,7 @@ } } - icmp_pointers[icmph->type].input[smp_processor_id() * 2 * - sizeof(struct icmp_mib) / - sizeof(unsigned long)]++; + ICMP_INC_STATS_BH_FIELD(icmp_pointers[icmph->type].input_off); (icmp_pointers[icmph->type].handler)(skb); drop: @@ -1024,122 +1018,122 @@ static struct icmp_control icmp_pointers[NR_ICMP_TYPES + 1] = { /* ECHO REPLY (0) */ [0] = { - .output = &icmp_statistics[0].IcmpOutEchoReps, - .input = &icmp_statistics[0].IcmpInEchoReps, + .output_off = offsetof(struct icmp_mib, IcmpOutEchoReps), + .input_off = offsetof(struct icmp_mib, IcmpInEchoReps), .handler = icmp_discard, }, [1] = { - .output = &icmp_statistics[0].dummy, - .input = &icmp_statistics[0].IcmpInErrors, + .output_off = offsetof(struct icmp_mib, dummy), + .input_off = offsetof(struct icmp_mib,IcmpInErrors), .handler = icmp_discard, .error = 1, }, [2] = { - .output = &icmp_statistics[0].dummy, - .input = &icmp_statistics[0].IcmpInErrors, + .output_off = offsetof(struct icmp_mib, dummy), + .input_off = offsetof(struct icmp_mib,IcmpInErrors), .handler = icmp_discard, .error = 1, }, /* DEST UNREACH (3) */ [3] = { - .output = &icmp_statistics[0].IcmpOutDestUnreachs, - .input = &icmp_statistics[0].IcmpInDestUnreachs, + .output_off = offsetof(struct icmp_mib, IcmpOutDestUnreachs), + .input_off = offsetof(struct icmp_mib, IcmpInDestUnreachs), .handler = icmp_unreach, .error = 1, }, /* SOURCE QUENCH (4) */ [4] = { - .output = &icmp_statistics[0].IcmpOutSrcQuenchs, - .input = &icmp_statistics[0].IcmpInSrcQuenchs, + .output_off = offsetof(struct icmp_mib, IcmpOutSrcQuenchs), + .input_off = offsetof(struct icmp_mib, IcmpInSrcQuenchs), icmp_unreach, .error = 1, }, /* REDIRECT (5) */ [5] = { - .output = &icmp_statistics[0].IcmpOutRedirects, - .input = &icmp_statistics[0].IcmpInRedirects, + .output_off = offsetof(struct icmp_mib, IcmpOutRedirects), + .input_off = offsetof(struct icmp_mib, IcmpInRedirects), .handler = icmp_redirect, .error = 1, }, [6] = { - .output = &icmp_statistics[0].dummy, - .input = &icmp_statistics[0].IcmpInErrors, + .output_off = offsetof(struct icmp_mib, dummy), + .input_off = offsetof(struct icmp_mib, IcmpInErrors), .handler = icmp_discard, .error = 1, }, [7] = { - .output = &icmp_statistics[0].dummy, - .input = &icmp_statistics[0].IcmpInErrors, + .output_off = offsetof(struct icmp_mib, dummy), + .input_off = offsetof(struct icmp_mib, IcmpInErrors), .handler = icmp_discard, .error = 1, }, /* ECHO (8) */ [8] = { - .output = &icmp_statistics[0].IcmpOutEchos, - .input = &icmp_statistics[0].IcmpInEchos, + .output_off = offsetof(struct icmp_mib, IcmpOutEchos), + .input_off = offsetof(struct icmp_mib, IcmpInEchos), .handler = icmp_echo, .error = 0, }, [9] = { - .output = &icmp_statistics[0].dummy, - .input = &icmp_statistics[0].IcmpInErrors, + .output_off = offsetof(struct icmp_mib, dummy), + .input_off = offsetof(struct icmp_mib, IcmpInErrors), .handler = icmp_discard, .error = 1, }, [10] = { - .output = &icmp_statistics[0].dummy, - .input = &icmp_statistics[0].IcmpInErrors, + .output_off = offsetof(struct icmp_mib, dummy), + .input_off = offsetof(struct icmp_mib, IcmpInErrors), .handler = icmp_discard, .error = 1, }, /* TIME EXCEEDED (11) */ [11] = { - .output = &icmp_statistics[0].IcmpOutTimeExcds, - .input = &icmp_statistics[0].IcmpInTimeExcds, + .output_off = offsetof(struct icmp_mib, IcmpOutTimeExcds), + .input_off = offsetof(struct icmp_mib,IcmpInTimeExcds), .handler = icmp_unreach, .error = 1, }, /* PARAMETER PROBLEM (12) */ [12] = { - .output = &icmp_statistics[0].IcmpOutParmProbs, - .input = &icmp_statistics[0].IcmpInParmProbs, + .output_off = offsetof(struct icmp_mib, IcmpOutParmProbs), + .input_off = offsetof(struct icmp_mib, IcmpInParmProbs), .handler = icmp_unreach, .error = 1, }, /* TIMESTAMP (13) */ [13] = { - .output = &icmp_statistics[0].IcmpOutTimestamps, - .input = &icmp_statistics[0].IcmpInTimestamps, + .output_off = offsetof(struct icmp_mib, IcmpOutTimestamps), + .input_off = offsetof(struct icmp_mib, IcmpInTimestamps), .handler = icmp_timestamp, }, /* TIMESTAMP REPLY (14) */ [14] = { - .output = &icmp_statistics[0].IcmpOutTimestampReps, - .input = &icmp_statistics[0].IcmpInTimestampReps, + .output_off = offsetof(struct icmp_mib, IcmpOutTimestampReps), + .input_off = offsetof(struct icmp_mib, IcmpInTimestampReps), .handler = icmp_discard, }, /* INFO (15) */ [15] = { - .output = &icmp_statistics[0].dummy, - .input = &icmp_statistics[0].dummy, + .output_off = offsetof(struct icmp_mib, dummy), + .input_off = offsetof(struct icmp_mib, dummy), .handler = icmp_discard, }, /* INFO REPLY (16) */ [16] = { - .output = &icmp_statistics[0].dummy, - .input = &icmp_statistics[0].dummy, + .output_off = offsetof(struct icmp_mib, dummy), + .input_off = offsetof(struct icmp_mib, dummy), .handler = icmp_discard, }, /* ADDR MASK (17) */ [17] = { - .output = &icmp_statistics[0].IcmpOutAddrMasks, - .input = &icmp_statistics[0].IcmpInAddrMasks, + .output_off = offsetof(struct icmp_mib, IcmpOutAddrMasks), + .input_off = offsetof(struct icmp_mib, IcmpInAddrMasks), .handler = icmp_address, }, /* ADDR MASK REPLY (18) */ [18] = { - .output = &icmp_statistics[0].IcmpOutAddrMaskReps, - .input = &icmp_statistics[0].IcmpInAddrMaskReps, + .output_off = offsetof(struct icmp_mib, IcmpOutAddrMaskReps), + .input_off = offsetof(struct icmp_mib, IcmpInAddrMaskReps), .handler = icmp_address_reply, } }; diff -ruN -X dontdiff linux-2.5.50/net/ipv4/ip_input.c mibstats-2.5.50/net/ipv4/ip_input.c --- linux-2.5.50/net/ipv4/ip_input.c Thu Nov 28 04:05:49 2002 +++ mibstats-2.5.50/net/ipv4/ip_input.c Wed Dec 4 12:13:33 2002 @@ -149,7 +149,7 @@ * SNMP management statistics */ -struct ip_mib ip_statistics[NR_CPUS*2]; +DEFINE_SNMP_STAT(struct ip_mib, ip_statistics); /* * Process Router Attention IP option diff -ruN -X dontdiff linux-2.5.50/net/ipv4/proc.c mibstats-2.5.50/net/ipv4/proc.c --- linux-2.5.50/net/ipv4/proc.c Thu Nov 28 04:05:48 2002 +++ mibstats-2.5.50/net/ipv4/proc.c Wed Dec 4 12:13:33 2002 @@ -86,16 +86,21 @@ .release = single_release, }; -static unsigned long fold_field(unsigned long *begin, int sz, int nr) +static unsigned long +fold_field(void *mib[], int nr) { unsigned long res = 0; int i; - sz /= sizeof(unsigned long); - for (i = 0; i < NR_CPUS; i++) { - res += begin[2 * i * sz + nr]; - res += begin[(2 * i + 1) * sz + nr]; + if (!cpu_possible(i)) + continue; + res += + *((unsigned long *) (((void *) per_cpu_ptr(mib[0], i)) + + sizeof (unsigned long) * nr)); + res += + *((unsigned long *) (((void *) per_cpu_ptr(mib[0], i)) + + sizeof (unsigned long) * nr)); } return res; } @@ -118,8 +123,7 @@ for (i = 0; i < offsetof(struct ip_mib, __pad) / sizeof(unsigned long); i++) seq_printf(seq, " %lu", - fold_field((unsigned long *)ip_statistics, - sizeof(struct ip_mib), i)); + fold_field((void **) ip_statistics, i)); seq_printf(seq, "\nIcmp: InMsgs InErrors InDestUnreachs InTimeExcds " "InParmProbs InSrcQuenchs InRedirects InEchos " @@ -132,8 +136,7 @@ for (i = 0; i < offsetof(struct icmp_mib, dummy) / sizeof(unsigned long); i++) seq_printf(seq, " %lu", - fold_field((unsigned long *)icmp_statistics, - sizeof(struct icmp_mib), i)); + fold_field((void **) icmp_statistics, i)); seq_printf(seq, "\nTcp: RtoAlgorithm RtoMin RtoMax MaxConn ActiveOpens " "PassiveOpens AttemptFails EstabResets CurrEstab " @@ -142,17 +145,15 @@ for (i = 0; i < offsetof(struct tcp_mib, __pad) / sizeof(unsigned long); i++) seq_printf(seq, " %lu", - fold_field((unsigned long *)tcp_statistics, - sizeof(struct tcp_mib), i)); + fold_field((void **) tcp_statistics, i)); seq_printf(seq, "\nUdp: InDatagrams NoPorts InErrors OutDatagrams\n" "Udp:"); for (i = 0; i < offsetof(struct udp_mib, __pad) / sizeof(unsigned long); i++) - seq_printf(seq, " %lu", - fold_field((unsigned long *)udp_statistics, - sizeof(struct udp_mib), i)); + seq_printf(seq, " %lu", + fold_field((void **) udp_statistics, i)); seq_putc(seq, '\n'); return 0; @@ -206,10 +207,10 @@ " TCPAbortFailed TCPMemoryPressures\n" "TcpExt:"); for (i = 0; - i < offsetof(struct linux_mib, __pad) / sizeof(unsigned long); i++) - seq_printf(seq, " %lu", - fold_field((unsigned long *)net_statistics, - sizeof(struct linux_mib), i)); + i < offsetof(struct linux_mib, __pad) / sizeof(unsigned long); + i++) + seq_printf(seq, " %lu", + fold_field((void **) net_statistics, i)); seq_putc(seq, '\n'); return 0; } diff -ruN -X dontdiff linux-2.5.50/net/ipv4/tcp.c mibstats-2.5.50/net/ipv4/tcp.c --- linux-2.5.50/net/ipv4/tcp.c Thu Nov 28 04:05:54 2002 +++ mibstats-2.5.50/net/ipv4/tcp.c Wed Dec 4 12:13:33 2002 @@ -258,13 +258,15 @@ #include #include #include +#include + #include #include int sysctl_tcp_fin_timeout = TCP_FIN_TIMEOUT; -struct tcp_mib tcp_statistics[NR_CPUS * 2]; +DEFINE_SNMP_STAT(struct tcp_mib, tcp_statistics); kmem_cache_t *tcp_openreq_cachep; kmem_cache_t *tcp_bucket_cachep; @@ -1395,8 +1397,7 @@ struct sk_buff *skb; struct tcp_opt *tp = tcp_sk(sk); - net_statistics[smp_processor_id() * 2 + 1].TCPPrequeued += - skb_queue_len(&tp->ucopy.prequeue); + NET_ADD_STATS_USER(TCPPrequeued, skb_queue_len(&tp->ucopy.prequeue)); /* RX process wants to run with disabled BHs, though it is not * necessary */ @@ -1676,7 +1677,7 @@ /* __ Restore normal policy in scheduler __ */ if ((chunk = len - tp->ucopy.len) != 0) { - net_statistics[smp_processor_id() * 2 + 1].TCPDirectCopyFromBacklog += chunk; + NET_ADD_STATS_USER(TCPDirectCopyFromBacklog, chunk); len -= chunk; copied += chunk; } @@ -1687,7 +1688,7 @@ tcp_prequeue_process(sk); if ((chunk = len - tp->ucopy.len) != 0) { - net_statistics[smp_processor_id() * 2 + 1].TCPDirectCopyFromPrequeue += chunk; + NET_ADD_STATS_USER(TCPDirectCopyFromPrequeue, chunk); len -= chunk; copied += chunk; } @@ -1770,7 +1771,7 @@ tcp_prequeue_process(sk); if (copied > 0 && (chunk = len - tp->ucopy.len) != 0) { - net_statistics[smp_processor_id() * 2 + 1].TCPDirectCopyFromPrequeue += chunk; + NET_ADD_STATS_USER(TCPDirectCopyFromPrequeue, chunk); len -= chunk; copied += chunk; } diff -ruN -X dontdiff linux-2.5.50/net/ipv4/tcp_input.c mibstats-2.5.50/net/ipv4/tcp_input.c --- linux-2.5.50/net/ipv4/tcp_input.c Thu Nov 28 04:06:16 2002 +++ mibstats-2.5.50/net/ipv4/tcp_input.c Wed Dec 4 12:13:33 2002 @@ -3043,8 +3043,8 @@ /* First, purge the out_of_order queue. */ if (skb_queue_len(&tp->out_of_order_queue)) { - net_statistics[smp_processor_id() * 2].OfoPruned += - skb_queue_len(&tp->out_of_order_queue); + NET_ADD_STATS_BH(OfoPruned, + skb_queue_len(&tp->out_of_order_queue)); __skb_queue_purge(&tp->out_of_order_queue); /* Reset SACK state. A conforming SACK implementation will diff -ruN -X dontdiff linux-2.5.50/net/ipv4/tcp_minisocks.c mibstats-2.5.50/net/ipv4/tcp_minisocks.c --- linux-2.5.50/net/ipv4/tcp_minisocks.c Thu Nov 28 04:06:16 2002 +++ mibstats-2.5.50/net/ipv4/tcp_minisocks.c Wed Dec 4 12:13:33 2002 @@ -464,7 +464,7 @@ if ((tcp_tw_count -= killed) != 0) mod_timer(&tcp_tw_timer, jiffies+TCP_TWKILL_PERIOD); - net_statistics[smp_processor_id()*2].TimeWaited += killed; + NET_ADD_STATS_BH(TimeWaited, killed); out: spin_unlock(&tw_death_lock); } @@ -628,7 +628,7 @@ out: if ((tcp_tw_count -= killed) == 0) del_timer(&tcp_tw_timer); - net_statistics[smp_processor_id()*2].TimeWaitKilled += killed; + NET_ADD_STATS_BH(TimeWaitKilled, killed); spin_unlock(&tw_death_lock); } diff -ruN -X dontdiff linux-2.5.50/net/ipv4/tcp_timer.c mibstats-2.5.50/net/ipv4/tcp_timer.c --- linux-2.5.50/net/ipv4/tcp_timer.c Thu Nov 28 04:06:19 2002 +++ mibstats-2.5.50/net/ipv4/tcp_timer.c Wed Dec 4 12:13:33 2002 @@ -237,7 +237,8 @@ if (skb_queue_len(&tp->ucopy.prequeue)) { struct sk_buff *skb; - net_statistics[smp_processor_id()*2].TCPSchedulerFailed += skb_queue_len(&tp->ucopy.prequeue); + NET_ADD_STATS_BH(TCPSchedulerFailed, + skb_queue_len(&tp->ucopy.prequeue)); while ((skb = __skb_dequeue(&tp->ucopy.prequeue)) != NULL) sk->backlog_rcv(sk, skb); diff -ruN -X dontdiff linux-2.5.50/net/ipv4/udp.c mibstats-2.5.50/net/ipv4/udp.c --- linux-2.5.50/net/ipv4/udp.c Thu Nov 28 04:05:51 2002 +++ mibstats-2.5.50/net/ipv4/udp.c Wed Dec 4 12:13:33 2002 @@ -110,7 +110,7 @@ * Snmp MIB for the UDP layer */ -struct udp_mib udp_statistics[NR_CPUS*2]; +DEFINE_SNMP_STAT(struct udp_mib, udp_statistics); struct sock *udp_hash[UDP_HTABLE_SIZE]; rwlock_t udp_hash_lock = RW_LOCK_UNLOCKED; From kiran@in.ibm.com Wed Dec 4 04:37:05 2002 Received: with ECARTIS (v1.0.0; list netdev); Wed, 04 Dec 2002 04:37:07 -0800 (PST) Received: from e5.ny.us.ibm.com (e5.ny.us.ibm.com [32.97.182.105]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB4Cb5uR022366 for ; Wed, 4 Dec 2002 04:37:05 -0800 Received: from northrelay03.pok.ibm.com (northrelay03.pok.ibm.com [9.56.224.151]) by e5.ny.us.ibm.com (8.12.2/8.12.2) with ESMTP id gB4Ce8rb230850; Wed, 4 Dec 2002 07:40:08 -0500 Received: from asterix.in.ibm.com (asterix.in.ibm.com [9.182.12.249]) by northrelay03.pok.ibm.com (8.12.3/NCO/VER6.4) with ESMTP id gB4Ce4ot048652; Wed, 4 Dec 2002 07:40:05 -0500 Received: (from kirant@localhost) by asterix.in.ibm.com (8.11.2/8.11.2) id gB4CdM621450; Wed, 4 Dec 2002 18:09:22 +0530 Date: Wed, 4 Dec 2002 18:09:22 +0530 From: Ravikiran G Thirumalai To: linux-kernel@vger.kernel.org Cc: netdev , "David S. Miller " , Andrew Morton , dipankar@in.ibm.com Subject: Re: [patch] Change Networking mibs to use kmalloc_percpu -- 3/3 Message-ID: <20021204180922.E17375@in.ibm.com> References: <20021204180510.C17375@in.ibm.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20021204180510.C17375@in.ibm.com>; from kiran@in.ibm.com on Wed, Dec 04, 2002 at 06:05:10PM +0530 X-archive-position: 1292 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kiran@in.ibm.com Precedence: bulk X-list: netdev Content-Length: 3224 Lines: 108 This patch is 3 of 3 D: Name: sctpmibs-2.5.50-1.patch D: Author: Ravikiran Thirumalai D: Description: Changes sctp stats to use kmalloc_percpu from the traditional D: padded NR_CPUS arrays include/net/sctp/sctp.h | 2 +- net/sctp/protocol.c | 44 +++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 44 insertions(+), 2 deletions(-) diff -ruN -X dontdiff linux-2.5.50/include/net/sctp/sctp.h mibstats-2.5.50/include/net/sctp/sctp.h --- linux-2.5.50/include/net/sctp/sctp.h Thu Nov 28 04:05:53 2002 +++ mibstats-2.5.50/include/net/sctp/sctp.h Wed Dec 4 12:13:33 2002 @@ -203,7 +203,7 @@ #define SCTP_SOCK_SLEEP_POST(sk) SOCK_SLEEP_POST(sk) /* SCTP SNMP MIB stats handlers */ -extern struct sctp_mib sctp_statistics[NR_CPUS * 2]; +DECLARE_SNMP_STAT(struct sctp_mib, sctp_statistics); #define SCTP_INC_STATS(field) SNMP_INC_STATS(sctp_statistics, field) #define SCTP_INC_STATS_BH(field) SNMP_INC_STATS_BH(sctp_statistics, field) #define SCTP_INC_STATS_USER(field) SNMP_INC_STATS_USER(sctp_statistics, field) diff -ruN -X dontdiff linux-2.5.50/net/sctp/protocol.c mibstats-2.5.50/net/sctp/protocol.c --- linux-2.5.50/net/sctp/protocol.c Thu Nov 28 04:06:05 2002 +++ mibstats-2.5.50/net/sctp/protocol.c Wed Dec 4 12:13:33 2002 @@ -59,7 +59,7 @@ /* Global data structures. */ sctp_protocol_t sctp_proto; struct proc_dir_entry *proc_net_sctp; -struct sctp_mib sctp_statistics[NR_CPUS * 2]; +DEFINE_SNMP_STAT(struct sctp_mib, sctp_statistics); /* This is the global socket data structure used for responding to * the Out-of-the-blue (OOTB) packets. A control sock will be created @@ -597,6 +597,40 @@ } } +static int __init init_sctp_mibs(void) +{ + int i; + + sctp_statistics[0] = kmalloc_percpu(sizeof (struct sctp_mib), + GFP_KERNEL); + if (!sctp_statistics[0]) + return -ENOMEM; + sctp_statistics[1] = kmalloc_percpu(sizeof (struct sctp_mib), + GFP_KERNEL); + if (!sctp_statistics[1]) { + kfree_percpu(sctp_statistics[0]); + return -ENOMEM; + } + + /* Zero all percpu versions of the mibs */ + for (i = 0; i < NR_CPUS; i++) { + if (cpu_possible(i)) { + memset(per_cpu_ptr(sctp_statistics[0], i), 0, + sizeof (struct sctp_mib)); + memset(per_cpu_ptr(sctp_statistics[1], i), 0, + sizeof (struct sctp_mib)); + } + } + return 0; + +} + +static void cleanup_sctp_mibs(void) +{ + kfree_percpu(sctp_statistics[0]); + kfree_percpu(sctp_statistics[1]); +} + /* Initialize the universe into something sensible. */ int sctp_init(void) { @@ -610,6 +644,11 @@ /* Add SCTP to inetsw linked list. */ inet_register_protosw(&sctp_protosw); + /* Allocate and initialise sctp mibs. */ + status = init_sctp_mibs(); + if (status) + goto err_init_mibs; + /* Initialize proc fs directory. */ sctp_proc_init(); @@ -745,6 +784,8 @@ err_ahash_alloc: sctp_dbg_objcnt_exit(); sctp_proc_exit(); + cleanup_sctp_mibs(); +err_init_mibs: inet_del_protocol(&sctp_protocol, IPPROTO_SCTP); inet_unregister_protosw(&sctp_protosw); return status; @@ -774,6 +815,7 @@ sctp_dbg_objcnt_exit(); sctp_proc_exit(); + cleanup_sctp_mibs(); inet_del_protocol(&sctp_protocol, IPPROTO_SCTP); inet_unregister_protosw(&sctp_protosw); From hadi@cyberus.ca Wed Dec 4 04:56:21 2002 Received: with ECARTIS (v1.0.0; list netdev); Wed, 04 Dec 2002 04:56:22 -0800 (PST) Received: from cyberus.ca (mail.cyberus.ca [216.191.240.111]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB4CuKuR023212 for ; Wed, 4 Dec 2002 04:56:21 -0800 Received: from shell.cyberus.ca (shell [216.191.240.114]) by cyberus.ca (8.9.3/8.9.3/Cyberus Online Inc.) with ESMTP id HAA17854; Wed, 4 Dec 2002 07:59:29 -0500 (EST) Received: from localhost (hadi@localhost) by shell.cyberus.ca (8.11.6+Sun/8.11.6) with ESMTP id gB4CpYr03925; Wed, 4 Dec 2002 07:51:34 -0500 (EST) X-Authentication-Warning: shell.cyberus.ca: hadi owned process doing -bs Date: Wed, 4 Dec 2002 07:51:34 -0500 (EST) From: jamal To: Ronghua Zhang cc: Subject: Re: synchronization between bottom half and user context In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 1293 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev Content-Length: 657 Lines: 21 On Tue, 3 Dec 2002, Ronghua Zhang wrote: > Can someone tell me when lock should be used to synchronize bh and user > context? > > For examle: in tcp_accept(), lock_sock() is called to avoid any change > made by net_bh during its operation. But if tcp_accept() is called, it's > impossilbe that net_bh is also executing. (i think this is true for UP, > what about SMP?), then why need lock_sock()? > to serialize packets towards the socket; The linux network stack is threaded, you can have upto n packets to the same socket coming up the stack in parallel on an n-SMP machine; think of that and the fact that TCP data must be sequenced. cheers, jamal From hadi@cyberus.ca Wed Dec 4 05:10:55 2002 Received: with ECARTIS (v1.0.0; list netdev); Wed, 04 Dec 2002 05:10:59 -0800 (PST) Received: from cyberus.ca (mail.cyberus.ca [216.191.240.111]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB4DAsuR023738 for ; Wed, 4 Dec 2002 05:10:55 -0800 Received: from shell.cyberus.ca (shell [216.191.240.114]) by cyberus.ca (8.9.3/8.9.3/Cyberus Online Inc.) with ESMTP id IAA22127; Wed, 4 Dec 2002 08:14:02 -0500 (EST) Received: from localhost (hadi@localhost) by shell.cyberus.ca (8.11.6+Sun/8.11.6) with ESMTP id gB4D67G03951; Wed, 4 Dec 2002 08:06:07 -0500 (EST) X-Authentication-Warning: shell.cyberus.ca: hadi owned process doing -bs Date: Wed, 4 Dec 2002 08:06:07 -0500 (EST) From: jamal To: Stefan Rompf cc: , Subject: Re: Patch resubmission: RFC2863 operstatus for 2.5.49 In-Reply-To: <3DED3869.5E47D7A2@isg.de> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 1294 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev Content-Length: 1237 Lines: 34 On Wed, 4 Dec 2002, Stefan Rompf wrote: > Hi Jamal, > > > Just thought of something, it may be a little tricky but valuable and > > i am not quiet sure if it should part of your patch: We probably need to > > flush the qdiscs software queues; maybe even the DMA ring i.e simulate > > admin down followed by admin up. > > dev_close() is doing quite a lot of stuff, so we should do nothing more > than flush the qdiscs when the link comes up. But is it really useful? Indeed it is, infact i have been bitten by buffered packets confusing an upstream switch with buffered VRRP packets when someone stepped on a cable that i later reconnected. Typically about 30 seconds later an arp fixes the problem - if the queue had been properly flushed thered be no problem. I think maybe flushing all routes and neighbors pointing out that device is also useful. > Normally, the queues are short anyway to keep latencies low, and fifty > additional packets don't hurt. I rather think about clearing neighbor > tables and the route cache whenever the operstatus goes down. > > Anyway, implementation and usage of the notification should not mix up, > so it let it be a separate patch. Agreed; just queueing more work for you ;-> cheers, jamal From hadi@cyberus.ca Wed Dec 4 05:16:13 2002 Received: with ECARTIS (v1.0.0; list netdev); Wed, 04 Dec 2002 05:16:15 -0800 (PST) Received: from cyberus.ca (mail.cyberus.ca [216.191.240.111]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB4DGDuR027539 for ; Wed, 4 Dec 2002 05:16:13 -0800 Received: from shell.cyberus.ca (shell [216.191.240.114]) by cyberus.ca (8.9.3/8.9.3/Cyberus Online Inc.) with ESMTP id IAA23749; Wed, 4 Dec 2002 08:19:22 -0500 (EST) Received: from localhost (hadi@localhost) by shell.cyberus.ca (8.11.6+Sun/8.11.6) with ESMTP id gB4DBRG03961; Wed, 4 Dec 2002 08:11:27 -0500 (EST) X-Authentication-Warning: shell.cyberus.ca: hadi owned process doing -bs Date: Wed, 4 Dec 2002 08:11:27 -0500 (EST) From: jamal To: Jeff Garzik cc: Stefan Rompf , "David S. Miller" , Subject: Re: Patch resubmission: RFC2863 operstatus for 2.5.50 In-Reply-To: <3DED51E9.1080408@pobox.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 1296 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev Content-Length: 669 Lines: 23 On Tue, 3 Dec 2002, Jeff Garzik wrote: > Pardon my dumb question, but what parts of RFC2863 require kernel > additions over and above your link state patch? > > Your second patch I am less enthusiastic about than the first... :( > > I wonder if userspace cannot figure out ifOperStatus from existing data > we make available? > Stefans curtrent patch makes the info available via netlink. What dont you like about it Jeff? Take a quick look at RFC2863 and scan for IfAdminStatus and IfOperStatus. The modelling RFC2863 has is pretty good and thats what Stefan has followed (after we weeded a few crappy pieces off the RFC; we discussed on netdev). cheers, jamal From hadi@cyberus.ca Wed Dec 4 05:22:34 2002 Received: with ECARTIS (v1.0.0; list netdev); Wed, 04 Dec 2002 05:22:35 -0800 (PST) Received: from cyberus.ca (mail.cyberus.ca [216.191.240.111]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB4DMXuR028256 for ; Wed, 4 Dec 2002 05:22:34 -0800 Received: from shell.cyberus.ca (shell [216.191.240.114]) by cyberus.ca (8.9.3/8.9.3/Cyberus Online Inc.) with ESMTP id IAA25822; Wed, 4 Dec 2002 08:25:42 -0500 (EST) Received: from localhost (hadi@localhost) by shell.cyberus.ca (8.11.6+Sun/8.11.6) with ESMTP id gB4DHkk03973; Wed, 4 Dec 2002 08:17:47 -0500 (EST) X-Authentication-Warning: shell.cyberus.ca: hadi owned process doing -bs Date: Wed, 4 Dec 2002 08:17:46 -0500 (EST) From: jamal To: SVR Anand cc: Subject: Re: tcp flows hash function In-Reply-To: <200212040738.NAA02825@eis.iisc.ernet.in> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 1297 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev Content-Length: 1594 Lines: 53 Anand, why dont you give a shot at the comparisons and post your results. You have already extracted the hashing function; so write a small program in user space that you feed headers from real world webservers for example to tcp_hashfn(); see how well the hashing is distributed amongst the hash buckets; this should give you the rough number of lookups needed in the linux hash function. Then try some of the trie schemes with the same data. I'd be interested in your results. cheers, jamal On Wed, 4 Dec 2002, SVR Anand wrote: > Hi, > > I am writing a program that requires maintanence of per-connection statistics > of many tcp connections based on multi-field classification. The problem is > that of fast search and update when there are 1000s of connections. Couple of > questions. > > 1. In many places of network code, I am seeing hashing being used. Can I also > use the same ? > > >From the recently published literature on packet classification, people are > talking about some sort of trie variants. I would like to know if someone has > done performance comparison of the algorithm used in Linux against other > schemes. > > > 2. While going through tcp_ipv4.c, I happened to see tcp_hashfn. What is the > algorithm behind this ? > > static __inline__ int tcp_hashfn(__u32 laddr, __u16 lport, > __u32 faddr, __u16 fport) > { > int h = ((laddr ^ lport) ^ (faddr ^ fport)); > h ^= h>>16; > h ^= h>>8; > return h & (tcp_ehash_size - 1); > } > > > Any inputs from you will help me proceed further. > > Anand > > From yan@intruvert.com Wed Dec 4 06:57:35 2002 Received: with ECARTIS (v1.0.0; list netdev); Wed, 04 Dec 2002 06:57:40 -0800 (PST) Received: from ivexs1.intruvert.com (mx2.intruvert.com [65.209.235.21]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB4EvYuR003176 for ; Wed, 4 Dec 2002 06:57:35 -0800 Received: by ivexs1.intruvert.com with Internet Mail Service (5.5.2653.19) id ; Wed, 4 Dec 2002 07:00:37 -0800 Message-ID: From: Yan-Fa Li To: "'Ben Greear'" , Yan-Fa Li Cc: netdev@oss.sgi.com Subject: RE: TG3 Performance Question Date: Tue, 3 Dec 2002 13:04:51 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Disposition: inline Content-Type: text/plain Content-Transfer-Encoding: 7bit X-archive-position: 1298 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: yan@intruvert.com Precedence: bulk X-list: netdev Content-Length: 1642 Lines: 55 Yes, lspci: 00:08.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5701 Gigabit Ethernet (rev 15) Subsystem: 3Com Corporation 3C996B-T 1000BaseTX Flags: bus master, 66Mhz, medium devsel, latency 248, IRQ 10 Memory at f8200000 (64-bit, non-prefetchable) [size=64K] Capabilities: [40] PCI-X non-bridge device. Capabilities: [48] Power Management version 2 Capabilities: [50] Vital Product Data Capabilities: [58] Message Signalled Interrupts: 64bit+ Queue=0/3 Enable- would seem to indicate that it is 66/66. The latency is my fault, I set it manually to see if it would help performance. Unfortunately I don't think the E1000 with will with the AMD chipset, and I don't think Intel will be releasing a fix somehow. Do you have access to an AMD760MPX ? I would love it if someone could verify this is a working combination before I go out and spend more money. Y -----Original Message----- From: Ben Greear [mailto:greearb@candelatech.com] Sent: Tuesday, December 03, 2002 12:45 PM To: Yan-Fa Li Cc: netdev@oss.sgi.com Subject: Re: TG3 Performance Question Yan-Fa Li wrote: > Hello > > I'm trying to figure out some performance issues I have with the tg3 > driver. Are you sure your tg3 NIC is 66/64 PCI? For 66/64, try an e1000 NIC, you should be able to get 800Mbps or better out of it on such a system. Ben -- Ben Greear President of Candela Technologies Inc http://www.candelatech.com ScryMUD: http://scry.wanfear.com http://scry.wanfear.com/~greear [[HTML alternate version deleted]] From rz5b@cs.virginia.edu Wed Dec 4 07:28:11 2002 Received: with ECARTIS (v1.0.0; list netdev); Wed, 04 Dec 2002 07:28:12 -0800 (PST) Received: from ares.cs.Virginia.EDU (mail.cs.Virginia.EDU [128.143.137.19]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB4FSAuR003816 for ; Wed, 4 Dec 2002 07:28:10 -0800 Received: from bobbidi.cs.Virginia.EDU (bobbidi.cs.Virginia.EDU [128.143.67.48]) by ares.cs.Virginia.EDU (8.9.3+Sun/8.9.2/UVACS-2000040300) with ESMTP id KAA19829; Wed, 4 Dec 2002 10:31:15 -0500 (EST) Date: Wed, 4 Dec 2002 10:31:49 -0500 (EST) From: Ronghua Zhang To: jamal cc: netdev@oss.sgi.com Subject: Re: synchronization between bottom half and user context In-Reply-To: Message-ID: References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 1299 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: rz5b@cs.virginia.edu Precedence: bulk X-list: netdev Content-Length: 922 Lines: 30 Do you mean that net_bh can be running on several CPU at the same time? If it is juat a UP, can I say that socket system call and net_bh is always serialized even without lock_sock()? Ronghua On Wed, 4 Dec 2002, jamal wrote: > > > On Tue, 3 Dec 2002, Ronghua Zhang wrote: > > > Can someone tell me when lock should be used to synchronize bh and user > > context? > > > > For examle: in tcp_accept(), lock_sock() is called to avoid any change > > made by net_bh during its operation. But if tcp_accept() is called, it's > > impossilbe that net_bh is also executing. (i think this is true for UP, > > what about SMP?), then why need lock_sock()? > > > > to serialize packets towards the socket; The linux network stack > is threaded, you can have upto n packets to the same socket coming up the > stack in parallel on an n-SMP machine; think of that and the fact that TCP > data must be sequenced. > > cheers, > jamal > From davem@redhat.com Wed Dec 4 09:01:26 2002 Received: with ECARTIS (v1.0.0; list netdev); Wed, 04 Dec 2002 09:01:29 -0800 (PST) Received: from pizda.ninka.net (IDENT:root@pizda.ninka.net [216.101.162.242]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB4H1QuR006462 for ; Wed, 4 Dec 2002 09:01:26 -0800 Received: from localhost (IDENT:davem@localhost.localdomain [127.0.0.1]) by pizda.ninka.net (8.9.3/8.9.3) with ESMTP id JAA23638; Wed, 4 Dec 2002 09:01:53 -0800 Date: Wed, 04 Dec 2002 09:01:52 -0800 (PST) Message-Id: <20021204.090152.97851491.davem@redhat.com> To: kiran@in.ibm.com Cc: linux-kernel@vger.kernel.org, netdev@oss.sgi.com, akpm@digeo.com, dipankar@in.ibm.com Subject: Re: [patch] Change Networking mibs to use kmalloc_percpu -- 1/3 From: "David S. Miller" In-Reply-To: <20021204180510.C17375@in.ibm.com> References: <20021204180510.C17375@in.ibm.com> X-FalunGong: Information control. X-Mailer: Mew version 2.1 on Emacs 21.1 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-archive-position: 1300 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: davem@redhat.com Precedence: bulk X-list: netdev Content-Length: 530 Lines: 17 From: Ravikiran G Thirumalai Date: Wed, 4 Dec 2002 18:05:10 +0530 Here's a patchset to enable networking mibs to use kmalloc_percpu instead of the traditional padded NR_CPUS arrays. Advantages: 1. Removes NR_CPUS bloat due to static definition 2. Can support node local allocation 3. Will work with modules I totally support this work. Once the kmalloc percpu bits hit Linus's tree, just retransmit these diffs to me privately and I'll put them into my net-2.5 tree. Thanks. From greearb@candelatech.com Wed Dec 4 09:17:02 2002 Received: with ECARTIS (v1.0.0; list netdev); Wed, 04 Dec 2002 09:17:03 -0800 (PST) Received: from grok.yi.org (IDENT:XZJt15nWDGVGF6FpDZeC0ln1IIcQSRJg@dhcp101-dsl-usw4.w-link.net [208.161.125.101]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB4HH1uR007438 for ; Wed, 4 Dec 2002 09:17:01 -0800 Received: from candelatech.com (IDENT:rZ1/LYC0sTY0llXJv+oFL8TgbGBeTEBY@localhost.localdomain [127.0.0.1]) by grok.yi.org (8.11.6/8.11.2) with ESMTP id gB4HK7q30759; Wed, 4 Dec 2002 09:20:07 -0800 Message-ID: <3DEE3947.8000604@candelatech.com> Date: Wed, 04 Dec 2002 09:20:07 -0800 From: Ben Greear Organization: Candela Technologies User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2a) Gecko/20020910 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Yan-Fa Li CC: netdev@oss.sgi.com Subject: Re: TG3 Performance Question References: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 1301 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: greearb@candelatech.com Precedence: bulk X-list: netdev Content-Length: 452 Lines: 17 Yan-Fa Li wrote: > to see if it would help performance. Unfortunately I don't think the E1000 > with > will with the AMD chipset, and I don't think Intel will be releasing a fix The e1000 works with my dual AMD motherboard... Ben -- Ben Greear President of Candela Technologies Inc http://www.candelatech.com ScryMUD: http://scry.wanfear.com http://scry.wanfear.com/~greear From jgarzik@pobox.com Wed Dec 4 09:39:01 2002 Received: with ECARTIS (v1.0.0; list netdev); Wed, 04 Dec 2002 09:39:06 -0800 (PST) Received: from www.linux.org.uk (parcelfarce.linux.theplanet.co.uk [195.92.249.252]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB4Hd0uR011129 for ; Wed, 4 Dec 2002 09:39:01 -0800 Received: from nat-pool-rdu.redhat.com ([66.187.233.200] helo=pobox.com) by www.linux.org.uk with esmtp (Exim 3.33 #5) id 18JdXF-00010D-00; Wed, 04 Dec 2002 17:42:09 +0000 Message-ID: <3DEE3E6E.30407@pobox.com> Date: Wed, 04 Dec 2002 12:42:06 -0500 From: Jeff Garzik User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2) Gecko/20021202 X-Accept-Language: en-us, en MIME-Version: 1.0 To: jamal CC: Stefan Rompf , "David S. Miller" , netdev@oss.sgi.com Subject: Re: Patch resubmission: RFC2863 operstatus for 2.5.50 References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 1302 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jgarzik@pobox.com Precedence: bulk X-list: netdev Content-Length: 2264 Lines: 56 jamal wrote: > Stefans curtrent patch makes the info available via netlink. He has two patches: link change notification, and RFC2863 operstatus. I agree with the first one and support its inclusion; the second one I question its need. (just for context, I am referring to message <3DED2EA9.D812C881@isg.de> dated Dec 3) > What dont you like about it Jeff? Take a quick look at RFC2863 > and scan for IfAdminStatus and IfOperStatus. The modelling RFC2863 > has is pretty good and thats what Stefan has followed (after we weeded > a few crappy pieces off the RFC; we discussed on netdev). I had looked at ifAdminStatus and ifOperStatus before I posted :) My argument is, _after_ Stefan's link state patch is applied, why do we need the additional patch? [this question is meant to be delivered in an honest, not snide way...] For ifAdminStatus, you have "up", "down", and "testing" states. Since we have no concept of a testing state, if we eliminate that we have "up" and "down", two states we can obviously handle. For ifOperStatus, we have "up", "down" and "testing", which are applicable (or not) to Linux as with ifAdminStatus. Further we have states "dormant", "unknown", "notPresent", "lowerLayerDown". I'll discuss each of these in detail. "dormant" - not used in Stefan's patch, which I agree with. "unknown" - only used in Stefan's patch before interface is first up'd, which is IMO inaccurate. Accurate use of "up" and "down", to me, implies -no- use of "unknown" state. Because as soon as we are initialized, all ethernet details are known, thus "down" is more applicable. The Linux net stack's atomicity is such that leaking an "unknown" state would be a bug, too. "notPresent" - analagous to Linux's netif_device_xxx, and Stefan's patch acknowledges this. However, the use of netif_device_xxx in drivers is really only used when the hardware is suspended. If hardware goes away, the interface goes away too, almost immediately. "lowerLayerDown" - not used in Stefan's patch, which I agree with. So, Stefan's 2nd patch really only adds "unknown" and "notPresent" states to current behavior -- and the applicability of those states to Linux is IMO questionable. Questions/comments requested. Regards, Jeff From jgarzik@pobox.com Wed Dec 4 10:12:28 2002 Received: with ECARTIS (v1.0.0; list netdev); Wed, 04 Dec 2002 10:12:29 -0800 (PST) Received: from www.linux.org.uk (parcelfarce.linux.theplanet.co.uk [195.92.249.252]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB4ICRuR011880 for ; Wed, 4 Dec 2002 10:12:27 -0800 Received: from nat-pool-rdu.redhat.com ([66.187.233.200] helo=pobox.com) by www.linux.org.uk with esmtp (Exim 3.33 #5) id 18Je3c-0001kP-00; Wed, 04 Dec 2002 18:15:36 +0000 Message-ID: <3DEE463E.7050303@pobox.com> Date: Wed, 04 Dec 2002 13:15:26 -0500 From: Jeff Garzik User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2) Gecko/20021202 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Stefan Rompf CC: "David S. Miller" , netdev@oss.sgi.com Subject: Re: Patch resubmission: RFC2863 operstatus for 2.5.50 References: <3DE33D6D.25B9C9B4@isg.de> <20021126.021546.91313706.davem@redhat.com> <3DED2EA9.D812C881@isg.de> <3DED51E9.1080408@pobox.com> <3DEDCE9A.61FDEBBC@isg.de> In-Reply-To: <3DEDCE9A.61FDEBBC@isg.de> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 1304 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jgarzik@pobox.com Precedence: bulk X-list: netdev Content-Length: 370 Lines: 12 Stefan Rompf wrote: >>Your second patch I am less enthusiastic about than the first... :( > > > Well, with your opinion I count two against two: I want it, Jamal has > proposed the semantics, and Alexey doesn't want to waste a single bit of > a netlink message for this. Well, for generic net stack issues, I weight Jamal and Alexey's opinion more than my own ;-) From davem@redhat.com Wed Dec 4 11:39:01 2002 Received: with ECARTIS (v1.0.0; list netdev); Wed, 04 Dec 2002 11:39:07 -0800 (PST) Received: from pizda.ninka.net (IDENT:root@pizda.ninka.net [216.101.162.242]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB4Jd0uR014354 for ; Wed, 4 Dec 2002 11:39:01 -0800 Received: from localhost (IDENT:davem@localhost.localdomain [127.0.0.1]) by pizda.ninka.net (8.9.3/8.9.3) with ESMTP id LAA24250; Wed, 4 Dec 2002 11:39:24 -0800 Date: Wed, 04 Dec 2002 11:39:23 -0800 (PST) Message-Id: <20021204.113923.16680312.davem@redhat.com> To: srompf@isg.de Cc: netdev@oss.sgi.com Subject: Re: Patch resubmission: RFC2863 operstatus for 2.5.50 From: "David S. Miller" In-Reply-To: <3DED2EA9.D812C881@isg.de> References: <3DE33D6D.25B9C9B4@isg.de> <20021126.021546.91313706.davem@redhat.com> <3DED2EA9.D812C881@isg.de> X-FalunGong: Information control. X-Mailer: Mew version 2.1 on Emacs 21.1 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-archive-position: 1305 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: davem@redhat.com Precedence: bulk X-list: netdev Content-Length: 125 Lines: 6 I've applied your first patch to 2.5.x, I'm awaiting more discussion wrt. Jeff's comments on your second one :-) Thanks. From acme@conectiva.com.br Wed Dec 4 14:31:50 2002 Received: with ECARTIS (v1.0.0; list netdev); Wed, 04 Dec 2002 14:31:54 -0800 (PST) Received: from orion.netbank.com.br (orion.netbank.com.br [200.203.199.90]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB4MVmuR012927 for ; Wed, 4 Dec 2002 14:31:50 -0800 Received: from [200.181.171.189] (helo=brinquendo.conectiva.com.br) by orion.netbank.com.br with asmtp (Exim 3.33 #1) id 18Ji92-0003ZJ-00; Wed, 04 Dec 2002 20:37:29 -0200 Received: by brinquendo.conectiva.com.br (Postfix, from userid 500) id 7E7071966C; Wed, 4 Dec 2002 22:34:28 +0000 (UTC) Date: Wed, 4 Dec 2002 20:34:27 -0200 From: Arnaldo Carvalho de Melo To: "David S. Miller" Cc: kiran@in.ibm.com, linux-kernel@vger.kernel.org, netdev@oss.sgi.com, akpm@digeo.com, dipankar@in.ibm.com Subject: Re: [patch] Change Networking mibs to use kmalloc_percpu -- 1/3 Message-ID: <20021204223427.GA23578@conectiva.com.br> Mail-Followup-To: Arnaldo Carvalho de Melo , "David S. Miller" , kiran@in.ibm.com, linux-kernel@vger.kernel.org, netdev@oss.sgi.com, akpm@digeo.com, dipankar@in.ibm.com References: <20021204180510.C17375@in.ibm.com> <20021204.090152.97851491.davem@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20021204.090152.97851491.davem@redhat.com> User-Agent: Mutt/1.4i X-Url: http://advogato.org/person/acme X-archive-position: 1306 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: acme@conectiva.com.br Precedence: bulk X-list: netdev Content-Length: 777 Lines: 21 Em Wed, Dec 04, 2002 at 09:01:52AM -0800, David S. Miller escreveu: > From: Ravikiran G Thirumalai > Date: Wed, 4 Dec 2002 18:05:10 +0530 > > Here's a patchset to enable networking mibs to use kmalloc_percpu instead > of the traditional padded NR_CPUS arrays. > > Advantages: > 1. Removes NR_CPUS bloat due to static definition > 2. Can support node local allocation > 3. Will work with modules > > I totally support this work. Once the kmalloc percpu bits hit > Linus's tree, just retransmit these diffs to me privately and > I'll put them into my net-2.5 tree. Cool stuff! I was planning to macroise this so that things like this would be possible without source impact but now its just there, keep it up :-) - Arnaldo From cwalker@thezenith.com Wed Dec 4 15:29:48 2002 Received: with ECARTIS (v1.0.0; list netdev); Wed, 04 Dec 2002 15:29:52 -0800 (PST) Received: from whlexch.thezenith.com (host-65-119-25-226.thezenith.com [65.119.25.226] (may be forged)) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB4NTmuR013714 for ; Wed, 4 Dec 2002 15:29:48 -0800 Received: by whlexch.thezenith.com with Internet Mail Service (5.5.2653.19) id ; Wed, 4 Dec 2002 15:32:52 -0800 Message-ID: <1FB25D2C0FC3D11193C10050046E9254079A4952@whlexch.thezenith.com> From: "Walker, Caleb" To: "'netdev@oss.sgi.com'" Subject: Where do I ask Vortex Driver Questions? Date: Wed, 4 Dec 2002 15:32:51 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain X-archive-position: 1307 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: cwalker@thezenith.com Precedence: bulk X-list: netdev Content-Length: 921 Lines: 18 I apologize for taking your time with this email, but I am wondering if you could tell me where I can ask such questions as: How do I set the card up to run at 100MB/Half Duplex? The document is a little hard to understand for someone as ignorant as me. I get everything but what half of the hex numbers mean. Thank you, Caleb Walker ------------------------------------------------------------------------- This email and any attachments constitute non-public information for use only by intended recipient(s) and may contain confidential information. If you are not an intended recipient, please notify our Information Security Officer immediately at (941) 906-5844 and permanently delete the original of this email and any attachments, as well as any copies or printouts thereof.The unauthorized use, dissemination, distribution or reproduction of this email and any attachments is prohibited and may be unlawful. From bogdan.costescu@iwr.uni-heidelberg.de Thu Dec 5 07:07:24 2002 Received: with ECARTIS (v1.0.0; list netdev); Thu, 05 Dec 2002 07:07:26 -0800 (PST) Received: from mail.iwr.uni-heidelberg.de (mail.iwr.uni-heidelberg.de [129.206.104.30]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB5F7NuR006289 for ; Thu, 5 Dec 2002 07:07:24 -0800 Received: from kenzo.iwr.uni-heidelberg.de (kenzo.iwr.uni-heidelberg.de [129.206.120.29]) by mail.iwr.uni-heidelberg.de (8.11.1/8.11.1) with ESMTP id gB5FAaG24055; Thu, 5 Dec 2002 16:10:36 +0100 (MET) Received: from localhost (bogdan@localhost) by kenzo.iwr.uni-heidelberg.de (8.11.6/8.11.6) with ESMTP id gB5FAZl13945; Thu, 5 Dec 2002 16:10:35 +0100 Date: Thu, 5 Dec 2002 16:10:35 +0100 (CET) From: Bogdan Costescu To: "Walker, Caleb" cc: "'netdev@oss.sgi.com'" Subject: Re: Where do I ask Vortex Driver Questions? In-Reply-To: <1FB25D2C0FC3D11193C10050046E9254079A4952@whlexch.thezenith.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 1310 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: bogdan.costescu@iwr.uni-heidelberg.de Precedence: bulk X-list: netdev Content-Length: 859 Lines: 26 On Wed, 4 Dec 2002, Walker, Caleb wrote: > I apologize for taking your time with this email, but I am wondering if you > could tell me where I can ask such questions as: Most probably on vortex@scyld.com which is a list dedicated to vortex driver related discussions. > How do I set the card up to run at 100MB/Half Duplex? This is answered in Documentation/networking/vortex.txt in your kernel source tree or at http://www.scyld.com/network/vortex.html (depending on the driver that you're using). > The document is a little hard to understand for someone as ignorant as me. Err, which document ? -- Bogdan Costescu IWR - Interdisziplinaeres Zentrum fuer Wissenschaftliches Rechnen Universitaet Heidelberg, INF 368, D-69120 Heidelberg, GERMANY Telephone: +49 6221 54 8869, Telefax: +49 6221 54 8868 E-mail: Bogdan.Costescu@IWR.Uni-Heidelberg.De From rz5b@cs.virginia.edu Thu Dec 5 13:32:06 2002 Received: with ECARTIS (v1.0.0; list netdev); Thu, 05 Dec 2002 13:32:11 -0800 (PST) Received: from ares.cs.Virginia.EDU (ares.cs.Virginia.EDU [128.143.137.19]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB5LW5uR026609 for ; Thu, 5 Dec 2002 13:32:06 -0800 Received: from bobbidi.cs.Virginia.EDU (bobbidi.cs.Virginia.EDU [128.143.67.48]) by ares.cs.Virginia.EDU (8.9.3+Sun/8.9.2/UVACS-2000040300) with ESMTP id QAA18803; Thu, 5 Dec 2002 16:35:18 -0500 (EST) Date: Thu, 5 Dec 2002 16:36:09 -0500 (EST) From: Ronghua Zhang To: kernelnewbies@nl.linux.org cc: netdev@oss.sgi.com, , Subject: synchronization between net_bh and user-context Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 1311 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: rz5b@cs.virginia.edu Precedence: bulk X-list: netdev Content-Length: 1134 Lines: 30 I am reading the TCP/IP code of kernel 2.2.15 and doing some development based on it(yes, I know it's an old version, but I have to). I got a little confused about the synchronization between net_bh and user-context, and hope someone can help me out. Specifically, why the following is impossible? destroy_sock is called on CPU1, and a little bit later net_bh() is executed on CPU2, which will grab the pointer to the socket just before it's destroyed, and its later access becomes invalid. CPU 1 CPU2 destroy_sock() lock_sock() sk->sock_readers++ synchronize_bh(), no bh is running now net_bh() get called =>tcp_v4_rcv() sk = __tcp_v4_lookup(...) sk has not been destroyed tcp_v4_destroy_sock() kill_sk_now() free sk now sk has been destroyed if (!atomic_read(&sk->sock_readers)) <-- sk become invalid ronghua From Sam.Bingner@hickam.af.mil Thu Dec 5 15:54:55 2002 Received: with ECARTIS (v1.0.0; list netdev); Thu, 05 Dec 2002 15:54:58 -0800 (PST) Received: from hic-sr1.hickam.af.mil (HIC-SR1.hickam.af.mil [131.38.214.75]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB5NssuR004762 for ; Thu, 5 Dec 2002 15:54:55 -0800 Received: from fsknmd01.hickam.pacaf.ds.af.mil (FSKNMD01.hickam.af.mil [131.38.158.106]) by hic-sr1.hickam.af.mil (Switch-2.2.2/Switch-2.2.0) with ESMTP id gB5Nra409885; Thu, 5 Dec 2002 13:53:36 -1000 Received: by fsknmd01.hickam.pacaf.ds.af.mil with Internet Mail Service (5.5.2653.19) id ; Thu, 5 Dec 2002 23:50:51 -0000 Message-ID: From: Bingner Sam J Contractor PACAF CSS/SCHE To: "'Roberto Nibali'" , Phil Oester Cc: "David S. Miller" , "'ja@ssi.bg'" , "'linux-kernel@vger.kernel.org'" , "'netdev@oss.sgi.com'" Subject: RE: hidden interface (ARP) 2.4.20 Date: Thu, 5 Dec 2002 23:57:18 -0000 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" X-archive-position: 1312 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: Sam.Bingner@hickam.af.mil Precedence: bulk X-list: netdev Content-Length: 3175 Lines: 82 to risk getting jumped on again, I still don't see why an address that is -=ASSIGNED TO AN INTERFACE=- should be responded to on a completely different interface... if we wanted the ip address to be assigned to the system, there should be a pseudo interface that will work on any of the interfaces attached. Why assign an address to an interface if it would work just the same if you assigned it to the loopback adapter? Why would you assign an address to the loopback adapter if you wanted it to be accessed from the world? Anyways, just wasting my breath by expressing my point of view... have fun Also, if anybody has a link to something that explains how to do this using an alternate method, or usage for arp_filter... I'd appreciate it if you could email me... I've been searching for like 2 hours and I havn't found anything useful. Sam Bingner -----Original Message----- From: Roberto Nibali [mailto:ratz@drugphish.ch] Sent: Thursday, December 05, 2002 12:51 PM To: Phil Oester Cc: David S. Miller; Bingner Sam J Contractor PACAF CSS/SCHE; 'ja@ssi.bg'; 'linux-kernel@vger.kernel.org' Subject: Re: hidden interface (ARP) 2.4.20 Hello, First I would like to ask people not to post such patches to lkml but rather to the LVS list, because this affects only LVS so far and we cover all kernel versions pretty much up to date. Julian just needs to do the s/__constant_htons/htons/ fixes and upload the changes to his site ;) The inclusion of the hidden feature has been discussed almost to death on netdev (where these questions should have gone in the first place) and it was decided against inclusion of this patch for various reasons. Phil Oester wrote: > So we should enable netfilter for all x-hundred webservers we have? Or play games with routing tables? Yes. What is the problem? You need to setup the x-hundred webservers anyway, 2 routing entry lines certainly won't hurt. Yes, I understand that if you're in process of upgrading your webservers from 2.2.x to 2.4.x this is a bit of an additional pain. There are also other solutions to this arp problem, but please address this on the LVS mailinglist. > Why was something which: > > a) works > b) was present in 2.2.xx kernels > c) is trivial to include and doesn't seem to 'hurt' anything > > ripped from 2.4 kernels? http://marc.theaimsgroup.com/?t=95743539800002&r=1&w=2 > What some people fail to grasp is that _many_ people in the real world are using > the hidden flag in load balancing scenarios for its simplicity. > Removing it (without any particularly valid reason that anyone is > aware of) doesn't make much sense. Depends if it was a hack before that shouldn't have been there in the first place. In an evolutionary process things get optimized ... as has happened with the network stack code. > -Phil > > p.s. flame away, Dave Search the LVS and the netdev archives for constructive discussions about it. No need to flame anyone. But hey, if people keep coming up with this, DaveM and Alexey might get weak and put it back in 2.5.x :) Best regards, Roberto Nibali, ratz -- echo '[q]sa[ln0=aln256%Pln256/snlbx]sb3135071790101768542287578439snlbxq'|dc From zjp@iscas.ac.cn Thu Dec 5 17:18:00 2002 Received: with ECARTIS (v1.0.0; list netdev); Thu, 05 Dec 2002 17:18:05 -0800 (PST) Received: from mail.iscas.ac.cn ([159.226.5.56]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB61HxuR006397 for ; Thu, 5 Dec 2002 17:18:00 -0800 Received: (qmail 25948 invoked by uid 104); 6 Dec 2002 01:21:16 -0000 Received: from zjp@iscas.ac.cn by mail.iscas.ac.cn by uid 0 with qmail-scanner-1.14 (hbedv: 6.15.0.1. hbedv: operating system: Linux (glibc). hbedv: product version: 2.0.4. hbedv: engine version: 6.15.0.1. hbedv: packlib version: 2.0.0.8 (supports 19 formats). hbedv: vdf version: 6.15.0.7 (66928 recognized forms). hbedv: . hbedv: product: AntiVir Workstation. hbedv: key file: hbedv.key. hbedv: registered user: irene, 123. hbedv: serial number: 1001020203. hbedv: key expires: 31 May 2003. hbedv: run mode: PRIVATE. hbedv: . hbedv: product: AntiVir MailGate. hbedv: key file: hbedv.key. hbedv: registered user: irene, 123. hbedv: serial number: 1001020203. hbedv: key expires: 31 May 2003. hbedv: run mode: PRIVATE. hbedv: . hbedv: product: AntiVir (command line scanner). hbedv: key file: hbedv.key. hbedv: registered user: irene, 123. hbedv: serial number: 1001020203. hbedv: key expires: 31 May 2003. hbedv: run mode: PRIVATE. Clear:. Processed in 0.213769 secs); 06 Dec 2002 01:21:16 -0000 Received: from unknown (HELO zhengjp) (zjp@192.168.6.108) by mail.iscas.ac.cn with SMTP; 6 Dec 2002 01:21:16 -0000 Message-ID: <001801c29cc6$25688660$6c06a8c0@zhengjp> From: "Zheng Jianping" To: Subject: ip6_ra_control Date: Fri, 6 Dec 2002 09:23:55 +0800 MIME-Version: 1.0 X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2919.6700 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700 Content-Disposition: inline Content-Type: text/plain Content-Transfer-Encoding: 7bit X-archive-position: 1313 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: zjp@iscas.ac.cn Precedence: bulk X-list: netdev Content-Length: 191 Lines: 11 Hi, What's this function mean, and when&how to call it? int ip6_ra_control(struct sock *sk, int sel, void (*destructor)(struct sock *)) Thanks, Zheng [[HTML alternate version deleted]] From yoshfuji@linux-ipv6.org Thu Dec 5 17:53:09 2002 Received: with ECARTIS (v1.0.0; list netdev); Thu, 05 Dec 2002 17:53:11 -0800 (PST) Received: from yue.hongo.wide.ad.jp (yue.hongo.wide.ad.jp [203.178.139.94]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB61r8uR007339 for ; Thu, 5 Dec 2002 17:53:08 -0800 Received: from localhost (localhost [127.0.0.1]) by yue.hongo.wide.ad.jp (8.12.3+3.5Wbeta/8.12.3/Debian -4) with ESMTP id gB61vFGR011517; Fri, 6 Dec 2002 10:57:16 +0900 Date: Fri, 06 Dec 2002 10:57:15 +0900 (JST) Message-Id: <20021206.105715.03276611.yoshfuji@linux-ipv6.org> To: zjp@iscas.ac.cn Cc: netdev@oss.sgi.com Subject: Re: ip6_ra_control From: YOSHIFUJI Hideaki / =?iso-2022-jp?B?GyRCNUhGIzFRTEAbKEI=?= In-Reply-To: <001801c29cc6$25688660$6c06a8c0@zhengjp> References: <001801c29cc6$25688660$6c06a8c0@zhengjp> Organization: USAGI Project X-URL: http://www.yoshifuji.org/%7Ehideaki/ X-Fingerprint: 90 22 65 EB 1E CF 3A D1 0B DF 80 D8 48 07 F8 94 E0 62 0E EA X-PGP-Key-URL: http://www.yoshifuji.org/%7Ehideaki/hideaki@yoshifuji.org.asc X-Face: "5$Al-.M>NJ%a'@hhZdQm:."qn~PA^gq4o*>iCFToq*bAi#4FRtx}enhuQKz7fNqQz\BYU] $~O_5m-9'}MIs`XGwIEscw;e5b>n"B_?j/AkL~i/MEaZBLP X-Mailer: Mew version 2.2 on Emacs 20.7 / Mule 4.1 (AOI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-archive-position: 1314 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: yoshfuji@linux-ipv6.org Precedence: bulk X-list: netdev Content-Length: 525 Lines: 12 In article <001801c29cc6$25688660$6c06a8c0@zhengjp> (at Fri, 6 Dec 2002 09:23:55 +0800), "Zheng Jianping" says: > What's this function mean, and when&how to call it? > int ip6_ra_control(struct sock *sk, int sel, void (*destructor)(struct sock *)) setsockopt(sock, IPPPROTO_IPV6, IPV6_ROUTER_ALERT, &i, sizeof(i)) to receive packet with Router Alert Hop-by-Hop option on routers. -- Hideaki YOSHIFUJI @ USAGI Project GPG FP: 9022 65EB 1ECF 3AD1 0BDF 80D8 4807 F894 E062 0EEA From zjp@iscas.ac.cn Fri Dec 6 01:21:09 2002 Received: with ECARTIS (v1.0.0; list netdev); Fri, 06 Dec 2002 01:21:14 -0800 (PST) Received: from mail.iscas.ac.cn ([159.226.5.56]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB69L7uR016439 for ; Fri, 6 Dec 2002 01:21:09 -0800 Received: (qmail 6255 invoked by uid 104); 6 Dec 2002 09:24:26 -0000 Received: from zjp@iscas.ac.cn by mail.iscas.ac.cn by uid 0 with qmail-scanner-1.14 (hbedv: 6.15.0.1. hbedv: operating system: Linux (glibc). hbedv: product version: 2.0.4. hbedv: engine version: 6.15.0.1. hbedv: packlib version: 2.0.0.8 (supports 19 formats). hbedv: vdf version: 6.15.0.7 (66928 recognized forms). hbedv: . hbedv: product: AntiVir Workstation. hbedv: key file: hbedv.key. hbedv: registered user: irene, 123. hbedv: serial number: 1001020203. hbedv: key expires: 31 May 2003. hbedv: run mode: PRIVATE. hbedv: . hbedv: product: AntiVir MailGate. hbedv: key file: hbedv.key. hbedv: registered user: irene, 123. hbedv: serial number: 1001020203. hbedv: key expires: 31 May 2003. hbedv: run mode: PRIVATE. hbedv: . hbedv: product: AntiVir (command line scanner). hbedv: key file: hbedv.key. hbedv: registered user: irene, 123. hbedv: serial number: 1001020203. hbedv: key expires: 31 May 2003. hbedv: run mode: PRIVATE. Clear:. Processed in 0.217447 secs); 06 Dec 2002 09:24:26 -0000 Received: from unknown (HELO zhengjp) (zjp@192.168.6.108) by mail.iscas.ac.cn with SMTP; 6 Dec 2002 09:24:26 -0000 Message-ID: <004701c29d09$a58e7b40$6c06a8c0@zhengjp> From: "Zheng Jianping" To: Subject: How kernel send message to daemon? Date: Fri, 6 Dec 2002 17:27:07 +0800 MIME-Version: 1.0 X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2919.6700 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700 Content-Disposition: inline Content-Type: text/plain Content-Transfer-Encoding: 7bit X-archive-position: 1318 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: zjp@iscas.ac.cn Precedence: bulk X-list: netdev Content-Length: 206 Lines: 10 Hi, In IPv6 packet forwarding processing, the kernel receives a 'bad' message. Now I want the kernel inform the daemon( a routing protocl ) . Howto? Thanks, Zheng [[HTML alternate version deleted]] From jmorris@intercode.com.au Fri Dec 6 05:24:12 2002 Received: with ECARTIS (v1.0.0; list netdev); Fri, 06 Dec 2002 05:24:18 -0800 (PST) Received: from blackbird.intercode.com.au (IDENT:root@blackbird.intercode.com.au [203.32.101.10]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB6DO9uR028932 for ; Fri, 6 Dec 2002 05:24:10 -0800 Received: from localhost (jmorris@localhost) by blackbird.intercode.com.au (8.9.3/8.9.3) with ESMTP id AAA13452; Sat, 7 Dec 2002 00:27:16 +1100 Date: Sat, 7 Dec 2002 00:27:16 +1100 (EST) From: James Morris To: Zheng Jianping cc: netdev@oss.sgi.com Subject: Re: How kernel send message to daemon? In-Reply-To: <004701c29d09$a58e7b40$6c06a8c0@zhengjp> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 1319 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jmorris@intercode.com.au Precedence: bulk X-list: netdev Content-Length: 293 Lines: 15 On Fri, 6 Dec 2002, Zheng Jianping wrote: > Hi, > > In IPv6 packet forwarding processing, the kernel receives a 'bad' > message. Now I want the kernel inform the daemon( a routing protocl ) . > Howto? Try Netlink (have a look at rtnetlink.c). -- James Morris From jgarzik@pobox.com Sat Dec 7 14:34:36 2002 Received: with ECARTIS (v1.0.0; list netdev); Sat, 07 Dec 2002 14:34:40 -0800 (PST) Received: from www.linux.org.uk (parcelfarce.linux.theplanet.co.uk [195.92.249.252]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB7MYZuR016319 for ; Sat, 7 Dec 2002 14:34:35 -0800 Received: from c-24-98-88-232.atl.client2.attbi.com ([24.98.88.232] helo=pobox.com) by www.linux.org.uk with esmtp (Exim 3.33 #5) id 18KnaA-0002qp-00; Sat, 07 Dec 2002 22:37:58 +0000 Message-ID: <3DF2781D.3030209@pobox.com> Date: Sat, 07 Dec 2002 17:37:17 -0500 From: Jeff Garzik User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2) Gecko/20021202 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Linux Kernel Mailing List , netdev@oss.sgi.com CC: "David S. Miller" Subject: [RFC][PATCH] net drivers and cache alignment Content-Type: multipart/mixed; boundary="------------070601080505050708040803" X-archive-position: 1320 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jgarzik@pobox.com Precedence: bulk X-list: netdev Content-Length: 5525 Lines: 197 This is a multi-part message in MIME format. --------------070601080505050708040803 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit One of the [many] nice properties of the traditional Don Becker drivers has been that often the driver-private structures are arranged such that the structure is broken up on cacheline boundaries. The RX thread has a cacheline, the TX thread has a cacheline, etc. Jes Sorensen also independently, in his review of 8139cp.c, suggested that the driver-private struct be update with attention to cacheline boundaries. Early next year, I would like to start cleaning up some of the net drivers along these lines (no pun intended). To make it easier for vendors and random coders to cacheline-align struct members, I would like to make more explicit these cacheline boundaries, in a manner that is portable between 32-bit and 64-bit systems. I attach a sample implementation, and request feedback on this approach. The general idea is to make implementing this sort of concept "harder to screw up." --------------070601080505050708040803 Content-Type: text/plain; name="patch" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="patch" ===== drivers/net/tg3.c 1.41 vs edited ===== --- 1.41/drivers/net/tg3.c Wed Nov 20 00:49:23 2002 +++ edited/drivers/net/tg3.c Sat Dec 7 17:12:38 2002 @@ -25,6 +25,7 @@ #include #include #include +#include #include #include ===== drivers/net/tg3.h 1.19 vs edited ===== --- 1.19/drivers/net/tg3.h Mon Nov 11 05:27:52 2002 +++ edited/drivers/net/tg3.h Sat Dec 7 17:20:25 2002 @@ -1741,17 +1741,66 @@ * necessary for acquisition of 'tx_lock'. */ spinlock_t lock; - spinlock_t tx_lock; + spinlock_t indirect_lock; + + unsigned long regs; + struct net_device *dev; + struct pci_dev *pdev; + + struct tg3_hw_status *hw_status; + dma_addr_t status_mapping; + + u32 msg_enable; + + /* end "general, frequently-used members" cacheline section */ + ALIGNED_PAD(_pad1_) u32 tx_prod; u32 tx_cons; + u32 tx_pending; + + spinlock_t tx_lock; + + /* TX descs are only used if TG3_FLAG_HOST_TXDS is set. */ + struct tg3_tx_buffer_desc *tx_ring; + struct tx_ring_info *tx_buffers; + dma_addr_t tx_desc_mapping; + + /* end "tx thread" cacheline section */ + ALIGNED_PAD(_pad2_) + u32 rx_rcb_ptr; u32 rx_std_ptr; u32 rx_jumbo_ptr; #if TG3_MINI_RING_WORKS u32 rx_mini_ptr; #endif - spinlock_t indirect_lock; + u32 rx_pending; +#if TG3_MINI_RING_WORKS + u32 rx_mini_pending; +#endif + u32 rx_jumbo_pending; +#if TG3_VLAN_TAG_USED + struct vlan_group *vlgrp; +#endif + + struct tg3_rx_buffer_desc *rx_std; + struct ring_info *rx_std_buffers; + dma_addr_t rx_std_mapping; +#if TG3_MINI_RING_WORKS + struct tg3_rx_buffer_desc *rx_mini; + struct ring_info *rx_mini_buffers; + dma_addr_t rx_mini_mapping; +#endif + struct tg3_rx_buffer_desc *rx_jumbo; + struct ring_info *rx_jumbo_buffers; + dma_addr_t rx_jumbo_mapping; + + struct tg3_rx_buffer_desc *rx_rcb; + dma_addr_t rx_rcb_mapping; + + /* end "rx thread" cacheline section */ + ALIGNED_PAD(_pad3_) struct net_device_stats net_stats; struct net_device_stats net_stats_prev; @@ -1791,8 +1840,6 @@ #define TG3_FLAG_SPLIT_MODE 0x40000000 #define TG3_FLAG_INIT_COMPLETE 0x80000000 - u32 msg_enable; - u32 split_mode_max_reqs; #define SPLIT_MODE_5704_MAX_REQ 3 @@ -1806,13 +1853,6 @@ struct tg3_link_config link_config; struct tg3_bufmgr_config bufmgr_config; - u32 rx_pending; -#if TG3_MINI_RING_WORKS - u32 rx_mini_pending; -#endif - u32 rx_jumbo_pending; - u32 tx_pending; - /* cache h/w values, often passed straight to h/w */ u32 rx_mode; u32 tx_mode; @@ -1865,38 +1905,10 @@ (X) == PHY_ID_BCM5703 || (X) == PHY_ID_BCM5704 || \ (X) == PHY_ID_BCM8002 || (X) == PHY_ID_SERDES) - unsigned long regs; - struct pci_dev *pdev; - struct net_device *dev; -#if TG3_VLAN_TAG_USED - struct vlan_group *vlgrp; -#endif - - struct tg3_rx_buffer_desc *rx_std; - struct ring_info *rx_std_buffers; - dma_addr_t rx_std_mapping; -#if TG3_MINI_RING_WORKS - struct tg3_rx_buffer_desc *rx_mini; - struct ring_info *rx_mini_buffers; - dma_addr_t rx_mini_mapping; -#endif - struct tg3_rx_buffer_desc *rx_jumbo; - struct ring_info *rx_jumbo_buffers; - dma_addr_t rx_jumbo_mapping; - - struct tg3_rx_buffer_desc *rx_rcb; - dma_addr_t rx_rcb_mapping; - - /* TX descs are only used if TG3_FLAG_HOST_TXDS is set. */ - struct tg3_tx_buffer_desc *tx_ring; - struct tx_ring_info *tx_buffers; - dma_addr_t tx_desc_mapping; - - struct tg3_hw_status *hw_status; - dma_addr_t status_mapping; - struct tg3_hw_stats *hw_stats; dma_addr_t stats_mapping; + + /* end "everything else" cacheline(s) section */ }; #endif /* !(_T3_H) */ ===== include/linux/cache.h 1.5 vs edited ===== --- 1.5/include/linux/cache.h Tue Aug 27 16:04:10 2002 +++ edited/include/linux/cache.h Sat Dec 7 17:12:13 2002 @@ -53,4 +53,14 @@ #endif #endif +/* helper used inside struct definitions to break up struct at + * cacheline boundaries. + * Note: do not add a semi-colon (';') after an ALIGNED_PAD use. + */ +struct dummy_cacheline_struct { + int x; +} ____cacheline_aligned; +#define ALIGNED_PAD(name) \ + struct dummy_cacheline_struct name; + #endif /* __LINUX_CACHE_H */ --------------070601080505050708040803-- From davem@redhat.com Sat Dec 7 14:39:56 2002 Received: with ECARTIS (v1.0.0; list netdev); Sat, 07 Dec 2002 14:39:57 -0800 (PST) Received: from pizda.ninka.net (IDENT:root@pizda.ninka.net [216.101.162.242]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB7MdtuR016674 for ; Sat, 7 Dec 2002 14:39:55 -0800 Received: from localhost (IDENT:davem@localhost.localdomain [127.0.0.1]) by pizda.ninka.net (8.9.3/8.9.3) with ESMTP id OAA03473; Sat, 7 Dec 2002 14:40:05 -0800 Date: Sat, 07 Dec 2002 14:40:04 -0800 (PST) Message-Id: <20021207.144004.45605764.davem@redhat.com> To: jgarzik@pobox.com Cc: linux-kernel@vger.kernel.org, netdev@oss.sgi.com Subject: Re: [RFC][PATCH] net drivers and cache alignment From: "David S. Miller" In-Reply-To: <3DF2781D.3030209@pobox.com> References: <3DF2781D.3030209@pobox.com> X-FalunGong: Information control. X-Mailer: Mew version 2.1 on Emacs 21.1 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-archive-position: 1321 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: davem@redhat.com Precedence: bulk X-list: netdev Content-Length: 141 Lines: 5 Can't the cacheline_aligned attribute be applied to individual struct members? I remember doing this for thread_struct on sparc ages ago. From jgarzik@pobox.com Sat Dec 7 14:43:50 2002 Received: with ECARTIS (v1.0.0; list netdev); Sat, 07 Dec 2002 14:43:52 -0800 (PST) Received: from www.linux.org.uk (parcelfarce.linux.theplanet.co.uk [195.92.249.252]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB7MhnuR017052 for ; Sat, 7 Dec 2002 14:43:50 -0800 Received: from c-24-98-88-232.atl.client2.attbi.com ([24.98.88.232] helo=pobox.com) by www.linux.org.uk with esmtp (Exim 3.33 #5) id 18Knj7-0002ya-00; Sat, 07 Dec 2002 22:47:13 +0000 Message-ID: <3DF27A48.2090903@pobox.com> Date: Sat, 07 Dec 2002 17:46:32 -0500 From: Jeff Garzik User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2) Gecko/20021202 X-Accept-Language: en-us, en MIME-Version: 1.0 To: "David S. Miller" CC: linux-kernel@vger.kernel.org, netdev@oss.sgi.com Subject: Re: [RFC][PATCH] net drivers and cache alignment References: <3DF2781D.3030209@pobox.com> <20021207.144004.45605764.davem@redhat.com> In-Reply-To: <20021207.144004.45605764.davem@redhat.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 1322 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jgarzik@pobox.com Precedence: bulk X-list: netdev Content-Length: 257 Lines: 10 David S. Miller wrote: > Can't the cacheline_aligned attribute be applied to individual > struct members? I remember doing this for thread_struct on > sparc ages ago. I was hoping someone who knows gcc better than me knew that, and would speak up ;-) From jgarzik@pobox.com Sat Dec 7 15:03:33 2002 Received: with ECARTIS (v1.0.0; list netdev); Sat, 07 Dec 2002 15:03:38 -0800 (PST) Received: from www.linux.org.uk (parcelfarce.linux.theplanet.co.uk [195.92.249.252]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB7N3WuR017552 for ; Sat, 7 Dec 2002 15:03:33 -0800 Received: from c-24-98-88-232.atl.client2.attbi.com ([24.98.88.232] helo=pobox.com) by www.linux.org.uk with esmtp (Exim 3.33 #5) id 18Ko2B-0003HO-00; Sat, 07 Dec 2002 23:06:55 +0000 Message-ID: <3DF27EE7.4010508@pobox.com> Date: Sat, 07 Dec 2002 18:06:15 -0500 From: Jeff Garzik User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2) Gecko/20021202 X-Accept-Language: en-us, en MIME-Version: 1.0 To: "David S. Miller" CC: linux-kernel@vger.kernel.org, netdev@oss.sgi.com, Andrew Morton Subject: [RFC][PATCH] net drivers and cache alignment References: <3DF2781D.3030209@pobox.com> <20021207.144004.45605764.davem@redhat.com> In-Reply-To: <20021207.144004.45605764.davem@redhat.com> Content-Type: multipart/mixed; boundary="------------080805080000030003020904" X-archive-position: 1323 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jgarzik@pobox.com Precedence: bulk X-list: netdev Content-Length: 4802 Lines: 179 This is a multi-part message in MIME format. --------------080805080000030003020904 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit David S. Miller wrote: > Can't the cacheline_aligned attribute be applied to individual > struct members? I remember doing this for thread_struct on > sparc ages ago. Looks like it from the 2.4 processor.h code. Attached is cut #2. Thanks for all the near-instant feedback so far :) Andrew, does the attached still need padding on SMP? --------------080805080000030003020904 Content-Type: text/plain; name="patch" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="patch" ===== drivers/net/tg3.c 1.41 vs edited ===== --- 1.41/drivers/net/tg3.c Wed Nov 20 00:49:23 2002 +++ edited/drivers/net/tg3.c Sat Dec 7 17:12:38 2002 @@ -25,6 +25,7 @@ #include #include #include +#include #include #include ===== drivers/net/tg3.h 1.19 vs edited ===== --- 1.19/drivers/net/tg3.h Mon Nov 11 05:27:52 2002 +++ edited/drivers/net/tg3.h Sat Dec 7 18:01:08 2002 @@ -1728,6 +1728,8 @@ }; struct tg3 { + /* begin "general, frequently-used members" cacheline section */ + /* SMP locking strategy: * * lock: Held during all operations except TX packet @@ -1740,20 +1742,63 @@ * be disabled to take 'lock' but only softirq disabling is * necessary for acquisition of 'tx_lock'. */ - spinlock_t lock; - spinlock_t tx_lock; + spinlock_t lock ____cacheline_aligned; + spinlock_t indirect_lock; - u32 tx_prod; + unsigned long regs; + struct net_device *dev; + struct pci_dev *pdev; + + struct tg3_hw_status *hw_status; + dma_addr_t status_mapping; + + u32 msg_enable; + + /* begin "tx thread" cacheline section */ + u32 tx_prod ____cacheline_aligned; u32 tx_cons; - u32 rx_rcb_ptr; + u32 tx_pending; + + spinlock_t tx_lock; + + /* TX descs are only used if TG3_FLAG_HOST_TXDS is set. */ + struct tg3_tx_buffer_desc *tx_ring; + struct tx_ring_info *tx_buffers; + dma_addr_t tx_desc_mapping; + + /* begin "rx thread" cacheline section */ + u32 rx_rcb_ptr ____cacheline_aligned; u32 rx_std_ptr; u32 rx_jumbo_ptr; #if TG3_MINI_RING_WORKS u32 rx_mini_ptr; #endif - spinlock_t indirect_lock; + u32 rx_pending; +#if TG3_MINI_RING_WORKS + u32 rx_mini_pending; +#endif + u32 rx_jumbo_pending; +#if TG3_VLAN_TAG_USED + struct vlan_group *vlgrp; +#endif + + struct tg3_rx_buffer_desc *rx_std; + struct ring_info *rx_std_buffers; + dma_addr_t rx_std_mapping; +#if TG3_MINI_RING_WORKS + struct tg3_rx_buffer_desc *rx_mini; + struct ring_info *rx_mini_buffers; + dma_addr_t rx_mini_mapping; +#endif + struct tg3_rx_buffer_desc *rx_jumbo; + struct ring_info *rx_jumbo_buffers; + dma_addr_t rx_jumbo_mapping; - struct net_device_stats net_stats; + struct tg3_rx_buffer_desc *rx_rcb; + dma_addr_t rx_rcb_mapping; + + /* begin "everything else" cacheline(s) section */ + struct net_device_stats net_stats ____cacheline_aligned; struct net_device_stats net_stats_prev; unsigned long phy_crc_errors; @@ -1791,8 +1836,6 @@ #define TG3_FLAG_SPLIT_MODE 0x40000000 #define TG3_FLAG_INIT_COMPLETE 0x80000000 - u32 msg_enable; - u32 split_mode_max_reqs; #define SPLIT_MODE_5704_MAX_REQ 3 @@ -1806,13 +1849,6 @@ struct tg3_link_config link_config; struct tg3_bufmgr_config bufmgr_config; - u32 rx_pending; -#if TG3_MINI_RING_WORKS - u32 rx_mini_pending; -#endif - u32 rx_jumbo_pending; - u32 tx_pending; - /* cache h/w values, often passed straight to h/w */ u32 rx_mode; u32 tx_mode; @@ -1864,36 +1900,6 @@ (X) == PHY_ID_BCM5411 || (X) == PHY_ID_BCM5701 || \ (X) == PHY_ID_BCM5703 || (X) == PHY_ID_BCM5704 || \ (X) == PHY_ID_BCM8002 || (X) == PHY_ID_SERDES) - - unsigned long regs; - struct pci_dev *pdev; - struct net_device *dev; -#if TG3_VLAN_TAG_USED - struct vlan_group *vlgrp; -#endif - - struct tg3_rx_buffer_desc *rx_std; - struct ring_info *rx_std_buffers; - dma_addr_t rx_std_mapping; -#if TG3_MINI_RING_WORKS - struct tg3_rx_buffer_desc *rx_mini; - struct ring_info *rx_mini_buffers; - dma_addr_t rx_mini_mapping; -#endif - struct tg3_rx_buffer_desc *rx_jumbo; - struct ring_info *rx_jumbo_buffers; - dma_addr_t rx_jumbo_mapping; - - struct tg3_rx_buffer_desc *rx_rcb; - dma_addr_t rx_rcb_mapping; - - /* TX descs are only used if TG3_FLAG_HOST_TXDS is set. */ - struct tg3_tx_buffer_desc *tx_ring; - struct tx_ring_info *tx_buffers; - dma_addr_t tx_desc_mapping; - - struct tg3_hw_status *hw_status; - dma_addr_t status_mapping; struct tg3_hw_stats *hw_stats; dma_addr_t stats_mapping; --------------080805080000030003020904-- From Andrew.Morton@digeo.com Sat Dec 7 15:25:59 2002 Received: with ECARTIS (v1.0.0; list netdev); Sat, 07 Dec 2002 15:26:01 -0800 (PST) Received: from packet.digeo.com (packet.digeo.com [12.110.80.53]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB7NPxuR018148 for ; Sat, 7 Dec 2002 15:25:59 -0800 Received: from digeo-nav01.digeo.com (digeo-nav01.digeo.com [192.168.1.233]) by packet.digeo.com (8.9.3+Sun/8.9.3) with SMTP id PAA18670 for ; Sat, 7 Dec 2002 15:29:17 -0800 (PST) Received: from digeo-e2k03.digeo.com ([192.168.2.44]) by digeo-nav01.digeo.com (NAVGW 2.5.2.12) with SMTP id M2002120715310031349 ; Sat, 07 Dec 2002 15:31:00 -0800 Received: from pao-ex01.pao.digeo.com ([172.17.144.34]) by digeo-e2k03.digeo.com with Microsoft SMTPSVC(5.0.2195.5329); Sat, 7 Dec 2002 15:29:18 -0800 Received: from digeo.com ([172.17.144.18]) by pao-ex01.pao.digeo.com with Microsoft SMTPSVC(5.0.2195.4905); Sat, 7 Dec 2002 15:29:17 -0800 Message-ID: <3DF2844C.F9216283@digeo.com> Date: Sat, 07 Dec 2002 15:29:16 -0800 From: Andrew Morton X-Mailer: Mozilla 4.79 [en] (X11; U; Linux 2.5.46 i686) X-Accept-Language: en MIME-Version: 1.0 To: Jeff Garzik CC: "David S. Miller" , linux-kernel@vger.kernel.org, netdev@oss.sgi.com Subject: Re: [RFC][PATCH] net drivers and cache alignment References: <3DF2781D.3030209@pobox.com> <20021207.144004.45605764.davem@redhat.com> <3DF27EE7.4010508@pobox.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 07 Dec 2002 23:29:17.0171 (UTC) FILETIME=[75D7DC30:01C29E48] X-archive-position: 1324 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: akpm@digeo.com Precedence: bulk X-list: netdev Content-Length: 1043 Lines: 42 Jeff Garzik wrote: > > David S. Miller wrote: > > Can't the cacheline_aligned attribute be applied to individual > > struct members? I remember doing this for thread_struct on > > sparc ages ago. > > Looks like it from the 2.4 processor.h code. > > Attached is cut #2. Thanks for all the near-instant feedback so far :) > Andrew, does the attached still need padding on SMP? It needs padding _only_ on SMP. ____cacheline_aligned_in_smp. #define offsetof(t, m) ((int)(&((t *)0)->m)) struct foo { int a; int b __attribute__((__aligned__(1024))); int c; } foo; main() { printf("%d\n", sizeof(struct foo)); printf("%d\n", offsetof(struct foo, a)); printf("%d\n", offsetof(struct foo, b)); printf("%d\n", offsetof(struct foo, c)); } ./a.out 2048 0 1024 1028 So your patch will do what you want it to do. You should just tag the first member of a group with ____cacheline_aligned_in_smp, and keep an eye on things with offsetof(). Not sure why sizeof() returned 2048 though. From davem@redhat.com Sat Dec 7 15:30:37 2002 Received: with ECARTIS (v1.0.0; list netdev); Sat, 07 Dec 2002 15:30:43 -0800 (PST) Received: from pizda.ninka.net (IDENT:root@pizda.ninka.net [216.101.162.242]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB7NUbuR018504 for ; Sat, 7 Dec 2002 15:30:37 -0800 Received: from localhost (IDENT:davem@localhost.localdomain [127.0.0.1]) by pizda.ninka.net (8.9.3/8.9.3) with ESMTP id PAA03608; Sat, 7 Dec 2002 15:30:45 -0800 Date: Sat, 07 Dec 2002 15:30:45 -0800 (PST) Message-Id: <20021207.153045.26640406.davem@redhat.com> To: akpm@digeo.com Cc: jgarzik@pobox.com, linux-kernel@vger.kernel.org, netdev@oss.sgi.com Subject: Re: [RFC][PATCH] net drivers and cache alignment From: "David S. Miller" In-Reply-To: <3DF2844C.F9216283@digeo.com> References: <20021207.144004.45605764.davem@redhat.com> <3DF27EE7.4010508@pobox.com> <3DF2844C.F9216283@digeo.com> X-FalunGong: Information control. X-Mailer: Mew version 2.1 on Emacs 21.1 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-archive-position: 1325 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: davem@redhat.com Precedence: bulk X-list: netdev Content-Length: 432 Lines: 13 From: Andrew Morton Date: Sat, 07 Dec 2002 15:29:16 -0800 Jeff Garzik wrote: > Attached is cut #2. Thanks for all the near-instant feedback so far :) > Andrew, does the attached still need padding on SMP? It needs padding _only_ on SMP. ____cacheline_aligned_in_smp. non-smp machines lack L2 caches? That's new to me :-) More seriously, there are real benefits on non-SMP systems. From jgarzik@pobox.com Sat Dec 7 15:34:08 2002 Received: with ECARTIS (v1.0.0; list netdev); Sat, 07 Dec 2002 15:34:09 -0800 (PST) Received: from www.linux.org.uk (parcelfarce.linux.theplanet.co.uk [195.92.249.252]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB7NY7uR018867 for ; Sat, 7 Dec 2002 15:34:07 -0800 Received: from c-24-98-88-232.atl.client2.attbi.com ([24.98.88.232] helo=pobox.com) by www.linux.org.uk with esmtp (Exim 3.33 #5) id 18KoVm-0003kA-00; Sat, 07 Dec 2002 23:37:30 +0000 Message-ID: <3DF28610.3000004@pobox.com> Date: Sat, 07 Dec 2002 18:36:48 -0500 From: Jeff Garzik User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2) Gecko/20021202 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Andrew Morton CC: "David S. Miller" , linux-kernel@vger.kernel.org, netdev@oss.sgi.com Subject: Re: [RFC][PATCH] net drivers and cache alignment References: <3DF2781D.3030209@pobox.com> <20021207.144004.45605764.davem@redhat.com> <3DF27EE7.4010508@pobox.com> <3DF2844C.F9216283@digeo.com> In-Reply-To: <3DF2844C.F9216283@digeo.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 1326 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jgarzik@pobox.com Precedence: bulk X-list: netdev Content-Length: 670 Lines: 22 Andrew Morton wrote: > It needs padding _only_ on SMP. ____cacheline_aligned_in_smp. [...] > So your patch will do what you want it to do. You should just tag the > first member of a group with ____cacheline_aligned_in_smp, and keep an > eye on things with offsetof(). thanks. For this case, though, I want to align on cacheline bounaries even on UP, right? That's why I picked ____cacheline_aligned. It uses L1_CACHE_BYTES when !CONFIG_SMP. Other uses of ____cacheline_aligned in the kernel seem to relate to irq matters, just like my groupings in tg3.h. [obviously benchmarking can answer some of this, but I want to hammer out silliness first] Jeff From jamagallon@able.es Sat Dec 7 15:34:52 2002 Received: with ECARTIS (v1.0.0; list netdev); Sat, 07 Dec 2002 15:34:54 -0800 (PST) Received: from smtp09.retemail.es (smtp09.iddeo.es [62.81.186.19]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB7NYouR019123 for ; Sat, 7 Dec 2002 15:34:51 -0800 Received: from werewolf.able.es ([212.97.172.115]) by smtp09.retemail.es (InterMail vM.5.01.05.12 201-253-122-126-112-20020820) with ESMTP id <20021207233811.GORR1188.smtp09.retemail.es@werewolf.able.es>; Sun, 8 Dec 2002 00:38:11 +0100 Date: Sun, 8 Dec 2002 00:37:46 +0100 From: "J.A. Magallon" To: Andrew Morton Cc: Jeff Garzik , "David S. Miller" , linux-kernel@vger.kernel.org, netdev@oss.sgi.com Subject: Re: [RFC][PATCH] net drivers and cache alignment Message-ID: <20021207233745.GE3183@werewolf.able.es> References: <3DF2781D.3030209@pobox.com> <20021207.144004.45605764.davem@redhat.com> <3DF27EE7.4010508@pobox.com> <3DF2844C.F9216283@digeo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-15 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <3DF2844C.F9216283@digeo.com>; from akpm@digeo.com on Sun, Dec 08, 2002 at 00:29:16 +0100 X-Mailer: Balsa 1.4.1 X-archive-position: 1327 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jamagallon@able.es Precedence: bulk X-list: netdev Content-Length: 1441 Lines: 62 On 2002.12.08 Andrew Morton wrote: >Jeff Garzik wrote: >> >> David S. Miller wrote: >> > Can't the cacheline_aligned attribute be applied to individual >> > struct members? I remember doing this for thread_struct on >> > sparc ages ago. >> >> Looks like it from the 2.4 processor.h code. >> >> Attached is cut #2. Thanks for all the near-instant feedback so far :) >> Andrew, does the attached still need padding on SMP? > What do you all think about this: #include #define CACHE_LINE_SIZE 128 #define ____cacheline_aligned __attribute__((__aligned__(CACHE_LINE_SIZE))) #define __cacheline_start struct { } ____cacheline_aligned; #define offsetof(t, m) ((int)(&((t *)0)->m)) struct S { __cacheline_start int x; __cacheline_start int y; int z; }; int main() { struct S s; printf("%d\n",sizeof(struct S)); printf("%d\n",offsetof(struct S,x)); printf("%d\n",offsetof(struct S,y)); printf("%d\n",offsetof(struct S,z)); } werewolf:~> vi kk.c werewolf:~> kk 256 0 128 132 So you don't have to modify any field, just put __cacheline_start where needed ? (and does not add any extra sizeof(int) overhead). -- J.A. Magallon \ Software is like sex: werewolf.able.es \ It's better when it's free Mandrake Linux release 9.1 (Cooker) for i586 Linux 2.4.20-jam1 (gcc 3.2 (Mandrake Linux 9.1 3.2-4mdk)) From Andrew.Morton@digeo.com Sat Dec 7 15:38:43 2002 Received: with ECARTIS (v1.0.0; list netdev); Sat, 07 Dec 2002 15:38:47 -0800 (PST) Received: from packet.digeo.com (packet.digeo.com [12.110.80.53]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB7NcguR019615 for ; Sat, 7 Dec 2002 15:38:42 -0800 Received: from digeo-nav01.digeo.com (digeo-nav01.digeo.com [192.168.1.233]) by packet.digeo.com (8.9.3+Sun/8.9.3) with SMTP id PAA18874 for ; Sat, 7 Dec 2002 15:42:01 -0800 (PST) Received: from digeo-e2k03.digeo.com ([192.168.2.44]) by digeo-nav01.digeo.com (NAVGW 2.5.2.12) with SMTP id M2002120715434417363 ; Sat, 07 Dec 2002 15:43:44 -0800 Received: from pao-ex01.pao.digeo.com ([172.17.144.34]) by digeo-e2k03.digeo.com with Microsoft SMTPSVC(5.0.2195.5329); Sat, 7 Dec 2002 15:42:01 -0800 Received: from digeo.com ([172.17.144.18]) by pao-ex01.pao.digeo.com with Microsoft SMTPSVC(5.0.2195.4905); Sat, 7 Dec 2002 15:42:00 -0800 Message-ID: <3DF28748.186AB31F@digeo.com> Date: Sat, 07 Dec 2002 15:42:00 -0800 From: Andrew Morton X-Mailer: Mozilla 4.79 [en] (X11; U; Linux 2.5.46 i686) X-Accept-Language: en MIME-Version: 1.0 To: "David S. Miller" CC: jgarzik@pobox.com, linux-kernel@vger.kernel.org, netdev@oss.sgi.com Subject: Re: [RFC][PATCH] net drivers and cache alignment References: <3DF2844C.F9216283@digeo.com> <20021207.153045.26640406.davem@redhat.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 07 Dec 2002 23:42:00.0630 (UTC) FILETIME=[3CE67560:01C29E4A] X-archive-position: 1328 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: akpm@digeo.com Precedence: bulk X-list: netdev Content-Length: 630 Lines: 19 "David S. Miller" wrote: > > From: Andrew Morton > Date: Sat, 07 Dec 2002 15:29:16 -0800 > > Jeff Garzik wrote: > > Attached is cut #2. Thanks for all the near-instant feedback so far :) > > Andrew, does the attached still need padding on SMP? > > It needs padding _only_ on SMP. ____cacheline_aligned_in_smp. > > non-smp machines lack L2 caches? That's new to me :-) > > More seriously, there are real benefits on non-SMP systems. Then I am most confused. None of these fields will be put under busmastering or anything like that, so what advantage is there in spreading them out? From jgarzik@pobox.com Sat Dec 7 15:40:22 2002 Received: with ECARTIS (v1.0.0; list netdev); Sat, 07 Dec 2002 15:40:24 -0800 (PST) Received: from www.linux.org.uk (parcelfarce.linux.theplanet.co.uk [195.92.249.252]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB7NeLuR019913 for ; Sat, 7 Dec 2002 15:40:21 -0800 Received: from c-24-98-88-232.atl.client2.attbi.com ([24.98.88.232] helo=pobox.com) by www.linux.org.uk with esmtp (Exim 3.33 #5) id 18Kobl-0003pC-00; Sat, 07 Dec 2002 23:43:41 +0000 Message-ID: <3DF28781.3070405@pobox.com> Date: Sat, 07 Dec 2002 18:42:57 -0500 From: Jeff Garzik User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2) Gecko/20021202 X-Accept-Language: en-us, en MIME-Version: 1.0 To: "J.A. Magallon" CC: Andrew Morton , "David S. Miller" , linux-kernel@vger.kernel.org, netdev@oss.sgi.com Subject: Re: [RFC][PATCH] net drivers and cache alignment References: <3DF2781D.3030209@pobox.com> <20021207.144004.45605764.davem@redhat.com> <3DF27EE7.4010508@pobox.com> <3DF2844C.F9216283@digeo.com> <20021207233745.GE3183@werewolf.able.es> In-Reply-To: <20021207233745.GE3183@werewolf.able.es> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 1329 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jgarzik@pobox.com Precedence: bulk X-list: netdev Content-Length: 819 Lines: 34 J.A. Magallon wrote: > On 2002.12.08 Andrew Morton wrote: > >>Jeff Garzik wrote: >> >>>David S. Miller wrote: >>> >>>>Can't the cacheline_aligned attribute be applied to individual >>>>struct members? I remember doing this for thread_struct on >>>>sparc ages ago. >>> >>>Looks like it from the 2.4 processor.h code. >>> >>>Attached is cut #2. Thanks for all the near-instant feedback so far :) >>> Andrew, does the attached still need padding on SMP? >> > > What do you all think about this: > > #include > > #define CACHE_LINE_SIZE 128 > #define ____cacheline_aligned __attribute__((__aligned__(CACHE_LINE_SIZE))) > > #define __cacheline_start struct { } ____cacheline_aligned; if you can mark struct members with attributes, as it appears you can, there's no need to define a struct. Jeff From Andrew.Morton@digeo.com Sat Dec 7 15:42:05 2002 Received: with ECARTIS (v1.0.0; list netdev); Sat, 07 Dec 2002 15:42:06 -0800 (PST) Received: from packet.digeo.com (packet.digeo.com [12.110.80.53]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB7Ng5uR020334 for ; Sat, 7 Dec 2002 15:42:05 -0800 Received: from digeo-nav01.digeo.com (digeo-nav01.digeo.com [192.168.1.233]) by packet.digeo.com (8.9.3+Sun/8.9.3) with SMTP id PAA18945 for ; Sat, 7 Dec 2002 15:45:23 -0800 (PST) Received: from digeo-e2k03.digeo.com ([192.168.2.44]) by digeo-nav01.digeo.com (NAVGW 2.5.2.12) with SMTP id M2002120715470616840 ; Sat, 07 Dec 2002 15:47:06 -0800 Received: from pao-ex01.pao.digeo.com ([172.17.144.34]) by digeo-e2k03.digeo.com with Microsoft SMTPSVC(5.0.2195.5329); Sat, 7 Dec 2002 15:45:23 -0800 Received: from digeo.com ([172.17.144.18]) by pao-ex01.pao.digeo.com with Microsoft SMTPSVC(5.0.2195.4905); Sat, 7 Dec 2002 15:45:22 -0800 Message-ID: <3DF28811.F6580BA6@digeo.com> Date: Sat, 07 Dec 2002 15:45:21 -0800 From: Andrew Morton X-Mailer: Mozilla 4.79 [en] (X11; U; Linux 2.5.46 i686) X-Accept-Language: en MIME-Version: 1.0 To: "J.A. Magallon" CC: Jeff Garzik , "David S. Miller" , linux-kernel@vger.kernel.org, netdev@oss.sgi.com Subject: Re: [RFC][PATCH] net drivers and cache alignment References: <3DF2781D.3030209@pobox.com> <20021207.144004.45605764.davem@redhat.com> <3DF27EE7.4010508@pobox.com> <3DF2844C.F9216283@digeo.com> <20021207233745.GE3183@werewolf.able.es> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 07 Dec 2002 23:45:22.0378 (UTC) FILETIME=[B526C2A0:01C29E4A] X-archive-position: 1330 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: akpm@digeo.com Precedence: bulk X-list: netdev Content-Length: 446 Lines: 13 "J.A. Magallon" wrote: > > #define __cacheline_start struct { } ____cacheline_aligned; That will generate a warning on faster^Wolder versions of gcc. mnm:/home/akpm> gcc t2.c t2.c:11: warning: unnamed struct/union that defines no instances t2.c:15: warning: unnamed struct/union that defines no instances mnm:/home/akpm> gcc -v Reading specs from /usr/local/lib/gcc-lib/i686-pc-linux-gnu/2.95.3/specs gcc version 2.95.3 20010315 (release) From Andrew.Morton@digeo.com Sat Dec 7 15:48:19 2002 Received: with ECARTIS (v1.0.0; list netdev); Sat, 07 Dec 2002 15:48:21 -0800 (PST) Received: from packet.digeo.com (packet.digeo.com [12.110.80.53]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB7NmJuR020748 for ; Sat, 7 Dec 2002 15:48:19 -0800 Received: from digeo-nav01.digeo.com (digeo-nav01.digeo.com [192.168.1.233]) by packet.digeo.com (8.9.3+Sun/8.9.3) with SMTP id PAA19081 for ; Sat, 7 Dec 2002 15:51:38 -0800 (PST) Received: from digeo-e2k03.digeo.com ([192.168.2.44]) by digeo-nav01.digeo.com (NAVGW 2.5.2.12) with SMTP id M2002120715532000144 ; Sat, 07 Dec 2002 15:53:20 -0800 Received: from pao-ex01.pao.digeo.com ([172.17.144.34]) by digeo-e2k03.digeo.com with Microsoft SMTPSVC(5.0.2195.5329); Sat, 7 Dec 2002 15:51:38 -0800 Received: from digeo.com ([172.17.144.18]) by pao-ex01.pao.digeo.com with Microsoft SMTPSVC(5.0.2195.4905); Sat, 7 Dec 2002 15:51:36 -0800 Message-ID: <3DF28988.93F268EA@digeo.com> Date: Sat, 07 Dec 2002 15:51:36 -0800 From: Andrew Morton X-Mailer: Mozilla 4.79 [en] (X11; U; Linux 2.5.46 i686) X-Accept-Language: en MIME-Version: 1.0 To: "David S. Miller" , jgarzik@pobox.com, linux-kernel@vger.kernel.org, netdev@oss.sgi.com Subject: Re: [RFC][PATCH] net drivers and cache alignment References: <3DF2844C.F9216283@digeo.com> <20021207.153045.26640406.davem@redhat.com> <3DF28748.186AB31F@digeo.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 07 Dec 2002 23:51:36.0841 (UTC) FILETIME=[94594790:01C29E4B] X-archive-position: 1331 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: akpm@digeo.com Precedence: bulk X-list: netdev Content-Length: 897 Lines: 27 Andrew Morton wrote: > > "David S. Miller" wrote: > > > > From: Andrew Morton > > Date: Sat, 07 Dec 2002 15:29:16 -0800 > > > > Jeff Garzik wrote: > > > Attached is cut #2. Thanks for all the near-instant feedback so far :) > > > Andrew, does the attached still need padding on SMP? > > > > It needs padding _only_ on SMP. ____cacheline_aligned_in_smp. > > > > non-smp machines lack L2 caches? That's new to me :-) > > > > More seriously, there are real benefits on non-SMP systems. > > Then I am most confused. None of these fields will be put under > busmastering or anything like that, so what advantage is there in > spreading them out? Oh I see what you want - to be able to pick up all the operating fields in a single fetch. That will increase the overall cache footprint though. I wonder if it's really a net win, over just keeping it small. From jamagallon@able.es Sat Dec 7 15:49:54 2002 Received: with ECARTIS (v1.0.0; list netdev); Sat, 07 Dec 2002 15:49:56 -0800 (PST) Received: from smtp07.retemail.es (smtp07.iddeo.es [62.81.186.17]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB7NnruR021013 for ; Sat, 7 Dec 2002 15:49:54 -0800 Received: from werewolf.able.es ([212.97.172.115]) by smtp07.retemail.es (InterMail vM.5.01.05.12 201-253-122-126-112-20020820) with ESMTP id <20021207235316.HCKY1229.smtp07.retemail.es@werewolf.able.es>; Sun, 8 Dec 2002 00:53:16 +0100 Date: Sun, 8 Dec 2002 00:52:55 +0100 From: "J.A. Magallon" To: Andrew Morton Cc: "J.A. Magallon" , Jeff Garzik , "David S. Miller" , linux-kernel@vger.kernel.org, netdev@oss.sgi.com Subject: Re: [RFC][PATCH] net drivers and cache alignment Message-ID: <20021207235255.GA3754@werewolf.able.es> References: <3DF2781D.3030209@pobox.com> <20021207.144004.45605764.davem@redhat.com> <3DF27EE7.4010508@pobox.com> <3DF2844C.F9216283@digeo.com> <20021207233745.GE3183@werewolf.able.es> <3DF28811.F6580BA6@digeo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-15 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <3DF28811.F6580BA6@digeo.com>; from akpm@digeo.com on Sun, Dec 08, 2002 at 00:45:21 +0100 X-Mailer: Balsa 1.4.1 X-archive-position: 1332 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jamagallon@able.es Precedence: bulk X-list: netdev Content-Length: 1006 Lines: 29 On 2002.12.08 Andrew Morton wrote: >"J.A. Magallon" wrote: >> >> #define __cacheline_start struct { } ____cacheline_aligned; > >That will generate a warning on faster^Wolder versions of gcc. > >mnm:/home/akpm> gcc t2.c >t2.c:11: warning: unnamed struct/union that defines no instances >t2.c:15: warning: unnamed struct/union that defines no instances >mnm:/home/akpm> gcc -v >Reading specs from /usr/local/lib/gcc-lib/i686-pc-linux-gnu/2.95.3/specs >gcc version 2.95.3 20010315 (release) > And how 'bout this (do not have any gcc oldie available to test): #define __cacheline_start union { int :0; } ____cacheline_aligned; It passes gcc-3.2 -Wall... I think it's nicer to insert __c_s than to go field by field marking them... -- J.A. Magallon \ Software is like sex: werewolf.able.es \ It's better when it's free Mandrake Linux release 9.1 (Cooker) for i586 Linux 2.4.20-jam1 (gcc 3.2 (Mandrake Linux 9.1 3.2-4mdk)) From drow@false.org Sat Dec 7 17:10:37 2002 Received: with ECARTIS (v1.0.0; list netdev); Sat, 07 Dec 2002 17:10:40 -0800 (PST) Received: from crack.them.org (mail@crack.them.org [65.125.64.184]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB81AauR022423 for ; Sat, 7 Dec 2002 17:10:37 -0800 Received: from nevyn.them.org ([66.93.61.169] ident=mail) by crack.them.org with asmtp (Exim 3.12 #1 (Debian)) id 18Krt9-0004GZ-00; Sat, 07 Dec 2002 21:13:52 -0600 Received: from drow by nevyn.them.org with local (Exim 3.36 #1 (Debian)) id 18Kq1D-00028p-00; Sat, 07 Dec 2002 20:14:03 -0500 Date: Sat, 7 Dec 2002 20:14:03 -0500 From: Daniel Jacobowitz To: Andrew Morton Cc: Jeff Garzik , "David S. Miller" , linux-kernel@vger.kernel.org, netdev@oss.sgi.com Subject: Re: [RFC][PATCH] net drivers and cache alignment Message-ID: <20021208011403.GA7729@nevyn.them.org> Mail-Followup-To: Andrew Morton , Jeff Garzik , "David S. Miller" , linux-kernel@vger.kernel.org, netdev@oss.sgi.com References: <3DF2781D.3030209@pobox.com> <20021207.144004.45605764.davem@redhat.com> <3DF27EE7.4010508@pobox.com> <3DF2844C.F9216283@digeo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3DF2844C.F9216283@digeo.com> User-Agent: Mutt/1.5.1i X-archive-position: 1333 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: dan@debian.org Precedence: bulk X-list: netdev Content-Length: 1511 Lines: 55 On Sat, Dec 07, 2002 at 03:29:16PM -0800, Andrew Morton wrote: > Jeff Garzik wrote: > > > > David S. Miller wrote: > > > Can't the cacheline_aligned attribute be applied to individual > > > struct members? I remember doing this for thread_struct on > > > sparc ages ago. > > > > Looks like it from the 2.4 processor.h code. > > > > Attached is cut #2. Thanks for all the near-instant feedback so far :) > > Andrew, does the attached still need padding on SMP? > > It needs padding _only_ on SMP. ____cacheline_aligned_in_smp. > > #define offsetof(t, m) ((int)(&((t *)0)->m)) > > struct foo { > int a; > int b __attribute__((__aligned__(1024))); > int c; > } foo; > > main() > { > printf("%d\n", sizeof(struct foo)); > printf("%d\n", offsetof(struct foo, a)); > printf("%d\n", offsetof(struct foo, b)); > printf("%d\n", offsetof(struct foo, c)); > } > > ./a.out > 2048 > 0 > 1024 > 1028 > > So your patch will do what you want it to do. You should just tag the > first member of a group with ____cacheline_aligned_in_smp, and keep an > eye on things with offsetof(). > > Not sure why sizeof() returned 2048 though. The structure contains an __aligned__(1024) item. Think about an array of 'struct foo' items. They have to be 2048 bytes or you won't align correctly. C allows for empty space in structure padding, but not in arrays, AFAIK. -- Daniel Jacobowitz MontaVista Software Debian GNU/Linux Developer From davem@redhat.com Sun Dec 8 12:00:41 2002 Received: with ECARTIS (v1.0.0; list netdev); Sun, 08 Dec 2002 12:00:45 -0800 (PST) Received: from pizda.ninka.net (IDENT:root@pizda.ninka.net [216.101.162.242]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB8K0fuR006713 for ; Sun, 8 Dec 2002 12:00:41 -0800 Received: from localhost (IDENT:davem@localhost.localdomain [127.0.0.1]) by pizda.ninka.net (8.9.3/8.9.3) with ESMTP id MAA21843; Sun, 8 Dec 2002 12:00:45 -0800 Date: Sun, 08 Dec 2002 12:00:44 -0800 (PST) Message-Id: <20021208.120044.08024570.davem@redhat.com> To: akpm@digeo.com Cc: jgarzik@pobox.com, linux-kernel@vger.kernel.org, netdev@oss.sgi.com Subject: Re: [RFC][PATCH] net drivers and cache alignment From: "David S. Miller" In-Reply-To: <3DF28748.186AB31F@digeo.com> References: <3DF2844C.F9216283@digeo.com> <20021207.153045.26640406.davem@redhat.com> <3DF28748.186AB31F@digeo.com> X-FalunGong: Information control. X-Mailer: Mew version 2.1 on Emacs 21.1 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-archive-position: 1335 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: davem@redhat.com Precedence: bulk X-list: netdev Content-Length: 830 Lines: 23 From: Andrew Morton Date: Sat, 07 Dec 2002 15:42:00 -0800 "David S. Miller" wrote: > non-smp machines lack L2 caches? That's new to me :-) > > More seriously, there are real benefits on non-SMP systems. Then I am most confused. None of these fields will be put under busmastering or anything like that, so what advantage is there in spreading them out? When you are in the "tx path" you'll take one L2 cache miss to bring all the necessary information into the cpu's caches. Otherwise, when data is arbitrarily scattered over multiple L2 cache lines, you'll need to service potentially more L2 cache misses. This optimization has nothing to do with false data sharing amoungst multiple processors. It's about packing the data accesses optimally for specific code paths. From david-b@pacbell.net Sun Dec 8 14:37:09 2002 Received: with ECARTIS (v1.0.0; list netdev); Sun, 08 Dec 2002 14:37:13 -0800 (PST) Received: from mta7.pltn13.pbi.net (mta7.pltn13.pbi.net [64.164.98.8]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB8Mb9uR010738 for ; Sun, 8 Dec 2002 14:37:09 -0800 Received: from pacbell.net ([67.118.246.27]) by mta7.pltn13.pbi.net (iPlanet Messaging Server 5.1 HotFix 1.6 (built Oct 18 2002)) with ESMTP id <0H6T009YOO6FKV@mta7.pltn13.pbi.net> for netdev@oss.sgi.com; Sun, 08 Dec 2002 14:37:29 -0800 (PST) Date: Sun, 08 Dec 2002 14:42:16 -0800 From: David Brownell Subject: Re: 2.5.50 BUG_TRAP on !dev->deadbeaf, and oopses To: Stefan Rompf Cc: netdev@oss.sgi.com Message-id: <3DF3CAC8.5070309@pacbell.net> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=us-ascii Content-transfer-encoding: 7bit X-Accept-Language: en-us, en, fr User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.9) Gecko/20020513 References: <3DE9290A.7070502@pacbell.net> <3DEA0452.B1F15BFD@isg.de> <3DEBA9F5.6000606@pacbell.net> X-archive-position: 1336 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: david-b@pacbell.net Precedence: bulk X-list: netdev Content-Length: 3031 Lines: 82 Following up on my earlier reply ... > Hi Stefan, > >>> KERNEL: assertion (!dev->deadbeaf) failed at net/core/dev.c(2544) >>> >>> I think there's another bug, beyond the obvious speling erorz. Namely, >>> that "deadbeaf" is only set after that BUG_TRAP, or on one error path. >>> The assertion prevents hotpluggable network drivers from unregistering >>> when the hardware goes away ... which is a regression. >> >> >> >> actually, the assertion is triggered when someone tries to unregister a >> netdevice twice, and that's also why you get FWIW I just added a printk so I could see if the disconnect() method was called more than once (by USB) per your guess ... no, it wasn't. It's called once, leading to flakey diagnostices and a BUG(). So this is clearly some kind of network layer problem, as I described in my original message (and then in this one). Behavioral proof, as well as the one that came from inspecting the kernel source code and noticing that "deadbeaf" clearly can't be achieving what it seems to be intending to do... Is there someone who has a clear explanation of exactly how "deadbeaf" was once expected to work -- and now (since sometime before about 2.5.40) evidently doesn't? It seems to be driven by side effects, and whatever comments are in the code aren't any help. The only case "deadbeaf" could be set is still documented as an error path ... but evidently those USB drivers don't hit that "error" path any more on 2.5 (but they do on 2.4, and did earlier in 2.5 also). My thought is that there were some bugs covering for each other, and one of them got fixed ... exposing this. But without knowing what the networking code was really expecting to do, I can't fix anything. > Then why will grep of all kernel files not turn up other places where > 'deadbeaf' gets set? There's strange stuff going on here regardless > (as well as speling issue), which looks pretty buglike. > > Plus: this kind of bugcatch should use magic numbers, or maybe zero. > Assuming "any nonzero value is valid", like this assertion does, is > clearly going to fail for any of the class of bugs highlighted by > slab poisoning. (0xa5a5a5a5 gets accepted as valid...) > > >>> unregister_netdevice: device /dfd74058 never was registered >> >> >> >> From a short browsing through usb.c I don't see a similiar bug catcher >> in usb_device_remove(), so have a look if the USB subsystem itself >> removes a unplugged device twice for some reason. > > > At least one failure path also involves "rmmod" of the network > drivers, where the device hardware is still around; so that code > would not always be called. > > I wouldn't rule out problems in the relevant usbcore/sysfs bits, > even now that they seem to have stabilized again (and yes, I was > wondering about multiple disconnects too), but all that deadbeaf > logic still looks fishy to me. Right now I _would_ absolutely rule out such a problem. And that "deadbeaf" stuff still looks more than a little bit dubious. > - Dave > > > From hadi@cyberus.ca Mon Dec 9 05:32:15 2002 Received: with ECARTIS (v1.0.0; list netdev); Mon, 09 Dec 2002 05:32:19 -0800 (PST) Received: from cyberus.ca (mail.cyberus.ca [216.191.240.111]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB9DWEuR020182 for ; Mon, 9 Dec 2002 05:32:15 -0800 Received: from shell.cyberus.ca (shell [216.191.240.114]) by cyberus.ca (8.9.3/8.9.3/Cyberus Online Inc.) with ESMTP id IAA27259; Mon, 9 Dec 2002 08:35:44 -0500 (EST) Received: from localhost (hadi@localhost) by shell.cyberus.ca (8.11.6+Sun/8.11.6) with ESMTP id gB9DRdU19286; Mon, 9 Dec 2002 08:27:39 -0500 (EST) X-Authentication-Warning: shell.cyberus.ca: hadi owned process doing -bs Date: Mon, 9 Dec 2002 08:27:39 -0500 (EST) From: jamal To: Jeff Garzik cc: Stefan Rompf , "David S. Miller" , Subject: Re: Patch resubmission: RFC2863 operstatus for 2.5.50 In-Reply-To: <3DEE3E6E.30407@pobox.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 1337 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev Content-Length: 3494 Lines: 79 On Wed, 4 Dec 2002, Jeff Garzik wrote: > jamal wrote: > > Stefans curtrent patch makes the info available via netlink. > > He has two patches: link change notification, and RFC2863 operstatus. > I agree with the first one and support its inclusion; the second one I > question its need. > > (just for context, I am referring to message <3DED2EA9.D812C881@isg.de> > dated Dec 3) > > > > What dont you like about it Jeff? Take a quick look at RFC2863 > > and scan for IfAdminStatus and IfOperStatus. The modelling RFC2863 > > has is pretty good and thats what Stefan has followed (after we weeded > > a few crappy pieces off the RFC; we discussed on netdev). > > I had looked at ifAdminStatus and ifOperStatus before I posted :) > > My argument is, _after_ Stefan's link state patch is applied, why do we > need the additional patch? [this question is meant to be delivered in > an honest, not snide way...] > I somehow deleted the original email he sent with the patches. What two patches? I thought he had one which was a backport and another which was for 2.5.x (sorry, i actually have seen those patches a few times so i didnt bother reviewing anything); In any case, when you look at this stuff think as well of devices that are software netddevices example VLANs or PPP or some of the USB, Irda etc and you want the status properly reflected (and some of that status may not make sense to ethernet for example). > For ifAdminStatus, you have "up", "down", and "testing" states. Since > we have no concept of a testing state, if we eliminate that we have "up" > and "down", two states we can obviously handle. > > For ifOperStatus, we have "up", "down" and "testing", which are > applicable (or not) to Linux as with ifAdminStatus. Further we have > states "dormant", "unknown", "notPresent", "lowerLayerDown". I'll > discuss each of these in detail. > > "dormant" - not used in Stefan's patch, which I agree with. > "unknown" - only used in Stefan's patch before interface is first up'd, > which is IMO inaccurate. Accurate use of "up" and "down", to me, > implies -no- use of "unknown" state. Because as soon as we are > initialized, all ethernet details are known, thus "down" is more > applicable. The Linux net stack's atomicity is such that leaking an > "unknown" state would be a bug, too. > "notPresent" - analagous to Linux's netif_device_xxx, and Stefan's patch > acknowledges this. However, the use of netif_device_xxx in drivers is > really only used when the hardware is suspended. If hardware goes away, > the interface goes away too, almost immediately. > "lowerLayerDown" - not used in Stefan's patch, which I agree with. > > So, Stefan's 2nd patch really only adds "unknown" and "notPresent" > states to current behavior -- and the applicability of those states to > Linux is IMO questionable. > If all he is adding are some enumerated types, theres no harm. I cant remember the details of the discussions - but we had long winding discussions on the different states. Note you need both admin and operational status and physically thet may mean different things (think PPP and ethernet for example). You also need to properly reflect the status for all sorts of netdevices. As long as those requirements are met, then all is good. The RFC is not the final word but it draws from experiences people had for years doing this kind of stuff - so it is a good idea to draw from their experiences (but ok to ignore it if it sounds unrealistic). cheers, jamal From hadi@cyberus.ca Mon Dec 9 05:34:15 2002 Received: with ECARTIS (v1.0.0; list netdev); Mon, 09 Dec 2002 05:34:17 -0800 (PST) Received: from cyberus.ca (mail.cyberus.ca [216.191.240.111]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB9DYEuR020394 for ; Mon, 9 Dec 2002 05:34:15 -0800 Received: from shell.cyberus.ca (shell [216.191.240.114]) by cyberus.ca (8.9.3/8.9.3/Cyberus Online Inc.) with ESMTP id IAA27953; Mon, 9 Dec 2002 08:37:45 -0500 (EST) Received: from localhost (hadi@localhost) by shell.cyberus.ca (8.11.6+Sun/8.11.6) with ESMTP id gB9DTjG19296; Mon, 9 Dec 2002 08:29:45 -0500 (EST) X-Authentication-Warning: shell.cyberus.ca: hadi owned process doing -bs Date: Mon, 9 Dec 2002 08:29:45 -0500 (EST) From: jamal To: Ronghua Zhang cc: Subject: Re: synchronization between bottom half and user context In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 1338 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev Content-Length: 1369 Lines: 50 To reitarate: 1) serialization is against n-CPUs towards the socket. 2) ordered data (dont care whether its 1 or n-CPUs). TCP cares, UDP doesnt. The main TCP path towards the socket runs in softirq context - you can have the same softirq code running on multiple CPUs at the same time. The system call is in process context. cheers, jamal On Wed, 4 Dec 2002, Ronghua Zhang wrote: > Do you mean that net_bh can be running on several CPU at the same time? > If it is juat a UP, can I say that socket system call and net_bh is always > serialized even without lock_sock()? > > Ronghua > > On Wed, 4 Dec 2002, jamal wrote: > > > > > > > On Tue, 3 Dec 2002, Ronghua Zhang wrote: > > > > > Can someone tell me when lock should be used to synchronize bh and user > > > context? > > > > > > For examle: in tcp_accept(), lock_sock() is called to avoid any change > > > made by net_bh during its operation. But if tcp_accept() is called, it's > > > impossilbe that net_bh is also executing. (i think this is true for UP, > > > what about SMP?), then why need lock_sock()? > > > > > > > to serialize packets towards the socket; The linux network stack > > is threaded, you can have upto n packets to the same socket coming up the > > stack in parallel on an n-SMP machine; think of that and the fact that TCP > > data must be sequenced. > > > > cheers, > > jamal > > > > From david-b@pacbell.net Mon Dec 9 11:54:42 2002 Received: with ECARTIS (v1.0.0; list netdev); Mon, 09 Dec 2002 11:54:47 -0800 (PST) Received: from mta7.pltn13.pbi.net (mta7.pltn13.pbi.net [64.164.98.8]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB9JsguR030853 for ; Mon, 9 Dec 2002 11:54:42 -0800 Received: from pacbell.net ([67.118.246.52]) by mta7.pltn13.pbi.net (iPlanet Messaging Server 5.1 HotFix 1.6 (built Oct 18 2002)) with ESMTP id <0H6V005K7BGZ82@mta7.pltn13.pbi.net> for netdev@oss.sgi.com; Mon, 09 Dec 2002 11:58:14 -0800 (PST) Date: Mon, 09 Dec 2002 11:51:18 -0800 From: David Brownell Subject: Re: 2.5.50 BUG_TRAP on !dev->deadbeaf, and oopses To: Stefan Rompf Cc: netdev@oss.sgi.com Message-id: <3DF4F436.6020807@pacbell.net> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=us-ascii Content-transfer-encoding: 7bit X-Accept-Language: en-us, en, fr User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.9) Gecko/20020513 References: <3DE9290A.7070502@pacbell.net> <3DEA0452.B1F15BFD@isg.de> <3DEBA9F5.6000606@pacbell.net> <3DF3CAC8.5070309@pacbell.net> X-archive-position: 1339 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: david-b@pacbell.net Precedence: bulk X-list: netdev Content-Length: 1101 Lines: 27 > Is there someone who has a clear explanation of exactly how "deadbeaf" > was once expected to work -- and now (since sometime before about > 2.5.40) evidently doesn't? > > It seems to be driven by side effects, and whatever comments are in > the code aren't any help. The only case "deadbeaf" could be set is > still documented as an error path ... All that still holds true. There's something fishy going on, or just old cruft that's lingered. I suppose I should just patch it and see if contradictory information appears then. >> Plus: this kind of bugcatch should use magic numbers, or maybe zero. >> Assuming "any nonzero value is valid", like this assertion does, is >> clearly going to fail for any of the class of bugs highlighted by >> slab poisoning. (0xa5a5a5a5 gets accepted as valid...) Actually I found a place where the wrong pointer was being used. Heh -- lucky me, to make my point that way ... a stray pointer happened to point to a zero, and so triggered that warning. As a bugcatch it's pretty poor: it wouldn't normally trigger on that kind of bug, either. - Dave From srompf@isg.de Mon Dec 9 15:13:52 2002 Received: with ECARTIS (v1.0.0; list netdev); Mon, 09 Dec 2002 15:13:54 -0800 (PST) Received: from mail.isg.de (rzfoobar.is-asp.com [217.11.194.155]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB9NDpuR005573 for ; Mon, 9 Dec 2002 15:13:52 -0800 Received: from isg.de (Bd37e.pppool.de [213.7.211.126]) by mail.isg.de (Postfix) with ESMTP id DB08DEC3172; Tue, 10 Dec 2002 00:17:16 +0100 (CET) Message-ID: <3DF52302.D0DBE637@isg.de> Date: Tue, 10 Dec 2002 00:10:58 +0100 From: Stefan Rompf X-Mailer: Mozilla 4.79 [en] (X11; U; Linux 2.4.20 i686) X-Accept-Language: en MIME-Version: 1.0 To: netdev@oss.sgi.com Cc: jamal , Jeff Garzik , "David S. Miller" Subject: Re: Patch resubmission: RFC2863 operstatus for 2.5.50 References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-archive-position: 1340 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: srompf@isg.de Precedence: bulk X-list: netdev Content-Length: 1905 Lines: 45 Hi, > > My argument is, _after_ Stefan's link state patch is applied, why do we > > need the additional patch? [this question is meant to be delivered in > > an honest, not snide way...] > > > > I somehow deleted the original email he sent with the patches. What two > patches? I thought he had one which was a backport and another which > was for 2.5.x I've splitted the patch into userspace notification and RFC2863 part to separate the features clearly and get the discussion running again. That seem to have worked ;-) > (sorry, i actually have seen those patches a few times so i > didnt bother reviewing anything); In any case, when you look at this stuff > think as well of devices that are software netddevices example VLANs or > PPP or some of the USB, Irda etc and you want the status properly > reflected (and some of that status may not make sense to ethernet for > example). As an example, how do we flag a sleeping dial on demand device with the current Linux semantics? Is it oper up, because it may be able to transmit packets and should be considered, or is it oper down, simply as no protocol has been negotiated. RFC2863 can provice a clear state: Dormant. Beside this, we currently have the situation that we can put a device into admin up and not present, simply by calling netdev_carrier_on() and netdev_detach(). I consider that broken, a device cannot be both removed and ready. I'm aware that I did not provide updates to drivers to use the new states, but that's just the nature of infrastructure creating patches. Also, I cannot forwarding the RFC2863 state to userspace as long as Alexey refuses space in the netlink message for it. Ok, IMHO the pro and contra arguments are said. So let's decide: Do we want RFC2863 semantics (of course yes ;-) and David either accepts the second part of the patch or I continue working on it, or do we just drop that part? Stefan From garzik@gtf.org Mon Dec 9 15:22:34 2002 Received: with ECARTIS (v1.0.0; list netdev); Mon, 09 Dec 2002 15:22:35 -0800 (PST) Received: from havoc.gtf.org (havoc.daloft.com [64.213.145.173]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB9NMXuR006008 for ; Mon, 9 Dec 2002 15:22:33 -0800 Received: by havoc.gtf.org (Postfix, from userid 500) id 2E2DC6644; Mon, 9 Dec 2002 18:25:59 -0500 (EST) Date: Mon, 9 Dec 2002 18:25:59 -0500 From: Jeff Garzik To: Stefan Rompf Cc: netdev@oss.sgi.com Subject: Re: Patch: link state detection for 8139too against 2.4.20rc2 / 2.5 Message-ID: <20021209232559.GA9473@gtf.org> References: <3DA96BCC.B2589AC0@isg.de> <3DE0CA4C.730699FC@isg.de> <3DF51ADB.47E83F81@isg.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3DF51ADB.47E83F81@isg.de> User-Agent: Mutt/1.3.28i X-archive-position: 1341 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jgarzik@pobox.com Precedence: bulk X-list: netdev Content-Length: 1139 Lines: 37 On Mon, Dec 09, 2002 at 11:36:11PM +0100, Stefan Rompf wrote: > Hi, > > Jeff Garzik wrote: > > > > ...that was some days ago. I've seen that Jeff is enhancing the MII > > > interface library, so I used the functions provided there. Patch is > > > against 2.4.20rc2 and 2.5 with some offset. Please apply if it looks > > > good. > > [...] > > > close -- you don't want to unconditionally "initialize" the media > > (mii_check_media second arg). > > I've just seen the reason: No need to read the MII_ADVERTISE register > once the structure has been initialized. Expect another patch the next > days. Thanks. > > the phy code is gonna be pretty darned similar. Eventually the phy code > > needs to move to a common 8139lib.c, because both old-8139 line and > > May be. However, I cannot help there. I just have one card of the 8139 > family, and all I could produce is some compiles for me stuff - > something that I would not consider helpful. I would consider it helpful. If you are willing to do the coding and worksforme testing, I can review and do further testing. Part of the power of open source :) Jeff From srompf@isg.de Mon Dec 9 15:50:22 2002 Received: with ECARTIS (v1.0.0; list netdev); Mon, 09 Dec 2002 15:50:26 -0800 (PST) Received: from mail.isg.de (rzfoobar.is-asp.com [217.11.194.155]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gB9NoBuR006561 for ; Mon, 9 Dec 2002 15:50:12 -0800 Received: from isg.de (Bd37e.pppool.de [213.7.211.126]) by mail.isg.de (Postfix) with ESMTP id 24E00EC316B; Tue, 10 Dec 2002 00:17:07 +0100 (CET) Message-ID: <3DF51ADB.47E83F81@isg.de> Date: Mon, 09 Dec 2002 23:36:11 +0100 From: Stefan Rompf X-Mailer: Mozilla 4.79 [en] (X11; U; Linux 2.4.20 i686) X-Accept-Language: en MIME-Version: 1.0 To: jgarzik@pobox.com Cc: netdev@oss.sgi.com Subject: Re: Patch: link state detection for 8139too against 2.4.20rc2 / 2.5 References: <3DA96BCC.B2589AC0@isg.de> <3DE0CA4C.730699FC@isg.de> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-archive-position: 1342 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: srompf@isg.de Precedence: bulk X-list: netdev Content-Length: 848 Lines: 28 Hi, Jeff Garzik wrote: > > ...that was some days ago. I've seen that Jeff is enhancing the MII > > interface library, so I used the functions provided there. Patch is > > against 2.4.20rc2 and 2.5 with some offset. Please apply if it looks > > good. [...] > close -- you don't want to unconditionally "initialize" the media > (mii_check_media second arg). I've just seen the reason: No need to read the MII_ADVERTISE register once the structure has been initialized. Expect another patch the next days. > the phy code is gonna be pretty darned similar. Eventually the phy code > needs to move to a common 8139lib.c, because both old-8139 line and May be. However, I cannot help there. I just have one card of the 8139 family, and all I could produce is some compiles for me stuff - something that I would not consider helpful. Stefan From garzik@gtf.org Mon Dec 9 19:52:51 2002 Received: with ECARTIS (v1.0.0; list netdev); Mon, 09 Dec 2002 19:53:00 -0800 (PST) Received: from havoc.gtf.org (havoc.daloft.com [64.213.145.173]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBA3qKuR009739 for ; Mon, 9 Dec 2002 19:52:51 -0800 Received: by havoc.gtf.org (Postfix, from userid 500) id 34FF36643; Mon, 9 Dec 2002 22:55:32 -0500 (EST) Date: Mon, 9 Dec 2002 22:55:32 -0500 From: Jeff Garzik To: jamal Cc: Stefan Rompf , "David S. Miller" , netdev@oss.sgi.com Subject: Re: Patch resubmission: RFC2863 operstatus for 2.5.50 Message-ID: <20021210035532.GA22559@gtf.org> References: <3DEE3E6E.30407@pobox.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.3.28i X-archive-position: 1344 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jgarzik@pobox.com Precedence: bulk X-list: netdev Content-Length: 2710 Lines: 56 On Mon, Dec 09, 2002 at 08:27:39AM -0500, jamal wrote: > On Wed, 4 Dec 2002, Jeff Garzik wrote: > > For ifAdminStatus, you have "up", "down", and "testing" states. Since > > we have no concept of a testing state, if we eliminate that we have "up" > > and "down", two states we can obviously handle. > > > > For ifOperStatus, we have "up", "down" and "testing", which are > > applicable (or not) to Linux as with ifAdminStatus. Further we have > > states "dormant", "unknown", "notPresent", "lowerLayerDown". I'll > > discuss each of these in detail. > > > > "dormant" - not used in Stefan's patch, which I agree with. > > "unknown" - only used in Stefan's patch before interface is first up'd, > > which is IMO inaccurate. Accurate use of "up" and "down", to me, > > implies -no- use of "unknown" state. Because as soon as we are > > initialized, all ethernet details are known, thus "down" is more > > applicable. The Linux net stack's atomicity is such that leaking an > > "unknown" state would be a bug, too. > > "notPresent" - analagous to Linux's netif_device_xxx, and Stefan's patch > > acknowledges this. However, the use of netif_device_xxx in drivers is > > really only used when the hardware is suspended. If hardware goes away, > > the interface goes away too, almost immediately. > > "lowerLayerDown" - not used in Stefan's patch, which I agree with. > > > > So, Stefan's 2nd patch really only adds "unknown" and "notPresent" > > states to current behavior -- and the applicability of those states to > > Linux is IMO questionable. > > > > If all he is adding are some enumerated types, theres no harm. I cant > remember the details of the discussions - but we had long winding > discussions on the different states. I would summarize his patch as adding variable to represent literally ifOperStatus, along with a lock and apparatus to set this variable. The value may be deduced, without having to literally track it. > Note you need both admin and operational status and physically thet may > mean different things (think PPP and ethernet for example). You also need > to properly reflect the status for all sorts of netdevices. As long as > those requirements are met, then all is good. The RFC is not the final > word but it draws from experiences people had for years doing this kind > of stuff - so it is a good idea to draw from their experiences (but ok to > ignore it if it sounds unrealistic). I think we do agree on interpretation as you describe it here. And the linkwatch patch is now in Linux 2.5.51. I just think the further patch to "track literally" ifOperStatus is not needed. However, that is presented as an opinion and RFC, not a statement of fact :) Jeff From kiran@in.ibm.com Tue Dec 10 05:23:12 2002 Received: with ECARTIS (v1.0.0; list netdev); Tue, 10 Dec 2002 05:23:17 -0800 (PST) Received: from e5.ny.us.ibm.com (e5.ny.us.ibm.com [32.97.182.105]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBADNAuR022453 for ; Tue, 10 Dec 2002 05:23:11 -0800 Received: from northrelay04.pok.ibm.com (northrelay04.pok.ibm.com [9.56.224.206]) by e5.ny.us.ibm.com (8.12.2/8.12.2) with ESMTP id gBADQeQ0146262; Tue, 10 Dec 2002 08:26:40 -0500 Received: from asterix.in.ibm.com (asterix.in.ibm.com [9.182.12.249]) by northrelay04.pok.ibm.com (8.12.3/NCO/VER6.4) with ESMTP id gBADQQ3X049804; Tue, 10 Dec 2002 08:26:34 -0500 Received: (from kirant@localhost) by asterix.in.ibm.com (8.11.2/8.11.2) id gBADP2u16901; Tue, 10 Dec 2002 18:55:02 +0530 Date: Tue, 10 Dec 2002 18:55:02 +0530 From: Ravikiran G Thirumalai To: "David S. Miller " Cc: netdev , linux-kernel@vger.kernel.org Subject: [trivial] Use __init and __exit for sctp_init and sctp_exit Message-ID: <20021210185502.G17375@in.ibm.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i X-archive-position: 1345 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kiran@in.ibm.com Precedence: bulk X-list: netdev Content-Length: 777 Lines: 31 These module_init and module_exit routines should have been marked __init and __exit like every where else I guess... Please apply Thanks, Kiran diff -ruN -X dontdiff linux-2.5.51/net/sctp/protocol.c sctp_fix-2.5.51/net/sctp/protocol.c --- linux-2.5.51/net/sctp/protocol.c Tue Dec 10 08:16:10 2002 +++ sctp_fix-2.5.51/net/sctp/protocol.c Tue Dec 10 18:48:07 2002 @@ -598,7 +598,7 @@ } /* Initialize the universe into something sensible. */ -int sctp_init(void) +int __init sctp_init(void) { int i; int status = 0; @@ -751,7 +751,7 @@ } /* Exit handler for the SCTP protocol. */ -void sctp_exit(void) +void __exit sctp_exit(void) { /* BUG. This should probably do something useful like clean * up all the remaining associations and all that memory. From kaber@trash.net Tue Dec 10 05:32:51 2002 Received: with ECARTIS (v1.0.0; list netdev); Tue, 10 Dec 2002 05:32:54 -0800 (PST) Received: from el-zoido.localnet (port-212-202-202-93.reverse.qdsl-home.de [212.202.202.93]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBADWnuR023761 for ; Tue, 10 Dec 2002 05:32:50 -0800 Received: from trash.net (ws.localnet [192.168.0.23]) by el-zoido.localnet (8.11.6/linuxconf) with ESMTP id gBADaia04871; Tue, 10 Dec 2002 14:36:44 +0100 Message-ID: <3DF5EDBE.7000707@trash.net> Date: Tue, 10 Dec 2002 14:35:58 +0100 From: Patrick McHardy User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20021208 Debian/1.2.1-2 X-Accept-Language: en MIME-Version: 1.0 To: Netfilter Development Mailinglist CC: netdev@oss.sgi.com Subject: [PATCH]: forgotten dev_put for bridge-devices in nf_reinject Content-Type: multipart/mixed; boundary="------------000505020000010609080306" X-archive-position: 1346 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kaber@trash.net Precedence: bulk X-list: netdev Content-Length: 1201 Lines: 43 This is a multi-part message in MIME format. --------------000505020000010609080306 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Hi. The attached patch releases the bridge devices grabbed in nf_queue after reinjecting the packet. Seems someone forgot these (or planned to get Rusty killed) ;) Bye, Patrick --------------000505020000010609080306 Content-Type: text/plain; name="nfqueue-bridge-devput.diff" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="nfqueue-bridge-devput.diff" --- linux-2.5.50/net/core/netfilter.c.orig 2002-12-10 14:16:20.000000000 +0100 +++ linux-2.5.50/net/core/netfilter.c 2002-12-10 14:19:23.000000000 +0100 @@ -574,7 +574,15 @@ /* Release those devices we held, or Alexey will kill me. */ if (info->indev) dev_put(info->indev); if (info->outdev) dev_put(info->outdev); - +#if defined(CONFIG_BRIDGE) || defined(CONFIG_BRIDGE_MODULE) + if (skb->nf_bridge) { + if (skb->nf_bridge->physindev) + dev_put(skb->nf_bridge->physindev); + if (skb->nf_bridge->physoutdev) + dev_put(skb->nf_bridge->physoutdev); + } +#endif + kfree(info); return; } --------------000505020000010609080306-- From manfred@colorfullife.com Tue Dec 10 12:32:00 2002 Received: with ECARTIS (v1.0.0; list netdev); Tue, 10 Dec 2002 12:32:07 -0800 (PST) Received: from dbl.q-ag.de (IDENT:root@dbl.q-ag.de [80.146.160.66]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBAKVwuR012082 for ; Tue, 10 Dec 2002 12:31:59 -0800 Received: from colorfullife.com (IDENT:root@localhost.localdomain [127.0.0.1]) by dbl.q-ag.de (8.11.6/8.11.6) with ESMTP id gBAKZXH07537 for ; Tue, 10 Dec 2002 21:35:33 +0100 Message-ID: <3DF65013.2090303@colorfullife.com> Date: Tue, 10 Dec 2002 21:35:31 +0100 From: Manfred Spraul User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2) Gecko/20021202 X-Accept-Language: en-us, en MIME-Version: 1.0 To: netdev@oss.sgi.com Subject: ether_crc wrong in 2.5.50 Content-Type: multipart/mixed; boundary="------------040607060806030409070906" X-archive-position: 1347 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: manfred@colorfullife.com Precedence: bulk X-list: netdev Content-Length: 4800 Lines: 169 This is a multi-part message in MIME format. --------------040607060806030409070906 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit The ether_crc were converted to use the crc32 library in 2.5. The conversion is incorrect: crc32_le and _be are about the bit order in which the bits are processed: _le means least significant bit first, _be msb first. ethernet is always lsb first. ether_crc means the output should be in the cpu bit order [msb in bit 31], ether_crc_le means the output should be with msb in bit 0: that what crc32_le usually generates. The attached patch is tested with winbond-840, natsemi and 8139too, i.e. ether_crc works. ether_crc_le generates the same output as the inline functions from 2.4.18, but it's untested due to lack of nics. Could someone test it on a nic that uses ether_crc_le? -- Manfred --------------040607060806030409070906 Content-Type: text/plain; name="patch-crc" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="patch-crc" // $Header$ // Kernel Version: // VERSION = 2 // PATCHLEVEL = 5 // SUBLEVEL = 50 // EXTRAVERSION = --- 2.5/include/linux/crc32.h 2002-11-04 23:30:16.000000000 +0100 +++ build-2.5/include/linux/crc32.h 2002-12-10 19:15:50.000000000 +0100 @@ -9,9 +9,19 @@ extern u32 crc32_le(u32 crc, unsigned char const *p, size_t len); extern u32 crc32_be(u32 crc, unsigned char const *p, size_t len); +extern u32 bitreverse(u32 in); #define crc32(seed, data, length) crc32_le(seed, (unsigned char const *)data, length) + +/* + * Helpers for hash table generation of ethernet nics: + * + * Ethernet sends the least significant bit of a byte first, thus crc32_le + * is used. The output of crc32_le is bit reversed [most significant bit + * is in bit nr 0], thus it must be reversed before use. Except for + * nics that bit swap the result internally... + */ +#define ether_crc(length, data) bitreverse(crc32_le(~0, data, length)) #define ether_crc_le(length, data) crc32_le(~0, data, length) -#define ether_crc(length, data) crc32_be(~0, data, length) #endif /* _LINUX_CRC32_H */ --- 2.5/lib/crc32.c 2002-11-04 23:30:03.000000000 +0100 +++ build-2.5/lib/crc32.c 2002-12-10 19:16:47.000000000 +0100 @@ -255,6 +255,16 @@ } #endif +u32 bitreverse(u32 x) +{ + x = (x >> 16) | (x << 16); + x = (x >> 8 & 0x00ff00ff) | (x << 8 & 0xff00ff00); + x = (x >> 4 & 0x0f0f0f0f) | (x << 4 & 0xf0f0f0f0); + x = (x >> 2 & 0x33333333) | (x << 2 & 0xcccccccc); + x = (x >> 1 & 0x55555555) | (x << 1 & 0xaaaaaaaa); + return x; +} + /* * A brief CRC tutorial. * @@ -399,16 +409,6 @@ } #endif -static u32 attribute((const)) bitreverse(u32 x) -{ - x = (x >> 16) | (x << 16); - x = (x >> 8 & 0x00ff00ff) | (x << 8 & 0xff00ff00); - x = (x >> 4 & 0x0f0f0f0f) | (x << 4 & 0xf0f0f0f0); - x = (x >> 2 & 0x33333333) | (x << 2 & 0xcccccccc); - x = (x >> 1 & 0x55555555) | (x << 1 & 0xaaaaaaaa); - return x; -} - static void bytereverse(unsigned char *buf, size_t len) { while (len--) { --- 2.5/drivers/net/natsemi.c 2002-11-04 23:30:05.000000000 +0100 +++ build-2.5/drivers/net/natsemi.c 2002-12-10 19:20:01.000000000 +0100 @@ -164,6 +164,7 @@ #include #include #include +#include #include /* Processor type for cache alignment. */ #include #include @@ -1898,44 +1899,6 @@ return &np->stats; } -/** - * dp83815_crc - computer CRC for hash table entries - * - * Note - this is, for some reason, *not* the same function - * as ether_crc_le() or ether_crc(), though it uses the - * same big-endian polynomial. - */ -#define DP_POLYNOMIAL 0x04C11DB7 -static unsigned dp83815_crc(int length, unsigned char *data) -{ - u32 crc; - u8 cur_byte; - u8 msb; - u8 byte, bit; - - crc = ~0; - for (byte=0; byte> 31; - crc <<= 1; - if (msb ^ (cur_byte & 1)) { - crc ^= DP_POLYNOMIAL; - crc |= 1; - } - cur_byte >>= 1; - } - } - crc >>= 23; - - return (crc); -} - - -void set_bit_le(int offset, unsigned char * data) -{ - data[offset >> 3] |= (1 << (offset & 0x07)); -} #define HASH_TABLE 0x200 static void __set_rx_mode(struct net_device *dev) { @@ -1960,9 +1923,8 @@ memset(mc_filter, 0, sizeof(mc_filter)); for (i = 0, mclist = dev->mc_list; mclist && i < dev->mc_count; i++, mclist = mclist->next) { - set_bit_le( - dp83815_crc(ETH_ALEN, mclist->dmi_addr) & 0x1ff, - mc_filter); + int i = (ether_crc(ETH_ALEN, mclist->dmi_addr) >> 23) & 0x1ff; + mc_filter[i/8] |= (1 << (i & 0x07)); } rx_mode = RxFilterEnable | AcceptBroadcast | AcceptMulticast | AcceptMyPhys; --------------040607060806030409070906-- From rl@hellgate.ch Wed Dec 11 09:56:42 2002 Received: with ECARTIS (v1.0.0; list netdev); Wed, 11 Dec 2002 09:56:47 -0800 (PST) Received: from mta9n.bluewin.ch (mta9n.bluewin.ch [195.186.1.215]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBBHufuR024506 for ; Wed, 11 Dec 2002 09:56:42 -0800 Received: from k3.hellgate.ch (195.186.128.45) by mta9n.bluewin.ch (Bluewin AG 6.5.032) id 3DAA821D00AD585C; Wed, 11 Dec 2002 14:24:40 +0100 Received: by k3.hellgate.ch (MTA, from userid 500) id C806932215; Wed, 11 Dec 2002 14:24:36 +0100 (CET) Date: Wed, 11 Dec 2002 14:24:36 +0100 From: Roger Luethi To: netdev@oss.sgi.com Subject: pci-skeleton duplex check Message-ID: <20021211132436.GA12529@k3.hellgate.ch> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.3.27i X-Operating-System: Linux 2.5.44 on i686 X-GPG-Fingerprint: 92 F4 DC 20 57 46 7B 95 24 4E 9E E7 5A 54 DC 1B X-GPG: 1024/80E744BD wwwkeys.ch.pgp.net X-archive-position: 1349 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: rl@hellgate.ch Precedence: bulk X-list: netdev Content-Length: 300 Lines: 10 pci-skeleton (and some other net drivers) figure out the duplex setting like this (leaving duplex locks out here): int duplex = (lpar & 0x0100) || (lpar & 0x01C0) == 0x0040; If we get past the first condition, we already know bit 8 must be 0. Why do we check again in the second condition? Roger From mauro@deepspace6.net Wed Dec 11 16:22:39 2002 Received: with ECARTIS (v1.0.0; list netdev); Wed, 11 Dec 2002 16:22:46 -0800 (PST) Received: from smtp1.libero.it (smtp1.libero.it [193.70.192.51]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBC0McuR004953 for ; Wed, 11 Dec 2002 16:22:39 -0800 Received: from trantor.ferrara.linux.it (151.26.185.215) by smtp1.libero.it (6.7.015) id 3DE22B81007651A0; Wed, 11 Dec 2002 22:48:09 +0100 Received: by trantor.ferrara.linux.it (Postfix, from userid 501) id 2DC3F18F0D; Wed, 11 Dec 2002 22:38:07 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by trantor.ferrara.linux.it (Postfix) with ESMTP id 7E60D18F09; Wed, 11 Dec 2002 22:38:07 +0100 (CET) Date: Wed, 11 Dec 2002 22:38:06 +0100 (CET) From: Mauro Tortonesi To: ds6-www@deepspace6.net, , , , , , , Subject: deepspace6 official announcement Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 1350 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: mauro@deepspace6.net Precedence: bulk X-list: netdev Content-Length: 1905 Lines: 49 We are proud to announce the birth of a new Linux-related IPv6 portal: http://www.deepspace6.net. The Deep Space 6 initiative is born from the experience of two of the most important Linux- and IPv6-related websites: Project 6 and Peter Bieringer's www.bieringer.de website. We aim at being a reference website where the users can find information about the use of the new IPv6 protocol with the Linux operating system. Our site is still in construction at this moment, but already features: - Peter Bieringer's Linux+IPv6 HOWTO and many other interesting articles - many mailing lists for the discussion about every aspect of the use of IPv6 under Linux - a rich collection of IPv6-enabled opensource software - anonymous and web CVS access to our projects You are invited to visit our website and send us your comments about our work or, better, join us in our effort to: - provide useful information about the use of the new IPv6 protocol with the Linux operating system - enhance the IPv6 compliance of existing opensource software - develop new IPv6-based opensource solutions and services Deep Space 6 is an initiative which, right from its birth, is open to every kind of collaboration with volunteers and other similar projects. We are willing to: - publish IPv6-related articles and HOWTOs - host new IPv6-related opensource software projects and provide our help in their development - collaborate with other projects and/or add a link to their homepages - create new mailing lists for especially interesting arguments or projects We are looking forward to receive your feedback and, maybe, to start a new collaboration with you ;-) -- Aequam memento rebus in arduis servare mentem... Mauro Tortonesi mauro@deepspace6.net mauro@ferrara.linux.it Deep Space 6 - IPv6 on Linux http://www.deepspace6.net Ferrara Linux Users Group http://www.ferrara.linux.it From zjp@iscas.ac.cn Wed Dec 11 18:05:04 2002 Received: with ECARTIS (v1.0.0; list netdev); Wed, 11 Dec 2002 18:05:13 -0800 (PST) Received: from mail.iscas.ac.cn ([159.226.5.56]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBC252uR005919 for ; Wed, 11 Dec 2002 18:05:03 -0800 Received: (qmail 7402 invoked by uid 104); 12 Dec 2002 02:08:40 -0000 Received: from zjp@iscas.ac.cn by mail.iscas.ac.cn by uid 0 with qmail-scanner-1.14 (hbedv: 6.15.0.1. hbedv: operating system: Linux (glibc). hbedv: product version: 2.0.4. hbedv: engine version: 6.15.0.1. hbedv: packlib version: 2.0.0.8 (supports 19 formats). hbedv: vdf version: 6.15.0.7 (66928 recognized forms). hbedv: . hbedv: product: AntiVir Workstation. hbedv: key file: hbedv.key. hbedv: registered user: irene, 123. hbedv: serial number: 1001020203. hbedv: key expires: 31 May 2003. hbedv: run mode: PRIVATE. hbedv: . hbedv: product: AntiVir MailGate. hbedv: key file: hbedv.key. hbedv: registered user: irene, 123. hbedv: serial number: 1001020203. hbedv: key expires: 31 May 2003. hbedv: run mode: PRIVATE. hbedv: . hbedv: product: AntiVir (command line scanner). hbedv: key file: hbedv.key. hbedv: registered user: irene, 123. hbedv: serial number: 1001020203. hbedv: key expires: 31 May 2003. hbedv: run mode: PRIVATE. Clear:. Processed in 0.213252 secs); 12 Dec 2002 02:08:40 -0000 Received: from unknown (HELO zhengjp) (zjp@192.168.6.108) by mail.iscas.ac.cn with SMTP; 12 Dec 2002 02:08:39 -0000 Message-ID: <005f01c2a183$c590bce0$6c06a8c0@zhengjp> From: "Zheng Jianping" To: "yoshfuji" Cc: References: <000d01c2975d$a2ad3c10$6c06a8c0@zhengjp> <20021129.131656.766932499.yoshfuji@linux-ipv6.org> Subject: Re: How to get the IPv6 address of next hop towards destination? Date: Thu, 12 Dec 2002 10:11:24 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2919.6700 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700 X-archive-position: 1351 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: zjp@iscas.ac.cn Precedence: bulk X-list: netdev Content-Length: 365 Lines: 23 Hi, Yoshfuji, What's type of the address stored in primary_key[0], link-local or global? Thanks, Zheng > In article <000d01c2975d$a2ad3c10$6c06a8c0@zhengjp> (at Fri, 29 Nov 2002 12:13:13 +0800), "Zheng Jianping" says: > > > It seems that > > u8 primary_key[0]; > > is responsible for it. > > > > Is it right? > > yes. > > --yoshfuji > > From becker@scyld.com Wed Dec 11 18:38:55 2002 Received: with ECARTIS (v1.0.0; list netdev); Wed, 11 Dec 2002 18:38:57 -0800 (PST) Received: from beohost.scyld.com (pool-151-196-175-148.balt.east.verizon.net [151.196.175.148]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBC2cruR006543 for ; Wed, 11 Dec 2002 18:38:53 -0800 Received: from localhost (becker@localhost) by beohost.scyld.com (8.11.6/8.11.6) with ESMTP id gBC2gio12384; Wed, 11 Dec 2002 21:42:45 -0500 Date: Wed, 11 Dec 2002 21:42:44 -0500 (EST) From: Donald Becker To: Roger Luethi cc: netdev@oss.sgi.com Subject: Re: pci-skeleton duplex check In-Reply-To: <20021211132436.GA12529@k3.hellgate.ch> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 1352 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: becker@scyld.com Precedence: bulk X-list: netdev Content-Length: 1778 Lines: 48 On Wed, 11 Dec 2002, Roger Luethi wrote: > pci-skeleton (and some other net drivers) figure out the duplex setting > like this (leaving duplex locks out here): > > int duplex = (lpar & 0x0100) || (lpar & 0x01C0) == 0x0040; I'm not certain what version of pci-skeleton.c you are looking at, but the current version is int mii_reg5 = mdio_read(dev, np->phys[0], 5); int negotiated = mii_reg5 & np->advertising; int duplex = (negotiated & 0x0100) || (negotiated & 0x01C0) == 0x0040; We are looking for the highest negotiated capability, according to the rules described in http://scyld.com/expert/NWay.html You did read the reference documentation, didn't you? I put the documentation references at the top of the driver because you may need to read them to understand what is going on. Or was it so unusual to have written priciples of operation that you skipped that section? I'll explain the cases: (negotiated & 0x0100) 100baseTx full duplex or (negotiated & 0x01C0) == 0x0040 No capability above 10baseT full duplex Note that the second check ignores 100baseT4, despite it have priority over 10baseT-*. That was intentional to work around, "a specific issue" with a transceiver. > If we get past the first condition, we already know bit 8 must be 0. Why do > we check again in the second condition? The is no extra cost to the extra bit, and it makes it clear we are testing for 10baseT-FDX. (The test was originally implemented as part of a complete set of cases. The test code needed while building a driver is more complex than what you see in the concise final result.) -- Donald Becker becker@scyld.com Scyld Computing Corporation http://www.scyld.com 410 Severn Ave. Suite 210 Scyld Beowulf cluster system Annapolis MD 21403 410-990-9993 From hadi@cyberus.ca Thu Dec 12 05:25:42 2002 Received: with ECARTIS (v1.0.0; list netdev); Thu, 12 Dec 2002 05:25:46 -0800 (PST) Received: from cyberus.ca (mail.cyberus.ca [216.191.240.111]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBCDPguR000644 for ; Thu, 12 Dec 2002 05:25:42 -0800 Received: from shell.cyberus.ca (shell [216.191.240.114]) by cyberus.ca (8.9.3/8.9.3/Cyberus Online Inc.) with ESMTP id IAA17336; Thu, 12 Dec 2002 08:29:23 -0500 (EST) Received: from localhost (hadi@localhost) by shell.cyberus.ca (8.11.6+Sun/8.11.6) with ESMTP id gBCDLFh28966; Thu, 12 Dec 2002 08:21:15 -0500 (EST) X-Authentication-Warning: shell.cyberus.ca: hadi owned process doing -bs Date: Thu, 12 Dec 2002 08:21:14 -0500 (EST) From: jamal To: Jeff Garzik cc: Stefan Rompf , "David S. Miller" , Subject: Re: Patch resubmission: RFC2863 operstatus for 2.5.50 In-Reply-To: <20021210035532.GA22559@gtf.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 1353 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hadi@cyberus.ca Precedence: bulk X-list: netdev Content-Length: 509 Lines: 20 On Mon, 9 Dec 2002, Jeff Garzik wrote: > I would summarize his patch as adding variable to represent literally > ifOperStatus, along with a lock and apparatus to set this variable. > > The value may be deduced, without having to literally track it. If it can be deduced then no point in tracking it. i think my shepherding of the patch is done at this point. I'll let you take it from here. BTW, I still think theres need to clean up the qdisc queues when a link goes operationaly down. cheers, jamal From becker@scyld.com Thu Dec 12 06:07:43 2002 Received: with ECARTIS (v1.0.0; list netdev); Thu, 12 Dec 2002 06:07:46 -0800 (PST) Received: from beohost.scyld.com (pool-151-196-173-209.balt.east.verizon.net [151.196.173.209]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBCE7fuR001464 for ; Thu, 12 Dec 2002 06:07:42 -0800 Received: from localhost (becker@localhost) by beohost.scyld.com (8.11.6/8.11.6) with ESMTP id gBCEBYh17269; Thu, 12 Dec 2002 09:11:34 -0500 Date: Thu, 12 Dec 2002 09:11:34 -0500 (EST) From: Donald Becker To: Roger Luethi cc: netdev@oss.sgi.com Subject: Re: pci-skeleton duplex check In-Reply-To: <20021212132033.GA3224@k3.hellgate.ch> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 1354 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: becker@scyld.com Precedence: bulk X-list: netdev Content-Length: 1216 Lines: 28 On Thu, 12 Dec 2002, Roger Luethi wrote: > On Wed, 11 Dec 2002 21:42:44 -0500, Donald Becker wrote: > > Note that the second check ignores 100baseT4, despite it have priority > > over 10baseT-*. That was intentional to work around, "a specific issue" > > with a transceiver. > > That "specific issue" might be worth documenting. Information on such > quirks is hardest to come by. There are some quirks I'll refer to as bugs, and some as "issues". When I get a documented, reliable work-around for a problem that doesn't impact operation or performance, I consider that chip to be non-buggy. I try very hard to avoid NDAs, but I'm willing to sign one when it's clear on both sides what the limits of the NDA are. Now, if I have to track down the problems myself I can't be certain if I've found the full extent or correct fix for what is clearly a _bug_. [[ I don't know why I bother. The people that now control what goes into the kernel would rather put in random patches from other people than accept a correct fix from me. ]] -- Donald Becker becker@scyld.com Scyld Computing Corporation http://www.scyld.com 410 Severn Ave. Suite 210 Scyld Beowulf cluster system Annapolis MD 21403 410-990-9993 From krkumar@us.ibm.com Thu Dec 12 11:03:23 2002 Received: with ECARTIS (v1.0.0; list netdev); Thu, 12 Dec 2002 11:03:25 -0800 (PST) Received: from e2.ny.us.ibm.com (e2.ny.us.ibm.com [32.97.182.102]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBCJ3MuR013795 for ; Thu, 12 Dec 2002 11:03:23 -0800 Received: from northrelay03.pok.ibm.com (northrelay03.pok.ibm.com [9.56.224.151]) by e2.ny.us.ibm.com (8.12.2/8.12.2) with ESMTP id gBCJ718I313140; Thu, 12 Dec 2002 14:07:02 -0500 Received: from gateway1.beaverton.ibm.com (gateway1.beaverton.ibm.com [138.95.180.2]) by northrelay03.pok.ibm.com (8.12.3/NCO/VER6.4) with ESMTP id gBCJ6wBE020902; Thu, 12 Dec 2002 14:06:59 -0500 Received: from eng2.beaverton.ibm.com (eng2.beaverton.ibm.com [9.47.57.17]) by gateway1.beaverton.ibm.com (8.11.6/8.11.6) with ESMTP id gBCJ0jV27734; Thu, 12 Dec 2002 11:00:45 -0800 Received: (from kkumar@localhost) by eng2.beaverton.ibm.com (8.10.0.Beta10/8.8.5/token.aware-1.2) id gBCJ5hn18058; Thu, 12 Dec 2002 11:05:43 -0800 (PST) From: Krishna Kumar Message-Id: <200212121905.gBCJ5hn18058@eng2.beaverton.ibm.com> Subject: [PATCH RESEND] memory leak in ndisc_router_discovery To: kuznet@ms2.inr.ac.ru, davem@redhat.com Date: Thu, 12 Dec 2002 11:05:43 -0800 (PST) Cc: netdev@oss.sgi.com, linux-kernel@vger.kernel.org X-Mailer: ELM [version 2.5 PL3] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-archive-position: 1355 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: krkumar@us.ibm.com Precedence: bulk X-list: netdev Content-Length: 848 Lines: 25 Hi, I had sent this earlier, there is a bug in router advertisement handling code, where the reference (and memory) to an inet6_dev pointer can get leaked (this leak can happen atmost once for each interface on a system which receives invalid RA's). Below is the patch against 2.5.51 to fix it. thanks, - KK ------------------------------------------------------------------------------- diff -ruN linux.org/net/ipv6/ndisc.c linux/net/ipv6/ndisc.c --- linux.org/net/ipv6/ndisc.c Fri Nov 7 10:02:11 2002 +++ linux/net/ipv6/ndisc.c Fri Nov 8 14:37:27 2002 @@ -871,6 +871,7 @@ } if (!ndisc_parse_options(opt, optlen, &ndopts)) { + in6_dev_put(in6_dev); if (net_ratelimit()) ND_PRINTK2(KERN_WARNING "ICMP6 RA: invalid ND option, ignored.\n"); ------------------------------------------------------------------------------- From jgarzik@pobox.com Thu Dec 12 12:29:55 2002 Received: with ECARTIS (v1.0.0; list netdev); Thu, 12 Dec 2002 12:29:58 -0800 (PST) Received: from www.linux.org.uk (parcelfarce.linux.theplanet.co.uk [195.92.249.252]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBCKTsuR015698 for ; Thu, 12 Dec 2002 12:29:54 -0800 Received: from nat-pool-rdu.redhat.com ([66.187.233.200] helo=pobox.com) by www.linux.org.uk with esmtp (Exim 3.33 #5) id 18Ma1Z-0007Nz-00; Thu, 12 Dec 2002 20:33:38 +0000 Message-ID: <3DF8F26E.50306@pobox.com> Date: Thu, 12 Dec 2002 15:32:46 -0500 From: Jeff Garzik User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2) Gecko/20021202 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Donald Becker CC: Roger Luethi , netdev@oss.sgi.com, Linux Kernel Mailing List Subject: Re: pci-skeleton duplex check References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 1356 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jgarzik@pobox.com Precedence: bulk X-list: netdev Content-Length: 384 Lines: 15 Donald Becker wrote: > [[ I don't know why I bother. The people that now control what goes into > the kernel would rather put in random patches from other people than > accept a correct fix from me. ]] I'm very interested in applying fixes from you! I am publicly begging you to do so, and even CC'ing lkml on my request. Please re-post any patches I or Andrew missed. Jeff From becker@scyld.com Thu Dec 12 13:07:17 2002 Received: with ECARTIS (v1.0.0; list netdev); Thu, 12 Dec 2002 13:07:23 -0800 (PST) Received: from beohost.scyld.com (dsl093-058-082.blt1.dsl.speakeasy.net [66.93.58.82]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBCL7GuR017239 for ; Thu, 12 Dec 2002 13:07:17 -0800 Received: from localhost (becker@localhost) by beohost.scyld.com (8.11.6/8.11.6) with ESMTP id gBCLBDx20424; Thu, 12 Dec 2002 16:11:13 -0500 Date: Thu, 12 Dec 2002 16:11:13 -0500 (EST) From: Donald Becker To: Jeff Garzik cc: Roger Luethi , , Linux Kernel Mailing List Subject: Re: pci-skeleton duplex check In-Reply-To: <3DF8F26E.50306@pobox.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 1357 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: becker@scyld.com Precedence: bulk X-list: netdev Content-Length: 2188 Lines: 47 On Thu, 12 Dec 2002, Jeff Garzik wrote: > Donald Becker wrote: > > [[ I don't know why I bother. The people that now control what goes into > > the kernel would rather put in random patches from other people than > > accept a correct fix from me. ]] > > I'm very interested in applying fixes from you! I am publicly begging > you to do so, and even CC'ing lkml on my request. This is very disingenuous statement. The drivers in the kernel are now heavily modified and have significantly diverged from my version. Sure, you are fine with having someone else do the difficult and unrewarding debugging and maintainence work, while you work on just the latest cool hardware, change the interfaces and are concerned only with the current kernel version. I've been actively developing Linux drivers for over a decade, and run about two dozen mailing lists for specific drivers. I write diagnostic routines for every released driver. I thoroughly test and frequently update the driver set I maintain. And since about 2000, my patches were ignored while the first notice I've have gotten to changes in my drivers is the bug reports. And the response: "submit a patch to fix those newly introduced bugs". I've even had patches ignore in favor of people that wrote "I don't have the NIC, but here is a change". A good example is the tulip driver. You repeatedly restructured my driver in the kernel, splitting into different files. It was still 90+% my code, but the changes made it impossible to track the modification history. The kernel driver was long-broken with 21143-SYM cards, but no one took the responsibility for fixing it. It's easy to make the first few patches, when you don't have to deal with reversion testing, many different models, and have an unlimited sandbox where it doesn't matter if a specific release works or not. But it takes a huge of work to keep a stable, tracable driver development process that works with many different kernel versions and hardware environments. -- Donald Becker becker@scyld.com Scyld Computing Corporation http://www.scyld.com 410 Severn Ave. Suite 210 Scyld Beowulf cluster system Annapolis MD 21403 410-990-9993 From jgarzik@pobox.com Thu Dec 12 13:36:02 2002 Received: with ECARTIS (v1.0.0; list netdev); Thu, 12 Dec 2002 13:36:06 -0800 (PST) Received: from www.linux.org.uk (parcelfarce.linux.theplanet.co.uk [195.92.249.252]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBCLa1uR017737 for ; Thu, 12 Dec 2002 13:36:01 -0800 Received: from nat-pool-rdu.redhat.com ([66.187.233.200] helo=pobox.com) by www.linux.org.uk with esmtp (Exim 3.33 #5) id 18Mb3Z-000812-00; Thu, 12 Dec 2002 21:39:45 +0000 Message-ID: <3DF90208.3010208@pobox.com> Date: Thu, 12 Dec 2002 16:39:20 -0500 From: Jeff Garzik User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2) Gecko/20021202 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Donald Becker CC: Roger Luethi , netdev@oss.sgi.com, Linux Kernel Mailing List Subject: Re: pci-skeleton duplex check References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 1358 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jgarzik@pobox.com Precedence: bulk X-list: netdev Content-Length: 3779 Lines: 92 Donald Becker wrote: > On Thu, 12 Dec 2002, Jeff Garzik wrote: > >>Donald Becker wrote: >> >>>[[ I don't know why I bother. The people that now control what goes into >>>the kernel would rather put in random patches from other people than >>>accept a correct fix from me. ]] >> >>I'm very interested in applying fixes from you! I am publicly begging >>you to do so, and even CC'ing lkml on my request. > > > This is very disingenuous statement. Oh come on, it's far less disingenuous than what you said: [[ I don't know why I bother. The people that now control what goes into the kernel would rather put in random patches from other people than accept a correct fix from me. ]] I'm sure you'll continue making snide comments on every mailing list you maintain, but the fact remains: I would much rather accept a fix from you. That hasn't changed in the past year. or two. or any amount of time. Your input is very valuable, and I typically save quite a few of your emails. > The drivers in the kernel are now heavily modified and have significantly > diverged from my version. Sure, you are fine with having someone else > do the difficult and unrewarding debugging and maintainence work, while > you work on just the latest cool hardware, change the interfaces and are > concerned only with the current kernel version. While I disagree with this assessment, I think we can safely draw the conclusion that the problem is _not_ people ignoring your patches, or preferring other patches over yours. > I've been actively developing Linux drivers for over a decade, and run > about two dozen mailing lists for specific drivers. I write diagnostic > routines for every released driver. I thoroughly test and frequently > update the driver set I maintain. And since about 2000, my patches were > ignored while the first notice I've have gotten to changes in my drivers > is the bug reports. And the response: "submit a patch to fix those > newly introduced bugs". I've even had patches ignore in favor of people > that wrote "I don't have the NIC, but here is a change". I don't recall _ever_ getting a patch from you or seeing one posted on lkml or netdev. How can you be ignored if you're not sending patches? > A good example is the tulip driver. You repeatedly restructured my > driver in the kernel, splitting into different files. It was still 90+% > my code, but the changes made it impossible to track the modification > history. The kernel driver was long-broken with 21143-SYM cards, but no > one took the responsibility for fixing it. s/was/is/ I take responsibility for fixing it, I just haven't fixed it yet :) > It's easy to make the first few patches, when you don't have to deal > with reversion testing, many different models, and have an unlimited > sandbox where it doesn't matter if a specific release works or not. But > it takes a huge of work to keep a stable, tracable driver development > process that works with many different kernel versions and hardware > environments. We're slowly getting there, in terms of regression and stress testing. Since you don't send patches anymore for a long time, I was really the only one [stupid enough?] to stand up and even bother to help collecting and reviewing net driver changes. I would love to integrate your drivers directly, but they don't come anywhere close to using current kernel APIs. The biggie of biggies is not using the pci_driver API. So, given that we cannot directly merge your drivers, and you don't send patches to kernel developers, what is the next best alternative? (a) let kernel net drivers bitrot, or (b) maintain them as best we can without Don Becker patches? I say that "b" is far better than "a" for Linux users. Jeff From greearb@candelatech.com Thu Dec 12 15:05:15 2002 Received: with ECARTIS (v1.0.0; list netdev); Thu, 12 Dec 2002 15:05:23 -0800 (PST) Received: from grok.yi.org (IDENT:oqqFhnIvE1PsNkXfm2ajc4dMoTC0xc1m@dhcp101-dsl-usw4.w-link.net [208.161.125.101]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBCN5CuR018620 for ; Thu, 12 Dec 2002 15:05:14 -0800 Received: from candelatech.com (IDENT:IGMKQWvvUSV9E0hFjOuggClOHalYf98U@localhost.localdomain [127.0.0.1]) by grok.yi.org (8.11.6/8.11.2) with ESMTP id gBCN8Q001667; Thu, 12 Dec 2002 15:08:26 -0800 Message-ID: <3DF916EA.10504@candelatech.com> Date: Thu, 12 Dec 2002 15:08:26 -0800 From: Ben Greear Organization: Candela Technologies User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20021130 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Donald Becker CC: Jeff Garzik , Roger Luethi , netdev@oss.sgi.com, Linux Kernel Mailing List Subject: Re: pci-skeleton duplex check References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 1359 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: greearb@candelatech.com Precedence: bulk X-list: netdev Content-Length: 1889 Lines: 40 Donald Becker wrote: > I've been actively developing Linux drivers for over a decade, and run > about two dozen mailing lists for specific drivers. I write diagnostic > routines for every released driver. I thoroughly test and frequently > update the driver set I maintain. And since about 2000, my patches were > ignored while the first notice I've have gotten to changes in my drivers > is the bug reports. And the response: "submit a patch to fix those > newly introduced bugs". I've even had patches ignore in favor of people > that wrote "I don't have the NIC, but here is a change". > > A good example is the tulip driver. You repeatedly restructured my > driver in the kernel, splitting into different files. It was still 90+% > my code, but the changes made it impossible to track the modification > history. The kernel driver was long-broken with 21143-SYM cards, but no > one took the responsibility for fixing it. For what it's worth, I have yet to find a tulip driver that works with all of my 4-port NICs. Becker's fails with the Phobos 4-port NIC, a very recent kernel driver fails to negotiate correctly (sometimes) with the DFE-570tx NIC. Both of them failed a while back when I tried to put 3 DFE-570tx's into a single machine. On average, I've had better luck with the kernel driver than with Becker's, and since it is quite a pain to compile and test it, I have been ignoring it more and more. Mr Becker: Perhaps you could rename your tulip driver becker_tulip and have it separately buildable and configurable in the kernel config options? If it was back into the kernel proper it would be much easier to experiment with. Thanks, Ben -- Ben Greear President of Candela Technologies Inc http://www.candelatech.com ScryMUD: http://scry.wanfear.com http://scry.wanfear.com/~greear From hch@infradead.org Thu Dec 12 16:23:10 2002 Received: with ECARTIS (v1.0.0; list netdev); Thu, 12 Dec 2002 16:23:15 -0800 (PST) Received: from phoenix.infradead.org (carisma.slowglass.com [195.224.96.167]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBD0N9uR021022 for ; Thu, 12 Dec 2002 16:23:09 -0800 Received: from hch by phoenix.infradead.org with local (Exim 4.10) id 18MdfI-0004Ew-00; Fri, 13 Dec 2002 00:26:52 +0000 Date: Fri, 13 Dec 2002 00:26:51 +0000 From: Christoph Hellwig To: Jeff Garzik Cc: Linux Kernel Mailing List , netdev@oss.sgi.com Subject: Re: [PATCH 2.4.20] vanilla 3c59x Message-ID: <20021213002651.A15922@infradead.org> Mail-Followup-To: Christoph Hellwig , Jeff Garzik , Linux Kernel Mailing List , netdev@oss.sgi.com References: <3DF925AC.4020703@pobox.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <3DF925AC.4020703@pobox.com>; from jgarzik@pobox.com on Thu, Dec 12, 2002 at 07:11:24PM -0500 X-archive-position: 1360 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: hch@infradead.org Precedence: bulk X-list: netdev Content-Length: 243 Lines: 9 On Thu, Dec 12, 2002 at 07:11:24PM -0500, Jeff Garzik wrote: > > bk pull bk://kernel.bkbits.net/jgarzik/3c59x-2.4 > > This will update the following files: Marcelo, please don't pull that. Having the pci-scan* junk in mainline is bogus. From becker@scyld.com Thu Dec 12 17:14:55 2002 Received: with ECARTIS (v1.0.0; list netdev); Thu, 12 Dec 2002 17:15:03 -0800 (PST) Received: from beohost.scyld.com (dsl093-058-082.blt1.dsl.speakeasy.net [66.93.58.82]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBD1EtuR022152 for ; Thu, 12 Dec 2002 17:14:55 -0800 Received: from localhost (becker@localhost) by beohost.scyld.com (8.11.6/8.11.6) with ESMTP id gBD1IqQ21999; Thu, 12 Dec 2002 20:18:52 -0500 Date: Thu, 12 Dec 2002 20:18:51 -0500 (EST) From: Donald Becker To: Jeff Garzik cc: Roger Luethi , , Linux Kernel Mailing List Subject: Re: pci-skeleton duplex check In-Reply-To: <3DF90208.3010208@pobox.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 1361 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: becker@scyld.com Precedence: bulk X-list: netdev Content-Length: 4803 Lines: 104 On Thu, 12 Dec 2002, Jeff Garzik wrote: > Donald Becker wrote: > > On Thu, 12 Dec 2002, Jeff Garzik wrote: > >>Donald Becker wrote: > [[ I don't know why I bother. The people that now control what > goes into the kernel would rather put in random patches from > other people than accept a correct fix from me. ]] > I'm sure you'll continue making snide comments on every mailing list you > maintain, but the fact remains: > I would much rather accept a fix from you. Perhaps we have a different idea of "fix". You want are looking for a patch to modifications you have made to code I have written. In the meantime I have been providing working code, and have been updating that code to work with new hardware. So a fix is the working, continuously maintained version of the driver. To put an admittedly simplified spin on it, you are saying "I want you to tell me what I have broken when I changed this", and to continuously monitor and test your changes, made for unknown reasons on a very divergent source base. > > The drivers in the kernel are now heavily modified and have significantly > > diverged from my version. Sure, you are fine with having someone else > > do the difficult and unrewarding debugging and maintenance work, while > > you work on just the latest cool hardware, change the interfaces and are > > concerned only with the current kernel version. > > While I disagree with this assessment, I think we can safely draw the > conclusion that the problem is _not_ people ignoring your patches, or > preferring other patches over yours. I can point to specific instances. Just looking at the drivers in the kernel, it is clear that this has happened. > > A good example is the tulip driver. You repeatedly restructured my ... > I take responsibility for fixing it, I just haven't fixed it yet :) > > It's easy to make the first few patches, when you don't have to deal > > with reversion testing, many different models, and have an unlimited > > sandbox where it doesn't matter if a specific release works or not. I think that these two statements fit well together. > > But > > it takes a huge of work to keep a stable, traceable driver development > > process that works with many different kernel versions and hardware > > environments. > > We're slowly getting there, in terms of regression and stress testing. But it existed before, and was discarded! Yes, the kernel is now _returning_ to a stable state while making improvements. But there was a period of time when interface stability and detailed correctness was thrown away in favor of many inexperienced people quickly and repeatedly restructuring interfaces without understanding the underlying requirements. I could mention VM, but I'll go back to one that had a very large direct impacted on me: CardBus. CardBus is a difficult case of hot-swap PCI -- the device can go away without warning, and it's usually implemented on machines where suspend and resume must work correctly. We had perhaps the best operational implementation in the industry, and I had written about half of the CardBus drivers. Yet my proven interface implementation was completely discarded in favor one that needed to be repeatedly changed as the requirements were slowly understood. > I would love to integrate your drivers directly, but they don't come > anywhere close to using current kernel APIs. The biggie of biggies is > not using the pci_driver API. So, given that we cannot directly merge Yup, that is just what I was talking about. Let me continue: The result is that today other systems now have progressed to a great implementation of suspend/resume, while Linux distributions now default to unloading and reloading drivers to avoid various suspend bugs. And when the driver cannot be unloaded because some part of the networking stack is holding the interface, things go horribly wrong... You might be able to naysay the individual details, but the end technical result is clear. > your drivers, and you don't send patches to kernel developers, what is > the next best alternative? (a) let kernel net drivers bitrot, or (b) > maintain them as best we can without Don Becker patches? I say that "b" > is far better than "a" for Linux users. Or perhaps recognizing that when someone that has been a significant, continuous contributer since the early days of Linux says "this is screwed up", they might have a point. When Linux suddenly had thousands of people wanting to submit patches, that didn't means that there were more people that could understand, implement and maintain complex systems. -- Donald Becker becker@scyld.com Scyld Computing Corporation http://www.scyld.com 410 Severn Ave. Suite 210 Scyld Beowulf cluster system Annapolis MD 21403 410-990-9993 From jgarzik@pobox.com Fri Dec 13 00:09:08 2002 Received: with ECARTIS (v1.0.0; list netdev); Fri, 13 Dec 2002 00:09:16 -0800 (PST) Received: from www.linux.org.uk (parcelfarce.linux.theplanet.co.uk [195.92.249.252]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBD897uR025051 for ; Fri, 13 Dec 2002 00:09:08 -0800 Received: from rdu57-8-131.nc.rr.com ([66.57.8.131] helo=pobox.com) by www.linux.org.uk with esmtp (Exim 3.33 #5) id 18MdRA-00025E-00; Fri, 13 Dec 2002 00:12:16 +0000 Message-ID: <3DF925AC.4020703@pobox.com> Date: Thu, 12 Dec 2002 19:11:24 -0500 From: Jeff Garzik User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2) Gecko/20021202 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Linux Kernel Mailing List , netdev@oss.sgi.com Subject: [PATCH 2.4.20] vanilla 3c59x Content-Type: multipart/mixed; boundary="------------020908010405090006090002" X-archive-position: 1362 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jgarzik@pobox.com Precedence: bulk X-list: netdev Content-Length: 1406 Lines: 42 This is a multi-part message in MIME format. --------------020908010405090006090002 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit This patch, against 2.4.20, updates 3c59x to vanilla Don Becker version that is available at ftp://www.scyld.com/pub/network/3c59x.c It's too big post on a list, so I put it up at ftp://ftp.kernel.org/pub/linux/kernel/people/jgarzik/patches/2.4.20/3c59x-vanilla-2.4.20.patch.gz It's also available from BitKeeper at the 'bk pull' URL attached to this email. But be warned you are pulling from 2.4.21-bk-current, plus 3c59x.c changes, not vanilla 2.4.20 as the patch is diff'd against. --------------020908010405090006090002 Content-Type: text/plain; name="3c59x-2.4.txt" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="3c59x-2.4.txt" bk pull bk://kernel.bkbits.net/jgarzik/3c59x-2.4 This will update the following files: drivers/net/3c59x.c | 3254 +++++++++++++++++++--------------------------- drivers/net/Makefile | 2 drivers/net/kern_compat.h | 284 ++++ drivers/net/pci-scan.c | 658 +++++++++ drivers/net/pci-scan.h | 90 + 5 files changed, 2440 insertions(+), 1848 deletions(-) through these ChangeSets: (02/12/12 1.812) [netdrvr 3c59x] update to vanilla Becker version 0.99Xf (11/17/2002) --------------020908010405090006090002-- From davem@redhat.com Fri Dec 13 00:47:03 2002 Received: with ECARTIS (v1.0.0; list netdev); Fri, 13 Dec 2002 00:47:07 -0800 (PST) Received: from rth.ninka.net (rth.ninka.net [216.101.162.244]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBD8l2uR025741 for ; Fri, 13 Dec 2002 00:47:03 -0800 Received: from rth.ninka.net (localhost.localdomain [127.0.0.1]) by rth.ninka.net (8.12.5/8.12.5) with ESMTP id gBD9HH1O001913; Fri, 13 Dec 2002 01:17:17 -0800 Received: (from davem@localhost) by rth.ninka.net (8.12.5/8.12.5/Submit) id gBD9HGaF001909; Fri, 13 Dec 2002 01:17:16 -0800 X-Authentication-Warning: rth.ninka.net: davem set sender to davem@redhat.com using -f Subject: Re: pci-skeleton duplex check From: "David S. Miller" To: Donald Becker Cc: Jeff Garzik , Roger Luethi , netdev@oss.sgi.com, Linux Kernel Mailing List In-Reply-To: References: Content-Type: text/plain Content-Transfer-Encoding: 7bit X-Mailer: Ximian Evolution 1.0.8 (1.0.8-10) Date: 13 Dec 2002 01:17:16 -0800 Message-Id: <1039771036.1509.5.camel@rth.ninka.net> Mime-Version: 1.0 X-archive-position: 1363 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: davem@redhat.com Precedence: bulk X-list: netdev Content-Length: 880 Lines: 20 On Thu, 2002-12-12 at 17:18, Donald Becker wrote: > Or perhaps recognizing that when someone that has been a significant, > continuous contributer since the early days of Linux Until you learn to play nice with people and mesh within the fabric of Linux development, I adamently do not classify you as you appear to self-classify yourself. You don't contribute, you sit in your sandbox and then point fingers at the people who do know how to work with other human beings and say "see how much that stuff sucks? well my stuff works, nyah!" I fear you will hold a grudge about this forever. If Linux itself is worse off and went backwards in time for a while, it is because of your inability to work together with people. I know it may be hard for you to accept this fact, but I can tell you that continuing to point the fingers elsewhere is going to be a repeated dead end. From rl@hellgate.ch Fri Dec 13 02:30:41 2002 Received: with ECARTIS (v1.0.0; list netdev); Fri, 13 Dec 2002 02:30:46 -0800 (PST) Received: from mta2n.bluewin.ch (mta2n.bluewin.ch [195.186.4.220]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBDAUduR026841 for ; Fri, 13 Dec 2002 02:30:40 -0800 Received: from k3.hellgate.ch (62.203.102.214) by mta2n.bluewin.ch (Bluewin AG 6.5.032) id 3DE760510014ED9F; Thu, 12 Dec 2002 14:20:42 +0100 Received: by k3.hellgate.ch (MTA, from userid 500) id 315A9323C0; Thu, 12 Dec 2002 14:20:33 +0100 (CET) Date: Thu, 12 Dec 2002 14:20:33 +0100 From: Roger Luethi To: Donald Becker Cc: netdev@oss.sgi.com Subject: Re: pci-skeleton duplex check Message-ID: <20021212132033.GA3224@k3.hellgate.ch> References: <20021211132436.GA12529@k3.hellgate.ch> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.3.27i X-Operating-System: Linux 2.4.21-pre1 on i686 X-GPG-Fingerprint: 92 F4 DC 20 57 46 7B 95 24 4E 9E E7 5A 54 DC 1B X-GPG: 1024/80E744BD wwwkeys.ch.pgp.net X-archive-position: 1364 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: rl@hellgate.ch Precedence: bulk X-list: netdev Content-Length: 734 Lines: 19 On Wed, 11 Dec 2002 21:42:44 -0500, Donald Becker wrote: > Note that the second check ignores 100baseT4, despite it have priority > over 10baseT-*. That was intentional to work around, "a specific issue" > with a transceiver. That "specific issue" might be worth documenting. Information on such quirks is hardest to come by. > The is no extra cost to the extra bit, and it makes it clear we are > testing for 10baseT-FDX. > (The test was originally implemented as part of a complete set of > cases. The test code needed while building a driver is more complex > than what you see in the concise final result.) That was exactly the kind of answer I've been looking for. Nothing beats some historic background. Thank you. Roger From becker@scyld.com Fri Dec 13 08:52:31 2002 Received: with ECARTIS (v1.0.0; list netdev); Fri, 13 Dec 2002 08:52:35 -0800 (PST) Received: from beohost.scyld.com (pool-151-196-176-15.balt.east.verizon.net [151.196.176.15]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBDGqTuR003376 for ; Fri, 13 Dec 2002 08:52:30 -0800 Received: from localhost (becker@localhost) by beohost.scyld.com (8.11.6/8.11.6) with ESMTP id gBDGuHo02102; Fri, 13 Dec 2002 11:56:18 -0500 Date: Fri, 13 Dec 2002 11:56:17 -0500 (EST) From: Donald Becker To: "David S. Miller" cc: Jeff Garzik , Roger Luethi , , Linux Kernel Mailing List Subject: Re: pci-skeleton duplex check In-Reply-To: <1039771036.1509.5.camel@rth.ninka.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 1366 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: becker@scyld.com Precedence: bulk X-list: netdev Content-Length: 1356 Lines: 32 On 13 Dec 2002, David S. Miller wrote: > On Thu, 2002-12-12 at 17:18, Donald Becker wrote: > > Or perhaps recognizing that when someone that has been a significant, > > continuous contributer since the early days of Linux > > Until you learn to play nice with people and mesh within the > fabric of Linux development, I adamently do not classify you > as you appear to self-classify yourself. You don't contribute, > you sit in your sandbox and then point fingers at the people who > do know how to work with other human beings and say "see how much > that stuff sucks? well my stuff works, nyah!" .. > If Linux itself is worse off and went backwards in time for a while... The development criteria used to be technically based, and that is still the public statement. Now, as your statement makes clear, working code is an irrelevant criteria. You comments immediately moved the subject from the technical merit and correctness of the code to an ad hominem attack. The facts, and the code, clearly show the long term interaction and contribution. In most cases the code and interfaces we are talking about were written and defined by me throughout the past decade. -- Donald Becker becker@scyld.com Scyld Computing Corporation http://www.scyld.com 410 Severn Ave. Suite 210 Scyld Beowulf cluster system Annapolis MD 21403 410-990-9993 From lm@bitmover.com Fri Dec 13 09:18:43 2002 Received: with ECARTIS (v1.0.0; list netdev); Fri, 13 Dec 2002 09:18:45 -0800 (PST) Received: from mail.bitmover.com (bitmover.com [192.132.92.2]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBDHIhuR003966 for ; Fri, 13 Dec 2002 09:18:43 -0800 Received: from work.bitmover.com (work.bitmover.com [10.3.3.1]) by mail.bitmover.com (8.11.6/8.11.6) with ESMTP id gBDHeNp00373; Fri, 13 Dec 2002 09:40:23 -0800 Received: (from lm@localhost) by work.bitmover.com (8.11.6/8.11.6) id gBDHMTn17407; Fri, 13 Dec 2002 09:22:29 -0800 Date: Fri, 13 Dec 2002 09:22:29 -0800 From: Larry McVoy To: Donald Becker Cc: "David S. Miller" , Jeff Garzik , Roger Luethi , netdev@oss.sgi.com, Linux Kernel Mailing List Subject: Networking/Becker et al [was Re: pci-skeleton duplex check] Message-ID: <20021213092229.D9973@work.bitmover.com> Mail-Followup-To: Larry McVoy , Donald Becker , "David S. Miller" , Jeff Garzik , Roger Luethi , netdev@oss.sgi.com, Linux Kernel Mailing List Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i X-MailScanner: Found to be clean X-archive-position: 1367 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: lm@bitmover.com Precedence: bulk X-list: netdev Content-Length: 5111 Lines: 108 In the for what it is worth department... My brain dump on this situation. I know Dave Miller, Don Becker, and Jeff Garzik personally, we've spent time face to face. These are all good guys, very important guys to the kernel effort. I've known Don for a long time and we've had a bunch of talks about why his style and the Linux style doesn't mesh well. Here's my view, which may or may not be shared. Don is a careful, thoughtful guy. He is quite conservative when it comes to programming. His style most closely matches the Sun kernel style of development; it does not match the Linux style at all. The Linux style is a lot more free wheeling, stuff changes a lot and the kernel team depends heavily on the fact that it is has this vast army of free testers. Without that army, I shudder to think what things would be like, I do not think the current development style would work anywhere near as well. But it does work, and it violates a lot of engineering disciplines that old farts, like me and Don, respect. I've learned to live with it, even respect the fact that the Linux team does so well. Don is having a much tougher time. He really wants the Linux team to work more like he works and the Linux team doesn't want to do that at all, they point at their success and believe that their development style is part of that success. It is worth putting on the record that Don has done a lot for the Linux effort, a huge amount, in fact. Without Don, Linux would be dramatically less far along than it is. I've been here since before it had networking and it really took off when Don started writing drivers. It's also pointing out that I think he's right about the networking regressions, suspend/resume on laptops used to work and now the network is almost always hosed after I do that. I doubt that either side is likely to change their view. But, the real point is how to we get Don's brain engaged on the kernel networking drivers? A few thoughts: a) Don is going to have to accept that the Linux kernel approach is the way it is. Sitting on the sidelines and whining is not going to change how the kernel is developed. Either get with the program or not, but don't sit there and complain but refuse to work the way the rest of the people work. b) The kernel folks need to listen to Don more. Draw him into the conversations about interface changes, try and extract the knowledge he has, it's worth it. Not doing so just means you are wasting time. c) Don needs to kill those mailing lists he maintains or merge them with the appropriate kernel lists. That is a big part of the problem, the interesting stuff seems to be happening over in Don's part of the world and the mainstream kernel team isn't aware of it. d) Beer. More beer. Much more beer and some face time. If there is a tech conference coming up, I think we should get Don to come and give him the first lifetime achievement award for Linux kernel development. Then shove him and the other network hackers into a room with a keg and not let them out until they are smiling. BitMover will kick in some money towards this if needed. --lm On Fri, Dec 13, 2002 at 11:56:17AM -0500, Donald Becker wrote: > On 13 Dec 2002, David S. Miller wrote: > > On Thu, 2002-12-12 at 17:18, Donald Becker wrote: > > > Or perhaps recognizing that when someone that has been a significant, > > > continuous contributer since the early days of Linux > > > > Until you learn to play nice with people and mesh within the > > fabric of Linux development, I adamently do not classify you > > as you appear to self-classify yourself. You don't contribute, > > you sit in your sandbox and then point fingers at the people who > > do know how to work with other human beings and say "see how much > > that stuff sucks? well my stuff works, nyah!" > .. > > If Linux itself is worse off and went backwards in time for a while... > > The development criteria used to be technically based, and that is still > the public statement. Now, as your statement makes clear, working code > is an irrelevant criteria. > > You comments immediately moved the subject from the technical merit and > correctness of the code to an ad hominem attack. The facts, and the > code, clearly show the long term interaction and contribution. In most > cases the code and interfaces we are talking about were written and > defined by me throughout the past decade. > > > > -- > Donald Becker becker@scyld.com > Scyld Computing Corporation http://www.scyld.com > 410 Severn Ave. Suite 210 Scyld Beowulf cluster system > Annapolis MD 21403 410-990-9993 > > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ -- --- Larry McVoy lm at bitmover.com http://www.bitmover.com/lm From davem@redhat.com Fri Dec 13 10:29:47 2002 Received: with ECARTIS (v1.0.0; list netdev); Fri, 13 Dec 2002 10:29:48 -0800 (PST) Received: from pizda.ninka.net (IDENT:root@pizda.ninka.net [216.101.162.242]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBDITkuR005335 for ; Fri, 13 Dec 2002 10:29:47 -0800 Received: from localhost (IDENT:davem@localhost.localdomain [127.0.0.1]) by pizda.ninka.net (8.9.3/8.9.3) with ESMTP id KAA11375; Fri, 13 Dec 2002 10:29:19 -0800 Date: Fri, 13 Dec 2002 10:29:18 -0800 (PST) Message-Id: <20021213.102918.102902274.davem@redhat.com> To: becker@scyld.com Cc: jgarzik@pobox.com, rl@hellgate.ch, netdev@oss.sgi.com, linux-kernel@vger.kernel.org Subject: Re: pci-skeleton duplex check From: "David S. Miller" In-Reply-To: References: <1039771036.1509.5.camel@rth.ninka.net> X-FalunGong: Information control. X-Mailer: Mew version 2.1 on Emacs 21.1 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-archive-position: 1368 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: davem@redhat.com Precedence: bulk X-list: netdev Content-Length: 752 Lines: 19 From: Donald Becker Date: Fri, 13 Dec 2002 11:56:17 -0500 (EST) The development criteria used to be technically based, and that is still the public statement. Now, as your statement makes clear, working code is an irrelevant criteria. No, working code is only part of the equation. If you're a total and complete asshole, your work is likely to get lost to the sands of time. In such a case nobody wants to deal with you. Welcome to the real world where you have to interact with other human beings (not just be technically capable) in order to accomplish things. It's always been like this Donald. If you piss off, or are a jerk to, the primary maintainers you're going to get the short end of the stick. From jgarzik@pobox.com Fri Dec 13 13:16:24 2002 Received: with ECARTIS (v1.0.0; list netdev); Fri, 13 Dec 2002 13:16:29 -0800 (PST) Received: from www.linux.org.uk (parcelfarce.linux.theplanet.co.uk [195.92.249.252]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBDLGN3v009756 for ; Fri, 13 Dec 2002 13:16:24 -0800 Received: from nat-pool-rdu.redhat.com ([66.187.233.200] helo=pobox.com) by www.linux.org.uk with esmtp (Exim 3.33 #5) id 18MxE9-0003aZ-00; Fri, 13 Dec 2002 21:20:09 +0000 Message-ID: <3DFA4ED1.7090209@pobox.com> Date: Fri, 13 Dec 2002 16:19:13 -0500 From: Jeff Garzik User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2) Gecko/20021202 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Larry McVoy CC: Donald Becker , "David S. Miller" , Roger Luethi , netdev@oss.sgi.com, Linux Kernel Mailing List Subject: Re: Networking/Becker et al [was Re: pci-skeleton duplex check] References: <20021213092229.D9973@work.bitmover.com> In-Reply-To: <20021213092229.D9973@work.bitmover.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 1369 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jgarzik@pobox.com Precedence: bulk X-list: netdev Content-Length: 4276 Lines: 92 Larry McVoy wrote: > Don is a careful, thoughtful guy. He is quite conservative when it comes > to programming. His style most closely matches the Sun kernel style of > development; it does not match the Linux style at all. The Linux style > is a lot more free wheeling, stuff changes a lot and the kernel team > depends heavily on the fact that it is has this vast army of free testers. > Without that army, I shudder to think what things would be like, I do not > think the current development style would work anywhere near as well. > > But it does work, and it violates a lot of engineering disciplines that > old farts, like me and Don, respect. I've learned to live with it, even > respect the fact that the Linux team does so well. Don is having a much > tougher time. He really wants the Linux team to work more like he works > and the Linux team doesn't want to do that at all, they point at their > success and believe that their development style is part of that success. This meshes with what I've observed, too... I'm not sure which is the bigger issue, Don's devel style versus Linux devel style, or use of kernel APIs, but both I think cut to the core of the differences in this situation. > It is worth putting on the record that Don has done a lot for the Linux > effort, a huge amount, in fact. Without Don, Linux would be dramatically > less far along than it is. I've been here since before it had networking > and it really took off when Don started writing drivers. I give him a lot of credit too, though it's often in the way of trying to apply lessons learned from him to current net drivers and such. > It's also pointing out that I think he's right about the networking > regressions, suspend/resume on laptops used to work and now the network > is almost always hosed after I do that. suspend/resume in Linux has always been a hack, and will continue to be until the 2.5.x sysfs/device model is fully fleshed out. Specifically for 2.4.x, let me know if your net driver doesn't suspend/resume correctly. The cases I've tested work fine. Make sure your distro is properly calling the /sbin/hotplug agent when suspend/resume occurs though. > I doubt that either side is likely to change their view. But, the real > point is how to we get Don's brain engaged on the kernel networking > drivers? A few thoughts: > > a) Don is going to have to accept that the Linux kernel approach is > the way it is. Sitting on the sidelines and whining is not going > to change how the kernel is developed. Either get with the > program or not, but don't sit there and complain but refuse > to work the way the rest of the people work. indeed... central problem > b) The kernel folks need to listen to Don more. Draw him into the > conversations about interface changes, try and extract the > knowledge he has, it's worth it. Not doing so just means you > are wasting time. I try to do this, and I can point to many specific instances in the past when he's been very helpful. I would love to get Don more involved in interface discussions though... typically where he pops up [where I see him] is more in the area of hardware experience and knowledge than interface discussions. > c) Don needs to kill those mailing lists he maintains or merge them > with the appropriate kernel lists. That is a big part of the > problem, the interesting stuff seems to be happening over in > Don's part of the world and the mainstream kernel team isn't > aware of it. This is really a matter of getting his driver changes into the kernel, too... Some mailing list users [not me] would probably complain about seeing support traffic for drivers that are not in the kernel. > d) Beer. More beer. Much more beer and some face time. If there > is a tech conference coming up, I think we should get Don to > come and give him the first lifetime achievement award for > Linux kernel development. Then shove him and the other network > hackers into a room with a keg and not let them out until they > are smiling. BitMover will kick in some money towards this if > needed. mmmmmmm, beer :) From mcr@sandelman.ottawa.on.ca Fri Dec 13 16:16:20 2002 Received: with ECARTIS (v1.0.0; list netdev); Fri, 13 Dec 2002 16:16:25 -0800 (PST) Received: from noxmail.sandelman.ottawa.on.ca (cyphermail.sandelman.ottawa.on.ca [192.139.46.78]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBE0GF3v011688 for ; Fri, 13 Dec 2002 16:16:20 -0800 Received: from sandelman.ottawa.on.ca (marajade.sandelman.ottawa.on.ca [192.139.46.20]) by noxmail.sandelman.ottawa.on.ca (8.11.6/8.11.6) with ESMTP id gBE0Jth09668 (using TLSv1/SSLv3 with cipher EDH-RSA-DES-CBC3-SHA (168 bits) verified OK); Fri, 13 Dec 2002 19:20:02 -0500 (EST) Received: from sandelman.ottawa.on.ca (marajade [127.0.0.1]) by sandelman.ottawa.on.ca (8.12.3/8.12.3/Debian -4) with ESMTP id gBE0JncX006748 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=OK); Fri, 13 Dec 2002 19:19:50 -0500 Received: from marajade.sandelman.ottawa.on.ca (mcr@localhost) by sandelman.ottawa.on.ca (8.12.3/8.12.3/Debian -4) with ESMTP id gBE0JmDs006744; Fri, 13 Dec 2002 19:19:48 -0500 Message-Id: <200212140019.gBE0JmDs006744@sandelman.ottawa.on.ca> To: netdev@oss.sgi.com, Linux Kernel Mailing List Subject: Re: pci-skeleton duplex check In-reply-to: Your message of "Thu, 12 Dec 2002 16:11:13 EST." Mime-Version: 1.0 (generated by tm-edit 1.8) Content-Type: text/plain; charset=US-ASCII Date: Fri, 13 Dec 2002 19:19:47 -0500 From: Michael Richardson X-archive-position: 1370 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: mcr@sandelman.ottawa.on.ca Precedence: bulk X-list: netdev Content-Length: 1565 Lines: 42 -----BEGIN PGP SIGNED MESSAGE----- >>>>> "Donald" == Donald Becker writes: Donald> The drivers in the kernel are now heavily modified and have significantly Donald> diverged from my version. Sure, you are fine with having someone else Donald> do the difficult and unrewarding debugging and maintainence work, while Donald> you work on just the latest cool hardware, change the interfaces and are Donald> concerned only with the current kernel version. I agree strongly with Donald. Interfaces should NEVER change in patch level versions. Just *DO NOT DO IT*. Go wild in odd-numbered.. get the interfaces right there. But leave them alone afterward. This is a fundamental tenant of being professional. Otherwise, the kernel people are the biggest reason I've ever seen for using *BSD. Microsoft is not the real enemy. Gratuitous change is. ] ON HUMILITY: to err is human. To moo, bovine. | firewalls [ ] Michael Richardson, Sandelman Software Works, Ottawa, ON |net architect[ ] mcr@sandelman.ottawa.on.ca http://www.sandelman.ottawa.on.ca/ |device driver[ ] panic("Just another Debian GNU/Linux using, kernel hacking, security guy"); [ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.7 (GNU/Linux) Comment: Finger me for keys iQCVAwUBPfp5IIqHRg3pndX9AQHW7gP9FC0BgskaVBb9HNjKUp8DhR5bJK+YTVa7 YeVGZFRxuFi2O9oDiMuUvYq++y+8PR4LXpJZuNoShA36wqV38QS8pBFhqFt/JrEb xHNozohQ/7IyncJsG0UkBTfhqIbxbfsd19DUx0ehcqNAh7N3c95qeEEHODTs2DKy jqtgSrXvOBY= =JsKT -----END PGP SIGNATURE----- From jgarzik@pobox.com Fri Dec 13 17:14:16 2002 Received: with ECARTIS (v1.0.0; list netdev); Fri, 13 Dec 2002 17:14:20 -0800 (PST) Received: from www.linux.org.uk (parcelfarce.linux.theplanet.co.uk [195.92.249.252]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBE1EE3v012902 for ; Fri, 13 Dec 2002 17:14:15 -0800 Received: from nat-pool-rdu.redhat.com ([66.187.233.200] helo=pobox.com) by www.linux.org.uk with esmtp (Exim 3.33 #5) id 18MwtV-0003My-00; Fri, 13 Dec 2002 20:58:49 +0000 Message-ID: <3DFA49CF.9060106@pobox.com> Date: Fri, 13 Dec 2002 15:57:51 -0500 From: Jeff Garzik User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2) Gecko/20021202 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Donald Becker CC: Roger Luethi , netdev@oss.sgi.com, Linux Kernel Mailing List Subject: Re: pci-skeleton duplex check References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 1371 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jgarzik@pobox.com Precedence: bulk X-list: netdev Donald Becker wrote: > On Thu, 12 Dec 2002, Jeff Garzik wrote: > >>Donald Becker wrote: >> >>>On Thu, 12 Dec 2002, Jeff Garzik wrote: >>> >>>>Donald Becker wrote: >> >> [[ I don't know why I bother. The people that now control what >> goes into the kernel would rather put in random patches from >> other people than accept a correct fix from me. ]] >>I'm sure you'll continue making snide comments on every mailing list you >>maintain, but the fact remains: >>I would much rather accept a fix from you. > > > Perhaps we have a different idea of "fix". > > You want are looking for a patch to modifications you have made to code > I have written. In the meantime I have been providing working code, and > have been updating that code to work with new hardware. So a fix is the > working, continuously maintained version of the driver. > > To put an admittedly simplified spin on it, you are saying "I want you > to tell me what I have broken when I changed this", and to continuously > monitor and test your changes, made for unknown reasons on a very > divergent source base. No, that's not it at all. I would ecstatic if you even posted the changes made to your own drivers to netdev@oss.sgi.com or similar... I'm asking for _any_ contributions at all. The more fine-grained the better... >>>The drivers in the kernel are now heavily modified and have significantly >>>diverged from my version. Sure, you are fine with having someone else >>>do the difficult and unrewarding debugging and maintenance work, while >>>you work on just the latest cool hardware, change the interfaces and are >>>concerned only with the current kernel version. >> >>While I disagree with this assessment, I think we can safely draw the >>conclusion that the problem is _not_ people ignoring your patches, or >>preferring other patches over yours. > > > I can point to specific instances. Just looking at the drivers in the > kernel, it is clear that this has happened. There is an admitted preference to people who actually send me patches. That sometimes translates to "other change" being preferred over logic in one of your drivers. I would still greatly prefer patches from you, however. And your comments on other people's patches are very welcome [and there have been plenty of those in past -- thanks]. > But it existed before, and was discarded! > Yes, the kernel is now _returning_ to a stable state while making > improvements. But there was a period of time when interface stability > and detailed correctness was thrown away in favor of many inexperienced > people quickly and repeatedly restructuring interfaces without > understanding the underlying requirements. > > I could mention VM, but I'll go back to one that had a very large direct > impacted on me: CardBus. CardBus is a difficult case of hot-swap PCI -- > the device can go away without warning, and it's usually implemented on > machines where suspend and resume must work correctly. We had perhaps > the best operational implementation in the industry, and I had written > about half of the CardBus drivers. Yet my proven interface > implementation was completely discarded in favor one that needed to be > repeatedly changed as the requirements were slowly understood. But... this is how Linux development works. Believe me, I understand you don't like that very much, but here is a central question to you: what can we do to move forward? The CardBus implementation still fails on some systems, and still wants work. However, the pci_driver API is not only codified in 2.4.x, but it is extended to the more generic driver model in 2.5.x. _And_ I have proven it works just fine under 2.2.x (see kcompat24 toolkit). What can we as kernel developers do to reintegrate you back into kernel development? Some of the APIs you obviously don't like, but pretending they don't exist is not a solution. This is the Linux game, for better or worse. At the end of the day, if we don't like Linus's decisions, we can either swallow our pride and continue with Linux, or fork a Linux tree and make it work "the right way." The driver model (nee pci_driver) is the direction of Linux. >>I would love to integrate your drivers directly, but they don't come >>anywhere close to using current kernel APIs. The biggie of biggies is >>not using the pci_driver API. So, given that we cannot directly merge > > > Yup, that is just what I was talking about. Let me continue: > > The result is that today other systems now have progressed to a great > implementation of suspend/resume, while Linux distributions now default > to unloading and reloading drivers to avoid various suspend bugs. And > when the driver cannot be unloaded because some part of the networking > stack is holding the interface, things go horribly wrong... > > You might be able to naysay the individual details, but the end > technical result is clear. That's what is currently in development in 2.5.x: sane suspend and resume. I would dispute that other systems have a decently designed suspend/resume -- that said, working is obviously better right now than non-working but nicer design ;-) >>your drivers, and you don't send patches to kernel developers, what is >>the next best alternative? (a) let kernel net drivers bitrot, or (b) >>maintain them as best we can without Don Becker patches? I say that "b" >>is far better than "a" for Linux users. > > > Or perhaps recognizing that when someone that has been a significant, > continuous contributer since the early days of Linux says "this is > screwed up", they might have a point. When Linux suddenly had thousands > of people wanting to submit patches, that didn't means that there were > more people that could understand, implement and maintain complex > systems. Shit, dude, _I_ recognize this. Probably better than most people, since I see on a daily basis the benefits of your overall design in the net drivers, and want to push good elements of that design into the kernel net drivers. At the end of the day you'd be surprised how much I wind up defending your code to other kernel hackers, and educating them on why -not- to do certain things. IMO the bigger sticking point is - at what point do you say "yeah, 2.4.x/2.5.x APIs may suck in my opinion, but they are the official APIs so I will use them"? There are tons of reasons why Red Hat (my current employer) is very leery of taking patches which will not eventually find their way to the mainline kernel.org kernel. A lot of those reasons apply in the case of your drivers, too. Using non-standard APIs has all sorts of software engineering implications which wind up with a negative developer and user experience. Jeff From oxymoron@waste.org Fri Dec 13 17:40:03 2002 Received: with ECARTIS (v1.0.0; list netdev); Fri, 13 Dec 2002 17:40:05 -0800 (PST) Received: from waste.org (waste.org [209.173.204.2]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBE1e23v013325 for ; Fri, 13 Dec 2002 17:40:03 -0800 Received: from waste.org (localhost [127.0.0.1]) by waste.org (8.12.3/8.12.3/Debian -4) with ESMTP id gBE1hklZ018026 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO); Fri, 13 Dec 2002 19:43:46 -0600 Received: (from oxymoron@localhost) by waste.org (8.12.3/8.12.3/Debian -4) id gBE1hkQ1018023; Fri, 13 Dec 2002 19:43:46 -0600 Date: Fri, 13 Dec 2002 19:43:46 -0600 From: Oliver Xymoron To: Michael Richardson Cc: netdev@oss.sgi.com, Linux Kernel Mailing List Subject: Re: pci-skeleton duplex check Message-ID: <20021214014346.GA6029@waste.org> References: <200212140019.gBE0JmDs006744@sandelman.ottawa.on.ca> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200212140019.gBE0JmDs006744@sandelman.ottawa.on.ca> User-Agent: Mutt/1.3.28i X-Virus-Scanned: by AMaViS-ng (Milter interface) X-archive-position: 1372 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: oxymoron@waste.org Precedence: bulk X-list: netdev On Fri, Dec 13, 2002 at 07:19:47PM -0500, Michael Richardson wrote: > > > >>>>> "Donald" == Donald Becker writes: > Donald> The drivers in the kernel are now heavily modified and have significantly > Donald> diverged from my version. Sure, you are fine with having someone else > Donald> do the difficult and unrewarding debugging and maintainence work, while > Donald> you work on just the latest cool hardware, change the interfaces and are > Donald> concerned only with the current kernel version. > > I agree strongly with Donald. > > Interfaces should NEVER change in patch level versions. > Just *DO NOT DO IT*. > > Go wild in odd-numbered.. get the interfaces right there. > But leave them alone afterward. Umm, if I recall correctly, they're rehashing a flame war about stuff that occurred in 2.3. It doesn't need any additional kindling. -- "Love the dolphins," she advised him. "Write by W.A.S.T.E.." From arun4linux@indiatimes.com Sat Dec 14 06:33:48 2002 Received: with ECARTIS (v1.0.0; list netdev); Sat, 14 Dec 2002 06:33:49 -0800 (PST) Received: from WS0005.indiatimes.com ([203.199.93.15]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBEEXk3v030958 for ; Sat, 14 Dec 2002 06:33:47 -0800 Received: from 192.168.57.15 (a1 [192.168.57.21]) by WS0005.indiatimes.com (8.9.3/8.9.3) with SMTP id TAA32351; Sat, 14 Dec 2002 19:58:31 +0530 From: "arun4linux" Message-Id: <200212141428.TAA32351@WS0005.indiatimes.com> To: "Michael Richardson", , "Linux Kernel Mailing List" Reply-To: "arun4linux" Subject: Re: Re: pci-skeleton duplex check Date: Sat, 14 Dec 2002 20:05:30 +0530 X-URL: http://indiatimes.com Content-Type: text/plain; charset=us-ascii X-archive-position: 1374 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: arun4linux@indiatimes.com Precedence: bulk X-list: netdev <<Interfaces should NEVER change in patch level versions. Just *DO NOT DO IT*. >>I do agree on this. This is a common complaint about linux kernel developers. And this always gives an insecure feeling :-) for the device driver or kernel module programmers. This was one of the issues in my earlier company/work and they have gone for another OS. Warm Regards Arun "Michael Richardson" wrote: -----BEGIN PGP SIGNED MESSAGE----- >>>>> "Donald" == Donald Becker writes: Donald> The drivers in the kernel are now heavily modified and have significantly Donald> diverged from my version. Sure, you are fine with having someone else Donald> do the difficult and unrewarding debugging and maintainence work, while Donald> you work on just the latest cool hardware, change the interfaces and are Donald> concerned only with the current kernel version. I agree strongly with Donald. Interfaces should NEVER change in patch level versions. Just *DO NOT DO IT*. Go wild in odd-numbered.. get the interfaces right there. But leave them alone afterward. This is a fundamental tenant of being professional. Otherwise, the kernel people are the biggest reason I've ever seen for using *BSD. Microsoft is not the real enemy. Gratuitous change is. ] ON HUMILITY: to err is human. To moo, bovine. | firewalls [ ] Michael Richardson, Sandelman Software Works, Ottawa, ON |net architect[ ] mcr@sandelman.ottawa.on.ca http://www.sandelman.ottawa.on.ca/ |device driver[ ] panic("Just another Debian GNU/Linux using, kernel hacking, security guy"); [ Get Your Private, Free E-mail from Indiatimes at http://email.indiatimes.com Buy the best in Movies at http://www.videos.indiatimes.com Change the way you talk. Indiatimes presents Valufon, Your PC to Phone service with clear voice at rates far less than the normal ISD rates. Go to http://www.valufon.indiatimes.com. Choose your plan. BUY NOW. From arun4linux@indiatimes.com Sat Dec 14 06:33:38 2002 Received: with ECARTIS (v1.0.0; list netdev); Sat, 14 Dec 2002 06:33:45 -0800 (PST) Received: from WS0005.indiatimes.com ([203.199.93.15]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBEEXZ3v030949 for ; Sat, 14 Dec 2002 06:33:37 -0800 Received: from 192.168.57.15 (a1 [192.168.57.21]) by WS0005.indiatimes.com (8.9.3/8.9.3) with SMTP id TAA32276; Sat, 14 Dec 2002 19:58:18 +0530 From: "arun4linux" Message-Id: <200212141428.TAA32276@WS0005.indiatimes.com> To: "Michael Richardson", , "Linux Kernel Mailing List" Reply-To: "arun4linux" Subject: Re: Re: pci-skeleton duplex check Date: Sat, 14 Dec 2002 20:05:17 +0530 X-URL: http://indiatimes.com MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain Content-Transfer-Encoding: 7bit Content-length: 1975 X-archive-position: 1373 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: arun4linux@indiatimes.com Precedence: bulk X-list: netdev <<Interfaces should NEVER change in patch level versions. Just *DO NOT DO IT*. >>I do agree on this. This is a common complaint about linux kernel developers. And this always gives an insecure feeling :-) for the device driver or kernel module programmers. This was one of the issues in my earlier company/work and they have gone for another OS. Warm Regards Arun "Michael Richardson" wrote: -----BEGIN PGP SIGNED MESSAGE----- >>>>> "Donald" == Donald Becker writes: Donald> The drivers in the kernel are now heavily modified and have significantly Donald> diverged from my version. Sure, you are fine with having someone else Donald> do the difficult and unrewarding debugging and maintainence work, while Donald> you work on just the latest cool hardware, change the interfaces and are Donald> concerned only with the current kernel version. I agree strongly with Donald. Interfaces should NEVER change in patch level versions. Just *DO NOT DO IT*. Go wild in odd-numbered.. get the interfaces right there. But leave them alone afterward. This is a fundamental tenant of being professional. Otherwise, the kernel people are the biggest reason I've ever seen for using *BSD. Microsoft is not the real enemy. Gratuitous change is. ] ON HUMILITY: to err is human. To moo, bovine. | firewalls [ ] Michael Richardson, Sandelman Software Works, Ottawa, ON |net architect[ ] mcr@sandelman.ottawa.on.ca http://www.sandelman.ottawa.on.ca/ |device driver[ ] panic("Just another Debian GNU/Linux using, kernel hacking, security guy"); [ Get Your Private, Free E-mail from Indiatimes at http://email.indiatimes.com Buy the best in Movies at http://www.videos.indiatimes.com Change the way you talk. Indiatimes presents "Valufon", Your PC to Phone service with clear voice at rates far less than the normal ISD rates. Go to http://www.valufon.indiatimes.com. Choose your plan. BUY NOW. [[HTML alternate version deleted]] From rmk@arm.linux.org.uk Sat Dec 14 08:11:07 2002 Received: with ECARTIS (v1.0.0; list netdev); Sat, 14 Dec 2002 08:11:11 -0800 (PST) Received: from caramon.arm.linux.org.uk (caramon.arm.linux.org.uk [212.18.232.186]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBEGB33v032748 for ; Sat, 14 Dec 2002 08:11:07 -0800 Received: from flint.arm.linux.org.uk ([3ffe:8260:2002:1:201:2ff:fe14:8fad]) by caramon.arm.linux.org.uk with asmtp (TLSv1:DES-CBC3-SHA:168) (Exim 4.04) id 18NEwB-0006Dz-00; Sat, 14 Dec 2002 16:14:47 +0000 Received: from rmk by flint.arm.linux.org.uk with local (Exim 4.04) id 18NEwA-0006ub-00; Sat, 14 Dec 2002 16:14:46 +0000 Date: Sat, 14 Dec 2002 16:14:46 +0000 From: Russell King To: arun4linux Cc: Michael Richardson , netdev@oss.sgi.com, Linux Kernel Mailing List Subject: Re: Re: pci-skeleton duplex check Message-ID: <20021214161446.B23020@flint.arm.linux.org.uk> Mail-Followup-To: arun4linux , Michael Richardson , netdev@oss.sgi.com, Linux Kernel Mailing List References: <200212141428.TAA32351@WS0005.indiatimes.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <200212141428.TAA32351@WS0005.indiatimes.com>; from arun4linux@indiatimes.com on Sat, Dec 14, 2002 at 08:05:30PM +0530 X-archive-position: 1375 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: rmk@arm.linux.org.uk Precedence: bulk X-list: netdev On Sat, Dec 14, 2002 at 08:05:30PM +0530, arun4linux wrote: > Interfaces should NEVER change in patch level versions. > Just *DO NOT DO IT*. > I do agree on this. Rubbish. Think about what you've just said. Patch level version changes are things like 2.5.43 to 2.5.44 or 2.4.19 to 2.4.20. You are saying that we shouldn't change any interfaces between (eg) 2.5.43 and 2.5.44, but we should change every interface we want to change between 2.4.15 and 2.5.0. This is obviously completely bogus. 2.5 is a _development_ tree. Everyone should expect anything, including interfaces to change between each development patch level. > This is a common complaint about linux kernel developers. And this always > gives an insecure feeling :-) for the device driver or kernel module > programmers. If interfaces are changed without extremely good reason between two _stable_ patch level versions, that would be a bug. > This was one of the issues in my earlier company/work and they have > gone for another OS. Was the problem against a stable kernel version? Did you report the problem when you found it? Was there a response? Unless you have done at least the above, I, for one, have very little sympathy. -- Russell King (rmk@arm.linux.org.uk) The developer of ARM Linux http://www.arm.linux.org.uk/personal/aboutme.html From babydr@baby-dragons.com Sat Dec 14 13:22:58 2002 Received: with ECARTIS (v1.0.0; list netdev); Sat, 14 Dec 2002 13:23:04 -0800 (PST) Received: from filesrv1.baby-dragons.com (ns1.baby-dragons.com [199.33.245.254]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBELMv3v011632 for ; Sat, 14 Dec 2002 13:22:58 -0800 Received: from localhost (babydr@localhost) by filesrv1.baby-dragons.com (8.11.4/8.11.4) with ESMTP id gBELQKd01272; Sat, 14 Dec 2002 16:26:20 -0500 Date: Sat, 14 Dec 2002 16:26:20 -0500 (EST) From: "Mr. James W. Laferriere" To: Russell King cc: arun4linux , Michael Richardson , "" , Linux Kernel Mailing List Subject: Re: Re: pci-skeleton duplex check In-Reply-To: <20021214161446.B23020@flint.arm.linux.org.uk> Message-ID: References: <200212141428.TAA32351@WS0005.indiatimes.com> <20021214161446.B23020@flint.arm.linux.org.uk> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 1376 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: babydr@baby-dragons.com Precedence: bulk X-list: netdev On Sat, 14 Dec 2002, Russell King wrote: ... > Rubbish. > > Think about what you've just said. Patch level version changes are > things like 2.5.43 to 2.5.44 or 2.4.19 to 2.4.20. > > You are saying that we shouldn't change any interfaces between (eg) > 2.5.43 and 2.5.44, but we should change every interface we want to > change between 2.4.15 and 2.5.0. Put very simply yes . x.odd-number.y IS for DEVELOPEMENT , x.even-number.y IS for Stability . If people can not understand that I feel sorry for them . JimL -- +------------------------------------------------------------------+ | James W. Laferriere | System Techniques | Give me VMS | | Network Engineer | P.O. Box 854 | Give me Linux | | babydr@baby-dragons.com | Coudersport PA 16915 | only on AXP | +------------------------------------------------------------------+ From sp@scali.com Sat Dec 14 16:33:20 2002 Received: with ECARTIS (v1.0.0; list netdev); Sat, 14 Dec 2002 16:33:24 -0800 (PST) Received: from elin.scali.no (IDENT:root@elin.scali.no [62.70.89.10]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBF0XI3v014157 for ; Sat, 14 Dec 2002 16:33:19 -0800 Received: from pc-23.office.scali.no (pc-23.office.scali.no [172.16.0.123]) by elin.scali.no (8.12.3/8.12.3) with ESMTP id gBF0asmo012856; Sun, 15 Dec 2002 01:36:55 +0100 Date: Sun, 15 Dec 2002 01:37:27 +0100 (CET) From: Steffen Persvold X-X-Sender: sp@sp-laptop.isdn.scali.no To: Russell King cc: arun4linux , Michael Richardson , , Linux Kernel Mailing List Subject: Re: Re: pci-skeleton duplex check In-Reply-To: <20021214161446.B23020@flint.arm.linux.org.uk> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by amavisd-milter (http://amavis.org/) X-archive-position: 1377 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: sp@scali.com Precedence: bulk X-list: netdev On Sat, 14 Dec 2002, Russell King wrote: > On Sat, Dec 14, 2002 at 08:05:30PM +0530, arun4linux wrote: > > Interfaces should NEVER change in patch level versions. > > Just *DO NOT DO IT*. > > I do agree on this. > > Rubbish. > > Think about what you've just said. Patch level version changes are > things like 2.5.43 to 2.5.44 or 2.4.19 to 2.4.20. > > You are saying that we shouldn't change any interfaces between (eg) > 2.5.43 and 2.5.44, but we should change every interface we want to > change between 2.4.15 and 2.5.0. > > This is obviously completely bogus. 2.5 is a _development_ tree. > Everyone should expect anything, including interfaces to change > between each development patch level. > > > This is a common complaint about linux kernel developers. And this always > > gives an insecure feeling :-) for the device driver or kernel module > > programmers. > > If interfaces are changed without extremely good reason between two > _stable_ patch level versions, that would be a bug. > There have been a few during 2.4... The alloc_kiovec stuff for instance and zap_page_range. 2.2 was much more stable. Interface changes in development series is (or atleast should be to everyone using linux) a known "feature". Regards, -- Steffen Persvold | Scali AS mailto:sp@scali.com | http://www.scali.com Tel: (+47) 2262 8950 | Olaf Helsets vei 6 Fax: (+47) 2262 8951 | N0621 Oslo, NORWAY From jes@trained-monkey.org Sun Dec 15 10:27:20 2002 Received: with ECARTIS (v1.0.0; list netdev); Sun, 15 Dec 2002 10:27:26 -0800 (PST) Received: from trained-monkey.org (IDENT:root@trained-monkey.org [209.217.122.11]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBFIRJ3v032027 for ; Sun, 15 Dec 2002 10:27:20 -0800 Received: (from jes@localhost) by trained-monkey.org (8.11.6/8.9.3) id gBFIVFq23347; Sun, 15 Dec 2002 13:31:15 -0500 To: Andrew Morton Cc: "David S. Miller" , jgarzik@pobox.com, linux-kernel@vger.kernel.org, netdev@oss.sgi.com Subject: Re: [RFC][PATCH] net drivers and cache alignment References: <3DF2844C.F9216283@digeo.com> <20021207.153045.26640406.davem@redhat.com> <3DF28748.186AB31F@digeo.com> <3DF28988.93F268EA@digeo.com> From: Jes Sorensen Date: 15 Dec 2002 13:31:15 -0500 In-Reply-To: Andrew Morton's message of "Sat, 07 Dec 2002 15:51:36 -0800" Message-ID: Lines: 21 X-Mailer: Gnus v5.7/Emacs 20.7 X-archive-position: 1378 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jes@trained-monkey.org Precedence: bulk X-list: netdev >>>>> "Andrew" == Andrew Morton writes: Andrew> Andrew Morton wrote: >> Then I am most confused. None of these fields will be put under >> busmastering or anything like that, so what advantage is there in >> spreading them out? Andrew> Oh I see what you want - to be able to pick up all the Andrew> operating fields in a single fetch. Andrew> That will increase the overall cache footprint though. I Andrew> wonder if it's really a net win, over just keeping it small. There's another case where it matters, I guess one could look at it as similar to the SMP case, but between CPU and device. Some devices have producer indices in host memory which they update whenever it receiving a packet. By putting that seperate from TX data structures you avoid the CPU and the NIC fighting over cache lines. Cheers, Jes From riel@conectiva.com.br Sun Dec 15 12:31:22 2002 Received: with ECARTIS (v1.0.0; list netdev); Sun, 15 Dec 2002 12:31:28 -0800 (PST) Received: from 4-090.ctame701-1.telepar.net.br (root@4-090.ctame701-1.telepar.net.br [200.193.162.90]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBFKUx3v001157 for ; Sun, 15 Dec 2002 12:31:18 -0800 Received: from localhost ([IPv6:::ffff:127.0.0.1]:59073 "EHLO localhost") by imladris.surriel.com with ESMTP id ; Sun, 15 Dec 2002 18:34:11 -0200 Date: Sun, 15 Dec 2002 18:34:06 -0200 (BRST) From: Rik van Riel X-X-Sender: riel@imladris.surriel.com To: netdev@oss.sgi.com cc: linux-kernel@vger.kernel.org Subject: RFC: p&p ipsec without authentication Message-ID: X-spambait: aardvark@kernelnewbies.org X-spammeplease: aardvark@nl.linux.org MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 1379 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: riel@conectiva.com.br Precedence: bulk X-list: netdev Hi, I've got a crazy idea. I know it's not secure, but I think it'll add some security against certain attacks, while being non-effective against some others. The idea I have is letting the ipsec layer do opportunistic encryption even when there are no ipsec keys known for the destination address, ie. negotiate a key when none is in the configuration or DNS. I know this gives absolutely no protection against man-in-the-middle attacks (except maybe being able to detect them), but it should prevent passive sniffing of network traffic, as done by some governments. If this "random" encryption could be turned on with one argument to ip or ifconfig and millions of hosts would use it, sniffing internet traffic might just become impractical (or too expensive) for large organisations. Furthermore, even if just 0.1% of the hosts were to use ipsec authentication, the 3-letter agencies would be faced with the additional challenge of identifying which connections could safely be intercepted with man-in-the-middle attacks and which couldn't. Not to mention the fact that the port number on many communications would be invisible, vastly increasing the difficulty of doing any kind of statistical analysis on the traffic that's traversing the network. Is this idea completely crazy or only slightly ? regards, Rik -- Bravely reimplemented by the knights who say "NIH". http://www.surriel.com/ http://guru.conectiva.com/ Current spamtrap: october@surriel.com From andrew@indranet.co.nz Sun Dec 15 14:07:35 2002 Received: with ECARTIS (v1.0.0; list netdev); Sun, 15 Dec 2002 14:07:39 -0800 (PST) Received: from mail.acheron.indranet.co.nz (ns.indranet.co.nz [210.54.239.210]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBFM7W3v004718 for ; Sun, 15 Dec 2002 14:07:34 -0800 Received: from localhost.localdomain (enso.acheron.indranet.co.nz [192.168.1.1]) by mail.acheron.indranet.co.nz (8.11.2/8.11.2) with ESMTP id gBFM7LQ24631; Mon, 16 Dec 2002 11:07:21 +1300 Date: Mon, 16 Dec 2002 10:59:10 +1300 From: Andrew McGregor To: Rik van Riel , netdev@oss.sgi.com cc: linux-kernel@vger.kernel.org Subject: Re: RFC: p&p ipsec without authentication Message-ID: <3050000.1039989550@localhost.localdomain> In-Reply-To: References: X-Mailer: Mulberry/3.0.0b9 (Linux/x86) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-archive-position: 1380 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: andrew@indranet.co.nz Precedence: bulk X-list: netdev It's not crazy at all. Perfectly practical, now that lots of people have fast enough machines and slow enough connections that it won't drive them mad with the performance issues :-) Actually, it can be done (fairly) securely against MITM attacks as well. Check out a keying protocol called HIP, most of the resources are linked to from www.hip4inter.net. The basic idea is that each end prove to the other that they know a private key. The MITM protection is quite hard to describe :-) And it can be done (at least on IPv6) with almost zero cost in time for connections that don't support HIP, as well as only one round trip + compute time for those that do. There are four implementations in progress, two for linux. It would be very nice to get the necessary hooks into the mainline kernel. Cool, eh? Andrew --On Sunday, December 15, 2002 18:34:06 -0200 Rik van Riel wrote: > Hi, > > I've got a crazy idea. I know it's not secure, but I think it'll > add some security against certain attacks, while being non-effective > against some others. > > The idea I have is letting the ipsec layer do opportunistic encryption > even when there are no ipsec keys known for the destination address, > ie. negotiate a key when none is in the configuration or DNS. > > I know this gives absolutely no protection against man-in-the-middle > attacks (except maybe being able to detect them), but it should prevent > passive sniffing of network traffic, as done by some governments. > > If this "random" encryption could be turned on with one argument to > ip or ifconfig and millions of hosts would use it, sniffing internet > traffic might just become impractical (or too expensive) for large > organisations. Furthermore, even if just 0.1% of the hosts were to > use ipsec authentication, the 3-letter agencies would be faced with > the additional challenge of identifying which connections could safely > be intercepted with man-in-the-middle attacks and which couldn't. > > Not to mention the fact that the port number on many communications > would be invisible, vastly increasing the difficulty of doing any > kind of statistical analysis on the traffic that's traversing the > network. > > Is this idea completely crazy or only slightly ? > > regards, > > Rik > -- > Bravely reimplemented by the knights who say "NIH". > http://www.surriel.com/ http://guru.conectiva.com/ > Current spamtrap: href=mailto:"october@surriel.com">october@surriel.com - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ > > From yoshfuji@linux-ipv6.org Sun Dec 15 23:57:34 2002 Received: with ECARTIS (v1.0.0; list netdev); Sun, 15 Dec 2002 23:57:39 -0800 (PST) Received: from yue.hongo.wide.ad.jp (yue.hongo.wide.ad.jp [203.178.139.94]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBG7vH3v000813 for ; Sun, 15 Dec 2002 23:57:18 -0800 Received: from localhost (localhost [127.0.0.1]) by yue.hongo.wide.ad.jp (8.12.3+3.5Wbeta/8.12.3/Debian -4) with ESMTP id gBG6ZWGR024187; Mon, 16 Dec 2002 15:35:33 +0900 Date: Mon, 16 Dec 2002 15:35:32 +0900 (JST) Message-Id: <20021216.153532.87456123.yoshfuji@wide.ad.jp> To: linux-kernel@vger.kernel.org CC: netdev@oss.sgi.com, davem@redhat.com, kuznet@ms2.inr.ac.ru, krkumar@us.ibm.com Subject: Re: [PATCH RESEND] memory leak in ndisc_router_discovery From: YOSHIFUJI Hideaki / =?iso-2022-jp?B?GyRCNUhGIzFRTEAbKEI=?= In-Reply-To: <200212121905.gBCJ5hn18058@eng2.beaverton.ibm.com> References: <200212121905.gBCJ5hn18058@eng2.beaverton.ibm.com> Organization: USAGI Project X-URL: http://www.yoshifuji.org/%7Ehideaki/ X-Fingerprint: 90 22 65 EB 1E CF 3A D1 0B DF 80 D8 48 07 F8 94 E0 62 0E EA X-PGP-Key-URL: http://www.yoshifuji.org/%7Ehideaki/hideaki@yoshifuji.org.asc X-Face: "5$Al-.M>NJ%a'@hhZdQm:."qn~PA^gq4o*>iCFToq*bAi#4FRtx}enhuQKz7fNqQz\BYU] $~O_5m-9'}MIs`XGwIEscw;e5b>n"B_?j/AkL~i/MEaZBLP Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Mailer: Mew version 2.2 on XEmacs 21.4.6 (Common Lisp) Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-archive-position: 1381 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: yoshfuji@linux-ipv6.org Precedence: bulk X-list: netdev In article <200212121905.gBCJ5hn18058@eng2.beaverton.ibm.com> (at Thu, 12 Dec 2002 11:05:43 -0800 (PST)), Krishna Kumar says: > I had sent this earlier, there is a bug in router advertisement handling code, > where the reference (and memory) to an inet6_dev pointer can get leaked (this > leak can happen atmost once for each interface on a system which receives > invalid RA's). Below is the patch against 2.5.51 to fix it. (It would be called "refcnt leakage," or some thing like that, but anyway) This seems correct fix. please apply... > ------------------------------------------------------------------------------- > diff -ruN linux.org/net/ipv6/ndisc.c linux/net/ipv6/ndisc.c > --- linux.org/net/ipv6/ndisc.c Fri Nov 7 10:02:11 2002 > +++ linux/net/ipv6/ndisc.c Fri Nov 8 14:37:27 2002 > @@ -871,6 +871,7 @@ > } > > if (!ndisc_parse_options(opt, optlen, &ndopts)) { > + in6_dev_put(in6_dev); > if (net_ratelimit()) > ND_PRINTK2(KERN_WARNING > "ICMP6 RA: invalid ND option, ignored.\n"); > ------------------------------------------------------------------------------- -- Hideaki YOSHIFUJI @ USAGI Project GPG FP: 9022 65EB 1ECF 3AD1 0BDF 80D8 4807 F894 E062 0EEA From kiran@in.ibm.com Mon Dec 16 05:50:24 2002 Received: with ECARTIS (v1.0.0; list netdev); Mon, 16 Dec 2002 05:50:26 -0800 (PST) Received: from e31.co.us.ibm.com (e31.co.us.ibm.com [32.97.110.129]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBGDoN3v015686 for ; Mon, 16 Dec 2002 05:50:23 -0800 Received: from westrelay05.boulder.ibm.com (westrelay05.boulder.ibm.com [9.17.193.33]) by e31.co.us.ibm.com (8.12.2/8.12.2) with ESMTP id gBGDsKcQ019606; Mon, 16 Dec 2002 08:54:20 -0500 Received: from asterix.in.ibm.com (asterix.in.ibm.com [9.182.12.249]) by westrelay05.boulder.ibm.com (8.12.3/NCO/VER6.4) with ESMTP id gBGDtRMB145966; Mon, 16 Dec 2002 06:55:28 -0700 Received: (from kirant@localhost) by asterix.in.ibm.com (8.11.2/8.11.2) id gBGDqCq13251; Mon, 16 Dec 2002 19:22:12 +0530 Date: Mon, 16 Dec 2002 19:22:12 +0530 From: Ravikiran G Thirumalai To: "David S. Miller " Cc: linux-kernel@vger.kernel.org, netdev , dipankar@in.ibm.com, Andrew Morton Subject: [patch] Make rt_cache_stat use kmalloc_percpu Message-ID: <20021216192212.C26076@in.ibm.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i X-archive-position: 1382 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kiran@in.ibm.com Precedence: bulk X-list: netdev Hi, Here's another patch to use kmalloc_percpu. As usual, this removes NR_CPUS bloat, can work when modulized and helps node local allocation. Thanks, Kiran D: Name: rtstats-2.5.52-1.patch D: Author: Ravikiran Thirumalai D: Description: Make rt_cache_stat use kmalloc_percpu include/net/route.h | 6 +++- net/ipv4/route.c | 71 +++++++++++++++++++++++++++++----------------------- 2 files changed, 45 insertions(+), 32 deletions(-) diff -ruN -X dontdiff linux-2.5.52/include/net/route.h rtstats-2.5.52/include/net/route.h --- linux-2.5.52/include/net/route.h Mon Dec 16 07:37:42 2002 +++ rtstats-2.5.52/include/net/route.h Mon Dec 16 14:56:01 2002 @@ -102,7 +102,11 @@ unsigned int gc_ignored; unsigned int gc_goal_miss; unsigned int gc_dst_overflow; -} ____cacheline_aligned_in_smp; +}; + +extern struct rt_cache_stat *rt_cache_stat; +#define RT_CACHE_STAT_INC(field) \ + (per_cpu_ptr(rt_cache_stat, smp_processor_id())->field++) extern struct ip_rt_acct *ip_rt_acct; diff -ruN -X dontdiff linux-2.5.52/net/ipv4/route.c rtstats-2.5.52/net/ipv4/route.c --- linux-2.5.52/net/ipv4/route.c Mon Dec 16 07:38:23 2002 +++ rtstats-2.5.52/net/ipv4/route.c Mon Dec 16 16:14:09 2002 @@ -195,7 +195,7 @@ static unsigned rt_hash_mask; static int rt_hash_log; -struct rt_cache_stat rt_cache_stat[NR_CPUS]; +struct rt_cache_stat *rt_cache_stat; static int rt_intern_hash(unsigned hash, struct rtable *rth, struct rtable **res); @@ -319,24 +319,26 @@ int len = 0; for (i = 0; i < NR_CPUS; i++) { + if (!cpu_possible(i)) + continue; len += sprintf(buffer+len, "%08x %08x %08x %08x %08x %08x %08x %08x %08x %08x %08x %08x %08x %08x %08x \n", dst_entries, - rt_cache_stat[i].in_hit, - rt_cache_stat[i].in_slow_tot, - rt_cache_stat[i].in_slow_mc, - rt_cache_stat[i].in_no_route, - rt_cache_stat[i].in_brd, - rt_cache_stat[i].in_martian_dst, - rt_cache_stat[i].in_martian_src, - - rt_cache_stat[i].out_hit, - rt_cache_stat[i].out_slow_tot, - rt_cache_stat[i].out_slow_mc, - - rt_cache_stat[i].gc_total, - rt_cache_stat[i].gc_ignored, - rt_cache_stat[i].gc_goal_miss, - rt_cache_stat[i].gc_dst_overflow + per_cpu_ptr(rt_cache_stat, i)->in_hit, + per_cpu_ptr(rt_cache_stat, i)->in_slow_tot, + per_cpu_ptr(rt_cache_stat, i)->in_slow_mc, + per_cpu_ptr(rt_cache_stat, i)->in_no_route, + per_cpu_ptr(rt_cache_stat, i)->in_brd, + per_cpu_ptr(rt_cache_stat, i)->in_martian_dst, + per_cpu_ptr(rt_cache_stat, i)->in_martian_src, + + per_cpu_ptr(rt_cache_stat, i)->out_hit, + per_cpu_ptr(rt_cache_stat, i)->out_slow_tot, + per_cpu_ptr(rt_cache_stat, i)->out_slow_mc, + + per_cpu_ptr(rt_cache_stat, i)->gc_total, + per_cpu_ptr(rt_cache_stat, i)->gc_ignored, + per_cpu_ptr(rt_cache_stat, i)->gc_goal_miss, + per_cpu_ptr(rt_cache_stat, i)->gc_dst_overflow ); } @@ -592,11 +594,11 @@ * do not make it too frequently. */ - rt_cache_stat[smp_processor_id()].gc_total++; + RT_CACHE_STAT_INC(gc_total); if (now - last_gc < ip_rt_gc_min_interval && atomic_read(&ipv4_dst_ops.entries) < ip_rt_max_size) { - rt_cache_stat[smp_processor_id()].gc_ignored++; + RT_CACHE_STAT_INC(gc_ignored); goto out; } @@ -664,7 +666,7 @@ We will not spin here for long time in any case. */ - rt_cache_stat[smp_processor_id()].gc_goal_miss++; + RT_CACHE_STAT_INC(gc_goal_miss); if (expire == 0) break; @@ -683,7 +685,7 @@ goto out; if (net_ratelimit()) printk(KERN_WARNING "dst cache overflow\n"); - rt_cache_stat[smp_processor_id()].gc_dst_overflow++; + RT_CACHE_STAT_INC(gc_dst_overflow); return 1; work_done: @@ -1387,7 +1389,7 @@ if (!LOCAL_MCAST(daddr) && IN_DEV_MFORWARD(in_dev)) rth->u.dst.input = ip_mr_input; #endif - rt_cache_stat[smp_processor_id()].in_slow_mc++; + RT_CACHE_STAT_INC(in_slow_mc); in_dev_put(in_dev); hash = rt_hash_code(daddr, saddr ^ (dev->ifindex << 5), tos); @@ -1472,7 +1474,7 @@ } free_res = 1; - rt_cache_stat[smp_processor_id()].in_slow_tot++; + RT_CACHE_STAT_INC(in_slow_tot); #ifdef CONFIG_IP_ROUTE_NAT /* Policy is applied before mapping destination, @@ -1629,7 +1631,7 @@ } flags |= RTCF_BROADCAST; res.type = RTN_BROADCAST; - rt_cache_stat[smp_processor_id()].in_brd++; + RT_CACHE_STAT_INC(in_brd); local_input: rth = dst_alloc(&ipv4_dst_ops); @@ -1674,7 +1676,7 @@ goto intern; no_route: - rt_cache_stat[smp_processor_id()].in_no_route++; + RT_CACHE_STAT_INC(in_no_route); spec_dst = inet_select_addr(dev, 0, RT_SCOPE_UNIVERSE); res.type = RTN_UNREACHABLE; goto local_input; @@ -1683,7 +1685,7 @@ * Do not cache martian addresses: they should be logged (RFC1812) */ martian_destination: - rt_cache_stat[smp_processor_id()].in_martian_dst++; + RT_CACHE_STAT_INC(in_martian_dst); #ifdef CONFIG_IP_ROUTE_VERBOSE if (IN_DEV_LOG_MARTIANS(in_dev) && net_ratelimit()) printk(KERN_WARNING "martian destination %u.%u.%u.%u from " @@ -1700,7 +1702,7 @@ martian_source: - rt_cache_stat[smp_processor_id()].in_martian_src++; + RT_CACHE_STAT_INC(in_martian_src); #ifdef CONFIG_IP_ROUTE_VERBOSE if (IN_DEV_LOG_MARTIANS(in_dev) && net_ratelimit()) { /* @@ -1749,7 +1751,7 @@ rth->u.dst.lastuse = jiffies; dst_hold(&rth->u.dst); rth->u.dst.__use++; - rt_cache_stat[smp_processor_id()].in_hit++; + RT_CACHE_STAT_INC(in_hit); read_unlock(&rt_hash_table[hash].lock); skb->dst = (struct dst_entry*)rth; return 0; @@ -2046,7 +2048,7 @@ rth->u.dst.output=ip_output; - rt_cache_stat[smp_processor_id()].out_slow_tot++; + RT_CACHE_STAT_INC(out_slow_tot); if (flags & RTCF_LOCAL) { rth->u.dst.input = ip_local_deliver; @@ -2056,7 +2058,7 @@ rth->rt_spec_dst = fl.fl4_src; if (flags & RTCF_LOCAL && !(dev_out->flags & IFF_LOOPBACK)) { rth->u.dst.output = ip_mc_output; - rt_cache_stat[smp_processor_id()].out_slow_mc++; + RT_CACHE_STAT_INC(out_slow_mc); } #ifdef CONFIG_IP_MROUTE if (res.type == RTN_MULTICAST) { @@ -2114,7 +2116,7 @@ rth->u.dst.lastuse = jiffies; dst_hold(&rth->u.dst); rth->u.dst.__use++; - rt_cache_stat[smp_processor_id()].out_hit++; + RT_CACHE_STAT_INC(out_hit); read_unlock_bh(&rt_hash_table[hash].lock); *rp = rth; return 0; @@ -2634,6 +2636,11 @@ ipv4_dst_ops.gc_thresh = (rt_hash_mask + 1); ip_rt_max_size = (rt_hash_mask + 1) * 16; + rt_cache_stat = kmalloc_percpu(sizeof (struct rt_cache_stat), + GFP_KERNEL); + if (!rt_cache_stat) + goto out_enomem1; + devinet_init(); ip_fib_init(); @@ -2659,6 +2666,8 @@ out: return rc; out_enomem: + kfree_percpu(rt_cache_stat); +out_enomem1: rc = -ENOMEM; goto out; } From pb@bieringer.de Mon Dec 16 08:34:05 2002 Received: with ECARTIS (v1.0.0; list netdev); Mon, 16 Dec 2002 08:34:07 -0800 (PST) Received: from smtp2.aerasec.de (gromit.aerasec.de [195.226.187.57]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBGGXs3v000595 for ; Mon, 16 Dec 2002 08:33:55 -0800 Received: from localhost (localhost [127.0.0.1]) by smtp2.aerasec.de (Postfix) with SMTP id 031A913866 for ; Mon, 16 Dec 2002 17:10:46 +0100 (CET) X-AV-Checked: Mon Dec 16 17:10:46 2002 smtp2.aerasec.de Received: from [192.168.1.2] (pD950F045.dip.t-dialin.net [217.80.240.69]) (using TLSv1 with cipher EDH-RSA-DES-CBC3-SHA (168/168 bits)) (Client did not present a certificate) by smtp2.aerasec.de (Postfix) with ESMTP id 6D34913865 for ; Mon, 16 Dec 2002 17:10:45 +0100 (CET) Date: Mon, 16 Dec 2002 17:10:45 +0100 From: Peter Bieringer To: Maillist netdev Subject: Mailling list archive URL broken? Message-ID: <70060000.1040055045@worker.muc.bieringer.de> X-Mailer: Mulberry/3.0.0b9 (Linux/x86) X-URL: http://www.bieringer.de/pb/ X-OS: Linux MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline X-archive-position: 1383 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: pb@bieringer.de Precedence: bulk X-list: netdev Hi, some time ago I found this archive URL: http://oss.sgi.com/projects/netdev/archive/ Unfortunately, it's broken (forbidden). Temp failure or is archive moved? If second, pls. send me the new URL. Thank you very much, Peter --- Dr. Peter Bieringer mailto: pb at bieringer dot de http://www.bieringer.de/pb/ Key 0x958F422D : B501 24F4 9418 23E2 C0F3 F833 7B57 AA7B 958F 422D From Larry.Sendlosky@storigen.com Mon Dec 16 09:55:59 2002 Received: with ECARTIS (v1.0.0; list netdev); Mon, 16 Dec 2002 09:56:02 -0800 (PST) Received: from xchangeserver2.storigen.com ([65.193.106.66]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBGHtw3v002481 for ; Mon, 16 Dec 2002 09:55:59 -0800 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Subject: 2.4.20, tg3 , and 5704 X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Date: Mon, 16 Dec 2002 12:59:54 -0500 Message-ID: <7BFCE5F1EF28D64198522688F5449D5A03C0CE@xchangeserver2.storigen.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: 2.4.20, tg3 , and 5704 Thread-Index: AcKh/I6Nmqmd/VAPR/WmSMP3gHX6mADL+zOQ From: "Larry Sendlosky" To: Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by oss.sgi.com id gBGHtw3v002481 X-archive-position: 1384 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: Larry.Sendlosky@storigen.com Precedence: bulk X-list: netdev Hi, I got a preproduction Dell server with 2 5704 NICs onboard. 2.4.20 w/tg3 doesn't want to work. When loaded the link light on the switch just goes on-off-on-off ... seeming like it is reinitializing all the time. I just did get the current Broadcom drivers sources and it seems to work ok (ping and ftp works on 100Mb ...) built against 2.4.20. I prefer to use tg3 even though I have to backport to 2.4.18 (not a big fan of the Broadcom code and it's not part of linux). thanks for any help larry From aravind1001@speedpost.net Mon Dec 16 10:51:14 2002 Received: with ECARTIS (v1.0.0; list netdev); Mon, 16 Dec 2002 10:51:19 -0800 (PST) Received: from server2.fastmail.fm (ny2.fastmail.fm [66.111.4.3]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBGIpD3v003336 for ; Mon, 16 Dec 2002 10:51:14 -0800 Received: from server2.fastmail.fm (localhost [127.0.0.1]) by fastmail.fm (Postfix) with ESMTP id 48B1717B43; Mon, 16 Dec 2002 13:55:13 -0500 (EST) Received: from 127.0.0.1 ([127.0.0.1] helo=server2.fastmail.fm) by fastmail.fm with SMTP; Mon, 16 Dec 2002 13:55:13 -0500 X-Mail-from: aravind1001@speedpost.net Received: by server2.fastmail.fm (Postfix, from userid 99) id 43F023D56F; Mon, 16 Dec 2002 13:55:13 -0500 (EST) Content-Disposition: inline Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="ISO-8859-1" MIME-Version: 1.0 X-Mailer: MIME::Lite 1.2 (F2.71; T1.001; A1.51; B2.12; Q2.03) From: "Aravind Ceyardass" To: "arun4linux" , "Michael Richardson" , netdev@oss.sgi.com, "Linux Kernel Mailing List" Date: Mon, 16 Dec 2002 14:55:13 -0400 X-Epoch: 1040064913 X-Sasl-enc: K1tEy+9myIhjIDMkv6MRmQ Cc: aravind1001@speedpost.net Subject: Re: Re: pci-skeleton duplex check References: <200212141428.TAA32351@WS0005.indiatimes.com> In-Reply-To: <200212141428.TAA32351@WS0005.indiatimes.com> Message-Id: <20021216185513.43F023D56F@server2.fastmail.fm> X-archive-position: 1385 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: aravind1001@speedpost.net Precedence: bulk X-list: netdev Hi, A good scheme for numbering kernels or software components in general is as follows For stable releases. (x.even.y=major.minor.patch) increment patch for any bug fixes. increment minor for any enhancements or new interfaces. increment major for interface changes or interface deletions.(dangerous or poor design) We should increment major even if interface remains same but behaviour has changed.(again may be poor design) For development releases we can't follow the above scheme, because the interfaces are in a flux and we may end up in version 589.201.700 from 2.4.20. So, we decide to increment patch number for all changes and deletions. Hope it helps! Regards Aravind -- http://fastmail.fm - IMAP accessible web-mail From ralf@linux-mips.org Mon Dec 16 16:38:43 2002 Received: with ECARTIS (v1.0.0; list netdev); Mon, 16 Dec 2002 16:38:49 -0800 (PST) Received: from dea.linux-mips.net (p508B6B70.dip.t-dialin.net [80.139.107.112]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBH0ce3v006783 for ; Mon, 16 Dec 2002 16:38:42 -0800 Received: (from ralf@localhost) by dea.linux-mips.net (8.11.6/8.11.6) id gBH0get15155; Tue, 17 Dec 2002 01:42:40 +0100 Date: Tue, 17 Dec 2002 01:42:40 +0100 From: Ralf Baechle To: Peter Bieringer Cc: Maillist netdev Subject: Re: Mailling list archive URL broken? Message-ID: <20021217014240.A14491@linux-mips.org> References: <70060000.1040055045@worker.muc.bieringer.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <70060000.1040055045@worker.muc.bieringer.de>; from pb@bieringer.de on Mon, Dec 16, 2002 at 05:10:45PM +0100 X-archive-position: 1386 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: ralf@linux-mips.org Precedence: bulk X-list: netdev On Mon, Dec 16, 2002 at 05:10:45PM +0100, Peter Bieringer wrote: > some time ago I found this archive URL: > > http://oss.sgi.com/projects/netdev/archive/ > > Unfortunately, it's broken (forbidden). Temp failure or is archive > moved? If second, pls. send me the new URL. A glitch in the recent conversion from Majordomo to Ecartis, I just fixed this. This is just the URL of the mbox archives maintained by Ecartis. There's also a webarchive ... Ralf From kernel@itris.fr Tue Dec 17 02:16:14 2002 Received: with ECARTIS (v1.0.0; list netdev); Tue, 17 Dec 2002 02:16:19 -0800 (PST) Received: from mail.itris.fr (AGrenoble-203-2-2-11.abo.wanadoo.fr [80.13.134.11]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBHAGD3v015954 for ; Tue, 17 Dec 2002 02:16:14 -0800 Received: from itris.fr (localhost [127.0.0.1]) by mail.itris.fr (Postfix) with SMTP id 040DA3FF4 for ; Tue, 17 Dec 2002 11:16:38 +0100 (CET) Received: from 192.168.0.31 (SquirrelMail authenticated user kernel) by www.itris.fr with HTTP; Tue, 17 Dec 2002 11:16:39 +0100 (CET) Message-ID: <1939.192.168.0.31.1040120199.squirrel@www.itris.fr> Date: Tue, 17 Dec 2002 11:16:39 +0100 (CET) Subject: 3c59x AUI and sockets AF_PACKET From: To: X-Priority: 3 Importance: Normal X-MSMail-Priority: Normal X-Mailer: SquirrelMail (version 1.2.6) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-archive-position: 1387 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kernel@itris.fr Precedence: bulk X-list: netdev Hi, I have got an important problem to solve : When I load the 3c59x module with no option, my 2 network cards are working well with 10baseT, and vortex-diag tells me then : Transceiver type in use: Autonegociate. I can open a socket with AF_PACKET level, and bind() works fine ! I can send data to the socket through RJ45. But, when I load 3c59x with options=1, vortex-diag tells me Transceiver type in use : AUI10baseFL. I can configure my 2 network cards with ifconfig. Everything is ok, but, when i want to open my socket AF_PACKET to send data to AUI port, bind () returns -1, as though all my interfaces are ok (ifconfig sees them all). I would like to send data to AUI, what is the problem ? Thanks !! Christophe Baillon From greearb@candelatech.com Tue Dec 17 12:06:43 2002 Received: with ECARTIS (v1.0.0; list netdev); Tue, 17 Dec 2002 12:06:48 -0800 (PST) Received: from grok.yi.org (IDENT:HTLCq4W5nkXFjSCAvZkSqhCA+oOVfq/2@dhcp101-dsl-usw4.w-link.net [208.161.125.101]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBHK6e3v032602 for ; Tue, 17 Dec 2002 12:06:42 -0800 Received: from candelatech.com (IDENT:buTF2ZfWbwpiFLbR+OwjUwO9KcG3EhBV@localhost.localdomain [127.0.0.1]) by grok.yi.org (8.11.6/8.11.2) with ESMTP id gBHKAg002188; Tue, 17 Dec 2002 12:10:42 -0800 Message-ID: <3DFF84C2.9090305@candelatech.com> Date: Tue, 17 Dec 2002 12:10:42 -0800 From: Ben Greear Organization: Candela Technologies User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20021130 X-Accept-Language: en-us, en MIME-Version: 1.0 To: kernel@itris.fr CC: netdev@oss.sgi.com Subject: Re: 3c59x AUI and sockets AF_PACKET References: <1939.192.168.0.31.1040120199.squirrel@www.itris.fr> In-Reply-To: <1939.192.168.0.31.1040120199.squirrel@www.itris.fr> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 1388 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: greearb@candelatech.com Precedence: bulk X-list: netdev kernel@itris.fr wrote: > Hi, > > I have got an important problem to solve : > > When I load the 3c59x module with no option, my 2 network cards > are working well with 10baseT, and vortex-diag tells me then : > Transceiver type in use: Autonegociate. > > I can open a socket with AF_PACKET level, and bind() works fine ! > I can send data to the socket through RJ45. > > But, when I load 3c59x with options=1, vortex-diag tells me > Transceiver type in use : AUI10baseFL. > > I can configure my 2 network cards with ifconfig. > > Everything is ok, but, when i want to open my socket AF_PACKET to > send data to AUI port, bind () returns -1, as though all my interfaces > are ok (ifconfig sees them all). After the call fails, do this (it may offer a clue): printf("%i: %s\n", errno, strerror(errno)); /* Hopes syntax is correct */ > > I would like to send data to AUI, what is the problem ? > > Thanks !! > > Christophe Baillon > > -- Ben Greear President of Candela Technologies Inc http://www.candelatech.com ScryMUD: http://scry.wanfear.com http://scry.wanfear.com/~greear From scott.feldman@intel.com Tue Dec 17 13:59:52 2002 Received: with ECARTIS (v1.0.0; list netdev); Tue, 17 Dec 2002 13:59:55 -0800 (PST) Received: from caduceus.jf.intel.com (fmr06.intel.com [134.134.136.7]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBHLxp3v009132 for ; Tue, 17 Dec 2002 13:59:52 -0800 Received: from petasus.jf.intel.com (petasus.jf.intel.com [10.7.209.6]) by caduceus.jf.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.51 2002/09/23 20:43:23 dmccart Exp $) with ESMTP id gBHM0cN27535 for ; Tue, 17 Dec 2002 22:00:38 GMT Received: from orsmsxvs040.jf.intel.com (orsmsxvs040.jf.intel.com [192.168.65.206]) by petasus.jf.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.27 2002/10/16 23:46:59 dmccart Exp $) with SMTP id gBHM0fL25370 for ; Tue, 17 Dec 2002 22:00:41 GMT Received: from orsmsx26.jf.intel.com ([192.168.65.26]) by orsmsxvs040.jf.intel.com (NAVGW 2.5.2.11) with SMTP id M2002121714091001513 ; Tue, 17 Dec 2002 14:09:10 -0800 Received: by orsmsx26.jf.intel.com with Internet Mail Service (5.5.2653.19) id ; Tue, 17 Dec 2002 14:03:52 -0800 Message-ID: <288F9BF66CD9D5118DF400508B68C44604758F6C@orsmsx113.jf.intel.com> From: "Feldman, Scott" To: linux-kernel@vger.kernel.org, netdev@oss.sgi.com Cc: LOSTeam Subject: [ANNOUNCE] Intel PRO/100 software developer manual released Date: Tue, 17 Dec 2002 14:03:50 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain X-archive-position: 1389 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: scott.feldman@intel.com Precedence: bulk X-list: netdev Available at https://sourceforge.net/projects/e1000. The full title is: Intel 8255x 10/100 Mbps Ethernet Controller Family Open Source Software Developer Manual Revision 1.0 The manual is intended to support the maintenance of the e100 driver (or the best driver for the PRO/100 networking hardware ;-). The manual covers the 82557, 82558, 82559, 82550, and 82551 Ethernet controllers. I would like to thank Jeff Garzik for encouraging this publication, and for having the patience to deal with the Intel machine. I would also like to thank the Intel editors and reviewers: Carolyn Abrigana, Larry Bates, Julie Donnelly, John Ronciak, Wen-Hwa Tao, Eli Kupermann, David Valdez, Colleen Culbertson, and especially Glenn Begis for not giving up. -scott From jgarzik@pobox.com Tue Dec 17 14:56:27 2002 Received: with ECARTIS (v1.0.0; list netdev); Tue, 17 Dec 2002 14:56:31 -0800 (PST) Received: from www.linux.org.uk (parcelfarce.linux.theplanet.co.uk [195.92.249.252]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBHMuP3v009889 for ; Tue, 17 Dec 2002 14:56:26 -0800 Received: from nat-pool-rdu.redhat.com ([66.187.233.200] helo=pobox.com) by www.linux.org.uk with esmtp (Exim 3.33 #5) id 18OQhT-0001dg-00; Tue, 17 Dec 2002 23:00:31 +0000 Message-ID: <3DFFAC6F.9010903@pobox.com> Date: Tue, 17 Dec 2002 17:59:59 -0500 From: Jeff Garzik User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2) Gecko/20021202 X-Accept-Language: en-us, en MIME-Version: 1.0 To: "Feldman, Scott" CC: linux-kernel@vger.kernel.org, netdev@oss.sgi.com, LOSTeam Subject: Re: [ANNOUNCE] Intel PRO/100 software developer manual released References: <288F9BF66CD9D5118DF400508B68C44604758F6C@orsmsx113.jf.intel.com> In-Reply-To: <288F9BF66CD9D5118DF400508B68C44604758F6C@orsmsx113.jf.intel.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 1390 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jgarzik@pobox.com Precedence: bulk X-list: netdev Feldman, Scott wrote: > Available at https://sourceforge.net/projects/e1000. > > The full title is: > > Intel 8255x 10/100 Mbps Ethernet Controller Family > Open Source Software Developer Manual > Revision 1.0 > > The manual is intended to support the maintenance of the e100 driver (or the > best driver for the PRO/100 networking hardware ;-). The manual covers the > 82557, 82558, 82559, 82550, and 82551 Ethernet controllers. > > I would like to thank Jeff Garzik for encouraging this publication, and for > having the patience to deal with the Intel machine. I'm surprised that this David (me) actually got through to the Goliath :) Let me publicly thank Intel NIC team [and related persons] for some great work. You guys have really been responsive both to technical issues and also political/community-related issues like this one of opening docs. This is a situation where I really think that opening docs will lead to a better overall user experience with Intel hardware on Linux, and hopefully serves as a model for other vendors to follow. Thanks and kudos, Intel! Jeff From anton@samba.org Tue Dec 17 19:25:06 2002 Received: with ECARTIS (v1.0.0; list netdev); Tue, 17 Dec 2002 19:25:08 -0800 (PST) Received: from lists.samba.org (dp.samba.org [66.70.73.150]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBI3P53v014400 for ; Tue, 17 Dec 2002 19:25:06 -0800 Received: by lists.samba.org (Postfix, from userid 504) id A270D2C122; Tue, 17 Dec 2002 22:29:13 -0500 (EST) Date: Wed, 18 Dec 2002 14:27:32 +1100 From: Anton Blanchard To: "Feldman, Scott" Cc: linux-kernel@vger.kernel.org, netdev@oss.sgi.com, LOSTeam Subject: Re: [ANNOUNCE] Intel PRO/100 software developer manual released Message-ID: <20021218032732.GC22434@krispykreme> References: <288F9BF66CD9D5118DF400508B68C44604758F6C@orsmsx113.jf.intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <288F9BF66CD9D5118DF400508B68C44604758F6C@orsmsx113.jf.intel.com> User-Agent: Mutt/1.4i X-archive-position: 1391 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: anton@samba.org Precedence: bulk X-list: netdev > Available at https://sourceforge.net/projects/e1000. > > The full title is: > > Intel 8255x 10/100 Mbps Ethernet Controller Family > Open Source Software Developer Manual > Revision 1.0 > > The manual is intended to support the maintenance of the e100 driver (or the > best driver for the PRO/100 networking hardware ;-). The manual covers the > 82557, 82558, 82559, 82550, and 82551 Ethernet controllers. Nice work guys! This has already answered a few questions I had when tuning the e100 on ppc64. Any plans to release similar docs for the e1000? Anton From scott.feldman@intel.com Tue Dec 17 19:42:22 2002 Received: with ECARTIS (v1.0.0; list netdev); Tue, 17 Dec 2002 19:42:26 -0800 (PST) Received: from hermes.fm.intel.com (fmr01.intel.com [192.55.52.18]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBI3gL3v014919 for ; Tue, 17 Dec 2002 19:42:21 -0800 Received: from petasus.fm.intel.com (petasus.fm.intel.com [10.1.192.37]) by hermes.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.51 2002/09/23 20:43:23 dmccart Exp $) with ESMTP id gBI3iGo05888 for ; Wed, 18 Dec 2002 03:44:16 GMT Received: from fmsmsxvs041.fm.intel.com (fmsmsxvs041.fm.intel.com [132.233.42.126]) by petasus.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.27 2002/10/16 23:46:59 dmccart Exp $) with SMTP id gBI3fuU22045 for ; Wed, 18 Dec 2002 03:41:56 GMT Received: from FMSMSX017.fm.intel.com ([132.233.42.196]) by fmsmsxvs041.fm.intel.com (NAVGW 2.5.2.11) with SMTP id M2002121719444010328 ; Tue, 17 Dec 2002 19:44:40 -0800 Received: by fmsmsx017.fm.intel.com with Internet Mail Service (5.5.2653.19) id ; Tue, 17 Dec 2002 19:46:28 -0800 Message-ID: <288F9BF66CD9D5118DF400508B68C44604758F76@orsmsx113.jf.intel.com> From: "Feldman, Scott" To: "'Anton Blanchard'" Cc: linux-kernel@vger.kernel.org, netdev@oss.sgi.com, LOSTeam Subject: RE: [ANNOUNCE] Intel PRO/100 software developer manual released Date: Tue, 17 Dec 2002 19:46:27 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain X-archive-position: 1392 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: scott.feldman@intel.com Precedence: bulk X-list: netdev > Nice work guys! This has already answered a few questions I > had when tuning the e100 on ppc64. Great! Let us know if you figure out that checksum weirdness you saw the other day. > Any plans to release similar docs for the e1000? It's a logical next step, but no definitive schedule. Let's just leave it as a goal. -scott From greg@kroah.com Tue Dec 17 22:32:36 2002 Received: with ECARTIS (v1.0.0; list netdev); Tue, 17 Dec 2002 22:32:37 -0800 (PST) Received: from kroah.com (12-231-249-244.client.attbi.com [12.231.249.244]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBI6WX3v018168 for ; Tue, 17 Dec 2002 22:32:35 -0800 Received: (qmail 29030 invoked by uid 500); 18 Dec 2002 06:34:05 -0000 Date: Tue, 17 Dec 2002 22:34:05 -0800 From: Greg KH To: "Feldman, Scott" Cc: linux-kernel@vger.kernel.org, netdev@oss.sgi.com, LOSTeam Subject: Re: [ANNOUNCE] Intel PRO/100 software developer manual released Message-ID: <20021218063405.GP28629@kroah.com> References: <288F9BF66CD9D5118DF400508B68C44604758F6C@orsmsx113.jf.intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <288F9BF66CD9D5118DF400508B68C44604758F6C@orsmsx113.jf.intel.com> User-Agent: Mutt/1.4i X-archive-position: 1394 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: greg@kroah.com Precedence: bulk X-list: netdev Content-Length: 286 Lines: 11 On Tue, Dec 17, 2002 at 02:03:50PM -0800, Feldman, Scott wrote: > > Available at https://sourceforge.net/projects/e1000. Yeah! Thank you for doing this, I know it must not have been an easy thing to accomplish. Hopefully, Linux will be better off because of it. thanks, greg k-h From jmorris@intercode.com.au Wed Dec 18 00:30:12 2002 Received: with ECARTIS (v1.0.0; list netdev); Wed, 18 Dec 2002 00:30:36 -0800 (PST) Received: from blackbird.intercode.com.au (IDENT:root@blackbird.intercode.com.au [203.32.101.10]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBI8U83v020869 for ; Wed, 18 Dec 2002 00:30:10 -0800 Received: from localhost (jmorris@localhost) by blackbird.intercode.com.au (8.9.3/8.9.3) with ESMTP id TAA16543; Wed, 18 Dec 2002 19:34:06 +1100 Date: Wed, 18 Dec 2002 19:34:06 +1100 (EST) From: James Morris To: "Feldman, Scott" cc: linux-kernel@vger.kernel.org, , LOSTeam Subject: Re: [ANNOUNCE] Intel PRO/100 software developer manual released In-Reply-To: <288F9BF66CD9D5118DF400508B68C44604758F6C@orsmsx113.jf.intel.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 1395 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jmorris@intercode.com.au Precedence: bulk X-list: netdev Content-Length: 649 Lines: 21 On Tue, 17 Dec 2002, Feldman, Scott wrote: > The manual is intended to support the maintenance of the e100 driver (or the > best driver for the PRO/100 networking hardware ;-). The manual covers the > 82557, 82558, 82559, 82550, and 82551 Ethernet controllers. Hi Scott, This looks great. Any chance of releasing documentation on the crypto hardware which resides on the PRO/100 S? We're currently looking at adding support for hardware crypto (including IPsec offload) to the kernel, and it would be good to include the requirements and capabilities of this card in the design process. - James -- James Morris From scott.feldman@intel.com Wed Dec 18 09:03:59 2002 Received: with ECARTIS (v1.0.0; list netdev); Wed, 18 Dec 2002 09:04:01 -0800 (PST) Received: from caduceus.jf.intel.com (fmr06.intel.com [134.134.136.7]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBIH3w3v029508 for ; Wed, 18 Dec 2002 09:03:59 -0800 Received: from talaria.jf.intel.com (talaria.jf.intel.com [10.7.209.7]) by caduceus.jf.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.51 2002/09/23 20:43:23 dmccart Exp $) with ESMTP id gBIH4m306435 for ; Wed, 18 Dec 2002 17:04:48 GMT Received: from orsmsxvs040.jf.intel.com (orsmsxvs040.jf.intel.com [192.168.65.206]) by talaria.jf.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.27 2002/10/16 23:46:59 dmccart Exp $) with SMTP id gBIGq4R01569 for ; Wed, 18 Dec 2002 16:52:04 GMT Received: from orsmsx26.jf.intel.com ([192.168.65.26]) by orsmsxvs040.jf.intel.com (NAVGW 2.5.2.11) with SMTP id M2002121809132206376 ; Wed, 18 Dec 2002 09:13:22 -0800 Received: by orsmsx26.jf.intel.com with Internet Mail Service (5.5.2653.19) id ; Wed, 18 Dec 2002 09:08:02 -0800 Message-ID: <288F9BF66CD9D5118DF400508B68C44604758F7C@orsmsx113.jf.intel.com> From: "Feldman, Scott" To: "'linux-kernel@vger.kernel.org'" , "'netdev@oss.sgi.com'" Cc: LOSTeam Subject: RE: [ANNOUNCE] Intel PRO/100 software developer manual released Date: Wed, 18 Dec 2002 09:07:59 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain X-archive-position: 1396 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: scott.feldman@intel.com Precedence: bulk X-list: netdev Content-Length: 343 Lines: 10 > I would also like to thank the Intel editors and reviewers: > Carolyn Abrigana, Larry Bates, Julie Donnelly, John Ronciak, > Wen-Hwa Tao, Eli Kupermann, David Valdez, Colleen Culbertson, > and especially Glenn Begis for not giving up. Credit is important, and I unintentionally omitted Shaun Sloan. Thanks Shaun for your help. -scott From scott.feldman@intel.com Wed Dec 18 14:37:02 2002 Received: with ECARTIS (v1.0.0; list netdev); Wed, 18 Dec 2002 14:37:05 -0800 (PST) Received: from hermes.fm.intel.com (fmr01.intel.com [192.55.52.18]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBIMb13v012535 for ; Wed, 18 Dec 2002 14:37:02 -0800 Received: from talaria.fm.intel.com (talaria.fm.intel.com [10.1.192.39]) by hermes.fm.intel.com (8.11.6/8.11.6/d: outer.mc,v 1.51 2002/09/23 20:43:23 dmccart Exp $) with ESMTP id gBIMcxU02478 for ; Wed, 18 Dec 2002 22:38:59 GMT Received: from fmsmsxv040-1.fm.intel.com (fmsmsxvs040.fm.intel.com [132.233.42.124]) by talaria.fm.intel.com (8.11.6/8.11.6/d: inner.mc,v 1.27 2002/10/16 23:46:59 dmccart Exp $) with SMTP id gBIMhUp13179 for ; Wed, 18 Dec 2002 22:43:30 GMT Received: from FMSMSX016.fm.intel.com ([132.233.42.195]) by fmsmsxv040-1.fm.intel.com (NAVGW 2.5.2.11) with SMTP id M2002121814413404925 ; Wed, 18 Dec 2002 14:41:34 -0800 Received: by fmsmsx016.fm.intel.com with Internet Mail Service (5.5.2653.19) id ; Wed, 18 Dec 2002 14:41:11 -0800 Message-ID: <288F9BF66CD9D5118DF400508B68C44604758F86@orsmsx113.jf.intel.com> From: "Feldman, Scott" To: "'James Morris'" Cc: linux-kernel@vger.kernel.org, netdev@oss.sgi.com, LOSTeam Subject: RE: [ANNOUNCE] Intel PRO/100 software developer manual released Date: Wed, 18 Dec 2002 14:41:07 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain X-archive-position: 1397 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: scott.feldman@intel.com Precedence: bulk X-list: netdev Content-Length: 125 Lines: 7 > Any chance of releasing documentation on > the crypto hardware which resides on the PRO/100 S? No plans, sorry. -scott From sri@us.ibm.com Wed Dec 18 15:06:49 2002 Received: with ECARTIS (v1.0.0; list netdev); Wed, 18 Dec 2002 15:06:52 -0800 (PST) Received: from e6.ny.us.ibm.com (e6.ny.us.ibm.com [32.97.182.106]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBIN6l3v014046 for ; Wed, 18 Dec 2002 15:06:48 -0800 Received: from northrelay04.pok.ibm.com (northrelay04.pok.ibm.com [9.56.224.206]) by e6.ny.us.ibm.com (8.12.2/8.12.2) with ESMTP id gBINArYA119370 for ; Wed, 18 Dec 2002 18:10:53 -0500 Received: from dyn9-47-18-86.beaverton.ibm.com (dyn9-47-18-86.beaverton.ibm.com [9.47.18.86]) by northrelay04.pok.ibm.com (8.12.3/NCO/VER6.4) with ESMTP id gBINApgC122444 for ; Wed, 18 Dec 2002 18:10:51 -0500 Date: Wed, 18 Dec 2002 15:12:56 -0800 (PST) From: Sridhar Samudrala X-X-Sender: To: Subject: ip6_route_output() does not fill in the src. addr. Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-archive-position: 1398 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: sri@us.ibm.com Precedence: bulk X-list: netdev Content-Length: 491 Lines: 14 The ipv6 version of the 'get route entry' function ip6_route_output() does not fill in the source address of the route even when a valid appropriate source address is passed. Is this the expected behavior or a bug which is going to be fixed in the future releases of 2.5? The ipv4 version 'ip_route_output_key()' works as expected. SCTP calls these routines to get and cache a route entry and uses the cached entry to fill in skb->dst before passing the skb to ip. Thanks Sridhar From aravind1001@speedpost.net Wed Dec 18 20:32:20 2002 Received: with ECARTIS (v1.0.0; list netdev); Wed, 18 Dec 2002 20:32:22 -0800 (PST) Received: from server2.fastmail.fm (ny2.fastmail.fm [66.111.4.3]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBJ4WJ3v028239 for ; Wed, 18 Dec 2002 20:32:20 -0800 Received: from server2.fastmail.fm (localhost [127.0.0.1]) by fastmail.fm (Postfix) with ESMTP id 8B8D83123E for ; Wed, 18 Dec 2002 23:36:28 -0500 (EST) Received: from 127.0.0.1 ([127.0.0.1] helo=server2.fastmail.fm) by fastmail.fm with SMTP; Wed, 18 Dec 2002 23:36:28 -0500 Received: by server2.fastmail.fm (Postfix, from userid 99) id 876C33EDEC; Wed, 18 Dec 2002 23:36:28 -0500 (EST) Content-Disposition: inline Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="ISO-8859-1" MIME-Version: 1.0 X-Mailer: MIME::Lite 1.2 (F2.71; T1.001; A1.51; B2.12; Q2.03) From: "Aravind Ceyardass" To: netdev@oss.sgi.com Date: Thu, 19 Dec 2002 00:36:28 -0400 X-Epoch: 1040272588 X-Sasl-enc: UHvlpySUNzmJfhjdNUO+zA Subject: IPv6 Programming Message-Id: <20021219043628.876C33EDEC@server2.fastmail.fm> X-archive-position: 1399 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: aravind1001@speedpost.net Precedence: bulk X-list: netdev Hi, New to IPv6 on Linux. I would like to know which version of Linux has got the header files for IPv6 programming? Thanks in advance. Aravind -- http://fastmail.fm - Accessible with your email software or over the web From yoshfuji@linux-ipv6.org Wed Dec 18 20:37:37 2002 Received: with ECARTIS (v1.0.0; list netdev); Wed, 18 Dec 2002 20:37:38 -0800 (PST) Received: from yue.hongo.wide.ad.jp (yue.hongo.wide.ad.jp [203.178.139.94]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBJ4bZ3v028751 for ; Wed, 18 Dec 2002 20:37:36 -0800 Received: from localhost (localhost [127.0.0.1]) by yue.hongo.wide.ad.jp (8.12.3+3.5Wbeta/8.12.3/Debian -4) with ESMTP id gBJ4fjGR012244; Thu, 19 Dec 2002 13:41:51 +0900 Date: Thu, 19 Dec 2002 13:41:45 +0900 (JST) Message-Id: <20021219.134145.53383055.yoshfuji@linux-ipv6.org> To: aravind1001@speedpost.net Cc: netdev@oss.sgi.com Subject: Re: IPv6 Programming From: YOSHIFUJI Hideaki / =?iso-2022-jp?B?GyRCNUhGIzFRTEAbKEI=?= In-Reply-To: <20021219043628.876C33EDEC@server2.fastmail.fm> References: <20021219043628.876C33EDEC@server2.fastmail.fm> Organization: USAGI Project X-URL: http://www.yoshifuji.org/%7Ehideaki/ X-Fingerprint: 90 22 65 EB 1E CF 3A D1 0B DF 80 D8 48 07 F8 94 E0 62 0E EA X-PGP-Key-URL: http://www.yoshifuji.org/%7Ehideaki/hideaki@yoshifuji.org.asc X-Face: "5$Al-.M>NJ%a'@hhZdQm:."qn~PA^gq4o*>iCFToq*bAi#4FRtx}enhuQKz7fNqQz\BYU] $~O_5m-9'}MIs`XGwIEscw;e5b>n"B_?j/AkL~i/MEaZBLP X-Mailer: Mew version 2.2 on Emacs 20.7 / Mule 4.1 (AOI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-archive-position: 1400 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: yoshfuji@linux-ipv6.org Precedence: bulk X-list: netdev In article <20021219043628.876C33EDEC@server2.fastmail.fm> (at Thu, 19 Dec 2002 00:36:28 -0400), "Aravind Ceyardass" says: > New to IPv6 on Linux. I would like to know which version of Linux has got > the header files for > IPv6 programming? It depends on what you mean by "IPv6 programming," but anyway, glibc 2.1.3 or later, preferably, glibc 2.2.x or later. --yoshfuji From abcd@e-mail.nwx.com.br Thu Dec 19 04:00:40 2002 Received: with ECARTIS (v1.0.0; list netdev); Thu, 19 Dec 2002 04:00:46 -0800 (PST) Received: from e-mail.nwx.com.br (iplus-fac-244.xdsl-dinamico-nat.ctbcnetsuper.com.br [200.225.214.244]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBJC0V3v008830 for ; Thu, 19 Dec 2002 04:00:38 -0800 Message-Id: <200212191200.gBJC0V3v008830@oss.sgi.com> From: "Adauto" To: Subject: Mailing Canada Mime-Version: 1.0 Content-Type: text/html; charset="ISO-8859-1" Date: Thu, 19 Dec 2002 10:04:57 -0200 Reply-To: "Adauto" Content-Transfer-Encoding: 8bit X-archive-position: 1401 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: abcd@e-mail.nwx.com.br Precedence: bulk X-list: netdev Olá, Você conhece alguém que vende cadastro de e-mails da Argentina ou México ou Canadá ou outras partes do mundo para fazer mala direta?
Caso não conhece não é necessário responder essa mensagem.

Obrigado Adauto

e-mail e-mail@e-mail.nwx.com.br

Hi, Do you know who sells a mailing list to direct marketing from Argentina,Mexico and canada? Or from others parts os the world?
If you do not know,is not necessary to answer this message.

Thank you Adauto

e-mail e-mail@e-mail.nwx.com.br From rockshows@rockshows.com.br Thu Dec 19 19:54:29 2002 Received: with ECARTIS (v1.0.0; list netdev); Thu, 19 Dec 2002 19:54:35 -0800 (PST) Received: from rockshows.com.br (200227204161-dial-user-BOL.acessonet.com.br [200.227.204.161]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBK3sO3v009519 for ; Thu, 19 Dec 2002 19:54:27 -0800 Message-Id: <200212200354.gBK3sO3v009519@oss.sgi.com> From: "Rock Shows Produções Artísticas" To: Subject: =?ISO-8859-1?Q?Lan=E7amento?= do CD da Fator Cinco Mime-Version: 1.0 Content-Type: text/html; charset="ISO-8859-1" Date: Fri, 20 Dec 2002 01:59:44 -0200 Content-Transfer-Encoding: 8bit X-archive-position: 1402 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: rockshows@rockshows.com.br Precedence: bulk X-list: netdev

Grande show de lançamento do CD da banda Fator Cinco no DirecTV - Music Hall em Fevereiro de 2003,
clique aqui e saiba como adquirir o seu ingresso gratuitamente e conhecer esta nova revelação do rock nacional.

 

Caso não queira mais receber nossos informativos, Click aqui !


 From todoarg@uol.com.ar Thu Dec 19 22:47:15 2002 Received: with ECARTIS (v1.0.0; list netdev); Thu, 19 Dec 2002 22:47:20 -0800 (PST) Received: from company.mail ([209.99.231.135]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBK6l73v010920 for ; Thu, 19 Dec 2002 22:47:10 -0800 Received: from uol.com.ar [127.0.0.1] by company.mail [169.254.245.246] with SMTP (MDaemon.PRO.v5.0.1.T) for ; Fri, 06 Sep 2002 02:19:10 -0300 Message-ID: <123114-220029565199641@uol.com.ar> X-EM-Version: 6, 0, 0, 4 X-EM-Registration: #00F06206106618006920 X-Priority: 3 Reply-To: datacenter@uol.com.ar From: "DataCenter" To: netdev@oss.sgi.com Subject: Mejor Imposible... Date: Fri, 6 Sep 2002 02:19:09 -0300 MIME-Version: 1.0 Content-type: text/plain; charset="US-ASCII" X-MDRemoteIP: 127.0.0.1 X-Return-Path: todoarg@uol.com.ar X-MDaemon-Deliver-To: netdev@oss.sgi.com Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by oss.sgi.com id gBK6l73v010920 X-archive-position: 1403 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: todoarg@uol.com.ar Precedence: bulk X-list: netdev HOLA: Soy, Gaston Roviralta y les ofrezco los mejores 20 Programas del mundo. estos programas fueron los mas vedidos de Argentina y del resto del mundo *AUREATE: ¿ Publicidad por mail? ¿ Necesitas un programa para envió masivo de mensajes? Acá tenes la solución AUREATE GROUP MAIL PRO 3.3 Versión FULL Registrada, con Código de Registro y Plugins... Este programa te permite entre otras cosas: *ENVIAR MENSAJES EN FORMA MASIVA. * ELIMINAR AUTOMÁTICAMENTE LAS DIRECCIONES REBOTADAS. * CHEQUEADOS DE DIRECCIONES. * REMOVER USUARIOS AUTOMÁTICAMENTE * CREAR MENSAJES HTML: con la mejor tecnología ETC... ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- *BASES DE E-MAIL / GUÍAS DE E-MAIL DE TODO EL MUNDO: BASE DE DATOS DE EMAILS ARGENTINOS MAS DE 1.000.000 DE DIRECCIONES PARTICULARES, DE EMPRESAS, PYMES, PROFESIONALES, Y MUCHOS OTROS RUBROS. COMO POR EJEMPLO: ESTA DIVIDIDA POR CONTINENTES, PAÍSES, (ARGENTINA EN CAPITAL, GRAN BUENOS AIRES, PROVINCIAS Y LOCALIDADES. PROGRAMAS DE ENVIÓ MASIVO DE MENSAJES, PARA PODER UTILIZAR ESTAS BASES DE DATOS. ESPECTACULAR PARA PODER EMPEZAR TU PROPIO NEGOCIO CHEQUEADAS AL 15-11 VERSIÓN FULL DE TODOS LOS PROGRAMAS CON CRACK INCLUIDOS. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- *COMBO XP CONTIENE LOS MEJORES PROGRAMAS DE MICROSOFT WINDOWS XP Y OFFICE XP. La ultima version del Sistema Operativo de Microsoft: Windows XP Profesional en castellano, es un CD con el sistema completo, un diez de calificación para este producto, que tiene mas de 25 nuevas utilidades, y el resto totalmente actualizados. La ultima versión del Paquete de Programas Office: se trata del Office XP 2002 en Castellano, versión profesional, son tres CDS, con la ultima tecnología y modificaciones para este producto. y la versión 2002 en español de la Enciclopedia Encarta junto con el atlas, que en total son otros cuatro CDS... ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- *GUÍA TELEFÓNICA 2002 / DATEL 2002: Nueva Guía Telefónica 2002 en CD ( BUSCA POR NOMBRE, TELÉFONO Y DOMICILIO !!! ) + Nueva Guía de Cod Postales Alfanuméricos 2001. CONTENIDO DEL CD1: Guía Telefónica Invertida con todos los abonados del país de telefónica y Telecom, busca por nombre, teléfono o dirección. Guía Comercial (para realizar búsquedas por empresas, rubro, provincia o localidad, con variables filtros definidos por el usuario: Ej., que tenga e-mail, que tenga URL, que tenga aviso; cuyo resultado podrá ser visualizado en formato de fichas o tablas. Guía Profesional, la más completa de las guías de profesionales del país, podrá realizar búsquedas por profesión: Ej., si UD. desea buscar un abogado, le dará datos de todos los que se encuentren en la zona que UD delimite, o también buscar por apellido y nombre. Guía de Organismos Oficiales. CONTENIDO 2: Nueva guía con los códigos alfanuméricos de toda la Republica Argentina, por ejemplo: el Cód. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- GRAN COLECCIÓN 10 LIBROS SOBRE HACKING a un precio increible:EN 2 CDS. Son 10 CDS que traen de todo!!! LIBRO 1: Hackers 2000 la ultima tecnología en programas de Hacking, los mejores virus, troyanos, programas para crackear software, programas para sacar passwords con sus correspondientes listas de posibles contraseñas, utilidades para icq, para obtener direcciones ip, y muchas pero muchas otras utilidades relacionadas con el hacking. LIBRO 2: Exelsior: Hacking en linux: las ultimas herramientas y los mejores manuales que te enseñaran todos los secretos para poder hackear bajo el sistema operativo Linux. LIBRO 3: Master Hacker: Siguiendo la misma temática que el CD 1, este CD aporta otra gran colección de programas para muchísimas cosas. Excelente. LIBRO 4 y 5: Dr. HACKER and MR. CRACK: Los grandes secretos del hacking y cracking con manuales escritos y programas realizados por dos hackers muy importantes del mundo. LIBRO 6 Este CD contiene muchisimas herramientas para el hacking: TROYANOS: BackOriffice120, bo2k (BackOriffice2000), Netbus160, Netbus170, NetbusPro2, WinSatan, SubSeven y otros, Sacadores de password para los servidores. ANTITROYANOS, Virus (una gran colección), Creadores de VIRUS (versiones en castellanos de estos generadores de Virus), BOMBAS e-mail, la mas grande colección de NUCKES, SCANNEADORES (una gran cantidad de scanneadores de IP, de puertos y MODEMS), CRACKS, Sacadores de password de todo tipo: Para paginas WEB, Para programas como: Word, Excel, Access y otros para obtener las claves de conexión a Internet o redes locales. Muchos programas para telefonía; como llamar de teléfonos gratuitamente, mensajes a celulares y carga de pulsos de estos. Muchos programas para la Administración Remota. LO MEJOR DE ESTE CD ES QUE TIENE UNA GRAN CANTIDAD DE MANUALES Y TUTORES EN CASTELLANO, INCLUSO MUCHOS DE LOS PROGRAMAS ESTÁN EN CASTELLANO (GENERADORES DE VIRUS, TROYANOS COMO EL BACK ORÍFICE Y EL NETBUS TAMBIÉN ESTA EN CASTELLANO). ESTE CD FUE CREADO POR UN GRUPO DE SEGUIDORES DEL CONOCIDO HACKER VeIrUS (DE HABLA HISPANA). LIBRO 7 (continuación del CD 1) Este CD contiene muchísimas herramientas para el hacking: TROYANOS: BackOriffice120, bo2k (BackOriffice2000), Netbus160, Netbus170, NetbusPro2, WinSatan, SubSeven y otros, Sacadores de password para los servidores. ANTITROYANOS, Virus (una gran coleccion), Creadores de VIRUS (versiones en castellanos de estos generadores de Virus), BOMBAS e-mail, la mas grande colección de NUCKES, SCANNEADORES (una gran cantidad de scanneadores de IP, de puertos y MODEMS), CRACKS, Sacadores de password de todo tipo: Para paginas WEB, Para programas como: Word, Excel, Access y otros para obtener las claves de conexión a Internet o redes locales. Muchos programas para telefonía; como llamar de teléfonos gratuitamente, mensajes a celulares y carga de pulsos de estos. LIBRO 8 Hackers Handbook este CD contiene las herramientas y técnicas del arte de hacking(hackear). Herramientas para hackear (Hacking Tools) Mas de 800 Herramientas creadas y usadas por los mejores hackers del mundo para explotar todo tipo de seguridad de computación incluyendo: Password Crackers, Trojan horse wrappers, unix y linux scripts, windows 95/98 remote hacking, nucking, hacks para icq, Windows nt crackers admin, laboratorio de virus y mucho mas. Herramientas de Phreaking. Herramientas de Phreaking que realmente funcionan incluyendo emuladores de red box y plans, generadores de tonos para controlar líneas telefónicas, líen Tes. devices y mucho mas.. Librería de Hacking y Phreaking. La mas completa librería jamás compilada, no son las mismas filas de 10 anos de antigüedad que se venden en otros CDS, estas filas son todas nuevas y explican las verdaderas técnicas usadas por los hackers reales que aparecen en las noticias, incluyendo:ip spoofing(cambia el numero de ip para que nadie sepa quien SOS realmente), hacking icq(tecnicas para hackear el programa icq)y hacks para remote win que no vas a creer, plus exploiting browsers, cracks de web pages y Web Server, escaneo de ports, y miles de cosas mas. El contenido de este CD-rom es de propósitos de información y material difícil de conseguir para estudio o educación propia el contenido de este CD-rom no viola ninguna ley pero el uso de alguno de estos programas si , por eso se ruega que el uso de este CD-rom sea conciente, el vendedor no se responsabiliza de ninguna forma por el mal uso de la información dada en este cumuchos programas para la Administración Remota LIBRO 9 CD que contiene 420 Mb. de pura información, programas, herramientas ... De nuestros amigos Españoles, Mexicanos y de hache ARGENTINA ... La mayoria de su material se encuentra en español ... EL ultimo CD ... recién compilado ... complementa todo lo que le falta al Veirus - y lo actualiza - ... un Cd no solo para aprender sino para ... LIBRO 10 ATENCION PROGRAMADORES, WEBMASTER, MANAGER O CEO "200 DE LOS MEJORES CURSOS, MANUALES Y MONOGRAFÍAS " "EL SABER NO OCUPA LUGAR" YO ME PREGUNTO SI SE INSPIRO EN NUETRO PACK YA QUE EN UN CD HEMOS COMPILADO 200 DE LOS MEJORES CURSOS, MANUALES Y MONOGRAFÍAS; QUE DE OTRA FORMA SERIA IMPOSIBLE COMERCIALIZAR POR SU TAMAÑO Y COSTO. ENTRE LOS TEMAS MAS IMPORTANTES PODES ENCONTRAR: NT, LINUX, JAVA, JSCRIPT, ASP, CGI, HTML, MARKETING, FIREWALL, VISUAL BASIC, VISUAL FOX, INTERNET, UNIX, MACROMEDIA, PERL, C++, SQL, TCP/IP, FRONT PAGE, ORACLE, ENTRE OTROS. NIVELES BÁSICOS, INTERMEDIOS Y AVANZADOS. MAS DE 40.000 HOJAS LOS ARCHIVOS ESTÁN EN FORMATO HTML, PDF Y DOC E INCLUIMOS UTILIDADES Textos en Castellano ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- • NORTON ANTIVIRUS 2002 La mas actualizada versión del mas potente y conocido Antivirus, totalmente funcional bajo Windows Xp. Trae mas de 125 herramientas que mantendran a su equipo junto con la tecnología XP. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- • GLOBAL LINK TRASLATOR PRO 7.0 La ultima versión del mejor traductor para archivos y frases que existan en el mercado. Traduce 6 idiomas: Ingles, Español, Francés, Italiano, Alemán y Portugués. Compatible con Windows y Office XP. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- *JUEGOS / ESTRATEGIA: TITULOS: * FIFA 2001 (FÚTBOL MUNDIAL Y LOCAL DE ALTO NIVEL) * MIDTOWN MADNEES 2 (LAS MEJORES CARRERAS DE AUTOS EN LAS CALLES DE CALIFORNIA, NEW YORK, LONDRES ETC...) * SIM CITY 3000 CASTELLANO ( ES UN JUEGO DE ESTRATEGIA QUE TRATA DE GOBERNAR UN PAIS EN LOS TIEMPOS QUE CORREN) * THE SIMS (JUEGO DE ESTRATEGIA: ES PODER ORGANIZAR UNA FAMILIA Y COMO LLEVARLA A DELANTE) * TOMB RAIDER 4 (JUEGO DE GUERRA Y ARMAS NUCLEARES) * TOMB RAIDER 5 (VERCION MAS AVANZADA DEL TOMB RAIDER CUATRO) * PC FÚTBOL 7 (LOS MEJORES CLUBS DEL PAÍS COMO BOCA, RIVER, VELEZ. PODER LLEGAR A GANAR LA COPA APERTURA Y CLAUSURA) * NEED FOR SPEED 5 (COCHES IMPORTADOS JUEGAN LA CARRERA MAS IMPORTANTE DE TODO LOS TIEMPOS CON FERRARRI, LANBORGINI, ALFA RONEO, JAGUAR Y PORCHE.. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- *PARA ADULTOS LO MEJOR DE KAMASUTRA: Si te interesa el Libro de Kamasutra, entonces tenes que tener este CD. que muestra las mejores poses sexuales en una forma interactiva. Tenes lo mismo que el libro pero en CD. así que si te interesa, solo tense que encargarlo. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- *PADRÓN ELECTORAL: año 1999 incluye hasta la categoría 1980. Todos los ciudadanos de la Republica Argentina reunidos en estos tres CDS. Separados por provincias e importado a archivos de MICROSOFT ACCESS para su mejor utilización. Podes buscar por nombre, apellido o DNI, podes filtra por nombre, apellido, clase, DNI, lugar de votación, dirección, mesa, etc. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- *"PADRÓN DE OBRAS SOCIALES" Estos 2 CDS contienen el registro nacional de todas las obras sociales, y sus correspondientes afiliados. Los datos que incluyen, entre otros, son: nombre y apellido, dirección, DNI, obra social, y otros datos personales de interés. En este caso los datos también pueden exportarse o imprimirse. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- *"REGISTRO DEL AUTOMOTOR" Base de datos del Registro Del Automotor Nacional, que incluye información sobre cada persona que posea vehículos, así como datos sobre el vehículo en si. Además de otros datos de interés. nombre y apellido, dirección, DNI, obra social, y otros datos personales de interés. En este caso los datos también pueden exportarse o imprimirse. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- *"GUÍA COMERCIAL MERCOSUR" Este CD contiene todos los datos de las empresas y negocios que forman parte del MERCOSUR, con posibilidad de filtrar los resultados por país, región, provincia, localidad, rubro, etc. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- *"GUÍA DE LA INDUSTRIA 2002" Este CD contiene los datos de todas las industrias nacionales con posibilidad de buscar por: Razón Social, nombre, dirección, rubro, localidad, etc. *"PAGINAS AMARILLAS 2002" Este CD contiene los datos de todas las empresas nacionales, además de la guía completa de Profesionales y guía telefónica de todo el país. Permite buscar por: razón social, nombre, dirección, teléfono, etc. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- "GUÍA DE TELÉFONO INVERTIDA 2000" Con llave de registro. Este CD contiene la guía telefónica Nacional. Permite buscar por nombre, apellido, dirección parcial, completa, todo un edificio, cuadra par, impar, por nº de teléfono entre otros. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- *PLANO DE CALLES INTELIGENTE: * El CD contiene un plano de calles de Gran Bs.As., La Plata, Ensenada, Beriso y Capital Federal. * Este plano permite buscar direcciones completas, aproximadas, calles, avenidas, lugares, entrecalles, etc. * Además contiene una guía de Códigos Postales, con un buscador que permite encontrar un cod. postal por dirección, localidad, etc. * El plano muestra el resultado y se lo puede agrandar hasta un 600% mas de lo normal, Muestra también todos los nombres de las calles con sus correspondientes alturas. * Permite guardar las búsquedas ya realizadas para luego poder consultarlas. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- *TÉCNICAS Web: ¿NECESITAS HERRAMIENTAS PARA TU Web? ¿ IDEAS ? ¿SCRIPTS? ENTONCES ESTE CD TE VA A INTERESA DEMASIADO. UN CD COMPLETO CON MAS DE 2.000 HERRAMIENTAS PARA LOS WEBMASTERS, HERRAMIENTAS DE AUDIO, video, IMAGEN, INTERNET, Y MUCHO MAS. TENSE SCRIPTS .ASP / .PHP LIBROS DE VISITAS ADMINISTRACIÓN DE BD CONTADORES ONLINE ENCUESTAS FOROS FORMULARIOS PASSWORD Y MUCHOS PERO MUCHOS OTROS SCRIPTS. TENES TAMBIÉN IMÁGENES DE PRUEBA, SONIDOS Y MUCHAS COSAS MAS PARA UTILIZAR EN TU PAGINA Web. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- CADA PRODUCTO CAPITAL : CUESTA $15 PARA LA PROVINCIAS: CUESTA $25 *EL HACKING Y EL COMBO XP CUESTA CAPITAL $25 PARA LA PROVINCIAS:CUESTA $37 !!!!ATENCIÓN... CAPITAL FEDERAL Y GRAN BUENOS AIRES, CADA COMPRA QUE REALICE LES DESCONTAMOS UN %50 DE SERVICIE DE REPARACIÓN DE PC. IMPORTANTE: SOBRE OTRO NOMBRE DE PROGRAMA QUE LE INTERESE, DEBE DECIRNOS EL NOMBRE Y SE LO ENVIAMOS EN 24 HORAS. PARA COMPRARLO SOLO TENES QUE RESPONDERME ESTE MENSAJE ESPECIFICANDO: NOMBRE COMPLETO DEL DESTINATARIO DIRECCIÓN COMPLETA (CALLE - N° - PISO/Dpto. - CP - LOCALIDAD - PCIA.) TELÉFONO (MAYORMENTE PARA CONFIRMAR LOS DATOS, NO IMPORTA QUE SEAN CELULARES O TELÉFONOS DEL INTERIOR.) ATENCIÓN CAPITAL FEDERAL!!! ACEPTAMOS PESOS - DÓLARES - PATACONES- LECOPS (SOLO VALIDO PARA CAPITAL FEDERAL PORQUE SE ENTREGA POR MENSAJERO, EL RESTO DE LAS LOCALIDADES SE ENVÍA POR CORREO ARGENTINO Y NO ACEPTAN ESAS FORMAS DE PAGO POR EL MOMENTO.) ESTA OFERTA NO SE VOLVERÁ A REPETIR. EN CASO DE QUE ESTE EMAIL TE MOLESTE, TE PIDO DISCULPAS, Y NO HACE FALTA QUE TE BORRES DE NINGUNA LISTA PORQUE NO RECIBIRÁS OTRA OFERTA SOBRE ESTE MISMO PRODUCTO. LOS EMAILS LOS ENVIÓ EN FORMATO TEXTO PARA HACERTE MAS RÁPIDA LA DESCARGA. Por sección 301, párrafo (a)(2)(C) de S.1618. Bajo el decreto S.1618 titulo 3ro. Aprobado por el 105 congreso base de las normativas Internacionales sobre SPAM, un E-mail no podrá ser considerado SPAM mientras incluya una forma de ser removido.Si usted desea ser removido de nuestra base de datos en forma definitiva por favor responda a este e-mail indicando "Remover" en el subject. From todoarg@uol.com.ar Fri Dec 20 21:23:46 2002 Received: with ECARTIS (v1.0.0; list netdev); Fri, 20 Dec 2002 21:23:51 -0800 (PST) Received: from company.mail ([209.99.231.135]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBL5Ne3v024943 for ; Fri, 20 Dec 2002 21:23:42 -0800 Received: from uol.com.ar [127.0.0.1] by company.mail [169.254.245.246] with SMTP (MDaemon.PRO.v5.0.1.T) for ; Sat, 07 Sep 2002 01:30:42 -0300 Message-ID: <397514-2200296743041276@uol.com.ar> X-EM-Version: 6, 0, 0, 4 X-EM-Registration: #00F06206106618006920 X-Priority: 3 Reply-To: datacenter@uol.com.ar From: "DataCenter" To: netdev@oss.sgi.com Subject: Mejor Imposible... Date: Sat, 7 Sep 2002 01:30:41 -0300 MIME-Version: 1.0 Content-type: text/plain; charset="US-ASCII" X-MDRemoteIP: 127.0.0.1 X-Return-Path: todoarg@uol.com.ar X-MDaemon-Deliver-To: netdev@oss.sgi.com Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by oss.sgi.com id gBL5Ne3v024943 X-archive-position: 1404 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: todoarg@uol.com.ar Precedence: bulk X-list: netdev HOLA: Soy, Gaston Roviralta y les ofrezco los mejores 20 Programas del mundo. estos programas fueron los mas vedidos de Argentina y del resto del mundo *AUREATE: ¿ Publicidad por mail? ¿ Necesitas un programa para envió masivo de mensajes? Acá tenes la solución AUREATE GROUP MAIL PRO 3.3 Versión FULL Registrada, con Código de Registro y Plugins... Este programa te permite entre otras cosas: *ENVIAR MENSAJES EN FORMA MASIVA. * ELIMINAR AUTOMÁTICAMENTE LAS DIRECCIONES REBOTADAS. * CHEQUEADOS DE DIRECCIONES. * REMOVER USUARIOS AUTOMÁTICAMENTE * CREAR MENSAJES HTML: con la mejor tecnología ETC... ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- *BASES DE E-MAIL / GUÍAS DE E-MAIL DE TODO EL MUNDO: BASE DE DATOS DE EMAILS ARGENTINOS MAS DE 1.000.000 DE DIRECCIONES PARTICULARES, DE EMPRESAS, PYMES, PROFESIONALES, Y MUCHOS OTROS RUBROS. COMO POR EJEMPLO: ESTA DIVIDIDA POR CONTINENTES, PAÍSES, (ARGENTINA EN CAPITAL, GRAN BUENOS AIRES, PROVINCIAS Y LOCALIDADES. PROGRAMAS DE ENVIÓ MASIVO DE MENSAJES, PARA PODER UTILIZAR ESTAS BASES DE DATOS. ESPECTACULAR PARA PODER EMPEZAR TU PROPIO NEGOCIO CHEQUEADAS AL 15-11 VERSIÓN FULL DE TODOS LOS PROGRAMAS CON CRACK INCLUIDOS. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- *COMBO XP CONTIENE LOS MEJORES PROGRAMAS DE MICROSOFT WINDOWS XP Y OFFICE XP. La ultima version del Sistema Operativo de Microsoft: Windows XP Profesional en castellano, es un CD con el sistema completo, un diez de calificación para este producto, que tiene mas de 25 nuevas utilidades, y el resto totalmente actualizados. La ultima versión del Paquete de Programas Office: se trata del Office XP 2002 en Castellano, versión profesional, son tres CDS, con la ultima tecnología y modificaciones para este producto. y la versión 2002 en español de la Enciclopedia Encarta junto con el atlas, que en total son otros cuatro CDS... ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- *GUÍA TELEFÓNICA 2002 / DATEL 2002: Nueva Guía Telefónica 2002 en CD ( BUSCA POR NOMBRE, TELÉFONO Y DOMICILIO !!! ) + Nueva Guía de Cod Postales Alfanuméricos 2001. CONTENIDO DEL CD1: Guía Telefónica Invertida con todos los abonados del país de telefónica y Telecom, busca por nombre, teléfono o dirección. Guía Comercial (para realizar búsquedas por empresas, rubro, provincia o localidad, con variables filtros definidos por el usuario: Ej., que tenga e-mail, que tenga URL, que tenga aviso; cuyo resultado podrá ser visualizado en formato de fichas o tablas. Guía Profesional, la más completa de las guías de profesionales del país, podrá realizar búsquedas por profesión: Ej., si UD. desea buscar un abogado, le dará datos de todos los que se encuentren en la zona que UD delimite, o también buscar por apellido y nombre. Guía de Organismos Oficiales. CONTENIDO 2: Nueva guía con los códigos alfanuméricos de toda la Republica Argentina, por ejemplo: el Cód. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- GRAN COLECCIÓN 10 LIBROS SOBRE HACKING a un precio increible:EN 2 CDS. Son 10 CDS que traen de todo!!! LIBRO 1: Hackers 2000 la ultima tecnología en programas de Hacking, los mejores virus, troyanos, programas para crackear software, programas para sacar passwords con sus correspondientes listas de posibles contraseñas, utilidades para icq, para obtener direcciones ip, y muchas pero muchas otras utilidades relacionadas con el hacking. LIBRO 2: Exelsior: Hacking en linux: las ultimas herramientas y los mejores manuales que te enseñaran todos los secretos para poder hackear bajo el sistema operativo Linux. LIBRO 3: Master Hacker: Siguiendo la misma temática que el CD 1, este CD aporta otra gran colección de programas para muchísimas cosas. Excelente. LIBRO 4 y 5: Dr. HACKER and MR. CRACK: Los grandes secretos del hacking y cracking con manuales escritos y programas realizados por dos hackers muy importantes del mundo. LIBRO 6 Este CD contiene muchisimas herramientas para el hacking: TROYANOS: BackOriffice120, bo2k (BackOriffice2000), Netbus160, Netbus170, NetbusPro2, WinSatan, SubSeven y otros, Sacadores de password para los servidores. ANTITROYANOS, Virus (una gran colección), Creadores de VIRUS (versiones en castellanos de estos generadores de Virus), BOMBAS e-mail, la mas grande colección de NUCKES, SCANNEADORES (una gran cantidad de scanneadores de IP, de puertos y MODEMS), CRACKS, Sacadores de password de todo tipo: Para paginas WEB, Para programas como: Word, Excel, Access y otros para obtener las claves de conexión a Internet o redes locales. Muchos programas para telefonía; como llamar de teléfonos gratuitamente, mensajes a celulares y carga de pulsos de estos. Muchos programas para la Administración Remota. LO MEJOR DE ESTE CD ES QUE TIENE UNA GRAN CANTIDAD DE MANUALES Y TUTORES EN CASTELLANO, INCLUSO MUCHOS DE LOS PROGRAMAS ESTÁN EN CASTELLANO (GENERADORES DE VIRUS, TROYANOS COMO EL BACK ORÍFICE Y EL NETBUS TAMBIÉN ESTA EN CASTELLANO). ESTE CD FUE CREADO POR UN GRUPO DE SEGUIDORES DEL CONOCIDO HACKER VeIrUS (DE HABLA HISPANA). LIBRO 7 (continuación del CD 1) Este CD contiene muchísimas herramientas para el hacking: TROYANOS: BackOriffice120, bo2k (BackOriffice2000), Netbus160, Netbus170, NetbusPro2, WinSatan, SubSeven y otros, Sacadores de password para los servidores. ANTITROYANOS, Virus (una gran coleccion), Creadores de VIRUS (versiones en castellanos de estos generadores de Virus), BOMBAS e-mail, la mas grande colección de NUCKES, SCANNEADORES (una gran cantidad de scanneadores de IP, de puertos y MODEMS), CRACKS, Sacadores de password de todo tipo: Para paginas WEB, Para programas como: Word, Excel, Access y otros para obtener las claves de conexión a Internet o redes locales. Muchos programas para telefonía; como llamar de teléfonos gratuitamente, mensajes a celulares y carga de pulsos de estos. LIBRO 8 Hackers Handbook este CD contiene las herramientas y técnicas del arte de hacking(hackear). Herramientas para hackear (Hacking Tools) Mas de 800 Herramientas creadas y usadas por los mejores hackers del mundo para explotar todo tipo de seguridad de computación incluyendo: Password Crackers, Trojan horse wrappers, unix y linux scripts, windows 95/98 remote hacking, nucking, hacks para icq, Windows nt crackers admin, laboratorio de virus y mucho mas. Herramientas de Phreaking. Herramientas de Phreaking que realmente funcionan incluyendo emuladores de red box y plans, generadores de tonos para controlar líneas telefónicas, líen Tes. devices y mucho mas.. Librería de Hacking y Phreaking. La mas completa librería jamás compilada, no son las mismas filas de 10 anos de antigüedad que se venden en otros CDS, estas filas son todas nuevas y explican las verdaderas técnicas usadas por los hackers reales que aparecen en las noticias, incluyendo:ip spoofing(cambia el numero de ip para que nadie sepa quien SOS realmente), hacking icq(tecnicas para hackear el programa icq)y hacks para remote win que no vas a creer, plus exploiting browsers, cracks de web pages y Web Server, escaneo de ports, y miles de cosas mas. El contenido de este CD-rom es de propósitos de información y material difícil de conseguir para estudio o educación propia el contenido de este CD-rom no viola ninguna ley pero el uso de alguno de estos programas si , por eso se ruega que el uso de este CD-rom sea conciente, el vendedor no se responsabiliza de ninguna forma por el mal uso de la información dada en este cumuchos programas para la Administración Remota LIBRO 9 CD que contiene 420 Mb. de pura información, programas, herramientas ... De nuestros amigos Españoles, Mexicanos y de hache ARGENTINA ... La mayoria de su material se encuentra en español ... EL ultimo CD ... recién compilado ... complementa todo lo que le falta al Veirus - y lo actualiza - ... un Cd no solo para aprender sino para ... LIBRO 10 ATENCION PROGRAMADORES, WEBMASTER, MANAGER O CEO "200 DE LOS MEJORES CURSOS, MANUALES Y MONOGRAFÍAS " "EL SABER NO OCUPA LUGAR" YO ME PREGUNTO SI SE INSPIRO EN NUETRO PACK YA QUE EN UN CD HEMOS COMPILADO 200 DE LOS MEJORES CURSOS, MANUALES Y MONOGRAFÍAS; QUE DE OTRA FORMA SERIA IMPOSIBLE COMERCIALIZAR POR SU TAMAÑO Y COSTO. ENTRE LOS TEMAS MAS IMPORTANTES PODES ENCONTRAR: NT, LINUX, JAVA, JSCRIPT, ASP, CGI, HTML, MARKETING, FIREWALL, VISUAL BASIC, VISUAL FOX, INTERNET, UNIX, MACROMEDIA, PERL, C++, SQL, TCP/IP, FRONT PAGE, ORACLE, ENTRE OTROS. NIVELES BÁSICOS, INTERMEDIOS Y AVANZADOS. MAS DE 40.000 HOJAS LOS ARCHIVOS ESTÁN EN FORMATO HTML, PDF Y DOC E INCLUIMOS UTILIDADES Textos en Castellano ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- • NORTON ANTIVIRUS 2002 La mas actualizada versión del mas potente y conocido Antivirus, totalmente funcional bajo Windows Xp. Trae mas de 125 herramientas que mantendran a su equipo junto con la tecnología XP. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- • GLOBAL LINK TRASLATOR PRO 7.0 La ultima versión del mejor traductor para archivos y frases que existan en el mercado. Traduce 6 idiomas: Ingles, Español, Francés, Italiano, Alemán y Portugués. Compatible con Windows y Office XP. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- *JUEGOS / ESTRATEGIA: TITULOS: * FIFA 2001 (FÚTBOL MUNDIAL Y LOCAL DE ALTO NIVEL) * MIDTOWN MADNEES 2 (LAS MEJORES CARRERAS DE AUTOS EN LAS CALLES DE CALIFORNIA, NEW YORK, LONDRES ETC...) * SIM CITY 3000 CASTELLANO ( ES UN JUEGO DE ESTRATEGIA QUE TRATA DE GOBERNAR UN PAIS EN LOS TIEMPOS QUE CORREN) * THE SIMS (JUEGO DE ESTRATEGIA: ES PODER ORGANIZAR UNA FAMILIA Y COMO LLEVARLA A DELANTE) * TOMB RAIDER 4 (JUEGO DE GUERRA Y ARMAS NUCLEARES) * TOMB RAIDER 5 (VERCION MAS AVANZADA DEL TOMB RAIDER CUATRO) * PC FÚTBOL 7 (LOS MEJORES CLUBS DEL PAÍS COMO BOCA, RIVER, VELEZ. PODER LLEGAR A GANAR LA COPA APERTURA Y CLAUSURA) * NEED FOR SPEED 5 (COCHES IMPORTADOS JUEGAN LA CARRERA MAS IMPORTANTE DE TODO LOS TIEMPOS CON FERRARRI, LANBORGINI, ALFA RONEO, JAGUAR Y PORCHE.. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- *PARA ADULTOS LO MEJOR DE KAMASUTRA: Si te interesa el Libro de Kamasutra, entonces tenes que tener este CD. que muestra las mejores poses sexuales en una forma interactiva. Tenes lo mismo que el libro pero en CD. así que si te interesa, solo tense que encargarlo. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- *PADRÓN ELECTORAL: año 1999 incluye hasta la categoría 1980. Todos los ciudadanos de la Republica Argentina reunidos en estos tres CDS. Separados por provincias e importado a archivos de MICROSOFT ACCESS para su mejor utilización. Podes buscar por nombre, apellido o DNI, podes filtra por nombre, apellido, clase, DNI, lugar de votación, dirección, mesa, etc. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- *"PADRÓN DE OBRAS SOCIALES" Estos 2 CDS contienen el registro nacional de todas las obras sociales, y sus correspondientes afiliados. Los datos que incluyen, entre otros, son: nombre y apellido, dirección, DNI, obra social, y otros datos personales de interés. En este caso los datos también pueden exportarse o imprimirse. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- *"REGISTRO DEL AUTOMOTOR" Base de datos del Registro Del Automotor Nacional, que incluye información sobre cada persona que posea vehículos, así como datos sobre el vehículo en si. Además de otros datos de interés. nombre y apellido, dirección, DNI, obra social, y otros datos personales de interés. En este caso los datos también pueden exportarse o imprimirse. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- *"GUÍA COMERCIAL MERCOSUR" Este CD contiene todos los datos de las empresas y negocios que forman parte del MERCOSUR, con posibilidad de filtrar los resultados por país, región, provincia, localidad, rubro, etc. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- *"GUÍA DE LA INDUSTRIA 2002" Este CD contiene los datos de todas las industrias nacionales con posibilidad de buscar por: Razón Social, nombre, dirección, rubro, localidad, etc. *"PAGINAS AMARILLAS 2002" Este CD contiene los datos de todas las empresas nacionales, además de la guía completa de Profesionales y guía telefónica de todo el país. Permite buscar por: razón social, nombre, dirección, teléfono, etc. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- "GUÍA DE TELÉFONO INVERTIDA 2000" Con llave de registro. Este CD contiene la guía telefónica Nacional. Permite buscar por nombre, apellido, dirección parcial, completa, todo un edificio, cuadra par, impar, por nº de teléfono entre otros. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- *PLANO DE CALLES INTELIGENTE: * El CD contiene un plano de calles de Gran Bs.As., La Plata, Ensenada, Beriso y Capital Federal. * Este plano permite buscar direcciones completas, aproximadas, calles, avenidas, lugares, entrecalles, etc. * Además contiene una guía de Códigos Postales, con un buscador que permite encontrar un cod. postal por dirección, localidad, etc. * El plano muestra el resultado y se lo puede agrandar hasta un 600% mas de lo normal, Muestra también todos los nombres de las calles con sus correspondientes alturas. * Permite guardar las búsquedas ya realizadas para luego poder consultarlas. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- *TÉCNICAS Web: ¿NECESITAS HERRAMIENTAS PARA TU Web? ¿ IDEAS ? ¿SCRIPTS? ENTONCES ESTE CD TE VA A INTERESA DEMASIADO. UN CD COMPLETO CON MAS DE 2.000 HERRAMIENTAS PARA LOS WEBMASTERS, HERRAMIENTAS DE AUDIO, video, IMAGEN, INTERNET, Y MUCHO MAS. TENSE SCRIPTS .ASP / .PHP LIBROS DE VISITAS ADMINISTRACIÓN DE BD CONTADORES ONLINE ENCUESTAS FOROS FORMULARIOS PASSWORD Y MUCHOS PERO MUCHOS OTROS SCRIPTS. TENES TAMBIÉN IMÁGENES DE PRUEBA, SONIDOS Y MUCHAS COSAS MAS PARA UTILIZAR EN TU PAGINA Web. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- CADA PRODUCTO CAPITAL : CUESTA $15 PARA LA PROVINCIAS: CUESTA $25 *EL HACKING Y EL COMBO XP CUESTA CAPITAL $25 PARA LA PROVINCIAS:CUESTA $37 !!!!ATENCIÓN... CAPITAL FEDERAL Y GRAN BUENOS AIRES, CADA COMPRA QUE REALICE LES DESCONTAMOS UN %50 DE SERVICIE DE REPARACIÓN DE PC. IMPORTANTE: SOBRE OTRO NOMBRE DE PROGRAMA QUE LE INTERESE, DEBE DECIRNOS EL NOMBRE Y SE LO ENVIAMOS EN 24 HORAS. PARA COMPRARLO SOLO TENES QUE RESPONDERME ESTE MENSAJE ESPECIFICANDO: NOMBRE COMPLETO DEL DESTINATARIO DIRECCIÓN COMPLETA (CALLE - N° - PISO/Dpto. - CP - LOCALIDAD - PCIA.) TELÉFONO (MAYORMENTE PARA CONFIRMAR LOS DATOS, NO IMPORTA QUE SEAN CELULARES O TELÉFONOS DEL INTERIOR.) ATENCIÓN CAPITAL FEDERAL!!! ACEPTAMOS PESOS - DÓLARES - PATACONES- LECOPS (SOLO VALIDO PARA CAPITAL FEDERAL PORQUE SE ENTREGA POR MENSAJERO, EL RESTO DE LAS LOCALIDADES SE ENVÍA POR CORREO ARGENTINO Y NO ACEPTAN ESAS FORMAS DE PAGO POR EL MOMENTO.) ESTA OFERTA NO SE VOLVERÁ A REPETIR. EN CASO DE QUE ESTE EMAIL TE MOLESTE, TE PIDO DISCULPAS, Y NO HACE FALTA QUE TE BORRES DE NINGUNA LISTA PORQUE NO RECIBIRÁS OTRA OFERTA SOBRE ESTE MISMO PRODUCTO. LOS EMAILS LOS ENVIÓ EN FORMATO TEXTO PARA HACERTE MAS RÁPIDA LA DESCARGA. Por sección 301, párrafo (a)(2)(C) de S.1618. Bajo el decreto S.1618 titulo 3ro. Aprobado por el 105 congreso base de las normativas Internacionales sobre SPAM, un E-mail no podrá ser considerado SPAM mientras incluya una forma de ser removido.Si usted desea ser removido de nuestra base de datos en forma definitiva por favor responda a este e-mail indicando "Remover" en el subject. From davem@redhat.com Fri Dec 20 23:06:39 2002 Received: with ECARTIS (v1.0.0; list netdev); Fri, 20 Dec 2002 23:06:44 -0800 (PST) Received: from pizda.ninka.net (IDENT:root@pizda.ninka.net [216.101.162.242]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBL76c3v026140 for ; Fri, 20 Dec 2002 23:06:38 -0800 Received: from localhost (IDENT:davem@localhost.localdomain [127.0.0.1]) by pizda.ninka.net (8.9.3/8.9.3) with ESMTP id XAA28457; Fri, 20 Dec 2002 23:05:29 -0800 Date: Fri, 20 Dec 2002 23:05:28 -0800 (PST) Message-Id: <20021220.230528.106417474.davem@redhat.com> To: kiran@in.ibm.com Cc: linux-kernel@vger.kernel.org, netdev@oss.sgi.com, dipankar@in.ibm.com, akpm@digeo.com Subject: Re: [patch] Make rt_cache_stat use kmalloc_percpu From: "David S. Miller" In-Reply-To: <20021216192212.C26076@in.ibm.com> References: <20021216192212.C26076@in.ibm.com> X-FalunGong: Information control. X-Mailer: Mew version 2.1 on Emacs 21.1 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-archive-position: 1405 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: davem@redhat.com Precedence: bulk X-list: netdev From: Ravikiran G Thirumalai Date: Mon, 16 Dec 2002 19:22:12 +0530 Here's another patch to use kmalloc_percpu. As usual, this removes NR_CPUS bloat, can work when modulized and helps node local allocation. I can't consider this seriously until the kmalloc_percpu stuff actually makes it into Linus's tree. Last I checked, Andrew had a lot of legitimate gripes with the ideas. From davem@redhat.com Fri Dec 20 23:10:16 2002 Received: with ECARTIS (v1.0.0; list netdev); Fri, 20 Dec 2002 23:10:18 -0800 (PST) Received: from pizda.ninka.net (IDENT:root@pizda.ninka.net [216.101.162.242]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBL7AG3v026569 for ; Fri, 20 Dec 2002 23:10:16 -0800 Received: from localhost (IDENT:davem@localhost.localdomain [127.0.0.1]) by pizda.ninka.net (8.9.3/8.9.3) with ESMTP id XAA28508; Fri, 20 Dec 2002 23:08:37 -0800 Date: Fri, 20 Dec 2002 23:08:36 -0800 (PST) Message-Id: <20021220.230836.40590865.davem@redhat.com> To: krkumar@us.ibm.com Cc: kuznet@ms2.inr.ac.ru, netdev@oss.sgi.com, linux-kernel@vger.kernel.org Subject: Re: [PATCH RESEND] memory leak in ndisc_router_discovery From: "David S. Miller" In-Reply-To: <200212121905.gBCJ5hn18058@eng2.beaverton.ibm.com> References: <200212121905.gBCJ5hn18058@eng2.beaverton.ibm.com> X-FalunGong: Information control. X-Mailer: Mew version 2.1 on Emacs 21.1 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-archive-position: 1406 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: davem@redhat.com Precedence: bulk X-list: netdev From: Krishna Kumar Date: Thu, 12 Dec 2002 11:05:43 -0800 (PST) I had sent this earlier, there is a bug in router advertisement handling code, where the reference (and memory) to an inet6_dev pointer can get leaked (this leak can happen atmost once for each interface on a system which receives invalid RA's). Below is the patch against 2.5.51 to fix it. Applied, thanks. From Andrew.Morton@digeo.com Fri Dec 20 23:27:42 2002 Received: with ECARTIS (v1.0.0; list netdev); Fri, 20 Dec 2002 23:27:44 -0800 (PST) Received: from packet.digeo.com (packet.digeo.com [12.110.80.53]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBL7Rf3v027026 for ; Fri, 20 Dec 2002 23:27:42 -0800 Received: from digeo-nav01.digeo.com (digeo-nav01.digeo.com [192.168.1.233]) by packet.digeo.com (8.9.3+Sun/8.9.3) with SMTP id XAA19836 for ; Fri, 20 Dec 2002 23:31:56 -0800 (PST) Received: from digeo-e2k04.digeo.com ([192.168.2.24]) by digeo-nav01.digeo.com (NAVGW 2.5.2.12) with SMTP id M2002122023340519723 ; Fri, 20 Dec 2002 23:34:05 -0800 Received: from pao-ex01.pao.digeo.com ([172.17.144.34]) by digeo-e2k04.digeo.com with Microsoft SMTPSVC(5.0.2195.5329); Fri, 20 Dec 2002 23:31:58 -0800 Received: from digeo.com ([172.17.144.18]) by pao-ex01.pao.digeo.com with Microsoft SMTPSVC(5.0.2195.4905); Fri, 20 Dec 2002 23:31:57 -0800 Message-ID: <3E0418EC.B55941EE@digeo.com> Date: Fri, 20 Dec 2002 23:31:56 -0800 From: Andrew Morton X-Mailer: Mozilla 4.79 [en] (X11; U; Linux 2.5.52 i686) X-Accept-Language: en MIME-Version: 1.0 To: "David S. Miller" CC: kiran@in.ibm.com, linux-kernel@vger.kernel.org, netdev@oss.sgi.com, dipankar@in.ibm.com Subject: Re: [patch] Make rt_cache_stat use kmalloc_percpu References: <20021216192212.C26076@in.ibm.com> <20021220.230528.106417474.davem@redhat.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 21 Dec 2002 07:31:57.0350 (UTC) FILETIME=[0AD33C60:01C2A8C3] X-archive-position: 1407 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: akpm@digeo.com Precedence: bulk X-list: netdev "David S. Miller" wrote: > > From: Ravikiran G Thirumalai > Date: Mon, 16 Dec 2002 19:22:12 +0530 > > Here's another patch to use kmalloc_percpu. As usual, this removes > NR_CPUS bloat, can work when modulized and helps node local allocation. > > I can't consider this seriously until the kmalloc_percpu stuff > actually makes it into Linus's tree. Last I checked, Andrew had > a lot of legitimate gripes with the ideas. Kiran's latest (vastly simpler) version looks fine to my eye. From davem@redhat.com Fri Dec 20 23:30:20 2002 Received: with ECARTIS (v1.0.0; list netdev); Fri, 20 Dec 2002 23:30:23 -0800 (PST) Received: from pizda.ninka.net (IDENT:root@pizda.ninka.net [216.101.162.242]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBL7UJ3v027421 for ; Fri, 20 Dec 2002 23:30:19 -0800 Received: from localhost (IDENT:davem@localhost.localdomain [127.0.0.1]) by pizda.ninka.net (8.9.3/8.9.3) with ESMTP id XAA28616; Fri, 20 Dec 2002 23:29:10 -0800 Date: Fri, 20 Dec 2002 23:29:09 -0800 (PST) Message-Id: <20021220.232909.88324816.davem@redhat.com> To: akpm@digeo.com Cc: kiran@in.ibm.com, linux-kernel@vger.kernel.org, netdev@oss.sgi.com, dipankar@in.ibm.com Subject: Re: [patch] Make rt_cache_stat use kmalloc_percpu From: "David S. Miller" In-Reply-To: <3E0418EC.B55941EE@digeo.com> References: <20021216192212.C26076@in.ibm.com> <20021220.230528.106417474.davem@redhat.com> <3E0418EC.B55941EE@digeo.com> X-FalunGong: Information control. X-Mailer: Mew version 2.1 on Emacs 21.1 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-archive-position: 1408 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: davem@redhat.com Precedence: bulk X-list: netdev From: Andrew Morton Date: Fri, 20 Dec 2002 23:31:56 -0800 "David S. Miller" wrote: > I can't consider this seriously until the kmalloc_percpu stuff > actually makes it into Linus's tree. Last I checked, Andrew had > a lot of legitimate gripes with the ideas. Kiran's latest (vastly simpler) version looks fine to my eye. Ok, so once that is in, he can feel free to resubmit the rt_cache_state patch. From werner@almesberger.net Sat Dec 21 01:41:18 2002 Received: with ECARTIS (v1.0.0; list netdev); Sat, 21 Dec 2002 01:41:22 -0800 (PST) Received: from host.almesberger.net (IDENT:4CWNKCAzITRSRT85bSlJCtDl5c2/1qE3@almesberger.net [63.105.73.239] (may be forged)) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBL9fI3v028444 for ; Sat, 21 Dec 2002 01:41:18 -0800 Received: from almesberger.net (vpnwa-home [10.200.0.2]) by host.almesberger.net (8.9.3/8.9.3) with ESMTP id BAA10503 for ; Sat, 21 Dec 2002 01:45:41 -0800 Received: (from werner@localhost) by almesberger.net (8.11.6/8.11.6) id gBL6m9P17945 for netdev@oss.sgi.com; Sat, 21 Dec 2002 03:48:09 -0300 Date: Sat, 21 Dec 2002 03:48:09 -0300 From: Werner Almesberger To: netdev@oss.sgi.com Subject: TCP connection passing, version 5 Message-ID: <20021221034809.A17934@almesberger.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-archive-position: 1409 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: wa@almesberger.net Precedence: bulk X-list: netdev A very minor update: http://www.almesberger.net/tcpcp/tcpcp-5.tar.gz md5sum d30310aec5e5c22e5fbc178274cd7c46 This one mainly flushes lots of extensions for a testing framework I've made over the last few weeks. The testing framework isn't there yet, and the extensions aren't particularly pretty either. The patch is now relative to 2.5.49, and there's also a patch for UML as of 2.5.47. tcpcp on UML is fun for demos :-) - Werner ----------------------------------- CHANGES ----------------------------------- Version 5 (21-DEC-2002) ----------------------- - upgraded to the 2.5.49 kernel - make "upload" now runs md5sum after uploading - tentatively added -b ("background") option to cpshd - added UML build process in uml/ - added framework for running programs under UML in umlrun/ - mkpatch now exits with non-zero exit status on error -- _________________________________________________________________________ / Werner Almesberger, Buenos Aires, Argentina wa@almesberger.net / /_http://www.almesberger.net/____________________________________________/ From jgarzik@pobox.com Sat Dec 21 12:28:40 2002 Received: with ECARTIS (v1.0.0; list netdev); Sat, 21 Dec 2002 12:28:47 -0800 (PST) Received: from www.linux.org.uk (parcelfarce.linux.theplanet.co.uk [195.92.249.252]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBLKSc3v003597 for ; Sat, 21 Dec 2002 12:28:39 -0800 Received: from c-24-98-88-232.atl.client2.attbi.com ([24.98.88.232] helo=pobox.com) by www.linux.org.uk with esmtp (Exim 3.33 #5) id 18PqIu-0000jp-00; Sat, 21 Dec 2002 20:33:00 +0000 Message-ID: <3E04CFBE.2060909@pobox.com> Date: Sat, 21 Dec 2002 15:31:58 -0500 From: Jeff Garzik User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2) Gecko/20021202 X-Accept-Language: en-us, en MIME-Version: 1.0 To: "David S. Miller" CC: Manish Lachwani , netdev@oss.sgi.com Subject: [PATCH] tg3 intr masking update Content-Type: multipart/mixed; boundary="------------000409090806070708070106" X-archive-position: 1410 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jgarzik@pobox.com Precedence: bulk X-list: netdev This is a multi-part message in MIME format. --------------000409090806070708070106 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Here is a better patch, after I mulled it over on the trip to Atlanta. It is based on the premise, derived from the chip documentation, that 1) intr-mbox-0 register usage is used to indicate to the NIC that the driver is "in interrupt handler". 2) (intr-mbox-0 = 1) == pci-inta-mask + pci-inta-clear + stuff 3) the docs seem to be saying "just use intr-mbox-0 and ignore pci-inta-mask" Based on these three observations, the attached patch updates tg3 for the following changes: 1) Mask ints in tg3_interrupt and unmask in tg3_poll. We not only eliminate several MMIO writes/reads, but we also eliminate all usage of functions tg3_[un]mask_ints(), which bit-bangs the MISC_HOST_CTRL_MASK_PCI_INT bit in the TG3PCI_MISC_HOST_CTRL register. 2) If tg3_main_interrupt_work() returns no-work-exists, we [obviously must] unmask ints in tg3_interrupt. But this is an unlikely case. 3) Simply call netif_rx_schedule(dev) in tg3_interrupt_main_work(). This was a change previously discussed. Eliminates the "poll already scheduled" message which was often actually a bogus message. 4) Because of #2, tg3_interrupt_main_work returns boolean 'work_exists' now. 5) There are some Bugzilla bug reports for tg3 1.2, where NMI watchdog triggers in tg3_timer. I did s/spin_lock_irq/spin_lock_irqsave/ in tg3_poll for safety, because we might be locking against a timer not only an interrupt handler. And just for general paranoia and safety as an "obviously more safe" change. Comments/feedback/corrections/flames requested :) --------------000409090806070708070106 Content-Type: text/plain; name="patch" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="patch" ===== drivers/net/tg3.c 1.40 vs edited ===== --- 1.40/drivers/net/tg3.c Mon Dec 9 21:32:02 2002 +++ edited/drivers/net/tg3.c Sat Dec 21 15:06:34 2002 @@ -217,22 +217,6 @@ tr32(MAILBOX_INTERRUPT_0 + TG3_64BIT_REG_LOW); } -static inline void tg3_mask_ints(struct tg3 *tp) -{ - tw32(TG3PCI_MISC_HOST_CTRL, - (tp->misc_host_ctrl | MISC_HOST_CTRL_MASK_PCI_INT)); -} - -static inline void tg3_unmask_ints(struct tg3 *tp) -{ - tw32(TG3PCI_MISC_HOST_CTRL, - (tp->misc_host_ctrl & ~MISC_HOST_CTRL_MASK_PCI_INT)); - if (tp->hw_status->status & SD_STATUS_UPDATED) { - tw32(GRC_LOCAL_CTRL, - tp->grc_local_ctrl | GRC_LCLCTRL_SETINT); - } -} - static void tg3_switch_clocks(struct tg3 *tp) { if (tr32(TG3PCI_CLOCK_CTRL) & CLOCK_CTRL_44MHZ_CORE) { @@ -2014,8 +1998,9 @@ struct tg3 *tp = netdev->priv; struct tg3_hw_status *sblk = tp->hw_status; int done; + unsigned long flags; - spin_lock_irq(&tp->lock); + spin_lock_irqsave(&tp->lock, flags); if (!(tp->tg3_flags & (TG3_FLAG_USE_LINKCHG_REG | @@ -2052,15 +2037,18 @@ if (done) { netif_rx_complete(netdev); - tg3_unmask_ints(tp); + tw32_mailbox(MAILBOX_INTERRUPT_0 + TG3_64BIT_REG_LOW, + 0x00000000); + /* don't care about flushing this write immediately */ } - spin_unlock_irq(&tp->lock); + spin_unlock_irqrestore(&tp->lock, flags); return (done ? 0 : 1); } -static __inline__ void tg3_interrupt_main_work(struct net_device *dev, struct tg3 *tp) +static inline unsigned int tg3_interrupt_main_work(struct net_device *dev, + struct tg3 *tp) { struct tg3_hw_status *sblk = tp->hw_status; int work_exists = 0; @@ -2075,19 +2063,10 @@ sblk->idx[0].rx_producer != tp->rx_rcb_ptr) work_exists = 1; - if (!work_exists) - return; + if (work_exists) + netif_rx_schedule(dev); - if (netif_rx_schedule_prep(dev)) { - /* NOTE: These writes are posted by the readback of - * the mailbox register done by our caller. - */ - tg3_mask_ints(tp); - __netif_rx_schedule(dev); - } else { - printk(KERN_ERR PFX "%s: Error, poll already scheduled\n", - dev->name); - } + return work_exists; } static void tg3_interrupt(int irq, void *dev_id, struct pt_regs *regs) @@ -2102,13 +2081,16 @@ if (sblk->status & SD_STATUS_UPDATED) { tw32_mailbox(MAILBOX_INTERRUPT_0 + TG3_64BIT_REG_LOW, 0x00000001); + /* flush the intr clear-and-mask before updating status block */ + tr32(MAILBOX_INTERRUPT_0 + TG3_64BIT_REG_LOW); sblk->status &= ~SD_STATUS_UPDATED; - tg3_interrupt_main_work(dev, tp); - - tw32_mailbox(MAILBOX_INTERRUPT_0 + TG3_64BIT_REG_LOW, - 0x00000000); - tr32(MAILBOX_INTERRUPT_0 + TG3_64BIT_REG_LOW); + if (!tg3_interrupt_main_work(dev, tp)) { + /* if work doesn't exist... */ + tw32_mailbox(MAILBOX_INTERRUPT_0 + TG3_64BIT_REG_LOW, + 0x00000000); + tr32(MAILBOX_INTERRUPT_0 + TG3_64BIT_REG_LOW); + } } spin_unlock_irqrestore(&tp->lock, flags); --------------000409090806070708070106-- From jgarzik@pobox.com Sat Dec 21 12:34:48 2002 Received: with ECARTIS (v1.0.0; list netdev); Sat, 21 Dec 2002 12:34:50 -0800 (PST) Received: from www.linux.org.uk (parcelfarce.linux.theplanet.co.uk [195.92.249.252]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBLKYl3v004000 for ; Sat, 21 Dec 2002 12:34:48 -0800 Received: from c-24-98-88-232.atl.client2.attbi.com ([24.98.88.232] helo=pobox.com) by www.linux.org.uk with esmtp (Exim 3.33 #5) id 18PqOs-0000m6-00; Sat, 21 Dec 2002 20:39:10 +0000 Message-ID: <3E04D130.8040504@pobox.com> Date: Sat, 21 Dec 2002 15:38:08 -0500 From: Jeff Garzik User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2) Gecko/20021202 X-Accept-Language: en-us, en MIME-Version: 1.0 To: "David S. Miller" CC: Manish Lachwani , netdev@oss.sgi.com Subject: Re: [PATCH] tg3 intr masking update References: <3E04CFBE.2060909@pobox.com> In-Reply-To: <3E04CFBE.2060909@pobox.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 1411 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: jgarzik@pobox.com Precedence: bulk X-list: netdev Additional note about this change, The NAPI docs mention specific areas where races may occur, and coincedentally so do the tg3 docs. The tg3 docs suggest to me that in tg3_poll, after we unmask ints, we should * flush the MMIO write * check the status block for updates again * tell tg3 to deliver to us an artificial interrupt just in case we raced [IMO preferred versus simply returning 'not-done' in tg3_poll retval] But this is just a theory and not tested yet... From rl@hellgate.ch Sat Dec 21 13:40:35 2002 Received: with ECARTIS (v1.0.0; list netdev); Sat, 21 Dec 2002 13:40:39 -0800 (PST) Received: from mta3n.bluewin.ch (mta3n.bluewin.ch [195.186.1.212]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBLLeX3v005271 for ; Sat, 21 Dec 2002 13:40:34 -0800 Received: from k3.hellgate.ch (195.186.229.68) by mta3n.bluewin.ch (Bluewin AG 6.5.032) id 3DEBA8F700257DBA; Sat, 21 Dec 2002 22:44:51 +0100 Received: by k3.hellgate.ch (MTA, from userid 500) id E1C3E37F4C; Sat, 21 Dec 2002 22:44:48 +0100 (CET) Date: Sat, 21 Dec 2002 22:44:48 +0100 From: Roger Luethi To: linux-kernel@vger.kernel.org Cc: netdev@oss.sgi.com Subject: VIA Rhine 1.15exp1, patch for testing/discussion Message-ID: <20021221214448.GA19739@k3.hellgate.ch> Mail-Followup-To: linux-kernel@vger.kernel.org, netdev@oss.sgi.com Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="ZPt4rx8FFjLCG7dd" Content-Disposition: inline User-Agent: Mutt/1.3.27i X-Operating-System: Linux 2.5.52 on i686 X-GPG-Fingerprint: 92 F4 DC 20 57 46 7B 95 24 4E 9E E7 5A 54 DC 1B X-GPG: 1024/80E744BD wwwkeys.ch.pgp.net X-archive-position: 1412 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: rl@hellgate.ch Precedence: bulk X-list: netdev --ZPt4rx8FFjLCG7dd Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Main points (more details below): - Tx underrun handling redone - Give chip 10 us (instead of 10 ms) to reset - Force a reset on VT6102 if necessary Minor changes: - use defined constants (especially MII), nuke mii_status_bits - move clear_tally_counters() up so it can get inlined (pointed out by Luca Barbieri) - fix namespace pollution According to the spec and my own tests, the flag previously labeled IntrMIIChange in fact indicates a Tx (FIFO, desc, or buffer, the specs are not quite sure about that) underrun. The flag we've been testing supposedly indicates a Tx underrun, too, but I've never seen it trigger. -- The new code tests for both. IntrMIIChange handling got dropped in the process. Tx underrun is more difficult to trigger than the abort, but I have seen the Tx threshold increase to 0x40 and then to 0x60. With 0x60, I have not been able to produce any Tx underruns. The fact that increasing the threshold fixes the error is another strong hint that the specs are correct in this case. The current driver will not only fail to recognize a Tx underrun, it will actually treat it as an abort _and_ restart autonegotiation (the interrupt status is 0208). Instead of testing for TxError (used to be called TxAbort), the new driver now tests for TxAborted. The autonegotiation issue was due to IntrMIIChange as discussed above. For the first time ever, and only once, I have seen a Rhine-II that refused to reset with the current driver. That was just enough to confirm that the force reset flag actually works, so that's in now. In the same function, I nuked the code that waited up to 10 ms for the chip to reset. I have yet to see a chip reset in 10 ms after it had failed to reset in 10 us. This change may cause a regression, but I doubt it. If you know it does, holler. I've got more changes in the works, this is a snapshot with some of the better tested pieces. Roger --ZPt4rx8FFjLCG7dd Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="via-rhine.c-1.15exp1.diff" --- linux-2.5.52/drivers/net/via-rhine.c.orig Fri Dec 20 20:13:55 2002 +++ linux-2.5.52/drivers/net/via-rhine.c Sat Dec 21 21:09:25 2002 @@ -101,11 +101,14 @@ LK1.1.15 (jgarzik): - Use new MII lib helper generic_mii_ioctl + LK1.1.15exp1 (Roger Luethi): + - [...] + */ #define DRV_NAME "via-rhine" -#define DRV_VERSION "1.1.15" -#define DRV_RELDATE "November-22-2002" +#define DRV_VERSION "1.1.15exp1" +#define DRV_RELDATE "December-21-2002" /* A few user-configurable values. @@ -416,29 +419,15 @@ int mmio_verify_registers[] = { /* Bits in the interrupt status/mask registers. */ enum intr_status_bits { IntrRxDone=0x0001, IntrRxErr=0x0004, IntrRxEmpty=0x0020, - IntrTxDone=0x0002, IntrTxError=0x0008, IntrTxUnderrun=0x0010, + IntrTxDone=0x0002, IntrTxError=0x0008, IntrTxUnderrun=0x0210, IntrPCIErr=0x0040, - IntrStatsMax=0x0080, IntrRxEarly=0x0100, IntrMIIChange=0x0200, + IntrStatsMax=0x0080, IntrRxEarly=0x0100, IntrRxOverflow=0x0400, IntrRxDropped=0x0800, IntrRxNoBuf=0x1000, IntrTxAborted=0x2000, IntrLinkChange=0x4000, IntrRxWakeUp=0x8000, IntrNormalSummary=0x0003, IntrAbnormalSummary=0xC260, }; -/* MII interface, status flags. - Not to be confused with the MIIStatus register ... */ -enum mii_status_bits { - MIICap100T4 = 0x8000, - MIICap10100HdFd = 0x7800, - MIIPreambleSupr = 0x0040, - MIIAutoNegCompleted = 0x0020, - MIIRemoteFault = 0x0010, - MIICapAutoNeg = 0x0008, - MIILink = 0x0004, - MIIJabber = 0x0002, - MIIExtended = 0x0001 -}; - /* The Rx and Tx buffer descriptors. */ struct rx_desc { s32 rx_status; @@ -533,30 +522,31 @@ static void via_rhine_set_rx_mode(struct static struct net_device_stats *via_rhine_get_stats(struct net_device *dev); static int netdev_ioctl(struct net_device *dev, struct ifreq *rq, int cmd); static int via_rhine_close(struct net_device *dev); -static inline void clear_tally_counters(long ioaddr); -static inline void via_restart_tx(struct net_device *dev); static void wait_for_reset(struct net_device *dev, int chip_id, char *name) { long ioaddr = dev->base_addr; - int i; + + udelay(5); + + if (readw(ioaddr + ChipCmd) & CmdReset) { + printk(KERN_INFO "%s: Reset did not complete in 5 us.\n", name); + } /* VT86C100A may need long delay after reset (dlink) */ if (chip_id == VT86C100A) udelay(100); - i = 0; - do { - udelay(5); - i++; - if(i > 2000) { - printk(KERN_ERR "%s: reset did not complete in 10 ms.\n", name); - break; - } - } while(readw(ioaddr + ChipCmd) & CmdReset); + /* The VT6102 needs to be forced sometimes */ + if (chip_id == VT6102) { + writeb(0x40, ioaddr + 0x81); + } + + udelay(5); + if (debug > 1) - printk(KERN_INFO "%s: reset finished after %d microseconds.\n", - name, 5*i); + printk(KERN_INFO "%s: Reset %s.\n", name, + (readw(ioaddr + ChipCmd) & CmdReset) ? "failed" : "succeeded"); } #ifdef USE_MEM @@ -787,17 +777,17 @@ static int __devinit via_rhine_init_one int phy, phy_idx = 0; np->phys[0] = 1; /* Standard for this chip. */ for (phy = 1; phy < 32 && phy_idx < MAX_MII_CNT; phy++) { - int mii_status = mdio_read(dev, phy, 1); + int mii_status = mdio_read(dev, phy, MII_BMSR); if (mii_status != 0xffff && mii_status != 0x0000) { np->phys[phy_idx++] = phy; - np->mii_if.advertising = mdio_read(dev, phy, 4); + np->mii_if.advertising = mdio_read(dev, phy, MII_ADVERTISE); printk(KERN_INFO "%s: MII PHY found at address %d, status " "0x%4.4x advertising %4.4x Link %4.4x.\n", dev->name, phy, mii_status, np->mii_if.advertising, - mdio_read(dev, phy, 5)); + mdio_read(dev, phy, MII_LPA)); /* set IFF_RUNNING */ - if (mii_status & MIILink) + if (mii_status & BMSR_LSTATUS) netif_carrier_on(dev); else netif_carrier_off(dev); @@ -820,8 +810,8 @@ static int __devinit via_rhine_init_one (option & 0x220 ? "full" : "half")); if (np->mii_cnt) mdio_write(dev, np->phys[0], MII_BMCR, - ((option & 0x300) ? 0x2000 : 0) | /* 100mbps? */ - ((option & 0x220) ? 0x0100 : 0)); /* Full duplex? */ + ((option & 0x300) ? BMCR_SPEED100 : 0) | /* 100mbps? */ + ((option & 0x220) ? BMCR_FULLDPLX : 0)); /* Full duplex? */ } } @@ -1021,7 +1011,7 @@ static void init_registers(struct net_de writew(IntrRxDone | IntrRxErr | IntrRxEmpty| IntrRxOverflow | IntrRxDropped | IntrRxNoBuf | IntrTxAborted | IntrTxDone | IntrTxError | IntrTxUnderrun | - IntrPCIErr | IntrStatsMax | IntrLinkChange | IntrMIIChange, + IntrPCIErr | IntrStatsMax | IntrLinkChange, ioaddr + IntrEnable); np->chip_cmd = CmdStart|CmdTxOn|CmdRxOn|CmdNoTxPoll; @@ -1066,10 +1056,10 @@ static void mdio_write(struct net_device if (phy_id == np->phys[0]) { switch (regnum) { case MII_BMCR: /* Is user forcing speed/duplex? */ - if (value & 0x9000) /* Autonegotiation. */ + if (value & (BMCR_ANENABLE | BMCR_RESET)) /* Autonegotiation. */ np->mii_if.force_media = 0; else - np->mii_if.full_duplex = (value & 0x0100) ? 1 : 0; + np->mii_if.full_duplex = (value & BMCR_FULLDPLX) ? 1 : 0; break; case MII_ADVERTISE: np->mii_if.advertising = value; @@ -1175,8 +1165,8 @@ static void via_rhine_timer(unsigned lon /* make IFF_RUNNING follow the MII status bit "Link established" */ mii_status = mdio_read(dev, np->phys[0], MII_BMSR); - if ( (mii_status & MIILink) != (np->mii_status & MIILink) ) { - if (mii_status & MIILink) + if ( (mii_status & BMSR_LSTATUS) != (np->mii_status & BMSR_LSTATUS) ) { + if (mii_status & BMSR_LSTATUS) netif_carrier_on(dev); else netif_carrier_off(dev); @@ -1320,7 +1310,7 @@ static void via_rhine_interrupt(int irq, via_rhine_tx(dev); /* Abnormal error summary/uncommon events handlers. */ - if (intr_status & (IntrPCIErr | IntrLinkChange | IntrMIIChange | + if (intr_status & (IntrPCIErr | IntrLinkChange | IntrStatsMax | IntrTxError | IntrTxAborted | IntrTxUnderrun)) via_rhine_error(dev, intr_status); @@ -1515,7 +1505,7 @@ static void via_rhine_rx(struct net_devi writew(CmdRxDemand | np->chip_cmd, dev->base_addr + ChipCmd); } -static inline void via_restart_tx(struct net_device *dev) { +static inline void via_rhine_restart_tx(struct net_device *dev) { struct netdev_private *np = dev->priv; int entry = np->dirty_tx % TX_RING_SIZE; @@ -1526,6 +1516,17 @@ static inline void via_restart_tx(struct writew(CmdTxDemand | np->chip_cmd, dev->base_addr + ChipCmd); } +/* Clears the "tally counters" for CRC errors and missed frames(?). + It has been reported that some chips need a write of 0 to clear + these, for others the counters are set to 1 when written to and + instead cleared when read. So we clear them both ways ... */ +static inline void clear_tally_counters(const long ioaddr) +{ + writel(0, ioaddr + RxMissed); + readw(ioaddr + RxCRCErrs); + readw(ioaddr + RxMissed); +} + static void via_rhine_error(struct net_device *dev, int intr_status) { struct netdev_private *np = dev->priv; @@ -1533,7 +1534,7 @@ static void via_rhine_error(struct net_d spin_lock (&np->lock); - if (intr_status & (IntrMIIChange | IntrLinkChange)) { + if (intr_status & (IntrLinkChange)) { if (readb(ioaddr + MIIStatus) & 0x02) { /* Link failed, restart autonegotiation. */ if (np->drv_flags & HasDavicomPhy) @@ -1551,11 +1552,11 @@ static void via_rhine_error(struct net_d np->stats.rx_missed_errors += readw(ioaddr + RxMissed); clear_tally_counters(ioaddr); } - if (intr_status & IntrTxError) { + if (intr_status & IntrTxAborted) { if (debug > 1) printk(KERN_INFO "%s: Abort %4.4x, frame dropped.\n", dev->name, intr_status); - via_restart_tx(dev); + via_rhine_restart_tx(dev); } if (intr_status & IntrTxUnderrun) { if (np->tx_thresh < 0xE0) @@ -1564,9 +1565,9 @@ static void via_rhine_error(struct net_d printk(KERN_INFO "%s: Transmitter underrun, Tx " "threshold now %2.2x.\n", dev->name, np->tx_thresh); - via_restart_tx(dev); + via_rhine_restart_tx(dev); } - if (intr_status & ~( IntrLinkChange | IntrStatsMax | + if (intr_status & ~( IntrLinkChange | IntrStatsMax | IntrTxUnderrun | IntrTxError | IntrTxAborted | IntrNormalSummary)) { if (debug > 1) printk(KERN_ERR "%s: Something Wicked happened! %4.4x.\n", @@ -1593,17 +1594,6 @@ static struct net_device_stats *via_rhin return &np->stats; } -/* Clears the "tally counters" for CRC errors and missed frames(?). - It has been reported that some chips need a write of 0 to clear - these, for others the counters are set to 1 when written to and - instead cleared when read. So we clear them both ways ... */ -static inline void clear_tally_counters(const long ioaddr) -{ - writel(0, ioaddr + RxMissed); - readw(ioaddr + RxCRCErrs); - readw(ioaddr + RxMissed); -} - static void via_rhine_set_rx_mode(struct net_device *dev) { struct netdev_private *np = dev->priv; --ZPt4rx8FFjLCG7dd-- From kiran@in.ibm.com Sun Dec 22 20:32:39 2002 Received: with ECARTIS (v1.0.0; list netdev); Sun, 22 Dec 2002 20:32:43 -0800 (PST) Received: from e6.ny.us.ibm.com (e6.ny.us.ibm.com [32.97.182.106]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBN4Wc3v002191 for ; Sun, 22 Dec 2002 20:32:39 -0800 Received: from northrelay04.pok.ibm.com (northrelay04.pok.ibm.com [9.56.224.206]) by e6.ny.us.ibm.com (8.12.2/8.12.2) with ESMTP id gBN4b3Bv101044; Sun, 22 Dec 2002 23:37:03 -0500 Received: from asterix.in.ibm.com (asterix.in.ibm.com [9.182.12.249]) by northrelay04.pok.ibm.com (8.12.3/NCO/VER6.4) with ESMTP id gBN4arEo049978; Sun, 22 Dec 2002 23:36:56 -0500 Received: (from kirant@localhost) by asterix.in.ibm.com (8.11.2/8.11.2) id gBN4Y6V05388; Mon, 23 Dec 2002 10:04:06 +0530 Date: Mon, 23 Dec 2002 10:04:05 +0530 From: Ravikiran G Thirumalai To: "David S. Miller" Cc: akpm@digeo.com, linux-kernel@vger.kernel.org, netdev@oss.sgi.com, dipankar@in.ibm.com Subject: Re: [patch] Make rt_cache_stat use kmalloc_percpu Message-ID: <20021223100405.B23413@in.ibm.com> References: <20021216192212.C26076@in.ibm.com> <20021220.230528.106417474.davem@redhat.com> <3E0418EC.B55941EE@digeo.com> <20021220.232909.88324816.davem@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20021220.232909.88324816.davem@redhat.com>; from davem@redhat.com on Fri, Dec 20, 2002 at 11:29:09PM -0800 X-archive-position: 1413 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kiran@in.ibm.com Precedence: bulk X-list: netdev On Fri, Dec 20, 2002 at 11:29:09PM -0800, David S. Miller wrote: >... > Kiran's latest (vastly simpler) version looks fine to my eye. > > Ok, so once that is in, he can feel free to resubmit the > rt_cache_state patch. Ok, I'll do that Thanks, Kiran From anurag_co@rediffmail.com Sun Dec 22 23:36:18 2002 Received: with ECARTIS (v1.0.0; list netdev); Sun, 22 Dec 2002 23:36:21 -0800 (PST) Received: from rediffmail.com (webmail18.rediffmail.com [203.199.83.28] (may be forged)) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBN7aF3v003559 for ; Sun, 22 Dec 2002 23:36:17 -0800 Received: (qmail 18716 invoked by uid 510); 23 Dec 2002 07:38:48 -0000 Date: 23 Dec 2002 07:38:48 -0000 Message-ID: <20021223073848.18715.qmail@webmail18.rediffmail.com> Received: from unknown (203.163.163.9) by rediffmail.com via HTTP; 23 dec 2002 07:38:48 -0000 MIME-Version: 1.0 From: "anurag madhavjibhai barvadiya" Reply-To: "anurag madhavjibhai barvadiya" To: netdev@oss.sgi.com Subject: How to get CBQ algorithm & code Content-type: text/plain; format=flowed Content-Disposition: inline X-archive-position: 1414 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: anurag_co@rediffmail.com Precedence: bulk X-list: netdev Hi, My project is bandwidth limitation,that is i do with the help of CBQ. So please how i can get material & source code about cbq. Can anyone tell me where can I download the document talk about CBQ algorithm. Thanks Xiang Ji From kiran@in.ibm.com Mon Dec 23 05:37:16 2002 Received: with ECARTIS (v1.0.0; list netdev); Mon, 23 Dec 2002 05:37:22 -0800 (PST) Received: from e6.ny.us.ibm.com (e6.ny.us.ibm.com [32.97.182.106]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBNDbF3v011532 for ; Mon, 23 Dec 2002 05:37:15 -0800 Received: from northrelay03.pok.ibm.com (northrelay03.pok.ibm.com [9.56.224.151]) by e6.ny.us.ibm.com (8.12.2/8.12.2) with ESMTP id gBNDfeBv012258; Mon, 23 Dec 2002 08:41:41 -0500 Received: from asterix.in.ibm.com (asterix.in.ibm.com [9.182.12.249]) by northrelay03.pok.ibm.com (8.12.3/NCO/VER6.4) with ESMTP id gBNDfarO087144; Mon, 23 Dec 2002 08:41:37 -0500 Received: (from kirant@localhost) by asterix.in.ibm.com (8.11.2/8.11.2) id gBNDcmY07202; Mon, 23 Dec 2002 19:08:48 +0530 Date: Mon, 23 Dec 2002 19:08:48 +0530 From: Ravikiran G Thirumalai To: "David S. Miller " Cc: netdev , linux-kernel@vger.kernel.org Subject: [patch] Convert sockets_in_use to use per_cpu areas Message-ID: <20021223190847.G23413@in.ibm.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i X-archive-position: 1415 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kiran@in.ibm.com Precedence: bulk X-list: netdev Hi, Here's a simple patch to change sockets_in_use to make use of per-cpu areas. I have couple of questions though... 1. Was this var made per-cpu just to avoid atomic_t or locking or are there real life workloads which cause too many sock_alloc and sock_releases to cause cacheline bouncing? 2. Is this var required? since we can just sum up all proto->stats.inuse and remove this var altogether? (This var is read only for /proc reporting) Thanks, Kiran diff -ruN -X dontdiff linux-2.5.52/net/socket.c sockets_in_use-2.5.52/net/socket.c --- linux-2.5.52/net/socket.c Mon Dec 16 07:37:53 2002 +++ sockets_in_use-2.5.52/net/socket.c Mon Dec 23 11:48:44 2002 @@ -189,10 +189,7 @@ * Statistics counters of the socket lists */ -static union { - int counter; - char __pad[SMP_CACHE_BYTES]; -} sockets_in_use[NR_CPUS] __cacheline_aligned = {{0}}; +static DEFINE_PER_CPU(int, sockets_in_use); /* * Support routines. Move socket addresses back and forth across the kernel/user @@ -475,7 +472,8 @@ inode->i_uid = current->fsuid; inode->i_gid = current->fsgid; - sockets_in_use[smp_processor_id()].counter++; + get_cpu_var(sockets_in_use)++; + put_cpu_var(sockets_in_use); return sock; } @@ -511,7 +509,8 @@ if (sock->fasync_list) printk(KERN_ERR "sock_release: fasync list not empty!\n"); - sockets_in_use[smp_processor_id()].counter--; + get_cpu_var(sockets_in_use)--; + put_cpu_var(sockets_in_use); if (!sock->file) { iput(SOCK_INODE(sock)); return; @@ -1851,7 +1850,7 @@ int counter = 0; for (cpu = 0; cpu < NR_CPUS; cpu++) - counter += sockets_in_use[cpu].counter; + counter += per_cpu(sockets_in_use, cpu); /* It can be negative, by the way. 8) */ if (counter < 0) From brandy40augusta@crazyfiles.com Mon Dec 23 07:24:14 2002 Received: with ECARTIS (v1.0.0; list netdev); Mon, 23 Dec 2002 07:24:15 -0800 (PST) Received: from eitp.gov.eg (subnet64.idsc.gov.eg [163.121.64.4] (may be forged)) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBNFOA3v015809 for ; Mon, 23 Dec 2002 07:24:12 -0800 Received: from webmailpro-com.mr.outblaze.com [65.179.9.10] by eitp.gov.eg with ESMTP (SMTPD32-5.04) id A74C720116; Mon, 23 Dec 2002 17:10:04 +0200 Message-ID: <0000187f1c8e$0000633d$00001ade@charlenechoi-com.mr.outblaze.com> To: From: "dawn hellam" Subject: Unlock Every Channel on Your New DSS Satellite System Date: Mon, 23 Dec 2002 10:22:12 -0800 MIME-Version: 1.0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Reply-To: brandy40augusta@crazyfiles.com X-MAILER: Mailman v2.0.4 X-archive-position: 1416 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: brandy40augusta@crazyfiles.com Precedence: bulk X-list: netdev

 

Watch Every Channel on YOUR Satellite System
Regardless of the type or brand

Don't waste $400 - $600 buying AN
After market card when they aren't
guaranteed to stay up.

When you can program your card
yourself in as little as 5-10 min.

Unlock Every Channel on your satellite system

All Sporting Events
All Pay-Per-Views
All Premiums
All NBA
ALL NFL Football
Extra Hidden Channels
100'S of channels
Every Channel is Unlocked


<= b>Find out how= by Clicking here now





Click Linkto be taken 0ff our l= ist



















HJFHS93SLLPO LKKJ903 JF90S93F XXCDFFS







F 4 FG 54G HTHG 65YT G 5YG5GTR THHC83H HDSGF93 GHGGKS98 HSHFS92F SKJS= DHF93








From davem@redhat.com Mon Dec 23 12:17:55 2002 Received: with ECARTIS (v1.0.0; list netdev); Mon, 23 Dec 2002 12:17:57 -0800 (PST) Received: from pizda.ninka.net (IDENT:root@pizda.ninka.net [216.101.162.242]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBNKHs3v022831 for ; Mon, 23 Dec 2002 12:17:54 -0800 Received: from localhost (IDENT:davem@localhost.localdomain [127.0.0.1]) by pizda.ninka.net (8.9.3/8.9.3) with ESMTP id MAA28292; Mon, 23 Dec 2002 12:16:32 -0800 Date: Mon, 23 Dec 2002 12:16:32 -0800 (PST) Message-Id: <20021223.121632.105420794.davem@redhat.com> To: kiran@in.ibm.com Cc: netdev@oss.sgi.com, linux-kernel@vger.kernel.org Subject: Re: [patch] Convert sockets_in_use to use per_cpu areas From: "David S. Miller" In-Reply-To: <20021223190847.G23413@in.ibm.com> References: <20021223190847.G23413@in.ibm.com> X-FalunGong: Information control. X-Mailer: Mew version 2.1 on Emacs 21.1 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-archive-position: 1417 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: davem@redhat.com Precedence: bulk X-list: netdev From: Ravikiran G Thirumalai Date: Mon, 23 Dec 2002 19:08:48 +0530 -static union { - int counter; - char __pad[SMP_CACHE_BYTES]; -} sockets_in_use[NR_CPUS] __cacheline_aligned = {{0}}; +static DEFINE_PER_CPU(int, sockets_in_use); You have to provide an explicit initializer for DEFINE_PER_CPU declarations or you break some platforms with older GCC's which otherwise won't put it into the proper section. From cw@f00f.org Mon Dec 23 15:56:15 2002 Received: with ECARTIS (v1.0.0; list netdev); Mon, 23 Dec 2002 15:56:19 -0800 (PST) Received: from tapu.f00f.org (tapu.f00f.org [202.49.232.129]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBNNuF3v032668 for ; Mon, 23 Dec 2002 15:56:15 -0800 Received: by tapu.f00f.org (Postfix, from userid 1000) id 37B031ED6D9; Mon, 23 Dec 2002 16:00:48 -0800 (PST) Date: Mon, 23 Dec 2002 16:00:48 -0800 From: Chris Wedgwood To: "David S. Miller" Cc: kiran@in.ibm.com, netdev@oss.sgi.com, linux-kernel@vger.kernel.org Subject: Re: [patch] Convert sockets_in_use to use per_cpu areas Message-ID: <20021224000048.GA14346@tapu.f00f.org> References: <20021223190847.G23413@in.ibm.com> <20021223.121632.105420794.davem@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20021223.121632.105420794.davem@redhat.com> User-Agent: Mutt/1.4i X-No-Archive: Yes X-archive-position: 1418 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: cw@f00f.org Precedence: bulk X-list: netdev On Mon, Dec 23, 2002 at 12:16:32PM -0800, David S. Miller wrote: > You have to provide an explicit initializer for DEFINE_PER_CPU > declarations or you break some platforms with older GCC's which > otherwise won't put it into the proper section. I wonder if "some platforms with older GCC's" will ever have these issues resolved... --cw From davem@redhat.com Mon Dec 23 16:25:16 2002 Received: with ECARTIS (v1.0.0; list netdev); Mon, 23 Dec 2002 16:25:23 -0800 (PST) Received: from pizda.ninka.net (IDENT:root@pizda.ninka.net [216.101.162.242]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBO0PF3v000784 for ; Mon, 23 Dec 2002 16:25:16 -0800 Received: from localhost (IDENT:davem@localhost.localdomain [127.0.0.1]) by pizda.ninka.net (8.9.3/8.9.3) with ESMTP id QAA28844; Mon, 23 Dec 2002 16:23:52 -0800 Date: Mon, 23 Dec 2002 16:23:51 -0800 (PST) Message-Id: <20021223.162351.40761424.davem@redhat.com> To: cw@f00f.org Cc: kiran@in.ibm.com, netdev@oss.sgi.com, linux-kernel@vger.kernel.org Subject: Re: [patch] Convert sockets_in_use to use per_cpu areas From: "David S. Miller" In-Reply-To: <20021224000048.GA14346@tapu.f00f.org> References: <20021223190847.G23413@in.ibm.com> <20021223.121632.105420794.davem@redhat.com> <20021224000048.GA14346@tapu.f00f.org> X-FalunGong: Information control. X-Mailer: Mew version 2.1 on Emacs 21.1 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-archive-position: 1419 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: davem@redhat.com Precedence: bulk X-list: netdev From: Chris Wedgwood Date: Mon, 23 Dec 2002 16:00:48 -0800 On Mon, Dec 23, 2002 at 12:16:32PM -0800, David S. Miller wrote: > You have to provide an explicit initializer for DEFINE_PER_CPU > declarations or you break some platforms with older GCC's which > otherwise won't put it into the proper section. I wonder if "some platforms with older GCC's" will ever have these issues resolved... I still don't have gcc-3.2.1 working properly on sparc64. I hope to have it working soon, but this does mean that 2.6.x cannot deprecate it, whereas 2.7.x certainly can. From meg_trplx@yahoo.com.ar Mon Dec 23 19:16:18 2002 Received: with ECARTIS (v1.0.0; list netdev); Mon, 23 Dec 2002 19:16:24 -0800 (PST) Received: from localhost.localdomain (host54.200-43-224.telecom.net.ar [200.43.224.54] (may be forged)) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBO3GF3v002255 for ; Mon, 23 Dec 2002 19:16:17 -0800 Received: from localhost.localdomain (vaio [127.0.0.1]) by localhost.localdomain (8.12.5/8.12.5) with ESMTP id gBO3L8Ad002650 for ; Tue, 24 Dec 2002 00:21:09 -0300 Received: (from mailman@localhost) by localhost.localdomain (8.12.5/8.12.5/Submit) id gBO3L7Gr002646; Tue, 24 Dec 2002 00:21:07 -0300 Date: Tue, 24 Dec 2002 00:21:07 -0300 Message-Id: <200212240321.gBO3L7Gr002646@localhost.localdomain> X-Authentication-Warning: localhost.localdomain: mailman set sender to meg_trplx@yahoo.com.ar using -f From: Meg Reply-To: Meg Subject: Fw: Meg Triplex To: netdev@oss.sgi.com X-archive-position: 1420 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: meg_trplx@yahoo.com.ar Precedence: bulk X-list: netdev > >Hola, mis fotos ya estan online, espero que las disfrutes... >Te muestro todo! Te hago todo! > >http://ar.geocities.com/meg_triplex_xxx/ >http://ar.geocities.com/fotos_privadas_triplex/ > >Solo para adultos! > > > > From kiran@in.ibm.com Mon Dec 23 22:47:18 2002 Received: with ECARTIS (v1.0.0; list netdev); Mon, 23 Dec 2002 22:47:23 -0800 (PST) Received: from e34.co.us.ibm.com (e34.co.us.ibm.com [32.97.110.132]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBO6lH3v003933 for ; Mon, 23 Dec 2002 22:47:17 -0800 Received: from westrelay05.boulder.ibm.com (westrelay05.boulder.ibm.com [9.17.193.33]) by e34.co.us.ibm.com (8.12.2/8.12.2) with ESMTP id gBO6ppf2047720; Tue, 24 Dec 2002 01:51:51 -0500 Received: from asterix.in.ibm.com (asterix.in.ibm.com [9.182.12.249]) by westrelay05.boulder.ibm.com (8.12.3/NCO/VER6.4) with ESMTP id gBO6r3nu028114; Mon, 23 Dec 2002 23:53:05 -0700 Received: (from kirant@localhost) by asterix.in.ibm.com (8.11.2/8.11.2) id gBO6mqf09198; Tue, 24 Dec 2002 12:18:52 +0530 Date: Tue, 24 Dec 2002 12:18:52 +0530 From: Ravikiran G Thirumalai To: "David S. Miller" Cc: netdev@oss.sgi.com, linux-kernel@vger.kernel.org Subject: Re: [patch] Convert sockets_in_use to use per_cpu areas Message-ID: <20021224121852.H23413@in.ibm.com> References: <20021223190847.G23413@in.ibm.com> <20021223.121632.105420794.davem@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20021223.121632.105420794.davem@redhat.com>; from davem@redhat.com on Mon, Dec 23, 2002 at 12:16:32PM -0800 X-archive-position: 1421 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: kiran@in.ibm.com Precedence: bulk X-list: netdev On Mon, Dec 23, 2002 at 12:16:32PM -0800, David S. Miller wrote: >... > -static union { > - int counter; > - char __pad[SMP_CACHE_BYTES]; > -} sockets_in_use[NR_CPUS] __cacheline_aligned = {{0}}; > +static DEFINE_PER_CPU(int, sockets_in_use); > > You have to provide an explicit initializer for DEFINE_PER_CPU > declarations or you break some platforms with older GCC's which > otherwise won't put it into the proper section. > Ok, here's the modified patch... Thanks, Kiran diff -ruN -X dontdiff linux-2.5.52/net/socket.c sockets_in_use-2.5.52/net/socket.c --- linux-2.5.52/net/socket.c Mon Dec 16 07:37:53 2002 +++ sockets_in_use-2.5.52/net/socket.c Tue Dec 24 05:11:36 2002 @@ -189,10 +189,7 @@ * Statistics counters of the socket lists */ -static union { - int counter; - char __pad[SMP_CACHE_BYTES]; -} sockets_in_use[NR_CPUS] __cacheline_aligned = {{0}}; +static DEFINE_PER_CPU(int, sockets_in_use) = 0; /* * Support routines. Move socket addresses back and forth across the kernel/user @@ -475,7 +472,8 @@ inode->i_uid = current->fsuid; inode->i_gid = current->fsgid; - sockets_in_use[smp_processor_id()].counter++; + get_cpu_var(sockets_in_use)++; + put_cpu_var(sockets_in_use); return sock; } @@ -511,7 +509,8 @@ if (sock->fasync_list) printk(KERN_ERR "sock_release: fasync list not empty!\n"); - sockets_in_use[smp_processor_id()].counter--; + get_cpu_var(sockets_in_use)--; + put_cpu_var(sockets_in_use); if (!sock->file) { iput(SOCK_INODE(sock)); return; @@ -1851,7 +1850,7 @@ int counter = 0; for (cpu = 0; cpu < NR_CPUS; cpu++) - counter += sockets_in_use[cpu].counter; + counter += per_cpu(sockets_in_use, cpu); /* It can be negative, by the way. 8) */ if (counter < 0) From davem@redhat.com Mon Dec 23 22:52:13 2002 Received: with ECARTIS (v1.0.0; list netdev); Mon, 23 Dec 2002 22:52:17 -0800 (PST) Received: from pizda.ninka.net (IDENT:root@pizda.ninka.net [216.101.162.242]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBO6qD3v004322 for ; Mon, 23 Dec 2002 22:52:13 -0800 Received: from localhost (IDENT:davem@localhost.localdomain [127.0.0.1]) by pizda.ninka.net (8.9.3/8.9.3) with ESMTP id WAA29389; Mon, 23 Dec 2002 22:50:48 -0800 Date: Mon, 23 Dec 2002 22:50:47 -0800 (PST) Message-Id: <20021223.225047.131927865.davem@redhat.com> To: kiran@in.ibm.com Cc: netdev@oss.sgi.com, linux-kernel@vger.kernel.org Subject: Re: [patch] Convert sockets_in_use to use per_cpu areas From: "David S. Miller" In-Reply-To: <20021224121852.H23413@in.ibm.com> References: <20021223190847.G23413@in.ibm.com> <20021223.121632.105420794.davem@redhat.com> <20021224121852.H23413@in.ibm.com> X-FalunGong: Information control. X-Mailer: Mew version 2.1 on Emacs 21.1 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-archive-position: 1422 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: davem@redhat.com Precedence: bulk X-list: netdev From: Ravikiran G Thirumalai Date: Tue, 24 Dec 2002 12:18:52 +0530 Ok, here's the modified patch... Ok I'll apply this when I get back from vacation in the new year. From juantolomeo@hotmail.com Tue Dec 24 10:55:35 2002 Received: with ECARTIS (v1.0.0; list netdev); Tue, 24 Dec 2002 10:55:40 -0800 (PST) Received: from 169.254.24.24 (adsl-63-196-57-64.dsl.lsan03.pacbell.net [63.196.57.64]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBOItY3v019687 for ; Tue, 24 Dec 2002 10:55:35 -0800 Message-Id: <200212241855.gBOItY3v019687@oss.sgi.com> From: "Juan Tolomeo" To: Subject: Argentino! Mime-Version: 1.0 Date: Tue, 24 Dec 2002 11:00:02 Reply-To: "Juan Tolomeo" X-Mailer: Microsoft Outlook Express 6.00.2600.0000ams Content-Disposition: inline Content-Type: text/plain Content-Transfer-Encoding: 7bit Content-length: 365 X-archive-position: 1423 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: juantolomeo@hotmail.com Precedence: bulk X-list: netdev Hola sos Agentino? Te encontre en una comunidad. Me llamo Juan Tolomeo, soy nuevo. Encontre un sitio para llamar a Buenos Aires desde USA muy barato. Solo cuesta 2 centavos el minuto desde USA. Creo que tambien se puede llamar desde arg y espana... Espero que te sirva, el sitio es http://www.TarjetasTelefonicas.com Saludos! [[HTML alternate version deleted]] From werner@almesberger.net Tue Dec 24 17:46:14 2002 Received: with ECARTIS (v1.0.0; list netdev); Tue, 24 Dec 2002 17:46:16 -0800 (PST) Received: from host.almesberger.net (IDENT:hC7avFxizN7mT6bmOQoYtIPEosHZk6Q8@almesberger.net [63.105.73.239] (may be forged)) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBP1kD3v022182 for ; Tue, 24 Dec 2002 17:46:14 -0800 Received: from almesberger.net (vpnwa-home [10.200.0.2]) by host.almesberger.net (8.9.3/8.9.3) with ESMTP id RAA01538; Tue, 24 Dec 2002 17:50:47 -0800 Received: (from werner@localhost) by almesberger.net (8.11.6/8.11.6) id gBP1oe022378; Tue, 24 Dec 2002 22:50:40 -0300 Date: Tue, 24 Dec 2002 22:50:40 -0300 From: Werner Almesberger To: netdev@oss.sgi.com Cc: chengjin@cs.caltech.edu Subject: snd_cwnd drawn and quartered Message-ID: <20021224225040.A22201@almesberger.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-archive-position: 1424 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: wa@almesberger.net Precedence: bulk X-list: netdev Hi all, how about a little bug for Christmas ? :-) There seems to be a bug in how fast recovery halves cwnd: Linux TCP decrements snd_cwnd for every second incoming ACK until we leave fast recovery. Using the NewReno procedure, recovery ends between the time of the inital loss plus RTT (no further losses), or about one RTT later (if we lose the packet just sent before we detected the initial loss). So, in the worst case (i.e. the drawn-out recovery), cwnd gets decremented for every second incoming ACK during two RTTs. In the first RTT, we get the equivalent of the old cwnd of ACKs, while in the second RTT, we've slowed down, and get only half as many ACKs. So we end up with cwnd being a quarter of its initial value. Now, one could argue that this actually kind of makes sense (i.e. no discontinuity for loss near the end of recovery), but the code quite clearly tries to avoid this case: net/ipv4/tcp_input.c:tcp_cwnd_down: if (decr && tp->snd_cwnd > tp->snd_ssthresh/2) tp->snd_cwnd -= decr; Unfortunately, snd_ssthresh has already been halved at this point, so the test actually does nothing. So I'd suggest to change this to if (decr && tp->snd_cwnd > tp->snd_ssthresh) This was found by Cheng Jin and me in his simulator based on 2.4.18, but the code still seems to be the same in 2.5.53. BTW, ironically, at least gcc 3.1 generates slightly worse code if I change the second line to tp->snd_cwnd--; It may still be desirable to change this for clarity, though. - Werner -- _________________________________________________________________________ / Werner Almesberger, Buenos Aires, Argentina wa@almesberger.net / /_http://www.almesberger.net/____________________________________________/ From chinaalarm@hotmail.com Wed Dec 25 14:32:45 2002 Received: with ECARTIS (v1.0.0; list netdev); Wed, 25 Dec 2002 14:32:47 -0800 (PST) Received: from 110alarm ([61.144.241.34]) by oss.sgi.com (8.12.5/8.12.5) with SMTP id gBPMUl3v003054 for ; Wed, 25 Dec 2002 14:30:48 -0800 Message-Id: <200212252230.gBPMUl3v003054@oss.sgi.com> From: "ÓÀ»ªÖÇÄÜ" To: Subject: =?ISO-8859-1?Q?=C9=F1=C1=CB=A3=AC=D5=E2=D5=C5=CE=DE=D0=CE=CA=A4=D3=D0=D0=CE=B5=C4=CD=F8?= Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="= Multipart Boundary 1226020642" Date: Thu, 26 Dec 2002 06:42:48 GMT X-Priority: 3 X-Mailer: JiXing V1.70 DesignBy JohnnieHuang X-archive-position: 1425 X-ecartis-version: Ecartis v1.0.0 Sender: netdev-bounce@oss.sgi.com Errors-to: netdev-bounce@oss.sgi.com X-original-sender: chinaalarm@hotmail.com Precedence: bulk X-list: netdev This is a multipart MIME message. --= Multipart Boundary 1226020642 Content-Type: text/plain; charset="GB2312" Content-Transfer-Encoding: 7bit ÉñÁË£¬ÕâÕÅÎÞÐÎʤÓÐÐεÄÍø ---ÉñÓ¥SY-995·ÀµÁ±¨¾¯ÏµÍ³ 2002/6/25 ©úÌÒÔ´´å£¬Õâ¸ö×ÔÑǫ̂·ÀµÁÍø·¢Ã÷ÒÔÀ´¹úÄÚµÚÒ»¸ö¹«¿ªÈ«²¿²»×°·ÀµÁÍøµÄ´óÐÍסլÇø£¬ÓÖÒ»´Î»ñµÃÎÒÊС°2002Ä갲ȫÖÇÄÜСÇø±ê±ø¡±µÄÃÀÓþ¡£ ÌÒÔ´´åÊÇÎÒÊÐסլ¾ÖÐ˽¨µÄ³¬´ó¹æÄ£µÄСÇø£¬×ÜÉè¼ÆÈÝÁ¿Îª9800¶à»§¡£ÔÚÕâÀÄã»á¸Ð¾õµ½ÕâÊÇÒ»¸öÍêÈ«×ÔÓÉ¿ª·ÅµÄ¿Õ¼ä£ºÕû¸öСÇø¾¹È»Ã»ÓÐÒ»ÕÅ´«Í³µÄ·ÀµÁÍø£¡ÌÒÔ´´åÊÇÈçºÎ×ß³ö¡°ÌúÁý×Ó¡±µÄÄØ£¿©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú©ú »¹ÊÇÀ´ÕâÀï¿´¿´°É¡£ÕâÊÇÒ»¼ÒÈý·¿Á½ÌüµÄµ¥Ôª¡£ÔÚ´óÌüµÄÌ컨ÉÏÎü¶¥°²×°Ò»¸öÉñÓ¥SY-995IIÖ÷»ú£»´óÃÅ°²×°Ò»¸öÎÞÏßÃÅ´Å£»ÔÚÑǫ̂ÉÏ·ÅÒ»¸öºìÍâ̽²âÆ÷£»Èý¸öÎÔÊҵĴ°»§±ß·Ö±ð·ÅÒ»¸öÄ»Á±Ì½²âÆ÷£»³ø·¿Àï°²ÖÃÁËÒ»¸öúÆøй©̽²âÆ÷ºÍÑ̸Ð̽²âÆ÷£¬¸Ãµ¥ÔªµÄÖ÷È˸æËß¼ÇÕß˵ÅäÉÏÒ»¸öÒ£¿ØÆ÷£¬Ò»Ì×ÎÞÏßÁªÍø·ÀµÁ±¨¾¯Æ÷¾ÍÐγÉÁË£¬Ò»µ©Óо¯Çé·¢Éú£¬Ö÷»ú¼ÈÄÜÔÚÏÖ³¡×Ô¶¯·¢³öÉù¹â±¨¾¯£¬Í¬Ê±ÓÖÄÜ×Ô¶¯½«¾¯ÇéÎÞÏß·¢Éäµ½ÌÒÔ´´åÎïÒµ¹ÜÀí¹«Ë¾Öµ°à±£°²ËæÉíBPºÍÌÒÔ´´å110CASÁªÍø±¨¾¯ÖÐÐÄ£¬´Ó¶øÍê³É×î¿ìËÙ¡¢×îÁ®¼Û¡¢×îʵÓõÄÎÞÏß±¨¾¯¡£ÓÉÓڸòúÆ·ÎÞÏßµÄÌص㣬°²×°¼°Æä¼òÒ×£¬²»ÓÃÀ­Ïß¡¢²»Óÿª²Û£¬ÃÀ¹ÛÓÖʵÓ㻶øÇÒ¾ßÓзÀµÁ±¨¾¯¡¢·´½Ù±¨¾¯¡¢ÃºÆøй©¡¢½ô¼±¾È»¤µÈ¹¦ÄÜ¡£ ÔÚÕâÀïÿ¼Òÿ»§¶¼°²×°ÁËÕâÌ×ÉñÓ¥SY-995ÎÞÏß·ÀµÁ±¨¾¯ÏµÍ³£¬ÕýÊÇÕâÌ×ϵͳ£¬¶à´Î×èÖ¹Á˵ÁÇÔµÈÖΰ²ºÍÐÌÊ°¸¼þ·¢Éú¡£´ÓÌÒÔ´´åÎïÒµ¹ÜÀí¹«Ë¾Á˽⵽£¬¸ÃСÇøÒý½øµÄÕâÌ×ÏȽøµÄÎÞÏßÁªÍø·ÀµÁ±¨¾¯ÏµÍ³ÊÇÓÉÉîÛÚÊÐÓîÊ¢ÓÀ»ªÖÇÄܿƼ¼ÓÐÏÞ¹«Ë¾²ÉÓùú¼ÊÏȽøµÄÎÞÏßÊý×Ö´«¸Ð¼¼Êõ¿ª·¢µÄÒ»Ì׸߿Ƽ¼¡¢¸ßÖÇÄÜ¡¢¶à¹¦ÄܵÄÁªÍø±¨¾¯ÏµÍ³£¬¼¯Ò£¿Ø²Ù×÷¡¢ºìÍâ̽²â¡¢ÏÖ³¡±¨¾¯ÓëÎÞÏß±¨¾¯µÈ¹¦ÄÜÓÚÒ»Ìå¡£¸Ãϵͳ×î´óµÄÌصãÊǽ«ÅÓ´óµÄBP»ú·¢Éą̈ѹËõµ½Ò»¸ö2ÀåÃ×X 2ÀåÃ×µÄÑÚĤµç·ÉÏ£¬¾ßÓÐÊ¡Ç®¡¢Ê¡Á¦¡¢Ê¡ÐÄ¡¢ÊµÓ÷¶Î§¹ã¡¢ÊµÓ÷½±ãµÈÌص㣬¸Ã²úÆ·»¹»ñµÃ¹ú¼Ò¼¶ÖصãвúÆ·µÄ³ÆºÅ¡£¡°±íÃæ¿´À´ÕâÀïûÓзÀµÁÍø£¬¿Éʵ¼ÊÉÏ£¬Ã¿¼Ò¶¼ÓÐÒ»ÕÅÎÞÐÎʤÓÐÐεġ¢Óø߿Ƽ¼±à³ÉµÄ·ÀµÁÍø¡±£¬ÎïÒµ¹ÜÀí¹«Ë¾¸ºÔðÈË˵£¬¡°ÕýÊÇÕâÕÅÍø£¬ÈÃÎÒÃÇΪÌÒÔ´´åµÄÒµÖ÷´´ÔìÁËÒ»¸ö°²È«¡¢¿ª·ÅµÄÉú»î»·¾³¡£¡± ÌÒÔ´´åÖÕÓڳɹ¦µØ×ß³öÁËÌúÁý×Ó£¬¾ÓÈ»°²È«µÃ²»ÄÜÔÙ°²È«£¡ÉñÁË£¬ÉñÓ¥SY-995£¬ÕâÕÅÎÞÐεÄÍø£¡ ¾ÝÁ˽⣬ĿǰÎÒÊÐâù¾°»¨Ô°¡¢Ô°ÁëСÇø¡¢ÄþË®»¨Ô°¡¢ÄÏ·½Ã÷Ö黨԰¡¢ºÏ·Ê½ðÔó»¨Ô°µÈоɶþÊ®¶à¸öСÇøÒѾ­Òý½øÕâÌ×·ÀµÁ±¨¾¯ÏµÍ³£¬Ò»¹²·ÀÖ¹µÁÇÔ°¸¼þÈýÊ®ÓàÆð£¬ÓÐÁ¦µØ´ò»÷Á˵ÁÇÔ·Ý×Ó¡£ ©ú©ú´´ÔìÎÞ¾ÐÎÞÊøµÄ¿Õ¼ä¡¢ÏíÊÜËæÐÄËùÓûµÄÉú»î£¬ÉñÓ¥SY-995ÐÂÒ»´úÎÞÏß¾Ó¼Ò·ÀµÁ±¨¾¯ÏµÍ³£¬ÕæÕý¿ª·Å¡¢°²È«¡¢ÊµÓõÄÖÇÄܾӼҷÀµÁÍø¡£ ¸ÐлÄúä¯ÀÀ±¾ÍøÒ³ http://www.110alarm.com ,Ô¸ÔÚδÀ´µÄÈÕ×ÓÀïÄܸúÄúºÏ×÷! ³ÏÑ°¸÷µØ´úÀíÉÌ,±¾ÍøÕ¾ËùÓвúÆ·¾ù¿É°´Åú·¢¼ÛÓʼÄ,ÑùÆ·Èý¸öÔ°üÍË°ü»»¡£ ========================================================================== ÖÐ×ÊÔ´¹«Ë¾http://www.net001.net ÌṩÓòÃûÏÈ×¢²áºó¸¶¿î;Ö÷»úÏÈ¿ªÍ¨ºóÊÕ·Ñ·þÎñ, ÖÐ;¿ÉÍË¿î¡£ÌØ»ÝÍƼö£ºÓòÃû+100MÖ÷»ú²¢¸½ËÍ5¸ö10MÆóÒµÓÊÏ䣬ÿÄêÖ»Ðè350Ôª£¡£¡ ʹÓü«ÐÇÓʼþȺ·¢£¬ÎÞÐëͨ¹ýÓʼþ·þÎñÆ÷£¬Ö±´ï¶Ô·½ÓÊÏ䣬ËٶȾø¶ÔÒ»Á÷£¡ ÏÂÔØÍøÖ·£ºhttp://www.lovexin.com£¬¸ü¶àÃâ·ÑµÄ³¬¿áÈí¼þµÈÄãÀ´Ï¡­¡­ --= Multipart Boundary 1226020642 Content-Type: application/octet-stream; name="girl smile.gif" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="girl smile.gif" R0lGODlheABaAPeIAAkMB3c9RRk7VYWWdEAkI0lUVIyiqxkxMVpuXXpxckZK NZmRgxUnKJZtdCMVDba0rmFWUV02OVRwexQyQHmLjXdrX3uSk1pOQEM6NYB+ daCZkhAeEyg5ODVSWqG1sWdvYEdhbJWTi2JeVXeBYKGtlLnMy4mAeGdxaSQj FUZNRkI3JUpcVy5FQGFoaA4ZFmmChS5FU4tpbBMNBl1kTJWij3tVWpaEfi8x IkVCOZ+pqiotJm6AalddVGxgWGBDQ6W7wUhZTXpsZ2tNTKmgmZOkezNUaDQo GIxtcihASXyDgLzEvRofE56EiS80KtPU0FJPRRQ0SI2Ffk9EO2V6fY6cnihG Vk5AMIWWf6Oel3FxaJysf1AvMFxQSRMuP662r6yqo4deY3yTnDY5Nn11azA9 MQsTCCQeFo6NimiDj1VkYUMvLCxGS56yul1APT5VWSgoHR4sJ0E6LxMVCZes rlVpbGRmWxolHFFuccfJxGV5Ym5nXaypqV5XTIZ9bm09QkpfYXhRVdjb1lBK PTBNYj1ZZ3lHTiVBU5yUjd/i35Cjo1VVUq7ExMzOy6eHjSQxJZdzeKq7lbq7 t6Cti46ddZOlq2w3PREgGiE+SZOSha7CvW5TUx02PnpyYnt/aqOro25pZjtO U3lfX6qknoZ5c2Z9h1I3NxgaFUNGRlRaTCVJZDUcGl15f5aNhXKNkxsuO2ph TjVJSkFealBFRj4wIniOll5VQqO4uTQtJlpwcYB5ciMcEG+EhYadoXhDSTdG PYl3bmpbTTJCRqSknpiutnVdVk9XPZqjiiMzLZt4f3BuYUtnbqKAhUFdX6e3 jZmmfh1BXkdJPRIrNIOUkzgxIW2AdJSkg7jJxIlhaUdWXqq+wh04R6u8uiFC Xqu6ogkMDVxwY7a5slt1fIFpYnuMdKGanSs/Pqa2lo6JemR0dDxFJjJMVaCw sDMrIHuLg5iOiTg1K5CKgm98fYOYnTZAOTk9MQwXDnCIjxsuMkVBMRUYD1Jq dnB6YK2wq21ER97f2QAAACH/C05FVFNDQVBFMi4wAwEAAAAh/h1CdWlsdCB3 aXRoIEdJRiBNb3ZpZSBHZWFyIDMuMAAh+QQEDQD/ACwAAAAAeABaAAAI/gBJ aBnojMikAePG5UHwB9SmCQ8nQJygTdsmi6DSoHuBa0oaXwzgbCiTz5QOHUvK AMiH4sa0OAoUCILwSg+nPlHgsXJ3qKfPn0B9LrBhos8YPa/4XMAQ592NfEs2 bKhXph6Aq1izasVaptlALQWvIKS2cAUQba4gRqy46SKSAmlwkQo3BRsHMo7s 2DFJD4UcAGVQOFJBD5+gYrV66KkwxoSNnTx/agh6aLLQBUX7VEhaCweOdDeW 2LE0larKraivqmzmFSyRAQh3lHXz0JU2KGvbblpToCOuXej++Dq2wY4jevPo 5QMgZ8mNdPhk1uIzo0KQPibOLTjERHJlDZZ7/lq2zApe5mQ9XgHDEQcHPtGj pwKwitqMKdVYvZWR9NXZpIMJlXVWRBJpQxFum8ACygm7kMIRNmQwsMExZNBD zy1llGEKCu8UpgBiM7zCSS59nGPDAqz81EhlKx7SSHgtYqaHCBBUwIkggkAj xRef0PNOVFFt4AJVWp3WjghYyZAVawQRIVaACDADClpqSXSRRQmCgssL9tgz xR/lMEDGmPSI8YY3+djxxjsyCVJLiMmQaEIUrCzgzgItuggeeOKFdwgrECRw xhlPPHFIBrnUgUcIT7zTCQL0UGNJPfX4wsMZOACQxRl1mMJPCPOgsBV/TcIG JQIOTeQKblAYWI42/uosOEUr9qDDDBm+3DXPPE3kU5IZOqQj0wV8vJLMTXM+ ZidQkzUb1Irw8EBOCIxkUUAINASSACKMPCACHmekwIhUG6SBhSc0mHIGD/xk 4QQj/PAAWFaStEaEk7EtxAxtBOZmCAygwLJCCxRQMAUusJABSwrQNIECS2+w w+YFxC7GSWPn6LRdZJXp+eKK45GnBzynaBCFIlGc4YQseJhATwoYKOFJGsVt QIYOPIBzyycYMEwONXa8M2qTr431DUMdXDKRlRRdYkgHoAQDSwvSUMARLCyk kAI9oS3xxg3vGFaLCK9UYGMu2cGznU8atNgsn+N1DA8fX+SCRySn5EAO/iOK YHE3P3YIo4QjKYADxAlJ5LAyOOR4kUUSeIDDz1VKqiSQFtUYdEVCOxy9b78U bVKFOh0QAgoHvqRhgdV0sDAPw/TgdcMNTeCDA7Gv9HDxGDipnSLbLHbcyAJv d3wIPD2I8EQdIpySwRlKyAIN87Kg8LIdKXgBRBNNvG7HKTycYooZKTyhw1Yk SJJ5NQCOQw0CUXZg4EPaXILE04SAoIwbwZQDhNXUCMfCigEEXxQjBU14B0ye UCw96GEMYxhFieBxp6A060XgeZGejAePXIwhC59AhTBCoAdFiIALhXrDEuph BxaaYkipAYA3tKIkEtAgc09y3/tWwAx1POQi/oYgHSGUoQ8JrIIOoCiAL9Cx iymgIw2nKAYO2pGEbwDhCagQQYhsxDvsmICCaxMPi/a0J+MV7xALgIcNzmEC E4zBg5zQQw8gwA4UFIdSU5lKhmKolRteoRpX2Bw1drCDKAFhDRC5xCVIFwtl SCAcpJhLAQqQAlC0IB7xUAQqoPEEKkgjHh+ogyh7cKwx/KIojqkTGp1FxraB h3gY/E7benKic5yjD30YxRiCUIEZ3eANacpHPYRJKSLxUTU0SGYgr9AOakxB Xx1QxyXqd4kixAIERiQFGtAggT9gww0p4MEJWsADRUBDEYmgAgXiAcJkfIIx vMuOLRewNg0YA4Pg/hnCECRhzz0ZYwitPMSd1BiFNvYhF0GIYw+cIgchlaE0 VdnjMQGgTGa2YxfUoMYdmMGMDqxBaZeoQv4kIIFtooEWpNAHNgpABjH4QhFp 4EIKWpADKoSAAklIQBayoIdkBKExNqCTOyKDz1by6RDGkOVR0cgKVtiyjRkw CicqsNBbbCAfkxJSMekzUWl4lZkZDQcuNuoGdayhflXoQBG1iYYwuPWkdMCG DhyBA1RwQRGKyEAOKEEOSlAgC+UURApk8QQRfCIXGQAjGmU5hEbwcwhJTeqe sLDUP7FijZmJ6hiSUYd3/KgMLaxHaSglUT5Kw6IYDeA3lNFRadZPHUUw/uI2 3RoGA4RBHujARhPeYFcIyIIHSVhHDoZ7BsCmwDM4gAY0eFCHOM2pThqApVGn u1R3lOepbOxDJ+Iognc0wQ5ysMNI9BjRiQKgau14wS5eEI473OEP0fyo6AZB iJKioRW1ta0B5IGGP7xjHqh4giL4IIIzCFe40mjHCUTAgycwDBrlU0Rz36g2 MX5Hsv6050/s1NQoFLQoGeAsHwRxg1usUCpVGQlVjMnHgtnjBdRYhXtZ24Eq nNVp1lzFbPWbX2ngonzQCDAf6nAGNuxhuDZtRwI4ywM+KFe5AhZBHcaQgROF kW0B7cnGrGseE2Tgg3poMDTo8QZHLKGhlkhx/jFLmxUlYWW9U1hFe5UBAmYU oQqGEMBrY7tN/Brgz7wwwDAoQYsWoCKLT4DAB0Jw5BwIY4SYuEIuOpGFDyaD swGGBj44yYUe5CIKyroyn0LACkycYxS54GkdeHDoFPgCB2TgHkr00lAUj5bN qYlHnNt7h1jEogg1niYULpGKWOjjvm7985+HwewwnKDJIuCDHrIQAuHqjQYh CMEZMpCBTnC7jeNQtQiUK4gnVABtQr2yQMsThc3qgQ9PEIQvXA0NX9jbF/T4 rh3MsIEgPfTWE51COCRwB2Uo49cdGMQ0b3OJQRi7rclWNrOHYQByRpsPyUiA Bvjh6ERogAZngAcb/gtqy3N82IMzggAEenBuOkHGJ9YtaAb0UIcZxBsaB9Qa AYGQgmL4QgdvgENUxKtm0k40HPowOAh+PYgqVEEAUIh6w2NRUtoKurYTr3gL lCeIaVf7yOSwqbbb6OGcnIHdbfRgAirwiZ/OSSdN5bCHTZALzjJPynVAwAdO sNNQ8gAIZDgGSoKkYtKyuEhKWvqvi1CEVDwjz1GX+iCwCfH8GoANmKc4OlrA h3eYAQNE5odww96TEMCjoB5u6lC5TNA2mmAUc8pJU8tDlNd/GbFQzQkm4HH6 DOxUBLIgwztQIpWUiFbFp1GJm7NCiMY33vECiH7ku1BNqkNc2Zf/ARv+/nyC OkDjHbp4gwjgIXph2NQdIXAHPKKK7sdYtzynn3sbYQ9quJ8e1blArBejkDF4 nAETmBAFHoQegoABw2cHS4BVKUZexzQIqfCAz/AM0Qd1kQcFAjB5VWd5w6B9 zEYJCcAD7JGAOGACXsAPnoAFIbB7URAEPQBvT8AFfNAD3FZ2QTV/r+dhNmBd 7mByuEQisPd28JBtQ3V6EJQMeiAITREaG9BQgFEPaVYVh5cVZQADjheBE1iB XQAFhkBfGZhfbMCBlEAJLaAIFuIXcoAKXrAOnkACf2IePfAETWAK87AEVzEP b2AKKVAHCYA2r9dGNkBBQ/UnUdCDB0V/X1YH/p/wCXrwCSIgAsnDB+wxDV2j YoABUVwlhVdRBVYofRQYdVmoDYNQBMfWVn9GCV/IBpRAcS2QBvjwDm8gBzLQ BCXoCcLQE9ZlAgmgBwwjBiphBy7AAfGweT2QAKNQjKMAD0wwVK9kA0aRf6OA KCKgC74ShzogBjqgQIURB01hYgkoWk1oCit2GkUCAIawidI3fROghVXwcPiF dZmHivKQAD3iiroQGLMoCh3DZSbwCdAgCzqQD3IAB1SDDgWAAbLgejaQg7KU RriEHV8mCG8gA0oiXvamA7DYHAp0C6GBAkO3R/kwJFGYFUhwhZ0IBRLxiaFo X8k2DJhnipQQBtJw/gI48A4cKQOm4AUe8AUkEB6s4GG5kDwQoAinIALokAB1 oAhSoAaxpxPu8CIAxZC41EYQYAbs8BeAcQon8BEAKQfA8g63cAsogIBYtWL5 wICpcRsViI4T0AVdAIqiOFvuaIrDQAUJQAXYgAN94Rf5wA9e8AWeUEYx52UZ MAo5lQSjsId7eIyr5w4agAX6NAR/EpX8h2pBkIjMc5SK8ASysJmCgAM4ggM6 EJb5AFpkCZIZsnxYsQnTBwVs2ZpsSRGG8JbbtGyDxgbkgFLyIJQ/8jBl4Akm +AWUZYvqFwWoBnuDmX8JEEEmsHrioU+iAJmsUIyOQRT591MJQA25oFOf/gAB XHBXKPQEUrBbdgRaLgAYZSlMcnCJWLEqWegKWZiFrLmWFFEFskkLWDdoFGcP SXAGf5ACYrAEKACOvrkHolBdgYlqe+iMsAeIPdEIj/mcf+J6FGQeEOSMSZAB OnV3NHIBmUlmYUkSLfRQZZmeuAYAqwIFtuGerPmabKkNIqUM22SfgiZoOWAA 8UAOFJAGsiA+S5ASntANOQCcfmKLaPdhORGIfeKYQ/CcGuAOEsplUZCce4go O/UJ7tRLXAABT4ABb6AXlPIGKlaW4SiOV3Gi7emaXSCfUPCiMRoGlCBoFBcP 0hAJSUCQp7AXo0kDHmCClBUeZGSLgNoxZKSk/qJQqJOBg/DwEzH3YYLpZR7U A8nzfV0qTHoxH3hEoqhpolHnCuz5nvIZDQ3HZzFKm2dACuTwA8BhTg8jFVfA D1+wB8G5JxgEUEe1VI1ADhpADs5ZqADlDgmZMctiGax3eqc3f5+gGLLglXdY EnYAGBmyQsYkjie6Ku6JpmkKBaUzirRAC3/GBsNgDwnAksFoTggoFUnAD3zK Jy+CBX06BH3aSrhKRkvKqwJ1enEnqKXnDpjACmfHewWVC4u4pbv1QiyUIVRh ClOxHFrBqeyZFmnamhMAqikZo9yaipQwBUlgC6iKDiswDyjhK9SArsDZJ42p Afpksih7snvSWA9K/q9N9YcnIkbk4BPZln76OAqK+ATQcAs6YArCJEzzIaLe YBVkaqZdUK2uGQ3UN7Ftxa1/RgXxkAM/kAMGwwMe+4/1gAC+yQ+16qC5+pi6 OgS6mrJOuasAZVl/qDbSRQ7kgAXxqqj/WkKd2bNXJVpUgSYukJ70cRrsuaLW 2gXRACvW11ZooGy0QAF78AN+lUlm4jD5wAM/SovvClBg+5j6pKsOOrn69AX6 1AiXBXePURlj25huG5wCVVA4q6U4cAt2ABX1sEL6UUwlkXwmerSuuZYQOwHE Nri0YA9/Fga0QAXaRwkWQApW+wa7tQQpQAPo+q4o+7WUK7YnS6saZLaT/uEO qKcsotu2woAFNMCuwoq6QaA8M8lvQdJC5lkVaTYvJloRCDI/EhG/l9ABlLet hWtbFHB5bKBOmTQPdnASS+ALG7eGsZqy0hu2KlurZjsEdkIUqbcdJtu2WNC9 4Nuk2Pt6CUC+YoCAptBCIVoGIImw+BEM/4IEMIAESLAGhoDCSGA/6kAI+qBj YXBSthUPCZAD3hoGSZAGY8aRb6ADN5AEfknBBoy5l/uYlsEE9HQIm/ucjXUI h5mcJvA77DoEj8auwRlzxZjBXIADZtLBxQSOQzsVLgCOWKEO6gADaLzGbGxW bmBs2jTDwUsL6IAOLSkPu8ADFrJv/+sIWUAC/rRIq+ARtmHrteTwIgugxA3q nJwLHu4wCj9lSjELHliMxdfbZR6kPPQQB10KJA9Vmg1lGgDgBqTsBqBQyqdM yqDQAcygDJC0raRgn0nQAnZMcbTAuGRmBl5jByJAA1csq5SrqwjMJ5fVMcSz ufrUEwilS42hyKRbyesWVLiUBcqDD7tlB3ZkB1B4FcNUDw11GtgQztjgTeIc zvsyzriADid1y7QgDaRAyzjMvwWAl6HBEtgTAt3LSmQ0tlhwq7R0zCzrnMns DmNAjKNQAaMQut9hyevWZWPACTMgCPhwA0C3BJawQhGVIQk7EipBBy3QAh6d BiI90nUg0h+ADvGQ/gRVEw84hQ6KELWUIA8U0ALHlW+v2KN2UAchsCf0hCds kyc9ocQ20FiSULYnK1DE2EYIrUZNlW00UBnpB3+6xAnJ0MUb6QhvMCkYrRJl 0FBy8M0AcAIn8AHfINZjPdYfoHcIcAI7EA/tQAGftE50kAZJEIZ4nAYFadMJ CBXQcAUhYIvbYSduM0s8YV02YLKOZbn6hCKn1EZBMAouxwrZVnptGFRfFkdI 6BdekxeiZXzOKgcPlRIAQEg7INalnQV8h9pjvQPUMA7tUDUJQAEuTdeUQAW3 LJTz4FmheWZ2gAPPM3snsgx2MlTJaLNaNhT01AgOisSNYAOj0NgRVH+8/ifZ sxd/RbFZPRAHfrES/xtaocXVIwHWWZAACbADH5QF+7ADO7UP6L0DGZAEV3AG 65QEn6AILZAEVKBOLm2A72CAKoQCTzAGSZALp1R7CZmQszd7d0JPt7RGsNRY 9GQDvzDhEATZ/gpqCW5yBhVHXOAXShJeczUVKwS0ZbBC31wGnZCdO9AJY8Di ncDe+zAGMf7ert0OGUANhvYBKk0Bu9ACXswFy+ncWSAM7qoBNlABQgAMwCAE ryAOv9AHp+RF2QGzt/QLjPEL2sFGvyAOxMByECTlDflUBiVBFcAHt2AGZuAA EmkKKtSzlPKRRBsVnwwAk9YJdn7nLt7i6T0O/kkwDlcQBdSA2oqg4wWTBLgw D4KQAW2jT0R+shI+BjYgDsBgBVLw5E8OQRfTRiZCFFIFQRQuDmNgNpzg5CSi XbnACZwgQbdUFL8wBhAwC7fABcB3h8DC5j77yXuEsHgEAHde53j+6xkwDu/d Dp3Ad3lIDW/d4zggAiY7r4UKvuIxPEduBcTQ6lOVPD0QT08F5WZjI6g+BuIQ BOJQAaAOQQ/NCVlgI7mEHUZRAXHADhDwC5DcA/54C+Oj66NJJBsAjt7obXfe bb/OCXYu7Alx43mwD3n4ODzu4wngmKLwAF/wnI7Z7Cq7DBVQC6jOCcDQmVLA GNjxqybQ6gm1S63+/tDisDtFKEe500VT3QNmcAEJTZ2fgANNYB+m4LP1MLRU kSa7jucA/+vbNfDcduMfMAMpwHcpTQ1pkALwIApfEPFOHL2KjUZWwAc2oh58 UAsX0AO/kJBDoUYS3upObkpiv+UP3QPAwAUUA09mIwJWMAu/wJjXGwXjhgIO kIAkang4bwoobufaBfRAf+MZsAMfwGp1QA2YhA7MdQhQr9hIjMTRJQhSAAwO hBSvQDHAIA4IPhRrROEUDu4Tzgm8pPZ8wAdcwPavUAtxIAjncAjQjgmf8AR3 2KNf/dUrBsZL4AKAz+IjMAJ2LvD+fuOdEEqoUAdZgA7UUPjwIAy82rJR/p/A hwAMW5AYU7UYPaD2QhAK4vAI05mQfRADp0Thox4KwOADbSAEiiEi7Z76s9AD rLAnTx0CY6AI7AAVAFkVX/2sS3hVu3/nAu/7ANFJYCdqWT7MQIXgBDpqdb4J E4VFlKghE0V9oThEowYNhzQsGGPFyqsxYzidrKBJyEpNNUKJizHGxC+aMSoQ I+ajzZYtQohVqJBsTKc+FV5dmNbn0KEQIZjSyCDilq4l+eTIsVqvTJklS+ot 2dDp5D5O+0ZwOls2y75O+3aM2XEiC4IZQHggyJIkhLAhfCFa/KJRsAYmrBbY OMxJhZWTJn/1GfUrFDFALolpaqnJh49SnSNE/mjTBtjPCpz69DH5qpYVfAuW woMdBVOUOjjMoLCKdevWfGC77ls7Zp9JsvuMqwWeRW6eOqic10lCQxgWLH4n Ypw4+JCNKDZMID7HRU3pxn1MRDkiThwxl6GuXQtVQ+XmNj5Ghyo95rRJTnwu 4JPCNWPgScKEJOAxoQcc3tDFqjIc3GqDrt6wg6y1kilun2Q2TOYD49Y64YQP EODBOQRoIIc6i1bETqNGtjPhtFFcW4CTOPTgxKQxcvnOBBNiCCKUUMAIJYgY QlGpn36EAKrJHFHjJJkeahFEhVoOcQ2eXGQ6J0Ep3jDjKqzk2E3CJexw5IMO 1fygTQ3r2GeGNtXM/gOBD+ooEZUPkiBnur8u+gK77IY47Jw+OKlgRo5MEKQH ToJwzEcbzjnnF3GEdC++HnoQUpwggIrStEOTeYUPQfCZRoSPWDlHSh/H4EMK dnQRM7et5OhqiRvSSEMaGqT5pg4EhOWh2Blm4AFZHlbgAYhO9sJiiEADfeAB ajHaIyMs+xBnllrGYIWjQ4Lgo4KScnyMUkpHyeWXT98t6Vz+OCGqkx74KAYp XRwQgRVWuAtyjCCCgICLWfJxoNZbcUWhqxXQsWWbHNA5oZ0kEAAi44xX2HiF FRCgbjp+pv3igYtMXpGij/rowYE4gBljgZW56MFcHWka5bsofBxl4Aog/s3x JKHj7a+W/1CQQQ5BTDjDHXdMGOPnIPTgI46Er8pnAzKT7gq3PyzIZhFbKJDG g0QocAMIUDJeG4i0V2hnuolMxqjkaenOdiN4+oBABpeJ0WCIQzK4gA8c493x tO8W+DfGURAPWkeTKjBaEKtlkAGDLEaxofMxQvmFEz1EwCdhWssg89Z8UGiY Dl4WWSSTRDxYZBtK3IAFFhZYyF2dDmBBx5PqrrvIoopSzg6LQ87hBHMpRtFo AUa5AMbcX/S7/hcTzukcnih29hG1XIJujJNX8lVgGjkwf+eTZH4Z5YggNEmg AqpxoHWJq3ZDXY6Gl6AA7WAHuxL8wAMv+AMs/jaxiXJcohwsYEYi+BEtiUSL IkvxSEUEtZGOuIML7+iBDYbwkZBcoBaO6gNNVEgT86TQhadBzX5QohpBpGMa KJADAADwhizoQQpxkAL9oCYCPuCDVvtD3W56U5VERCwTmVhECRaRDQ/IgwLK 2MQBtLgJFoCCCsKjjuAOgYWOYOkQgkHjEKjTERPoQQ828MgCKgDEwpkrhSyE HwxP85j9lASG9jMaqoyAgjLo0A5j6IEPVCAFcXxnDHV4ghHFlLpbdSUf+cjB OjxgCyhGMRu2oII0cFGOaERjgRCkQSKixRHXxNFfrglcRTRCnTUiKAg2+IgN KjALINYCR3c8CU3G/vCLLQ3zNE+CofmOUox03KBhMtBhPvRwryB+hxUmEMET FLCEI/IPdRK6ZDsoQAUPbGMb2cjGNmxhAXGG4wAL5IAbqLCOECzgRYfxl9O8 Bw/DZDCMtCRjCAy1gAWIAxhvMMI04lCLV3CCJigBShAepaMYJpNyF2CNM2ml Qx2+Awel4IIJ4MGEM5igDiIQRMOQmLoNaA0r8YhHO2YXsXT+YA6UWEciQLGG YCBhBTgNAWL85b18sgIeNmCFO2RGHRJUZwgkEEZHWGUDll3ACFedhiD4QAyH cst+EdWRqPpgqJMcRQr4UMEgN8rRfJiBHXqIgtPcEZU6FAOHuPImVjZQ/g85 oAOmZdvkOc1pi0++ABdpwAY2qJADcnTHBrB5rGHc8T1+usMjYYTqdLBAg+0U BRhSGORVrSCI6jk0dHoAlUSN2cKivAIYCrCCCtKKw0JyFACZM4HTDnGGUYig DoK4gZj4x1et8RWmu5AGTjeZjUz8QLChPEMSWhA3Dezse1GATVLdsc+kXlZF wgDvdD7yiynFwQxXVcUsVEBaoSEKtfk5CWvNZzTWqOCGtIImR71xiwSc4RDu IMfTePABBdw1dSzV2lZ2kYRd7KId8pjDJm3xA+faYh2ZbOyBZuOj83QnCqwI gb+wCw+nGOO7oggvOTQAj1H0YBYOMIIurqpe/itcABiveIX93muu9uqBvmiV 7Q3XZ1sAnCIK/vUvOUKAzQ/gw8C3+iYlGzxlmSZiDhFWpwfWsQ4vrKOkIv2e j64bVxAbFbus8K4wPCEKT3gCvFgIQRSCEAcZq0LGRpCtFYpRixs7SWg3ca0g rADbdNg3xlch8ieSTA4quCO6vqWHgckkB77KYQO7ScJxdxFTC0hDGldexzDW YYuInSEKBRrzdfmJCXeEADawKXF1wutm8T4NGDCGsRHUuxiRmBDHOQYKMUpF JZFYodCG1gXmbOuNM5xByf6lgql5UIc4vEG4ZNJagrdy3HY0+AVTtoCV18EG UtPT1NvTmYe5w88Q/mBi1SHgiAbUzGY1C4McrxnFVXON58UUo8b5uoAJazHw CwhaJPiIrWyvmuza6vANi6aCU0oqAtK94clRpuRx40GNKVDDwdSghjTWcdNE JIIXwqDBuTus7hG7AxNJBbFHNCBrjNSbs9sFhhHsPGNei0QQgoYtbAtebJGk I7bT0Hd+dSiGDDiFHIx2ylxFgIo43MDaMkiipfVXyDLAFB1fZ8g42tEOalwh EZSYQ8nPnoNoc3jl2D2zO5KKiQU4ZeayprW9NRACd/zCClflebExilHB15jo CFfBLJCOX46a4ROWffq9Q0COM0TlCSpA6KTLoBVcaaWQSVhIC1qAiyk4/pid 0pCHla88BzZcOOLtUNx5uHMOfvLTX5hoyhijBZHwYiHilXfxnfFM9MH3mujF LrQgD21bHMQ18nCO9hlyIYJivMPqWKe0pcuQYACUQZQUo8M3qJEEX3maCmqn RA4mrOVEVD4XuTDP9iZFKdsbdQHuuDveUc4UTDDPqoEvPMIzPAE0uiBTq4Yz gwQgKY8gBxoIgTN4wE74rRu4ASOAJr7aPErbCh3yAgpoga9LgjOgAmHIgRyg hF9JJWE4v0SgghZMBBVMgqEwD0qZv+w6g+7KP/BSJRroiC7phJzTtZ47PsM7 PtlSOF04HY5SQKeBNxrwvQeEh2TggwKrwBzS/kDO20AAACVciAfyI4dEyIE9 yAEVpAEqoAFPULuSy4E5yAFPaIdcyAKi8BF1eawbZLWlmDlhyKwGVDJ4MJQK uKo8E4meCzgiRD4DRIG1AoB6gIemYQpGCyhT+4RawAEUsLr1ObDN2w0AsJ14 IBsqIIc2u7AypIEroAIS8IQSLEE2bMEoGIcMkImx2h5Vyyc8pCXwogIN4KwF oBTF4DeiE8IhlC3FW7iE4SgxYIJDSEYq8IgWfMB28DEjegN2OLB6wMLa+gE2 kIYW/MJUbLMcwAJPCAEqwIJEQENP8zSLaYcQaAc/ipF0GzFWs6wxAi8a4MEQ oIH+qxQr2LViE0SR/lgNf0y8aSDIQyOT7hMBADuEpzuEXNytdjCpJ4gDFHiD 4MK6DNRErosYNmC0VCKBNlyHbtiyC8uBREiAAoApWGgBC2jBK0iCDAiCZOgq dUkC+mM1eLs7J+TBiJuNc3iFxFuMNhCJfjy+Y9M1wNuXIaOHKFiKe2vGEFCE djgDWRAH4GIdaysDaNoNDSykCduGOfA0c/SELeOHLZMYtksCRfCrFfjAM6CA JCCIdsiCT9CP+FMXWGMKjsCCFOEIewwxeKiAhCPCWVAvozO2QbQvwkRKOUiY fFAE/2IKpsiAEGyCEOQCTfgEMliCN7A4rNPKjNQhCcuBbWxDD/AEfsik/nX4 pNEkh3jgFYdoAR5IggQYvzPohE+ogNhTl3N4ObubOZ1UsaVwtTHgAkHEqCAU QiOchRmLMSNwgH1BgSCYx73LgBQQgxR4g2+gAsyUBfN6gyZ4g9sqA2/ovk0s g4jJKWkYSzYsSSqgBHKIhG1YB0qgBOk6gWJBBRxQhBaIghBIAuX4pfjbnuzK vfzbLN/roD4ghtiygoAMxMWALXwAsmlQzIVDQl3AgFEAMQg8gxN4ghRwBDJI giSAgCCQAjWgCtbhKK7jutAcORb0tDDcMkajhEhIp22Ihz8olmKBhiaQBS6A gJM6gXHIAhEQClk8DwINgVt0ws2SqigYA8JrnIOBK7ThE4SgM8IsBbyrCikb yACDyIB2gAdfSQFG04RQSAAxoMg3yAet1K8sZD8KuM8WmKccsKLkorBt+AFp iE0eSAEEaIFTeAJZeAIcgIDJ/AA+KBcuGdBGxMkDRbmn27ug6oNamNKBW43E HDSBnIY8G0ZdmwUuGIUoSIZcSAFUwIQzoAF0IINd2AMkEQFZsAPwvAql2wry 1KGAAAAh+QQEDQD/ACwAAAAAeABaAAAI/gBJaBnojMikAePG5UGwAtSlTRMm QIk4QduEh6DS4JoSbsofWMcYbNjw5g09evnKlFlyIw4+fMUu8JlRYUwfE/BY sXJ3qOchDT99ChW6gJWNPn3GJNPzCoICHO/eoLAzsl6ZegCyat26tQwAlQCa ESx4BSG1hcyAWNwEkaI2V9o2gfozZdcUXCtgwbG0AY4dMjjoqVyC4t1LBTJn vOJ08xyrBTyHAu05mXJPyAvOmUBaoUKPV4JwpDOyhGpVq1xTf80KtplYLQUH INyxcIWbSxYtup3AQVuwP/GmtPIYrBwLO3bozaP3Jp+dNzfwKRD0RHEyxibO 2dg5NKiG7z/B/h9qdMidO3hRRiHlVIEphpM3lsAZuaGeN6+qzZhivVqlQIIG IaQQQ0CwRZErB2qDBCwrTFHXKh/5woEdjviCDxlLmILCG01Id0EtPehRAScm 4LQTT+T1RJ4GKQalYk+sjKIHBHxcJ8iNgnzxwTs35LNEfVXhl5UM+EWRhlZe JfmaM2QJWBsocU3QVkSbaKMNDLAUsJE9d61QDhkckKEAPewQZsct6UhxgUw9 jDhGdjot4JN4jQAlnnjlwQNBAmec8cQTh2SQQR14hJDCDQl8MA81ltRTjy88 nIFDGVmcUYcp/Jwxjx2rsWbOQLAd5CQCf7jBVhddtAWRglhmhI4F/i/g4sY8 xxzjCzRkSAWdYYLUAsErenASxJuOQVbeeJTZaRl4k8HDgzAhMJKFIiHQ4E8C iDDyAAR4nJECI3zVg4AG/NBgiqX8JOAEI/zw0J9KkoBqUFnj0EaqQwhS1EVF l8AASjnHnILOLrvg8kc5vqTgCz0KKOCIYdPNIEKbnOSCVBQLGFsZi991vLF4 rHwCDzQ0RKFIEmcEUgAeJjSBwTuReMLDBlSRoQMP4DTxiRgpQEMONXa8w19W 8WpBBBHVCEjNNwyZCtG+UCCojSHquMEBB/O0QPALdADhiwIppIAAEPM0/AQf r7Q57C+jmHiInD7V6XHHLlbm7Be54KHE/il7kMPIE1jgEQk/S3iiBBwpgKPI CUl44sQpkZDjRRZJ4AEOP+8CUHQ1RNBbL9PMdBDXvrldgoQ6hGBTDtan7PLC XWEr8MQH8dSRAiqozKRHMmPYVCJOGS/bsdx0hyceJj2I8EQdIpyCMh6y4CAC BCjRk0I+KXgBRBNNzJOCHafwcIopS6TwhA74gSUQ55wrzXRDm+wL19QwdEAI CCDAEsw8ZGwUKyjFQMUT+rQ0EdShDrvjxBhy8btivS08dSKe3IjnsZ7AIxdj yIIeCiAMDdRBERCARlR8ZIcy0CwfI/GGagCgwqGVgQbViOEVrjAOatiQVHlp izaqoI4iEEIZ/voIRziwAYoUiMEXQFgBD3iAO0XkgBzSkEY8OpGFT+wuGUEw QRRs4Ji3fYeCQ5ibncaoLHewAh7nOEcG1KMUPsThHUsoDX1G0ighCQlJSCKB DGdYQ9rcYQXMgAVbrDSIQeBPAuFYBSn+UABQzIN783gCDxTxhCxQghJUoIIJ EnACPvABAiIQwRgyEIWcuKMRCxCjKo03t/IY5RxRKFEuflEBN6KkNHzJhwvo c0euJEkrNAjmDK+wC2rsYBUIYIYb1rAqQxQhFiCQgCLRQAo6YKMAvnDEG95x ChGg4mQ5yAElyJGELPDgCdCABnVEkIxcZAAemJBTxjSQSlV6LIx4/jIjGjVj ggzkIhmowMc7SAikgmJlhb4U5hXa0Y5iruIbzAgdM7VxCUPcTwISIAUa0GAP CdChAEAgTBNQwQNPniGcOUhEEk6giFPgAAcYwAE0UIFAYsHDO6kMozHstFMx 9sQ8Z4xCLDPQzw8U440joUo9wlWP+vQSodK4gjQY6rpV4OIOylTHRA0xCGVk dKNhCCstdsHIDaAggCLgQh1osI4ctBWKJqjiAVGRzpdC4wmoYCcDgxeeZOEJ bkBFj6B28IE0KEIBN5BKfUoDABfwpT6OQihXotiOVhSTGlfFagfW8BCqFUGa YA2DAcIgD1rQwQ0s4cMTlFcHcuwhnOQg/kcIzpCEDGYhGd4UoALm8dK86iEI NsiYPMkoFHdgAj0myEUW6oAAJkJDAU2wQ2nkQDNLeGOpQIqsZLUiDQo01KF3 uIMyOtCBKuCmCvaTwEZbIVoD0GK06CjAG2R3NlTogRxthS0NzhCFDPRuDAnI wm1n8IRi4AAfMwXGm4IrT8rIsyjwGEUGNDhJIITNF/OYhyOk+4aklnCpjRoJ WLYLAApQwHXUsKp4Q2deK1XhmasIrQFGO2MKpAGJgsgrH7IgjD28lhw0mO0Z TNAJQZVIMxnUA4EFIQsu6IGBUdgJ3HyiE3jEVQ/eTCc0eoZhXzRhKo6gLlXu Y4lGNRUrT00N/sEcFA4JKCMWhCBEFQwhAIr2UL1oCOuMR0uJYRggHmkw38Qq 1mPYyjYEyNWiUHXCCqFuxr9B+EQQoLwdwAY1uVbswQH18IEPZGEfn4ZGOt6B IZptQCVNze6IJRsPUlhVAneIRejKixsoXGIQsdAHGtjb3hn72b2B5oMI9JCA DGDBE0+kQgjcsWxHb1En7oBwoteYCwaeAx6mNKMNSjSKXAQhCwCutpFhmQEu BIYeG/jRqVVCn6biEaEcwagylEGIIgxizlaydSpiod6whuHXBhjGMCgRhhas tilZyAW5PBHbn7pj2yUq5U4mjm30xNIEoxiFUOGxnYdbGeMYzEXG/ktUcYvz ITRx+NGPUL0BM7sgzanBhZuVAYJYFKG8VRAAFHZ+62jmWc++ZgMbhlFwHoiA D9DQwxjYmgNPaMA85hlFAoIwrG6XEts6sYGjf6dFG5jS0dUWedsUnZPjwhIC F5BCHFAQR6eakC8usI9kyyADAMTC5kUoQipS8Qw675znhuw30A3Ahh8MXR4t YB4OdIGBLISAH/wQBg3MGIUEcAGOcthPVsSggwKIgJRaxPiRt3MiNPZhlmMY eykv7s8AX+ClcPSRHJaA6vmc2gVdWWHe8773Zwjg9zqHQhcu8UzBt3cYPzC8 AajQAm/GwQFvqEMIvOAJhh/CKLkI4S3s/tAErRQgBdRAxyeykPHym8Dr3DkE epJS7bGPQhBiwAAGxBCYOLwxDreIYxxLeBU7mPnUqnFHg9B7z+B7v/d3wicA gfdzezYMhccGM4YOyvMOpiAHUgAPXsAPniAMh7Bs8DAGnyAC0IASWTEP8ZBJ EigCYtc2jsETQNFoffALFiODFRAHdTcP9zAPvnAMcLAEukAYbLcE9cB/S6VU TjV3MFCABQh8CNgFUMBVhKBevDZjDzhwbJAAivAyKEAkT+AF67CB4KFPyaVp oZQASZAECdACEKB2QoV+4YEFQAEPMXgTfSAFZtAcMuACzgNop4ACbGcGbCdd VGFCLWcaADgk/rkHAFWghAcYfDuHKtowCEWgawxoAIYHgQIXX/TwDiggB2XQ BF5AfR0EHmdUIhmQAKj4CamYAKMwBhVwbY/hgkMwBFigfknRNqMwMTMiC7Lw CR8UGDfQBLegA1JBFf53FS0XLoeIiOkDAEhggEz4d/vSBYZQBbn2c6IlcGzQ Z2yQA58gCzyCAvkAAG/AD+vwBcIAhzDCCqYYcSWSBEfQbSLXcS6IBUMgCrW4 AH3QNkLVbRmUAJ/wCc03SSL0MjcwFcZ4FakWYqiRGviBBEzoiBMxAaSTCp+1 XmHlgNsodMOQBoGhA2y3EqH4BaIgHvqUHu70O1KHim1jHnCjAVgg/gqiMARv sxlaZGXVlgAAJiin+AlPgAGhQQ9++CN24CgLeWZ31EubgIBNGBGoIgAvRok/ 54AamQMGUAAfGUcm5An8sAeeAIeVwRNMAA/8tXVbZB6UITf3OJNPd3FRIIYA lgUJoFy3pQeKkFY4QIxF6X+OklR92ZCIuBpL2YSoUpioUlFFoAwYGXBDZ3g5 EAayMA8YoAOyVw8e0A2Rp47dUR7ucAZMcAYnMifhMYv32EHuEEuwCFQg1wkA ZltkCA3T8AZFuQGOUFDKeFCpQSSu8IjC15vCR5FdsEOTuJgCR5U5AGgpABXU ZRVc6QlfgE/K0ko8ERlz0jGzSJIl+XBb/nQOkSGGsSRh3aYHViQIzNFhtHlq dGRmqPFUZYAgTliYwocqwLlD97NReRZw2sgGVJAA6NBS70Bd+VAPntANOfAF 6sgsrSSaZKQBpCmTolAeWpeaDhdUQiVUYxAEyfCTstlUG9aXc7SeXEEkT9Ob hjmNwWmNiglWfkYJQmcACZAG8dBSTVAfKUED3VB9B2onQ0AODAqd9tSja0mT TGADXgcPwaMsITBbYuhtIiAITfAGS2AJdjAffSkSIvZLXbGbEzERqPKbhXlr F1mJVEkF+tAC8bACOPBlTVUG0rAOXZmjdTKLDAqkYVSnQ9AI+LSWM3l92OZ1 cmIn5OAiSXoe/v6UVu+gA+rmCB5qiFjaFUspEU5Zol3aAfy2UbRwfH5GCy2Q AIA2mc3hKFfQDV5Zi5MBk/Zop3Nap3H6HbMok7Mobc/mghrAoxUUAuyYCz3A B5NpCummVI6yFyLWKVshB/MjnxQRqcOHa3gGdAJnABTQAhQQD9hwCjqAqI7S OBqIBSGwoHRKmuRQp6naqs8JFNq5cazQE7HlU402Crmal6ZRlH3pCI8FFk+l pZJamBQJpssaBrwQcAQXD7twBmRFrTpQm/WQB9X3lcxypyxip6RJmnfSqmxZ Ht8JPCFwCFiQsRrLoxSLq1wgC9FliFZBM8a4AcKqFd7gCpvgnvFp/pjREA3a QKnLiga+5l4JQAvyAGicpwPxsQEIQALVl4466rCzuqNGWycLgEo9+gXPeacL kB4Z10U/QQ4aC5Z5kgt6wAUp8A7I4Zcm1LWD2EtlkLLyWbaSegn7tqytsGfO OgVnQAuANg868By1mQJUgKNgOTcOC7FAIVx4eo9N2xNSR3UmcK4/gQXpiAU0 AIfnkR6fwAVPELJJ5VSmNoip4Q35MD9SMhFtsS8TEA2X0AE+x15o0K8GIA9J EA9hQAsJgA2c5xyy6Qsh4JyJG67e+q08WidvkzGodJ2BewijQHWfMArb8YZY QLXfQXkmoAd8IAtxIIi06ShfW11WkZT1/hAMhoAEMAAD2bsGSPC94Is6+rAK rUC6pksLpJAEq9sCpyAG0SUHKOAIjtAJwlC7fsWgPKqqQDGkDBqnrRpGqDR1 3RYEowCa6BpbtKqdJhAEyoMBG4YCc8RutRm2nXIV6gAD6pDBoAAKsMDBsPDB MOAGlcpetHCfqBsPtCANtJAG7UuM1DUVa0UDtPpFLNIIWICncqOO7nAOeCoJ fzuTcuoOvbNGw4ttzPYdPMqjRpEeetAD5Cmb6nalNFMa/JeUbnDFbgAKV6zF WKzFzKAM4UAKtDDGeUYFYRAPKBwG0iA+YnALzSEH0pUCZ0ADd+IdrJq3P2UD /cuqEPs2Ilci/gT8lmZkPBb0naEAAaEBxSW0AZ5oQhuGHHbEGtjwB5M8ycyA DSuAyco0yXQwBWK8uvYQVmeADkmQwrQwre7bHCuxAfRADcXzNqj0EytSGRC6 AHL6sGF0CEwwcsFLvKDJCmdwsT7haDKCyA4ciISIntJVGqumFS3wDS2QBtCM AGnAXAhQB2mAAN+ADvGQBBTQXaV8BvFAByZGAaRQAKdAD2IApfngiaycBIPK BBmzDBkjz91JFESqHUl7p0OwAFzUO78AYBpXlkacddw2Bj3ABWp3kLJpQitH M4Fosid7AhT9AS3QaR/wDRit0SdADfHQDt98YhRAC+iQBiaWBKRw/grLMYz6 VwZy8AbJgHVGEVz0HFw7wQRo6Up9wAmcIA7nIFz+/Au0RAx6IA6pd35uiW3E 7G098AQL7QgQzMgrVw9TURq010tZcAI7kNVZsAMnkNUUnQdaTQ3jgDIUkAS7 QAFn0AKKcAbmjA7pfAvcM5SDgQoxaJM28Av5zEUltxPBdQ47XRO/oB3bJg4V 8ArEMCJCjXHqwYIq2YoMLAhxcAOUDaVyUA9TjRxW3YxZsQOdsNUjsAOgtg+k LWA7sAMZMA7tAM4n3QItYA/xkIb0cAuU/aRSAQAycAN6EAVjYNTiIA5Cnddc dHFkl88BbdS/EIOc8AviQNQVgNwxuEC9/sN1Qu2KuooCZlCwprESQtirvfpL KlEGncCanj3eOzAG450FVNQJ1JAE43AG7XCGSYAOesAD8YAOJ8ADUYECN6AA M5BBMZ2xukxLzF0LbQDc1e0ZPeA7RMpFrcgeCtTbrtgZyK1AmbYYSIFBPN0D RuAA7BAV2vQjs0d7Q9hhctQfX1FkrDneLI7enZALRZbaUcBQ7R0PJ1AHqIDG +Z1Y/P0B8AAUMWmP9ATUjyAOtQDcfVBLUiAISqceP81FQn3cUh4E4sDTEu4Z NKJ0/8UefOAAGAABXEDZUE17WonZpiGELsTiav7i4+3inVDWcJ4ERZYGJOXR O14S6QAP9uiq/rR4CHL6RQtQC8Cw3D1QC4IgBXyQetrxGKygGdXN3Dtd5cOi QJyQq7XABT2w3GPAHsBgBbPQA0HQA7bNdnIwey6t2ab2FeENACq+5mqO3qlt Agnh3vFADZNk3/HQAitwkDdADxogk017qg/bCD5MDEfOCU3hSRew4FEmZQ3+ 6GMQ0BLO07XEBRcgCAveO55xAewAAQyUBSrA46Vu6maebj/CH16h5n3g6iv+ 5qld1hlADSeAO3Vw30uUWDcgAqJAkv+Ly/ysATYADG2wGBAuDq/ABUIACMSA 4I9gA23zCzFQ3YYNDBdQCmrgA8SQNoyx069QCyrADsQbS28kFUtQ/uq0NxJB WJTgzerszuKc0Akj0Anr3gcZUGSENUl1QNGGlViCcA5ArKdsmacssgDiYAW1 wNM8HYOhAAiaoAmAUAMKrwnEkAChUPXEIAQ+0AalsAVbYAXOPSLrARrTgAFe 118vww6kLgcpMXuCGEcn2/Jr/vLjLSjUEO96kFc8sDRM1N8mEJMz6aB8Potz khm1YAU8LfN9cA7MTQw1UAOhUAOaIARCEPk+4AMRwPVbsPURIASJDfbrzgkQ gA8qwAdnRFQ4QNt+qPbtLGZ2UNWp0ebsHvMw7+rUsFwkxQMnwDRhkwGuCvgO eo+vuruHYANFj+EkQqQmIA6PDwhVfw1V/h8KlN8GlRABbeADwND5EZ4UnFAL cSAFFcCO/nT6aK8LJu/SPlIaUrFq4r3p+/Dyck/anND+6k0NtuXVOL5EH8AD dbADDwvse4CdADFEYKMFJs7ZMKFhwQJOFypw4tSnjw14UUz8ChJKY6iMoTR9 rBHqIaeHY8b0gajnlRV8PQ5GCdIDwy0zunTly+dAThk5KJagsPNmiRyiZcpw 2jcC6Zh9+5A2zbJvRxaqWaae4IEqa50TSbAIFBV2T1iyQxodWjhmVJ8xh4wd siHoFcQ+JgzawGsiSMeNGoMkCDJmL8Qxvzol6/GqFj58nPBGScYlDjt2+Zbc zEd0yc+fPn0O/pWzL2rUZKP3JftwOpnoqCc+fOCRFdWOMzREDSH75QvZsEM0 sEIorkKPXws0KLQyl9PJPlFsHDQx6ghgv39H/TIJcW/2V8BqXZiGbwwrixUu 3GJnxGbmzDxRvHmDwvNmnCf2XUkiOtkJ1K8/1HktD/8+QAAIVLQ6A4uwdNNt jy8eIGsP3xY455eVuGjLmCEOuaAWiCL65ReDTEDoOsEyCiIGk7LjJJmIIOLj O0F0wWEiE8boQRZ2VHlDF/qIksMyRxz5TD4fl5iChky2kecEaWigBoEV0phh Bh6snCGN2NJI4YMEh2BQtwcYHFMU3W5TqA81ZDDjgj7OOUSDlV4Z/umkX+r6 BS897brzF3GyG+NDTl4R5AIrppFDkCB+gWcUPSCY5Q0jlsgHBSB3yueGd27w jFOi2pljkUxyoMADD6iIZ4UVgGB1VVVfbSEELL4ac7cHwYTQTN6GcMcEI2Rg s49DfNNDgVrm6iNEtkTUE6FH7OpjlJMADbQCYAQRRAUUZLggmSDOGUWcHtR4 gx1d5KB0J54qTQc+XeSzdAkqPLBmEVsosWZUC4Bwg1VYgACF1QLcKEAaYYbA 4sHdcBNIoN10FeUrDdyRAlh2FhAIC06ssAKY5dgqLFkRRTQhWWgLu5OTX0hS TBB81JPjiZjU0kOTWcyY5rNLy9jsBk6X/rghHUvlSMSWRY7epoRsPJAHFFie hlodWJwGhR9hvjrzNrMa5q03cjQ4JBd2ZMBgWN8qYKkWcSQKUdk7L3IbZJXZ QomY76zQVieZPwGmByHAM+OdN9BVd6fNUPB5CSM8LWMdL7bJ5OijbZnDAnTU 2STzci5Zo3Mg9rha4jjBdsed48DSGmGE3WElCBGMA/sVKYxQYa4QVVbZ7RU/ pFuiQWXM+dwycMiiAikIMIIAyqCJA7SdDB9Kvp88lYOSHCDPRvJtKo8HlAM2 OWCCTS5hgYUUhBkW7EPcOaT90hdoRIOGGxamfg3OiCKBBQ4565WXjThWRFAi nOUs508QWZlE/nzHCRlZAWYyKAM+PlEBLqiAHWa4hSqegI/NFI5w6PKMfIgi DWmsYxvbyEY2MkG5XeyCDuWIRjS+t7k1wAJsxzkEK0rXPhuwwofso58wSDAE YdDgNyRq3wKAoQKbJKcCbYPISApokt75jhgXEIQV0vHAMrwDAj241izUoIYb KEIKHzTKpTbDmQ5SIx7SyIEttvGDTPxgG7awBdPcAAsklKNzsEBHCOBhgwUA Bx46zCE8Dsk+DXylfo/EwtdCQKIoiOMCRnCAA4yAjwDergKf/BAVJ4KSCiiG YyqYxrmAJYMl3AIDVpCCICAQh3fU4QlyONfzCMeTDXwGNG6kQCLW/sEGPC7p hD9Yxy7i0YI/gAIUbogHDQ5CkSgMUoesGOQi3TIrYXiim4/UwDmS1QMr6CKT umCHFTz0oU+C8oAK/IUevJNFWppLDjIAAABkIAcHsEOWPYgDBvRwyw/KYQOX QlcHdxKPeLRDHnNYhweOecds2CIRVDhDO1qQhjRQAwtnINEgq6lD1kUhCj6M Eze96Qlv0oAcrNALH5h4zuQ5URwkKaUUd2etWghCCulAJTscAKx8FlUVeviE Hlz5AVSgUTMHNegaQdOOduyCArxIhAdsYUc52jEHOSAHFTKQgQ9kQSEmMGlF nMM68ox0mwryhChY6k30RWEUPZhFJjOp/gojzOICwHgFndpZEna2jCVxQCUm iVpUb4hgFEHQQxPo8Qk+4LIoBx0KT9S4E2W+IB4tlAcveEEJNnhgGMO8Yw6i kIQbnaMidjGpSX0oyIrAIwTbrN8XhCFX++UwCrMwJ01plw6OHSuwdFKJd7DI MXzgTQVGsFRR82kKDKA1CVm4xTv4AAzLQhU0Q1GouhjaQmq0sB0UsEAiILoO E9oiB6zNQCcMco7YVlOR18QEJg55Ww088gue0K0wjiNIKQRXF0ZAsApUoEUr CKIYtYAwhLGYRZYoAKjTgC4+i+oAMfQgCiBNwDvegYrKasagQAIvVHcCgClQ o6ot/GwLqTBM/jbUeB1zkEYUMsCW+dbXvpgIgTswwQogp7R+LBUG6L62PnHM NJMJXjDHpGwFQ0mByg1ubnOnETwN51MHIkjAh/MnYgejmHAdBA1U8zlealBj CuZ9Y0Q9YD1KJOJ+FlFgbNVq3x/Cwx23pQE3v+nNWYUgBH0ABiYVTbsoT1nK yzWUApqbWMUWNR+ySEASzhCCM4SYHtAgaJrPjNAy5LMF1MDFN3BxAqtSgALS wCp75zAHSnjAE5weay5yMaK0ntSHJMVE+2aFhd1+UxiSjEIfKgBcBDP60VQ2 VLQ5ZijiPhe6Qy0qBEZxBiaE1R0hFsQTnpCZE2/AeUAySj7LgAtq/pwgDd/4 hqapEAJpUGHWlICoLSo6h1uP9SQZQGt970tkImPiOIKG5Lwr8otaLHoWCp7y BaR9ZQYrGMHnKmoZwnyGHZ4hF+8ItyDOXVCoGqXUAKDGN9ChampYQBqJSAQN qECFiybiq9nYxjpyQAUdZyAXO86Ac1yryEOegciMPM4j60cFDdAAf6OogCYR /HCIS3za0J6ygp9rEwcU1RRJYEK3ycHpEItbAUDaQD0OugGobiDd6qbACk6A jobSIBFzTUQIaC4MmOeAEjMHfAgysCK7TJO+PsSEkG97iK+QQBT1+xo59Fuy RWu90c929IVhJod8egMH7jjDIc4w7yhQ/kMM0ODD2U9chrXz8u1F9cQu2vFy GngiB/zwAMzJQQO7s/SrOdD5HHbejmRTsfB6ZoXRQ7C/RmKBBJB3aZDhYQJg II/qWnd01p2L4eDiUwZiSMAOD8HpM2TgBvh4ggI0m8aD1kMO9Xh9qdcB+Lvf PaKeqH1Yj5yDel+UCiQ8gwDMgAo4CbsIKaLDhAVgH8YbtmGjgfEbJHFwoOuj uoiTMq3DMPXIJQDIhzpgAncgB3eYN05LAh2IA5/SpXpQuzKAv/jLJ2SaObvL AU+wtd/zpkTguySghkyLBwoIwJkLwCBIhgIMOPpyrQQMtkYiAWILNKYLAVao EGCYhVmwgi1Q/oEKzD7nmoW+Uo9Mwic10LTxA5t5Kz92eAdowIfngT+DYkGT A4CTy6c78gQqoIFu4jcP4Af28oDfowIeSIBd4CgKALyM0rFPGMK6sAGTOocj RKnjaL4iwgIN4DR4+AViuELnukKIyzwF28KLyyU5UAR4YIL2IYf2IUFNoSXQ MDm3Mzk4LLUy+AH3kgcq8AT2Yq+vMqFt2DkqoICN+oYT2KgPoIYMuIIQaIck gCxOyIW6WMRF1KaUMgZiOzYsMKIQsKtXaLRM5BgFmDKgusJmc4DgigMTKB0N KEVyKL924IE3IAPKuAWeMDl1eb0uw6OZs7kZ/CoqoARyYIOKogRK/jiDFjiB LGgBHoCGFBABTaOBJBir4omIwnNGH2K+WXm++olEHjIBYsi64WowK+BGLeLE KfREXeCDKPjA0TkERRADHOCAJWgCVzICdWlBN5SuorIFNvi7HIC5RCAH64E5 FIqEH8iBE0CA16gDHsABHFCET7gCaaAKhuwBAnyTwhukHTo4YnPAsYOHcxiD NpiyKYyyj8TEKRTJAzMCPXjGAEyBFIADO9ABwZGCN4Agk2tBdZOuUmOD+ePF eKAEfrAFvSQHFLIjNtASHhABHkgBMcAACBCBHqiDLMiAJOgElSDAwjMIbTo4 pXOpBxykPqiFqzOCaVgwbKEwkLS8ZjMC/k5YJJBiyExLgyY4BRwwA3a4BV2A IPiDvzeUri5bB3lAhxZoAXnowV1IAlejAn44oRSighZYy3hIghQ4BlnAABzA gDqITNeoAz4QgbqALedghdFBuKv5GlZYABtYCYm7AHzQRkNxGZZwNIhTAccg sgzgAUWogzOQhiRgFR7AgDeIAzOYy7mkSX0qKkpABzEIhmBgqDOggB4URDb4 gR/IBmSShjT4gALwhXfQAWh4h1NYSxE4galIhjrwGO6krx5qn0aCJCYMwekz jw4xFPWcBo/cRve8gDYwlCmrABNYgFx4AlapA184BTJ4Ax1ABUXAAFPIBxnw Bt3sMrssqoAAACH5BAQNAP8ALAAAAAB4AFoAAAj+ACVpGeiMyIAB43bkQbAC 1CYoEyJK3DRB26ZgadDtenHnDyw4cFzUK5NviZ13jpYAWHIjDj4pFy7weZWM U65z8Fi5O8STpwYNPnsKPbRggY1zJvpw0vOKzwVocW7ks2Nnw4Z6IwFo3cp1 axkAZb42a6bFmbNJB8eN+4aAGSiL2iZQlAtl0yZtsHhMaTdF2QoWDCxh3WCy SZM3csq8eRcHh6AnXIDp4TSmjw2dh3YKBXqIs9Ci7lgd7aO0Qo9XFwTFmUaY wQZLV792nS07bBlJZM2iRThuYcOHc4PL1QZqRTx74XB5PHDsmKN8jsjQ05Gv 5LuXgrhAeKWnsmVWC4b+9uT8E2ijnu5CR0mqlFOyHjje0WtisqolwfVmazVj SuvX/7cNVJZBCCnUlkMSSaRNF5tccgkoQJywCy7KwcIBB47Y8UYT6dCDghxL oOCSFILw0YMeFVQWBTzpHXIeeRo08tN5PdHozig9QABBTYL0KMgXH0xzwxKE WWWVbFrJIINWUfDgn21gjVVWQWmtxdBbCUYE3CVrwJLCN7usEg4dbvjiSIbS 0aPAEvmg8AY+ClxQyys9VFDBLyacY4Nm4nX2U5+HwCNCAvCc8YQih2SQQR14 hJBCE53UMQ81Rp4iwhk4AJDFGXWYwg+mdoDllUAETXJFgQu5NVeCcBmiTjD+ KaRxAgUv4AJEOc7ZQQ8+9JCxxBJv3KBAdhCgWAEno5hwGZ8ylgfUs+PJGCgE woSARxaKhECDPwkgwsgDPOAhTQpOvFZPGuTwQ4MpnPKTgBOM8MMDgGGRqgUR uyWUKiyrdjEBg65cok4HHBwjBipToEMHM2Q0N88TOIjRxA0cQlPLDCL0EMQY lZkAT1FBddassz6RjMkn8EBDwxmKJHFGIAUoMc4785ARibwb2FGPGHDwAE4T J9CTAjTkUHMSWFCSdS8RBqm1AwJt8ftvRFC4UpGrbhzDARkpxDMFLggAkQI9 0AABDdm7QlMMBCKgOEYupJ1zDnhCjVxeyCHDw8P+F7ngocQpe5CDxxNY4HHz Ep4oAUcKkQCRRRKeOHFKJOR48Tge4PAjqm2kElHNqU6zxYzUCmpzCRLquIHN hU2IkTAuygBR8xNPQIMKDyk8gUrGk3lnQhRzg+zs8Hf7+ecCmPQgwhN11IFD EtamgIMIfNCTj9D5pOAFKobNk4IdvvBwiilL5K7DfwDkR4IW1VRDBOg7UJMH M0B0sKo22hgCgzqExELIhWIgQxruQCFYKCAFqICGIuJxAh5grHnJ4JgJJrii BYSgKDP6yRAaMTK7kQweuRhDFj4BBHJooA6ogMDZhjQVq9jBBRtwAZK64o0n haUeJGjf566gFvm1BQj+a5iLNqqgjiIQAgT6oBAoTjGPJvjiCQXgwROAgIo6 pOEMVJBGBqiRhffoIRlByIDHPjYeZw1BA2c847OI15nMLAAecsvAKDIwhqbg Y0hLcMRVrHKf/HhlhptLHw12eAUeUkN+d6BfEOVyiUEMAgTKkIAEwhEON8Bi HouBxhNRwUk+JIAKoExCBj5RB92xTQ8JyACL+BQj4pHMeK+0wQLOsR726EEE +LjFr/RYD1PwMSui6gqAwEKDYhbyCrugxg7AxgxmLNJB/YtkOEhBTWykgAx2 kI73eMADEaAiBJTIATnOEI86KAIaOICG7XiQhU7AAxPuCM8CgDJPV5rxT+T+ yIxo9DTBUeSiB3yIwzt+BQfBuACGsAGkfkRlzEK245CrwEUzOxBE01WBEMrQ xzTRwFEQFIAFGiJDOkXAAwjwIBE5yAEltFiHoeEDB76ARgq62c4KHsIYbTSG BnTaGZ3e7W47YQU8olDLPuQiGQGNA/kKWg8YviY2C52NNK4w1XaMgxoRTaQb 1BHES1ShA/qQACk4Goay6qMA5YDDEpqAgSeQVAQfWEdKc5CIEJwhCV1s3tBk WjYu8CAZWcjAOTARnjbCkic47UkIQgOPoWZgHCFEBT7eMSRL2MES3rAEQnVW m6hqRRrSaEc7konVOyiDGRS9hOk6UAR9rIKsZQ3+Ay6wAYuUvAEH3RSBCKKQ Ukp4QhjjjEInRJiF5nESGgqYhwKewIM6ZCEKmCAsyHJKT54s9gyYGKooE5CF D/AAB7d4wxsIc1lvmOupM1SoqNpBgdEmMxx3+AYIOjCIKlwCCl+NhQQ42goD hMG/pGgBKMiAAmHptnkhmCs5sLCyM2SgE7noBIRFiMLapZO5WcjFijBBFLxl xh1nGKoJcpGF7nIzgU2gyhvyYVmDwsZIwPSKfiww2he8IKun7YB9HVSFIugX tgbwbxjC8QdfBOsCbRNBFr6QUmEkggqHsGsUFCXGJMgtFztIhh6eUAxoCELJ YmSFTgrLk9C4Ax4jFuH+iYEgNl9g0w4mydlly1CPyz7Vj+mdzS52MYUpUFIZ /iOEfQXgINbuFw1lDbJ/DUALEJyiCdDgQx2UdwZ+5MATlABulM8MPLnlJLsi NuoYgqAHPQThJstKD2OJOgpSU+8JgkhB7ooxtDc4QrwmIRKd7fxU/Sg0HqQQ kwTuEAvUdsAQhtAGFBr541YkWtGUkIeAFcCFJ/Rg0pXegzjJEYIy26CWcxuz mIk6wQzk4twaXpFO4smKoo7hE3roQQ9I+YF6J+MDTXhHE1ICZzuE5ak586OM 9eNnCehDH8ogRBHqm2woLHsQzX52kIdhAHSAQndunXelLS0MDYQgn+5Yzyj+ +hAFG7BoAaER6lBrOYpR0BIey2r3BEW9sQQkIBdUNgEdX6rvOPu7DDn7ZTAH zhVcGBwEIIjFwqtQBQEoewKXKEIkES1xA1CcFNjggSK+rDF47GEdewCuqm80 hgq47dwraqxoRDxBCpo8PSKGG7pzkazfpX09goAGBlIM5yX8+z4HxexsltQV /xXh8EV4RtMFIACHdyHqINhv1dlA+TDQQRFP4EPGxnCGPfCDH8ANQbsTAAEc oKAkskmxpcTY9rrrSczpaXcfKnPuZBE1J41dTy6esvdc//w1sKmHJfJMeK4o /PCpSMUzGM94h09AAFKXgLMlzoYfsIEKuEgBH9j+JgvO78ELl8ZCZuCBI0Xc kR6y+cMp4sHAEySg5S1XVk40w4pzkIZjv7D9lMeQABFqGZ3voEsoQBh/ZxXC p15IsxWDkHzP0IDM53CO10iRR3WKVn3WV3GHIgIQoAN1EAWe4AX8IArCUGY3 kgClhAFmoBUcgAuJkAjx0AKDAn/KghneNntjMAodkws4oAPi1TpNoAPsEF5v QBVTcUMwtgFDt1BlAAMN6IDNB4Fd8HgQt1/TF2QWyAbygA7dNAOZkg+oQAOe F3Z/cghCFQW58AkZMygUcAYUkAAtIAtxMApEtSydgQVAYQM2mBSc8A6EZwnH sGd0AAr5IAdyYApvYAn+UwEWvxQbSFJ8/qEVVeCEjdd4jheFAjCFaFCFw1B9 bDAMYdACW8iHAPAGZ7ANISgKakSGQzUKqWRzruiK8TYKl4EZaDQE4rcAvwA3 eQIBQYgCMuACYkAB8UAHKbAEZpAPZmAHvuRv9bCIWKGEj2gIkkiJlfhwhECF z0Z5bMBooPgJsvAGXyEHGfAAYCcKdlhmMld3OpcEJnAER+BPcrhufjIE5kiG syeH56Bl73EoadACaXAKYsCDyagDpmAKzIgVL9aMYeFZAIAED0iNEPEvj5cK RXBo07eJbEAJlEABdPAJn4ABSyAHACAHWbANe7AHovAnGgB3UeBPVfYIScD+ iu83CvJ4CFiABaKAihqwAOwBPCMnQp/ghjDIA7KAAe/Ag0O4BMMHdEHXjBvQ iE/CFQ8BgVDoL1NzCRSpDxyFaFZHeRQXD2nwCU9weokBAIrgBWD3BXboGR/G BGdAVLVUcixSMrVIj2cUcrYXAkMVQvyHc6PUAxAjBuFVFb8nfC8mGIBUBo7o ClQJBVH4mFG4bBelDECGkcNADgEmAjhwjPmwJNrDD2K4RmyZGW5pZpphN0NA jzm5k+CWcmbIMTbnkaX0BEapA2ZgWYNRZ1WBXlzRWVrBmI7pmJDpL1HoVa3F X/+1iZsYBlPwCYpAD8o4EmXgC2i5B56wljMCS0D+sRN8op30+AWrGXLAEwVj p3shlABBoDx8cDY8SBUj4UKHeRVJqJhbwZhdIJz3mZ8R0QUCUAWxQJkcZXXD MKCeaHOyQA8rJp1ksA6g6QkqKZprKR7Ek5qqmZKiMYesgB6iMUHnBm/LgwE6 oAN2AAdYgRVUYVmMSBsAsCQUkZ/4OZxe9Z/IOaBWZwBJEA+fIAbv8AZ+FxZ2 0A3r8AUdxxMcpEF2mJoaNDxHWpeoOARk+G0lRzc+wVirGASfAAFPcAMiyllG YgeO0EcxJmNf8RD+AhGRCaODUJGVaQDXRwovSFmOgAK2AQef5wkpuUYygqRp VJdptEFqlJo5OQTuwAT+MEdUUsoZi8UKZ2ACWVAByxOiTMWUrwESR7KQtSED svEQExCRw/mYXjWBmRgGFDegtJAECYANYiBe4OijntANnuCgbfQTN6mnFEqh jfCnGhSo4tduUCqlJqRYIcd/ABVecHAkXZohlZqEIompdyGRxGmVnpoK+vVa YdBfE0cBpIAOsnALGuIItmEHDPpb4gcjGlSrfXpGt8qkOpkecKluPmFCY8gK ucAJPcAFt8CDrwF0sHFZB2AVSagV3iCSALAJVtOpjxkNUvhjiEYLijYMaJAA aSALhwEsPYo4/CCua4QFMpKuFEoOfeoZt6qaZ5QZLZksOfGuGoCdh2ACnFD+ B/Z6A3BAFfq6ATFbrEioUHTmDWXgCgRrsAcLBR0wrWhAC4lGcQmADoogsW9g kI7wnq/6qjdZHn+KpB5btc0iPKn5BV+Qmo3ACiYAfx/TbRqwYHYIciaQnrXw Du+gVnAws3CArEupounjCq7gswjbBdHAbJLHlYqGDi2gCCnQnnm0BCNBA69q jq40tRS6lkzABEVxqzm5tU56CKy4MVFAi1F7NywrAlwgMUPYtE9lCV9qWQv5 iABbQ66AP1YzNcMJdR0Aqv+VnGHwCRF7CoJ7a1hhuMIgDLZYrrRKDsBrQhtb FAvQCAuQmg/QpDzRakFQASaAuSk7hmf7ClwgUHD+YArHcBX7egz7Gqbe4A31 UEPegATkS76GUL4OcjpI0AGE4FocRbSKRgtpgA2KEB/tiQJUsQRJsLsL9qv3 ZEJDILzbaQNo5Ls6abyjlgtBkACXuxMpC7ydcSNBQL3zQZBN8JR8lL24WbpP orMAAAowAAOwoA6wsAZcpQ4nDAtu8Af6QAq0MLQGIA8GQAXyywMF4AtHaZv4 C2cnQAP9+yfNwkEaq6c/ERoEPASSELK1SoYJrMDuajz5tLJj8KGHYQc6IBiE mb3umT7zqbNuAAqg8MVh3AFi3AFgjA13ME0vTLRUIA/ykARpIAIpUJsEST6/ ki0+rJJBIbV3cx6igUb+SmyXO7kAHGNUQVByZ+AOIVAeIWBXrca5YhCQphCz ZbCvekQVTylwYaGzZeAN2MAM2BDKn/zJK8AMbrAC2JAGuDAFLjy0YUAFrxwP ioA78yCYOhCSTjkP7fATUVwj4REjxtNG7mADIcunI3sOEtQHCeAxiazI3ZYZ Z9BqmkCb96ohgkEkXprNlpCAAMDJ/uGP/ogA4ZwGCEDO5HwC6BAPSUAB9nAG 8hAGZ/CJaaAIpxAx+mYHgxgW+QAEUXAGdhUaRbEMIENmPBHFRnHQGzuyNmAD v4B/y9zPaSdm7RYFzcu5cdAE3Oqe2JwhGbKU4RtIWvEBJ1BvLfABJd0C31D+ 0t8g0tSwC+1gD0lQVkR7BlkHMWo7HytmG3LwDoSidiuy0AItS5jRne4we5zA CecwXQv9C+JQAeKALHZnd3d3tnqApbWpYvVAuNmcIT9nqUOXBScQ1mAN1mEd 1nlwAvGTBEmwCxRAC/JAC3dVAPQ8H/IRB7cssHJgB3VgAmPwC/y00IC90BIN e/FkAr/ACXeiLAttAk/dA8Tg1L+AJ/0U1V8LbxBwARggMVRBokRyJtnsb1wM Fh6sFTvQCSU2AjuwA1mwDyXG2vuwAxkgWuu8hhTQhqdQv/omH/PBJuH4BnUA D0ohDpEt3H59FLQElzAnGjZg2BvzC6Qxe0392Cn+0tDOPXt9MHP+FG/VFge3 wHdvkNU5w9FBl4AcDAASNgY7MAYSlt4SttoSlgTj0A7qXNvsBwGn8AT0cAO3 sCv08A51oCgaZpPix9DEIA6o4QPiYH94yDE3mCeAfQ5MzQlPLQ5j8NTS7dRj QBlaliLnluEAJQg4IFAiKl7CZxJvZgcoIJ2ikh8L2QnmJmHDdd7nvQ9jMA4m cFUukwTokACAmwIXfQP8rQCYIKu2mJpEIUtF8QvE8ArOfdh6oHkdoydGYQOj wNQcQ+FNrQehcCxHnZ7KkwyiRq8X4BJG2QRmAAc6kNVUcQOfbRsePEMuDuMw HmEcA+NqodbjsAvthzv+gtDfOsDfEIAFFWqL2ImkGmADtUAMfaDkF2AFggDV SD7lhk3dWG4nxxIETQ0BfGAiIdS8fCAFKgAx9JCqZpDm5HUm/HZDYaoVcS7n rs4JEvZYGZAE1CBK1PANqEA79HALN/BS0JAFg+6xgl6LGtQIxAAMR40asNYD xQ17RbHc1N3XEm4niG12fMAFgtADHBMEyfAEKoABrxDiICpeckAYjnADZ/Ir trEBNQTnME4art4JsN4J4+Disq5MPMBJ+H0Lt+BlPHAGgQqoTVqXBXwOwJDo lh4EwMAHmhAKv7DQTLAMNrAMXxvZDS0O0a0JtVAKMcEFnNDQTGEEs6AxL/X+ BIeBGL+CAqhuEl+BFbWxkA9G7/Eu750wAp3QBzc/DluUASfgTajQ59NwC25l ArvbpIHapOjaSgtQAVaA7JQxe8QACFIPCBj/Cw0Q2aNwBL9wBOIQCsSgCVKw BSpQCgv/ChzDCa9QC29wARWQCxjgGEL4K2uu8mdCZ1AiYzMv5zY/7y4eDzpP DZ/ATbWjtu8gAueABbyLijm5+LX6E/OEi7VwAZzw3Ibt9cQQCqFQA4AgBELQ 8JpADMTgA20QAaWgBltgBUJAJylSdsAgBWqg7bnQGE8QXijwKyHipcFyA4lh 90kY6/FOGTVv87F+VdTQCR/ASaiAA7seB/CA+Iv+//zmupNEYbxLbwWT3wd6 AuHiUAM1AAbXwP2awPk+EAERsAXmT/5tkPrUzjF6AAwqwPZ8/Q44IAs8eHom cWsqHywbkA/1wP+dVQYA0anTmH2dOO07iHAfwiydqGXZATELD1SCUuCD9ukQ FmFYRH0RFVKkKHKNNCz4ZSOKiUMLFhB7xalPn3M2FtgYFaNCKJ7XPl0LBUhT Px/9hBAjVkEpp4EVXl2IE8REnx7vMGB4Y8ZMviVLHKGwg8LRmyVc7SwpUwZA 2jKdsnDKMmLhviwL4WbBm4VaxBMneDwRBO1JLg1DPIYEuWekKA0amNgI8mvM mEMn+wDjxOkXTRusbJj++BWE5+gamoSOVrp0TGY9taSoGHMulwg1cd6pQrFE 1xKwKHy/uZHb9xI5bNXipXuwbrJ9yT4kY543T54PH1ChelLsXOEhojx9/AJS 5J4hGhqdMyFOD7E+CxovqKWH05g+JmraUDkqiOjRoYIkiGG/ZCoYMLPMgLkA H3zq64SPW+K4RSvieOttiRtu6Mo3FDaQo7jjmtvnuepGrK4OEqtDAAEgnuAh Cyy6A++LPcJ7QDGRNDjEhl+AsaIWyho7pJaYZPrFBPtMiGIU/YL4pIce9pts NaUK5KSCHlxTAZ+pKhBklluMMGMJB+TgrauvgMNQOOLy8bCbRFI8IY80ZkD+ gAceZsCTTjrT4CGNFRDABAuP+OEnpAceACk8RUMqz50+1JBBFS5oOuQQYISU aYzN+igSSfRAi3K+zKjk5JVXarFiGlVwqLIHDFTAgJ2tyOwqrNwsbCId4d5g s0MAMtmmhXiukIaHFVABItlkV1B2BWdXCKEjLL54YKQHhkj0I22HGOKQKIyQ QY43+ihMg6deqYBITjU1sl0TRpnpl8ykrNKpWhI0QhcpengFAhXikCJCXeTI RxfhHNlgCeCaCA4FsjpMa5FM2qHCgxzocAMIUEDROIUUOM444xU86Qjbbhjl NuXwRjJMA3eekEEGNRbgVgNgBKmlgj6I1JTTTX/+AXqqUA3kRI9TrWDHgXxw 4OMCQR6UggAzOuyKt6+6Ai6dht/QpQyIM8mEEluU2AadjWFhAZY10oYFFnXU gQWUP/iRluXyuFuMJHJwHOWdfKQ4pOZXeowp3s2KnGmmnvtYjejMnlLAChTk cAAHCARRw4g4Yp1ajoEpPEtcNNUsrrhtslnEmkUWkQcXN9bYZJNyYpf9kkvW pkPaFyut7BB3FqiUW1FSFlTQEOAJogf3TuJDBRVq0UzezIAGmnHHNTOItVoE UYGdcB3ApwcuZjHDiFveUEWXXjX0rcMb3nhDQ4jT2iaT1BfJZg5ppnDjgAOi 8d8mDrCJNaytBY0pjO/+3ME7d7CCFUwwD/GGIAwKauAMJggCE4C3gAsYYRpW EEf0OCGOdM3LcZNJXJV4ZIVZoCBmS8ABgmahCzOg7w1ymBytykQch2mIOB0y 3elWlz95tIMO0YgGA5AIwE1cggWg2EUIKtOIQziQFZXyjAOBV7wJkmCCNNAA PExgkxxJAQVGUEFMpEfCEs5LHKtJ3M7sZQUV5OuFGLjXNM4YB/TlY0w6vBUO h5NDr6UlEV44XSZ+4IFEJIISdxAgEl0RO9wlIgRR6Aw88MOKBbICHvCwogYE NUEKUnBvh/gFJsUxDRk4QBeCAIYaQ8hGom3qHDN5BYJYCCYHAEAG+XhQHXX+ wQVBvIEdlKOVIG9Vta7MLy3SyIEttvGDbdhCmraQRxpggYTYlaNtsEgDFTQQ BUzCIwqatKJnQHlFYwhKGCQjmTDIcQhQAQNcrZzFBYDRxtRUyUD1mQkn7qWA OqJgYADwZT5qaDAISIEd7yAYmeSwgd4MJ0MT9VBaLJAIW3hAmmD7QSbmYIFd oGMFp4gbKFKwgiRY8DOYxCQn3eEOcjrwEO3kiCd0SjIsaAAnoeCCLlopAzRK ATMhVIqp2riZKr1CEFagoxFQ4AAZIBShlNMFO3qAg2nc4pe7oZVvwsJMIBqH AlQYxjo8+gNpZsMW66ACFZKAjjqkwWMpyMIhNDn+RnJ+kpOePOcVK8ORd3qH p54JAh+oGrPyqcAKpkpKatpYJaOhKh0qmEa+5FBVqxoMBXHoQRzUcAsHlGVC vIlfMz/nzDLsIh4WkAcl5uBRa3pgHWxYBzmkYYIkJEMEgDmHJ40E08B68pPw wIQUjTEEEojinaXEETzGQNUxudIIs3DsBUyFLnQ55VRPdR5mNctZhMoABzi4 hSzqgIP3Ua1DnhWORMvKFgDEYxe7aAdsEzGHHNx2rTmgwgXH4FtBxOYc5Dyn OdfpDkxgArlSFKUwREGCwgpDAyG4pPeoqgtdGOG6ju1RLS5VCxKjCqr4cF5m ddFLq5bBDFyQxTsuoAf+KcSvYBKlUFfAOl+1rAW/u3hBPNpBrESsw5ps2AaA pXGGDBRtBj2gyYEVrGBMuCMEDU7uYEv5HZKdEh5cWKx10RheqFpBEIK4gNOg mg46TiOzi7Xq0nrwBDXIQg84gB9XcNzDZsqXvghtBzWoYd9dUGMXifAAbnGb A3LQ4IImGEMFRKAzBB/Xrw5k8AIHiwUKP7cjNAiBO4LwDup2eMx0LHOqU51i D1PVqgDQAR8+wYV3SOETWLlhn9mnmx/OD6FpQUewDd2OJOxCGolI6xzmINsc COMMSTBBk5OhhzEcKcGXZkVyWSHF4nmklOQQp4Bx4AAHnPHUqobqBcp8WTf+ j/fX76hDAioAgXdw4c7A4coP1ZSbgfm6x2X4BgJOMIUknGHJ0qDCOtaR1tu+ lQrHS8IYgpCFIPQhA0iCqUqQu20sI5Cwz6UBOS45ih6wg8MeJnOZ1b1qFGfW jpw1BRASEIXk1VsEODDCDTzXzFsF0pm/XssJvvENaiRBGjSQBw2oQAlKOLKt 21gHgKOQC6qjkD7kPIelWXGGMzS4Mo0RBinJIQwq6BWDgjD1h0G8ciuwnY7i NehmqyoGPZjgDEEQAQaeIIJiToM3VCvLcIC42VdbFRcniIfRaeCJRAC4kVTI QTTd2nQqSAMeF89AlKodhfsc2JNWrJQomfsFU+7+zZwkd4CH1Y7uVKO4jlKF MwAUMXN3JIDe+yomhvoMePmSt8UAoEY7lC6MHHjAE5QQBg2OrtPI99fIUafC Fcw5lczUxz4HTvAZWOGSTVMQC4kgBzncIcY+vN7D2AUxulPs8hVzNh8z56QJ eqAGHNSBCyqY3BI45N4d+77wFDg2YUgET+CHHEgEGlC+aPGEyKMEtVqHOag8 rnOwKNgZ+kCPmrC0BpupjeA0aQm5KlqJWpiF6zq/9Cuz8HI5gyI3znqDKDiD mYoCPcAAaNADRXiH3CATrxm8svKlwlsLpRtAnbKtBRTA7yOZAKs8aTg4ckgE rhOjZJgP67MBeDiwrMv+NuBpDI5otEbzHTEiBhWYhTD8MNYDw1lgvzGJGRnA gDNggkqJglt7Aj3gg3TYPbTwmj7rQR8EgESgAgEkgdnyBIvJgQX0BCpohxNA hxNIA3TYqEQQwK5zsAGTifpQiXPIunPABJvKQo7AAjB6ODEKAjoSQ+xKNTZz HjNUPXJbrHyAACaQIndggk+wHDnUo16DmMErA/97tXWQhncKRA/wAH5Yh22o JgA7gylAhxZoAXQQsiU7gxBIhEM4gyjIAOiYREvExu3DQmMQJS3sxEMIAVYg OToigDFUNccCw1RsP5nJhQWapxBIAhyIA2jgDTL4of2rBzmoh7TQw1fLAWn+ 8IQ5YL5mowQvyIZtYANK+Mdl7AsESIMTyAUKcMQlawcT6AT56IT6qEIq1Mbl 6cBPu7AQMIEKEMURTD8pkIITA8N0zJcVcwBZiIJKcYd5MoF36IoyyAcF6LUN 2IAy2Mce68eqooRjY7xGciRyYINsUMocoIR4aIEsOIE64AFZeII6yAAsCIF2 OCdqzAL5mIkjyTqXcI/l4kRBCTlQi4IxQJU6CkOoUgBBQEmoCq8R9DAO04VX gIcNpKkEwIAb4JAb4IF7zKg/A8rC48MlgzznowIvyASlZIMwSIM0+IBPkEoc UAQR+ARiSwIXrMhOQBeZuD7Ogwex3ERT6ilwvCX1HnEeD4MquJSCp3q7U1Q9 I2AHPYiCBTqDBMiAXPgAX2iCJZiHOnCvn7MqXayqJGiBuUIHaaCA5pSGYcyG H1gkKjiBFACCDKCAU1AERZAFWVCEXIBHasiATkiGGXgFPdBIbLQJj+wIUwKj HDkHTlg5MGQzM7uAlFS/t4sKu/uEiUiBD8iAHbBHFACCfhvMf/u9wgSFNIgH avgGCkg8CkiCYZROtvKAYhG6FYADDJAFDlWEFkmCEJW2V+CDajOSKXQJBJqg kBCURtMAVjAB+bSCNohN1kOzNqDRVKu7BOjQOpgIMiADnKwHeriBfvNJflyL fiyvgAAAIfkEBA0A/wAsAAAAAHgAWgAACP4AJWkZ6IzIgAHjdnxD4AbUJigT oDzUtukSkntwUFGLZw/XH1AcXNQrQ5IkAABy6i25gUKOnHQ4BAm6wAfYqwpj xvSxwcrdoZ9Agwo95G6BDXjn+ozhpIfLBSn4WpbZsKFePksjy5zcynUryXrN mmlx5mzSwXHjFq5Qtyniw03a4mrjYAdVvCT26GCDdUykN5J/AZTJl0+Ho3xy UFhRILOWiFd6OPUxYQKez6Eahh5a8JOV0aRLK/C5IChqvjJ2LLmgWi9w165a /0oSS9YsQoUzmMFqO2FCF7kctLkSMwNdu7zYyjGoN5K51jL1TL1rYibfG3wz i9Xq0YOTZBPnbP74bBSUvIbzh8inP8SElQ0b50ws5ZQMGj4pON4sqZeaatV6 r5lhyknPAeBNPQIRZBBCaSGgm0S9tSVcXGLMw8MJSbxAByjlwNEacxtoBYAp TdDTxBIovBNTLbXwoUcFnIxBmQ2YYHYIeuot0MgC7vShhwgQ4HTBE6R9kUwc cVTn339lBCaDDCedwYOIgkEHFm0FnbUDAggAoc4lEfkmlzYwcDAPKi3Es8tH 5dzjgiWsnVSPDu/Qc4sp1ynA4gw34fQLeKz0BJQG6glVKDwQ5HJGCE+gckgG GdTBCKPvRCpGBljVk0IaZ8xTRhZn1GEKP2fgYAeBgiE40Fi2JZSHg/7q9ObW BHHBUKYYQPAQTzzYgBIMBwzAWc9/9dTZxBspMlbLK5B5N0Yu4Xn203nUogcU Zz8hKkwIeGShSAgaBJIAIoyAIwIeIaTgBGtp0MCPBvmEyk8CgTDiBQ9VWplg Qa3ukMcKuvEmKxRkgnKKL/Tgc+EKBZxyjLAb2GGHI03g8M4bb8QBDU2QwajT KCbw5M5l6GV2o2aHsPIJPKdgEYUiSZwRSAF4mEAPDmJEwg8PEW9Ahg48gNPE CfSkAA051NjxDoFf0UYEEWahtSWsvXXhGxSuaAOKOqfgcEMTT4jAAxCwcACH HRtYQoYv0Phi8Q3vCPIEHyLokUyMk1EGD/6PN1LbSLV9/6QePDx8kQEeSpyy BzmMPIEFHjrb4YkSjqQQCSoYeuLEKeCQ40UWSeABDj9eyTYWEdVo6S8CHewm 6wRwGQJDMAi/8Q40ivBQQAosOGIHHE0gkAYPKdBTJz21zPBYjLn0Mdk55wQ6 LeCAT/vTAgv8+EQddeAQzyF4PAFNHSJg8EbRS6TgBSpNkIFDCnb4wsMppqT/ hA4FCkaCFtX0P4nU1PgGwGDxurjITh3BmMexpgGNJ/BgBSkgAxmOQQYgxCML aSgGDhQADUHUATJ6eNZkohCFc+xNR9X6m9+sNahDwCMDncjCJ1AhjEPUQREi eAIO8rMfO/DHBf6rEQmAuuINr1SJBP3r3xXQQo08fIMZAesNFAyhjg4QghCg IMMN4Ca33I1NgimIxxnQwQNUfKAOH0gG9/TwiSw8KwNRgIdl+qaBIZzHjtSi Y2aqtwBWwCMKGRhFBsZQARHwQQHIqoeHhmWJRjKHaa/RCnNoUI0rWHKJ1KAG Lu7AjA6wIEKXGEQHYgECEIBCDDpgxzsYw4NWAoEMHOBBOyiQBeLlIh4JyIIh DamHOiRjDHKco3nuWC0NYCuPLOwj9CjjvAr0QIc3WMIGDhOdtElziIKBzXPq QQNKXnIcmXzVCjqwhrZcIpSEAIE+NqTAjOGggUBQBBDYho4QJOEEqP5QBDyS kIssiC8mT3gCDnORhDiGYFDFLOa0/kYO9CwAHu+hTC5+kYwe8MFrKLCEIw7k H7Rh8zVMK4M3LdmOF1AjHLiAoicncM4qEEIZEogpNuZxDB3EISbwBAI0UpAB KmTgE4rIQgjOQI0PCGKHO4TGO0XwgSxkYFEHpeMhhrAAOx7CGH0zxnn+RhT4 2ICZufgRF/BxookxZzWN3EA+jAhSANQDE9K4QjvakclVbJIZblhDOS9RhQ7A dBVoIIU+gtGEWxjvfflEBSrSEIUQ1DIL5KABFUIAjyycsQ5lNFpSgcBUp+4t KFi9kVa1KhR3+DGOJoBUJ5IxA2hcbAm/Y/5OI9GGNlRRCTZxnesLxrGKPNzB QeS8BBQu0YEixJQUaLBHOBz2jhuwAx8Czef22hECEdQhCRogR2TPkIFcJGAM WVAjH4ymAHr4Ym51kNEJNyMUkwFlZMEEZCfAywNBvEMHdoCtHQ4EJzj50Cu3 3YoMyiCNuZb0BeG4Ayc70IEqtHSUgEUDGmhBCmxwoAkosJ0iyKeIoMIjBDz4 xBnOQ44QhACQ3c0JeHtZRgVsEBo80EMWKsMjzmDVvURxB2X/CKkTnJEHODjW EqR5Ntmm9b9cMQls2mGBXZh0FXf4hjJi0eBz8rUIsYgwGsJgj5lSBwU3gMAn uCcCUJ2hBSagAv45qKABGpz4HCiGVAZSG0MRoCIFMeFC3SoTqMu8NwSswAQJ +zlmVADBaL7bwJAjhpU3HXmISs7mVv7SDidTA8pTvqKDhWuI4kpAwmEIgwHC oI8C6AfMdQjCs/SQgAxkAR6RXfMhTMwKEpIQzlE4QzApM4YgfKIHIujOKKC3 gBqZVtdR4PUn+NC28iKMDG+gLVUkBh1LSIzRsJG0V+LxglUk+A6xYEYROmAI QwhgiqP8dCtCPepwYANZb5jGLxMwih5koZ8hEIasR+YOOdoazvAI9GmTbQLv juG70OIJK0Jg2j9GwbtZ0AMfAhpQVBTjCdD2ncSoXY9G+vdUSQ7wSf6msIqY SmDK466CuSFyCSx/essGGDWv7PAGuOkhF8+qgB5cLQx9a+CgB4VoyExAwoCb 1o/+Jvgo+gAyE/ak1krvtR6mfoJkZOHqH1AAGZpwg2lGrNr9sTZbtU0gdEjg DlOORcqrIABtQKQKRYDpukUdczRgAw41fwcfgqDqT4SwE54QRiIycwb4JvsX vxjDKEZRdD+654+Uibze+gwPEyz+u73OSS5yQZlA6pAeN5AYbKGTtv6e6rZl gJIRQaD2Irg+Fc8QgOyh8Pa4A5bdMacFNnRAp3e8AwLP+oSqNfCFwJNj1qww QRAkjgEHyOFJcjCDDnDwiVwkG2SWpwz0MP7R52TjXMVNj6PD+yC36USb4xsI Flb2+xrVo6r1RUhFKtgu+3NDxBBF0Mft6T7qFugAbnaCAyHERlFgDvzQc1ig ASMTBQnQA1yAA2ZwA6cBAGLgCynAA4qADpy3eMwEPT2xgKMwUTnxJ+HxR/Ex SNDgWixBW6SXGv5Fdu7HFa8nf7E3e7R3f3GnbvwXBlPAewmDArfABzrXAyHw APzwBcJgVY3gDgVXSEgyIABwDAWwCxSADi0gAvQ2Cn8SMuIBFMm2FDnBdPGR DCKQAmboWveFMUNmB6SXVhHTVqgiGDAAe89QhzZIexHRV7HwabgXc/rwDhhw M8YjCCIgAp+gAf5ecISeMAQJmBmBYgNREAQQMIkYsAIUIA3ykARJ0AJS0ANL F0c9UUcagAXJB4a/oBTPBwAucA/fsCF2kA/R8QYRw4YdhzZwsgRkF3InUQV1 WINQYH94OAH4BwJ8iHthkAZN8A5SAA0YgA/QQDc9QANekIiLg0c59kcJEATZ +AnokAB3kQCfIAI4cAGT0WeHgAVDkI6lKBndxQcOkA/e4A1TGA90cCL5AFuz OBWM5h+5WBKoYgh2WH83CBG+0Wl7CGqiFgZ21wQYAA0aExMXhQMh4AH84Alf IArVMjLuEYK5MAoJ8JEg+ZHCtxOCMlWioI4m8Avf0Qd0AwFBhg7x0P4CBaAD 9SMx1uZDVOGGISJpMeh+SCCQwEiQVmMIg3CQ6zZqBkAKp2Qf9CATqMAFGHMF 03iESZhHRJF8m8d4SWACSRCSJMgK0yMKYjkETLgUjCcfvvYJn3CFBYAB92UG ZgAHsIUVVXFtaeMCVOJ+t+V2AzmQrxNKxgVqMUdqbUkP8yAIUvAEXMAHMiAH ScAPXrAHxYeOmWEeVwl5i2dwi8cTKHQeWCCWYukOUTAKMnIUJpAF2Xhw3qUI snBfOnB+dDmLaZM2AUYSMfgQA9kFuukbujlcVRALyoCQBkAL+qAITSAGOBBQ isBsjUkNkMkPeyAMjVg9RMEeuZZsUSAy0/4yBI2QjqJwkaJwCDawdIACDziX BUFwdZ/ABcz4mnZgBhIzLBHjCD1TFdrUFbh5g7sJBVZjNYD5cnSHBrwiBu8w Nx0mAu8gA/VwAnvAD+sQnY34NypULQzHBKbFb+lBLXY0BGK5B2T5VZwHit6X jTr3BLIQB+/ADvCZGh0XMXAwm1WRPwAAJVTiCrTXBfyJo/0pRQLwm8G5Za1g AIGlCICoCAIFAXxgBoJxAjnAD9ApDAlVLeTAVTmmR3W0oUMAnuQgmtkXBYFy WpaXjVyACjgQB/hFcz70cT0zLP4Yh1vRFjrKm7o5p13AV8AJahNGCrwiCzok CzmEAichA2ngCf7QuQeesFUJ1VDZRS2N2FBYeqXgOQRMMJogA4oaiZm/9oAX YwdNkKazSJ9VsZPZlD9aYaNSxJs5upudZlzIFQa0YA9okADYYKINpAg4IAcz CgA8kAPUiJF7RChXKoqUuaEN5ah/k44cugcnmXyRl50Lx3C69kJjoAcQ0Jpv AAcPwx9ps3HEEmmpVyACUzV0OqehZJS0QAvJhQ5poAjLyAX3435AIAyFKgpY cDKMuqFXiqxYiq8bCppDcAaUUaleamImhmy58AkQIAioxKnTNotwACf/4aZF NKNlkDUCg6Ny6p8uN2G0EAakYA9kJAs4IAtA9gYDNqMpkAhe8KD0uv5H3Zmv Q0AOMbuvo/iyd8ShF0kOJJR9IRMFftZwo/BrrYlfjuBfjZRoPlQSBaK0grEJ rpCx4xoNXYB/ygACCIlcZKRUAsUOOrAEIuIL7mKoLds33JmOolhHMiuKeEQo WHqRjMhjMqI3s+ZeyfdrgoAB+PUw++gIiZYVIiIiADIS3uAKhLub40qu6Xa1 MemSp8CeePKKJ0EG0lCRSIgF05mAMkusxNqdjbAjJoOs35mOleeRi5edJnZ8 ORYFhSQFeMuptHW0NtkcgPEXrdEkf1G4h9sFUqu7LUcIAEphpLCuySmypoAC +aUVOnAFnqCsLbtHiuo3+oqOCbgAxsQZ3f75nSepAWfwcKrGeHMELj8BsKu7 sMfwotemcf81EpP2F39BEq7gtE8LtbuJBKOkD1umkEkQk7LAp08QB/nwuCRh B0ngCaLQc897s8iqXdoFrJmBPTpyI+kYqaLZgKFQmpdxfAxnAtQqBW9Zvts6 Tb+TvgBiu+wbG0hwwid8CdqABBbBwiysDrGgD+GABvJAYUmADqxppPnxv4pm CmrFAwZsuTiyVTKbudrVuddjA9Q7TBowlv+qfHrQA3rAeCNzIwyXfD3gkm8J B0V7vhPzX9AhIu07xrAwOzBQxsFgxma8Bm4Qw+FgDzW8K3TABSlgou+wH2qV x/mQAg1luRHKnf7ZVbZpa40/ocRq653puKUmoJah0ANH4KVB0W8mACSsG206 8KI36QhczIZfcRLtK2llAAqiPModIMrqcMpbwwzKgAurAMf2EA9WKAt17DVm IBKD4RzvUDImU5mMSmIs1FVT1bn6arbwMApBkAzLp15VXHjJBgzMZj6mwAA6 MIuWcDa+M8IfJWl/gQ14xc3YwM1ugA1u4AZA4AZ/gAvhMAUdmwSkoK5yw4zR dBq0exJ2QLBRJTjGoB7ZpULWcz3dabb6eh7SyneqRnQ9cQZM8EejwAdcwLpw EJeySFt8ewxI9lETaxItgAAtkAYczdHCIzwc/Q0wiQ6XiAZWqAgdhP4PzYUC G+AazwFHHcmZ76Ej2LIeh4C6J7MATODAbIushzCaOREE4iAj4mdrHgkBF4AD GPDQ3DqffCvCnmwgkwYAH9AC3/ABH/ANV10HC5HVLYAO1ACTFkALN5wGsiwI dnILbwCLIVUGdWB5Y/AL7zHXNrAMO81v7sAEP3F8GsAK5/AnM30jf8MEKRkE MFLQO0tCixcEE4cDavAGpoBfprBxdnAMReu3TXISuDqjjXkCnv0BPgbaoO1j eXACWbAR8UABFADLKK1UvtcENJkVx0kdsmAC7sEJ4tAHiCfXSqxweD0emyEf 4oATIWOhoklI3BEKFfALIIN9cN0UPqDU0f52rZagX45ABr6jFXiZqzPqDVAi A96QBTuwD+J9deN9dVlA3juQC9SgiTd8Ak9wCrKAAbdwC831ivXQBK2GsFlw Mr8gDsRQAcRADOLA23MdcCWZHjagFAEe15A4ms4EAcBQAar2LM0TSN5VAVxw t+ZjB6+pX7/z1IEbhyZREp1w4mOwAye+A1lw4i0+BtTQCdSQAXdBDZ8Q3xbj eyiKAukTAo7KiNb4Ve/xC8RQC8TAhbgWR5zpwDaQeDEiDuLAd40sxUEQI85i 4cjd0BgQB28gffohMXz71M+hFQMWGAWiWig+Bie+5il+4hkw40mQAfDNwe9w Cyj6DiigA1lguf4niUd4dKyEYhTAkNt/QnBeypnssQApmXg5EeVTHuAUzgmG HSO95kxP4Za8555gft3ZLRhFFBhmfhKdgOZrngvzpeYwngHjoOpJsAPw/TaG FQf0cD5JIAz+2lAva7Z+A+C/kBTNOtcObBRNzuhj4OjcUQHITuHJgBOEFMVP 0IxxIAZvgF/6EeZiXhJrRaNJBkNr3u3ePuqqPg5J0AkfkAJueQtNEAdu+QbQ gAm23ufIGqzpqEJ9QOC6rdvkyWcWygQ7/VWIFwOVTgxTZzdRTOVj4HciYAXq jgE4oAPUUe1NEOa0KBib7VaRBu6j/u3dvupMZFk5rkoMHwc84O5JmP658T7M 1AIP4lALMFIBr8AHfEAMMqLT6bEj/P4Io3AEOYHsxMAdxAAMztwdTAEMF6AG VrDlDX8LKLD01n7NkkTmoHwSMNQH3j4COWH1I+A8zgNDWVAH+BAHt8AOGIAf UgAPw3z2apsenLEAv3ABE/4KzlwTxBAKM63XyyDsTa7z4hAKegD3XOAUXBAZ nCAaVgBdUnBT/7f0NNf0IRLGRfS3ns7tVH/iI9DtVr/mkBJe5QP2t9CQXJAF HIqO8A7v+rr2yvQKF+AsnPALoUAMgFADoSAOMfALj/AINmD7Oh8DjawJbSAF W4ABbXABgt8DF6ACf++WOHALXcvjjrBFjv7wBo4wZHIAHWQnIhi/5pzQCd7R CSNQ+RhvWXyg0mEPDSKQAaNouRx6kk4M0JvBE709Bm0gGX0QHr+g+7APBjUA +8QA++JQA8QgBADhI0IpNQQIWOFSSw+nCq8E3erxJA4Gejfe6FqCwtENRx1v LHEkJ18ZkmUAnPTmDUAnlp1GcOI0Yh/MfTVZUhuTJVkdidPe4QgxROgQUUVF fSlKVNSQRocW2Pj1a9QYp+cY9jl37ikrG+JChaoB5tqRazU0CfEhMMKWLRFq CXmlR0+FHrXi4NMj5R2GWzp0mdJ4w85GR29QGEZRL5+ckydlqBzDcszLLPsq Wx5ROcuOLFk+iP6A9i7Ok1xYlBr9gjRp0iFPrVaoMGZB1Vd9bNvgysrEEXFg vl4DDhZQIbX9gGl69aoHbIezLvTAIJrdm3y6UGjU2PHNm40o8plaTBKA+Mqc ZtasnCzZPvWaTyT7wAcavjrnNGApmjr1A/2rRWlYYIFfKmgDmDFC0ADA2mw7 xwYbnjIhqq8mBOssITQBpAcN5+KQizji4AOCONR45w0z5MjIERTsuIGjwJp4 Y4nq5CipHm/KmCm9D+D7oEcf8/ggD8/qQEWBTg7BwrQ99tAvNVEe4C+/pQDs oxYjLuBkgQQXeKUCTmzrwwQTzhEzlyAm1CRNTSYMgkNiGmpIkFnwAf5Glokw MBHFwwzj6KM3mqDnFl3yuY5GGUpCoI46PpgBAQRmgLTRGXh4FIE0eEAlBR48 EUaoLx44aqkHlOLPyaVaM8EIGRx4pw8tD6nlFZj6GAPMWmkdIwi6NAxlLtg4 bOiVWuREQQpg1LgFAylQ0EXP6+x4w6Il/nyHHuvkeEMOGmksIw0E8viGBx6A AGIFVMgtd4UVMF0BCGjGWco0U08dSkqihNLgEBxkkCGOBYRaIFbYvuQkKjBt +2WMMcRhqIJkHH5Yj+SAEcQKXRww9hZ2JrpIDoyWyIjFG667QbTrriWJsTJa oEaaeMZFN+YU0HUDiBRSgKdTcvBLcsp8Nf4gSrWiksw3iDcceOKQ+w654AJZ +yi4j18K/kVqhBUeg5OHOfG1oVqsUMEBXawQYToC3jFjUJAzyghGkKOlJ0YU luDWJAAoSMQDaegAJQVY/GYBljVgIRyUvn0B4gwsyBmC8UOUliRfLYE+bQgs EjwkCkXegOCQIfKtuEuDEbatVpiyThgm2PZhblgr2JGhWD7eUGU7XQadG6Ql RJZxuhIxwqgMbU3KZJFs1qHADRbKQaKcTZ7fpBzmB0+hHWEQ1KARd7Z35/EA Z1NaKNOSJAGLEDAZpQImHrdBBRXaeGVqqaeOqmqCC2bYS9XjVAEFBxxghxQu 8AYj3MIMDqAbs/5yZxHE7IkdCeQWYwCwCAp6IBHocN7zDrCJA3TweZdgQRqE 4Ql3rO8Q7mAFCvOFQlYwYTb3sVyShHG5EEThFw9agAmmYQQrAONL9oNJELPW sAowjGAj0EMtVGAEOcROBRiwQgHjcCLvrCh30ULMtAyzhGaVRDyLsMYitjEH C6hjE/e4RzTUqMYNcsAXCPCEJ+ARBSawwo52fBwr4HFHd9zHNMIApDBocMIo 4IYJv1CFA6YhBYbZhhPiGFgQYaM/qzHkFRcwgi4OZQTRxOENt8DHAY/GNrq1 6CNz2yKKuFUP8WwjE5mwxTrkQYFVuCEaDLilGu+xCRb44glJ0EAIbP4QhUIS Ex7dQ+Eej5mgnglDFDNMUghG8QtxAEMGclAkMIgRk6k1LIiPjBpW+kCMr6lA kzIAwBKmcxgzTMMMh8mHYRazBIvcQAdbRIFIuliS8WwjG670wBzmQAl7rGGN arzE4E6Rgixgwh3wGGYU5oibE7rDBnucjTGSNAQSBHKGhzDBGMjJLweYwQpx MSIRJ8mJW0ltDMPaoQOaCAAZ3O5iDjAMAQmVT5Fs5x0j8w4XVelFk9hiG9v4 wTZs8YMfUAIdSHBFVJEAC3UsFBoz6EQIzhAFE0R0j9xjRRTwqDQsOFMYHQUk DeAxCk0Igl+rUoEgFLK/Sa40KmNoCNiMkP7PQznGFGJDwXTYYQZC0a0MWoSR PLW1mJHwEwDrWIdRj/pPNlAhHrj4AzYM1zcF0AMHItDjHCU6TDwmEx7wCMEh NFpWYXyBU+Qgh0WD4FaSGkEFVlBIl5gzl29yogfAuMASM8mv8QAgH2bIyC3e wQ53Xktb07pBYrm4GG3V7SRlkAYl1sGGoyo1ljnIASXOMAYiPYEit7iFCU5L TIka044oxMQZMJFajZqVU564nGzf6jEj2LZpwEhOXZPzNXyoYIfMcoDdjFsy XYwGBz/F3UjkgIJ6bnFGdfPi3SxAiRzMIbJsgKxRc0AFE+QiC3WARnTY0YM+ iPa0L35vCFgx3/7UaoAEQzirMzkVTHeYwAEkNYMZjDCL21rhArWoBTCSjGRB GHgaB24WOk/yBnzgo1hceMg7Eki3CddzZFwennjGY5Jd7MIC0pBHIjxsCzYs dQ+JOIMJaIXlONwiDno4h0RPe1GJztcdMg4BfTUQyGe+VgNbVcOPj9tf2xa5 GFaomBWsUOBpGLi//pPgSegBDRywAwcQkELJqki3DVAYizzlMlGvu4t47KId rp6DB2wxa8h6IgTnSEIf6sAHQcRhFlLghAmM+eI9vtehgdaARtF6XypowB1n gIAZVmUGVcyCyO4rsqRvi+0n7/VimS7DE54giL2IQC/04OLcRGLqL/6D2bpj LsMU0NGOV8cjER1ex0ApQQkqhOAKUegEXSiGD0G8YkxjIjYrzpBCQCMoSfgZ oTDIEegzjKIHqjAFo4kMNm13XAWz6HYmmyXBMtBDDyLgwixwEJFb0GNuGNHW nq7jbi+qJCVlQAc1kkCBM7SDChwGcb4hS4UzXCGkXOPDBWRxgU7YRtgXHSZu 7IgJqhuDmYHMgTCocEITBAEHqlBFf4m8BUl73H0hz6RMpYyCFDxMRLJAucnW FvPt5K664inDjW6U92/MWxr9psK9BxpZo3rAEzk4Qy4SMIaTX4ALQchFJ0Yx TGIX2441TlCnAkkODUijxBBgh8azbYV0gP4N2yC/9MjRWYZ3iEBhyYiDLPQA AdFwEWRy2EBGDgPmkRR3PKwkyQlapoFECBILiSDHvXPAhn9CtucmyMAYktED EfSApZ0Q9jkgqn14yNcdmHgcDNGKBSqQI3Ppk4Loi7x+bu9wr2o/ST7aPopc 5CIOCqiDCOgxDeA9F2R2mLnqkjLfq4eTkAZpiCMP8ABhSAQaoIE4Aq8cSAQO 8wB+WAcq4CoxyZogYKkwQbiLgof4woQAUZryAaTFabawSp/+IoBGy7bTC7nb mSkZqAcc+IQxICYT4IInEAGe2KLbI7Xbqy6aagySUAmTCDzwqkAaoAIsoAEG 9IR1AK9Y8wAphP6zMzgDMsGVL/FA7nux8+kePzrBBKGChwop9xsy9yG9bbM2 RrudH5MyM1CEIDABOzIBPeADEdCDGZg5IaSbPwyzxrgu3yOHOIpCT3jAHHBA CEyEv0s+KqCCv6OB84GHMvESD3QQeOA+qnMKpbkPQCIHGggmrhoF4bI2bDO9 j5sFN7wY4pIBHdCDBIiC7WGFZBABPRSBl6suLtsAVaIRxxDEMQOA4js8T6jC KNyDwxuxeGDGKTjA8tO6QMOEOTKBLPCSMRATytPEEVyAGns4JwxFrRITOVnF a/O4Hdohm4LDk8CATxiFKHicQ0iAk+tBZlmbJehF4TEsBSvCYAwBYf7IgXUw RsjyAKUaMWloAXRAhxOYgnjwOaLDhHxZOK6SPi4cE+07B0xghdl4IT8imkPQ qigAho9Lw9uqMnyIA1VkNLHhF5N4gk+wge2BrTMQtxjZMpDpRdxjpUEcwCLM uwmELMgCyBw4qhHLAVJIyIVsAUXIgiQ4AxoYMSzURBNgiC8hk6zQPo5MED8a vxWChwrAh1VstCqzgjhIRbGUwbd6giCIgjNApihoAmi4hSagG8PAvQ3YAODj x2BsDG+QA0ooRkpgwAn0gmzIhhwYBoXsjDpQBGhQhAQ4A4DEQq0ik4CLGqxs kI10igSxHGiioRCAh9kSSzWctElLSfdxQ/61k4EnyIJCOoQzyIBPeIcUeAJF cASQyBa8xDC+5MvHyAFpmMAILD9/yoYfGIY0aIEW+IA6SAEckIVPkMUakoaQ PIfoE50xsYGs+B7OBBoZ0gDOOwRWGAUl2iEVMEscsAL01DbU7C91fDyJOoMs SAAeaE5MQbfDEB450Eu72UtB9AYZ4IF4SIJIpIB2+LulMswfoIA0cJd2iIcU KABFkAVFeIIzILp2MIEkyIBOqAPlCLYoaBCtmBzKmaEZgq3wvEO9IrIqkwJl Kbtta79pEIQeyIUzYE4eyIAEEIPtkBEUEZ562ADHGkTeRKcCkAZ0oAAKcEgK oIJ1YCrDzIF4KFgAHjiBFDgGMTgFDDgFRfiAdui5oouCDEiGGQAGFuOqBvke phCKQCIacayASSvHD8EHK8AAKBo9SZvTOBAEYHiCFqACeJAGA42HedABFOki /Xy3/gxGGQgIACH5BAQNAP8ALAAAAAB4AFoAAAj+ACVp0eKMCJEB48bt+IbA DaxNEyJq07YJCaxTT15libdrFzYWxxhYAkCSpJySJeXIQcESxTRBgi7U4vNK T4UxfUxEYeXO3aGfQIFiYsVqQVGiUfrk4vSqliAp794sKVNvg50NWOvVQ8kV QJkyduw0G1hw0gCEC3kwg0URygRtb5EEewdN0MZ4L0CVOwanXpmSZbzJQCmn TD5HS3TdwBdzZs1kOE2cg9fTZ1Cg7o66W8DZxrk+vzj1AOY0zg0UVC1hXe3i 71+Ur8GKJWsQocJvM9ZucvsWLhIOOuLgy5JkFzpY5eDA+VpSMFdTb268eZNO gaBatWaI4MQpV85zRC/+i99cFJ7nBZ9DV5Aihc8TeqerWl29we9rM6YAk7Rk RyBBImahxZBDEfUWFwdNYIDPB8XhghwHy71GkgzeoPRGE46gkA4OjYlQEyeR wUOUT41cpsEhjSyQGSt96CHCDJz8MlMtXHyRTF3SLSHHBvzVV59fJJVxBg+x eWXVWAQVZNsOCCDAzBoQSUSRNmLMU9cH8cSDyykccMCAhCSZImEZOuhwGmPF ZPdKD5DlFIUNPC1wWYknHtITK+fwkAs8IXDBhwYZZKAHIyEo