Re: [PATCH 6/6] workqueue: reimplement WQ_HIGHPRI using a separate worker_pool

[Tejun Heo <tj@xxxxxxxxxx>]

Hello,

On Fri, Jul 13, 2012 at 10:08:00AM +0800, Fengguang Wu wrote:
> [    0.165669] Performance Events: unsupported Netburst CPU model 6 no PMU 
> driver, software events only.
> [    0.167001] XXX cpu=0 gcwq=ffff88000dc0cfc0 base=ffff88000dc11e80
> [    0.167989] XXX cpu=0 nr_running=0 @ ffff88000dc11e80
> [    0.168988] XXX cpu=0 nr_running=0 @ ffff88000dc11e88
> [    0.169988] XXX cpu=1 gcwq=ffff88000dd0cfc0 base=ffff88000dd11e80
> [    0.170988] XXX cpu=1 nr_running=0 @ ffff88000dd11e80
> [    0.171987] XXX cpu=1 nr_running=0 @ ffff88000dd11e88
> [    0.172988] XXX cpu=8 nr_running=0 @ ffffffff81d7c430
> [    0.173987] XXX cpu=8 nr_running=12 @ ffffffff81d7c438

Heh, I found it.  get_pool_nr_running() stores the nr_running array to
use in a local pointer to array and then returns pointer to the
specific element from there depending on the priority.

        atomic_t (*nr_running)[NR_WORKER_POOLS];

        /* set @nr_running to the array to use */
        return nr_running[worker_pool_pri(pool)];

The [] operator in the return statement is indexing to the arrays
instead of the array elements, so if the index is 1, the above
statement offsets nr_running by sizeof(atomic_t [NR_WORKER_POOLS])
instead of sizeof(atomic_t).  This should have been
&(*nr_running)[worker_pool_pri(pool)] instead.

So, highpri ends up dereferencing out-of-bounds and depending on
variable layout, it may see garbage value from the beginning (what you
were seeing) or get interfered afterwards (what Tony was seeing).
This also explains why I didn't see it and Tony can no longer
reproduce it after debug patch.

Will post updated patches.

Thank you.

-- 
tejun