Path: archiver1.google.com!news1.google.com!newsfeed.stanford.edu!news.tele.dk!small.news.tele.dk!213.204.128.162!news000.worldonline.se!newsfeed01.nntp.se.dataphone.net!nntp.se.dataphone.net!193.213.112.26.MISMATCH!newsfeed1.ulv.nextra.no!nextra.com!uninett.no!uio.no!nntp.uio.no!ifi.uio.no!internet-mailinglist
Newsgroups: fa.linux.kernel
Return-Path: <linux-kernel-ow...@vger.kernel.org>
Original-Date: Mon, 26 Nov 2001 11:46:10 -0800
From: Mike Kravetz <krav...@us.ibm.com>
To: linux-ker...@vger.kernel.org
Subject: Scheduler Cleanup
Original-Message-ID: <20011126114610.B1141@w-mikek2.des.beaverton.ibm.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.2.5i
Sender: linux-kernel-ow...@vger.kernel.org
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
Organization: Internet mailing list
Date: Mon, 26 Nov 2001 20:14:56 GMT
Message-ID: <fa.laomtfv.53gerh@ifi.uio.no>
Lines: 17
I'm happy to see the cleanup of scheduler code that went into
2.4.15/16. One small difference in behavior (I think) is that
the currently running task is not given preference over other
tasks on the runqueue with the same 'goodness' value. I would
think giving the current task preference is a good thing
(especially in light of recent discussions about too frequent
moving/rescheduling of tasks). Can someone provide the rational
for this change? Was it just the result of making the code
cleaner? Is it believed that this won't really make a difference?
--
Mike
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Path: archiver1.google.com!news1.google.com!newsfeed.stanford.edu!news.tele.dk!small.news.tele.dk!129.240.148.23!uio.no!nntp.uio.no!ifi.uio.no!internet-mailinglist
Newsgroups: fa.linux.kernel
Return-Path: <linux-kernel-ow...@vger.kernel.org>
Original-Date: Wed, 28 Nov 2001 02:07:08 +0100 (CET)
From: Ingo Molnar <mi...@elte.hu>
Reply-To: <mi...@elte.hu>
To: Mike Kravetz <krav...@us.ibm.com>
Cc: linux-kernel <linux-ker...@vger.kernel.org>
Subject: Re: Scheduler Cleanup
In-Reply-To: <20011126114610.B1141@w-mikek2.des.beaverton.ibm.com>
Original-Message-ID: <Pine.LNX.4.33.0111280145300.3429-100000@localhost.localdomain>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: linux-kernel-ow...@vger.kernel.org
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
Organization: Internet mailing list
Date: Tue, 27 Nov 2001 23:11:01 GMT
Message-ID: <fa.o48rfgv.tlg614@ifi.uio.no>
References: <fa.laomtfv.53gerh@ifi.uio.no>
Lines: 32
On Mon, 26 Nov 2001, Mike Kravetz wrote:
> I'm happy to see the cleanup of scheduler code that went into
> 2.4.15/16. One small difference in behavior (I think) is that the
> currently running task is not given preference over other tasks on the
> runqueue with the same 'goodness' value. I would think giving the
> current task preference is a good thing (especially in light of recent
> discussions about too frequent moving/rescheduling of tasks). Can
> someone provide the rational for this change? Was it just the result
> of making the code cleaner? Is it believed that this won't really
> make a difference?
i've done this change as part of the sched_yield() fixes/cleanups, and the
main reason for it is that the current process is preferred *anyway*, due
to getting the +1 boost via current->mm == this_mm in goodness().
(and besides, the percentage/probability of cases where we'd fail reselect
a runnable process where the previous scheduler would reselect it is very
very low. It does not justify adding a branch to the scheduler hotpath
IMO. In 99.9% of the cases if a runnable process is executing schedule()
then there is a higher priority process around that will win the next
selection. Or if there is a wakeup race, then the process will win the
selection very likely because it won the previous selection.)
Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Path: archiver1.google.com!news1.google.com!newsfeed.stanford.edu!newsfeeds.belnet.be!news.belnet.be!news.tele.dk!small.news.tele.dk!129.240.148.23!uio.no!nntp.uio.no!ifi.uio.no!internet-mailinglist
Newsgroups: fa.linux.kernel
Return-Path: <linux-kernel-ow...@vger.kernel.org>
Original-Date: Wed, 5 Dec 2001 13:58:51 -0800
From: Mike Kravetz <krav...@us.ibm.com>
To: Ingo Molnar <mi...@elte.hu>
Cc: linux-kernel <linux-ker...@vger.kernel.org>
Subject: Re: Scheduler Cleanup
Original-Message-ID: <20011205135851.D1193@w-mikek2.des.beaverton.ibm.com>
Original-References: <20011126114610.B1...@w-mikek2.des.beaverton.ibm.com> <Pine.LNX.4.33.0111280145300.3429-100...@localhost.localdomain>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.2.5i
In-Reply-To: <Pine.LNX.4.33.0111280145300.3429-100000@localhost.localdomain>; from mingo@elte.hu on Wed, Nov 28, 2001 at 02:07:08AM +0100
Sender: linux-kernel-ow...@vger.kernel.org
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
Organization: Internet mailing list
Date: Wed, 5 Dec 2001 22:06:01 GMT
Message-ID: <fa.lcouufv.73gdbo@ifi.uio.no>
References: <fa.o48rfgv.tlg614@ifi.uio.no>
Lines: 48
On Wed, Nov 28, 2001 at 02:07:08AM +0100, Ingo Molnar wrote:
>
> On Mon, 26 Nov 2001, Mike Kravetz wrote:
>
> > I'm happy to see the cleanup of scheduler code that went into
> > 2.4.15/16. One small difference in behavior (I think) is that the
> > currently running task is not given preference over other tasks on the
> > runqueue with the same 'goodness' value. I would think giving the
> > current task preference is a good thing (especially in light of recent
> > discussions about too frequent moving/rescheduling of tasks). Can
> > someone provide the rational for this change? Was it just the result
> > of making the code cleaner? Is it believed that this won't really
> > make a difference?
>
> i've done this change as part of the sched_yield() fixes/cleanups, and the
> main reason for it is that the current process is preferred *anyway*, due
> to getting the +1 boost via current->mm == this_mm in goodness().
>
> (and besides, the percentage/probability of cases where we'd fail reselect
> a runnable process where the previous scheduler would reselect it is very
> very low. It does not justify adding a branch to the scheduler hotpath
> IMO. In 99.9% of the cases if a runnable process is executing schedule()
> then there is a higher priority process around that will win the next
> selection. Or if there is a wakeup race, then the process will win the
> selection very likely because it won the previous selection.)
>
> Ingo
FYI - I have been having a heck of a time getting our MQ scheduler to
work well in 2.4.16/2.5.0. The problem was mainly with performance
when running (I'm afraid to say) VolanoMark. Turns out that the above
change makes a big difference in this benchmark when running with the
MQ scheduler. There is an almost 50% drop in performance on my 8 way.
I suspect that one would not see such a dramatic drop (if any) with
the current scheduler as its performance is mostly limited by lock
contention in this benchmark.
Now, I'm aware that very few people are actively using our MQ scheduler,
and even fewer care about VolanoMark performance (perhaps no one on this
list). However, this seemed like an interesting observation.
--
Mike
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Path: archiver1.google.com!news1.google.com!newsfeed.stanford.edu!news-spur1.maxwell.syr.edu!news.maxwell.syr.edu!nntp-relay.ihug.net!ihug.co.nz!newsfeed.media.kyoto-u.ac.jp!uio.no!nntp.uio.no!ifi.uio.no!internet-mailinglist
Newsgroups: fa.linux.kernel
Return-Path: <linux-kernel-ow...@vger.kernel.org>
Original-Date: Wed, 5 Dec 2001 15:44:42 -0800 (PST)
From: Davide Libenzi <davi...@xmailserver.org>
X-X-Sender: dav...@blue1.dev.mcafeelabs.com
To: Mike Kravetz <krav...@us.ibm.com>
cc: Ingo Molnar <mi...@elte.hu>, linux-kernel <linux-ker...@vger.kernel.org>
Subject: Re: Scheduler Cleanup
In-Reply-To: <20011205135851.D1193@w-mikek2.des.beaverton.ibm.com>
Original-Message-ID: <Pine.LNX.4.40.0112051539130.1644-100000@blue1.dev.mcafeelabs.com>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: linux-kernel-ow...@vger.kernel.org
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
Organization: Internet mailing list
Date: Wed, 5 Dec 2001 23:35:09 GMT
Message-ID: <fa.luqmoiv.u7mb0k@ifi.uio.no>
References: <fa.lcouufv.73gdbo@ifi.uio.no>
Lines: 62
On Wed, 5 Dec 2001, Mike Kravetz wrote:
> On Wed, Nov 28, 2001 at 02:07:08AM +0100, Ingo Molnar wrote:
> >
> > On Mon, 26 Nov 2001, Mike Kravetz wrote:
> >
> > > I'm happy to see the cleanup of scheduler code that went into
> > > 2.4.15/16. One small difference in behavior (I think) is that the
> > > currently running task is not given preference over other tasks on the
> > > runqueue with the same 'goodness' value. I would think giving the
> > > current task preference is a good thing (especially in light of recent
> > > discussions about too frequent moving/rescheduling of tasks). Can
> > > someone provide the rational for this change? Was it just the result
> > > of making the code cleaner? Is it believed that this won't really
> > > make a difference?
> >
> > i've done this change as part of the sched_yield() fixes/cleanups, and the
> > main reason for it is that the current process is preferred *anyway*, due
> > to getting the +1 boost via current->mm == this_mm in goodness().
> >
> > (and besides, the percentage/probability of cases where we'd fail reselect
> > a runnable process where the previous scheduler would reselect it is very
> > very low. It does not justify adding a branch to the scheduler hotpath
> > IMO. In 99.9% of the cases if a runnable process is executing schedule()
> > then there is a higher priority process around that will win the next
> > selection. Or if there is a wakeup race, then the process will win the
> > selection very likely because it won the previous selection.)
> >
> > Ingo
>
> FYI - I have been having a heck of a time getting our MQ scheduler to
> work well in 2.4.16/2.5.0. The problem was mainly with performance
> when running (I'm afraid to say) VolanoMark. Turns out that the above
> change makes a big difference in this benchmark when running with the
> MQ scheduler. There is an almost 50% drop in performance on my 8 way.
> I suspect that one would not see such a dramatic drop (if any) with
> the current scheduler as its performance is mostly limited by lock
> contention in this benchmark.
>
> Now, I'm aware that very few people are actively using our MQ scheduler,
> and even fewer care about VolanoMark performance (perhaps no one on this
> list). However, this seemed like an interesting observation.
Mike, before giving any scheduler benchmark a mean You've to verify if the
process distribution among CPUs is the same of the old scheduler.
Different process distributions can give very different numbers.
Anyway me too have verified an increased latency with sched_yield() test
and next days I'm going to try the real one with the cycle counter
sampler.
I've a suspect, but i've to see the disassembly of schedule() before
talking :)
- Davide
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Path: archiver1.google.com!news1.google.com!newsfeed.stanford.edu!news.tele.dk!small.news.tele.dk!129.240.148.23!uio.no!nntp.uio.no!ifi.uio.no!internet-mailinglist
Newsgroups: fa.linux.kernel
Return-Path: <linux-kernel-ow...@vger.kernel.org>
Original-Date: Wed, 5 Dec 2001 15:46:18 -0800
From: Mike Kravetz <krav...@us.ibm.com>
To: Davide Libenzi <davi...@xmailserver.org>
Cc: Ingo Molnar <mi...@elte.hu>, linux-kernel <linux-ker...@vger.kernel.org>
Subject: Re: Scheduler Cleanup
Original-Message-ID: <20011205154618.B24407@w-mikek2.des.beaverton.ibm.com>
Original-References: <20011205135851.D1...@w-mikek2.des.beaverton.ibm.com> <Pine.LNX.4.40.0112051539130.1644-100...@blue1.dev.mcafeelabs.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.2.5i
In-Reply-To: <Pine.LNX.4.40.0112051539130.1644-100000@blue1.dev.mcafeelabs.com>; from davidel@xmailserver.org on Wed, Dec 05, 2001 at 03:44:42PM -0800
Sender: linux-kernel-ow...@vger.kernel.org
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
Organization: Internet mailing list
Date: Wed, 5 Dec 2001 23:52:02 GMT
Message-ID: <fa.kd653kv.qg69qn@ifi.uio.no>
References: <fa.luqmoiv.u7mb0k@ifi.uio.no>
Lines: 19
On Wed, Dec 05, 2001 at 03:44:42PM -0800, Davide Libenzi wrote:
> Anyway me too have verified an increased latency with sched_yield() test
> and next days I'm going to try the real one with the cycle counter
> sampler.
> I've a suspect, but i've to see the disassembly of schedule() before
> talking :)
One thing to note is that possible acquisition of the runqueue
lock was reintroduced in sys_sched_yield(). From looking at
the code, it seems the purpose was to ?add fairness? in the case
of multiple yielders. Is that correct Ingo?
--
Mike
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Path: archiver1.google.com!news1.google.com!newsfeed.stanford.edu!news.tele.dk!small.news.tele.dk!129.240.148.23!uio.no!nntp.uio.no!ifi.uio.no!internet-mailinglist
Newsgroups: fa.linux.kernel
Return-Path: <linux-kernel-ow...@vger.kernel.org>
Original-Date: Thu, 6 Dec 2001 11:38:29 +0100 (CET)
From: Ingo Molnar <mi...@elte.hu>
Reply-To: <mi...@elte.hu>
To: Mike Kravetz <krav...@us.ibm.com>
Cc: Davide Libenzi <davi...@xmailserver.org>,
linux-kernel <linux-ker...@vger.kernel.org>
Subject: Re: Scheduler Cleanup
In-Reply-To: <20011205154618.B24407@w-mikek2.des.beaverton.ibm.com>
Original-Message-ID: <Pine.LNX.4.33.0112061123230.2778-100000@localhost.localdomain>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: linux-kernel-ow...@vger.kernel.org
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
Organization: Internet mailing list
Date: Thu, 6 Dec 2001 08:53:27 GMT
Message-ID: <fa.o3p9fov.t5m69d@ifi.uio.no>
References: <fa.kd653kv.qg69qn@ifi.uio.no>
Lines: 55
On Wed, 5 Dec 2001, Mike Kravetz wrote:
> One thing to note is that possible acquisition of the runqueue lock
> was reintroduced in sys_sched_yield(). From looking at the code, it
> seems the purpose was to ?add fairness? in the case of multiple
> yielders. Is that correct Ingo?
yes, it's to add fairness. You might remember that i did this
sched_yield() optimization originally to help Volanomark performance. But
it turned out that it's very unfair not to move yielded processes to the
end of the runqueue - it might even cause livelocks in user-space
spinlocks which use sched_yield(). (since the POLICY_YIELD bit prevents
the process from running only *once*, so it will handle a two-process lock
situation right, but if multiple processes are racing for the lock then
they might exclude the real owner of the spinlock for a *long* time.)
(plus the change also broke sched_yield() for RT-FIFO processes.)
(nevertheless i'm sure we can add any invariant optimizations or fixes to
sys_sched_yield(), but this one - no matter how nice it was to this
particular benchmark, was neither invariant, nor a fix.)
the pure fact that your workload is so sensitive to sched_yield() LIFO vs.
FIFO yielding of runnable threads shows that there is some serious locking
design problem. It shows that there are 1) too many threads running 2)
only a small subset of those runnable threads are doing the real work, and
the rest is only runnable for some unknown reason.
i think the user-space locking mechanizm the IBM JVM uses should *not* be
based on sched_yield(). [is it using LinuxThreads? LinuxThreads have some
serious design problems IMO.] One thing i remember from running Volanomark
was the huge amount of signal related, mostly bogus wakeups. This i
believe is mainly due to the manager thread design of LinuxThreads. That
aspect should be fixed i think, i believe it can be correctly done (we can
get rid of the manager thread) via the latest 2.4 kernel.
while i see the point that the multiqueue scheduler improves performance
visibly, i'm quite sure you could get an *order of magnitude* faster if
the basic threading model (and any possible underlying mechanizm, such as
LinuxThreads) was fixed. The current Linux scheduler just magnifies these
userspace design problems. LinuxThreads was done when there werent many
good mechanizms within the kernel to help threading. Things have changed
by today - but if something still cannot be done cleanly and efficiently
(such as userspace spinlocks or semaphores) then please let us know so we
can fix things, instead of trying to work around the symptoms. But this is
just a suggestion.
Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
|