Announcing CML2, a replacement for the kbuild system

From: "Eric S. Raymond" <e...@snark.thyrsus.com>
Subject: Announcing CML2, a replacement for the kbuild system
Date: 2000/05/24
Message-ID: <fa.gts9dev.1bg8r0l@ifi.uio.no>
X-Deja-AN: 626815711
Original-Date: Wed, 24 May 2000 13:39:17 -0400
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-Id: <200005241739.NAA09120@snark.thyrsus.com>
To: linux-ker...@vger.rutgers.edu, linux-kbu...@torque.net
X-Authentication-Warning: snark.thyrsus.com: esr set sender to e...@snark.thyrsus.com using -f
X-Orcpt: rfc822;linux-kernel-outgoing-dig
Organization: Internet mailing list
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu

For some weeks now, I have been developing a replacement for the kbuild 
system used to configure Linux kernels. This effort has had the support of
Michael Elizabeth Chastain, the principal kbuild maintainer, and has
benefited from input by others on the kbuild list.

The project is not yet complete, but it has reached a beta stage at
which it is usable and in significant ways functionally superior to the
present system. I am confident that it will complete. I am announcing
now rather than holding off until I'm completely done because there
are some preparations which, if begun now, will significantly reduce
total transition costs. These preparations will *not* break the
present kbuild system.

Why this project at all? It all started when I realized that building
kernels is way too hard. I wanted to simplify the configuration task
enough to make configuration accessible to non-gurus. It needs to
have more policy options. Rather than hundreds of questions like
"Include FOOBAR2317 driver?", the novice should see stuff like
"Compile in all modular drivers as modules without prompting?"

This just can't be done with the existing kbuild system. The existing
config-language programs are hard to read and modify, and the code
that interprets them has become a huge, unmaintainable hairball of
Tcl/Tk, C, makefiles, and shell. It has all become terminally
brittle, and the maintainers agree that it needs to be nuked and
rebuilt from scratch.

It happens that I love writing domain-specific minilanguages, so I have
tackled this problem head-on. I have designed a new configuration
language I call CML2 (the existing language I have retrospectively
named CML1). The implementation has two parts:

1. I have implemented a CML2 compiler that validates CML2 rulesets and
generates a rulebase that can be used to drive a configuration
process. I have translated almost all of the 7049 lines of CML1 in
the 2.3.99-pre9 source tree to validated CML2 (and *that*, believe me,
was hard work -- it took longer than the CML2 design and coding!).

2. I have written a configurator that is ready for testing. This
program reads in the rulebase and uses it to do the actual config-file
generation from a dialog with the user. Though not yet equipped with a
Tk interface, this program fully demonstrates the capabilities of
CML2. It runs in either line-oriented or curses mode depending on the
display environment (line-oriented mode can be forced with a
command-line switch).

The line-oriented mode of the new configurator is much more powerful
than the original Configure. It's possible to move backward or jump
around in the configuration sequence; the constraints that were
expressed by if-then-else logic in CML1 are now checked every time the
value of a relevant symbol is changed. It also has full access to the
help system.

The curses mode, unlike the old menuconfig code, also has full access
to the help text. It reports attempts to set symbol combinations that
would result in an invalid configuration.

The configurator should be able to present a Tk-based menu interface
when it detects that it's running on an X display. This is the part
I haven't written yet.

The code needs more testing, which is one reason I am announcing now.
It would be useful for configure maintainers to begin running through
odd configurations to see if they can get it to misbehave.

The first alpha of CML2 is available at

http://www.tuxedo.org/~esr/kbuild/

It includes the alpha implementation, documentation, and a transition
guide for maintainers of CML1 code.

OK, here's the bad news: the new system will not be an instant,
painless replacement for the old. I tried hard, but there was just
too much cruft to clean up for that to be possible.

The major source of problems is, as you might expect, that 6747-line
mass of old code -- the new language is nontrivially different than
the old, and the CML1 corpus is so tangled and nasty that I am certain
I have made at least a few mistakes in the translation. There are
a couple of places where I didn't understand the author's intentions
well enough to translate some particularly grotty CML1 code. I'll need
some help untangling these knots.

I apologize for this, but the translation overhead would only have
been avoidable if CML1 had been good enough not to replace. It's a
cost we have to pay to clean up a mess that would otherwise only have
gotten worse, and eventually have become a serious drag on kernel
development and porting.

There are some other minor problems, which we can fix up front. Mostly
they have to do with cleaning up the configuration-symbol namespace (which
would be a good idea even if we planned to keep CML1).

Now the good news: we will win big by changing over. Here are some
of the advantages of the new language:

1. Single parser and front end

CML1 had three different interpreters, none perfectly compatible with
any of the others. CML2 has one rule compiler and one rulebase-interpreter
front end. This will be good for consistency and economy.

2. A more expressive, easier-to-program configuration language

The rather spiky and cluttered shell-like syntax of CML1 is replaced
with a much simpler and more regular format resembling that of .netrc or
.fetchmailrc. More importantly, the semantics of the language are
declarative rather than imperative -- a better match for the problem
domain, and thus more expressive and easier to code in.

3. Drastic reduction in code size and complexity

The 7049 lines of CML1 in the 2.3.99-pre9 kernel translate to a hair
less than 2400 lines of CML2, a reduction by a factor of about three.
The CML2 compiler and prototype interpreter are the same factor of
three smaller than the nearly 10,000 lines of code in the CML1
interpreters and tools. Where CML1 is a complex mixture of C, shell,
Tcl/Tk, and Makefiles, CML2 is all be written in a single language
(Python).

4. Eliminates (or at least drastically reduces) lag between port configurations

The fact that the top-level CML1 files of the nine ports in the kernel
tree are separate means there have been plenty of opportunities for
the common code in them to suffer from version skew -- I point out
about a dozen bugs of this kind in the list of errors at the end of
this post. CML2's design and compilation rules should effectively
prevent future bugs of this kind.

5. Clean separation between configuration language and configuration UI

CML decouples the configuration language from the configuration user
interface (they communicate with each other only through the compiled
rulebase). This means that it will be relatively easy to improve the
UI and the language separately.

6. Internationalization

CML2 query prompts and menu banners are separated from the symbol
dependency declarations. Thus CML2 system definitions can be 
internationalized and localized.

7. Language is fully documented

CML2 has a complete, explicit description. Syntax, language semantics,
and front-end policy options are all discussed in detail. 

8. Policy-based options

The declarative semantics of CML2 makes it much easier to set up
and check interdependencies among symbols. I have done only 
enough of this in the CML1 translation for demonstration purposes (there
are new symbols TUNING, EXPERT and WIZARD that change some visibilities).
Once CML2 is in place, it should be a relatively small effort to
give the user a rich set of policy and don't-bother-me options.

So, how do we get there from here?

Obviously, I have to finish the CML2 front end. This is not a large
job; I already have a demonstrable prototype that runs in tty and
curses modes, and even on my heavy travel schedule I expect to have
the Tk version ready in about two weeks.

I have designed the CML2 implementation to coexist with CML1, so both
methods can be used while CML2 is being field-tested and debugged. I
anticipate a phase-in over three or four point releases during 2.5.x,
followed by a back-port to 2.4.x. Once CML2 is reported OK by the
various porting groups, Linus can quietly nuke the CML1 machinery.

There are a couple of preparation steps that can fruitfully begin now
and should happen before 2.4 in order to minimize backporting hassles
later. 

1. Notably, I would appreciate it if config-file maintainers made
the following changes in those files and relevant C code:

CONFIG_6xx -> CONFIG_PPC_6xx
CONFIG_4xx -> CONFIG_PPC_4xx
CONFIG_PPC64 -> CONFIG_PPC_64
CONFIG_8260 -> CONFIG_PPC_8260
CONFIG_8xx -> CONFIG_PPC_88x
CONFIG_060_WRITETHROUGH -> CONFIG_M68060_WRITETHROUGH
CONFIG_21285_WATCHDOG -> CONFIG_DC21285_WATCHDOG
CONFIG_3C515 -> CONFIG_ISA3C515
CONFIG_8139TOO -> CONFIG_RTL8139
CONFIG_82C710_MOUSE -> CONFIG_CT82C710_MOUSE
CONFIG_977_WATCHDOG -> CONFIG_WB83C977_WATCHDOG
CONFIG_3215 -> CONFIG_IBM3215
CONFIG_3215_CONSOLE -> CONFIG_IBM3215_CONSOLE

The reason for these is that CML2 symbol names drop the CONFIG_
prefix. It's unneeded clutter, and made CML1 programs harder to read
(the eye-brain systems that handle spelling look for prefix matches to
recognize things).

Also, I had to change the KEYBOARD and MOUSE symbols used in the MIPS branch
to MIPS_KEYBOARD and MIPS_MOUSE. This is because the MOUSE symbol
seems to be used in different ways on different architectures (notably
in the Intel branch).

2. I also found some apparent errors. I need these explained so I'll
know how to handle them in the translation. A summary of these apparent
errors is included at the end of this post.

3. Those are the easy parts. The hard part is that I'd like to ask
config maintainers to eyeball-check the CML2 translation of their work
*now*. Where I am most likely to have erred is in setting visibility
constraints by architecture. Ideally, I'd like everyone to have
confidence that the translation is correct by the time the Tk-based
front end comes out.

Presently the entire CML2 translation lives in a single file, rather than
being distributed into per-subdirectory files like the CML1 corpus. This
is a temporary expedient to make the transition easier. CML2's "source"
facility is quite powerful enough to support distributing the information
later on.

The current CML2 menu tree is ugly and poorly organized -- that is to
say, it has changed relatively little from the CML1 version. I am
deliberately refraining from large changes yet. Once we have tested
and switched over to CML2, it will be possible to do a complete
redesign of the kbuild user experience. The most important feature of
CML2 is that it will give us the capability to explore that design
space without risking breaking the ability to build kernels at all.

Here are the apparent errors I found in the CML1 corpus:

-----------------------------------------------------------------------------
There is what appears to be an error in the M68K configuration sequence.
Inside a PARPORT guard, the question 'Q40 Parallel port' sets PARPORT again.
I created a PARPORT_Q40 symbol and set it from this question.

The symbols SGI occurs in conditionals in config files but are never
set or associated with a query, nor are they used in C code anywere.

The symbol SUN3 is used in conditionals and set to n at one point, but there
is no place where it is set to y.

The symbol FB_CONSOLE is set at one point but
never used in either C or config language code.

The symbol ABSTRACT_CONSOLE is not used in C code, nor set anywhere in config
code.

The symbol AIC7XXX_TAGGED_QUEUEING is set in the sparc64 configuration
code, but not used in C code. I suspect it should be
AIC7XXX_TCQ_ON_BY_DEFAULT.

The symbol ADB_PMU68K defined in the M68K driver is not referenced anywhere 
in the config code and not used in the C code. It seems to be a misspelling
of ADB_PMU. I have eliminated it.

In the ARM port configuration, symbols ARCH_TBOX and ARCH_SHARK and
ARCH_NEXUSPCI and ARCH_NEXUSPCI are referenced but never associated
with a query or defined.

There are two symbols in the configuration code that seem to relate
to endianness on processors that can operate in either big-endian 
or little-endian mode. One is CPU_LITTLE_ENDIAN (in the MIPS ports)
and the other is LITTLE_ENDIAN (in the SuperH port). Neither is used 
in the C code; CPU_LITTLE_ENDIAN is used in a guard, once, in the
MIPS32 config. I have changed LITTLE_ENDIAN to CPU_LITTLE_ENDIAN.

The symbol PRINTER_READBACK is queried for once, but never used in the
config or C code. I suspect it should have been asking for PARPORT_1284.
-----------------------------------------------------------------------------

Note: this announcement was crossposted to the linux-kernel and linux-kbuild
lists. You may want to use group reply in respnding to it to reach both
populations.

Port maintainers and others with a continuing interest in the development
of CML2 should probably join the kbuild list -- subscribe in the usual
way via linux-kbuild-requ...@torque.net
-- 
<a href="http://www.tuxedo.org/~esr">Eric S. Raymond</a>

The Bible is not my book, and Christianity is not my religion. I could never
give assent to the long, complicated statements of Christian dogma.
-- Abraham Lincoln

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: Gianluca Anzolin <g.anzo...@libero.it>
Subject: Re: Announcing CML2, a replacement for the kbuild system
Date: 2000/05/24
Message-ID: <fa.mdrnhev.1d1ks0b@ifi.uio.no>#1/1
X-Deja-AN: 626862249
Original-Date: Wed, 24 May 2000 21:10:48 +0200 (CEST)
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-ID: <Pine.LNX.4.21.0005242110110.725-100000@dracula.home.intranet>
References: <fa.gts9dev.1bg8r0l@ifi.uio.no>
To: "Eric S. Raymond" <e...@snark.thyrsus.com>
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-Orcpt: rfc822;linux-kernel-outgoing-dig
Organization: Internet mailing list
MIME-Version: 1.0
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu

Only a problem:

File Not found

The requested URL /~esr/kbuild/cml2-0.1.0.tar.gz was not found on this
server.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: Alexander Viro <v...@math.psu.edu>
Subject: Re: Announcing CML2, a replacement for the kbuild system
Date: 2000/05/24
Message-ID: <fa.lctlgvv.272nb3@ifi.uio.no>#1/1
X-Deja-AN: 626891271
Original-Date: Wed, 24 May 2000 16:17:55 -0400 (EDT)
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-ID: <Pine.GSO.4.10.10005241608350.17448-100000@weyl.math.psu.edu>
References: <fa.mdrnhev.1d1ks0b@ifi.uio.no>
To: Gianluca Anzolin <g.anzo...@libero.it>
X-Authentication-Warning: weyl.math.psu.edu: viro owned process doing -bs
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-Orcpt: rfc822;linux-kernel-outgoing-dig
Organization: Internet mailing list
MIME-Version: 1.0
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu



On Wed, 24 May 2000, Gianluca Anzolin wrote:

> Only a problem:
> 
> File Not found
> 
> The requested URL /~esr/kbuild/cml2-0.1.0.tar.gz was not found on this
> server.

Not only. Dependency on Python is definitely a problem - not everyone uses
'everything and a kitchen sink' type of userland.


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: Alan Cox <a...@lxorguk.ukuu.org.uk>
Subject: Re: Announcing CML2, a replacement for the kbuild system
Date: 2000/05/24
Message-ID: <fa.gk118gv.1j1cs0s@ifi.uio.no>#1/1
X-Deja-AN: 626950206
Original-Date: Wed, 24 May 2000 22:23:41 +0100 (BST)
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-Id: <E12uicy-0006G4-00@the-village.bc.nu>
Content-Transfer-Encoding: 7bit
References: <fa.lctlgvv.272nb3@ifi.uio.no>
To: v...@math.psu.edu (Alexander Viro)
Content-Type: text/plain; charset=us-ascii
X-Orcpt: rfc822;linux-kernel-outgoing-dig
Organization: Internet mailing list
MIME-Version: 1.0
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu

> > The requested URL /~esr/kbuild/cml2-0.1.0.tar.gz was not found on this
> > server.
> 
> Not only. Dependency on Python is definitely a problem - not everyone uses
> 'everything and a kitchen sink' type of userland.

Python is actually pretty small and you need it on the build box not on the
runtime host.


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: Jonathan Walther <kroo...@debian.org>
Subject: Re: Announcing CML2, a replacement for the kbuild system
Date: 2000/05/27
Message-ID: <fa.i1gs5lv.b1a233@ifi.uio.no>#1/1
X-Deja-AN: 627796940
Original-Date: Fri, 26 May 2000 17:32:31 -0700 (PDT)
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-ID: <Pine.LNX.3.96.1000526173056.28897B-100000@localhost>
References: <fa.eqd4pvv.heg11@ifi.uio.no>
To: Dominik Kubla <dominik.ku...@uni-mainz.de>
X-Sender: jwalther@localhost
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-Orcpt: rfc822;linux-kernel-outgoing-dig
Organization: Internet mailing list
MIME-Version: 1.0
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu

-----BEGIN PGP SIGNED MESSAGE-----

I agree; why CAN'T we configure the linux kernel more like we do the
BSD ones? Their code and schema is very elegant and simple. And
the source shouldn't be too hard to port over. David Parsons? Oh
thats right, Linus ignores his patches. Oh well.

--sig--
It isn't true unless it makes you laugh, but you don't
really understand it till it makes you cry.

On Wed, 24 May 2000, Dominik Kubla wrote:
> I really don't like the added complexity. IMHO we should ditch our current
> config scheme in favour of the BSD tools if at all: They are written in
> C, they are simple to use and they have a proven track record. What more
> is there to ask?

-----BEGIN PGP SIGNATURE-----
Version: 2.6.3ia
Charset: noconv

iQCVAwUBOS8XoMK9HT/YfGeBAQE9IQP/QhVL5NSrDp+8AYtGAFROsp+A/pm1fSDa
pF03tUeffM+iudtUv3Z2rzalXqT6QyPUWgO6HKzsuED/5wfeSzfrUBWXW3O1STQd
2x9XiglqA+6oHIG5jGuIOMWisuYrUMLWVtcb1x4Ar88z/AzK7zW//KnNXg8UzPiy
3ZE+QwtiiDQ=
=ruRy
-----END PGP SIGNATURE-----


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: Alan Cox <a...@lxorguk.ukuu.org.uk>
Subject: Re: Announcing CML2, a replacement for the kbuild system
Date: 2000/05/27
Message-ID: <fa.gjgn30v.13i2qgr@ifi.uio.no>#1/1
X-Deja-AN: 627803177
Original-Date: Sat, 27 May 2000 01:34:21 +0100 (BST)
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-Id: <E12vUYc-0002Z9-00@the-village.bc.nu>
Content-Transfer-Encoding: 7bit
References: <fa.i1gs5lv.b1a233@ifi.uio.no>
To: kroo...@debian.org (Jonathan Walther)
Content-Type: text/plain; charset=us-ascii
X-Orcpt: rfc822;linux-kernel-outgoing-dig
Organization: Internet mailing list
MIME-Version: 1.0
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu

> I agree; why CAN'T we configure the linux kernel more like we do the
> BSD ones? Their code and schema is very elegant and simple. And

The BSD setup is not end user friendly. There is actually only one fundamental
end user problem with our current setup. When a user says 'I want XYZ' it
should turn on everything needed to get XYZ.

If I hit my capture card option it should turn on i2c, video4linux,procfs, etc

Alan


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: "Eric S. Raymond" <e...@thyrsus.com>
Subject: Re: [KBUILD] Re: Announcing CML2, a replacement for the kbuild system
Date: 2000/05/27
Message-ID: <fa.fvlhc7v.v6e0pi@ifi.uio.no>#1/1
X-Deja-AN: 627808266
Original-Date: Fri, 26 May 2000 21:22:18 -0400
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-ID: <20000526212218.C20668@thyrsus.com>
References: <fa.gjgn30v.13i2qgr@ifi.uio.no>
To: Alan Cox <a...@lxorguk.ukuu.org.uk>
Original-References: <Pine.LNX.3.96.1000526173056.28897B-100000@localhost> <E12vUYc-0002Z9...@the-village.bc.nu>
Content-Type: text/plain; charset=us-ascii
X-Orcpt: rfc822;linux-kernel-outgoing-dig
X-Eric-Conspiracy: There is no conspiracy
Organization: Eric Conspiracy Secret Labs
Mime-Version: 1.0
Reply-To: e...@thyrsus.com
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu

Alan Cox <a...@lxorguk.ukuu.org.uk>:
> The BSD setup is not end user friendly. There is actually only one
> fundamental end user problem with our current setup. When a user says 
> 'I want XYZ' it should turn on everything needed to get XYZ.
> 
> If I hit my capture card option it should turn on i2c, video4linux,procfs,
> etc

CML2 is aimed at this. It will never be perfect -- perfection would require
a full theorem prover -- but it's already good enough to handle normal
ancestry relationships. 

One of the purposes of making the CML2 language declarative rather than
imperative is to make all the dependencies explicit, rather than implied
by control structure as in CML1. Because that's so, over time it will
be possible to improve the deductive algoritms in the front end without
having to endure another replacement of the language.
-- 
<a href="http://www.tuxedo.org/~esr">Eric S. Raymond</a>

A human being should be able to change a diaper, plan an invasion,
butcher a hog, conn a ship, design a building, write a sonnet, balance
accounts, build a wall, set a bone, comfort the dying, take orders, give
orders, cooperate, act alone, solve equations, analyze a new problem,
pitch manure, program a computer, cook a tasty meal, fight efficiently,
die gallantly. Specialization is for insects.
-- Robert A. Heinlein Time Enough for Love

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: Alan Cox <a...@lxorguk.ukuu.org.uk>
Subject: Re: [KBUILD] Re: Announcing CML2, a replacement for the kbuild system
Date: 2000/05/27
Message-ID: <fa.gq3j40v.1s7qrgr@ifi.uio.no>#1/1
X-Deja-AN: 627815248
Original-Date: Sat, 27 May 2000 02:45:47 +0100 (BST)
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-Id: <E12vVfg-0002fZ-00@the-village.bc.nu>
Content-Transfer-Encoding: 7bit
References: <fa.fvlhc7v.v6e0pi@ifi.uio.no>
To: e...@thyrsus.com
Content-Type: text/plain; charset=us-ascii
X-Orcpt: rfc822;linux-kernel-outgoing-dig
Organization: Internet mailing list
MIME-Version: 1.0
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu

> CML2 is aimed at this. It will never be perfect -- perfection would require
> a full theorem prover -- but it's already good enough to handle normal
> ancestry relationships. 

Our relationships are heirarchical so you can build a dependancy graph and
then recursively backtrack to find the dependancies. 

Alan


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: "Eric S. Raymond" <e...@thyrsus.com>
Subject: Re: [KBUILD] Re: Announcing CML2, a replacement for the kbuild system
Date: 2000/05/27
Message-ID: <fa.fv5ha7v.omm1pn@ifi.uio.no>#1/1
X-Deja-AN: 627817903
Original-Date: Fri, 26 May 2000 22:07:36 -0400
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-ID: <20000526220736.A21045@thyrsus.com>
References: <fa.gq3j40v.1s7qrgr@ifi.uio.no>
To: Alan Cox <a...@lxorguk.ukuu.org.uk>
Original-References: <20000526212218.C20...@thyrsus.com> <E12vVfg-0002fZ...@the-village.bc.nu>
Content-Type: text/plain; charset=us-ascii
X-Orcpt: rfc822;linux-kernel-outgoing-dig
X-Eric-Conspiracy: There is no conspiracy
Organization: Eric Conspiracy Secret Labs
Mime-Version: 1.0
Reply-To: e...@thyrsus.com
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu

Alan Cox <a...@lxorguk.ukuu.org.uk>:
> > CML2 is aimed at this. It will never be perfect -- perfection would require
> > a full theorem prover -- but it's already good enough to handle normal
> > ancestry relationships. 
> 
> Our relationships are heirarchical so you can build a dependancy graph and
> then recursively backtrack to find the dependancies. 

Yeah, but it's a DAG rather than a tree. There are other things that 
complexify it, too. Like the difference between "suppress dependant"
and "suppress" -- sometimes ancestor symbols affect the valid range
of a symbol, sometimes they don't.

Consider the case where you fail a contraint while backtracking. It's
not obvious what the right policy is -- I've identified at least four
possibilities and I can think of circumstances where each one is right
and the other three wrong.

I certainly don't have all the answers. But I'm trying to get us to 
a place where we can explore the questions without tripping over our
own damn feet.
-- 
<a href="http://www.tuxedo.org/~esr">Eric S. Raymond</a>

"The bearing of arms is the essential medium through which the
individual asserts both his social power and his participation in
politics as a responsible moral being..."
-- J.G.A. Pocock, describing the beliefs of the founders of the U.S.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: Alan Cox <a...@lxorguk.ukuu.org.uk>
Subject: Re: [KBUILD] Re: Announcing CML2, a replacement for the kbuild system
Date: 2000/05/27
Message-ID: <fa.gr3j40v.1r7urgr@ifi.uio.no>#1/1
X-Deja-AN: 627818725
Original-Date: Sat, 27 May 2000 02:59:13 +0100 (BST)
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-Id: <E12vVsg-0002hM-00@the-village.bc.nu>
Content-Transfer-Encoding: 7bit
References: <fa.fv5ha7v.omm1pn@ifi.uio.no>
To: e...@thyrsus.com
Content-Type: text/plain; charset=us-ascii
X-Orcpt: rfc822;linux-kernel-outgoing-dig
Organization: Internet mailing list
MIME-Version: 1.0
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu

> Yeah, but it's a DAG rather than a tree. There are other things that 
> complexify it, too. Like the difference between "suppress dependant"

The backtracking is the same for the DAG except you need to remember
'already visited' nodes so that you don't look finding the solution path.
If you already visited a node then its dependancies are either resolved
or you are resolving them now and they are resolved if you can resolve
without rewalking the path.

Alan


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: Alexander Viro <v...@math.psu.edu>
Subject: Re: [KBUILD] Re: Announcing CML2, a replacement for the kbuild system
Date: 2000/05/27
Message-ID: <fa.lcthenv.472nj2@ifi.uio.no>#1/1
X-Deja-AN: 627811122
Original-Date: Fri, 26 May 2000 21:27:06 -0400 (EDT)
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-ID: <Pine.GSO.4.10.10005262116010.17448-100000@weyl.math.psu.edu>
References: <fa.fvlhc7v.v6e0pi@ifi.uio.no>
To: "Eric S. Raymond" <e...@thyrsus.com>
X-Authentication-Warning: weyl.math.psu.edu: viro owned process doing -bs
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-Orcpt: rfc822;linux-kernel-outgoing-dig
Organization: Internet mailing list
MIME-Version: 1.0
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu

On Fri, 26 May 2000, Eric S. Raymond wrote:

> One of the purposes of making the CML2 language declarative rather than
> imperative is to make all the dependencies explicit, rather than implied
> by control structure as in CML1. Because that's so, over time it will
> be possible to improve the deductive algoritms in the front end without
> having to endure another replacement of the language.

Sigh... I suspect that it's the real reason why your approach doesn't make
sense to me - you are developing complex stuff that works for FUBAR domain
instead of getting the domain fixed.

Try to grep the tree for CONFIG_FOO - you'll see that most of the places
that care are in Makefiles. 99% of the rest is not needed - check the
history of fs/filesystems.c for example.

IOW, instead of developing clever ways to handle complex dependencies
we'ld be better off simplifying them and leaving the rest to make(1)...

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: "Eric S. Raymond" <e...@thyrsus.com>
Subject: Re: [KBUILD] Re: Announcing CML2, a replacement for the kbuild system
Date: 2000/05/27
Message-ID: <fa.g055d0v.umq11p@ifi.uio.no>#1/1
X-Deja-AN: 627813691
Original-Date: Fri, 26 May 2000 21:52:22 -0400
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-ID: <20000526215222.H20668@thyrsus.com>
References: <fa.lcthenv.472nj2@ifi.uio.no>
To: Alexander Viro <v...@math.psu.edu>
Original-References: <20000526212218.C20...@thyrsus.com> <Pine.GSO.4.10.10005262116010.17448-100...@weyl.math.psu.edu>
Content-Type: text/plain; charset=us-ascii
X-Orcpt: rfc822;linux-kernel-outgoing-dig
X-Eric-Conspiracy: There is no conspiracy
Organization: Eric Conspiracy Secret Labs
Mime-Version: 1.0
Reply-To: e...@thyrsus.com
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu

Alexander Viro <v...@math.psu.edu>:
> > One of the purposes of making the CML2 language declarative rather than
> > imperative is to make all the dependencies explicit, rather than implied
> > by control structure as in CML1. Because that's so, over time it will
> > be possible to improve the deductive algoritms in the front end without
> > having to endure another replacement of the language.
> 
> Sigh... I suspect that it's the real reason why your approach doesn't make
> sense to me - you are developing complex stuff that works for FUBAR domain
> instead of getting the domain fixed.

One war at a time, Alex. If we try to replace the whole Makefile system
at the same time that we clean up the Augean stable that is CML1 it will
create havoc.

> Try to grep the tree for CONFIG_FOO - you'll see that most of the places
> that care are in Makefiles. 99% of the rest is not needed - check the
> history of fs/filesystems.c for example.

You may be right. But CML2 is simpler than what it's replacing by a
large (and readily measurable) factor, so it's a net win to make that
change anyway. *Then* we can go clean up the rest of the build
system.

> IOW, instead of developing clever ways to handle complex dependencies
> we'ld be better off simplifying them and leaving the rest to make(1)...

If that were possible, I have little doubt it would have been done already.
But I tell you what. After phase II, when we fix the rest of the build 
system, I promise to seriously re-examine the question of whether CML2
is really needed. If it isn't, we'll throw it away.

I mean that. There are other uses for CML2 -- VA wants me to make it
into a product configurator, among other things -- so the work won't
have been wasted even if we throw away the kernel rulebase.
--
<a href="http://www.tuxedo.org/~esr">Eric S. Raymond</a>

False is the idea of utility that sacrifices a thousand real advantages for
one imaginary or trifling inconvenience; that would take fire from men because
it burns, and water because one may drown in it; that has no remedy for evils
except destruction. The laws that forbid the carrying of arms are laws of
such a nature. They disarm only those who are neither inclined nor determined
to commit crimes.
-- Cesare Beccaria, as quoted by Thomas Jefferson's Commonplace book

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: Alexander Viro <v...@math.psu.edu>
Subject: Re: [KBUILD] Re: Announcing CML2, a replacement for the kbuild system
Date: 2000/05/27
Message-ID: <fa.lcdrf7v.5n8n37@ifi.uio.no>#1/1
X-Deja-AN: 627819349
Original-Date: Fri, 26 May 2000 22:02:16 -0400 (EDT)
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-ID: <Pine.GSO.4.10.10005262145530.17448-100000@weyl.math.psu.edu>
References: <fa.g055d0v.umq11p@ifi.uio.no>
To: "Eric S. Raymond" <e...@thyrsus.com>
X-Authentication-Warning: weyl.math.psu.edu: viro owned process doing -bs
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-Orcpt: rfc822;linux-kernel-outgoing-dig
Organization: Internet mailing list
MIME-Version: 1.0
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu

On Fri, 26 May 2000, Eric S. Raymond wrote:

> > Sigh... I suspect that it's the real reason why your approach doesn't make
> > sense to me - you are developing complex stuff that works for FUBAR domain
> > instead of getting the domain fixed.
> 
> One war at a time, Alex. If we try to replace the whole Makefile system
> at the same time that we clean up the Augean stable that is CML1 it will
> create havoc.

You are missing the real Augean stable - output of
% find /usr/src/linux -type f|xargs grep -n '#ifdef' /dev/null
- that's where the real dung is, problems with configuration complexity
are direct results of it...

> > Try to grep the tree for CONFIG_FOO - you'll see that most of the places
> > that care are in Makefiles. 99% of the rest is not needed - check the
> > history of fs/filesystems.c for example.
> 
> You may be right. But CML2 is simpler than what it's replacing by a
> large (and readily measurable) factor, so it's a net win to make that
> change anyway. *Then* we can go clean up the rest of the build
> system.

Oh, that's fine - the fact that new variant is more compact and readable
is a Good Thing(tm), no arguments here.

> > IOW, instead of developing clever ways to handle complex dependencies
> > we'ld be better off simplifying them and leaving the rest to make(1)...
> 
> If that were possible, I have little doubt it would have been done already.

It _is_ beeing done. Been there, done some parts of that. Seriously, one
of the goals to keep in mind is <strong> the less #ifdef's the better
</strong>. For a lot of obvious reasons, not the last of them being that
if the majority of config options will affect only the choice of object
files to be linked we will have a chance to get automated testing of the
whole thing - at least on the "does it compile?" level.

BTW, I'ld really recommend to run the search - you'll see the heaploads of
crap and there used to be more of that stuff. Some pieces resemble typical
GNU code - #ifdef on every third line...

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: "Eric S. Raymond" <e...@thyrsus.com>
Subject: Re: [KBUILD] Re: Announcing CML2, a replacement for the kbuild system
Date: 2000/05/27
Message-ID: <fa.fsl3bfv.q6s1hi@ifi.uio.no>#1/1
X-Deja-AN: 627844910
Original-Date: Sat, 27 May 2000 00:26:21 -0400
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-ID: <20000527002621.B21323@thyrsus.com>
References: <fa.lcdrf7v.5n8n37@ifi.uio.no>
To: Alexander Viro <v...@math.psu.edu>
Original-References: <20000526215222.H20...@thyrsus.com> <Pine.GSO.4.10.10005262145530.17448-100...@weyl.math.psu.edu>
Content-Type: text/plain; charset=us-ascii
X-Orcpt: rfc822;linux-kernel-outgoing-dig
X-Eric-Conspiracy: There is no conspiracy
Organization: Eric Conspiracy Secret Labs
Mime-Version: 1.0
Reply-To: e...@thyrsus.com
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu

Alexander Viro <v...@math.psu.edu>:
> You are missing the real Augean stable - output of
> % find /usr/src/linux -type f|xargs grep -n '#ifdef' /dev/null
> - that's where the real dung is, problems with configuration complexity
> are direct results of it...

Ghods. You're right, that is pretty nasty. The *next* war...
-- 
<a href="http://www.tuxedo.org/~esr">Eric S. Raymond</a>

The day will come when the mystical generation of Jesus by the Supreme
Being as his father, in the womb of a virgin, will be classed with the
fable of the generation of Minerva in the brain of Jupiter.
-- Thomas Jefferson, 1823

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: Alexander Viro <v...@math.psu.edu>
Subject: Re: [KBUILD] Re: Announcing CML2, a replacement for the kbuild system
Date: 2000/05/27
Message-ID: <fa.lbtff7v.674n31@ifi.uio.no>#1/1
X-Deja-AN: 627859528
Original-Date: Sat, 27 May 2000 01:48:33 -0400 (EDT)
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-ID: <Pine.GSO.4.10.10005270123130.17448-100000@weyl.math.psu.edu>
References: <fa.fsl3bfv.q6s1hi@ifi.uio.no>
To: "Eric S. Raymond" <e...@thyrsus.com>
X-Authentication-Warning: weyl.math.psu.edu: viro owned process doing -bs
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-Orcpt: rfc822;linux-kernel-outgoing-dig
Organization: Internet mailing list
MIME-Version: 1.0
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu

On Sat, 27 May 2000, Eric S. Raymond wrote:

> Alexander Viro <v...@math.psu.edu>:
> > You are missing the real Augean stable - output of
> > % find /usr/src/linux -type f|xargs grep -n '#ifdef' /dev/null
> > - that's where the real dung is, problems with configuration complexity
> > are direct results of it...
> 
> Ghods. You're right, that is pretty nasty. The *next* war...

Feel free to join... Another source of fun: take a random file
from inlude/*/, pick a random prototype defined there, grep the tree for
said beast. Estimate the probability of finding the function in question
a) called by somebody
b) defined somewhere
c) (a) && !(b)
Admittedly, P(c) is pretty low, but non-zero. P(!b) is _far_ from zero.
Now, for _real_ fun grep for LINUX_VERSION_CODE. Pay attention to
advansys.c - there are some real pearls (most of their, erm, analogs in
other SCSI drivers had been flushed in October, hell knows how did these
ones crawl back).

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: Brian Gerst <bge...@quark.vpplus.com>
Subject: Re: [KBUILD] Re: Announcing CML2, a replacement for the kbuild system
Date: 2000/05/27
Message-ID: <fa.hjmrqfv.1bg82jc@ifi.uio.no>#1/1
X-Deja-AN: 627871264
Original-Date: Sat, 27 May 2000 02:49:14 -0400
Sender: owner-linux-ker...@vger.rutgers.edu
Content-Transfer-Encoding: 7bit
Original-Message-ID: <392F6FEA.D8FA1F61@quark.vpplus.com>
References: <fa.lbtff7v.674n31@ifi.uio.no>
To: Alexander Viro <v...@math.psu.edu>
Original-References: <Pine.GSO.4.10.10005270123130.17448-100...@weyl.math.psu.edu>
X-Accept-Language: en
Content-Type: text/plain; charset=us-ascii
X-Orcpt: rfc822;linux-kernel-outgoing-dig
Organization: Internet mailing list
MIME-Version: 1.0
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu

Alexander Viro wrote:
> Admittedly, P(c) is pretty low, but non-zero. P(!b) is _far_ from zero.
> Now, for _real_ fun grep for LINUX_VERSION_CODE. Pay attention to
> advansys.c - there are some real pearls (most of their, erm, analogs in
> other SCSI drivers had been flushed in October, hell knows how did these
> ones crawl back).

I sent in a patch to Linus once to clean that that trash up but it was
only partially applied. Advansys.c was the part that wasn't applied. I
can shave off almost 100k from it by dropping stuff for earlier that
2.2.x.

--
Brian Gerst

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

From: Alan Cox <a...@lxorguk.ukuu.org.uk>
Subject: Re: [KBUILD] Re: Announcing CML2, a replacement for the kbuild system
Date: 2000/05/27
Message-ID: <fa.h2gur0v.12keigq@ifi.uio.no>#1/1
X-Deja-AN: 627956047
Original-Date: Sat, 27 May 2000 15:13:24 +0100 (BST)
Sender: owner-linux-ker...@vger.rutgers.edu
Original-Message-Id: <E12vhLC-0003eJ-00@the-village.bc.nu>
Content-Transfer-Encoding: 7bit
References: <fa.hjmrqfv.1bg82jc@ifi.uio.no>
To: bge...@quark.vpplus.com (Brian Gerst)
Content-Type: text/plain; charset=us-ascii
X-Orcpt: rfc822;linux-kernel-outgoing-dig
Organization: Internet mailing list
MIME-Version: 1.0
Newsgroups: fa.linux.kernel
X-Loop: majord...@vger.rutgers.edu

> only partially applied. Advansys.c was the part that wasn't applied. I
> can shave off almost 100k from it by dropping stuff for earlier that
> 2.2.x.

Advansys maintain the driver as a single common module. Last I heard they
wanted it left like it is, and since they do the maintaining who are we
to bitch 8)



-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/