Newsgroups: comp.unix.bsd
Path: sparky!uunet!gatech!usenet.ins.cwru.edu!agate!tfs.com!julian
From: jul...@tfs.com (Julian Elischer)
Subject: transcript of on-line discussion re-shared libs
Message-ID: <1992Dec5.011417.7070@tfs.com>
Organization: Trw Financial Systems
Date: Sat, 5 Dec 1992 01:14:17 GMT
Lines: 974
This is a brief transcript of the on-line sessionbetween various members
of the shared-libs SIG and the kernel SIG, and Bill jolitz.
it is mainly of interrest to people working in those areas.
sorry about the typing, but it was 'live online' and we were all making lots of typos.
people on-line were:
Name Host
Paul Richards thor.cf.ac.uk
stripes frob.eng.umd.edu
julian tfs.COM
Alistair Crooks charon.amdahl.com
Jonas Olsson caisr06169.CAISR.CWRU
Jim Bachesta pilchuck.data-io.com
Thorsten Lockert amanda.bbb.no
GlenOverby cherry01.cray.com
Jordan K. Hubbard 192.122.228.3
Wolfgang Solfrank kurt.TooLs.DE
Bruce Bauman donald.osf.org
bill ref.tfs.com
Chet odin.INS.CWRU.Edu
Tuomas Lukka lk-hp-8.hut.fi
Peter Dufault marley.alliant.com
Mike Dickson mike.inhouse.compuserve.com
Peng-Toh cory.Berkeley.EDU
terry icarus.weber.edu
Holger Veit du9ds3.fb9dv.uni-duisburg.de
Charles Hannum hal.gnu.ai.mit.edu
- session logging begins----
(9, bill) thanks, glad to
(9, bill) be here!
(1, julian) that's 10 people so far..
(1, julian) if this looks confusing, then you MAY like to try tf
(1, julian) (if you know the terminal type etc.
(1, julian)
(1, julian) it gives you a split screen
(9, bill) No, I'm fine
(9, bill) It's just the morning.
(1, julian) not your time of day?
(9, bill) I've got my X11 setup, so I
(9, bill) can reference source while we're talking.
(1, julian) ok, neat..
(1, julian) BTW the remrq error:
(1, julian) is called from:
(1, julian) panic,remrq,softclk,hardclk,Vclk,rexit,syscall
(1, julian) I've seen that sequence twice now
(9, bill) Interesting,
(1, julian) still people turning up.. should we begin everybody?
(4, Alistair Crooks) yes
(3, Jim Bachesta) sounds good to me.
(6, Chet Ramey) yes
(1, julian) ok, I suppose the best place to start would be to say where we are at the moment..
(1, julian) I'll summarise
(1, julian)
(1, julian) We have two working versions of shared libs.
(1, julian) they are:
(1, julian) the one ported onto the net, (which seems to be similar in style to the linux
(1, julian) version though
(1, julian) I'm not sure of it's lieage.
(1, julian) and we have the linux shared lib support ported to 386bsd by Terry.
(1, julian) There is also a user in er..Denmark (I think) that has told me that they
(1, julian) are a week or so
(1, julian) away from a working shared libs implimentation.
(1, julian)
(1, julian) We have looked at both of these
(1, julian) and they have severe limitations, but at least give us some idea
(1, julian) of
(1, julian) what can be done,
(1, julian) (and we can now say to linux types that we DO have shared libs too 8-)
(1, julian)
(1, julian) we have looked at the OSF loader, which is PD
(1, julian) (in ~boot on ref if you want to look)
(1, julian) and at the SUN loader (also PD, also in ~boot)
(1, julian) both seen quite....
(1, julian) "patched' shall we say.
(1, julian) anyhow,
(1, julian) bruce Bauman from OSF (here I see) says that they may be
(1, julian) rewriting part of the OSF loader.. maybe he can tell us if that will be
(1, julian) PD too.
(1, julian) bruce?
(8, Bruce Bauman) We are adding shared library support to the latest GNU linker (the one in
(8, Bruce Bauman)
(8, Bruce Bauman) Sorry, I had a phone call. We are only switching linkers,
(8, Bruce Bauman) the loader will remain pretty much the same, but with
(8, Bruce Bauman) ELF and DWARF support.
(9, bill) good ,
(9, bill) I 'd like to see ELF DWARf,
(9, bill) as that's been the format I'm
(9, bill) plannoing to support long-term
(9, bill) in 386bsd. is this by chance the same stuff
(9, bill) that cygnus is supportuing?
(8, Bruce Bauman) We will probably work with Cygnus to get a gas version with ELF/DWARF.
(8, Bruce Bauman) I have an 11:30 meeting, but I'll try to sneak back in a
(8, Bruce Bauman) few minutes.
(8, Bruce Bauman)
(1, julian) I notice the cygnus GAS supporter has joined the mailing lists
(1, julian) an he MAY turn up later too.
(1, julian) Bill, I know we are all dying to hear about what changes you envision
(1, julian) on 0.2 to support shared libs
(1, julian) can you give us a quick rundown on your relevant plans.
(9, bill) Thanks, (julian, in answer to your question,
(9, bill) I hope that the shared libararysd won't require the kernel
(9, bill) to change because
(9, bill) I'd like the implementation to still be able to be
(9, bill) modified, since the ideas are still evolving.
(9, bill) What I'd like to see is:
(9, bill) a shareldlib impl. that is not tied down to
(9, bill) specific VA spaces (for threads and other things), it
(9, bill) reasolve at crt time with a discrete boot loader, and that
(9, bill) the only action to the kernel be an mmap() of each library mapped.
(9, bill) what I enviassion is that the executable will load,
(9, bill) then crt in the image will be called (could be at any VA,
(9, bill) it
(9, bill) runs PC reletiver), and then it builds the transfer table
(9, bill) and maps consecutive libraryus in PAGE rouned up frames, adding
(9, bill) entries to the transfer table. Thius, it's the case that
(9, bill) the UNIX "break occurs AFTER alll of the shared text libraries. The
(9, bill) libaraery images are entirely RO, data and BSS ARE BOUND AT LINK TIME (sorry, hit caps lock)
(9, bill) so that the librarys only contributed text to the process. All executables use
(9, bill) shared libs, even on the boot or "Tiny" floppy.
(9, bill) go ahead all.
(1, julian) what about multiple instances of the same executable?
(1, julian) can they make use of each other's work?
(1, julian) or must they relink each time they are run?
(9, bill) yse, they can,
(9, bill) the problem is to distinguish a relative load from a thread load,
(9, bill) and this is done by an address space convention on inhreited entries in thge transfer table
(9, bill) (which is a shared resource in the address space). A relative vs absolute
(9, bill) mapped enttrie is a distinct relplicat of the objec.
(1, julian) how do they know what another version has linked in?
(1, julian) what do you mean by a relative load?
(1, julian) so the transfer table can be inherrited by other versions of the same binary?
(1, julian) (actually other instances)
(14, terry) hello all.
(9, bill) The idea is that each thread can have is own copy (if needed) of an object.
(1, julian) hmmm each THREAD?
(9, bill) go ahead all.
(14, terry) I assume this means a multithreaded kernel?
(9, bill) THis arrangement is in advance of a multithreaded kernel,
(1, julian) when you say "each thread can have it's own copy of an object,"
(1, julian) what do you call an "object"
(9, bill) An object is a mapped regionm ogf address space that has an
(9, bill) identitiy to the kernel.
(1, julian) e.g. a mmapped region
(9, bill) yes
(1, julian) and different THREADS can see different stuff in the same region,
(1, julian) or can they see each other's objects too?
(7, Wolfgang Solfrank) But I think the main difference between threads and procs is the VA?
(9, bill) Threads share VA
(9, bill) processes don't.
(1, julian) but you said Threads have their own objects..
(14, terry) A thread is a process with a diffrent heap but the same everything else...
(1, julian) (actually, different STACK, same everything else)
(7, Wolfgang Solfrank) So threads see the same objects at the same addresses
(9, bill) (to wolf) yes
(1, julian) yes.
(9, bill) and one of my major objections to linux like libraries is the VA congetion.
(1, julian) they are similar to the sysVr3
(1, julian) (linux libs I mean)
(9, bill) (groan)
(9, bill) (to terry) you can either have seperate heaps (stack inside) or
(9, bill) seperate stacks with heaps,
(9, bill) take you pick,
(1, julian) we gotta get rid of these <coming and going> messages...
(1, julian) terry keep the line size down
(7, Wolfgang Solfrank) Bill, do I understand you correctly that you want to
(7, Wolfgang Solfrank) unshare more than the stack between threads?
(15, terry) Oh you have to unshare more than the stack ... signals!
(9, bill) lets not talk about threads yet,
(9, bill) thats anotyher 100 minutes
(1, julian) ok
(1, julian) let's get back to the shared olibs
(14, mdickson) Bill, when you mentioned VM objects are these the same thing as
(14, mdickson) Mach VM objects
(9, bill) to mdickson: no. more on that in a vm discussion.
(9, bill) back to shared libs. I promise well talk more about threads another time.
(1, julian) can you describe the
(1, julian) actions taken by the crt0 code?
(7, Wolfgang Solfrank) Bill, the solution you describe is very similar to the
(7, Wolfgang Solfrank) way the Sun Shared libs code does it.
(7, Wolfgang Solfrank) (as far as I understand it, anyway)
(9, bill) to julian: crt0 is the crux of the matter.
(9, bill) to wolf: exactly. A combination of both OSF and Sun
(9, bill) is exactly the beast.
(15, terry) I would think a combination would be the loader benefits
(15, terry) to avoid glutting the crt0.
(15, terry) But sun libs otherwise.
(9, bill) yes
(15, terry) OK ... so we are talking about Sun Libs with an OSF loader mod
(15, terry) to avoid bunging crt0.
(9, bill) You gentleman have more patience than me
(1, julian) about the loaders and the sun/osf mix
(1, julian) how do ypou see crt0 acting on start-up.
(15, terry) It should act as it does now -- kern_exec is the guy who gets into the
(15, terry) loader act.
(15, terry) The problem I have with Sun libs (the only problem) is the
(15, terry) reexportation of symbols.
(1, julian) terry, I think that the loader can afford to keep a list of loaded
(1, julian) modules for (say the kernel) and re-evaluate
(1, julian) the symbols when you want to add a new module to
(1, julian) (say the kernel) (or any other address space.
(1, julian) it can always work out what symbols are already ther if it knows
(1, julian) what modules are ther.
(9, bill) to terry: You know why Sun does that right?
(15, terry) bill: why is that?
(9, bill) to terry: so that subsequent libraries can have forward resolution
(9, bill) of symbols in a single pass frame.
(9, bill) to julian: it passes responsibility to /sbin/ld.so ore somesuch.
(1, julian) (which it mmaps in? )
(9, bill) to julian: yes, after it resolves them.
(7, wolfgang) I don't want cluttering the kernel with the stuff needed for shared libraries.
(9, bill) to wolf: nore do I.
(1, julian) so I assume that crt0 mmaps in the loader, and jumps into it,
(1, julian) then the loader (which was already in ram) loads
(15, terry) I think you are misunderstanding. The kernel wouldn't add symbols
(15, terry) from shared libs explicitly to the available
(15, terry) symols. But shared libs aren't the only kernel modules we ant to load.
(9, bill) kern_execve: only loads images into a process or thread, it does not do libraries
(7, wolfgang) I don't think that I would want to load shared libs into the kernel
(15, terry) bill: now I'm confused... what was the OSF loader business???
(7, wolfgang) Shared libs should be used only for application programs.
(15, terry) wolf: yes, but the dynamic load mechanisms should be generalized and shared.
(9, bill) lets talk about the kernel dynamic loading AFTER we talk out shared libs.
(9, bill) terry: Yes, I'm sill wanting to use the OSF loader,
(9, bill) because the kernel should not have to be educated about obscure formats,
(9, bill) but that's a seperate issue.
(1, julian) I think that for each VA we can keep a mapped page of the modules
(1, julian) that have been loaded into it and their offsets.
(1, julian) So the loader can always work out what symbols are present..
(1, julian) That goes for the kernel too, for loasdable modules)
(15, terry) A problem occurs when you have a loaded module that depends on
(15, terry) ,symbols in a previously loaded module.
(1, julian) terry: The OSF loader is mmapped in and runs at the beginning of
(1, julian) the process's startup
(1, julian) but, it is also used to load modules intop the kernel.
(1, julian) My point is that for this to work well,
(1, julian) each VA needs an index of loaded modules.
(7, wolfgang) Is the OSF loader really used to load modules to the (micro-)kernel?
(7, wolfgang) Or is it used only for the U*X server(s)?
(1, julian) The OSF/1 kernel is not a micro kernel,
(1, julian) so, no,
(1, julian) but I believe that it does work for both the MK and the Un*x server as well.
(14, Mike Dickson) to julian: Doesn't the osf loader keep a "package table" that is
(14, Mike Dickson) essentially the index your asking for?
(1, julian) I don't know.. I think it SHOULD for it to work..
(14, Mike Dickson) I think (from reading the paper) that the package
(14, Mike Dickson) table is made accessable much like arg lists and
(17, terry) A problem occurs when you load a module that depends on symbols
(17, terry) exported from a previously loaded module
(1, julian) Terry, I think the OSF loader gets around that problem.
(1, julian) it keeps a table of previous;ly loaded modules,
(1, julian) and where their symbol tables are.
(1, julian) so all symbols are available.
(17, terry) That would seem to be a bad thing for libs; how can I have two
(17, terry) versions of a curses lib, for instance (lines/coles/etc.)
(1, julian) the table is kept PER VA
(17, terry) ah... I see.
(9, bill) Its this blurring of responsibilities that I don't like with
(9, bill) the OSF approach; a loader is just for finding a way of interpreting the
(9, bill) contents of an excutable image to start it up, while a shared libarary
(9, bill) resolver is for filling
(9, bill) out a executable's references.
(1, julian) I guess that's what Bruce was talking about..
(1, julian) they are going to use the new dld for linking binaries,
(1, julian) and their loader for keeping track of modules.
(1, julian) bill, does that solve your bluring problem?
(9, bill) Almost,
(9, bill) the problem is one that will be encountered only in the threaded user
(1, julian) now let's go to another part of sharedlibs..
(1, julian) the actual linking between modules..
(1, julian) do we patch the calls or call everything via a writable PAGE with a bounce
(1, julian) table.?
(9, bill) Heres where
(9, bill) it gets interesting.
(9, bill) The "main" program, that has been statically bound is responsible for
(9, bill) using the shared library reslover to update it's segment of
(9, bill) the transfer tables. It has the sole responsibility for it's tthread
(9, bill) for resolving addresses, which are always through the table (we don't
(9, bill) want to generate non-shared replicas for each file if we can help it, as
(9, bill) how the sharing occurs is via the aegis of the VM system..
(1, julian) what about data references?
(9, bill) Data references are also bound with the main thread,
(9, bill) which has had all of the data references resloved at link time, so
(9, bill) only the text in the shared libraries is actually used
(1, julian) If you have a single table though there are problems..
(1, julian) each module needs to already have the code correctly bound so it can get to the
(1, julian) trnsfer table.
(1, julian) therefor a PIC reference needs to be alreeady in existance in the module.
(1, julian) the module doesn't know how many other
(1, julian) modules it will be loaded with
(1, julian) so the transfer table
(1, julian) must be 'adjacent to" the code segment
(1, julian) for that module
(1, julian) so that it can 'find it'
(1, julian) therefore each module must have it's own tranfer table.
(1, julian) as each module is added , it's transfer
(1, julian) table must be fixed up and all the other transfer tables must
(1, julian) be examined fore inclomplete references
(1, julian) that are resloved by this module.
(16, Holger Veit) where do common data (such as in the libxview.a) fit here?
(9, bill) common data is relative to the main thread.
(15, Peng-Toh) excuse me, what does "thread" mean here?
(17, terry) Thread of execution within a process context.
(7, wolfgang) Do you mean the code in the libs has to use stub routines to access the data?
(9, bill) No
(17, terry) Julian: I believe the threads implementation is based on the standard
(17, terry) mechanism whereby heap is shared and text is
(17, terry) shared but stack is not.
(9, bill) yes
(14, Mike Dickson) Yes terry, that's how it's done in Mach.
(17, terry) When I modified the mutex mechanism in MTS on VMS, this is the way it was done.
(1, julian) therefore each module must have:
(1, julian) text:data:bss:table:
(1, julian)
(9, bill) no
(15, Peng-Toh) Does that mean the program is also compile for PIC?
(9, bill) everything is PIC
(9, bill) the format is
(9, bill) text:data:bss:libraries:break,
(9, bill) and potentially many of theese can be present in a VA,
(9, bill) with a shared transfer table at a "known" address
(17, terry) Thus the libraries are not allowed to change size once mapped.
(9, bill) correct.
(17, terry) this solves the table problem, julian.
(1, julian) I dissagree!
(17, terry) The known address is either an offset or a table link in text, is it not?
(9, bill) yes
(9, bill) also,
(9, bill) shared libraries could be referenced in other threads.
(17, terry) (offset for a fixup table, link for a static PIC reference not linked with
(17, terry) the program initially).
(9, bill) yes
(1, julian) I have a problem with what i THINK you are saying....
(17, terry) I assume we are going for a copy-on-read table to avoid linking a static
(17, terry) fixup table into an image?
(1, julian) if we have the libraries making references,
(1, julian) then they need to be able to find the entries in the table too
(1, julian) and they cannot POSSIBLY (at compile tim)
(1, julian) know the offset into the table for the particular refernce they are looking for,
(1, julian) unless the table tey are looking for come s witrh them
(1, julian)
(1, julian) THEREFORE
(1, julian) each library/module needs it's OWN TABLE
(1, julian)
(11, Tuomas Lukka) But does all library code also have to be PIC?
(1, julian) yes
(17, terry) Tuomas: only if it wants to be shared.
(11, Tuomas Lukka) Yes, but still that's rather inefficient, is it not?
(17, terry) Static libraries are still allowable.
(11, Tuomas Lukka) I was of course talking about shared libs.
(9, bill) The transfer table is a shared resource.
(17, terry) There would be a "libraries used" segment in the a.out used by the
(17, terry) loader to find things ... am I on track, bill?
(9, bill) yup
(17, terry) yes.
(17, terry) But only ONE table -- not a table linked initially with each image.
(9, bill) but, this is not in the library image, except for a prototype
(1, julian) are you agreeing with me, or each other?
(17, terry) "yes, each library/module needs it's OWN TABLE"
(1, julian) ok, and I suggest that even if it's not in the image
(1, julian) there needs to be a segment in the image that DEFINES it.
(1, julian)
(9, bill) The idea is that you don't want to have R/W libraries
(17, terry) The advantage is that fixed up tables can be shared between identical images.
(9, bill) yes yes yes
(9, bill) Thats why we
(9, bill) only have RO images from the libraries
(15, Peng-Toh) how do you tell if the images are identical???
(17, terry) open vnode reference for loaded image... it's unique.
(18, Charles Hannum) same file handle
(1, julian) so PIC in effect meas that the C compiler needs to make indirect
(1, julian) references via the table for all
(1, julian) EXTERN references?
(9, bill) julian: yes, but for libaries
(15, Peng-Toh) suppose difference LD_LIBRARY_PATH?
(15, Peng-Toh) (or equivalent)
[ note that this needs to be resolved -ed.]
(17, terry) Peng-Toh: different vnode reference (different file)
(1, julian) Then the loader notes the difference and creates new copies of thebpages.
(18, Charles Hannum) Then you have a different library. What difference does that make?
(17, terry) The upshot being that the pages fixed up in PIC are locked in core.
(1, julian) to terry: no not needed
(1, julian) you can always fault them in..
(1, julian) they of course would be heavily used though
(7, wolfgang) I assume the loader keeps the list of already fixup'ed libs/modules
(7, wolfgang) in some known directory?
(9, bill) no that's in the address space.
(11, Tuomas Lukka) But having PIC + indiret references everywhere will be even
(11, Tuomas Lukka) less efficient... Would it not be possible for the
(11, Tuomas Lukka) loader to patch the library's references?
(17, terry) julian: wouldn't that fail for a library in excess of 64k?
(17, terry) (the implication of your statement being that the tables themselves are PIC).
(15, Peng-Toh) PIC code on 386 does not have size limit... no 64k limit
(9, bill) The idea here is that the main
(9, bill) program can have everything tied down to a virtual segment,
(9, bill) but that libraries have to reference relative to the main segment that
(9, bill) reloves them
(1, julian) but the tables must be REACHED by PC-relative addressing!
(1, julian) Bill: I disagree.
(1, julian) (strongly)
(9, bill) Gee, a dissagreement?
(14, Mike Dickson) I agree with thomas, do we need a table?
(15, Peng-Toh) terry: ok, does that means you cache all the shared libs vnode
(15, Peng-Toh) for an executable so when the same executable is run
(15, Peng-Toh) with the same libs, you use the already fixed up pages?
(1, julian) basically yes, but since it's done outside the kernel it's more
(1, julian) that the loader knows it's the same file, than the actual vnode number.
(17, terry) Mike: even if we unPIC or fixup in code rather than a table,
(17, terry) that pretty much defeats shared text.
(11, Tuomas Lukka) terry: why?
(17, terry) Peng-Toh: You have to cache them anyway.
(11, Tuomas Lukka) terry: UnPIC:ing the code would have very little effect on sharing:
(11, Tuomas Lukka) all we need is to have the same libs at the same address IN EACH
(11, Tuomas Lukka) PROCESS'S PRIVATE address space.
(14, Mike Dickson) Can we resolve transfer tables vs. fixups via
(14, Mike Dickson) patching the code?
(1, julian) patching the code is a problem
(1, julian) because every page would have external references that need to be patched
(1, julian) so you would end up with no shared pages between unlike programs.
(1, julian) and that's what we are trying to achieve.
(1, julian) (SHARED<<_---- LIBS)
(9, bill) yes
(14, Mike Dickson) I'm not sure I understand why?
(11, Tuomas Lukka) julian: please elaborate!
(9, bill) exactly
(11, Tuomas Lukka) Why?
(9, bill) because they are not at the same VA
(7, wolfgang) Bill, I still don't understand where the fixup pages are maintained.
(11, Tuomas Lukka) Look: the loader would know each library's address in the "prototype"
(11, Tuomas Lukka) address space.
(11, Tuomas Lukka) Each process would only see the libraries from there it wants to.
(17, terry) Tuomas: Assume a jump table with fixup. This is the same as 35 machine
(17, terry) cycles per library reference. For a small
(17, terry) average reference size, PIC is cheaper. (35 for GCC on SPARC,
(17, terry) 85 for CC on sparc)
(11, Tuomas Lukka) Yes, but how about library code, I mean, in a cpu-intensive
library with many address references it would cost.
(9, bill) wolf: the fixup pages or transfer table are a shared resource among threads.
(7, wolfgang) I didn't mean why; where does a newly loaded pgm get the already
fixed'up pages?
(17, terry) Everybody but Tuomas: It's possible to have a jump table and not trash
all pages; thus the fixups are all in a single contiguous block.
(17, terry) This is what AZTEC C on a MAC does to do it large model scatter loading.
(9, bill) The difference between the main program and the libraries
(9, bill) is that the main program has statically bound
(9, bill) "relative" virtual addresses for everything
(9, bill) except shared library text references.
(7, wolfgang) But threads share their VA anyway, so they of course share the fixup pages.
(1, julian) Tuomas: it sounds to me like you are advocating fixed addresses for
the libraries (as in LINUX)
(11, Tuomas Lukka) No! Not at all. The libraries would have fixed addresses
ONCE HAVING BEEN LOADED. At load time, they would be patched.
(17, terry) Tuomas: bogus -- you limit you code size, library size, or both.
(1, julian) eh?
(11, Tuomas Lukka) terry: how? Code starts at bottom, libraries loading at top.
The limits are very little touched by this...
(11, Tuomas Lukka) The first library loaded into memory would be memtop-size, etc.
(17, terry) julian: bill said "a fixed location PIC vector table so that very
little gets patched" ... almost.
(9, bill) The transfer table is located above the stack at user top.
(1, julian) but a library cannot possibly know WHERE in that table the
references it wants ARE>>>>>
so each lib must have it's OWN TABLE because otherwise it
doesn't know the format of the table
(7, wolfgang) Tuomas: but your libraries wouldn't be SHARED.
(11, Tuomas Lukka) wolfgang: YES, they would! The pages would just be
mapped into the same high address IN EACH PROCESS.
(9, bill) yes
(17, terry) Tuomas: Unless you want multiple loaded/resolved shared lib code at
multiple fixup addresses, you must allocat a
portion of the address space per library (potentially limiting it's size).
(18, Charles Hannum) wolfgang: I believe he's saying only that they would be at
the same address in each address space.
(1, julian) NO
(11, Tuomas Lukka) terry: But the allocation would be run-time i.e. when
the library is first referenced by a proc.
(7, wolfgang) Bill, do you mean to reserve some amount of VA (hopefully large enough)
beyond the top of stack?
(18, Charles Hannum) wolfgang: AIX 3 does something like this; there is a
special 256MB "segment" which contains all the shared
libraries currently loaded. They are fixed up at load time.
(9, bill) Exactly
(17, terry) Tuomas: In addition, unless each lib were mapped before the break
(limiting code size) you would limit the size that
the break could grow to.
(1, julian) tuomas:only for copies of the same binary
(17, terry) tuomas,julian: yes, eactly.
(11, Tuomas Lukka) julian: why? The loader can mmap the library at the same
address for other binaries, can it not?
(1, julian) you may already havae some OTHER lib loaded at that address.
(11, Tuomas Lukka) terry: The break? Mapping at 0xFE000000-libsize would
not be such a hard limit...
(17, terry) tuomas: say I have "hello world" and the loader maps the libc at 0x2000
(17, terry) tuomas: then I load the X server.
(11, Tuomas Lukka) julian: No, the LOADER decides where to map them.
(18, Charles Hannum) terry: It wouldn't have to be below the break.
(11, Tuomas Lukka) terry: The TOP of the address space.
(17, terry) The top of the address space is ... special -- for a lot of reasons.
(11, Tuomas Lukka) You load hello world -> libc at 0xFDFF0000
(11, Tuomas Lukka) terry: just below the top that the kernel wants... there's an
address known WHEN THE LOADER IS FIRST RUN above
which we should not touch.
(11, Tuomas Lukka) Also, this way, the bookkeeping would be minimized...
(9, bill) The only "fixed" VA assignment need be the shared transfer table.
(17, terry) I see what you are getting at -- the size of the libs would not
be limited because the load would be dynamic,
(11, Tuomas Lukka) terry: YES!!!!
[at this point three people suddenly understand and all ask, the same question 8-) ]
(1, julian) in that scheme however EVERY PROCESS
(1, julian) loses the amount of VA of the SUM of the size of ALL the libs loaded.
(18, Charles Hannum) Tuomas: You need to think carefully about garbage-collecting
unused libraries.
(17, terry) but then what do you do about garbage collection? Seems a pretty
big obstacle to fixed addr loading.
(9, bill) But why I don't want to "patch" libaries is that I
(9, bill) want to cache them regardless of the VA they are loaded at.
(11, Tuomas Lukka) Yes, I can see the problem with that... but with 32 bit
addresses, there's plenty of addres space to spare, and
you could keep reference counts to libraries and
unmap them after done.
(16, Holger Veit) you want to do garbage collecting in the running process, or
after death of it?
(17, terry) julian is right -- steals memory out of everyones addrs for libs
they aren't using.
(11, Tuomas Lukka) terry: I know, but 32 bits.... is a lot.
(14, Mike Dickson) Hmm, ok, now I'm starting to like the transfer table...
(18, Charles Hannum) Tuomas: But how to you reuse regions? A simple algorithm
will leave holes.
(11, Tuomas Lukka) charles: Yes, but do they matter so much? And of course, you
can repatch the libraries at any time.
(18, Charles Hannum) Tuomas: You can repatch libraries, but you can't split text
segments. And the holes will accumulate.
(9, bill) One of the problems with threads is VA space congestion.
(1, julian) good point: threqads often have hundreds of
Va modules SPARSELY scattered
(17, terry) julian: nonapplicable to user space thread context switch.
You aren't talking threads -- you are talking LWP's.
(16, Holger Veit) i see.
(17, terry) Tuomas: scenerio:
load lib foo 1.0 -- fee 1.0 fum 1.0 --
unload lib foo 1.0 --
load lib fie (real small) 1.0 --
reload lib foo 1.0;
HUGE ADDR SPACE ~DECREASE.
(7, wolfgang) Bill, but the Sun scheme does share all the lib except the per lib
transfer table.
(9, bill) to wolf: Yes, and we also need
to allow for potential locally bound instance s of the same libary
per thread (e.g. locallyt unshared for processor clusters).
(18, Charles Hannum) Tuomas: You can arrange to be able to remap an entire
library at any time, but that's a bit hairy, and
potentially slow.
(11, Tuomas Lukka) Ok, but still, I dislike PIC with the current processors, as
they are planned for something else... though the
other points are rather heavy.
(11, Tuomas Lukka) charles: I know it's heavy but it would be done rarely.
(1, julian) There is that.. libraries are rarely replaced on running machines.
(14, Mike Dickson) Bill, are you saying each thread could have it's
(14, Mike Dickson) own libraries loaded?
(9, bill) to mike: yes, for replication on a clusterr procesor.
(7, wolfgang) Bill, I think you have to explain your threads concept a bit more for me
(7, wolfgang) to understand this (maybe in private email exchange?)
(7, wolfgang) Or on another occasion
(9, bill) to wolf: somtetime soon.
(11, Tuomas Lukka) bill: cc's?
(9, bill) ??
(18, Charles Hannum) Actually...
(18, Charles Hannum) Tuomas: I just got an inkling of an idea, but I need to think
about it more.
(18, Charles Hannum) lunch time
(11, Tuomas Lukka) charles: Free speech, please tell us.
(11, Tuomas Lukka) bill: carbon copies... (of emails you send to wolf about threads)
(9, bill) I must be going soon. Any thing I can help with first?
(1, julian) just do it on the mailing list
(1, julian) (and everybody remember to reply to it as well.
(1, julian) 386bsd_kernel and 386bsd_sharedlibs @ref.tfs.com..
(17, terry) wolfgang: See "Solaris SunOS 5.0 Multithreading and Real-Tile
(white paper) SunSoft, INc; 2550 Garcia Avenue;
Mountain View, CA 94023; 1-800-227-9227.
(9, bill) (gack. Solaris 2.0 almost did it right...
(9, bill) but what a monster...)
(17, terry) bill: loadable kernel modules, kernel configuration management,
threads: talk about it when?
(1, julian) next Friday?
(9, bill) soon. I'm already booked solid now.
(9, bill) I'll check back after tuesday
(17, terry) bill: streams?
(9, bill) not till 0.3
(17, terry) bill: I have a working getmsg/getpmsg/putmsg/putpmsg.
(17, terry) bill: and some of the kernel stuff for queueing/allocb.
(9, bill) terry: good. The hard part's the goddamn tty driver.
(10, Chet) terry and wolfgang: that paper (and many others) are available for ftp
(10, Chet) from opcom.sun.ca in /opcom/pub/docs
(10, Chet) opcom.sun.ca (aka suntalk.sun.ca) 192.75.19.1
(2, Alistair Crooks) I recall Charles Forsyth saying the main bloat in Sun kernels
(2, Alistair Crooks) was dure to streams
(2, Alistair Crooks) but all those buffers...
(2, Alistair Crooks) and the multiplexed abortions
(9, bill) Streams is a performmance hog due to it's base mechanism
(1, julian) I guess this is winding down..
I will mail the minutes (after considerable editing)
(1, julian) to all people omn te two mailing lists
(1, julian) anyone not on those?
(9, bill) Sounds great Terry, I'd like to hear more later.
Any more shared libs before I go?
(17, terry) streams is useful for telnet sessions without ptys/active telnetd's.
(1, julian) no, I think I will re-read the OSF stuff again
(1, julian) and look at the sun stuff, (and the sun white paper)
(1, julian) then I will edit the minutes up
(1, julian) and send them out..
(9, bill) yes please do
(1, julian) I think we will be surprised about what we've covered..
(0, stripes) Julian, where can I get the OSF paper (I am assuming it is a shar lib paper)?
(17, terry) ~boot:OSF on ref.tfs.com for the OSF paper.
(11, Tuomas Lukka) julian: are you also placing all this on ftp on ref also?
I think that would be nice...
(11, Tuomas Lukka) (in case some new people decide to join)
(1, julian) Bill can we expect to see you here again?
(9, bill) definitely
(9, bill) It's a little difficult
(9, bill) to keep track of, but if I can help
(9, bill) you , I'll do it.
(8, Bruce Bauman) I'll try and put some more OSF doc online, too.
(1, julian) I will attempt to make improvements to the format
(1, julian) (e.g. get rid of trhe coming and going messages)
(17, terry) moderated? 8-).
(1, julian) and maybe work out some way we can
(1, julian) serialise the discussion..
(1, julian) maybe with a 'chairman' or a VIRTUAL TOKEN
(1, julian) any ideas?
(9, bill) I'll see if I can contribute around
(9, bill) an hour to 3 a week for this,
(9, bill) how is it for our international group timewise?
(2, Alistair Crooks) can I say thanks for a very liveley & interesting time?
(1, julian) anybody have better time suggestions..
(11, Tuomas Lukka) What time GMT is it now?
(1, julian) I know bill prefers this time..
(2, Alistair Crooks) so suits me this time or thereabouts
(17, terry) I vote for bill's preference.
(12, Holger Veit) Germany, here its 7:23pm
(12, Holger Veit) Germany is GMT+1
(1, julian) only bad in japan and Australia really
(11, Tuomas Lukka) Finland,8:30 pm... very much ok!
(17, terry) Whatever we discuss will be meaningless if there isn't coordination with bill.
(9, bill) Actually,
(9, bill) you guys seem to get along quite well anyways,
(1, julian) ok, so This time on another date again!
(1, julian) bill, any ideas of your timetable?
(9, bill) I'm quite proud of what you have been able
(9, bill) to do with little input.
(1, julian) (we can also meet at other times without you of course)
(9, bill) sure
(11, Tuomas Lukka) bill: 8talking about input, how about making a file about your
plans on this and mailing it to the mailing list?
(11, Tuomas Lukka) In here, only a few issues could be touched...
(1, julian) we'll hack it to bits and mail it back 8-)
(9, bill) I don't exactly follow you
(11, Tuomas Lukka) I mean, talking in here is good but to get a good, clear general
picture, it would be nice to have a slightly .
(9, bill) You want the big picture
(1, julian) yes please
(11, Tuomas Lukka) Or a bigger picture on shlibs.
(9, bill) That I can do.
(1, julian) I expect the mailing list will be lively on the topic and easier to read ! 8-)
---- No world ----
|