|
|
On Mar 27, 2:41 pm, CBFalconer <cbfalco...@xxxxxxxxx> wrote:
> Paul Hsieh wrote:
> > CBFalconer <cbfalco...@xxxxxxxxx> wrote:
> >> Paul Hsieh wrote:
> >> ... snip ...
> >>> The one feature that there simply is NO controversy about at all:
> >>> Memory management. I don't think programmers could yell any louder
> >>> for better support for any (practical) other feature. (Obviously
> >>> the other one is bounds checked arrays, however, that runs into
> >>> practicality problems.) Show me a document anywhere that
> >>> demonstrates that the ISO C language committee gave a flying fig
> >>> about memory management; that they were working towards some sort
> >>> of library extensions or API enhancements to deal with the leaking,
> >>> corruption and examination of memory.
>
> >> You obviously don't realize that the fundamental C technique of
> >> 'taking addresses' and 'forming pointers' to things in various
> >> types of storage makes such memory checking totally impossible.
>
> > What the hell is your problem? You know very well that I am a supreme
> > expert about such things.
>
> >> Unless you are willing to give up all efficiency by making all
> >> pointers indirect, and keeping a master list, and modifying that on
> >> every *p++ coding.
>
> > When you are in a corrupted state, indeed there is nothing you can do
> > that is *guaranteed*. But there is a lot you can do with 99.99%
> > certainty. This is the whole premise behind the very concept of a
> > debugger, for example.
>
> > My point is that it is very typical that debuggers which are too
> > general, have very little insight into the state and structure of
> > memory. Yet, with some work, its easily possible to make a very
> > useful self-checking memory manager that takes very little performance
> > and very little extra space overhead. Having written a heap manager
> > yourself, you must be very well aware of this -- it costs basically
> > nothing to have a very rich debugging environment for memory.
>
> > For example, its trivial to track the base/system heap memory regions
> > that your manager allocates from. So writing a function like:
>
> > int isInAllocatedMemoryRange (void * ptr, int size);
>
> > is both useful and easy to do, and *you* of all people ought to know
> > that you can do this. Of course its not a guarantee that the pointer
> > is in the right format, or aligned properly, or pointing at live
> > allocated memory. So we can try to add the following:
>
> I have ignored the remainder. Getting too complex for me.
Ok, this I don't get. Did you not write your own heap manager
yourself?!?! I was pretty sure you did, and used it for the DOS port
of gcc or something.
Look, you fetch big blocks from system memory in some way, whether
through sbrk() or VirtualAlloc() or some DPMI memory allocation or
whatever. You can track these big blocks in a list. The
isInAllocatedMemoryRange function would just look through this list to
see if the pointer was pointing to the inside of one of these blocks.
What is the complication here?!?! There's no issues -- I *did* this
in order to revive some old buggy code while doing major invasive
refactoring; this level of debugging was critical in making the effort
practical.
> However, my point is that there is no way (apart from comprehensive
> runtime tables) to tell that a pointer is to malloced memory,
> static memory, or automatic memory.
What? If you are a compiler implementer (remember, someone was
challenging me on compiler extensions, so this is our premise) then in
fact, exactly those three memory regions represent exactly what you
*DO* know. In the effort that I referred to above, what I did is I
had the program read back in its own MAP file to figure what its
static ranges were and tracked the bottom of the stack from main (and
top from ESP in an _asm fragment.) This allowed me to implement a
memClassify function as well. Of course, its possible to get pointers
from other sources, so this does not represent a comprehensive memory
classifier.
But the compiler can do way better. At link time the memory map can
be determined, so hacks like reading back its own map file are
unnecessary when you can just append a small data region with the
start and length information for the static memory (and init/const
memory, if there is any.) For the stack, the beginning can be marked
before the call to main, and a macro be used for determining the
current stack top.
> [...] There is also no way to tell
> that that pointer is the the whole chunk, or to a miniscule chunk
> of that. This makes memory checking virtually impossible, if you
> want to preserve efficiency.
Huh? Look, you want the block with its header to be aligned. But you
also want the base pointer to be aligned. That means you have *LOTS*
of header space. If you do an approximation of (uint32_t)
SHA2({ptr,size}) (you can go waaay simpler, I am just giving an
overkill example to illustrate the point) and store it in the header
somewhere, then that's it; the memory allocation is marked in a pseudo-
unique fashion. You aren't going to run into flukey internal or
static pointers that randomly reproduce this signature.
> Sure, I can do some things with a subset of those pointers, which I
> have allocated and manipulated since birth. But those operations
> won't function on another system.
Huh? I am talking about how one implements this in the compiler
library for each specific system. Obviously there's no way the
implementations would be portable; but this is unnecessary.
> [...] They will be totally confused if
> ever passed non-malloced pointers, or even malloced pointers using
> another malloc package.
Huh? If it doesn't come from the malloc that corresponds to these
functions, then it will fail the isInAllocatedMemoryRange () function.
> I just don't like any software that 'works sometimes'.
Its a debugging tool, and "sometimes" is 99.99% of the time.
> [...] If its
> purpose is to tell me that memory is being misused, I want a
> definite answer. Barring that, I prefer no answer.
An automated system that is correct 99.99% of the time, is a higher
standard than I have ever seen in any human when tracking down a
memory corruption bug in a non-trivial amount of code.
--
Paul Hsieh
http://www.pobox.com/~qed/
bstring.sf.net/">http://bstring.sf.net/
|
|