I have some given code that uses variable length arrays on the stack in C. I cannot easily change that code to use malloced buffers on the heap, since I am working on some stuff where I have no operating system support for dynamic memory. However when I test my method the stack actually get's smashed instead (i.e. the SP get's set to a completely bogus address). To figure out what is going on I had a look at the assembly and I was completely confused by the compiler output.
I am running the code on a pandaboard ES (OMAP4460 ARM processor).
In order to test how variable size arrays on the stack are actually compiled, I produced a simple working test program. However again the code was much too confusing for me to understand.
Here is what I do not understand:
When I compile a simple function:
int test_method(size_t i)
{
unsigned char buffer[10];
return -1;
}
I get some very simple assembler code:
80003a68 <test_method>:
80003a68: b480 push {r7} ; store frame pointer
80003a6a: b087 sub sp, #28 ; make stack frame (size 28)
80003a6c: af00 add r7, sp, #0 ; set frame pointer
80003a6e: 6078 str r0, [r7, #4]; store parameter on stack
80003a70: f04f 0300 mov.w r3, #0 ; load return value
80003a74: 4618 mov r0, r3 ; put return value in return register
80003a76: f107 071c add.w r7, r7, #28 ; destroy stack frame
80003a7a: 46bd mov sp, r7 ; ...
80003a7c: bc80 pop {r7} ; pop stored frame pointer
80003a7e: 4770 bx lr ; return
But when I try this with a variable size array using the following code:
int test_method(size_t i)
{
unsigned char buffer[i];
return 0;
}
I get t