How to modify an array in function?

You have three options here, but maybe you don't need any of them, since Matlab used 'copy-on-write', i.e. variables are only copied if you modify them.

1. As @gnovice mentions, you can use a nested function. Variables used inside the nested function are shared between the nested function and the enclosing function. Nested functions are somewhat tricky to debug, and a bit more complicated to write/understand.
2. You can store your images as properties of a handle object, which is passed by reference.
3. You can write code differently in order to not use a recursive function, since Matlab isn't the best language for using those. If you have access to the image processing toolbox, you may be able to use functions like blockproc, or im2col to rewrite the function.

Finally, if you want to stay with your current scheme, I strongly suggest using persistent variables instead of globals.

MATLAB is not always pass-by-value, newer versions of MATLAB do pass-by-reference under some circumstances, see in-place operations and a more general discussion about MATLAB memory management in this SO post.

Without tail-call optimization it is inefficient to use recursion and MATLAB does not have it as far I know, but every recursion can be transformed into a loop.

If you make your recursive function a nested function within another function where the image data is stored, then the recursive function can modify the image data without needing to have it passed to it.