How to clean up temporary file used with send_file?

I'm currently developing a server side json interface where several temporary files are manipulating during requests.

My current solution for cleaning up these files at the end of the request looks like this:

@app.route("/method",methods=['POST'])
def api_entry():
    with ObjectThatCreatesTemporaryFiles() as object:
        object.createTemporaryFiles()
        return "blabalbal"

In this case, the cleanup takes lace in object.__exit__()

However in a few cases I need to return a temporary files to the client, in which case the code looks like this:

@app.route("/method",methods=['POST'])
def api_entry():
    with ObjectThatCreatesTemporaryFiles() as object:
        object.createTemporaryFiles()
        return send_file(object.somePath)

This currently does not work, because when I the cleanup takes place flask is in the process of reading the file and sending it to the client. ¨ How can I solve this?

Edit: I Forgot to mention that the files are located in temporary directories.