I forked a project on Github.

Let the remote upstream be upstream and my remote repository be origin. My local master branch is set to track the remote master branch. Then I added some stuff in local master, and I merged with the upstream every now and then.

Not until today when I want to issue a pull request did I find the problem: the pull request consists those merge commits, and those unwanted commits that I did previously without care. However what I want is just to submit the last commit I did, which should be pulled as a single commit. What can I do to rescue this?

Edit
Report