If you're trying to create a single jar that contains your application and its required libraries, there are two ways (that I know of) to do that. The first is One-Jar, which uses a special classloader to allow the nesting of jars. The second is UberJar, (or Shade), which explodes the included libraries and puts all the classes in the top-level jar.
I should also mention that UberJar and Shade are plugins for Maven1 and Maven2 respectively. As mentioned below, you can also use the assembly plugin (which in reality is much more powerful, but much harder to properly configure).
You do NOT want to use those "explode JAR contents" solutions. They definitely make it harder to see stuff (since everything is exploded at the same level). Furthermore, there could be naming conflicts (should not happen if people use proper packages, but you cannot always control this).
The feature that you want is one of the top 25 Sun RFEs: RFE 4648386, which Sun, in their infinite wisdom, has designated as being of low priority. We can only hope that Sun wakes up...
In the meanwhile, the best solution that I have come across (which I wish that Sun would copy in the JDK) is to use the custom class loader JarClassLoader.
I was about to advise to extract all the files at the same level, then to make a jar out of the result, since the package system should keep them neatly separated. That would be the manual way, I suppose the tools indicated by Steve will do that nicely.