return rows in a dataframe closest to a user-defined number

I have a user defined number which I want to compare to a certain column of a dataframe.

I would like to return the rows of a dataframe which contain (in a certain column of df, say, df.num) the 5 closest numbers to the given number x.

Any suggestions for the best way to do this without loops would be greatly appreciated.