Resolving RVA's for Import and Export tables within a PE file

I am currently writing a PE parser/loader. I have successfully loaded the PE file into memory using standard c file io, retrieved valid DOS and PE headers (the optional header) as well as gaining access to the PE's sections. My next target is to gain access to the Export table to retrieve exported symbols. To do this i have used the RVA stored in the optional headers data-dictionary array at index 0 (which i believe points to the export table) and added this address to the address of the PE file loaded into program memory, then casted this into a valid export table header. I am turning up NULL addresses and data when i do this. here is a small code snippet;

// RVA from optional headers data dictionaries array cast to Export directory type 
  IMAGE_EXPORT_DIRECTORY* ied(
  (IMAGE_EXPORT_DIRECTORY*)((void*)
  ((unsigned char*)buffer + ioh->DataDirectory[0].VirtualAddress)));

Do i have to use memory mapped IO to do this properly? Am i calculating the address wrong? Information on PE RVA's seems sparse. thanks in advance.