A couple of times, I have had issues in Ubuntu Server with apt-get not removing all files.
The usual syntax is:-
sudo apt-get remove packagename
I had problems with phpmyadmin, and could not get it to reinstall properly. To force apt-get to remove all files including configuration files use:-
sudo apt-get --purge remove packagename
This should do the trick.
