To remove it correctly you must add Disk Cleanup utility with this powershell command:
Add-WindowsFeature Desktop-Experience
It will appear Clean up system Files button, after that you will be prompted to remove windows.old or precedents windows installations.
[more articles :
How to install Clean up utility on windows 2012
http://social.technet.microsoft.com/wiki/contents/articles/15221.enabling-disk-cleanup-utility-in-windows-server-2012.aspx
Windows.old problem
http://blogs.technet.com/b/chad/archive/2012/10/08/tip-51-cleanup-on-isle-3-get-back-disk-cleanup-wizard-on-windows-server-2008-amp-2012.aspx]