We came across a weird problem on ESX5i. Occasionally, one of our hosts would suddenly be unable to start any VMs – the running VMs were fine, but any attempt to start new ones would fail with an “Unknown internal error”. The first time this happened, I restarted the management agents and finally suspended all VMs and rebooted the server, after which everything was OK again for a few months and then the problem occurred again. This time I decided to figure out what was going on. The log files in /var/log contain a lot of useful information and I was able to see that the problem was actually caused by ESX being out of disk space on the device[…]