Take a look at Task Manager on the machine where your TFS Repository lives. We have this problem whenever its memory utilization is greater than 95%.
http://stackoverflow.com/questions/20632215/tfs-build-controller-service-http-code-500-system-servicemodel-serviceactivatio