Only control I have over time is the max script execution time in php options which is currently set to 300s.
A manual refresh may end but often it takes so long I never see it finish as I go to do something else.
Do you know if there are any particular files that may slow things down? zip files etc?
No, it doesn't matter if there are zipped files because the "refresh task" only take basic info about the sites. In the nex version of Control Center I will include a log to check if something goes wrong to solve cases like this.