[转]Reducing script compile time or a better workflow to reduce excessive recompiling

http://forum.unity3d.com/threads/148078-Reducing-script-compile-time-or-a-better-workflow-to-reduce-excessive-recompiling

SOLUTIONS: Each of these solutions reduce compile time. Use one or all of them together to get the biggest speed increase.

#1 - Convert project to C# (extremely effective)
I was using UnityScript. Convert your project to C# -- it compiles 4X FASTER than UnityScript! Compile times went from 15s to 3.5s. See this post for some numbers.

#2 - Convert some/all scripts to DLLs (very effective but complex)
NOTE: This solution involves a complicated setup using externally compiled DLLs. While it works, it's probably far too complicated for most users. Compile times reduced from 15s to 2-3 seconds or less. See my answer here.

#3 - Convert ALL remaining .js files to C# (somewhat effective)
I was able to reduce compile times an additional 3.25 seconds by converting ALL lingering .js files in my project to C#, including Standard Assets. See this post for details.

Overall, using all 3 methods above, my script compile times are now down from 15 seconds every time I update a script to just 2 seconds. That's a 7.5X performance increase! See my happy face over there? 

I'd like to report that two weeks later, my problem is solved! I've reduced my compile time from 15 seconds to 2-3 seconds. Thanks for your help everyone in getting this figured out.

In a nutshell, I moved every script I possibly could to external DLLs, though this was no easy task, especially because I'm using UnityScript. Moving to DLLs is pretty easy for basic utility classes, but most of the speedup comes from moving MonoBehaviours to DLLs which is much more difficult considering this was not planned for at the beginning of the project.

Because of how I structured things, I can keep the scripts I'm currently working on as scripts in Unity and everything I'm mostly done with as DLLs. Once I'm done with them, I can shift them over into the DLL side. (Its just a little bit easier to work on things that need frequent changes as scripts rather than DLLs, but it's also workable just to work on everything as DLLs, though I doubt using Unity's debugger would work from precompiled DLLs).

I'll outline my steps for anyone interested. Note that this requires Unity Pro unless you can figure out how to edit the binary scene and prefab files. Also, I assume you are comfortable writing your own small tool programs outside of Unity to help with certain small tasks like parsing text files.

----------------------------------------------------------

STEP 1: *** BACK UP YOUR PROJECT! ***

///////////////////////////////////
STEP 2: PREPARING YOUR DLL SOLUTION
///////////////////////////////////

Create a new solution in MonoDevelop. Inside this solution, create one project for each category of your scripts. I organized mine into a 30 projects by type arranged just as they had been as scripts. (The more code you have in one category, the longer it will take to compile.) Copy all the scripts you want to convert to DLLs into their respective project folders.

Common libraries:
Add references to one or both UnityEngine.dll and UnityEditor.dll in all your projects that need them. (In Windows, find them in UnityEditorDataManaged.) Also add System from the Packages tab.

Utilities:
I had to do a little re-organizing to make sure I had a sensible dependency chain set up. My utility classes, enums, and some hardcoded data tables would need to be referenced by most/all other projects, so I organized accordingly. Add references to these projects in all your other projects that need them.

MonoBehaviours:
I wanted to be able to keep my existing code entirely as is. Because much of the code in my MonoBehaviour classes relies on other MonoBehaviour-extended types (being used in other scripts), there are a lot of cross dependencies. I didn't want to have to wait forever for one gigantic project to compile every time I made a change (how Unity does it), so I chose to split up all MonoBehaviours into two classes each -- one base class and one class extended from the base class. I then added references the BaseClasses project to every other project that needed access to those classes (all the MonoBehaviour-containing projects). The base classes can be passed around and accessed by any MonoBehaviour that needs them.

About Base Classes:
Because of the size of my project, splitting up all the MonoBehaviours into base and extended classes required that I write a little tool to help me with the task. Essentially I went through all the script files and extracted all class properties/fields and all public methods and generated a large file containing all the base classes. For properties, I copied the property declaration and assignment (if any) as is so as to . For the methods I just created stubs (blank methods) out of them. (When one of these stub methods is called through the base class, the extended class's override method will be called instead.) All the game code will be in the extended classes. The base classes just provide a base framework.

I used the original class name for my base classes because I didn't want to change any references in code. After I got my base classes file made, I modified each script file:
Comment out the properies because these need to be only set in the base.
Rename the class (and their .js/.cs files) with a suffix ("_Ext" in my case) to differentiate it from the base class. (You will never have to call these extended classes in code, just the base with the name you're used to.)
Make the class extend the new base class.

MonoDevelop configuration for C# users:
Make sure your projects are set to output to .NET 2.0 or 3.5 or you MIGHT get some errors when using the dlls in Unity. Also make sure you enable optimization. (VStudio: project properties -> Build -> optimize code, MonoDev: project properties -> Compiler -> Enable Optimizations). If you do not enable this, your code will run slower.


MonoDevelop configuration for UnityScript users:
Getting DLLs to compile from UnityScript is a bit of a challenge and carries with it a couple of issues I'll get to later. To get MonoDevelop to output a Library instead of an Exe you can't use the GUI like in C#. Instead, open all your .unityproj files in your new solution and change the line <OutputType>Exe</OutputType> to <OutputType>Library</OutputType>.

Compiling and Merging:
Unity has a big problem which makes it impossible to just import your compiled DLLs if you're using the base class method outlined above. Unity will not recognize any subclasses in a DLL extended from a MonoBehaviour-based base class in a separate DLL. See this post for details.

In order to get around this problem, you must merge your BaseClasses.dll with all your other BaseClass-extended DLLs. This way all the bases and their subclasses exist within one DLL by the time Unity gets it and there is no problem. (Well, except for the fact that your dropdown list will contain ALL your classes not in alphabetical order which is kind of ugly if you need to assign them.)

To facilitate this, MonoDevelop projects should be set up in a particular way. Set all projects to output to solution/bin/merging so every DLL will be in one place when compiled. Also, right click on every reference in every project and uncheck "Local Copy" -- this will prevent MD from copying all the dependencies into this output folder with every compile. Also, copy the common dependencies to this folder or a subfolder: UnityEngine.dll, UnityEditor.dll, and for UnityScript users, Boo.Lang.dll and UnityScript.Lang.dll (find them in UnityEditorDataMonolibmono2.0 in Windows).

Microsoft ILMerge is the tool to use for merging the dlls. It runs from the command line. It will also merge PDB files so you will have line numbers reported from Unity for bug testing, etc. Create a batch file to merge the base class and any other MonoBehaviour-derrived class dlls together into one dll. I call mine Core.dll. You don't have to merge your utility classes as they can just be copied as is. (I recommending forcing ILMerge to ouput a .NET 2.0 class with the switch /targetplatform:v2.)

It's unfortunate you have to go through this extra step, but merging only takes 2-3 seconds even with the 160 or so MonoBehaviours I have in my project and I only have to do it if I change something in a MonoBehaviour-derrived class.

PDBs, MDBs, and Merging:
Compiling with MonoDevelop outputs .PDB files, but Unity wants .MDB files. ILMerge also requires PDB files and outputs a merged PDB files. This PDB file must be converted to an MDB file for Unity. Unity comes with pdb2mdb.exe (in UnityEditorDataMonolibmono2.0 in Windows) which can convert PDBs to MDBs (well, most anyway). Run it from the command line pdb2mdb Assembly.dll, but your current directory MUST be the directory the .dll and .pdb reside in or you'll get an error. Convert your merged DLL and any non-merged utility class assemblies DLLs you have as well. (Add all this to your batch file created in the previous step.)

NOTE: pdb2mdb is buggy and doesn't work on all assemblies. I had some problems trying to convert one of my assemblies (IEnumerable error), so I ended up having to make my own pdb2mdb converter using Mono.Cecil which you can get here. It was pretty easy with help from here on where to start.

NOTE 2: .PDB files are output in the format AssemblyName.PDB, however .MDB files should be formatted AssemblyName.dll.mdb in order for Unity to recognize them. Plan for this accordingly.

Copy to your Unity project:
An addition to the merging and PDB converting batch file, also add copying the merged .dll + .mdb and any other loose .dlls + .mdbs you have to your UnityProject/Assets/Assemblies folder. (Anywhere under Assets is fine.)

Workflow:
I made a few helper programs and a pretty nice batch file to make this process smoother for me. After compiling a change to any of my DLLs, I just run a single batch file from a hotkey or any convenient launcher to detect changed DLLs, re-merge if necessary, and copy all the changed DLLs and MDBs to the Unity folder, and let me know of any problems such as locked files, etc. It's quick and only adds one additional button press beyond telling MD to compile. (I COULD add it to an "After Build" command but I don't want it running every time necessarily.)

Result:
Once your structure is set up, you should be able to compile each individual project relatively quickly instead of having to wait for everything to compile every time. You can also work on new scripts in the Unity Assets folder just like before, but you won't have to wait for hundreds of other scripts to compile because they're already dlls. Those scripts can later be moved to DLL format too once you're finished with them.

Note about .NET version for UnityScript users:
I have encountered an error before with some .dlls compiled from MonoDevelop because MonoDevelop ALWAYS compiles to .NET 4.0 even if you try to force it to use .NET 2.0. Most of the time this isn't a problem, but if you see some errors like in this postyou may have to change some code to work around them OR resort to using Unity's internal compiler to compile instead of MonoDevelop to force it to compile in .NET 2.0 as I outlined here. But I recommend against doing this as it requires a TON of extra steps and a series of custom programs to help make the workflow less awkward. In the end I abandoned this approach for many reasons AND the side benefit of using ILMerge to merge your DLLs is you can force it to output a .NET 2.0 even though the source DLLs are .NET 4.0.

/////////////////////////////////////////////////////////////////////////////
STEP 3: Converting your project to use the DLLs instead of the loose scripts
/////////////////////////////////////////////////////////////////////////////

First, change a couple of settings in the editor: (Requires Unity Pro)
Edit -> Project Settings -> Editor
Version Control: Meta files
Asset Serialization: Force text

Version control: Meta files will make Unity output .meta files for every asset which contain the important GUID for the asset. Asset serialization: Force text will make it so Unity outputs text files for all the assets which will make it far easier for us to find/replace all the references to the old loose scripts across the project. (Note: It will take quite a while for Unity to generate these files on a large project.)

Replacing Script References:
All .prefab and .unity (scene) files must have their references to the old loose scripts replaced with references to the new classes in the DLLs.

Unity uses a combination of a fileID and a guid to determine what class is referenced on a prefab or in a scene. These references are stored in the .prefab and .scene files and show up as a line of text because of the asset serialization setting above. This way, references can be easily changed in all objects across the project.

See my post here for information about the reference format.

The trick here is that you will have to build a reference table. You need to know both the guids/fileIds for all your loose scripts and the guids/fileIds for all your new extended classes. Once you get these, it's a simple matter of search and replace across the project referencing the table.

In order to build the table, I suggest starting a new Unity project and copy all your old loose scripts WITH their .meta files to it. Create one gameobject and apply all your scripts to it. Create a prefab out of it. Write a small editor script to give you a list of all components on the object and reverse the order of the list because the prefab stores them in reverse order of how Unity's inspector shows or the output of a for(Component c in GetComponents(Component)) loop. Filter out any components that may have been added automatically as a result of assigning a MonoBehaviour like CharacterController, etc. Then parse the prefab file looking for m_Script: {fileID: # guid: # lines and copy out all the fileIds and guids. Finally compile it into a table.

Delete the gameobject, prefab, and scripts and do the same thing again with the DLLs this time (also remember to bring the meta files as well so the guid will match when you copy it out.) Assign all the extended MonoBehaviour classes ("_Ext" classes in my case) to a gameobject and repeat. You don't need to do it for the base classes because you are never going to assign these directly to anything, just the extended ones.

I suggest backing up your .prefabs and .scenes again now in case something goes wrong and you need to re-do the reference replacement. It's a lot less hassle than restoring the entire backup if you have a lot of other assets.

Now with your table in hand, write a program to search the UnityProject/Assets folder for all .unity and .prefab files, search for fileId + guid pairs, and replace them with the fileId and guid of the matching extended class from the dll. (Using the suffix "_Ext" allowed me to easily find the replacement in the table.)

Now open the project in Unity and verify the MonoBehaviours converted properly by looking at some prefabs. Make sure they were replaced with the expected class and that all the serialized data fields are there. Also check and make sure any serialized object references that point to a script type now point to the new extended type.

If everything looks good, delete loose scripts.

---------------------------------------------------------------

DONE!

Your compile times should now vastly improved.

Also, as a bonus, the reference replacement process could also be used when converting an entire project to another language. I'm seriously thinking about switching my whole project to C# so I can finally have a decent IDE to work on.

EDIT: Since I made this post, I have indeed converted my project to C# and boy was I surprised at the results. C# compiles between 4X and 12.5X faster than UnityScript! Had I known this before, I wouldn't have bothered with the complex DLL setup because C# is fast enough to just keep using Unity's loose script scheme. Using the loose script setup, C# is 4X faster, and using the many-dll's set up, C# is 12.5X faster at compiling. Either way you go, you can't loose if you just bite the bullet and convert everything to C#.