Unity caches GameObjects based on their tags, so in a big scene with lots of objects,
GameObject.FindWithTag is several orders of magnitudes faster that GameObject.Find.
It still does not beat the simple deserialization of a direct public variable link (in terms of speed).
Just for the record: Object.FindObjectOfType is A LOT slower than GameObject.Find.
In our measurements, the relation is roughly: 10x GameObject.FindWithTag = GameObject.Find and 100x GameObject.Find = Object.FindObjectOfType.
Of course, all that only matters if it really matters. 99% of the cases, it just does not matter so if you don't write a tight Update() function in a big scene targeting iPhone 3, go on with any method you like until you run into trouble.