update-engine
https://code.google.com/archive/p/update-engine/
Update Engine is a flexible Mac OS X framework that can help developers keep their products up-to-date. It can update nearly any type of software, including Cocoa apps, screen savers, and preference panes. It can even update kernel extensions, regular files, and root-owned applications. Update Engine can even update multiple products just as easily as it can update one.
Google ships many different pieces of Mac software ranging from simple Cocoa applications to complicated software systems. We needed a robust software update solution that would be able to update all of our current and future Mac products. And we designed and built Update Engine to solve this problem.
The following are some unique features of Update Engine:
- Can update virtually any type of product (e.g., Cocoa app, pref pane, kernel extension)
- Can update non-bundle-based products (e.g., command-line tool, plain file)
- Can update many products at once
- Solid framework on which to build
We have two movies for you to watch.
The first is by Greg and it gives you and overview of Update Engine, why we built it, and what makes it tick. In short, HowDoesUpdateEngineWork. IMPORTANT: this demo uses "http:" URLs in the example tickets to save space. To keep things secure in the real world, https: URLs should be preferred when fetching the plist from the server. See UpdateEngineAndSecurity for more details.
The second is by Mark and it walks you through running the HelloEngine sample program.
To browse the Update Engine source code, visit the Source tab. Changes to Update Engine are documented in the release notes.
If you find a problem/bug or want a new feature to be included in Update Engine, please join the discussion group or submit an issue.
Update Engine follows the Google Objective-C Style Guide.
The Update Engine developers believe in TestingAndCoverage, with a goal to maintain 90% or better code coverage at all times. Our first release has 97.4% coverage.
https://code.google.com/archive/p/update-engine/wikis
wiki:
1 update-engine - PlistServerConfiguration.wiki
Update Engine provides KSPlistServer, a KSServer subclass, that reads a property list at a given URL. This property list contains the information that Update Engine needs to determine if an update is needed, and where to download that update from.
Rules
The property list should contain a dictionary with one entry called Rules. Rules is a an array of dictionaries, one for each product. Each rule dictionary should have these keys:
-
ProductID - The unique identifier for the product (same as the product ID used in the ticket). The identifier for the product. Update Engine doesn't care about the actual contents of the ProductID. They can be bundle IDs, UUIDs, blood types or shoe sizes. Update Engine just compares them for equality with the product IDs in the tickets.
-
Predicate - Any NSPredicate compatible string that determines whether the update described by the current rule should be applied. There are some Update Engine specific values you can use in your predicate.
-
Codebase - The URL where the update should be downloaded from. This URL must reference a disk image (DMG).
-
Hash - The Base64-encoded SHA-1 hash of the file at the
CodebaseURL. An easy way to calculate this is with the command:`openssl sha1 -binary dmg-filename | openssl base64` -
Size - The size in bytes of the file at the "Codebase" URL.
Predicates
The Predicate in the rule enables lots of flexibility and configurability. The string specified for the predicate should use Apple's predicate format string syntax and will be converted into an NSPredicate and run against an NSDictionary with two attributes:
- SystemVersion - This gives the predicate access to version information about the current OS. The value of this key is the contents of the file:
/System/Library/CoreServices/SystemVersion.plist - Ticket -This gives the predicate access to the product's currently installed version information via its corresponding KSTicket.
The ticket itself contains several attributes. Check out KSTicket.h for all of them. The most useful one would be version, the product version in the ticket. Also, any additional Key/Value pairs from the plist file will be included in the Ticket dictionary, so you can use this information in your predicate.
Example Plist 1
This plist contains one rule whose predicate says to install the update at "Codebase" if the currently installed version is not version 1.1. This rule may work for a product whose current version is 1.1 and all versions that are not 1.1 should be "upgraded" to version 1.1.
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "https://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>Rules</key> <array> <dict> <key>ProductID</key> <string>com.google.Foo</string> <key>Predicate</key> <string>Ticket.version != '1.1'</string> <key>Codebase</key> <string>https://www.google.com/engine/Foo.dmg</string> <key>Hash</key> <string>8O0cLNwuXQV79dMylImBvuSD9DY</string> <key>Size</key> <string>4443812</string> </dict> </array> </dict> </plist>
Example Plist 2
This plist lists two rules for two different products (Foo and Bar). The Foo product will only ever apply to Tiger machines because the predicate looks for "SystemVersion.ProductVersion beginswith '10.4'". The Bar product only targets Leopard systems because its predicate includes a similar check for a system version beginning with '10.5'. The rest of this plist should be pretty straight forward.
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "https://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>Rules</key> <array> <dict> <key>ProductID</key> <string>com.google.Foo</string> <key>Predicate</key> <string>SystemVersion.ProductVersion beginswith '10.4' AND Ticket.version != '1.1'</string> <key>Codebase</key> <string>https://www.google.com/engine/Foo.dmg</string> <key>Hash</key> <string>somehash=</string> <key>Size</key> <string>328882</string> </dict> <dict> <key>ProductID</key> <string>com.google.Bar</string> <key>Predicate</key> <string>SystemVersion.ProductVersion beginswith '10.5' AND Ticket.version != '1.1'</string> <key>Codebase</key> <string>https://www.google.com/engine/Bar.dmg</string> <key>Hash</key> <string>YL8O0cuwVNXd7Q9lyBIMmvSD9DYu=</string> <key>Size</key> <string>228122</string> </dict> </array> </dict> </plist>
Example Plist 3
This is the server response plist from HelloEngine
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>Rules</key> <array> <dict> <key>ProductID</key> <string>com.google.HelloEngineTest</string> <key>Predicate</key> <string>Ticket.version != '2.0'</string> <key>Codebase</key> <string>file:///tmp/TestApp.dmg</string> <key>Size</key> <string>51051</string> <key>Hash</key> <string>L8O0cuNwVXQd79lMyBImvuSD9DY=</string> </dict> </array> </dict> </plist>
2 update-engine - TestingAndCoverage.wiki
Testing and Code Coverage
We strive to write some of the best code here at Google, and to prove that it really is as good as we think we need to make sure it's very well tested. To make sure it can be very well tested, we need to design our code to be testable from the beginning. Code that is not designed to be testable is rarely easy to test, whether you're talking about unit tests, system tests, or even manual testing.
The Update Engine engineering team is very proud of the level of unit testing and test coverage that we've achieved. Every source file in Common and Core has a corresponding unit test.
Designing Testable Code
Designing code to be testable is extremely important and has a number of benefits:
- It's easier to unit test the code and find bugs before users (or even your coworkers) do
- The developer of a piece of code can use the code as a client, which can improve the code's API. If it's awkward to use in a test, it'll be awkward to use in real life.
- Testable code tends to be more modular
- Testable code tends to be more flexible
Only one of these benefits is about finding bugs. The remaining points are all about improving the code. We feel that the majority of the benefit of unit testing (also known as "developer testing") is about producing all-around higher quality code.
Write Object-Oriented Code
Writing object-oriented code may be an obvious point, but it's one one that's important and can be easily overlooked. Writing object-oriented code is more than just using the @interface keyword, or using the term "method".
Fundamentally, object-oriented code is made up of a number of separate objects that interact with each other through their interfaces to accomplish a given task. Simply wrapping a handful of C functions in a class definition is not object oriented.
Well-written object-oriented programs look and feel very different from their functional or procedural counterparts. Object-oriented code demands an object-oriented mindset just like functional programming demands a functional mindset. Be sure to think of your problem space in terms of objects, not functions. Design the solution to your problem in terms of objects and their interactions, nouns and verbs, and not functions and their arguments.
Classes describe objects, objects represent things, and things do stuff. Useful things typically have useful interfaces. Make sure you clearly understand the object model for your program before attempting to codify it in class implementations. Think about how the objects in your model will interact and let this information help drive the objects' interfaces.
Carefully create your class's interface. A class filled with a bunch of no-arg functions that return void are generally not very useful. This is also a sign of a weak object concept, or an attempt to cram a functional program into a class to make it appear object oriented.
Similarly, classes with few to no methods in the public interface indicate a class that is just one big implementation detail. This class will be difficult to work with, extend, test, and re-use. Classes like this look like functional code wearing an unconvincing OO costume.
Naming
The names of the classes, objects, variables, methods, and functions in your program creates a vocabulary that describes your program to the compiler and other engineers. These words should be very carefully chosen so that your program reads well and clearly shows its intent. Imagine a novelist or a poet struggling over word choice. We as developers should invest the same kind of effort on our names. The compiler doesn't care but your peers will. As will you when you reread the code a year or two later.
Classes should have a specific purpose and should have concise, descriptive names. Generic names like "Controller", "Manager", and "App" tend to be so vague that everything really "fits", which lends itself to kitchen sink syndrome. You should be able to describe the essence of a class in one sentence. If you can't clearly and concisely describe what a class is or what it does, it represents a poorly defined concept. Go back to the drawing board and refactor this class/concept before it infects other parts of your code. Take some more time to think about the problem, grab a cup of coffee, clarify the concept you're trying to model, have a snack, and a good name will reveal itself.
Perhaps the object you had trouble naming should really be broken up into two separate classes, each with a very descriptive name and purpose. Your code will end up being much clearer and easier to maintain, and you'll be glad you waited.
If at any point you think of a better name for an existing class/method/variable, rename it. There is no reason to continue using a poorly named object if you know of a better name. Xcode includes a Refactoring command that will do project wide renamings. There is also a command-line utility called tops that let you do similar things on a more localized basis.
Use Dependency Injection
Dependency Injection is awesome and easy. It can help you write more loosely coupled code that is much easier to test.
Rather than an object taking responsibility for acquiring some resources, Dependency Injection says to make that resource a property of the object instead. OK, so what does that actually mean?
Take a look at KSUpdateCheckAction. This is an action class that checks for updates using a KSServer to do the actual server communications. We could have designed KSUpdateCheckAction to use a default server, or to query some kind of Abstract Server Factory to acquire a global server object.
Instead, the server is an argument to the class creation and init methods:
+ (id)checkerWithServer:(KSServer *)server tickets:(NSArray *)tickets; - (id)initWithServer:(KSServer *)server tickets:(NSArray *)tickets;
This allows KSUpdateCheckActionTest to create special severers that assist in testing. The tests can just create a custom server object, and then create a new update check action with that new server.
Similarly, KSMockFetcherFactory is a testing class that let us provide mock factories with special behaviors, such as providing a fetcher that always fails with error, or will supply a given NSData to other objects in the system.
Tests that require a connected network or a database server are frequently a big problem. Network and database operations are usually slow. Slow tests bog down your entire develop / test / debug / curse cycle. Plus they can make your tests break for reasons completely unrelated to the code being tested: The database may be down. The cat may have unplugged the Time Capsule. Using dependency injection for the networking and database classes can let you supply simpler test objects that will exercise all the dark corners of your class without pulling in a lot extra complexity.
Oh by the way, one simple, but surprisingly non-obvious thing, is using file:// URLs with APIs that take URLs. file:// URLs let you put test data into a class without requiring a live network connection or running a web server somewhere.
Use Good Object-Oriented Design Patterns
You might have seen the term "GoF" before. It stands for Gang of Four, referring to the four authors of the quintessential book Design Patterns. Even though the book is somewhat dated, and should not be followed religiously (amen), you should certainly have a copy or have easy access to one. The ideas and terminology presented in the book are important and should be understood by all software engineers who write object oriented code. Some of the discussed patterns are certainly more common than others. Don't worry about memorizing the details of all of them. Just be familiar with them, and know where you can look to find the details when necessary.
Beyond the GoF's 1994 magnum opus, there are a number of other "patterns" that can be very useful. Formal refactoring techniques, as described by Martin Fowler in his Refactoring book, can help you improve the design of existing code.
Before you can refactor, you must know when to refactor code. Sometimes refactoring is needed almost on a daily or weekly basis. Some of the big signs that you need to stop, think, and refactor your code are when
- You and your co-workers curse a certain piece of code daily
- A method, function, or class is getting too big or complicated to understand or to change
- A class has too many responsibilities
- Classes are too closely coupled
- An object is "too difficult" to test
Strive for very loosely coupled classes. Understand the Liskov substitution principle, the Law of Demeter, and the Open/closed principle. Learn when to apply refactorings and patterns, and when not to. Know the names of refactorings and patterns so that you can more easily communicate with your fellow engineers. Lots of new ideas in software engineering like Agile Development, Test-Driven Development, and eXtreme Programming have deep roots in topics like OO design patterns and refactorings.
Know when and how to properly subclass. Subclass when you're defining a new type that truly "is a" refinement of the parent class's type. Do not subclass simply as a way to reuse code. Subclasses have a very intimate relationship with their parent class, which can lead to unwanted dependencies and relying on hidden assumptions. In general, prefer composition ("has a") to inheritance ("is a").
Singletons Considered Harmful
Also, keep in mind that the Singleton pattern is often overused and has become a kind of antipattern. If you can accomplish your goal without using a singleton, do it. Never use a singleton simply because you want global access to an object. Fear the Singleton just like you would fear any global variable.
Singletons are hard to test because they are a global resource. One test can mess up the Singleton's state for the next test. It can also be hard to get to the underlying functionality of a singleton to test it. It's ok to have a class method to give global semantics to something, such as NSUserDefaults' +standardUserDefaults, but go ahead and allow multiple instances of the class to be created, manipulated, and destroyed. If for no other reason to allow for better testability.
Prefer Instance Methods to Class Methods
It's not uncommon to see utility classes that are full of class methods and no instance methods. In this case, the class is merely defining a namespace, or scope, in which to group these hopefully related methods (in reality they're actually just "functions"). This is generally not a good idea for a number of reasons:
- Class methods can be more difficult to mock and for subclasses to override properly
- Class methods can make dependency injection more difficult
- Class utility methods may indicate a design flaw in your object model
- Class methods do not play well if you need to maintain state
Even if your class currently does not have any of these problems, don't forget that you still need to think about the future. Will the class ever need to maintain state, say by adding a cache? Will anyone ever want to "inject" an instance of your class for testing? Thankfully, there is a wonderful alternative to class methods -- instance methods!
Instance methods solve all of the problems with class methods, except for one: instance methods are slightly longer to type because you need to get an instance of the object first. However, with advanced IDEs like Xcode that have code completion, this is really not a problem. It's much better to prefer instance methods to class methods. Notice that Apple rarely uses class methods for utility classes either. Take a look at NSProcessInfo, NSFileManager, and NSWorkspace.
And for those who are curious what a good use of class methods is, it's convenience creation methods like NSString' +stringWithFormat:.
Also note that Objective-C classes cannot have static methods -- they have class methods. The difference is that class methods are dynamically bound at runtime, whereas static methods are bound at compile time. In other words, Objective-C class methods can be overridden by subclasses. However, class methods do not have access to any storage other than static variables. This overlap of dynamically bound class methods and static storage can cause bizarre problems that are no fun to debug. Do not use class methods if you need to maintain state. Again, instance methods are better in almost every way.
Update Engine, Testing, and Code Coverage
The Update Engine Xcode project is already set up to generate coverage when the tests are run in Debug mode. To see the coverage for Update Engine, build and run the "Test All" target. Once it's finished doing its thing, drag and drop your build directory on top of CoverStory.
To filter out the Google Toolbox for Mac source files, which we don't completely exercise in our tests, enter /update-engine/C in the search box at the bottom of the CoverStory window.
3 update-engine - EngineRunnerCommand.wiki
Engine Runner
The EngineRunner command is one of the Update Engine sample programs. It is a command-line tool that exposes some useful bits of Update Engine functionality. You can use it to experiment with Update Engine, or to maintain ticket stores, or even to run the update process for your applicaton. You can include the code directly in your application, or include the EngineRunner command in your application bundle and call it directly with NSTask or GTMScriptRunner
Command Syntax
The syntax for EngineRunner is
EngineRunner _command_ -arg1 value -arg2 value -arg3 value ...
To see all of the commands supported by EngineRunner, run it without any commands (these are all entered into the shell on one line):
$ EngineRunner EngineRunner supports these commands: add : Add a new ticket to a ticket store change : Change attributes of a ticket delete : Delete a ticket from the store dryrun : See if an update is needed, but don't install it dryrunticket : See available updates in a ticket store but don't install any list : Lists all of the tickets in a ticket store run : Update a single product runticket : Update all of the products in a ticket store Run 'EngineRunner commandName -help' for help on a particular command
To see help on a given command run the command with the -help flag:
$ EngineRunner change -help change : Change attributes of a ticket Required arguments: -productid : Product ID to change -store : Path to the ticket store Optional arguments: -url : New server URL -version : New product version -xcpath : New existence checker path
This will show you the optional and required parameters.
Running an Update
To run an update, provide all of the information for the product on the command line:
$ EngineRunner run -productid com.greeble.hoover -version 1.2 -url http://example.com/updateInfo finished update of com.greeble.hoover: Success
If you want to see if your application would be considered updatable, use the dryrun command. That goes through the motions of updating, but doesn't actually download or run anything.
$ EngineRunner dryrun -productid com.greeble.hoover -version 1.2 -url http://example.com/updateInfo Products that would update: com.greeble.hoover
Manipulating Tickets
If you have several products you're managing, you might want to use a ticket store to consolidate all of the update information in one place. To add a ticket to a store, use the add command:
$ EngineRunner add -store /tmp/store.tix -productid com.greeble.hoover -version 1.2 -url http://example.com/updateInfo -xcpath /Applications/Greebleator.app
To see what tickets you have in the store, use the list command:
$ EngineRunner list -store /tmp/store.tix 1 tickets at /tmp/store.tix Ticket 0: com.greeble.hoover version 1.2 exists? NO, with existence checker <KSPathExistenceChecker:0x317d60 path=/Applications/Greebleator.app> server URL http://example.com/updateInfo
The "NO" after "exists?" is the actual return value of the existence checker. In this case, there is no /Applications/Greebleator.app.
Updating With a Ticket Store
To see what products need an update (without actually running an update), use dryrunticket:
$ EngineRunner dryrunticket -store /tmp/store.tix No products to update
$ EngineRunner dryrunticket -store /some/other/ticket/store.tix Products that would update: com.google.greeble com.google.bork
To actually run an update, use runticket:
$ EngineRunner runticket -store /some/other/ticket/store.tix finished update of com.google.greeble: Success finished update of com.google.bork: Success
Or supply a productID to just update one product:
$ EngineRunner runticket -store /some/other/ticket/store.tix -productid com.google.bork finished update of com.google.bork: Success
4 update-engine - ActionProcessor.wiki
The Update Engine Action System
Writing and comprehending sequential programs of the form do_A(), then do_B(), then do_C() is generally very easy. You can simply look in one place in the source code and see exactly what's supposed to be happening. It's clear that do_C() is run after do_B(), which itself is run after do_A() completes. Even if we add a little logic to the flow so that do_C() only runs if do_B() finished successfully, it's generally pretty straightforward to understand.
You typically loose this simplicity and clarity when using asynchronous tasks - there is typically no one place in the source code that lists the 3 (in this example) tasks to be done. Furthermore, you often need to come up with convoluted ways to have one task kick off another task once it finishes. This can make the intent of the program much more difficult to discern, in addition to making the code less flexible. Nearly all of the tasks that Update Engine needs to accomplish are asynchronous in nature (e.g., checking for updates, downloading a file), so we knew this was a problem that needed to be solved.
We solved this problem in Update Engine with what we call the Action System. The action system is the mechanism by which Update Engine accomplishes and manages all of its tasks, asynchronous or not, in a simple and orderly manner. There are three primary abstractions that make up the action system:
- KSAction -- an abstract base class that defines a unit of work (e.g.,
KSDownloadAction,KSInstallAction) - KSActionProcessor -- a queue of
KSActioninstances that will be run one at a time in the order they appear in the queue - KSActionPipe -- connects the "output" of one
KSActionwith the "input" of another; analogous to a typical Unix command-line pipe
Each task that Update Engine needs to perform is encapsulated as an action, and that action is added to an action processor. The action processor is then responsible for running the action at the appropriate time. Actions, that is, subclasses of KSAction, each implement a -performAction method that the action processor calls when it is time to run. Actions may run for as long as they need, since they're asynchronous, but they must inform the action processor when they are finished. Once an action finishes, the action processor will start the next action in the queue, and so on, as shown here: (actions are shown running from left to right)
Actions may communicate with one another using KSActionPipes. KSActionPipes are analogous to typical Unix pipes, except that KSActionPipes pass objects rather than using a simple stream of bytes. Pipes allow multiple actions to work together to accomplish a greater task. For example, a task for downloading a file (KSDownloadAction) may be combined with another action for installing something from a disk image (KSInstallAction) to accomplish the greater task of "updating a product" (KSUpdateAction). This allows the intent of the code to be expressed at the appropriate level of abstraction.
The action system allows Update Engine code to clearly express the intent of otherwise very complex, asynchronous tasks. The following is an actual snippet from the KSUpdateEngine class that shows how the action system allowed us to very concisely express the essence of the entire Update Engine framework.
``` - (void)triggerUpdateForTickets:(NSArray *)tickets { // Build a KSMultiAction pipeline with output flowing as indicated: // // KSCheckAction -> KSPrefetchAction -> KSSilentUpdateAction -> KSPromptAction
KSAction *check = [KSCheckAction actionWithTickets:tickets params:params_]; KSAction *prefetch = [KSPrefetchAction actionWithEngine:self]; KSAction *silent = [KSSilentUpdateAction actionWithEngine:self]; KSAction *prompt = [KSPromptAction actionWithEngine:self];
[KSActionPipe bondFrom:check to:prefetch]; [KSActionPipe bondFrom:prefetch to:silent]; [KSActionPipe bondFrom:silent to:prompt];
[processor_ enqueueAction:check]; [processor_ enqueueAction:prefetch]; [processor_ enqueueAction:silent]; [processor_ enqueueAction:prompt];
[processor_ startProcessing]; } ```
It's not too difficult to see from this code that Update Engine first checks for updates, then it may pre-download some things, then it may silently install some updates, and finally it may ask the user if they would like to install some additional updates. These are the four main stages of a Update Engine update cycle. The action processor built in the previous code snippet is shown here:
The first action to run is the Check action. Its input is an array of tickets and its output is an array of dictionaries (KSUpdateInfos) that each represent one available update. All of the actions in this queue communicate with one another by emitting an array of KSUpdateInfo dictionaries, allowing allowing a uniform I/O interface on each of these actions. Now for a closer look at each of these high-level actions in more detail.
The "Check" Action
The KSCheckAction class takes the array of tickets from its input, groups them by the server they need to talk to, and then creates one KSUpdateCheckAction for each server back-end that needs to be contacted. It will then run all of these subactions using its own, internal, action processor. The KSCheckAction itself is not finished until all of its subactions are finished. Once that happens, KSCheckAction's output is the aggregate of all of its subactions' output, as shown here:
The KSUpdateCheck subactions will communicate with server backends to determine what software (if any) needs to be updated. However, we wanted to keep the design flexible enough so that we could change back-ends. For example, Update Engine ships with KSPlistServer, which is a different server protocol than what is used internally at Google. Update Engine users are also welcome to write their own server subclasses to satisfy their individual needs.
KSCheckActions use two primary abstractions when dealing with server backends: KSServer and KSUpdateCheckAction. KSServer is an abstract base class whose subclasses encapsulate all of the knowledge about a specific instance of a "Update Engine back-end server". Concrete instances of this class will contain all of the knowledge specific to a particular type of server, as well as a URL identifying a particular instance of the server. This class will handle creating NSURLRequest objects from an array of KSTicket objects. Similarly, it will handle creating KSUpdateInfo dictionary instances from the server's response.
The KSUpdateCheckAction class is a concrete KSAction subclass. The objects must be created with a KSServer instance and an array of KSTickets, and it will then handle the HTTP(S) communication with the server, and coordinate with the given KSServer instance to create KSUpdateActions from the server's response. The KSUpdateCheckAction will do HTTP(S) communication using GDataHTTPFetcher, so it will automatically have back off and retry logic.
The Prefetch Action
The KSPrefetchAction class takes as input the array of KSUpdateInfo dictionaries from KSCheckAction's output. The prefetch action sends a message to the Update Engine's delegate and asks which of the updates should be downloaded immediately. Based on the response from the delegate, KSPrefetchAction will create one KSDownloadAction for each update to be prefetched, and it will run them on its own internal action processor in the same way that KSCheckAction did previously. The output of KSPrefetchAction is always the same as its input, whether or not anything was actually downloaded. The act of downloading doesn't change the set of files that need updating.
KSDownloadAction is responsible for downloading all files. It downloads files by NSTask'ing a separate "ksurl" process to do the actual work. "ksurl" (like "curl") is a small command-line tool that simply uses Foundation's NSURLDownload to download a file. KSDownloadAction does this in a separate process as a security measure to ensure that downloads (a network transaction) never happen as root. Once downloaded, KSDownloadAction will verify that the downloaded file matches a known size and pre-computed SHA-1 hash to guarantee the downloaded file is intact and wasn't tampered with.
The Silent Update Action
KSSilentUpdateAction objects take as input the array of KSUpdateInfo dictionaries that were the output of the KSPrefetchAction. It then sends a message to the Update Engine's delegate asking which of the updates should be immediately installed silently. Depending on the delegate's response, some number of KSUpdateActions will be created to install the requested updates.
KSUpdateAction itself is a "composite" action, following the Composite design pattern, made up of a KSDownloadAction and a KSInstallAction. If the update to be installed was previously downloaded in the prefetch stage, then this download action will find the download cached and the action will complete immediately. Otherwise if the update was not prefetched, the download will happen at this point. Either way, once this download completes, the KSInstallAction will handle installing the downloaded update by running the scripts on the disk image.
The Prompt Update Action
The last stage of our main action processor's pipeline is the KSPromptAction. This action is almost identical to the KSSilentUpdateAction with the only difference being the callback that it sends to the Update Engine's delegate. This difference is necessary so that we can separate updates that require user permission and updates that do not. KSPromptAction and KSSilentUpdateAction are both subclasses of the KSMultiUpdateAction abstract base class.
The following diagram shows the hierarchy of all of the actions that we've discussed so far. Note that actions are processed from left to right.
Writing Your Own Actions
You're not limited to the action subclasses provided by Update Engine. You're welcome to create your own KSAction subclasses and operate your own KSActonProcessors.
Update Engine includes the "Actions" sample which uses KSAction subclasses for downloading a set of image files and displays them in an NSTableView, so you can see some custom actions in action.
Subclassing KSAction
There's only two things you must do when subclassing KSAction: perform the action, and then let the action processor know when you've finished.
Perform Action
The first thing to do is override -performAction. This is where you perform your work, whether it's something asynchronous like fetching something over the internet:
``` - (void)performAction { // Kick off an HTTP fetch for the catalog. NSURLRequest *request = [NSURLRequest requestWithURL:catalogURL_]; httpFetcher_ = [[GDataHTTPFetcher alloc] initWithRequest:request]; [httpFetcher_ beginFetchWithDelegate:self didFinishSelector:@selector(fetcher:epicWinWithData:) didFailSelector:@selector(fetcher:epicFailWithError:)];
} // performAction ```
or performing a synchronous processing action:
``` - (void)performAction { // Pick up the array of url strings from the previous stage in the pipeline. NSArray *imageURLStrings = [[self inPipe] contents];
// If we have a predicte, filter the incoming array. NSArray *filteredURLStrings; if (filterPredicate_) { filteredURLStrings = [imageURLStrings filteredArrayUsingPredicate:filterPredicate_]; } else { filteredURLStrings = imageURLStrings; }
// Send the results to the next action in the chain. [[self outPipe] setContents:filteredURLStrings];
// All done! [[self processor] finishedProcessing:self successfully:YES];
} // performAction ```
Finished Processing
You need to tell the action processor when you've completed your action by calling -finishedProcessing:successfully:
[[self processor] finishedProcessing:self successfully:YES];
This tells the action processor that it is time to move on to the next action.
Using Pipes
Recall that KSActionPipe is used to connect actions to each other. Before you add actions connected by a pipe to an action processor, you need to hook up the pipe:
``` UECatalogLoaderAction *catalogLoader = ...; UECatalogFilterAction *filter = ...;
[KSActionPipe bondFrom:catalogLoader to:filter]; ```
Then, in your first action subclass (the catalogLoader in this case), before you tell your action processor that you have -finishedProcessing:, you put the object into the pipe that you want the other action to find:
[[self outPipe] setContents:urlStrings];
In your second action subclass (the filter in this case), you read from the pipe in your -performAction :
NSArray *imageURLStrings = [[self inPipe] contents];
Why not NSOperation?
One good question is "why not use NSOperation for this stuff." NSOperation can be used for much of what the action processor does.
The main reason is that Update Engine needs to support Tiger (Mac OS X 10.4). NSOperation and friends are 10.5 and later.
NSOperations don't have a pipe concept. We would need to add that on top of NSOperations. Not difficult, but would be extra work. The operations are executed sequentially, so it would be a little extra work to set up the proper operation dependencies before adding them to the operation queue.
The last reason is that that the action processor does not need to use a new thread for its operations since the majority of the work done is either network-related (asynchronous, runloop based), or very fast (like the "Actions" sample image URL filter). To get non-threaded operations, we would need to use "concurrent" NSOperations for this, which are awkward to implement, and we would need to implement the operation's KVO notifications when the operations change state.
5 update-engine - UpdateEngineAndSecurity.wiki
Update Engine and Security
Update Engine's goal with security is to let you do the simplest thing in the most secure way possible.
The Recommended Approach
We recommend that you use https (a.k.a. Secure HTTP, which uses SSL, the Secure Sockets Layer) to retrieve your update plist. By using https, the server response is trusted.
Update Engine retrieves the server response from the URL listed in your ticket. When the server check happens via https you know the response comes from a verified server, so you can trust it.
Update Engine uses the download URL in the server response. At google we have product downloads happen via http for performance reasons. https is computationally expensive when you're serving a lot of downloads. But because you can trust the server response, you can trust the hash and file size.
The hash and file size from the server response are used to verify the download's integrity. The combination of SHA-1 hash and file size make it a sufficiently difficult problem for a malicious disk image to be created. Update Engine will automaticaly verify the download with the hash and file size.
Alternatives
Unfortunately, https is not universally available. Your host might not support it, SSL certifications can be expensive, plus the added overhead of configuring your webserver.
Update Engine actually does not impose any particular security model, so you are free to use whatever security mechanism you feel is appropriate.
For example, MacFUSE has shipped with an earlier version of Update Engine, called Keystone. MacFUSE lives on code.google.com, which does not have https support. To securely update McaFUSE, the team has added a server subclass which uses a public and private key to sign the server response plist. This signature is verified once the server response has been downloaded. You can see the code in the MacFUSE code browser, in particular the SignedPlistServer class. In MacFUSE's case the public key is compiled in to the code, but you could use a separate file with the key.
There are some caveats with this approach. You have to be very careful with your private key. Since there is no public key infrastructure involved, a key can't be revoked or changed once your application has shipped. If you lose your private key (natural disaster, bad backups, insane employee) you won't be able to update your app. What's worse is if the private key gets compromised an attacker can use your update mechanism to install something bad. Security is hard. (Let's go shopping.)
If you do decide to go with the key pair route, feel free to use MacFUSE a guide.
That's All Folks
To restate, our recommendation is to simply fetch your plist using https. By using SSL, Update Engine will do proper certification verification and avoid security issues. You can trust the response, so the SHA-1 hash can be trusted, so no reason to sign the download itself. It's easy, and you don't have to reimplement any security mechanisms.
6 update-engine - EngineInstallScripts.wiki
The Engine Install Scripts
Once Update Engine has downloaded an update to an application, it will install the update using the KSAction subclass KSInstallAction. The KSInstallAction class is very straightforward because it will only handle one specific type of install: an Update Engine install. It will not directly handle drag-installs or even pkg installs. These types of installs will be handled indirectly by scripts present on the disk image.
When Update Engine mounts (read-only) a disk image to install an upgrade to an application, it will look for the following scripts and will execute them in order.
.engine_preinstall .engine_install (required) .engine_postinstall
These scripts all have a leading dot so that the DMG used for updating the application could be the same one that customers download directly from your company website.
They don't have to be scripts, just something executable. So feel free to use ObjectFORTRAN++, or one of the built in OS X scripting languages. Be aware of language version differences between OS revisions. For example, python on Tiger is version 2.3.5, which is missing a number of features and libraries than python 2.5.1 on Leopard.
The Scripts
First up is the .engine_install script, which is the only script that is required. This is what kicks off the upgrade of your product.
"install" is used here, here because from Update Engine's perspective it is "installing" something; likely, it's installing an update for your application. You could use Update Engine as a metainstaller by providing it ticket for an application that has not been installed yet and doing an "update".
The .engine_preinstall and .engine_postinstall scripts are both optional. As you'd expect, the preinstall script is run before the install script, and then postinstall script is run last.
The distinction is academic for most products, you'll just use the install script. If you were installing things as root, you may want to run the pre or post install scripts as the console user so that they could put up user interface. Update Engine doesn't provide this particular functionality, though.
Script Arguments and Environment Variables
Update Engine makes some information available to these install scripts by way of command-line arguments and environment variables. The first argument to each script ($1 in bash-speak) will be the absolute path to the mounted disk image, which will be the path to where the scripts are located). This path may have spaces in it so your scripts should quote appropriately.
These scripts communicate with the other scripts downstream by making each script's standard output available to the other scripts in the form of an environment variable. This allows scripts to communicate by echoing output that the next script could read, grep, or process in some way. Your scripts can simply ignore the enviornment variables if they don't care about them. The output from .engine_preinstall script will be available in the KS_PREINSTALL_OUT variable, and the output from .engine_install will be stored in KS_INSTALL_OUT.
Update Engine will also set environment variables for all of the key/value pairs defined in the PlistServerConfiguration file. For example, if the config file for you product has a key/value par with "foo"="bar", Update Engine would set "KS_foo" to the value "bar" in your install scripts' environment.
Return Codes
Script return codes should follow normal Unix conventions where 0 means success and anything else is considered a failure, with two exceptions: * KS_INSTALL_TRY_AGAIN_LATER (77) from .engine_preinstall or .engine_install says "try again later". * KS_INSTALL_WANTS_REBOOT (66) from .engine_postinstall says that a reboot will be necessary.
If the preinstall script returns KS_INSTALL_TRY_AGAIN_LATER, Update Engine will immediately stop processing the update without considering it an error. The next time the install is attempted the .engine_preinstall script is free to return the same "try again later" value, in which case the same thing will happen again. This can be useful for applications that want to check "is it safe to update me now? oh, no? ok, well, I'll just have Update Engine try me again in a little bit".
KS_INSTALL_WANTS_REBOOT indicates to Update Engine that a reboot is necessary for successful completion of the update. Update Engine itself will not solicit or cause a reboot, but the -engine:finished:wasSuccess:wantsReboot: delegate method will be called (if supplied) with YES for wantsReboot. It is up to your program to arrange to reboot the system.
One easy way to do this is by asking System Events to do so with AppleScript, easily done via NSTask or GTMScriptRunner: /usr/bin/osascript -e 'tell application "System Events" to restart'
The downloaded payload for an update will be stored on-disk in a protected directory until the install succeeds. This will make it so the "retry later" case does not have to repeatedly download the same update; the cached update will be used. The hash and file size help ensure the integrity of the downloaded payload. Using https to serve your update plist file will add an extra layer of security.
7 update-engine - UpdateEngineDelegateMethods.wiki
Update Engine's Delegate Methods
These are methods that a KSUpdateEngine delegate may implement. There are no required methods, and optional methods will have some reasonable default action if not implemented.
The methods are listed in the relative order in which they're called.
- (void)engineStarted:(KSUpdateEngine *)engine
Called when Update Engine starts processing an update request. if not implemented, the return value is products.
- (NSArray *)engine:(KSUpdateEngine *)engine shouldPrefetchProducts:(NSArray *)products
Sent to the delegate when product updates are available. The products array is an array of NSDictionaries, each of with has keys defined in KSServer.h. The delegate must return an array containing the product dictionaries for the products which are to be prefetched (i.e., downloaded before possibly prompting the user about the update). The two most common return values for this delegate method are the following:
nil- Don't prefetch anything (same as empty array)products- Prefetch all of the products (this is the default)
- (NSArray *)engine:(KSUpdateEngine *)engine shouldSilentlyUpdateProducts:(NSArray *)products
Sent to the delegate when product updates are available. The products array is an array of KSUpdateInfos, each of with has keys defined in KSUpdateInfo.h. Much of the information in the KSUpdateInfos comes from the ticket, configurable by your PlistServerConfiguration file. The delegate should return an array of the products from the products list that should be installed silently.
If not implemented, the return value is products.
- (id<KSCommandRunner>)commandRunnerForEngine:(KSUpdateEngine *)engine
Returns a KSCommandRunner instance that can run commands on the delegates' behalf. Update Engine may call this method multiple times to get a KSCommandRunner for running Update Engine preinstall and postinstall scripts. See EngineInstallScripts for more details on these scripts.
If not implemented, a KSTaskCommandRunner is created.
- (void)engine:(KSUpdateEngine *)engine starting:(KSUpdateInfo *)updateInfo
Sent by engine when the update as defined by updateInfo starts.
- (void)engine:(KSUpdateEngine *)engine running:(KSUpdateInfo *)updateInfo progress:(NSNumber *)progress
Sent by engine when we have progress for updateInfo. progress is a float that specifies completeness, from 0.0 to 1.0.
- (void)engine:(KSUpdateEngine *)engine finished:(KSUpdateInfo *)updateInfo wasSuccess:(BOOL)wasSuccess wantsReboot:(BOOL)wantsReboot
Sent by engine when the update as defined by updateInfo has finished. wasSuccess indicates whether the update was successful, and wantsReboot indicates whether the update requested that the machine be rebooted. Update Engine will not reboot the system for you. That is the responsibility of your application.
- (NSArray *)engine:(KSUpdateEngine *)engine shouldUpdateProducts:(NSArray *)products
Sent to the delegate when product updates are available. The products array is an array of KSUpdateInfos, each of with has keys defined in KSUpdateInfo.h. The delegate can use this list of products to optionally display UI and ask the user what they want to install, or whatever. The return value should be an array containing the product dictionaries that should be updated. If a delegate simply wants to install all of the updates they can trivially implement this method to immediately return the same products array that they were given.
If not implemented, the return value is products.
- (void)engineFinished:(KSUpdateEngine *)engine wasSuccess:(BOOL)wasSuccess
Called when Update Engine is finished processing an update request. wasSuccess indicates whether the update check was successful or not. An update will fail if, for example, there is no network connection. It will not fail if an update was downloaded and that update's installer happened to fail.
8 update-engine - HowDoesUpdateEngineWork.wiki
Introduction
At a high level, Update Engine is very simple. It talks to a server to determine if installed software needs to be updated, and if so, it downloads and installs the new software.
Tickets
Update Engine knows about installed products through a set of tickets. A ticket describes a single product to Update Engine, including: * Product's name or identifier, like com.google.Earth * Version number, like 3.1.41.59 * URL for the product's update information, like http://example.com/snargleblaster/update.plist
Update Engine can update multiple products simply by specifying a ticket for each product. Tickets can be contained in a ticket store, whether persistently on disk or just in-memory.
Server
The URL in each ticket identifies a property list, abbreviated plist, that contains information about that product's updates. This plist contains a predicate that tells Update Engine whether or not the update is needed. It also contains the URL for the new software's disk image, the size and a hash of the new disk image, and a few other things. More information can be found at PlistServerConfiguration
Update Engine will download this plist from the specified URL, evaluate the rules, and then download and install all necessary updates.
You are free to make your own subclasses of KSServer to use any protocol with any server you want. The plist server makes it easy to drop a property list file on an internet host and use it for updates.
Installing
To install the new software, Update Engine mounts the downloaded disk image and runs a script that must be present at the disk image root. This required script is ultimately responsible for upgrading the software. This script provides a layer of abstraction that enables Update Engine to "update" virtually any type of software. More information can be found at EngineInstallScripts.