Valery Kholodkov <valery@grid.net.ru>, 2010
1. Introduction
This document describes application protocol that is used by nginx upload module to implement resumable file uploads. The first version of the module that supports this protocol is 2.2.0.
2. Purpose
The HTTP implements file uploads according to RFC 1867. When the request length is excessively large, the probability that connection will be interrupted is high. HTTP does not foresee a resumption mechanism. The goal of the protocol being described is to implement a mechanism of resumption of interrupted file transfer or suspension of upload upon user request.
2.1. Splitting file into segments
When TCP-connection interrupts abnormaly there is no way to determine what part of data stream has been succesfully delivered and what hasn't been delivered. Therefore a client cannot determine what position to resume from without communicating to server. In order to eliminate additional communication file is represented as an array of segments of reasonable length. When TCP-connection interrupts while transmitting certain segment, client retransmits the whole segment until a positive reponse will be received from server or maximal number of tries will be reached. In the protocol being described the client is responsible for choosing optimal length of a segment.
For tracking the progress of file upload client and server use identical numbering scheme for each byte of a file. The first byte of a file has number 0, the last byte has number n-1, where n is the length of file in bytes.
The order of transmission of a segment is not defined. Client may choose arbitrary order. However it is recommended to send segments in order ascention of byte numbers. Moreover, a user agent might decide to send multiple segments simultaneously using multiple independent connections. If a client exceeds maximal number of simultaneous connections allowed, server might return 503 "Service Unavailable" response.
In case of simultaneous transmission it is prohibited to send 2 or more requests with overlapping ranges within one session. Whenever server detects simultaneous requests with overlapping ranges it must return an errorneous response.
2.2. Encapsulation
Each segment of a file is encapsulated into a separate HTTP-request. The method of the request is POST. Each request contains following specific headers:
| Header name | Function |
|---|---|
| Content-Disposition | attachment, filename="name of the file being uploaded" |
| Content-Type | mime type of a file being uploaded (must not be multipart/form-data); |
| X-Content-Range or Content-Range |
byte range of a segment being uploaded; |
| X-Session-ID or Session-ID |
identifier of a session of a file being uploaded (see 2.3); |
The body of the request must contain a segment of the file, corresponding to the range that was specified in X-Content-Range or Content-Range headers.
Whenever a user agent is not able to determine mime type of a file, it may use application/octet-stream.
2.3. Session management
In order to identify requests containing segments of a file, a user agent sends a unique session identified in headers X-Session-ID or Session-ID. User agent is responsible for making session identifiers unique. Server must be ready to process requests from different IP-addresses corresponding to a single session.
2.4. Acknowledgment
Server acknowledges reception of each segment with a positive response. Positive responses are: 201 "Created" whenever at the moment of the response generation not all segments of the file were received or other 2xx and 3xx responses whenever at the moment of the response generation all segments of the file were received. Server must return positive response only when all bytes of a segment were successfully saved and information about which of the byte ranges were received was successfully updated.
Upon reception of 201 "Created" response client must proceed with transmission of a next segment. Upon reception of other positive response codes client must proceed according to their standart interpretation (see. RFC 2616).
In each 201 "Created" response server returns a Range header containing enumeration of all byte ranges of a file that were received at the moment of the response generation. Server returns identical list of ranges in response body.
Appendix A: Session examples
Example 1: Request from client containing the first segment of the file
POST /upload HTTP/1.1 Host: example.com Content-Length: 51201 Content-Type: application/octet-stream Content-Disposition: attachment; filename="big.TXT" X-Content-Range: bytes 0-51200/511920 Session-ID: 1111215056 <bytes 0-51200>
Example 2: Response to a request containing first segment of a file
HTTP/1.1 201 Created Date: Thu, 02 Sep 2010 12:54:40 GMT Content-Length: 14 Connection: close Range: 0-51200/511920 0-51200/511920
Example 3: Request from client containing the last segment of the file
POST /upload HTTP/1.1 Host: example.com Content-Length: 51111 Content-Type: application/octet-stream Content-Disposition: attachment; filename="big.TXT" X-Content-Range: bytes 460809-511919/511920 Session-ID: 1111215056 <bytes 460809-511919>
Example 4: Response to a request containing last segment of a file
HTTP/1.1 200 OK Date: Thu, 02 Sep 2010 12:54:43 GMT Content-Type: text/html Connection: close Content-Length: 2270 <response body>