RESTful for Flask API

• 1. RESTful Web API With Python, Flask and Mongo 
• 8. gestionaleamica.com invoicing & accounting
• 10. 进入 Python Flask 和 Mongo 的学习
• 11. 那么 REST 都是关于什么的？
• 12. REST 不是一个标准，也不是一个协议
• 14. REST 是一个架构风格的网络应用程序
• 15. REST 松散地定义了一组简单的规则以及大多数API的实现
• 16. #1 资源来源的具体信息， 一个web页面而不是资源准确的说它是资源的一种表现形式
• 18. #2 全球的每个资源永久标识是唯一标识 (想一想一个HTTP URI)
• 19. #3 标准接口用来交换资源的表示 (think the HTTP protocol)
• 20. #4 一组包括了关注点,无状态,缓存能力,分层系统,统一接口的分离，稍后将会讲到
• 21. World Wide Web 是建立在 REST 之上（人们使用）
• 22. RESTful Web APIs 也是建立在 REST（计算机使用）
• 23. 可以看看我是怎样向老婆描述REST的： Ryan Tomayko http://tomayko.com/writings/rest-to-my-wife
• 24. The Real Stuff Representational State Transfer (REST) by Roy Thomas Fielding http://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.ht
• 25. RESTful Web API Nuts & Bolts
• 26. Flask & Mongo
• 27. 让我们一点一滴地来看Flask 网络开发
• 28. 高雅简单，内置开发服务器和调试器，有明确的尚可执行的对象，100%符合WSGI规则，响应对象是WSGI应用程序本身

  from flask import Flask 
  app = Flask(__name__) 
  @app.route("/") 
  def hello(): 
      return "Hello World!" 
  
  if __name__ == "__main__": 
      app.run(debug=True)
  

  　　

• 29. RESTful 请求调度 

  @app.route(/user/<username>)
  def show_user_profile(username):
     return User %s % username @app.route(/post/<int:post_id>) 
  
  def show_post(post_id):
     return Post %d % post_id

• 33.最小的Footprint只有800行源代码
• 34. Heavily Tested 1500 lines of tests
• 35. Unittesting Support one day I will make good use of it
• 36. Bring Your Own Batteries we aim for flexibility
• 37. No built-in ORM we want to be as close to the bare metal as possible
• 38. No form validation we don’t need no freaking form validation
• 39. No data validation Python offers great tools to manipulate JSON, we can tinker something ourselves
• 40. Layered API built on Werkzeug, Jinja2, WSGI
• 41. Built by the Pros The Pocoo Team did Werkzeug, Jinja2, Sphinx, Pygments, and much more
• 42. Excellent Documentation Over 200 pages, lots of examples and howtos
• 43. Active Community Widely adopted, extensions for everything
• 44. “Flask is a sharp tool for building sharp services” Kenneth Reitz, DjangoCon 2012
• 45. MongoDB scalable, high-performance, open source NoSQL database
• 46. Similarity with RDBMS made NoSQL easy to grasp (even for a dumbhead like me)
• 47. Terminology RDBMS Mongo Database Database Table Collection Rows(s) JSON Document Index Index Join Embedding & Linking
• 48. JSON-style data store true selling point for me
• 49. JSON & RESTful API GET Client Mongo JSON JSON accepted media type (BSON) maybe we can push directly to client?
• 50. JSON & RESTful API GET Client API Mongo JSON JSON/dict JSON accepted media type maps to python dict (BSON) almost.
• 51. JSON & RESTful API POST Client API Mongo JSON JSON/dict JSON maps to python dict objects (BSON) (validation layer) also works when posting (adding) items to the database
• 52. What about Queries? Queries in MongoDB are represented as JSON-style objects //

  select * from things where x=3 and y="foo" db.things.find({x: 3, y: "foo”});

• 53. JSON & RESTful API FILTERING & SORTING ?where={x: 3, y: "foo”} Client API Mongo (very) thin native parsing JSON Mongo & validation (BSON) query syntax layer
• 54. JSON all along the pipeline mapping to and from the database feels more natural
• 55. Schema-less dynamic objects allow for a painless evolution of our schema (because yes, a schema exists at any point in time)
• 56. ORM Where we’re going we don’t need ORMs.
• 57. PyMongo official Python driver all we need to interact with the database
• 58. Also in MongoDB • setup is a breeze • lightweight • fast inserts, updates and queries • excellent documentation • great support by 10gen • great community
• 59. A Great Introduction To MongoDB The Little MongoDB Book by Karl Seguin http://openmymind.net/2011/3/28/The-Little-MongoDB-Book/
• 60. Shameless Plug Il Piccolo Libro di MongoDB by Karl Seguin, traduzione di Nicola Iarocci http://nicolaiarocci.com/il-piccolo-libro-di-mongodb-edizione-italiana/
• 61. MongoDB Interactive Tutorial http://tutorial.mongly.com/tutorial/index
• 62. RESTful Web APIs are really just collection of resources accesible through to a uniform interface
• 63. #1 each resource is identified by a persistent identifier We need to properly implement Request Dispatching
• 64. Collections API’s entry point + plural nouns http://api.example.com/v1/contacts
• 65. Collections Flask URL dispatcher allows for variables

   @app.route(/<collection>) 
  
  def collection(collection):
     if collection in DOMAIN.keys():
         (...) 
     abort(404) 

  api.example.com/contacts api.example.com/invoices etc.
• 66. Collections Flask URL dispatcher allows for variables

  @app.route(/<collection>) 
  
  def collection(collection): 
      if collection in DOMAIN.keys(): 
          (...) 
      abort(404)

   validation dictonary
• 67. Collections Flask URL dispatcher allows for variables

  @app.route(/<collection>) 
  
  def collection(collection): 
      if collection in DOMAIN.keys(): 
          (...) 
      abort(404)

  we don’t know this collection, return a 404
• 68. RegEx by design, collection URLs are plural nouns @app.route(/<regex("[w]*[Ss]"):collection>) def collection(collection): if collection in DOMAIN.keys(): (...) abort(404) regular expressions can be used to better narrow a variable part URL. However...
• 69. RegEx We need to build our own Custom Converter class RegexConverter(BaseConverter): def __init__(self, url_map, *items): super(RegexConverter, self).__init__(url_map) self.regex = items[0] app.url_map.converters[regex] = RegexConverter subclass BaseConverter and pass the new converter to the url_map
• 70. Document Documents are identified by ObjectID http://api.example.com/v1/contacts/4f46445fc88e201858000000 And eventually by an alternative lookup value http://api.example.com/v1/contacts/CUST12345
• 71. Document @app.route(/<regex("[w]*[Ss]"):collection>/<lookup>) @app.route(/<regex("[w]*[Ss]"):collection> /<regex("[a-f0-9]{24}"):object_id>) def document(collection, lookup=None, object_id=None): (...) URL dispatcher handles multiple variables http://api.example.com/v1/contacts/CUST12345
• 72. Document @app.route(/<regex("[w]*[Ss]"):collection>/<lookup>) @app.route(/<regex("[w]*[Ss]"):collection> /<regex("[a-f0-9]{24}"):object_id>) def document(collection, lookup=None, object_id=None): (...) and of course it also handles multiple RegEx variables http://api.example.com/v1/contacts/4f46445fc88e201858000000
• 73. Document @app.route(/<regex("[w]*[Ss]"):collection>/<lookup>) @app.route(/<regex("[w]*[Ss]"):collection> /<regex("[a-f0-9]{24}"):object_id>) def document(collection, lookup=None, object_id=None): (...) Different URLs can be dispatched to the same function just by piling up @app.route decorators.
• 74. #2 representation of resources via media types JSON, XML or any other valid internet media type dep ends on the reque st and not the identifier
• 75. Accepted Media Types mapping supported media types to corresponding renderer functions mime_types = {json_renderer: (application/json,), xml_renderer: (application/xml, text/xml, application/x-xml,)} JSON rendering function
• 76. Accepted Media Types mapping supported media types to corresponding renderer functions mime_types = {json_renderer: (application/json,), xml_renderer: (application/xml, text/xml, application/x-xml,)} corresponding JSON internet media type
• 77. Accepted Media Types mapping supported media types to corresponding renderer functions mime_types = {json_renderer: (application/json,), xml_renderer: (application/xml, text/xml, application/x-xml,)} XML rendering function
• 78. Accepted Media Types mapping supported media types to corresponding renderer functions mime_types = {json_renderer: (application/json,), xml_renderer: (application/xml, text/xml, application/x-xml,)} corresponding XML internet media types
• 79. JSON Render datetimes and ObjectIDs call for further tinkering class APIEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, datetime.datetime): return date_to_str(obj) elif isinstance(obj, ObjectId): return str(obj) return json.JSONEncoder.default(self, obj) def json_renderer(**data): return json.dumps(data, cls=APIEncoder) renderer function mapped to the appication/json media type
• 80. JSON Render datetimes and ObjectIDs call for further tinkering class APIEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, datetime.datetime): return date_to_str(obj) elif isinstance(obj, ObjectId): return str(obj) return json.JSONEncoder.default(self, obj) def json_renderer(**data): return json.dumps(data, cls=APIEncoder) standard json encoding is not enough, we need a specialized JSONEncoder
• 81. JSON Render datetimes and ObjectIDs call for further tinkering class APIEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, datetime.datetime): return date_to_str(obj) elif isinstance(obj, ObjectId): return str(obj) return json.JSONEncoder.default(self, obj) def json_renderer(**data): return json.dumps(data, cls=APIEncoder) Python datetimes are encoded as RFC 1123 strings: “Wed, 06 Jun 2012 14:19:53 UTC”
• 82. JSON Render datetimes and ObjectIDs call for further tinkering class APIEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, datetime.datetime): return date_to_str(obj) elif isinstance(obj, ObjectId): return str(obj) return json.JSONEncoder.default(self, obj) def json_renderer(**data): return json.dumps(data, cls=APIEncoder) Mongo ObjectId data types are encoded as strings: “4f46445fc88e201858000000”
• 83. JSON Render datetimes and ObjectIDs call for further tinkering class APIEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, datetime.datetime): return date_to_str(obj) elif isinstance(obj, ObjectId): return str(obj) return json.JSONEncoder.default(self, obj) def json_renderer(**data): return json.dumps(data, cls=APIEncoder) we let json/simplejson handle the other data types
• 84. Rendering Render to JSON or XML and get WSGI response object def prep_response(dct, status=200): mime, render = get_best_mime() rendered = globals()[render](**dct) resp = make_response(rendered, status) resp.mimetype = mime return resp best match between request Accept header and media types supported by the service
• 85. Rendering Render to JSON or XML and get WSGI response object def prep_response(dct, status=200): mime, render = get_best_mime() rendered = globals()[render](**dct) resp = make_response(rendered, status) resp.mimetype = mime return resp call the appropriate render function and retrieve the encoded JSON or XML
• 86. Rendering Render to JSON or XML and get WSGI response object def prep_response(dct, status=200): mime, render = get_best_mime() rendered = globals()[render](**dct) resp = make_response(rendered, status) resp.mimetype = mime return resp flask’s make_response() returns a WSGI response object wich we can use to attach headers
• 87. Rendering Render to JSON or XML and get WSGI response object def prep_response(dct, status=200): mime, render = get_best_mime() rendered = globals()[render](**dct) resp = make_response(rendered, status) resp.mimetype = mime return resp and finally, we set the appropriate mime type in the response header
• 88. Flask-MimeRender “Python module for RESTful resource representation using MIME Media-Types and the Flask Microframework” !"!#"$%&((#)(%*+,",-.-$/-.
• 89. Flask-MimeRender Render Functions render_json = jsonify render_xml = lambda message: <message>%s</message> % message render_txt = lambda message: message render_html = lambda message: <html><body>%s</body></html> % message
• 90. Flask-MimeRender then you just decorate your end-point function @app.route(/) @mimerender( default = html, html = render_html, xml = render_xml, json = render_json, txt = render_txt ) def index(): if request.method == GET: return {message: Hello, World!}
• 91. Flask-MimeRender Requests $ curl -H "Accept: application/html" example.com/ <html><body>Hello, World!</body></html> $ curl -H "Accept: application/xml" example.com/ <message>Hello, World!</message> $ curl -H "Accept: application/json" example.com/ {message:Hello, World!} $ curl -H "Accept: text/plain" example.com/ Hello, World!
• 92. #3 resource manipulation through HTTP verbs “GET, POST, PUT, DELETE and all that mess”
• 93. HTTP Methods Verbs are handled along with URL routing @app.route(/<collection>, methods=[GET, POST]) def collection(collection): if collection in DOMAIN.keys(): if request.method == GET: return get_collection(collection) elif request.method == POST: return post(collection) abort(404) accepted HTTP verbs a PUT will throw a 405 Command Not Allowed
• 94. HTTP Methods Verbs are handled along with URL routing @app.route(/<collection>, methods=[GET, POST]) def collection(collection): if collection in DOMAIN.keys(): if request.method == GET: return get_collection(collection) elif request.method == POST: return post(collection) abort(404) the global request object provides access to clients’ request headers
• 95. HTTP Methods Verbs are handled along with URL routing @app.route(/<collection>, methods=[GET, POST]) def collection(collection): if collection in DOMAIN.keys(): if request.method == GET: return get_collection(collection) elif request.method == POST: return post(collection) abort(404) we respond to a GET request for a ‘collection’ resource
• 96. HTTP Methods Verbs are handled along with URL routing @app.route(/<collection>, methods=[GET, POST]) def collection(collection): if collection in DOMAIN.keys(): if request.method == GET: return get_collection(collection) elif request.method == POST: return post(collection) abort(404) and here we respond to a POST request. Handling HTTP methods is easy!
• 97. CRUD via REST Acttion HTTP Verb Context Collection/ Get GET Document Create POST Collection Update PATCH* Document Delete DELETE Document * WTF?
• 98. GET Retrieve Multiple Documents (accepting Queries) http://api.example.com/v1/contacts?where={“age”: {“$gt”: 20}}
• 99. Collection GET http://api.example.com/v1/contacts?where={“age”: {“$gt”: 20}} def get_collection(collection): where = request.args.get(where) if where: args[spec] = json.loads(where, object_hook=datetime_parser) (...) response = {} documents = [] cursor = db(collection).find(**args) for document in cursor: documents.append(document) response[collection] = documents request.args returns the return prep_response(response) original URI’s query definition, in our example: where = {“age”: {“$gt”: 20}}
• 100. Collection GET http://api.example.com/v1/contacts?where={“age”: {“$gt”: 20}} def get_collection(collection): where = request.args.get(where) if where: args[spec] = json.loads(where, object_hook=datetime_parser) (...) response = {} documents = [] cursor = db(collection).find(**args) for document in cursor: documents.append(document) as the query already comes in as a Mongo expression: response[collection] = documents return prep_response(response) {“age”: {“$gt”: 20}} we simply convert it to JSON.
• 101. Collection GET http://api.example.com/v1/contacts?where={“age”: {“$gt”: 20}} def get_collection(collection): where = request.args.get(where) if where: args[spec] = json.loads(where, object_hook=datetime_parser) (...) response = {} documents = [] cursor = db(collection).find(**args) for document in cursor: documents.append(document) response[collection] = documents return prep_response(response) String-to-datetime conversion is obtained via the object_hook mechanis
• 102. Collection GET http://api.example.com/v1/contacts?where={“age”: {“$gt”: 20}} def get_collection(collection): where = request.args.get(where) if where: args[spec] = json.loads(where, object_hook=datetime_parser) (...) response = {} documents = [] cursor = db(collection).find(**args) for document in cursor: documents.append(document) response[collection] = documents find() accepts a python dict return prep_response(response) as query expression, and returns a cursor we can iterate
• 103. Collection GET http://api.example.com/v1/contacts?where={“age”: {“$gt”: 20}} def get_collection(collection): where = request.args.get(where) if where: args[spec] = json.loads(where, object_hook=datetime_parser) (...) response = {} documents = [] cursor = db(collection).find(**args) for document in cursor: documents.append(document) response[collection] = documents return prep_response(response) finally, we encode the response dict with the requested MIME media-type
• 104. Interlude On encoding JSON dates
• 105. On encoding JSON dates • We don’t want to force metadata into JSON representation: (“updated”: “$date: Thu 1, ..”) • Likewise, epochs are not an option • We are aiming for a broad solution not relying on the knoweldge of the current domain
• 106. uy e g ind th eh is b ed R Because, you know
• 107. Parsing JSON dates this is what I came out with >>> source = {"updated": "Thu, 1 Mar 2012 10:00:49 UTC"} >>> dct = json.loads(source, object_hook=datetime_parser) >>> dct {uupdated: datetime.datetime(2012, 3, 1, 10, 0, 49)} def datetime_parser(dct):     for k, v in dct.items():         if isinstance(v, basestring) and re.search(" UTC", v):             try:                 dct[k] = datetime.datetime.strptime(v, DATE_FORMAT)             except:                 pass     return dct object_hook is usually used to deserialize JSON to classes (rings a ORM bell?)
• 108. Parsing JSON dates this is what I came out with >>> source = {"updated": "Thu, 1 Mar 2012 10:00:49 UTC"} >>> dct = json.loads(source, object_hook=datetime_parser) >>> dct {uupdated: datetime.datetime(2012, 3, 1, 10, 0, 49)} def datetime_parser(dct):     for k, v in dct.items():         if isinstance(v, basestring) and re.search(" UTC", v):             try:                 dct[k] = datetime.datetime.strptime(v, DATE_FORMAT)             except:                 pass     return dct the resulting dct now has datetime values instead of string representations of dates
• 109. Parsing JSON dates this is what I came out with >>> source = {"updated": "Thu, 1 Mar 2012 10:00:49 UTC"} >>> dct = json.loads(source, object_hook=datetime_parser) >>> dct {uupdated: datetime.datetime(2012, 3, 1, 10, 0, 49)} def datetime_parser(dct):     for k, v in dct.items():         if isinstance(v, basestring) and re.search(" UTC", v):             try:                 dct[k] = datetime.datetime.strptime(v, DATE_FORMAT)             except:                 pass     return dct the function receives a dict representing the decoded JSON
• 110. Parsing JSON dates this is what I came out with >>> source = {"updated": "Thu, 1 Mar 2012 10:00:49 UTC"} >>> dct = json.loads(source, object_hook=datetime_parser) >>> dct {uupdated: datetime.datetime(2012, 3, 1, 10, 0, 49)} def datetime_parser(dct):     for k, v in dct.items():         if isinstance(v, basestring) and re.search(" UTC", v):             try:                 dct[k] = datetime.datetime.strptime(v, DATE_FORMAT)             except:                 pass     return dct strings matching the RegEx (which probably should be better defined)...
• 111. Parsing JSON dates this is what I came out with >>> source = {"updated": "Thu, 1 Mar 2012 10:00:49 UTC"} >>> dct = json.loads(source, object_hook=datetime_parser) >>> dct {uupdated: datetime.datetime(2012, 3, 1, 10, 0, 49)} def datetime_parser(dct):     for k, v in dct.items():         if isinstance(v, basestring) and re.search(" UTC", v):             try:                 dct[k] = datetime.datetime.strptime(v, DATE_FORMAT)             except:                 pass     return dct ...are converted to datetime values
• 112. Parsing JSON dates this is what I came out with >>> source = {"updated": "Thu, 1 Mar 2012 10:00:49 UTC"} >>> dct = json.loads(source, object_hook=datetime_parser) >>> dct {uupdated: datetime.datetime(2012, 3, 1, 10, 0, 49)} def datetime_parser(dct):     for k, v in dct.items():         if isinstance(v, basestring) and re.search(" UTC", v):             try:                 dct[k] = datetime.datetime.strptime(v, DATE_FORMAT)             except:                 pass     return dct if conversion fails we assume that we are dealing a normal, legit string
• 113. PATCH Editing a Resource
• 114. Why not PUT? • PUT means resource creation or replacement at a given URL • PUT does not allow for partial updates of a resource • 99% of the time we are updating just one or two fields • We don’t want to send complete representations of the document we are updating • Mongo allows for atomic updates and we want to take advantage of that
• 115. ‘atomic’ PUT updates are ok when each field is itself a resource http://api.example.com/v1/contacts/<id>/address
• 116. Enter PATCH “This specification defines the new method, PATCH, which is used to apply partial modifications to a resource.” RFC5789
• 117. PATCH • send a “patch document” with just the changes to be applied to the document • saves bandwidth and reduces traffic • it’s been around since 1995 • it is a RFC Proposed Standard • Widely adopted (will replace PUT in Rails 4.0) • clients not supporting it can fallback to POST with ‘X-HTTP-Method-Override: PATCH’ header tag
• 118. PATCHing def patch_document(collection, original): docs = parse_request(request.form) if len(docs) > 1: abort(400) request.form returns a dict with request form data. key, value = docs.popitem() response_item = {} object_id = original[ID_FIELD] # Validation validate(value, collection, object_id) response_item[validation] = value[validation] if value[validation][response] != VALIDATION_ERROR: # Perform the update updates = {"$set": value[doc]} db(collection).update({"_Id": ObjectId(object_id)}, updates) response_item[ID_FIELD] = object_id return prep_response(response_item)
• 119. PATCHing def patch_document(collection, original): docs = parse_request(request.form) if len(docs) > 1: abort(400) we aren’t going to accept more than one document here key, value = docs.popitem() response_item = {} object_id = original[ID_FIELD] # Validation validate(value, collection, object_id) response_item[validation] = value[validation] if value[validation][response] != VALIDATION_ERROR: # Perform the update updates = {"$set": value[doc]} db(collection).update({"_Id": ObjectId(object_id)}, updates) response_item[ID_FIELD] = object_id return prep_response(response_item)
• 120. PATCHing def patch_document(collection, original): docs = parse_request(request.form) if len(docs) > 1: retrieve the original abort(400) document ID, will be used by key, value = docs.popitem() the update command response_item = {} object_id = original[ID_FIELD] # Validation validate(value, collection, object_id) response_item[validation] = value[validation] if value[validation][response] != VALIDATION_ERROR: # Perform the update updates = {"$set": value[doc]} db(collection).update({"_Id": ObjectId(object_id)}, updates) response_item[ID_FIELD] = object_id return prep_response(response_item)
• 121. PATCHing def patch_document(collection, original): docs = parse_request(request.form) if len(docs) > 1: abort(400) validate the updates key, value = docs.popitem() response_item = {} object_id = original[ID_FIELD] # Validation validate(value, collection, object_id) response_item[validation] = value[validation] if value[validation][response] != VALIDATION_ERROR: # Perform the update updates = {"$set": value[doc]} db(collection).update({"_Id": ObjectId(object_id)}, updates) response_item[ID_FIELD] = object_id return prep_response(response_item)
• 122. PATCHing def patch_document(collection, original): docs = parse_request(request.form) if len(docs) > 1: abort(400) add validation results to the response dictionary key, value = docs.popitem() response_item = {} object_id = original[ID_FIELD] # Validation validate(value, collection, object_id) response_item[validation] = value[validation] if value[validation][response] != VALIDATION_ERROR: # Perform the update updates = {"$set": value[doc]} db(collection).update({"_Id": ObjectId(object_id)}, updates) response_item[ID_FIELD] = object_id return prep_response(response_item)
• 123. PATCHing def patch_document(collection, original): docs = parse_request(request.form) $set accepts a dict if len(docs) > 1: abort(400) with the updates for the db eg: {“active”: False}. key, value = docs.popitem() response_item = {} object_id = original[ID_FIELD] # Validation validate(value, collection, object_id) response_item[validation] = value[validation] if value[validation][response] != VALIDATION_ERROR: # Perform the update updates = {"$set": value[doc]} db(collection).update({"_Id": ObjectId(object_id)}, updates) response_item[ID_FIELD] = object_id return prep_response(response_item)
• 124. PATCHing def patch_document(collection, original): docs = parse_request(request.form) mongo update() method if len(docs) > 1: commits updates to the abort(400) database. Updates are key, value = docs.popitem() atomic. response_item = {} object_id = original[ID_FIELD] # Validation validate(value, collection, object_id) response_item[validation] = value[validation] if value[validation][response] != VALIDATION_ERROR: # Perform the update updates = {"$set": value[doc]} db(collection).update({"_Id": ObjectId(object_id)}, updates) response_item[ID_FIELD] = object_id return prep_response(response_item)
• 125. PATCHing def patch_document(collection, original): docs = parse_request(request.form) if len(docs) > 1: udpate() takes the unique Id abort(400) of the document andthe update expression ($set) key, value = docs.popitem() response_item = {} object_id = original[ID_FIELD] # Validation validate(value, collection, object_id) response_item[validation] = value[validation] if value[validation][response] != VALIDATION_ERROR: # Perform the update updates = {"$set": value[doc]} db(collection).update({"_Id": ObjectId(object_id)}, updates) response_item[ID_FIELD] = object_id return prep_response(response_item)
• 126. PATCHing def patch_document(collection, original): docs = parse_request(request.form) if len(docs) > 1: as always, our response abort(400) dictionary is returned with proper encding key, value = docs.popitem() response_item = {} object_id = original[ID_FIELD] # Validation validate(value, collection, object_id) response_item[validation] = value[validation] if value[validation][response] != VALIDATION_ERROR: # Perform the update updates = {"$set": value[doc]} db(collection).update({"_Id": ObjectId(object_id)}, updates) response_item[ID_FIELD] = object_id return prep_response(response_item)
• 127. POST Creating Resources
• 128. POSTing def post(collection): docs = parse_request(request.form) response = {} for key, item in docs.items(): response_item = {} validate(item, collection) if item[validation][response] != VALIDATION_ERROR: document = item[doc] response_item[ID_FIELD] = db(collection).insert(document) response_item[link] = get_document_link(collection, response_item[ID_FIELD]) response_item[validation] = item[validation] response[key] = response_item return {response: response} we accept multiple documents (remember, we are at collection level here)
• 129. POSTing def post(collection): docs = parse_request(request.form) response = {} for key, item in docs.items(): response_item = {} validate(item, collection) if item[validation][response] != VALIDATION_ERROR: document = item[doc] response_item[ID_FIELD] = db(collection).insert(document) response_item[link] = get_document_link(collection, response_item[ID_FIELD]) response_item[validation] = item[validation] response[key] = response_item return {response: response} we loop through the documents to be inserted
• 130. POSTing def post(collection): docs = parse_request(request.form) response = {} for key, item in docs.items(): response_item = {} validate(item, collection) if item[validation][response] != VALIDATION_ERROR: document = item[doc] response_item[ID_FIELD] = db(collection).insert(document) response_item[link] = get_document_link(collection, response_item[ID_FIELD]) response_item[validation] = item[validation] response[key] = response_item return {response: response} perform validation on the document
• 131. POSTing def post(collection): docs = parse_request(request.form) response = {} for key, item in docs.items(): response_item = {} validate(item, collection) if item[validation][response] != VALIDATION_ERROR: document = item[doc] response_item[ID_FIELD] = db(collection).insert(document) response_item[link] = get_document_link(collection, response_item[ID_FIELD]) response_item[validation] = item[validation] response[key] = response_item return {response: response} push document and get its ObjectId back from Mongo. like other CRUD operations, inserting is trivial in mongo.
• 132. POSTing def post(collection): docs = parse_request(request.form) response = {} for key, item in docs.items(): response_item = {} validate(item, collection) if item[validation][response] != VALIDATION_ERROR: document = item[doc] response_item[ID_FIELD] = db(collection).insert(document) response_item[link] = get_document_link(collection, response_item[ID_FIELD]) response_item[validation] = item[validation] response[key] = response_item return {response: response} a direct link to the resource we just created is added to the response
• 133. POSTing def post(collection): docs = parse_request(request.form) response = {} for key, item in docs.items(): response_item = {} validate(item, collection) if item[validation][response] != VALIDATION_ERROR: document = item[doc] response_item[ID_FIELD] = db(collection).insert(document) response_item[link] = get_document_link(collection, response_item[ID_FIELD]) response_item[validation] = item[validation] response[key] = response_item return {response: response} validation result is always returned to the client, even if the doc has not been inserted
• 134. POSTing def post(collection): docs = parse_request(request.form) response = {} for key, item in docs.items(): response_item = {} validate(item, collection) if item[validation][response] != VALIDATION_ERROR: document = item[doc] response_item[ID_FIELD] = db(collection).insert(document) response_item[link] = get_document_link(collection, response_item[ID_FIELD]) response_item[validation] = item[validation] response[key] = response_item return {response: response} standard response enconding applied
• 135. Data Validation We still need to validate incoming data
• 136. Data Validation DOMAIN = {} DOMAIN[contacts] = { secondary_id: name, fields: { name: { data_type: string, required: True, unique: True, max_length: 120, min_length: 1 }, DOMAIN is a Python dict containing our validation rules and schema structure
• 137. Data Validation DOMAIN = {} DOMAIN[contacts] = { secondary_id: name, fields: { name: { data_type: string, required: True, unique: True, max_length: 120, min_length: 1 }, every resource (collection) maintained by the API has a key in DOMAIN
• 138. Data Validation DOMAIN = {} DOMAIN[contacts] = { secondary_id: name, fields: { name: { data_type: string, required: True, unique: True, max_length: 120, min_length: 1 }, if the resource allows for a secondary lookup field, we define it here
• 139. Data Validation DOMAIN = {} DOMAIN[contacts] = { secondary_id: name, fields: { name: { data_type: string, required: True, unique: True, max_length: 120, min_length: 1 }, known fields go in the fields dict
• 140. Data Validation DOMAIN = {} DOMAIN[contacts] = { secondary_id: name, fields: { name: { data_type: string, required: True, unique: True, max_length: 120, min_length: 1 }, validation rules for ‘name’ field. data_type is mostly needed to process datetimes and currency values
• 141. Data Validation (...) iban: { data_type: string, custom_validation: { module: customvalidation, function: validate_iban } } (...) we can define custom validation functions when the need arises
• 142. Data Validation (...) contact_type: { data_type: array, allowed_values: [ client, agent, supplier, area manager, vector ] } (...) or we can define our own custom data types...
• 143. Data Validation (...) contact_type: { data_type: array, allowed_values: [ client, agent, supplier, area manager, vector ] } (...) ... like the array, which allows us to define a list of accepted values for the field
• 144. I will spare you the validation function It’s pretty simple really
• 145. Hey but! You’re building your own ORM! Just a thin validation layer on which I have total control AKA So What?
• 146. #4 Caching and concurrency control resource representation describes how when and if it can be used, discarded or re-fetched
• 147. Driving conditional requests Servers use Last-Modified and ETag response headers to drive conditional requests
• 148. Last-Modified Generally considered a weak validator since it has a one-second resolution “Wed, 06 Jun 2012 14:19:53 UTC”
• 149. ETag Entity Tag is a strong validator since its value can be changed every time the server modifies the representation 7a9f477cde424cf93a7db20b69e05f7b680b7f08
• 150. On ETags • Clients should be able to use ETag to compare representations of a resouce • An ETag is supposed to be like an object’s hash code. • Actually, some web frameworks and a lot of implementations do just that • ETag computed on an entire representation of the resource may become a performance bottleneck
• 151. Last-Modified or ETag? You can use either or both. Consider the types of client consuming your service. Hint: use both.
• 152. Validating cached representations Clients use If-Modified-Since and If-None-Match in request headers for validating cached representations
• 153. If-Mod-Since & ETag def get_document(collection, object_id=None, lookup=None): response = {} document = find_document(collection, object_id, lookup) if document: etag = get_etag(document) header_etag = request.headers.get(If-None-Match) if header_etag and header_etag == etag: return prep_response(dict(), status=304) if_modified_since = request.headers.get(If-Modified-Since) if if_modified_since: last_modified = document[LAST_UPDATED] if last_modified <= if_modified_since: return prep_response(dict(), status=304) response[collection.rstrip(s)] = document return prep_response(response, last_modified, etag) abort(404) retrieve the document from the database
• 154. If-Mod-Since & ETag def get_document(collection, object_id=None, lookup=None): response = {} document = find_document(collection, object_id, lookup) if document: etag = get_etag(document) header_etag = request.headers.get(If-None-Match) if header_etag and header_etag == etag: return prep_response(dict(), status=304) if_modified_since = request.headers.get(If-Modified-Since) if if_modified_since: last_modified = document[LAST_UPDATED] if last_modified <= if_modified_since: return prep_response(dict(), status=304) response[collection.rstrip(s)] compute ETag for the current = document return prep_response(response, last_modified, etag) representation. We test ETag abort(404) first, as it is a stronger validator
• 155. If-Mod-Since & ETag def get_document(collection, object_id=None, lookup=None): response = {} document = find_document(collection, object_id, lookup) if document: etag = get_etag(document) header_etag = request.headers.get(If-None-Match) if header_etag and header_etag == etag: return prep_response(dict(), status=304) if_modified_since = request.headers.get(If-Modified-Since) if if_modified_since: last_modified = document[LAST_UPDATED] if last_modified <= if_modified_since: return prep_response(dict(), status=304) response[collection.rstrip(s)] = document return prep_response(response, last_modified, etag) abort(404) retrieve If-None-Match ETag from request header
• 156. If-Mod-Since & ETag def get_document(collection, object_id=None, lookup=None): response = {} document = find_document(collection, object_id, lookup) if document: etag = get_etag(document) header_etag = request.headers.get(If-None-Match) if header_etag and header_etag == etag: return prep_response(dict(), status=304) if_modified_since = request.headers.get(If-Modified-Since) if if_modified_since: last_modified = document[LAST_UPDATED] if last_modified <= if_modified_since: return prep_response(dict(), status=304) response[collection.rstrip(s)] = document return prep_response(response, last_modified, etag) abort(404) if client and server representations match, return a 304 Not Modified
• 157. If-Mod-Since & ETag def get_document(collection, object_id=None, lookup=None): response = {} document = find_document(collection, object_id, lookup) if document: etag = get_etag(document) header_etag = request.headers.get(If-None-Match) if header_etag and header_etag == etag: return prep_response(dict(), status=304) if_modified_since = request.headers.get(If-Modified-Since) if if_modified_since: last_modified = document[LAST_UPDATED] if last_modified <= if_modified_since: return prep_response(dict(), status=304) response[collection.rstrip(s)] = document return prep_response(response, last_modified, if the resource likewise, etag) abort(404) has not been modified since If-Modifed-Since, return 304 Not Modified
• 158. Concurrency control Clients use If-Unmodified-Since and If-Match in request headers as preconditions for concurrency control
• 159. Concurrency control Create/Update/Delete are controlled by ETag def edit_document(collection, object_id, method): document = find_document(collection, object_id) if document: header_etag = request.headers.get(If-Match) if header_etag is None: return prep_response(If-Match missing from request header, status=403) if header_etag != get_etag(document[LAST_UPDATED]): # Precondition failed abort(412) else: if method in (PATCH, POST): return patch_document(collection, document) elif method == DELETE: return delete_document(collection, object_id) else: abort(404) retrieve client’s If-Match ETag from the request header
• 160. Concurrency control Create/Update/Delete are controlled by ETag def edit_document(collection, object_id, method): document = find_document(collection, object_id) if document: header_etag = request.headers.get(If-Match) if header_etag is None: return prep_response(If-Match missing from request header, status=403) if header_etag != get_etag(document[LAST_UPDATED]): # Precondition failed abort(412) else: if method in (PATCH, POST): return patch_document(collection, document) elif method == DELETE: return delete_document(collection, object_id) else: abort(404) editing is forbidden if ETag is not provided
• 161. Concurrency control Create/Update/Delete are controlled by ETag def edit_document(collection, object_id, method): document = find_document(collection, object_id) if document: header_etag = request.headers.get(If-Match) if header_etag is None: return prep_response(If-Match missing from request header, status=403) if header_etag != get_etag(document[LAST_UPDATED]): # Precondition failed abort(412) else: if method in (PATCH, POST): return patch_document(collection, document) elif method == DELETE: return delete_document(collection, object_id) else: client and server abort(404) representations don’t match. Precondition failed.
• 162. Concurrency control Create/Update/Delete are controlled by ETag def edit_document(collection, object_id, method): document = find_document(collection, object_id) if document: header_etag = request.headers.get(If-Match) if header_etag is None: client and server representation match, return prep_response(If-Match missing from request header, status=403) go ahead with the edit if header_etag != get_etag(document[LAST_UPDATED]): # Precondition failed abort(412) else: if method in (PATCH, POST): return patch_document(collection, document) elif method == DELETE: return delete_document(collection, object_id) else: abort(404)
• 163. Sending cache & concurrency directives back to clients
• 164. Cache & Concurrency def prep_response(dct, last_modified=None, etag=None, status=200): (...) resp.headers.add(Cache-Control, max-age=%s,must-revalidate & 30) resp.expires = time.time() + 30 if etag: resp.headers.add(ETag, etag) if last_modified: resp.headers.add(Last-Modified, date_to_str(last_modified)) return resp encodes ‘dct’ according to client’s accepted MIME Data-Type (click here see that slide)
• 165. Cache & Concurrency def prep_response(dct, last_modified=None, etag=None, status=200): (...) resp.headers.add(Cache-Control, max-age=%s,must-revalidate & 30) resp.expires = time.time() + 30 if etag: resp.headers.add(ETag, etag) if last_modified: resp.headers.add(Last-Modified, date_to_str(last_modified)) return resp Cache-Control, a directive for HTTP/1.1 clients (and later) -RFC2616
• 166. Cache & Concurrency def prep_response(dct, last_modified=None, etag=None, status=200): (...) resp.headers.add(Cache-Control, max-age=%s,must-revalidate & 30) resp.expires = time.time() + 30 if etag: resp.headers.add(ETag, etag) if last_modified: resp.headers.add(Last-Modified, date_to_str(last_modified)) return resp Expires, a directive for HTTP/1.0 clients
• 167. Cache & Concurrency def prep_response(dct, last_modified=None, etag=None, status=200): (...) resp.headers.add(Cache-Control, max-age=%s,must-revalidate & 30) resp.expires = time.time() + 30 if etag: resp.headers.add(ETag, etag) if last_modified: resp.headers.add(Last-Modified, date_to_str(last_modified)) return resp ETag. Notice that we don’t compute it on the rendered representation, this is by design.
• 168. Cache & Concurrency def prep_response(dct, last_modified=None, etag=None, status=200): (...) resp.headers.add(Cache-Control, max-age=%s,must-revalidate & 30) resp.expires = time.time() + 30 if etag: resp.headers.add(ETag, etag) if last_modified: resp.headers.add(Last-Modified, date_to_str(last_modified)) return resp And finally, we add the Last-Modified header tag.
• 169. Cache & Concurrency def prep_response(dct, last_modified=None, etag=None, status=200): (...) resp.headers.add(Cache-Control, max-age=%s,must-revalidate & 30) resp.expires = time.time() + 30 if etag: resp.headers.add(ETag, etag) if last_modified: resp.headers.add(Last-Modified, date_to_str(last_modified)) return resp the response object is now complete and ready to be returned to the client
• 170. that ’s o long ne acro ass nym #5 HATEOAS “Hypertext As The Engine Of Application State”
• 171. HATEOAS in a Nutshell • clients interact entirely through hypermedia provided dynamically by the server • clients need no prior knowledge about how to interact with the server • clients access an application through a single well known URL (the entry point) • All future actions the clients may take are discovered within resource representations returned from the server
• 172. It’s all about Links resource representation includes links to related resources
• 173. Collection { Representation  "links":[     "<link rel=parent title=home href=http://api.example.com/ />",     "<link rel=collection title=contacts href=http://api.example.com/Contacts />",     "<link rel=next title=next page  href=http://api.example.com/Contacts?page=2 />"    ],    "contacts":[       {          "updated":"Wed, 06 Jun 2012 14:19:53 UTC",          "name":"Jon Doe",          "age": 27,          "etag":"7a9f477cde424cf93a7db20b69e05f7b680b7f08",          "link":"<link rel=self title=Contact every resource href=http://api.example.com/Contacts/ 4f46445fc88e201858000000 />", representation provides a          "_id":"4f46445fc88e201858000000", links section with       }, navigational info for ] clients }
• 174. Collection { Representation  "links":[     "<link rel=parent title=home href=http://api.example.com/ />",     "<link rel=collection title=contacts href=http://api.example.com/Contacts />",     "<link rel=next title=next page  href=http://api.example.com/Contacts?page=2 />"    ],    "contacts":[       {          "updated":"Wed, 06 Jun 2012 14:19:53 UTC",          "name":"Jon Doe",          "age": 27,          "etag":"7a9f477cde424cf93a7db20b69e05f7b680b7f08",          "link":"<link rel=self title=Contact href=http://api.example.com/Contacts/ attribute provides the rel 4f46445fc88e201858000000 />", the relationship between the          "_id":"4f46445fc88e201858000000",       }, linked resource and the one ] currently represented }
• 175. Collection { Representation  "links":[     "<link rel=parent title=home href=http://api.example.com/ />",     "<link rel=collection title=contacts href=http://api.example.com/Contacts />",     "<link rel=next title=next page  href=http://api.example.com/Contacts?page=2 />"    ],    "contacts":[       {          "updated":"Wed, 06 Jun 2012 14:19:53 UTC",          "name":"Jon Doe",          "age": 27,          "etag":"7a9f477cde424cf93a7db20b69e05f7b680b7f08",          "link":"<link rel=self title=Contact the title attribute provides href=http://api.example.com/Contacts/ 4f46445fc88e201858000000 />", a tag (or description) for          "_id":"4f46445fc88e201858000000", the linked resource. Could       }, be used as a caption for a ] client button. }
• 176. Collection { Representation  "links":[     "<link rel=parent title=home href=http://api.example.com/ />",     "<link rel=collection title=contacts href=http://api.example.com/Contacts />",     "<link rel=next title=next page  href=http://api.example.com/Contacts?page=2 />"    ],    "contacts":[       {          "updated":"Wed, 06 Jun 2012 14:19:53 UTC",          "name":"Jon Doe",          "age": 27,          "etag":"7a9f477cde424cf93a7db20b69e05f7b680b7f08",          "link":"<link rel=self title=Contact href=http://api.example.com/Contacts/ attribute provides the href 4f46445fc88e201858000000 />", and absolute path to the          "_id":"4f46445fc88e201858000000",       }, resource (the “permanent ] identifier” per REST def.) }
• 177. Collection { Representation  "links":[     "<link rel=parent title=home href=http://api.example.com/ />",     "<link rel=collection title=contacts every resource listed href=http://api.example.com/Contacts />", exposes its own link, which     "<link rel=next title=next page  will allow the client to href=http://api.example.com/Contacts?page=2 />"    ], perform PATCH, DELETE etc.    "contacts":[ on the resource       {          "updated":"Wed, 06 Jun 2012 14:19:53 UTC",          "name":"Jon Doe",          "age": 27,          "etag":"7a9f477cde424cf93a7db20b69e05f7b680b7f08",          "link":"<link rel=self title=Contact href=http://api.example.com/Contacts/ 4f46445fc88e201858000000 />",          "_id":"4f46445fc88e201858000000",       }, ] }
• 178. Collection { Representation  "links":[ while we are here,     "<link rel=parent title=home href=http://api.example.com/ />",     "<link rel=collection title=contacts notice how every resource href=http://api.example.com/Contacts />", its own etag, also exposes     "<link rel=next title=next page  last-modified date. href=http://api.example.com/Contacts?page=2 />"    ],    "contacts":[       {          "updated":"Wed, 06 Jun 2012 14:19:53 UTC",          "name":"Jon Doe",          "age": 27,          "etag":"7a9f477cde424cf93a7db20b69e05f7b680b7f08",          "link":"<link rel=self title=Contact href=http://api.example.com/Contacts/ 4f46445fc88e201858000000 />",          "_id":"4f46445fc88e201858000000",       }, ] }
• 179. HATEOAS The API entry point (the homepage) @app.route(/, methods=[GET]) def home(): response = {} links = [] for collection in DOMAIN.keys(): links.append("<link rel=child title=%(name)s" "href=%(collectionURI)s />" % {name: collection, collectionURI: collection_URI(collection)}) response[links] = links return response the API homepage responds to GET requests and provides links to its top level resources to the clients
• 180. HATEOAS The API entry point (the homepage) @app.route(/, methods=[GET]) def home(): response = {} links = [] for collection in DOMAIN.keys(): links.append("<link rel=child title=%(name)s" "href=%(collectionURI)s />" % {name: collection, collectionURI: collection_URI(collection)}) response[links] = links return response for every collection of resources...
• 181. HATEOAS The API entry point (the homepage) @app.route(/, methods=[GET]) def home(): response = {} links = [] for collection in DOMAIN.keys(): links.append("<link rel=child title=%(name)s" "href=%(collectionURI)s />" % {name: collection, collectionURI: collection_URI(collection)}) response[links] = links return response ... provide relation, title and link, or the persistent identifier
• 182. Wanna see it running? Hopefully it won’t explode right into my face
• 183. Only complaint I have with Flask so far... Most recent HTTP methods not supported
• 184. 508 NOT MY FAULT Not supported
• 185. 208 WORKS FOR ME Not supported
• 186. en Just kidding! ev ! ’t ke sn jo ti y i
• 187. Open Source it? as a Flask extension maybe?
• 188. Web Resources • Richardson Maturity Model: steps toward the glory of REST by Richard Flowers • RESTful Service Best Practices by Todd Fredrich • What Exactly(lotsRESTful Programming? StackOverflow is of resources) • API Anti-Patterns: How to Avoid Common REST Mistakes by Tomas Vitvar
• 189. Excellent Books
• 190. Excellent Books I’m getting a cut. ish! Iw
• 191. Thank you. @nicolaiarocci