marshmallow基本使用

• What is marshmallow

marshmallow（Object serialization and deserialization, lightweight and fluffy.）用于对对象进行序列化和反序列化，并同步进行数据验证。

• What is schema

对对象进行序列化和反序列化需要一个中间载体，schema就是这个中间载体。

• Declaring Schemas

申明一个常见的模型类（Model）

import datetime as dt

class User(object):
    def __init__(self, name, email):
        self.name = name
        self.email = email
        self.created_at = dt.datetime.now()

    def __repr__(self):
        return '<User(name={self.name!r})>'.format(self=self)

给模型User申明一个对应的schema

from marshmallow import Schema, fields

class UserSchema(Schema):
    name = fields.Str()
    email = fields.Email()
    created_at = fields.DateTime()

或者使用from_dict从一个字典申明一个schema

from marshmallow import Schema, fields

UserSchema = Schema.from_dict(
    {
        "name": fields.Str(),
        "email": fields.Email(),
        "created_at": fields.DateTime()}
)

• Serializing Objects ("Dumping")

序列化对象，使用dump方法

from pprint import pprint

user = User(name="Monty", email="monty@python.org")
schema = UserSchema()
result = schema.dump(user)
pprint(result)

{'created_at': '2019-11-13T15:36:16.952377',
 'email': 'monty@python.org',
 'name': 'Monty'}

使用dumps方法，则转换成字符串

user = User(name="Monty", email="monty@python.org")
schema = UserSchema()
result = schema.dumps(user)
pprint(result)

('{"created_at": "2019-11-13T15:38:44.327872", "name": "Monty", "email": '
 '"monty@python.org"}')

• Filtering Output

过滤输出，如果只想输出特定字段，可以使用only或者exclude参数

user = User(name="Monty", email="monty@python.org")
schema = UserSchema(only=('name', ))
result = schema.dump(user)
pprint(result)

{'name': 'Monty'}

• Deserializing Objects （“Loading”）

默认情况下，反序列化的load方法会返回一个字典（没有触发ValidationError的情况下）

user_data = {
    'created_at': '2014-08-11T05:26:03.869245',
    'name': 'Ken',
    'email': 'ken@yahoo.com'
}

schema = UserSchema()
result = schema.load(user_data)
pprint(result)

{'created_at': datetime.datetime(2014, 8, 11, 5, 26, 3, 869245),
 'email': 'ken@yahoo.com',
 'name': 'Ken'}

为了反序列化成一个对象，需要给schema的一个方法加上post_load装饰器

from marshmallow import Schema, fields, post_load

class UserSchema(Schema):
    name = fields.Str()
    email = fields.Email()
    created_at = fields.DateTime()

    @post_load
    def make_user(self, data, **kwargs):
        return User(**data)

现在使用load方法就可以返回一个对象了

user_data = {
    'name': 'Ken',
    'email': 'ken@yahoo.com'
}

schema = UserSchema()
result = schema.load(user_data)
pprint(result)

<User(name='Ken')>

• Handling Collections of Objects

设置many=True可以一次处理一个集合的对象

user1 = User(name='Mick', email='mick@stones.com')
user2 = User(name='Keith', email='keith@stones.com')
users = [user1, user2]
schema = UserSchema(many=True) 
result = schema.dump(users) # OR UserSchema().dump(users, many=True)
pprint(result)

[{'created_at': '2019-11-13T16:07:03.841332',
  'email': 'mick@stones.com',
  'name': 'Mick'},
 {'created_at': '2019-11-13T16:07:03.841332',
  'email': 'keith@stones.com',
  'name': 'Keith'}]

• Validation

load方法可以对字段进行验证，并引发ValidationError异常，可以调用ValidationError.valid_data查看通过验证的字段，schema内置有常见类型的字段检查，例如：Email，URL等

try:
    result = UserSchema().load({'name': 'Lily', 'email': 'foo@yahoo'})
except ValidationError as err:
    print(err.messages)  # => {'email': ['Not a valid email address.']}
    print(err.valid_data)  # => {'name': 'Lily'}

当验证一个集合对象，未通过验证的对象的字段会以当前在集合中的序号为键的字典的形式抛出

from marshmallow import Schema, fields, ValidationError

class BandMembersSchema(Schema):
    name = fields.String(required=True)
    email = fields.Email()

user_data = [
    {'email': 'mick@stones.com', 'name': 'Mick'},
    {'email': 'invalid', 'name': 'Invalid'},  # invalid email
    {'email': 'keith@stones.com', 'name': 'Keith'},
    {'email': 'charlie@stones.com'},  # missing "name"
]

try:
    BandMembersSchema(many=True).load(user_data2)
except ValidationError as err:
    pprint(err.messages)

{1: {'email': ['Not a valid email address.']},
 3: {'name': ['Missing data for required field.']}}

可以使用validate参数来对字段进行验证，这里可以找到更多内建的validate方法marshmallow.validate

from marshmallow import Schema, fields, validate, ValidationError

class UserSchema(Schema):
    name = fields.Str(validate=validate.Length(min=1))
    permission = fields.Str(validate=validate.OneOf(["read", "write", "admin"]))
    age = fields.Int(validate=validate.Range(min=18, max=40))

in_data = {"name": "", "permission": "invalid", "age": 71}
try:
    UserSchema().load(in_data)
except ValidationError as err:
    pprint(err.messages)

{'age': ['Must be greater than or equal to 18 and less than or equal to 40.'],
 'name': ['Shorter than minimum length 1.'],
 'permission': ['Must be one of: read, write, admin.']}

也可以自定义验证方法

from marshmallow import Schema, fields, ValidationError

def validate_quantity(n):
    if n < 0:
        raise ValidationError('Quantity must be greater than 0.')
    if n > 30:
        raise ValidationError('Quantity must not be greater than 30.')

class ItemSchema(Schema):
    quantity = fields.Integer(validate=validate_quantity)

in_data = {'quantity': 31}
try:
    ItemSchema().load(in_data)
except ValidationError as err:
    pprint(err.messages)

{'quantity': ['Quantity must not be greater than 30.']}

有时候把字段验证写成方法会更方便，使用validates装饰器可以注册字段验证方法

from marshmallow import fields, Schema, validates, ValidationError

class ItemSchema(Schema):
    quantity = fields.Integer()

    @validates("quantity")
    def validate_quantity(self, value):
        if value < 0:
            raise ValidationError("Quantity must be greater than 0.")
        if value > 30:
            raise ValidationError("Quantity must not be greater than 30.")

如果在class的Meta里定义strict=True，则一次可对多个字段进行验证

from marshmallow import Schema, fields, ValidationError, validates_schema

class ItemSchema(Schema):
    quantity = fields.Integer()
    age = fields.Integer()

    class Meta:
        strict = True

    @validates_schema
    def validate_quantity(self, data, **kwargs):
        if data['quantity'] < 0:
            raise ValidationError('Quantity must be greater than 0.')
        if data['quantity'] > 30:
            raise ValidationError('Quantity must not be greater than 30.')

        if data['age'] < 0:
            raise ValidationError('Age must be greater than 0.')
        if data['age'] > 30:
            raise ValidationError('Age must not be greater than 30.')

try:
    ItemSchema().load({'quantity': 21, 'age': 31})
except ValidationError as err:
    pprint(err.messages)

{'_schema': ['Age must not be greater than 30.']}

• Required Fields

设置required=True，如果没有传入对应字段，当调用load方法时会抛出一个错误信息，设置error_message参数可以定制错误信息

from marshmallow import Schema, fields, ValidationError

class UserSchema(Schema):
    name = fields.String(required=True)
    age = fields.Integer(required=True, error_messages={"required": "Age is required."})
    city = fields.String(
        required=True,
        error_messages={"required": {"message": "City required", "code": 400}},
    )
    email = fields.Email()

try:
    result = UserSchema().load({"email": "foo@bar.com"})
except ValidationError as err:
    pprint(err.messages)

{'age': ['Age is required.'],
'city': {'code': 400, 'message': 'City required'},
'name': ['Missing data for required field.']}

 自定义全局的默认错误信息

from marshmallow import fields

fields.Field.default_error_messages = {
    "required": "缺少必要数据",
    "null": "数据不能为空",
    "validator_failed": "非法数据",
}

fields.Str.default_error_messages = {
    'invalid': "不是合法文本"
}

fields.Int.default_error_messages = {
    "invalid": "不是合法整数"
}

fields.Number.default_error_messages = {
    "invalid": "不是合法数字"
}

fields.Boolean.default_error_messages = {
    "invalid": "不是合法布尔值"
}

• Partial Loading

当schema中的字段设置了required=True，如果传入的对象缺少了相应的字段，则通不过schema的校验，这时候可以设置partial参数来忽略缺失的字段

class UserSchema(Schema):
    name = fields.String(required=True)
    age = fields.Integer(required=True)

result = UserSchema().load({"age": 42}, partial=("name",))
# OR result = UserSchema().load({"age": 42}, partial=True)

print(result)

{'age': 42}

• Specifying Defaults

 指定默认值，missing用来指定反序列化时默认缺省值，default用来指定序列化时默认缺省值

class UserSchema(Schema):
    id = fields.UUID(missing=uuid.uuid1)
    birthdate = fields.DateTime(default=dt.datetime(2017, 9, 29))

UserSchema().load({})
# {'id': UUID('337d946c-32cd-11e8-b475-0022192ed31b')}
UserSchema().dump({})
# {'birthdate': '2017-09-29T00:00:00+00:00'}

• Handling Unknown Fields

 默认情况下，使用load方法，如果传入的对象字段在schema中不存在，则会引发ValidationError异常

class UserSchema(Schema):
    name = fields.Str()

try:
    pprint(UserSchema().load({"name": "Mike", "age": 10}))
except ValidationError as err:
    print(err.messages)

{'age': ['Unknown field.']}

可以通过设置EXCLUDE或者INCLUDE来改变这种默认行为

from marshmallow import Schema, INCLUDE, EXCLUDE

class UserSchema(Schema):
    name = fields.Str()

    class Meta:
        unknown = INCLUDE
        # unknown = EXCLUDE

try:
    pprint(UserSchema().load({"name": "Mike", "age": 10}))
    # OR schema = UserSchema(unknown=INCLUDE)
    # OR UserSchema().load({"name": "Mike", "age": 10}, unknown=INCLUDE)
except ValidationError as err:
    print(err.messages)

{'age': 10, 'name': 'Mike'}

• Validation Without Deserialization

 如果仅是需要校验输入数据（而不需要反序列化对象），使用Schema.validate()方法即可

errors = UserSchema().validate({"name": "Ronnie", "email": "invalid-email"})
print(errors)  

{'email': ['Not a valid email address.']}

• "Read-only" and "Write-only" Fields

 在通常的web api中，dump_only和load_only参数就类似于“read-only”和“write-only”字段

class UserSchema(Schema):
    name = fields.Str()
    # password is "write-only"
    password = fields.Str(load_only=True)
    # created_at is "read-only"
    created_at = fields.DateTime(dump_only=True)

注意：当使用load方法反序列化对象时，dump-only字段会被认为是unknown未知字段

class UserSchema(Schema):
    name = fields.Str()
    password = fields.Str(load_only=True)
    created_at = fields.DateTime(required=True, dump_only=True)


try:
    UserSchema().load({"name": "Mike", "password": "111111", "created_at": "2014-08-11T05:26:03.869245"})
except ValidationError as err:
    print(err.messages)

{'created_at': ['Unknown field.']}

如果此时设置INCLUDE，则对应的字段将不会得到校验

class UserSchema(Schema):
    name = fields.Str()
    password = fields.Str(load_only=True)
    created_at = fields.DateTime(required=True, allow_none=False, dump_only=True)


try:
    UserSchema().load({"name": "Mike", "password": "111111", "created_at": None}, unknown=INCLUDE)
    print("success")
except ValidationError as err:
    print(err.messages)

success

• Specifying Serialization/Deserialization Keys

 如果在序列化和反序列化的时候，传入的数据字段不一致怎么办呢？可以指定data_key参数解决

class UserSchema(Schema):
    name = fields.String()
    email = fields.Email(data_key="emailAddress")

s = UserSchema()

data = {"name": "Mike", "email": "foo@bar.com"}
result = s.dump(data)
# {'name': u'Mike',
# 'emailAddress': 'foo@bar.com'}

data = {"name": "Mike", "emailAddress": "foo@bar.com"}
result = s.load(data)
# {'name': u'Mike',
# 'email': 'foo@bar.com'}

• Implicit Field Creation

隐式字段创建，当模型有太多的参数，为每一个参数指定字段会是一件很重复的工作，特别是很多参数本身就是python内置数据类型

schema允许指定创建隐式字段，marshmallow会根据参数的类型选择合适的字段类型

class UserSchema(Schema):
    uppername = fields.Function(lambda obj: obj.name.upper())

    class Meta:
        fields = ("name", "email", "created_at", "uppername")

class User(object):
    def __init__(self, name, email):
        self.name = name
        self.email = email
        self.created_at = dt.datetime.now()

user = User(name="Mike", email="mike@example.com")
try:
    pprint(UserSchema().dump(user))
except ValidationError as err:
    print(err.messages)

{'created_at': '2019-11-14T13:45:38.350031',
 'email': 'mike@example.com',
 'name': 'Mike',
 'uppername': 'MIKE'}

注意：如果想指定除了显示声明的字段外还要包括哪些字段名，可以设置additional。

下面的schema和上面的等同

class UserSchema(Schema):
    uppername = fields.Function(lambda obj: obj.name.upper())

    class Meta:
        # No need to include 'uppername'
        additional = ("name", "email", "created_at")

• Ordering Output

为了保持字段的顺序，可以设置ordered=True，Marshmallow会将对象序列化为collections.OrderedDict对象

class UserSchema(Schema):
    uppername = fields.Function(lambda obj: obj.name.upper())

    class Meta:
        additional = ("name", "email", "created_at")
        ordered = True

class User(object):
    def __init__(self, name, email):
        self.name = name
        self.email = email
        self.created_at = dt.datetime.now()

user = User(name="Mike", email="mike@example.com")
result = UserSchema().dump(user)
assert isinstance(result, OrderedDict)
pprint(result)

OrderedDict([('uppername', 'MIKE'),
             ('name', 'Mike'),
             ('email', 'mike@example.com'),
             ('created_at', '2019-11-14T14:02:29.680152')])

 参考：https://marshmallow.readthedocs.io/en/stable/quickstart.html