zoukankan      html  css  js  c++  java
  • Amazon S3 分布式存储的 python 接口实现

    Amazon s3 是一种分布式的对象存储。用键值对的方式,来存储数据。其中,存入的所有数据都是一个对象(object),每一个对象都有一个键(key)存在。

    具有非常方便的 web 访问接口,以及权限控制。

    下面是具体读、写、判断三个接口的具体实现

    1. 创建一个 bucket

    class ImageFeatIO:
        __read_singleton = None
        __write_singleton = None
        __read_count = 0
        __write_count = 0
    
        def __init__(self):
            pass
    
        @staticmethod
        def get_write_instance():  #创建一个 bucket 的单例
            if ImageFeatIO.__write_singleton is None:
                paras = get_config('config')
                access_key = paras['access_key']
                secret_key = paras['secret_key']
                write_host = paras['file_write_host']
                conn = boto.connect_s3(
                    aws_access_key_id=access_key,
                    aws_secret_access_key=secret_key,
                    host=write_host, is_secure=False,
                    calling_format=boto.s3.connection.OrdinaryCallingFormat()
                )
                bucket_name = paras['bucket_name']
                bucket = conn.get_bucket(bucket_name)
                ImageFeatIO.__write_singleton = bucket
            return ImageFeatIO.__write_singleton

    2. 写入文件

    写入文件的时候,可以进行权限控制

    官方说明如下:

    1. Create a custom ACL that grants specific rights to specific users. At the moment, the users that are specified within grants have to be registered users of Amazon Web Services so this isn’t as useful or as general as it could be.
    2. Use a “canned” access control policy. There are four canned policies defined:
      1. private: Owner gets FULL_CONTROL. No one else has any access rights.
      2. public-read: Owners gets FULL_CONTROL and the anonymous principal is granted READ access.
      3. public-read-write: Owner gets FULL_CONTROL and the anonymous principal is granted READ and WRITE access.
      4. authenticated-read: Owner gets FULL_CONTROL and any principal authenticated as a registered Amazon S3 user is granted READ access
    def write_image_feature_to_file(id, imageFeaturestring):
        bucket = ImageFeatIO.get_write_instance()
        k = Key(bucket)
        k.key = id
        res = k.set_contents_from_string(imageFeaturestring)
        k.set_acl('authenticated-read')
        return res

    3.读取(下载)文件

    s3 提供了一种非常方便的 web 服务接口,可以从任何地方以 http 协议获取数据

    def read_image_feature_from_file(id):
        url_head = ImageFeatIO.get_read_instance()
        url = url_head + '/' + id  #url= http://img.3weijia.com/bucket_name/key
        request = urllib2.Request(url=url)
        response = urllib2.urlopen(request, timeout=20)
        res_unicode = unicode(response.read())
        res = res_unicode.encode('unicode-escape').decode('string_escape')
        return res

    4. 判断键值是否存在

    def if_image_feature_exist(id):
        bucket = ImageFeatIO.get_write_instance()
        key = bucket.get_key(id)
    
        return key is not None

    参考:

    http://boto.readthedocs.io/en/latest/s3_tut.html

    https://www.cnblogs.com/web424/p/6840207.html

  • 相关阅读:
    Android数据的四种存储方式SharedPreferences、SQLite、Content Provider和File (二) —— SQLite
    理解 Continuation
    99种用Racket说I love you的方式
    Racket Cheat Sheet
    scheme 教程 #lang racket
    开始学习Scheme
    MIT Scheme 的基本使用
    CPS变换
    SECD machine
    scheme 之门
  • 原文地址:https://www.cnblogs.com/yaolin1228/p/9092599.html
Copyright © 2011-2022 走看看