zoukankan      html  css  js  c++  java
  • python获取hive表时间格式最大分区

    #获取表的最大分区
    import boto3
    from datetime import datetime,timedelta
    def get_max_partition(db_name,table_name,partition_format='%Y-%m-%d'):
        client=boto3.client('glue')
        yesterday=datetime.utcnow()
        max_partition=yesterday.strftime(partition_format)
        response = client.batch_get_partition(
            DatabaseName=db_name,
            TableName=table_name,
            PartitionsToGet=[
                {
                    'Values': [
                        max_partition,
                    ]
                },
            ]
        )
        while response['Partitions']==[]:
            if partition_format in ['%Y-%m-%d','%Y%m%d']:
                yesterday=yesterday-timedelta(days=1)
            elif partition_format in ['%Y-%m','%Y%m']:
                if yesterday.month >1:
                    yesterday = yesterday.replace(month=yesterday.month - 1)
                else:
                    yesterday = yesterday.replace(year=yesterday.year - 1, month=12)
            max_partition=yesterday.strftime(partition_format)
            response = client.batch_get_partition(
                DatabaseName=db_name,
                TableName=table_name,
                PartitionsToGet=[
                    {
                        'Values': [
                            max_partition,
                        ]
                    },
                ]
            )
        return response['Partitions'][0]['Values'][0]
  • 相关阅读:
    继承
    面向对象
    数据库的数据操作
    数据库数据类型以及建库语句
    第一天
    继承与多态
    C#面向对象——对象成员、方法重载、引用类库等
    C#面向对象初步
    SQL2008知识回顾
    C#知识回顾
  • 原文地址:https://www.cnblogs.com/wangbin2188/p/11811878.html
Copyright © 2011-2022 走看看