zoukankan      html  css  js  c++  java
  • pgloader 学习(七) 从归档文件加载数据

    我们可以直接从zip,tar,gzip 文件获取内容

    command file 参考格式

    LOAD ARCHIVE
       FROM /Users/dim/Downloads/GeoLiteCity-latest.zip
       INTO postgresql:///ip4r
       BEFORE LOAD
         DO $$ create extension if not exists ip4r; $$,
            $$ create schema if not exists geolite; $$,
         EXECUTE 'geolite.sql'
       LOAD CSV
            FROM FILENAME MATCHING ~/GeoLiteCity-Location.csv/
                 WITH ENCODING iso-8859-1
                 (
                    locId,
                    country,
                    region null if blanks,
                    city null if blanks,
                    postalCode null if blanks,
                    latitude,
                    longitude,
                    metroCode null if blanks,
                    areaCode null if blanks
                 )
            INTO postgresql:///ip4r?geolite.location
                 (
                    locid,country,region,city,postalCode,
                    location point using (format nil "(~a,~a)" longitude latitude),
                    metroCode,areaCode
                 )
            WITH skip header = 2,
                 fields optionally enclosed by '"',
                 fields escaped by double-quote,
                 fields terminated by ','
      AND LOAD CSV
            FROM FILENAME MATCHING ~/GeoLiteCity-Blocks.csv/
                 WITH ENCODING iso-8859-1
                 (
                    startIpNum, endIpNum, locId
                 )
            INTO postgresql:///ip4r?geolite.blocks
                 (
                    iprange ip4r using (ip-range startIpNum endIpNum),
                    locId
                 )
            WITH skip header = 2,
                 fields optionally enclosed by '"',
                 fields escaped by double-quote,
                 fields terminated by ','
       FINALLY DO
         $$ create index blocks_ip4r_idx on geolite.blocks using gist(iprange); $$;

    source 指南: from

    文件名或者一个http 的url,如果是一个url,在处理之前会先下载到本地

    sub command

    • command [AND command ...]
      目前仅支持CSV,“FIXED”和DBF命令

    Final SQL Commands

    • FINALLY DO
      数据加载完成之后执行的sql 命令,比如create index

    参考资料

    https://pgloader.readthedocs.io/en/latest/ref/archive.html

  • 相关阅读:
    Hive系统函数之collect_list和collect_set
    SpringBoot集成JWT实现token验证
    Hadoop添加LZO压缩支持
    常见函数封装(自用,随时更新)
    vs-code 的常用插件
    在移动端显示控制台信息
    SublimeText 3 常见快捷键
    vue的自定义指令
    修改滚动条样式
    vue、react中循环遍历为什么会有key,key有什么作用?
  • 原文地址:https://www.cnblogs.com/rongfengliang/p/11019307.html
Copyright © 2011-2022 走看看