zoukankan      html  css  js  c++  java
  • Query performance optimization of Vertica

    1. Don't fetch any data that you don't need,or don't fetch any columns that you don't need. Because retrieving more data or more columns, which can increase network,I/O,memory and CPU overhead for the server. For example, if you need several columns you can use
      AT EPOCH LATEST
      SELECT fi.name, fi.InvestmentKey,id.VendorId,id.CUSIP,id.ISIN,id.DomicileCountryId,id.CurrencyId
      FROM dbo.FixedIncome fi
      INNER JOIN dbo.InvestmentIdDimension id ON id.InvestmentKey = fi.InvestmentKey
      WHERE id.InvestmentId = 'B000023K1X'
      But do not use:
      AT EPOCH LATEST
      SELECT fi.*, id.*
      FROM dbo.FixedIncome fi
      INNER JOIN dbo.InvestmentIdDimension id ON id.InvestmentKey = fi.InvestmentKey
      WHERE id.InvestmentId = 'B000023K1X'
    2. To avoid blocking Vertica write process, we alway add the "AT EPOCH LATEST" for query,which is snapshot read. for example, You can use
      AT EPOCH LATEST SELECT ... FROM ...,
      But do not use:
      SELECT ... FROM ...
    3. Chop up a complex query to many simpler queries.
    4. Join decomposition, if posible, Sometimes, Using "In" clause or sub query clause instead of a complex "JOIN" clause. like this, we can use
      AT EPOCH LATEST
      SELECT s1.CompanyId, id.InvestmentId, s1.InvestmentKey,id.VendorId,id.CUSIP,id.ISIN,id.DomicileCountryId,id.CurrencyId
      FROM ( SELECT CompanyId,InvestmentKey FROM dbo.FixedIncome WHERE CompanyId = '0C00000BDL') s1
      INNER JOIN dbo.InvestmentIdDimension id ON id.InvestmentKey = s1.InvestmentKey
      WHERE id.VendorId = 101 OR id.VendorId = 102;
      But do not use:
      AT EPOCH LATEST
      SELECT s1.CompanyId, id.InvestmentId, s1.InvestmentKey,id.VendorId,id.CUSIP,id.ISIN,id.DomicileCountryId,id.CurrencyId
      FROM dbo.FixedIncome fi
      INNER JOIN dbo.InvestmentIdDimension id ON id.InvestmentKey = s1.InvestmentKey
      WHERE fi.CompanyId = '0C00000BDL' AND( id.VendorId = 101 OR id.VendorId = 102 );
    5. Try to use the temporary table to cache data, which can avoid scan an physical table for times.
    6. Try to push the outer predicate into the inner subquery clause, so that it is evaluated before the analytic computation
    7. For Top-K query, if posible, we'd better omit the order by clause, Or we'd better adding a filter condition for it. 
    8. For sort operation, We can create Pre-sorted projections, so the vertica can choose the faster Group By Pipeline over Group By Hash
    9. Please refer to the "Optimizing Query Performance" chapter in reference manual of vertica, which doc's name is "Communiti Vertica Community Edition 6.0"
      [https://my.vertica.com/docs/CE/6.0.1/HTML/index.htm#12525.htm ]
  • 相关阅读:
    最全的“大数据”学习资源
    民生银行十五年的数据体系建设,深入解读阿拉丁大数据生态圈、人人BI 是如何养成的?【转】
    大数据分析界的“神兽”Apache Kylin有多牛?【转】
    大数据环境下互联网行业数据仓库/数据平台的架构之漫谈-续【转】
    写给大数据开发初学者的话5[转]
    唯品会海量实时OLAP分析技术升级之路
    大数据学习笔记
    元数据
    数据仓库之数据模型
    官方教程:Apache Kylin和Superset集成,使用开源组件,完美打造OLAP系统
  • 原文地址:https://www.cnblogs.com/s021368/p/3208679.html
Copyright © 2011-2022 走看看