zoukankan      html  css  js  c++  java
  • SQLSERVER 分页

    自SQLSERVER 2008 起新增的RowNumber确实简化了很多分页的代码,目前百万数据中

    RowNumber 配合颠倒Top仍然是最佳实践

    现在项目上的搜索比较恶心,主表与从表是个一对多关系,搜索条件里可能带有从表数据,并将符合结果的主表数据以及第一条从表数据(按创建时间排序)取出,注意主表数据不能因为inner join而重复。

    像这么恶心的搜索我还是第一次遇到,听说还有种动态视图(可创建自适应索引),但我没去试,还是用了个比较靠谱的,思路如下:

    如果无从表搜索,则直接搜索主表,取出分页数,并join 从表

    如果有从表,则只能先join再分页。值得一提的是分页排序规则是根据当前页数与总页数相比较得到(所以要先得到总页数)

    而从表只取第一条我们也可以巧妙得在RowNumber 中用关键字 PARTITION BY 来达成

    最后结果,百万数据的两表join 搜索都在1秒内,部分字段的StartWith搜索也只需要2秒

    最后贴出代码,索引啥就不贴了

    有从表查询:

      1 USE [GEMS_DM]
      2 GO
      3 /****** Object:  StoredProcedure [dbo].[P_PM_SearchPatronByHasIdentifyCriteria]    Script Date: 12/23/2013 16:38:41 ******/
      4 SET ANSI_NULLS ON
      5 GO
      6 SET QUOTED_IDENTIFIER ON
      7 GO
      8 
      9 
     10 
     11 
     12 
     13 
     14 
     15 
     16 
     17 
     18 
     19 ALTER PROCEDURE [dbo].[P_PM_SearchPatronByHasIdentifyCriteria]    
     20      @IsActive bit = NULL,
     21      @PatronNo bigint=NULL,
     22      @BirthDate date=NULL,
     23      @PNFirstName nvarchar(50)=NULL,
     24      @PNMiddleName nvarchar(50)=NULL,
     25      @PNLastName nvarchar(50)=NULL,
     26      @PreferredName nvarchar(40)=NULL,
     27      @EXFirstName nvarchar(50)=NULL,
     28      @EXLastName nvarchar(50)=NULL,
     29      @DisplayNameType nvarchar(30)=NULL,
     30      @Gender nvarchar(5)=NULL,
     31      @MobileCountryCode nvarchar(5)=NULL,
     32      @Mobile nvarchar(15)=NULL,
     33      @Email nvarchar(30)=NULL,
     34      @MembershipClass nvarchar(30)=Null,
     35      @IDType  nvarchar(5)=NULL,
     36      @IDNumber nvarchar(20)=NULL,
     37      @CountryID  smallint=NULL,
     38      @CityName nvarchar(50)=NULL,
     39      @DocType nvarchar(5)='',     
     40      @Page int = 1,
     41      @PageSize int = 10,
     42      @OrderSQL nvarchar(max)=null,
     43      @TotalRow int output
     44 
     45     
     46 
     47       -- Add the parameters for the stored procedure here    
     48 as
     49 BEGIN
     50 
     51 SET NOCOUNT ON;
     52 SET FMTONLY OFF  
     53 
     54 
     55 declare @Start nvarchar(20),@End nvarchar(20),@RowNumberSQL nvarchar(200),@BetweenSQL nvarchar(100),
     56 @rowCountSql nvarchar(max);
     57 IF (@Page is null) or (@Page <= 0)
     58     SET @Page = 1
     59 IF (@PageSize is null) or (@PageSize <=0)
     60     SET @PageSize = 10
     61 SET @Start=CONVERT(nvarchar(10),(@Page-1)*@PageSize+1);
     62 SET @End=CONVERT(nvarchar(10),(@Page)*@PageSize);
     63 Set @BetweenSQL = ' ROWNUMVE BETWEEN {Start} AND {End} ';
     64 
     65 set @RowNumberSQL = ' ROW_NUMBER() over (order by Patron_No {0} ) AS ROWNUMVE';
     66 
     67 
     68 declare @PatronTableSql nvarchar(max),@IdentificationTableSql nvarchar(max),@sqlwhere nvarchar(max),
     69 @FinalPatronTableSql nvarchar(max)
     70 
     71 set @PatronTableSql = 'SELECT Patron_ID AS PatronID,Patron_No AS PatronNo,PN_First_Name AS PNFirstName,PN_Middle_Name as PNMiddleName,PN_Last_Name AS PNLastName,
     72               Preferred_Name AS PreferredName,EX_First_Name AS EXFirstName,EX_Last_Name AS EXLastName,Birth_Date AS BirthDate,
     73               IsActive,IsMerged,Gender,Membership_Class AS MembershipClass,eMail,Mobile_Country_Code AS MobileCountryCode,
     74               Mobile_No AS MobileNo,Doc_ID as DocID,Patron_No as Patron_No FROM GSMA_Patron ';
     75 if(@PatronNo is not null)
     76     set @sqlwhere = dbo.F_CombineWhereSql(@sqlwhere,concat('Patron_No=',@PatronNo));
     77 if(@BirthDate is not null)
     78     set @sqlwhere = dbo.F_CombineWhereSql(@sqlwhere,concat('Birth_Date=''',@BirthDate,''''));
     79 if(@IsActive is not null)
     80     set @sqlwhere = dbo.F_CombineWhereSql(@sqlwhere,concat('IsActive=',@IsActive));
     81 if(@PNFirstName is not null)
     82     set @sqlwhere = dbo.F_CombineWhereSql(@sqlwhere,concat('PN_First_Name like N''',@PNFirstName,'%'''));
     83 if(@PNMiddleName is not null)
     84     set @sqlwhere = dbo.F_CombineWhereSql(@sqlwhere,concat('PN_Middle_Name like N''',@PNMiddleName,'%'''));
     85 if(@PNLastName is not null)
     86     set @sqlwhere = dbo.F_CombineWhereSql(@sqlwhere,concat('PN_Last_Name like N''',@PNLastName,'%'''));
     87 if(@PreferredName is not null)
     88     set @sqlwhere = dbo.F_CombineWhereSql(@sqlwhere,concat('Preferred_Name like N''',@PreferredName,'%'''));
     89 if(@EXFirstName is not null)
     90     set @sqlwhere = dbo.F_CombineWhereSql(@sqlwhere,concat('EX_First_Name like N''',@EXFirstName+'%'''));
     91 if(@EXLastName is not null)
     92     set @sqlwhere = dbo.F_CombineWhereSql(@sqlwhere,concat('EX_Last_Name like N''',@EXLastName,'%'''));
     93 if(@Gender is not null)
     94     set @sqlwhere = dbo.F_CombineWhereSql(@sqlwhere,concat('Gender = N''',@Gender,''''));
     95 if(@MembershipClass is not null)
     96     set @sqlwhere = dbo.F_CombineWhereSql(@sqlwhere,concat('Membership_Class in (''',Replace(@MembershipClass,',',''','''),''')'));
     97 if(@Email is not null)
     98     set @sqlwhere = dbo.F_CombineWhereSql(@sqlwhere,concat('eMail like N''',@Email+'%'''));
     99 
    100 if(@sqlwhere is not null and @sqlwhere <> '')
    101     set @PatronTableSql = concat(@PatronTableSql,' where ',@sqlwhere);
    102 set @FinalPatronTableSql = concat('(',@PatronTableSql,') A ');
    103 
    104 
    105 --reset @sqlwhere for Identification filter
    106 set @sqlwhere = ' IsActive = 1 ';
    107 
    108 set @IdentificationTableSql = '(SELECT IDType,IDNo,Patron_ID,CountryID,CityName From 
    109 (SELECT ID_Type as IDType,ID_No as IDNo,Patron_ID,
    110 Country_ID AS CountryID,City_Name as CityName
    111 ,ROW_NUMBER() OVER(PARTITION BY Patron_ID ORDER BY ISNULL(Modified_Date,Created_Date) DESC) AS RK FROM GSMA_Patron_Identification WHERE ';
    112 if(@IDType is not null)
    113     set @sqlwhere = dbo.F_CombineWhereSql(@sqlwhere,concat('ID_Type=N''',@IDType,''''));
    114 if(@IDNumber is not null)
    115     set @sqlwhere = dbo.F_CombineWhereSql(@sqlwhere,concat('ID_No like N''',@IDNumber,'%'''));
    116 if(@CountryID is not null)
    117     set @sqlwhere = dbo.F_CombineWhereSql(@sqlwhere,concat('Country_ID=',@CountryID));
    118 if(@CityName is not null)
    119     set @sqlwhere = dbo.F_CombineWhereSql(@sqlwhere,concat('City_Name like N''',@CityName,'%'''));
    120 set @IdentificationTableSql = concat(@IdentificationTableSql,@sqlwhere,') ID WHERE RK = 1 ) B ');
    121 
    122 declare @PISql nvarchar(max) = concat('Select * from
    123 (Select * {0} from ',char(10),@FinalPatronTableSql,char(10),' inner join',char(10),@IdentificationTableSql,'
    124  ON A.PatronID = B.Patron_ID) PLI ')
    125 
    126 -- remove template string and get total count sql
    127 set @rowCountSql = concat('Select @rowCount = count(1)  from (',REPLACE(@PISql,'{0}',''),') WS ');
    128 -- add paging rownumber 
    129 set @PISql = REPLACE(@PISql,'{0}',concat(',',@RowNumberSQL))
    130 -- add between sql
    131 set @PISql = concat(@PISql,' WHERE ',@BetweenSQL);
    132 set @PISql = concat('(',@PISql,') PI');
    133 
    134 declare @pageSql nvarchar(max) = concat('SELECT PI.PatronID,PI.PatronNo,PI.PNFirstName,PI.PNMiddleName,PI.PNLastName,PI.PreferredName,PI.EXFirstName,
    135 PI.EXLastName,PI.BirthDate,G.Lkp_Value as GenderName,PI.MembershipClass,M.Lkp_Value as MembershipClassName,
    136 PI.MobileCountryCode,PI.MobileNo,PI.eMail,PI.IsActive,PI.IsMerged,I.Lkp_Value as IDTypeName,PI.IDNo, C.Country_Name as CountryName,
    137 PI.CityName, D.[FileName],T.Tier_ID as TierID, T.Security_level as Securitylevel,PC.Club_Name as ClubName FROM ',@PISql,char(10),
    138        'left join (SELECT [File_Name] as [FileName],Doc_ID FROM GSAL_Document_Detail WHERE IsActive = 1 AND Doc_Type = ''DOCPH'') D 
    139             on PI.DocID = D.Doc_ID         
    140         left join (Select Lkp_Value,Lookup_Cd FROM GSRE_GenLKP) G on PI.Gender =G.Lookup_Cd
    141         left join (Select Lkp_Value,Lookup_Cd FROM GSRE_GenLKP) M on PI.MembershipClass =M.Lookup_Cd
    142         left join (Select Lkp_Value,Lookup_Cd FROM GSRE_GenLKP) I on PI.IDType =I.Lookup_Cd
    143         left join (Select Country_ID,Country_Name FROM GSRE_Country) C on PI.CountryID =C.Country_ID
    144         left join (Select Patron_ID,Tier_ID FROM GSPM_Patron_Tier WHERE IsActive=1) PT on PI.PatronID = PT.Patron_ID
    145         join  (Select Tier_ID,Club_ID,Security_level FROM GSRE_Tier) T on PT.Tier_ID=T.Tier_ID
    146         left join (Select Club_Name,Club_ID FROM GSRE_Club where IsActive=1) PC on T.Club_ID =  PC.Club_ID  ')
    147 
    148     Execute sp_executesql @rowCountSql, N'@rowCount int output', @TotalRow output;
    149     
    150     if((@TotalRow/(@PageSize * 2)) >= @Page)
    151     begin        
    152         set @pageSql =  REPLACE(@pageSql,'{0}',' asc ');
    153         set @pageSql =  REPLACE(@pageSql,'{Start}',@Start);
    154         set @pageSql =  REPLACE(@pageSql,'{End}',@End);
    155     end
    156     else
    157     begin
    158         set @pageSql =  REPLACE(@pageSql,'{0}',' desc ');
    159         set @pageSql =  REPLACE(@pageSql,'{Start}',@TotalRow - @End + 1);
    160         set @pageSql =  REPLACE(@pageSql,'{End}',@TotalRow - @Start + 1);        
    161         set @pageSql = concat(' select * from (',@pageSql,') DE order by PatronNo asc');
    162     end
    163     exec (@pageSql)
    164 END
    View Code

    无从表查询:

      1 USE [GEMS_DM]
      2 GO
      3 /****** Object:  StoredProcedure [dbo].[P_PM_SearchPatronByNoIdentifyCriteria]    Script Date: 12/23/2013 16:41:24 ******/
      4 SET ANSI_NULLS ON
      5 GO
      6 SET QUOTED_IDENTIFIER ON
      7 GO
      8 
      9 
     10 
     11 
     12 
     13 
     14 
     15 
     16 
     17 ALTER PROCEDURE [dbo].[P_PM_SearchPatronByNoIdentifyCriteria]    
     18      @IsActive bit = NULL,
     19      @PatronNo bigint=NULL,
     20      @BirthDate date=NULL,
     21      @PNFirstName nvarchar(50)=NULL,
     22      @PNMiddleName nvarchar(50)=NULL,
     23      @PNLastName nvarchar(50)=NULL,
     24      @PreferredName nvarchar(40)=NULL,
     25      @EXFirstName nvarchar(50)=NULL,
     26      @EXLastName nvarchar(50)=NULL,
     27      @Gender nvarchar(5)=NULL,
     28      @MobileCountryCode nvarchar(5)=NULL,
     29      @Mobile nvarchar(15)=NULL,
     30      @Email nvarchar(30)=NULL,
     31      @MembershipClass nvarchar(30)=Null,    
     32      @DocType nvarchar(5)='',     
     33      @Page int = 1,
     34      @PageSize int = 10,
     35      @OrderSQL nvarchar(max)=null,
     36      @TotalRow int output
     37 
     38     
     39 
     40       -- Add the parameters for the stored procedure here    
     41 as
     42 BEGIN
     43 
     44 SET NOCOUNT ON;
     45 SET FMTONLY OFF  
     46 
     47 
     48 declare @Start nvarchar(20),@End nvarchar(20),@RowNumberSQL nvarchar(200),@BetweenSQL nvarchar(100),
     49 @rowCountSql nvarchar(max)
     50 IF (@Page is null) or (@Page <= 0)
     51     SET @Page = 1
     52 IF (@PageSize is null) or (@PageSize <=0)
     53     SET @PageSize = 10
     54 SET @Start=CONVERT(nvarchar(10),(@Page-1)*@PageSize+1);
     55 SET @End=CONVERT(nvarchar(10),(@Page)*@PageSize);
     56 Set @BetweenSQL = ' ROWNUMVE BETWEEN {Start} AND {End} ';
     57 set @RowNumberSQL = ' ROW_NUMBER() over (order by Patron_No {0} ) AS ROWNUMVE';
     58 
     59 
     60 declare @PatronTableSql nvarchar(max),@IdentificationTableSql nvarchar(max),@sqlwhere nvarchar(max),
     61 @FinalPatronTableSql nvarchar(max)
     62 
     63 set @PatronTableSql = 'SELECT Patron_ID AS PatronID,Patron_No AS PatronNo,PN_First_Name AS PNFirstName,PN_Middle_Name as PNMiddleName,PN_Last_Name AS PNLastName,
     64               Preferred_Name AS PreferredName,EX_First_Name AS EXFirstName,EX_Last_Name AS EXLastName,Birth_Date AS BirthDate,
     65               IsActive,IsMerged,Gender,Membership_Class AS MembershipClass,eMail,Mobile_Country_Code AS MobileCountryCode,
     66               Mobile_No AS MobileNo,Doc_ID as DocID {0} FROM GSMA_Patron ';
     67 if(@PatronNo is not null)
     68     set @sqlwhere = dbo.F_CombineWhereSql(@sqlwhere,concat('Patron_No=',@PatronNo));
     69 if(@BirthDate is not null)
     70     set @sqlwhere = dbo.F_CombineWhereSql(@sqlwhere,concat('Birth_Date=''',@BirthDate,''''));
     71 if(@IsActive is not null)
     72     set @sqlwhere = dbo.F_CombineWhereSql(@sqlwhere,concat('IsActive=',@IsActive));
     73 if(@PNFirstName is not null)
     74     set @sqlwhere = dbo.F_CombineWhereSql(@sqlwhere,concat('PN_First_Name like N''',@PNFirstName,'%'''));
     75 if(@PNMiddleName is not null)
     76     set @sqlwhere = dbo.F_CombineWhereSql(@sqlwhere,concat('PN_Middle_Name like N''',@PNMiddleName,'%'''));
     77 if(@PNLastName is not null)
     78     set @sqlwhere = dbo.F_CombineWhereSql(@sqlwhere,concat('PN_Last_Name like N''',@PNLastName,'%'''));
     79 if(@PreferredName is not null)
     80     set @sqlwhere = dbo.F_CombineWhereSql(@sqlwhere,concat('Preferred_Name like N''',@PreferredName,'%'''));
     81 if(@EXFirstName is not null)
     82     set @sqlwhere = dbo.F_CombineWhereSql(@sqlwhere,concat('EX_First_Name like N''',@EXFirstName+'%'''));
     83 if(@EXLastName is not null)
     84     set @sqlwhere = dbo.F_CombineWhereSql(@sqlwhere,concat('EX_Last_Name like N''',@EXLastName,'%'''));
     85 if(@Gender is not null)
     86     set @sqlwhere = dbo.F_CombineWhereSql(@sqlwhere,concat('Gender = N''',@Gender,''''));
     87 if(@MembershipClass is not null)
     88     set @sqlwhere = dbo.F_CombineWhereSql(@sqlwhere,concat('Membership_Class in (''',Replace(@MembershipClass,',',''','''),''')'));
     89 if(@Email is not null)
     90     set @sqlwhere = dbo.F_CombineWhereSql(@sqlwhere,concat('eMail like N''',@Email+'%'''));
     91 
     92 if(@sqlwhere is not null and @sqlwhere <> '')
     93     set @PatronTableSql = concat(@PatronTableSql,' where ',@sqlwhere);
     94     
     95 -- remove template string and get total account from patron search
     96 set @rowCountSql = concat('Select @rowCount = count(1)  from (',REPLACE(@PatronTableSql,'{0}',''),') WS ');
     97 -- add paging rownumber column when select Patron
     98 set @PatronTableSql = REPLACE(@PatronTableSql,'{0}',concat(',',@RowNumberSQL));    
     99 -- set final patron table search sql
    100 set @FinalPatronTableSql = concat('Select PatronID,PatronNo,PNFirstName,PNMiddleName,PNLastName,PreferredName,EXFirstName,
    101                 EXLastName,BirthDate,IsActive,IsMerged,Gender,MembershipClass,eMail,MobileCountryCode,MobileNo,DocID 
    102                  From (',@PatronTableSql,') P  WHERE ',@BetweenSQL);
    103 set @FinalPatronTableSql = concat('(',@FinalPatronTableSql,') A ');
    104 
    105 
    106 set @IdentificationTableSql = '(SELECT ID_Type as IDType,ID_No as IDNo,Patron_ID,
    107 Country_ID AS CountryID,City_Name as CityName,ISNULL(Modified_Date,Created_Date) as SortDate FROM GSMA_Patron_Identification WHERE IsActive = 1) B';
    108 
    109 
    110 declare @PISql nvarchar(max)
    111 set @PISql = concat('(Select PatronID,PatronNo,PNFirstName,PNMiddleName,PNLastName,PreferredName,EXFirstName,
    112 EXLastName,BirthDate,IsActive,IsMerged,Gender,MembershipClass,eMail,MobileCountryCode,MobileNo,DocID,
    113 IDType,IDNo,CountryID,CityName From
    114 (Select A.*,B.IDType,B.IDNo,B.CountryID,B.CityName,
    115 ROW_NUMBER() OVER(PARTITION BY PatronID ORDER BY SortDate DESC) AS RK from '
    116 ,char(10),@FinalPatronTableSql,char(10),' Left join ',char(10),@IdentificationTableSql,'
    117  ON A.PatronID = B.Patron_ID) PLI WHERE RK=1 ) PI')
    118 
    119 declare @pageSql nvarchar(max) = concat('SELECT PI.PatronID,PI.PatronNo,PI.PNFirstName,PI.PNMiddleName,PI.PNLastName,PI.PreferredName,PI.EXFirstName,
    120 PI.EXLastName,PI.BirthDate,G.Lkp_Value as GenderName,PI.MembershipClass,M.Lkp_Value as MembershipClassName,
    121 PI.MobileCountryCode,PI.MobileNo,PI.eMail,PI.IsActive,PI.IsMerged,I.Lkp_Value as IDTypeName,PI.IDNo, C.Country_Name as CountryName,
    122 PI.CityName, D.[FileName],T.Tier_ID as TierID, T.Security_level as Securitylevel,PC.Club_Name as ClubName FROM ',@PISql,char(10),
    123        'left join (SELECT [File_Name] as [FileName],Doc_ID FROM GSAL_Document_Detail WHERE IsActive = 1 AND Doc_Type = ''DOCPH'') D 
    124             on PI.DocID = D.Doc_ID         
    125         left join (Select Lkp_Value,Lookup_Cd FROM GSRE_GenLKP) G on PI.Gender =G.Lookup_Cd
    126         left join (Select Lkp_Value,Lookup_Cd FROM GSRE_GenLKP) M on PI.MembershipClass =M.Lookup_Cd
    127         left join (Select Lkp_Value,Lookup_Cd FROM GSRE_GenLKP) I on PI.IDType =I.Lookup_Cd
    128         left join (Select Country_ID,Country_Name FROM GSRE_Country) C on PI.CountryID =C.Country_ID
    129         left join (Select Patron_ID,Tier_ID FROM GSPM_Patron_Tier WHERE IsActive=1) PT on PI.PatronID = PT.Patron_ID
    130         join  (Select Tier_ID,Club_ID,Security_level FROM GSRE_Tier) T on PT.Tier_ID=T.Tier_ID
    131         left join (Select Club_Name,Club_ID FROM GSRE_Club where IsActive=1) PC on T.Club_ID =  PC.Club_ID  ')
    132         
    133     Execute sp_executesql @rowCountSql, N'@rowCount int output', @TotalRow output;
    134     
    135     if((@TotalRow/(@PageSize * 2)) >= @Page)
    136     begin
    137         set @pageSql =  REPLACE(@pageSql,'{0}',' asc ');
    138         set @pageSql =  REPLACE(@pageSql,'{Start}',@Start);
    139         set @pageSql =  REPLACE(@pageSql,'{End}',@End);
    140     end
    141     else
    142     begin
    143         set @pageSql =  REPLACE(@pageSql,'{0}',' desc ');
    144         set @pageSql =  REPLACE(@pageSql,'{Start}',@TotalRow - @End + 1);
    145         set @pageSql =  REPLACE(@pageSql,'{End}',@TotalRow - @Start + 1);        
    146         set @pageSql = concat(' select * from (',@pageSql,') DE order by PatronNo asc');        
    147     end
    148     exec (@pageSql)
    149 END
    View Code

    顺带最后一提:动态SQL的性能和普通SQL没区别

  • 相关阅读:
    excel数据 入库mysql 和 mysql数据 导入excel文件
    gson和fastjson将json对象转换成javaBean 简单对照
    docker入门
    jdbc 事务
    关于Java 实现抽象类的抽象方法的特性的利用---面向切面
    try}-with-resources
    关于虚拟机加载类的先后顺序测试
    MySQL api
    JS 截取字符串-全是干货
    JS截取字符串常用方法详细整理
  • 原文地址:https://www.cnblogs.com/vincentsun1234/p/3487690.html
Copyright © 2011-2022 走看看