zoukankan      html  css  js  c++  java
  • number精度问题

    SQL> show numw
    numwidth 10

    SQL> select 1/3*3 from dual;

         1/3*3
    ----------
             1

    SQL> select trunc(1/3*3,9) from dual;

    TRUNC(1/3*3,9)
    --------------
        .999999999

    SQL> select trunc(1/3*3,10) from dual;

    TRUNC(1/3*3,10)
    ---------------
                  1

    SQL> select trunc(1/3*3,19) from dual;

         TRUNC(1/3*3,19)
    --------------------
    .9999999999999999999

    SQL> select trunc(1/3*3,20) from dual;

         TRUNC(1/3*3,20)
    --------------------
                       1

    SQL> select dump(1/3) from dual;

    DUMP(1/3)
    -----------------------------------------------------------------------------
    Typ=2 Len=21: 192,34,34,34,34,34,34,34,34,34,34,34,34,34,34,34,34,34,34,34,34

    SQL> select dump(1/3*3) from dual;

    DUMP(1/3*3)
    -------------------------------------------------------------------------------------------------
    Typ=2 Len=21: 192,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100,100

    SQL> select dump(1) from dual;

    DUMP(1)
    ------------------
    Typ=2 Len=2: 193,2

    SQL> set numw 100000
    SP2-0267: numwidth option 100000 out of range (2 through 50)

    由实验可以看出number 最大能精确到小数点后面50位。

  • 相关阅读:
    luoguP2016 战略游戏
    [Usaco2006 Nov]Corn Fields牧场的安排
    [Ahoi2009]self 同类分布
    POJ3208:Apocalypse Someday
    [usaco2010 Oct]Soda Machine
    [Usaco2005 Dec]Scales 天平
    PTA的Python练习题(十九)
    堆叠注入
    PHP序列化与反序列化(三)总结实战
    攻防世界web进阶1-12总结篇
  • 原文地址:https://www.cnblogs.com/hehe520/p/6330611.html
Copyright © 2011-2022 走看看