zoukankan      html  css  js  c++  java
  • to improve sqlite performance

    INSERT is really slow - I can only do few dozen INSERTs per second

    http://www.sqlite.org/faq.html#q19

    Actually, SQLite will easily do 50,000 or more INSERT statements per second on an average desktop computer. But it will only do a few dozen transactions per second. Transaction speed is limited by the rotational speed of your disk drive. A transaction normally requires two complete rotations of the disk platter, which on a 7200RPM disk drive limits you to about 60 transactions per second.

    Transaction speed is limited by disk drive speed because (by default) SQLite actually waits until the data really is safely stored on the disk surface before the transaction is complete. That way, if you suddenly lose power or if your OS crashes, your data is still safe. For details, read about atomic commit in SQLite..

    By default, each INSERT statement is its own transaction. But if you surround multiple INSERT statements with BEGIN...COMMIT then all the inserts are grouped into a single transaction. The time needed to commit the transaction is amortized over all the enclosed insert statements and so the time per insert statement is greatly reduced.

    Another option is to run PRAGMA synchronous=OFF. This command will cause SQLite to not wait on data to reach the disk surface, which will make write operations appear to be much faster. But if you lose power in the middle of a transaction, your database file might go corrupt.

    http://blogs.msdn.com/b/andy_wigley/archive/2013/11/21/how-to-massively-improve-sqlite-performance-using-sqlwinrt.aspx

    http://blog.quibb.org/2010/08/fast-bulk-inserts-into-sqlite/

    http://www.codeproject.com/Articles/853842/Csharp-Avoiding-Performance-Issues-with-Inserts-in

    http://stackoverflow.com/questions/1711631/improve-insert-per-second-performance-of-sqlite

    threadsafe / sqlite in multithread

    https://www.sqlite.org/threadsafe.html

    I read everywhere that creating transactions is the solution to slow SQLite writes, but it can be long and painful to rewrite your code and wrap all your SQLite writes in transactions.

    I found a much simpler, safe and very efficient method: I enable a (disabled by default) SQLite 3.7.0 optimisation : the Write-Ahead-Log (WAL). The documentation says it works in all unix (i.e. Linux and OSX) and Windows systems.

    How ? Just run the following commands after initializing your SQLite connection:

    PRAGMA journal_mode = WAL
    PRAGMA synchronous = NORMAL

    My code now runs ~600% faster : my test suite now runs in 38 seconds instead of 4 minutes :)

  • 相关阅读:
    css将元素float和textalign方向一致时
    Asp.Net细节性问题技巧精萃
    ajax单个元素中实现多个回调函数
    javascript实现Observer模式来管理多个事件回调
    管闲事的小明
    字符串扩展
    (转的)计算组合数——整数拆分
    数学题杭电1002A+B。。。交了七遍,五遍竟然是没删除测试时候的输出= =。。。。
    快速排序。。。注意快速排序不适宜解决有重复数据的排序
    哈希查找统计元音
  • 原文地址:https://www.cnblogs.com/zyip/p/4941827.html
Copyright © 2011-2022 走看看