zoukankan      html  css  js  c++  java
  • scala slick mysql utf8mb4 支持

    语言  scala 

    sql包  slick 3.2.0

    数据库  mysql

    https://stackoverflow.com/questions/36741141/scala-slick-jdbc-encoding-configuration-with-typesafe-config/50153514

    简代版  https://github.com/cclient/ScalaMysqlSync 

    数据库同步程序报错

    java.sql.SQLException: Incorrect string value: 'xF0x9Fx87xB5xF0x9F...' for column 'content' at row 1

    这个错误见的太多,emoji字符和utf-8不兼容所致,utf8mp4字符集支持emoji

    源库(另外一家公司提供)改了表的字符集,未通知我方,从库和源库字符集不同,同步时部分字符报错

    改mysql,改同步程序字符集即可

    改mysql简单

    ALTER TABLE table  CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

    难在改程序的同步集,这个是个大坑,不同语言,不同访问方式的解决方式不一样

    该同步服务用scala实现,引用的slick,没有任何资料,用小众的东西就是遇到坑,很多时候得独立解决

    传统java项目思路 都是ini sql,不过ini方式不同。

     试了两种方案

    1 连接先执行,不生效

    val DEST_DB = Database.forURL("jdbc:mysql://127.0.0.1:3306/table?characterEncoding=UTF-8&charset=utf8mb4", user = "root", password = "123456", driver = "com.mysql.jdbc.Driver")
    
    ORIGIN_DB.run(sql"""SET NAMES utf8mb4 COLLATE utf8mb4_unicode_ci""".as[Int])

    2连接初始化,这是蒙的,也不生效

    val ORIGIN_DB = Database.forURL("jdbc:mysql://127.0.0.1:3306/table?charset=utf8mb4;INIT="SET NAMES utf8mb4 COLLATE utf8mb4_unicode_ci"", user = "root", password = "123456", driver = "com.mysql.jdbc.Driver",executor = AsyncExecutor("orgindb", minThreads=1, queueSize=10,maxThreads=10,maxConnections=10),keepAliveConnection=true)

    暂时走到死胡同了

    查slick文档也没有任何相关资料,但是注意到一个关键字  HikariCP

    这是个sql连接池

    之前开发spring-boot spring-cloud 项目接触过,也碰上过HikariCP utf8mb4的相关问题

     http://www.cnblogs.com/zihunqingxin/p/8650470.html

    https://github.com/brettwooldridge/HikariCP 支持 connectionInitSql 配置

    spring-boot当时是通过解决 

    spring.datasource.hikari.connection-init-sql=SET NAMES utf8mb4 COLLATE utf8mb4_unicode_ci;

    既然不能直接解决utf8mb4的问题,那通过HikariCP连接池,倒可能解决该问题

    配置文件设置,官网上信息很少,得参照代码实现设置

    https://github.com/slick/slick/blob/master/slick-hikaricp/src/main/scala/slick/jdbc/hikaricp/HikariCPJdbcDataSource.scala#L13

    application.conf

    mysql_orginal = {
    connectionPool = "HikariCP"
    driver = "com.mysql.jdbc.Driver"
    connectionInitSql ="SET NAMES utf8mb4 COLLATE utf8mb4_unicode_ci"
    url = "jdbc:mysql://127.0.0.1/table?useUnicode=true&characterEncoding=UTF-8"
    keepAliveConnection = true
    user = "root"
    password = "123456"
    numThreads = 4
    poolName="mysql_orginal_pool"
    }
    再 val ORIGIN_DB = Database.forConfig("mysql_orginal")

    编译通过,但执行报错

    java.lang.ExceptionInInitializerError
        at main.tables.LastActor$$anonfun$receive$1.applyOrElse(last.scala:53)
        at akka.actor.Actor.aroundReceive(Actor.scala:513)
        at akka.actor.Actor.aroundReceive$(Actor.scala:511)
        at main.tables.LastActor.aroundReceive(last.scala:31)
        at akka.actor.ActorCell.receiveMessage(ActorCell.scala:527)
        at akka.actor.ActorCell.invoke(ActorCell.scala:496)
        at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
        at akka.dispatch.Mailbox.run(Mailbox.scala:224)
        at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
        at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
        at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
        at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
        at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
    Caused by: java.lang.ClassNotFoundException: slick.jdbc.hikaricp.HikariCPJdbcDataSource$

    错误表示连接池生效,只是找不到相关类

    引入依赖

    libraryDependencies +=  "com.typesafe.slick" %% "slick-hikaricp" % "3.2.0"

    重新编译

    执行 utf8mb4 问题解决

     

    其他参考资料
    http://slick.lightbend.com/doc/3.2.0/database.html#databaseconfig
    https://github.com/slick/slick/blob/6ac40e3b7f241475a122eb575f7ce0286080dd52/doc/code/application.conf
    https://github.com/slick/slick/blob/4674b977d9eeaf825d57808d48dbbfc37e47858c/doc/src/config.md

  • 相关阅读:
    Java 编程规范
    Java常考面试题
    SQL 实战
    快速排序
    剑指Offer(51-67)
    剑指Offer(41-50)
    移动端图片编辑器
    css隐藏和显示table的第一列
    sweetAlert1 设置弹窗宽度,及使用自定义样式
    js获取yyyy-mm-dd hh:mm:ss格式的当前系统时间
  • 原文地址:https://www.cnblogs.com/zihunqingxin/p/8981931.html
Copyright © 2011-2022 走看看