一、基本概念
大数据也称之为LOB(Large Objects),LOB又分为:clob和blob,clob用于存储大文本,blob用于存储二进制数据,例如图像、声音、二进制文等。
但是,在mysql中没有提供tinyclob、clob、mediumclob、longclob四种类型,而是使用如下四种类型来处理文本大数据:
二、使用JDBC处理MySQL的二进制数据
2.1 修改mysql的配置,默认储存对象最大为4M,这里需要调整:
在my.ini中添加如下配置,然后重启
max_allowed_packet=10485760
否则会被如下异常:
2.2 创建表存储MP3
CREATE TABLE testblob ( id INT PRIMARY KEY AUTO_INCREMENT, mp3 LONGBLOB );
2.3 编写测试代码如下:
package cn.zy.test; import java.io.FileInputStream; import java.io.FileOutputStream; import java.io.InputStream; import java.io.OutputStream; import java.sql.Blob; import java.sql.Connection; import java.sql.PreparedStatement; import java.sql.ResultSet; import javax.sql.rowset.serial.SerialBlob; import org.apache.commons.io.IOUtils; import org.junit.Test; import cn.zy.utils.JdbcUtils; public class LobTest { @Test /* * 将MP3保存到数据库 */ public void addBlob(){ Connection conn = null; PreparedStatement st = null; ResultSet rs = null; try { conn = JdbcUtils.getConnection(); String sql = "insert into testblob(mp3) values(?)"; st = conn.prepareStatement(sql); //把文件转换成流 byte[] bytes = IOUtils.toByteArray(new FileInputStream("F:/十年.mp3")); // 使用byte[]创建Blob Blob blob = new SerialBlob(bytes); st.setBlob(1, blob); int num = st.executeUpdate(); if (num>0) { System.out.println("插入成功"); } } catch (Exception e) { e.printStackTrace(); }finally { JdbcUtils.release(conn, st, rs); } } @Test /* * 读取mysql中保存的二进制数据 */ public void readBlob(){ Connection conn = null; PreparedStatement st = null; ResultSet rs = null; try { conn = JdbcUtils.getConnection(); String sql = "select * from testblob where id=?"; st = conn.prepareStatement(sql); st.setInt(1, 1); rs = st.executeQuery(); if (rs.next()){ Blob blob = rs.getBlob("mp3"); InputStream in = blob.getBinaryStream(); OutputStream out = new FileOutputStream("c:/十年.mp3"); IOUtils.copy(in, out); } } catch (Exception e) { e.printStackTrace(); }finally{ JdbcUtils.release(conn, st, rs); } } }
查询表:
三、使用JDBC处理MySQL的大文本
1. 将nohup.txt放到src下面
2. 创建表
CREATE TABLE testclob
(
id INT PRIMARY KEY AUTO_INCREMENT,
RESUME TEXT
);
3.代码如下:
package cn.zy.test; import java.io.File; import java.io.FileReader; import java.io.FileWriter; import java.io.Reader; import java.sql.Connection; import java.sql.PreparedStatement; import java.sql.ResultSet; import org.junit.Test; import cn.zy.utils.JdbcUtils; public class LobTextTest { /* * 向数据库中插入大文本数据 */ @Test public void add(){ Connection conn = null; PreparedStatement st = null; ResultSet rs = null; Reader reader = null; try { conn = JdbcUtils.getConnection(); String sql = "insert into testclob(resume) values(?)"; st = conn.prepareStatement(sql); //这种方式获取的路径,其中的空格会被使用“%20”代替 String path = LobTextTest.class.getClassLoader().getResource("nohup.txt").getPath(); //将“20%”替换回空格 path = path.replaceAll("%20", " "); File file = new File(path); reader = new FileReader(file); st.setCharacterStream(1, reader,(int)file.length()); int num = st.executeUpdate(); if(num>0){ System.out.println("插入成功!"); } //关闭流 reader.close(); } catch (Exception e) { e.printStackTrace(); }finally { JdbcUtils.release(conn, st, rs); } } /* * 读取mysql中的文本数据 */ @Test public void read(){ Connection conn = null; PreparedStatement st = null; ResultSet rs = null; File file = null; Reader reader = null; try { conn = JdbcUtils.getConnection(); String sql = "select * from testclob where id=?"; st = conn.prepareStatement(sql); st.setInt(1, 1); rs = st.executeQuery(); String contentStr =""; String content = ""; if (rs.next()){ //使用resultSet.getString("字段名")获取大文本数据的内容 content = rs.getString("resume"); //使用resultSet.getCharacterStream("字段名")获取大文本数据的内容 reader = rs.getCharacterStream("resume"); char buffer[] = new char[1024]; int len = 0; FileWriter out = new FileWriter("D:\1.txt"); while((len=reader.read(buffer))>0){ contentStr += new String(buffer); out.write(buffer, 0, len); } out.close(); reader.close(); } System.out.println(content); System.out.println("-----------------------------------------------"); System.out.println(contentStr); } catch (Exception e) { e.printStackTrace(); }finally{ JdbcUtils.release(conn, st, rs); } } }
查看mysql中插入的数据: