zoukankan      html  css  js  c++  java
  • Kea DHCP MySQL

    1. 简介

    kea是下一代的dhcp技术,也是ISC开发的。它支持ipv4和ipv6的地址分配,并且还可以支持动态dns更新。kea从2014年初开始研发,也算是比较新的技术了,并且现在也在更新中。


     
    kea

    现在工作中遇到一些问题。新到货的服务器要配置带外ip,之前使用老的DHCP方法根据mac地址绑定IP,但是这样做总感觉不太好。
    第一、每个机房都有一个配置文件,管理起来不方便;
    第二、当我需要查询绑定的结果时,查询起来也不方便;
    第三、配置文件毕竟是文件格式,安全性不太好保障。
    因为遇到了上面的种种问题,我就想,有没有什么方法可以把mac与ip的记录放到数据库中。这样更安全可靠好管理,然后在凯哥的指点下,学会了kea这个dhcp服务。
    感觉非常好,唯一不足的就是有些很实用的功能需要花钱购买,还贵,还是dollar。

    1.1 支持的平台

    根据官方的说法,他支持的平台比较多,例如Red Hat Enterprise Linux, CentOS, Fedora 和 FreeBSD等。现在比较新的Kea版本是Kea 1.4.0 。在很多系统上测试都没问题。并且centos上安装epel源,可以直接yum安装kea(yum install kea),只不过版本比较低。

    2. 安装

    这里的安装过程,我就按照我的方法来讲了,有兴趣多了解的同学可以去官网查看。

    2.1 安装数据库

    [root@test01 ~]# cat /etc/centos-release
    CentOS Linux release 7.4.1708 (Core)
    [root@test01 ~]# yum install mariadb mariadb-server -y
    [root@test01 ~]# systemctl start mariadb.service
    [root@test01 ~]# systemctl enable mariadb.service
    Created symlink from /etc/systemd/system/multi-user.target.wants/mariadb.service to /usr/lib/systemd/system/mariadb.service.
    [root@test01 ~]# mysql_secure_installation
    

    2.2 安装必要的依赖

    [root@test01 ~]# yum install gcc gcc-c++ openssl-devel mariadb-devel libtool automake autoconf -y
    

    2.3 编译安装必要的依赖

    安装Boost C++库,要求版本在1.57以上,不然可能工作的不太好。
    这个安装可能比较慢,要有耐心。

    [root@test01 ~]# wget https://dl.bintray.com/boostorg/release/1.65.1/source/boost_1_65_1.tar.gz
    [root@test01 ~]# tar -zxvf boost_1_65_1.tar.gz
    [root@test01 ~]# cd boost_1_65_1/
    [root@test01 boost_1_65_1]# ./bootstrap.sh
    [root@test01 boost_1_65_1]# ./b2 install
    

    安装log4cplus,要求版本在1.0.3以上。

    [root@test01 ~]# wget https://jaist.dl.sourceforge.net/project/log4cplus/log4cplus-stable/2.0.1/log4cplus-2.0.1.tar.gz
    [root@test01 ~]# tar -zxvf log4cplus-2.0.1.tar.gz
    [root@test01 ~]# cd log4cplus-2.0.1/
    [root@test01 log4cplus-2.0.1]# ./configure
    [root@test01 log4cplus-2.0.1]# make
    [root@test01 log4cplus-2.0.1]# make install
    

    2.4 安装kea

    这里也非常慢,要有耐心。

    [root@test01 ~]# wget http://ftp.isc.org/isc/kea/1.4.0/kea-1.4.0.tar.gz
    [root@test01 ~]# tar -zxvf kea-1.4.0.tar.gz
    [root@test01 ~]# cd kea-1.4.0/
    [root@test01 kea-1.4.0]# ./configure --with-mysql
    [root@test01 kea-1.4.0]# make
    [root@test01 kea-1.4.0]# make install
    

    3. 配置

    3.1 简单的配置验证

    初始化数据库

    [root@test01 kea-1.4.0]# mysql -u root  -p
    Enter password: 
    ...
    
    MariaDB [(none)]> CREATE DATABASE kea;
    Query OK, 1 row affected (0.01 sec)
    
    MariaDB [(none)]> CREATE USER 'kea'@'%' IDENTIFIED BY 'kea';
    Query OK, 0 rows affected (0.02 sec)
    
    MariaDB [(none)]> GRANT ALL ON kea.* TO 'kea'@'%';
    Query OK, 0 rows affected (0.00 sec)
    
    MariaDB [(none)]> quit
    Bye
    [root@test01 kea-1.4.0]# kea-admin lease-init mysql -u kea -p kea -n kea
    Checking if there is a database initialized already. Please ignore errors.
    Initializing database using script /usr/local/share/kea/scripts/mysql/dhcpdb_create.mysql
    mysql returned status code 0
    Lease DB version reported after initialization: 6.0
    

    因为我的测试环境没有ipv6,所以去掉ipv6服务的启动。

    [root@test01 ~]# cd /usr/local/etc/kea/
    [root@test01 kea]# vim keactrl.conf 
    dhcp6=no
    

    配置dhcpv4的网络端口。

    [root@test01 kea]# ip a
    1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
        link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
        inet 127.0.0.1/8 scope host lo
           valid_lft forever preferred_lft forever
    2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
        link/ether 00:0c:29:9b:7b:dc brd ff:ff:ff:ff:ff:ff
        inet 192.168.1.30/24 brd 192.168.1.255 scope global ens33
           valid_lft forever preferred_lft forever
    3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
        link/ether 00:0c:29:9b:7b:e6 brd ff:ff:ff:ff:ff:ff
        inet 192.168.212.1/24 brd 192.168.212.255 scope global ens37
           valid_lft forever preferred_lft forever
    [root@test01 kea]# vim kea-dhcp4.conf 
    "interfaces-config": {
            "interfaces": ["ens37"]
        },
    

    配置IP地址段

    [root@test01 kea]# vim kea-dhcp4.conf
    "subnet4": [
    {
          "subnet": "192.168.212.0/24",
          "pools": [ { "pool": "192.168.212.100 - 192.168.212.200" } ],
          "option-data": [ { "name": "routers", "data": "192.168.212.201" } ]
    }
    ]
    

    启动kea服务并测试

    [root@test01 kea]# keactrl start
    INFO/keactrl: Starting /usr/local/sbin/kea-dhcp4 -c /usr/local/etc/kea/kea-dhcp4.conf
    INFO/keactrl: kea-ctrl-agent appears to be running, see: PID 97955, PID file: /usr/local/var/kea/kea-ctrl-agent.kea-ctrl-agent.pid.
    [root@test01 kea]# keactrl status
    DHCPv4 server: active
    DHCPv6 server: inactive
    DHCP DDNS: inactive
    Control Agent: active
    Kea DHCPv4 configuration file: /usr/local/etc/kea/kea-dhcp4.conf
    Kea DHCPv6 configuration file: /usr/local/etc/kea/kea-dhcp6.conf
    Kea DHCP DDNS configuration file: /usr/local/etc/kea/kea-dhcp-ddns.conf
    Kea Control Agent configuration file: /usr/local/etc/kea/kea-ctrl-agent.conf
    keactrl configuration file: /usr/local/etc/kea/keactrl.conf
    

    客户端进行测试

    [root@test01 ~]# dhclient -v ens33
    Internet Systems Consortium DHCP Client 4.2.5
    Copyright 2004-2013 Internet Systems Consortium.
    All rights reserved.
    For info, please visit https://www.isc.org/software/dhcp/
    
    Listening on LPF/ens33/00:0c:29:f9:21:91
    Sending on   LPF/ens33/00:0c:29:f9:21:91
    Sending on   Socket/fallback
    DHCPDISCOVER on ens33 to 255.255.255.255 port 67 interval 6 (xid=0x62beba25)
    DHCPREQUEST on ens33 to 255.255.255.255 port 67 (xid=0x62beba25)
    DHCPOFFER from 192.168.212.1
    DHCPACK from 192.168.212.1 (xid=0x62beba25)
    bound to 192.168.212.100 -- renewal in 877 seconds.
    [root@test01 ~]# ip r
    default via 192.168.212.201 dev ens33 
    192.168.128.0/24 dev ens37 proto kernel scope link src 192.168.128.128 
    192.168.212.0/24 dev ens33 proto kernel scope link src 192.168.212.100 
    [root@test01 ~]#
    

    查看kea的日志

    [root@test01 kea]# tail -n2 /usr/local/var/log/kea-dhcp4.log
    2018-07-07 00:37:53.599 INFO  [kea-dhcp4.leases/97980] DHCP4_LEASE_ADVERT [hwtype=1 00:0c:29:f9:21:91], cid=[no info], tid=0x25babe62: lease 192.168.212.100 will be advertised
    2018-07-07 00:37:53.600 INFO  [kea-dhcp4.leases/97980] DHCP4_LEASE_ALLOC [hwtype=1 00:0c:29:f9:21:91], cid=[no info], tid=0x25babe62: lease 192.168.212.100 has been allocated
    

    至此,基本的功能已经完成

    3.2 数据库绑定ip

    配置数据库的连接地址

    [root@test01 kea]# pwd
    /usr/local/etc/kea
    [root@test01 kea]# vim kea-dhcp4.conf 
        "hosts-database": {
            "type": "mysql",
            "name": "kea",
            "user": "kea",
            "password": "kea",
            "host": "192.168.1.30",
            "port": 3306
        },
    

    将mac和ip绑定的配置插到数据库中

    [root@test01 kea]# cat mac_ip.sql 
    START TRANSACTION;
    ---要分配的IP地址
    SET @ipv4_reservation='192.168.212.99';
    ---要分配的主机名,有这个功能但是很少用
    SET @hostname = 'myhost.example.org';
    ---绑定的类型,一般情况是hw-address,代表mac地址
    SET @identifier_type='hw-address';
    ---客户端的mac地址
    SET @identifier_value='00:0C:29:F9:21:91';
    ---网络id,这个可以自己定义
    SET @dhcp4_subnet_id=1;
    ---PXE启动的时候配置的nextserver,一般不在这里配,但是也可以指定
    SET @next_server='192.168.212.1';
    ---nextserver的主机名
    SET @server_hostname='server-name.example.org';
    ---PXE启动的文件,一般是pxelinux.0;如果是efi启动模式的话,文件不一样
    SET @boot_file_name='bootfile.efi';
    
    INSERT INTO hosts (dhcp_identifier,
                       dhcp_identifier_type,
                       dhcp4_subnet_id,
                       ipv4_address,
                       hostname,
                       dhcp4_next_server,
                       dhcp4_server_hostname,
                       dhcp4_boot_file_name)
    VALUES (UNHEX(REPLACE(@identifier_value, ':', '')),
            (SELECT type FROM host_identifier_type WHERE name=@identifier_type),
            @dhcp4_subnet_id,
            INET_ATON(@ipv4_reservation),
            @hostname,
            INET_ATON(@next_server),
            @server_hostname,
            @boot_file_name);
    
    COMMIT;
    
    [root@test01 kea]# mysql -u kea -h 192.168.212.1 -p
    Enter password: 
    MariaDB [(none)]> use kea;
    MariaDB [kea]> source mac_ip.sql;
    
    MariaDB [kea]> SELECT
        ->     HEX(h.dhcp_identifier) AS dhcp_identifier,
        ->     i.name AS dhcp_identifier_name,
        ->     h.dhcp4_subnet_id AS dhcp4_subnet_id,
        ->     INET_NTOA(h.ipv4_address) AS ipv4_address,
        ->     h.hostname AS hostname
        -> FROM
        ->     hosts AS h
        ->         INNER JOIN
        ->     host_identifier_type AS i ON h.dhcp_identifier_type = i.type;
    +-----------------+----------------------+-----------------+----------------+--------------------+
    | dhcp_identifier | dhcp_identifier_name | dhcp4_subnet_id | ipv4_address   | hostname           |
    +-----------------+----------------------+-----------------+----------------+--------------------+
    | 000C29F92191    | hw-address           |               1 | 192.168.212.99 | myhost.example.org |
    +-----------------+----------------------+-----------------+----------------+--------------------+
    1 row in set (0.00 sec)
    
    MariaDB [kea]> exit
    Bye
    [root@test01 kea]# cd
    [root@test01 ~]# keactrl reload
    INFO/keactrl: Reloading kea-dhcp4...
    INFO/keactrl: kea-dhcp6 isn't running.
    INFO/keactrl: kea-dhcp-ddns isn't running.
    INFO/keactrl: Reloading kea-ctrl-agent...
    

    测试成功

    [root@test01 ~]# ip r
    default via 192.168.212.201 dev ens33 
    192.168.128.0/24 dev ens37 proto kernel scope link src 192.168.128.128 
    192.168.212.0/24 dev ens33 proto kernel scope link src 192.168.212.100 
    [root@test01 ~]# 
    [root@test01 ~]# 
    [root@test01 ~]# dhclient -r ens33
    [root@test01 ~]# ip r
    192.168.128.0/24 dev ens37 proto kernel scope link src 192.168.128.128 
    [root@test01 ~]# dhclient -v ens33
    Internet Systems Consortium DHCP Client 4.2.5
    Copyright 2004-2013 Internet Systems Consortium.
    All rights reserved.
    For info, please visit https://www.isc.org/software/dhcp/
    
    Listening on LPF/ens33/00:0c:29:f9:21:91
    Sending on   LPF/ens33/00:0c:29:f9:21:91
    Sending on   Socket/fallback
    DHCPDISCOVER on ens33 to 255.255.255.255 port 67 interval 6 (xid=0x514a221b)
    DHCPREQUEST on ens33 to 255.255.255.255 port 67 (xid=0x514a221b)
    DHCPOFFER from 192.168.212.1
    DHCPACK from 192.168.212.1 (xid=0x514a221b)
    bound to 192.168.212.99 -- renewal in 703 seconds.
    [root@test01 ~]# ip r
    default via 192.168.212.201 dev ens33 
    192.168.128.0/24 dev ens37 proto kernel scope link src 192.168.128.128 
    192.168.212.0/24 dev ens33 proto kernel scope link src 192.168.212.99 
    [root@test01 ~]#
    

    至此,数据里面加入mac和ip的绑定已经成功了

    3.3 PXE启动配置

    1. 环境测试。
      之前上面加入数据库的有next-server,所以给服务器pxe启动的时候可以看到这么一步:


       
       

      好!接下来我们配置pxe启动!
      首先搭建PXE环境,我上一篇文章有介绍ipxe。

    2. 搭建tftp服务。
    [root@test01 ~]# yum install tftp tftp-server xinetd
    [root@test01 ~]# cat /etc/xinetd.d/tftp 
    service tftp
    {
        socket_type     = dgram
        protocol        = udp
        wait            = yes
        user            = root
        server          = /usr/sbin/in.tftpd
        server_args     = -s /var/lib/tftpboot
        disable         = no
        per_source      = 11
        cps         = 100 2
        flags           = IPv4
    }
    [root@test01 ~]# cd /var/lib/tftpboot/
    [root@test01 tftpboot]# wget http://boot.ipxe.org/ipxe.efi
    [root@test01 tftpboot]# wget http://boot.ipxe.org/undionly.kpxe
    [root@test01 tftpboot]# systemctl enable xinetd
    [root@test01 tftpboot]# systemctl restart xinetd
    [root@test01 ~]# cd /tmp/
    [root@test01 tmp]# tftp
    (to) 192.168.212.1
    tftp> get undionly.kpxe
    tftp> quit
    [root@test01 tmp]# md5sum undionly.kpxe 
    758e2d856b69a94fb52cf0b1acc091c1  undionly.kpxe
    [root@test01 tmp]# md5sum /var/lib/tftpboot/undionly.kpxe 
    758e2d856b69a94fb52cf0b1acc091c1  /var/lib/tftpboot/undionly.kpxe
    
    1. 搭建http服务。
    [root@test01 ~]# yum install httpd
    [root@test01 ~]# vim /etc/httpd/conf/httpd.conf
    ServerAdmin root@192.168.212.1
    [root@test01 ~]# systemctl enable httpd.service
    [root@test01 ~]# systemctl start httpd.service
    [root@test01 ~]# cat /var/www/html/boot/boot_menu.php 
    #!ipxe
    kernel http://192.168.212.1/boot/vmlinuz initrd=initrd.img
    initrd http://192.168.212.1/boot/initrd.img
    boot
    [root@test01 ~]# cd /var/www/html/boot/
    [root@test01 boot]# wget http://mirrors.163.com/centos/6.10/isos/x86_64/CentOS-6.10-x86_64-netinstall.iso
    [root@test01 boot]# mount -o loop ./CentOS-6.10-x86_64-netinstall.iso /mnt/
    [root@test01 boot]# cd /mnt/isolinux/
    [root@test01 isolinux]# cp vmlinuz /var/www/html/boot/
    [root@test01 isolinux]# cp initrd.img /var/www/html/boot/
    [root@test01 isolinux]# cd /var/www/html/boot/
    [root@test01 boot]# umount /mnt/
    [root@test01 boot]# mv CentOS-6.10-x86_64-netinstall.iso /tmp/
    [root@test01 ~]# curl http://192.168.212.1/boot/boot_menu.php
    #!ipxe
    kernel http://192.168.212.1/boot/vmlinuz initrd=initrd.img
    initrd http://192.168.212.1/boot/initrd.img
    boot
    [root@test01 ~]#
    
    1. 删除之前的记录。
      环境搭建好了,理论上没问题了,删掉之前hosts记录。使用sql语句:
    DELETE FROM `kea`.`hosts` WHERE  `host_id`=1;
    
    1. 配置kea-dhcp4文件。
      让我们来指定next-server和filename。默认我们指定的是undionly.kpxe,如果在client-classes中匹配到其它的,那么优先client-classes里的。这个我之前的ipxe文章中介绍过
    "client-classes":
        [
          {
            "name": "XClient_iPXE",
            "test": "substring(option[77].hex,0,4) == 'iPXE'",
            "boot-file-name": "http://192.168.212.1/boot/boot_menu.php"
          },
          {
            "name": "HTTPClient",
            "test": "substring(option[60].hex,0,20) == 'PXEClient:Arch:00010'",
            "boot-file-name": "http://192.168.212.1/boot/boot_menu.php"
          },
          {
            "name": "UEFI-32-1",
            "test": "substring(option[60].hex,0,20) == 'PXEClient:Arch:00006'",
            "boot-file-name": "ipxe/i386/ipxe.efi"
          },
          {
            "name": "UEFI-32-2",
            "test": "substring(option[60].hex,0,20) == 'PXEClient:Arch:00002'",
            "boot-file-name": "ipxe/i386/ipxe.efi"
          },
          {
            "name": "UEFI-64-1",
            "test": "substring(option[60].hex,0,20) == 'PXEClient:Arch:00007'",
            "boot-file-name": "ipxe.efi"
          },
          {
            "name": "UEFI-64-2",
            "test": "substring(option[60].hex,0,20) == 'PXEClient:Arch:00008'",
            "boot-file-name": "ipxe.efi"
          },
          {
            "name": "UEFI-64-3",
            "test": "substring(option[60].hex,0,20) == 'PXEClient:Arch:00009'",
            "boot-file-name": "ipxe.efi"
          }
          // {
          //   "name": "Legacy-1",
          //   "test": "substring(option[60].hex,0,20) == 'PXEClient:Arch:00000'",
          //   "boot-file-name": "undionly.kpxe"
          // }
        ],
    
        "next-server": "192.168.212.1",
        "boot-file-name": "undionly.kpxe",
    
        "subnet4": [
            {
                "subnet": "192.168.212.0/24",
                "pools": [ { "pool": "192.168.212.100 - 192.168.212.200" } ],
                "option-data": [ { "name": "routers", "data": "192.168.212.1" }]
            }
        ]
    

    注:如果测试不成功,可以把配置文件中的注释和一些无关的配置删掉。这样文档看起来更清晰,当然这可能会让你不清楚每段配置的含义,没关系,你可以参考/usr/local/share/doc/kea/kea-guide.txt或是官方文档。

    1. 最后测试并验证。


       
       

      这个画面不好捕捉,为了好看我们可以做一个ipxe的界面。

    2. 修改启动文件,制作界面
    [root@test01 boot]# cat /var/www/html/boot/boot_menu.php 
    #!ipxe
       set menu-timeout 5000
       set menu-default PXE
       isset ${ip} || dhcp
       isset ${next-server} || set next-server 192.168.212.1
    :start
      menu iPXE Boot Menu
      item --gap --             -------------------------------- TOOL --------------------------------
      item PXE                  PXE Tool
      item --gap --             ---------------------------- Advanced options -----------------------
      item --key c config       Configure settings                                   -- c
      item shell                Drop to iPXE shell
      item  Local               Boot from local drive 0x80
      item reboot               Reboot computer
      item --key x exit         Exit iPXE and continue BIOS boot                     -- x
      choose --timeout ${menu-timeout} --default ${menu-default} selected
      imgfree
      goto ${selected}
    :shell
      echo Type 'exit' to get the back to the menu
      shell
      goto start
    :failed
      echo Booting failed, dropping to shell
      goto shell
    :Local
    sanboot --no-describe --drive 0x80
    goto failed
    :reboot
      reboot
    :exit
      exit
    :config
      config
      goto start
    :PXE
      kernel http://${next-server}/boot/vmlinuz initrd=initrd.img
      initrd http://${next-server}/boot/initrd.img || goto failed
      boot || goto failed
      goto start
    
    1. 重启得到如下界面。


       
       

    3.4 API接口配置

    Kea Control Agent是一个守护进程,它用来提供一个管理kea服务的接口。通过http请求传过来的参数,对kea服务进行操作管理,十分的方便。下面我们做一下基本的配置,并验证

    [root@test01 kea]# pwd
    /usr/local/etc/kea
    [root@test01 kea]# vim kea-ctrl-agent.conf 
    {
        "Control-agent": {
            "http-host": "192.168.1.30",
            "http-port": 8080,
    
            "control-sockets": {
                "dhcp4": {
                    "comment": "main server",
                    "socket-type": "unix",
                    "socket-name": "/path/to/the/unix/socket-v4"
                },
                "dhcp6": {
                    "socket-type": "unix",
                    "socket-name": "/path/to/the/unix/socket-v4",
                    "user-context": { "version": 3 }
                }
            },
    
            "hooks-libraries": [
            {
                "library": "/opt/local/control-agent-commands.so",
                "parameters": {
                    "param1": "foo"
                }
            } ]
        },
    
        "Logging": {
            "loggers": [ {
                "name": "kea-ctrl-agent",
                "severity": "INFO"
            } ]
        }
    }
    [root@test01 kea]# vim kea-dhcp4.conf
    "Dhcp4": {
        "control-socket": {
            "socket-type": "unix",
            "socket-name": "/tmp/kea-dhcp4-ctrl.sock"
        },
    
    

    测试

    [root@test01 kea]# curl -X POST -H "Content-Type: application/json" -d '{ "command": "config-get" }' http://192.168.1.30:8080/
    [ { "arguments": { "Control-agent": { "control-sockets": { "dhcp4": { "socket-name": "/tmp/kea-dhcp4-ctrl.sock", "socket-type": "unix" }, "dhcp6": { "socket-name": "/tmp/kea-dhcp6-ctrl.sock", "socket-type": "unix" } }, "hooks-libraries": [  ], "http-host": "192.168.1.30", "http-port": 8080 } }, "result": 0 } ]
    [root@test01 kea]# 
    
    

    3.5 更高级的功能知晓

    1.mac地址和ip绑定的接口。
    配置钩子:

    "Dhcp6": { 
        "hooks-libraries": [
            {
                "library": "/path/libdhcp_host_cmds.so"
            }
            ...
        ] 
    }
    

    使用方法:

    curl -X POST -H "Content-Type: application/json" -d '
    {
        "command": "reservation-add",
        "arguments": {
            "reservation": {
                "subnet-id": 1,
                "hw-address": "1a:1b:1c:1d:1e:1f",
                "ip-address": "192.0.2.202"
            }
        }
    ' 
    http://192.168.1.30:8080/
    }
    

    参考:https://kea.isc.org/docs/kea-guide.html#host-cmds

    1. 网段信息的接口管理。
      参考:https://kea.isc.org/docs/kea-guide.html#subnet-cmds
    2. 还有很多其它很好的很实用的功能,可续需要收费。


       
       

    4. 总结



    作者:小小运维
    链接:https://www.jianshu.com/p/b333c4271939
    来源:简书
    著作权归作者所有。商业转载请联系作者获得授权,非商业转载请注明出处。
  • 相关阅读:
    Redis Hashes 巧用sort排序
    Redis 压缩存储的配置
    计算
    关于时间大小判断的坑和网上工具类的看法
    Mysql中字段类型之时间戳大坑2
    Mysql中字段类型之时间戳大坑
    Spring和springmvc父子容器注解扫描问题详解
    JXL导出Excel工具类
    Maven学习
    MySQL之账户管理
  • 原文地址:https://www.cnblogs.com/studio313/p/11783822.html
Copyright © 2011-2022 走看看