数据结构

简介

特点

架构

20200930-205650-0124.png

安装方式

常用命令

mysql -uroot -p
show databases;
use xxx(databasename);
show tables;
DESC xxx(datatable);
SELECT VERSION(),CURRENT_DATE(),CURRENT_TIME();
show processlist;
show variables like '%tx_isolation%';
show engine innodb status\G;
DDL
create database test1;
drop database test1;
alter table emp modify ename varchar(20);
alter table emp add column age int(3);alter table emp drop column age;
DML
insert into emp(ename,sal,deptno) values('zhangsan','2015-08-01','2000',1);insert into emp(ename,sal,deptno) values('lisi','2015-08-01','3000',1);create table dept(deptno int(3),deptname varchar(20);insert into dept values(1,'tech'),(2,'sales'),(3,'fin');
update emp set sal=4000 where ename='lisi';delete from emp where ename='lisi';
select ename,hiredate,sal,deptno from emp;select distinct deptno from emp;
select * from emp where deptno =1 and sal<3000;select * from emp order by sal;
select count(1) from emp;select deptno,count(1) as empnum from emp group by deptno
select deptno,count(1) as empnum from emp group by deptno with rollup;select deptno,count(1) as empnum from emp group by deptno having count(1)>1;
select sum(sal),max(sal),min(sal) from emp;select ename,deptname from emp,dept where emp.deptno=dept.deptno;

存储引擎

show engines;+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+| Engine             | Support | Comment                                                        | Transactions | XA   | Savepoints |+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+| InnoDB             | DEFAULT | Supports transactions, row-level locking, and foreign keys     | YES          | YES  | YES        || CSV                | YES     | CSV storage engine                                             | NO           | NO   | NO         || MyISAM             | YES     | MyISAM storage engine                                          | NO           | NO   | NO         || BLACKHOLE          | YES     | /dev/null storage engine (anything you write to it disappears) | NO           | NO   | NO         || PERFORMANCE_SCHEMA | YES     | Performance Schema                                             | NO           | NO   | NO         || MRG_MYISAM         | YES     | Collection of identical MyISAM tables                          | NO           | NO   | NO         || ARCHIVE            | YES     | Archive storage engine                                         | NO           | NO   | NO         || MEMORY             | YES     | Hash based, stored in memory, useful for temporary tables      | NO           | NO   | NO         || FEDERATED          | NO      | Federated MySQL storage engine                                 | NULL         | NULL | NULL       |+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+9 rows in set (0.03 sec)
  1. 数据和索引合并存储为一个文件,.frm(描述表的结构) .ibd(表数据文件)
  2. 支持外键,事务处理
  3. 行锁定
  4. 具有提交、回滚和崩溃恢复能力的事务安全
  5. 并行读写,适用于大量的写操作的表

备份

备份所有数据库

mysqldump -uroot -p --all-databases > itsca.sql
[root@xdja wch]# mysqldump -uroot -p --all-databases > itsca.sqlEnter password: Warning: A partial dump from a server that has GTIDs will by default include the GTIDs of all transactions, even those that changed suppressed parts of the database. If you don't want to restore GTIDs, pass --set-gtid-purged=OFF. To make a complete dump, pass --all-databases --triggers --routines --events.

备份指定数据库

mysqldump -u root -p --databases 数据库1 数据库2 > xxx.sql

还原 mysql 备份内容

mysql -uroot -p123456 < /data/mysqlDump/mydb.sql
mysql> source /data/mysqlDump/mydb.sql

基础优化

存储引擎的选择、字段设计、索引、SQL 语句等都是影响 MySQL 性能的重要因素,本次暂不详细讨论。

仅从 MySQL 数据库参数配置入手,探究设置合理的参数值来提升 MySQL 数据库性能。

innodb_page_size

https://dev.mysql.com/doc/refman/5.7/en/optimizing-innodb-diskio.html

innodb_buffer_pool_size

https://dev.mysql.com/doc/refman/5.7/en/innodb-parameters.html#sysvar_innodb_buffer_pool_size

sync_binlog

https://dev.mysql.com/doc/refman/5.7/en/replication-options-binary-log.html#sysvar_sync_binlog

innodb_flush_log_at_trx_commit

https://dev.mysql.com/doc/refman/5.7/en/innodb-parameters.html#sysvar_innodb_flush_log_at_trx_commit

innodb_lru_scan_depth

https://www.cnblogs.com/zengkefu/p/5692803.html
https://dev.mysql.com/doc/refman/5.7/en/innodb-parameters.html#sysvar_innodb_lru_scan_depth

innodb_io_capacity

https://dev.mysql.com/doc/refman/5.7/en/optimizing-innodb-diskio.html

innodb_io_capacity_max

https://dev.mysql.com/doc/refman/5.7/en/optimizing-innodb-diskio.html

https://dev.mysql.com/doc/refman/5.7/en/innodb-configuring-io-capacity.html

innodb_max_dirty_pages_pct

  1. REDO 日志快满的时候。
  2. 为了保证 MySQL 中的空闲页面的数量,会从 LRU 链表尾部淘汰一部分页面作为空闲页。如果对应的页面是脏页的话,就需要先将页面刷到磁盘。
  3. MySQL
  4. MySQL 实例正常关闭时候。

https://blog.csdn.net/w892824196/article/details/107411404

innodb_flush_neighbors

  1. 设置为 0 时,表示刷脏页时不刷其附近的脏页。
  2. 设置为 1 时,表示刷脏页时连带其附近毗连的脏页一起刷掉。
  3. 设置为 2 时,表示刷脏页时连带其附近区域的脏页一起刷掉。1 与 2 的区别是 2 刷的区域更大一些。

https://www.mytecdb.com/blogDetail.php?id=117
https://dev.mysql.com/doc/refman/5.7/en/innodb-parameters.html#sysvar_innodb_flush_neighbors

innodb_page_cleaners

https://www.jianshu.com/p/6991304a8e26
https://www.jianshu.com/p/ddb24f9afae0

https://dev.mysql.com/doc/refman/5.7/en/innodb-parameters.html#sysvar_innodb_flush_neighbors

innodb_log_file_size

https://blog.csdn.net/kai404/article/details/80242262

https://dev.mysql.com/doc/refman/5.7/en/innodb-parameters.html#sysvar_innodb_log_file_size

优化结果

[mysqld]########basic settings########server-id = 11port = 3306user = mysql#bind_address = 10.166.224.32   #根据实际情况修改autocommit = 1   #5.6.X安装时,需要注释掉,安装完成后再打开character_set_server=utf8mb4skip_name_resolve = 1max_connections = 800max_connect_errors = 1000datadir = /home/mysql/data      #根据实际情况修改,建议和程序分离存放transaction_isolation = READ-COMMITTED#explicit_defaults_for_timestamp = 1join_buffer_size = 134217728tmp_table_size = 67108864tmpdir = /tmpmax_allowed_packet = 16777216sql_mode = "STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION,NO_ZERO_DATE,NO_ZERO_IN_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER"interactive_timeout = 1800wait_timeout = 1800read_buffer_size = 16777216read_rnd_buffer_size = 33554432sort_buffer_size = 33554432########log settings########log_error = /home/mysql/logs/error.logslow_query_log = 1slow_query_log_file = /home/mysql/logs/slow.loglog_queries_not_using_indexes = 1log_slow_admin_statements = 1log_slow_slave_statements = 1log_throttle_queries_not_using_indexes = 10expire_logs_days = 60long_query_time = 1min_examined_row_limit = 100lower_case_table_names = 1 ########replication settings########master_info_repository = TABLErelay_log_info_repository = TABLElog_bin = /home/mysql/binlog/bin.logsync_binlog = 1gtid_mode = onenforce_gtid_consistency = 1log_slave_updatesbinlog_format = rowrelay_log = /home/mysql/relaylog/relay.logrelay_log_recovery = 1binlog_gtid_simple_recovery = 1slave_skip_errors = ddl_exist_errors########innodb settings#########innodb_page_size = 8192innodb_buffer_pool_size = 24G    #根据实际情况修改innodb_buffer_pool_instances = 12innodb_buffer_pool_load_at_startup = 1innodb_buffer_pool_dump_at_shutdown = 1innodb_lru_scan_depth = 2000innodb_lock_wait_timeout = 5innodb_io_capacity = 4000innodb_io_capacity_max = 8000innodb_flush_method = O_DIRECTinnodb_file_format = Barracudainnodb_file_format_max = Barracudainnodb_log_group_home_dir = /home/mysql/redolog/  #根据实际情况修改innodb_undo_directory = /home/mysql/undolog/      #根据实际情况修改innodb_undo_logs = 128innodb_undo_tablespaces = 3innodb_flush_neighbors = 1innodb_log_file_size = 8G               #根据实际情况修改innodb_log_buffer_size = 16777216innodb_purge_threads = 4innodb_large_prefix = 1innodb_thread_concurrency = 64innodb_print_all_deadlocks = 1innodb_strict_mode = 1innodb_sort_buffer_size = 67108864innodb_flush_log_at_trx_commit = 1[mysqld-5.7]innodb_buffer_pool_dump_pct = 40innodb_page_cleaners = 12innodb_undo_log_truncate = 1innodb_max_undo_log_size = 2Ginnodb_purge_rseg_truncate_frequency = 128binlog_gtid_simple_recovery=1log_timestamps=systemtransaction_write_set_extraction=MURMUR32show_compatibility_56=on

http://freshmeat.sourceforge.net/projects/fio/

[root@xdja wch]# tar -zxvf fio-2.1.10.tar.gz [root@xdja wch]# cd fio-2.1.10/[root@xdja fio-2.1.10]# ./configure[root@xdja fio-2.1.10]# make[root@xdja fio-2.1.10]# make Install[root@xdja fio-2.1.10]# cd ..[root@xdja wch]# fio -filename=test0628 -direct=1 -iodepth 1 -thread -rw=randrw -ioengine=psync -bs=16k -size=500M -numjobs=10 -runtime=10 -group_reporting -name=mytestdirect=1 测试过程绕过机器自带的buffer,使测试结果更真实rw=randrw 测试随机混合写和读的I/Oioengine=psync io引擎使用pync方式bs=16k 单次io的块文件大小为16knumjobs=10 本次的测试线程为10
mytest: (g=0): rw=randrw, bs=16K-16K/16K-16K/16K-16K, ioengine=psync, iodepth=1...fio-2.1.10Starting 10 threadsmytest: Laying out IO file(s) (1 file(s) / 500MB)Jobs: 10 (f=10): [mmmmmmmmmm] [100.0% done] [1328KB/896KB/0KB /s] [83/56/0 iops] [eta 00m:00s]mytest: (groupid=0, jobs=10): err= 0: pid=23464: Mon Jun 28 12:52:55 2021  read : io=11632KB, bw=1156.2KB/s, iops=72, runt= 10061msec    clat (msec): min=1, max=273, avg=84.36, stdev=55.83     lat (msec): min=1, max=273, avg=84.36, stdev=55.83    clat percentiles (msec):     |  1.00th=[    5],  5.00th=[   12], 10.00th=[   18], 20.00th=[   31],     | 30.00th=[   48], 40.00th=[   62], 50.00th=[   77], 60.00th=[   94],     | 70.00th=[  113], 80.00th=[  130], 90.00th=[  159], 95.00th=[  186],     | 99.00th=[  249], 99.50th=[  258], 99.90th=[  273], 99.95th=[  273],     | 99.99th=[  273]    bw (KB  /s): min=   56, max=  191, per=10.04%, avg=116.07, stdev=29.91  write: io=11424KB, bw=1135.5KB/s, iops=70, runt= 10061msec    clat (usec): min=277, max=205220, avg=54698.78, stdev=44032.60     lat (usec): min=277, max=205221, avg=54699.64, stdev=44032.65    clat percentiles (usec):     |  1.00th=[  828],  5.00th=[ 1272], 10.00th=[ 1672], 20.00th=[ 3472],     | 30.00th=[23168], 40.00th=[37632], 50.00th=[53504], 60.00th=[64768],     | 70.00th=[79360], 80.00th=[92672], 90.00th=[112128], 95.00th=[134144],     | 99.00th=[171008], 99.50th=[193536], 99.90th=[205824], 99.95th=[205824],     | 99.99th=[205824]    bw (KB  /s): min=   29, max=  206, per=10.19%, avg=115.64, stdev=38.23    lat (usec) : 500=0.21%, 750=0.21%, 1000=0.69%    lat (msec) : 2=4.93%, 4=5.27%, 10=3.68%, 20=5.55%, 50=18.18%    lat (msec) : 100=34.35%, 250=26.44%, 500=0.49%  cpu          : usr=0.01%, sys=0.06%, ctx=1492, majf=0, minf=7  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%     issued    : total=r=727/w=714/d=0, short=r=0/w=0/d=0     latency   : target=0, window=0, percentile=100.00%, depth=1Run status group 0 (all jobs):   READ: io=11632KB, aggrb=1156KB/s, minb=1156KB/s, maxb=1156KB/s, mint=10061msec, maxt=10061msec  WRITE: io=11424KB, aggrb=1135KB/s, minb=1135KB/s, maxb=1135KB/s, mint=10061msec, maxt=10061msecDisk stats (read/write):    dm-2: ios=719/1496, merge=0/0, ticks=60164/81876, in_queue=142913, util=100.00%, aggrios=727/1144, aggrmerge=0/370, aggrticks=61291/63730, aggrin_queue=125075, aggrutil=100.00%  sda: ios=727/1144, merge=0/370, ticks=61291/63730, in_queue=125075, util=100.00%
innodb_buffer_pool_size =6G
innodb_buffer_pool_instances = 1
innodb_lru_scan_depth = 200
innodb_io_capacity = 100
innodb_io_capacity_max = 200
innodb_log_file_size = 4G
innodb_page_cleaners= 1

其他一些概念

问题汇总

Deadlock found when trying to get lock; try restarting transaction; nested exception is com.mysql.jdbc.exceptions.jdbc4.MySQLTransactionRollbackException: Deadlock found when trying to get lock; try restarting transaction

https://cloud.tencent.com/developer/article/1710549

优化参考文档


↙↙↙阅读原文可查看相关链接,并与作者交流