Saturday, May 23, 2009

Importance of EXPLAIN Syntax in Mysql

Syntex:
EXPLAIN emp(table name)
EXPLAIN SELECT * from emp (Query)
EXPLAIN is a synonym for DESCRIBE table name .
When we use SELECT statement with the keyword EXPLAIN, MySQL explains how it would process the SELECT, and provide the information about How it works with other table in join.
It provide information about indexes with the help of EXPLAIN, you can see when you must add indexes to tables to get a faster SELECT that uses indexes to find the records. You can also see if the optimizer joins the tables in an optimal order.
For non-simple joins, EXPLAIN returns a row of information for each table used in the SELECT statement. The tables are listed in the order they would be read. MySQL resolves all joins using a single-sweep multi-join method. This means that MySQL reads a row from the first table, then finds a matching row in the second table, then in the third table and so on. When all tables are processed, it outputs the selected columns and backtracks through the table list until a table is found for which there are more matching rows. The next row is read from this table and the process continues with the next table.
Output from EXPLAIN includes the following columns:
table
The table to which the row of output refers.
type
The join type. Information about the various join types.
possible_keys
The possible_keys column indicates which indexes MySQL could use to find the rows in this table. Note that this column is totally independent of the order of the tables. That means that some of the keys in possible_keys may not be usable in practice with the generated table order. If this column is empty, there are no relevant indexes. In this case, you may be able to improve the performance of your query by examining the WHERE clause to see if it refers to some column or columns that would be suitable for indexing. If so, create an appropriate index and check the query with EXPLAIN again. ALTER TABLE Syntax. To see what indexes a table has, use SHOW INDEX FROM tbl_name.
key
The key column indicates the key that MySQL actually decided to use. The key is NULL if no index was chosen. If MySQL chooses the wrong index, you can probably force MySQL to use another index by using myisamchk --analyze, myisamchk Invocation Syntax, or by using USE INDEX/IGNORE INDEX. .
key_len
The key_len column indicates the length of the key that MySQL decided to use. The length is NULL if the key is NULL. Note that this tells us how many parts of a multi-part key MySQL will actually use.
ref
The ref column shows which columns or constants are used with the key to select rows from the table.
rows
The rows column indicates the number of rows MySQL believes it must examine to execute the query.
Extra
This column contains additional information of how MySQL will resolve the query. Here is an explanation of the different text strings that can be found in this column:
Distinct
MySQL will not continue searching for more rows for the current row combination after it has found the first matching row.
Not exists
MySQL was able to do a LEFT JOIN optimization on the query and will not examine more rows in this table for the previous row combination after it finds one row that matches the LEFT JOIN criteria. Here is an example for this:
SELECT * FROM t1 LEFT JOIN t2 ON t1.id=t2.id WHERE t2.id IS NULL;
Assume that t2.id is defined with NOT NULL. In this case MySQL will scan t1 and look up the rows in t2 through t1.id. If MySQL finds a matching row in t2, it knows that t2.id can never be NULL, and will not scan through the rest of the rows in t2 that has the same id. In other words, for each row in t1, MySQL only needs to do a single lookup in t2, independent of how many matching rows there are in t2.
range checked for each record (index map: #)
MySQL didn't find a real good index to use. It will, instead, for each row combination in the preceding tables, do a check on which index to use (if any), and use this index to retrieve the rows from the table. This isn't very fast but is faster than having to do a join without an index.
Using filesort
MySQL will need to do an extra pass to find out how to retrieve the rows in sorted order. The sort is done by going through all rows according to the join type and storing the sort key + pointer to the row for all rows that match the WHERE. Then the keys are sorted. Finally the rows are retrieved in sorted order.
Using index
The column information is retrieved from the table using only information in the index tree without having to do an additional seek to read the actual row. This can be done when all the used columns for the table are part of the same index.
Using temporary
To resolve the query MySQL will need to create a temporary table to hold the result. This typically happens if you do an ORDER BY on a different column set than you did a GROUP BY on.
Where used
A WHERE clause will be used to restrict which rows will be matched against the next table or sent to the client. If you don't have this information and the table is of type ALL or index, you may have something wrong in your query (if you don't intend to fetch/examine all rows from the table).
If you want to get your queries as fast as possible, you should look out for Using filesort and Using temporary.
The different join types are listed below, ordered from best to worst type:
system
The table has only one row (= system table). This is a special case of the const join type.
const
The table has at most one matching row, which will be read at the start of the query. Because there is only one row, values from the column in this row can be regarded as constants by the rest of the optimizer. const tables are very fast as they are read only once!
eq_ref
One row will be read from this table for each combination of rows from the previous tables. This is the best possible join type, other than the const types. It is used when all parts of an index are used by the join and the index is UNIQUE or a PRIMARY KEY.
ref
All rows with matching index values will be read from this table for each combination of rows from the previous tables. ref is used if the join uses only a leftmost prefix of the key, or if the key is not UNIQUE or a PRIMARY KEY (in other words, if the join cannot select a single row based on the key value). If the key that is used matches only a few rows, this join type is good.
range
Only rows that are in a given range will be retrieved, using an index to select the rows. The key column indicates which index is used. The key_len contains the longest key part that was used. The ref column will be NULL for this type.
index
This is the same as ALL, except that only the index tree is scanned. This is usually faster than ALL, as the index file is usually smaller than the data file.
ALL
A full table scan will be done for each combination of rows from the previous tables. This is normally not good if the table is the first table not marked const, and usually very bad in all other cases. You normally can avoid ALL by adding more indexes, so that the row can be retrieved based on constant values or column values from earlier tables.
You can get a good indication of how good a join is by multiplying all values in the rows column of the EXPLAIN output. This should tell you roughly how many rows MySQL must examine to execute the query. This number is also used when you restrict queries with the max_join_size variable.
The following example shows how a JOIN can be optimized progressively using the information provided by EXPLAIN.
Suppose you have the SELECT statement shown below, that you examine using EXPLAIN:
EXPLAIN SELECT tt.TicketNumber, tt.TimeIn,
tt.ProjectReference, tt.EstimatedShipDate,
tt.ActualShipDate, tt.ClientID,
tt.ServiceCodes, tt.RepetitiveID,
tt.CurrentProcess, tt.CurrentDPPerson,
tt.RecordVolume, tt.DPPrinted, et.COUNTRY,
et_1.COUNTRY, do.CUSTNAME
FROM tt, et, et AS et_1, do
WHERE tt.SubmitTime IS NULL
AND tt.ActualPC = et.EMPLOYID
AND tt.AssignedPC = et_1.EMPLOYID
AND tt.ClientID = do.CUSTNMBR;
For this example, assume that:
The columns being compared have been declared as follows:
Table
Column
Column type
tt
ActualPC
CHAR(10)
tt
AssignedPC
CHAR(10)
tt
ClientID
CHAR(10)
et
EMPLOYID
CHAR(15)
do
CUSTNMBR
CHAR(15)
The tables have the indexes shown below:
Table
Index
tt
ActualPC
tt
AssignedPC
tt
ClientID
et
EMPLOYID (primary key)
do
CUSTNMBR (primary key)
The tt.ActualPC values aren't evenly distributed.
Initially, before any optimizations have been performed, the EXPLAIN statement produces the following information:
table type possible_keys key key_len ref rows Extra
et ALL PRIMARY NULL NULL NULL 74
do ALL PRIMARY NULL NULL NULL 2135
et_1 ALL PRIMARY NULL NULL NULL 74
tt ALL AssignedPC,ClientID,ActualPC NULL NULL NULL 3872
range checked for each record (key map: 35)
Because type is ALL for each table, this output indicates that MySQL is doing a full join for all tables! This will take quite a long time, as the product of the number of rows in each table must be examined! For the case at hand, this is 74 * 2135 * 74 * 3872 = 45,268,558,720 rows. If the tables were bigger, you can only imagine how long it would take.
One problem here is that MySQL can't (yet) use indexes on columns efficiently if they are declared differently. In this context, VARCHAR and CHAR are the same unless they are declared as different lengths. Because tt.ActualPC is declared as CHAR(10) and et.EMPLOYID is declared as CHAR(15), there is a length mismatch.
To fix this disparity between column lengths, use ALTER TABLE to lengthen ActualPC from 10 characters to 15 characters:
mysql> ALTER TABLE tt MODIFY ActualPC VARCHAR(15);
Now tt.ActualPC and et.EMPLOYID are both VARCHAR(15). Executing the EXPLAIN statement again produces this result:
table type possible_keys key key_len ref rows Extra
tt ALL AssignedPC,ClientID,ActualPC NULL NULL NULL 3872 where used
do ALL PRIMARY NULL NULL NULL 2135
range checked for each record (key map: 1)
et_1 ALL PRIMARY NULL NULL NULL 74
range checked for each record (key map: 1)
et eq_ref PRIMARY PRIMARY 15 tt.ActualPC 1
This is not perfect, but is much better (the product of the rows values is now less by a factor of 74). This version is executed in a couple of seconds.
A second alteration can be made to eliminate the column length mismatches for the tt.AssignedPC = et_1.EMPLOYID and tt.ClientID = do.CUSTNMBR comparisons:
mysql> ALTER TABLE tt MODIFY AssignedPC VARCHAR(15),
MODIFY ClientID VARCHAR(15);
Now EXPLAIN produces the output shown below:
table type possible_keys key key_len ref rows Extra
et ALL PRIMARY NULL NULL NULL 74
tt ref AssignedPC,ClientID,ActualPC ActualPC 15 et.EMPLOYID 52 where used
et_1 eq_ref PRIMARY PRIMARY 15 tt.AssignedPC 1
do eq_ref PRIMARY PRIMARY 15 tt.ClientID 1
This is almost as good as it can get.
The remaining problem is that, by default, MySQL assumes that values in the tt.ActualPC column are evenly distributed, and that isn't the case for the tt table. Fortunately, it is easy to tell MySQL about this:
shell> myisamchk --analyze PATH_TO_MYSQL_DATABASE/tt
shell> mysqladmin refresh
Now the join is perfect, and EXPLAIN produces this result:
table type possible_keys key key_len ref rows Extra
tt ALL AssignedPC,ClientID,ActualPC NULL NULL NULL 3872 where used
et eq_ref PRIMARY PRIMARY 15 tt.ActualPC 1
et_1 eq_ref PRIMARY PRIMARY 15 tt.AssignedPC 1
do eq_ref PRIMARY PRIMARY 15 tt.ClientID 1
Note that the rows column in the output from EXPLAIN is an educated guess from the MySQL join optimizer. To optimize a query, you should check if the numbers are even close to the truth. If not, you may get better performance by using STRAIGHT_JOIN in your SELECT statement and trying to list the tables in a different order in the FROM clause.

Wednesday, May 20, 2009

Optimizing MySQL Queries-Part 1

Indexing Basic MySQL Queries
The database is just too slow. Queries are queuing up, backlogs growing, users being refused connection. Management is ready to spend millions on "upgrading" to some other system, when the problem is really that MySQL is simply not being used properly. Badly defined or non-existent indexes are one of the primary reasons for poor performance, and fixing these can often lead to phenomenal improvements. Indexes increase the performance of your query but one thing you should know that if indexes are not defined properly then it can decrease your performance.

So Let’s see the example:
First we must understand why the query is slow in the mind of MySQL and the only way to do that is to have MySQL explain it to us:
MySQL EXPLAIN
-> SELECT attach_id as total_attachment FROM exp
-> WHERE post_id IN (163246, 164224, 164894, 165146, 167931);
+-------------------+-------+---------------+-------------------+---------+------+-------+--------------------------+
| table | type | possible_keys | key | key_len | ref | rows | Extra |
+-------------------+-------+---------------+-------------------+---------+------+-------+--------------------------+
| exp | index | NULL | attach_id_post_id | 6 | NULL | 61834 | Using where; Using index |
+-------------------+-------+---------------+-------------------+---------+------+-------+--------------------------+

It's a thing of beauty to see inside the mind of the machine, but in this case perhaps not. Why is MySQL telling us there's no possible keys but it's using key attach_id_post_id? And if it's using a key then why does it suspect it will have to examine 61,834 rows (by performing a full index scan, denoted by "type: index")? It seems there's a problem with the keys so we must now understand them:
MySQL DESCRIBE exp;
+-------------+-----------------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+-----------------------+------+-----+---------+-------+
| attach_id | mediumint(8) unsigned | | MUL | 0 | |
| post_id | mediumint(8) unsigned | | | 0 | |
| privmsgs_id | mediumint(8) unsigned | | | 0 | |
| user_id_1 | mediumint(8) | | | 0 | |
| user_id_2 | mediumint(8) | | | 0 | |
+-------------+-----------------------+------+-----+---------+-------+

MySQL> SHOW INDEX FROM exp;
+-------------------+------------+-----------------------+--------------+-------------+-----------+-------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality |
+-------------------+------------+-----------------------+--------------+-------------+-----------+-------------+
| exp | 1 | attach_id_post_id | 1 | attach_id | A | 61834 |
| exp | 1 | attach_id_post_id | 2 | post_id | A | 61834 |
| exp | 1 | attach_id_privmsgs_id | 1 | attach_id | A | 61834 |
| exp | 1 | attach_id_privmsgs_id | 2 | privmsgs_id | A | 61834 |

Understanding indexes (or keys if you prefer) is two part: Understanding the structure of the table then understanding the indexes. You can't just slap an index on a table and think everything will be wonderful. In this example it looks like everything should be wonderful with key attach_id_post_id. Given that the SELECT statement is selecting attach_id and post_id and that's just what this key indexes, so why isn't it working? It is working, just not how we're intending; it's working for MySQL which is why in EXPLAIN it says "Using index." When MySQL says this in "Extra" is means "The column information is retrieved from the table using only information in the index tree without having to do an additional seek to read the actual row." In other words: It finds and returns matching columns from the index in memory not the table on disk, which is a good thing, unless it's doing this 12 million times for 1 matching column.

How very annoying: MySQL is using the index but still in effect examing every row of the table. The reason why in this example deals with how MySQL uses multiple column indexes. From DESCRIBE we see "MUL" for multi-column index, and from SHOW INDEX we see attach_id_post_id twice, first for attach_id second for post_id. A multiple column index acts like a single column index if the columns were put end-to-end in the order specified by "Seq_in_index" from SHOW INDEX. In this example if attach_id were 100 and post_id were 200 this is indexed as "100 200". Painfully simple correct? Throw this in the mix: MySQL will only use a multi-column index if a value is specified for the first column in the index. In this example the first column in the index is attach_id and we're not specifying a value for this column which is why MySQLwon't use the index like we want it to. What MySQLdoes do, and why it's able to use the index at all, is use any value for attach_id and the values we gave it for post_id. In effect it looks for '* 163246', '* 164224', '* 164894', '* 165146', and '* 167931'. Since attach_id is unique MySQLreally does have to look at every single one, all 61,00+. While doing that if it comes across one with a matching post_id lucky for us. I hope you see the obvious and simple solution: Swap the order of columns in the key, post_id then attach_id. Later we'll do this but first it's good learning to examine another possibility.

Wednesday, April 29, 2009

What's New in the Next Generation of MySQL Cluster?

Whether you’re racing to introduce a new service, or trying to manage an avalanche of data in real time, your database has to be scalable, fast and highly available to meet ever-changing market conditions and stringent SLAs.

By attending this webinar, you will learn more about MySQL Cluster 7, and how it enables you to deliver 99.999% database availability, with real time performance and linear scalability, while slashing TCO.

Boasting a range of breakthrough capabilities, the MySQL Cluster 7 will enable you to stay ahead of your most demanding, mission-critical application requirements. Enhanced scalability delivers higher database performance with fewer nodes to simplify deployment and administration. Expanded platform support and interoperability delivers more choice in both the development and deployment of MySQL Cluster. Simplified cluster monitoring tools reduce DBA administration overhead and operational costs.
WHO:

* Andrew Morgan, MySQL Product Management
* Matthew Keep, MySQL Product Management

WHAT: What's New in the Next Generation of MySQL Cluster? web presentation.

WHEN:
Thursday, April 30, 2009: 10:00 Pacific time (America)
Thu, Apr 30: 07:00 Hawaii time
Thu, Apr 30: 11:00 Mountain time (America)
Thu, Apr 30: 12:00 Central time (America)
Thu, Apr 30: 13:00 Eastern time (America)
Thu, Apr 30: 17:00 UTC
Thu, Apr 30: 18:00 Western European time
Thu, Apr 30: 19:00 Central European time
Thu, Apr 30: 20:00 Eastern European time


The presentation will be approximately 45 minutes long followed by Q&A.
WHERE: Simply access the web seminar from the comfort of your own office.

Mat Keep
MySQL Product Marketing
Sun Microsystems, Inc

Tuesday, April 21, 2009

Mysql cluster 7.0

Sunmicro System is announcing the General Availability (GA) of MySQL Cluster 7.0, a new version of the industry’s leading real-time database combining the flexibility of a high availability relational database with the low TCO of open source. MySQL Cluster 7.0 provides new features and management tools that deliver a powerful, more manageable way to meet business challenges.

MySQL Cluster 7.0 new features include:

* Enhanced scalability - delivers over 4x higher database performance with fewer nodes to simplify deployment and administration.
* Expanded platform support and interoperability - delivers more choice in both the development and deployment of MySQL Cluster.
* Flexible online backup - snapshots reduce DBA administration overhead and operational costs.

"We view MySQL Cluster Carrier Grade Edition as a strategic technology in our applications portfolio. With MySQL Cluster CGE 7.0 we are enabling our customers to expose the rich capabilities of their networks to drive a long-tailed innovation model. As a result, the operator can leverage the creativity and agility of the web ecosystem to deliver new compelling, personalized and targeted services to their subscribers without compromising reliability and scalability."
-- Jeff Cortley, VP Subscriber Data Management Alcatel-Lucent

MySQL Cluster Customers
As our valued customer, you will continue to receive maximum database performance, scalability, and availability at a cost that's 10x1 less than proprietary offerings. The newest version of MySQL Cluster 7.0 is available for download here:
http://dev.mysql.com/downloads/cluster/

You can learn more about upgrading to MySQL Cluster 7 here:
http://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-upgrade-downgrade.html

For assistance in evaluating or upgrading to MySQL Cluster 7.0, please do not hesitate to contact your MySQL account team. You can reach them via the contact mechanisms below

Learn more about the new MySQL Cluster 7.0:
- Download the new whitepaper: “Building a Carrier-Grade Platform for Data Management
http://www.mysql.com/why-mysql/white-papers/mysql_wp_cluster7_open_cgp.php

- New Technical Whitepaper: MySQL Cluster 7.0, Architecture and New Features
http://www.mysql.com/why-mysql/white-papers/mysql_wp_cluster7_architecture.php

- Register for the April 30 webinar, "MySQL Cluster 7, What's New" http://www.mysql.com/news-and-events/web-seminars/display-320.html

- To speak to a MySQL representative about how you can leverage MySQL Cluster to lower your database TCO, contact your MySQL sales representative online at http://www.mysql.com/about/contact/ or at the number for your region below:

USA - Toll Free: +1-866-221-0634
Latin America: +1 512 535 7751
UK: +44 845 399 1124
Ireland: +353 1 6919191
Germany: +49 89 420 95 98 95
France: +33 1 70 61 48 95
Sweden: +46 730 207 871
Benelux: +358 50 5710 528
Italy: +39 06-99268193
Israel: +358 50 5710 528
Spain & Portugal: + 34 933905461
Other EMEA countries: +353 1 6919191
Asia Pacific: +81 3 5843 1140

Regards,

The MySQL Team
Sun Microsystems

Article is on behalf of SUNMICRO SYSTEM

Thursday, April 16, 2009

Developers and DBA

Hello Everybody...
Please send your resume at amitabh.mishra83@gmail.com If you are a Mysql DBA or Developer.

Mysql Jobs


Welcome to all companies which are recruiting Mysql developers and DBA.
Please post your jobs here and I will provide you candidates.

Tuesday, January 15, 2008

Mysql

Replication enables data from one MySQL database server (called the master) to be replicated to one or more MySQL database servers (slaves). Replication is asynchronous - your replication slaves do not need to be connected permanently to receive updates from the master, which means that updates can occur over long-distance connections and even temporary solutions such as a dial-up service. Depending on the configuration, you can replicate all databases, selected databases, or even selected tables within a database.
The target uses for replication in MySQL include:
Scale-out solutions - spreading the load among multiple slaves to improve performance. In this environment, all writes and updates must take place on the master server. Reads, however, may take place on one or more slaves. This model can improve the performance of writes (since the master is dedicated to updates), while dramatically increasing read speed across an increasing number of slaves.
Data security - because data is replicated to the slave, and the slave can pause the replication process, it is possible to run backup services on the slave without corrupting the corresponding master data.
Analytics - live data can be created on the master, while the analysis of the information can take place on the slave without affecting the performance of the master.
Long-distance data distribution - if a branch office would like to work with a copy of your main data, you can use replication to create a local copy of the data for their use without requiring permanent access to the master.
Replication in MySQL features support for one-way, asynchronous replication, in which one server acts as the master, while one or more other servers act as slaves. This is in contrast to the synchronous replication which is a characteristic of MySQL Cluster (see
Chapter 16, MySQL Cluster).
There are a number of solutions available for setting up replication between two servers, but the best method to use depends on the presence of data and the engine types you are using. For more information on the available options, see
Section 15.1.1, “How to Set Up Replication”.
Replication is controlled through a number of different options and variables. These control the core operation of the replication, timeouts and the databases and filters that can be applied on databases and tables. For more information on the available options, see
Section 15.1.2, “Replication Startup Options and Variables”.
You can use replication to solve a number of different problems, including problems with performance, supporting the backup of different databases and for use as part of a larger solution to alleviate system failures. For information on how to address these issues, see
Section 15.2, “Replication Solutions”.
For notes and tips on how different data types and statements are treated during replication, including details of replication features, version compatibility, upgrades, and problems and their resolution, including an FAQ, see
Section 15.3, “Replication Notes and Tips”.

Powered by mysql