Table Design and MySQL Index Details

It's not often I get to work with a true homemade database design. In this case the customer (or their developer to be accurate) had designed the whole database backend of their website from scratch. However, problems arose as usual: An overly powerful server seeing sustained high loads and CPU usage by MySQL—which is actually an understatement given that the load would spike to 50+. After enabling slow query logging and examining the log a day later one query stood out in terms of frequency:

# Query_time: 7 Lock_time: 1 Rows_sent: 2 Rows_examined: 45023
SELECT * FROM hints WHERE game_id = 374 ORDER BY date DESC;

Seven seconds is bad for a query this simple, as is examining 45,000 rows to return 2. Furthermore, this query was logged over 4,000 times in one day, meaning MySQL spent about 28,000 seconds a day on this query, or 7 hours. Let's first consider design and later we'll see how it effects the details of indexes. (You may notice the numbers between examples don't match; this is because some of the examples were really taken from the production server and others were re-created lab experiements.)

      mysql> DESCRIBE hints;
      +------------+------------------+------+-----+---------+----------------+
      | Field      | Type             | Null | Key | Default | Extra          |
      +------------+------------------+------+-----+---------+----------------+
      | hints_id   | int(10) unsigned |      | PRI | NULL    | auto_increment |
      | game_id    | int(11)          |      |     | 0       |                |
      | hint_title | text             |      |     |         |                |
      | hint       | text             |      |     |         |                |
      | status     | text             |      |     |         |                |
      | date       | text             |      |     |         |                |
      +------------+------------------+------+-----+---------+----------------+
   

Unless there's some special condition I was not made aware of, the date column should not be type TEXT. What complicates the issue more is that the data in the date column is not normalized. That is, there are different representations of data (dates) in the column: Some dates are typical YYYY-MM-DD format and others are Unix timestamps. The first issue this creates is increased code complexity: Having to account for data that may be in different formats. Secondly, it's space inefficient. A date as TEXT will require 10 + 2 bytes or, times 45,000 rows, 540k. As a DATE column type, 3 bytes or 135k. The third issue involves fixinig the query: How to index a date column as TEXT. Indexes on TEXT columns require a prefix length; that is, since TEXT columns are truly variable you have to tell MySQL how much of it you want to index. In this case, since the data is not normalized and everything counts down to the last second for dates and times, we would have to index 10 bytes, another 450k for the index (minus compression). Finally, it's error prone: A DATE column type will reliaibly have a date. A TEXT column may have a DATE and a recipe for biscuits. Perhaps I'm just being overly zealous and picky; afterall, what's another 450k here or there, or a little more code to determine if the date is DATE or a Unix timestamp? Nothing, actually, but the point is: Why add these complications when they're completely unnecessary? Just use a DATE colum type. In short: KISS.

Index Details

      mysql> SHOW INDEXES FROM hints;
      +-------+------------+----------+--------------+-------------+-------------+----------+