Logging requests with MySQL

July 12th 2008 03:49 pm

One of my recent work projects was publishing a widget which others can add to their websites by copying some code and using it on their site. Similar to how YouTube makes it easy to embed their videos elsewhere.

Of course we want to know how and where this widget is being used so we can decide if it’s worth doing more widgets in the future, so we needed to include tracking. What we want to know is where the widget is being used, and how many times it is being displayed on those URLs.

The tracking table is very simple:

CREATE TABLE widget_impressions
(
	id INT(11) NOT NULL AUTO_INCREMENT,
        referral_url VARCHAR(255) NOT NULL,
        counter INT(11) NOT NULL,
        date_created DATE NOT NULL,
        PRIMARY KEY  (referral_url,date_created),
        UNIQUE KEY id (id)
)

Technically, the id column is not needed since the primary key is a combination of the referral_url and the date_viewed field, but it can be handy in some situations.

When the widget is displayed, the following query is run:

INSERT INTO widget_impressions
	SET
		referral_url = 'http://example.com',
		date_viewed = NOW(),
		counter = 1
	ON DUPLICATE KEY UPDATE
		counter = counter + 1;

If this is the first visit from http://example.com for the day, a new record will be added to the table. If it’s not the first record for the day, the counter will be incremented by one. Using the date type for the date_viewed field means only the date is stored, not the time. If a datetime field were used, this method would not work the same way; we’d have separate entries in the table every time someone viewed the widget.

With this table structure it’s very easy to get stats. Say you want to know all of the pages where the widget is displayed and how many times it has been displayed on each:

SELECT referral_url, SUM(counter) as num_views
FROM widget_impressions
GROUP BY referral_url;

Want to know the 10 sites that are displaying your widget the most?

SELECT referral_url, SUM(counter) as num_views
FROM widget_impressions
GROUP BY referral_url
ORDER BY num_views DESC
LIMIT 10;

References:

Posted by bradym under MySQL |

No Responses to “Logging requests with MySQL”

  1. Mark responded on 12 Jul 2008 at 7:59 pm #

    The only problem I see with your code is for organizations that require hourly statistics of usage. ie. What does the usage look like for the hours of 5pm to midnight. Other than that I applaud your simplicity in design and effort in getting the Database to do some of the lifting instead of bloating the page code.

  2. bradym responded on 12 Jul 2008 at 8:25 pm #

    Yeah, this definitely isn’t a do-all-end-all solution to logging. For more fine-grained reporting like that, you’d want to write a separate entry to the db table (or a text log, that is then loaded into a database for analysis, depending on expected usage) for every action.

    Even with this more complex, fine-grained log you can easily make the database do the heavy lifting (or at least most of it) for analysis.

Comments RSS

Leave a Reply