Introduction
URL rewriting can be one of the best and quickest ways to improve the
usability and search friendliness of your site. It can also be the
source of near-unending misery and suffering. Definitely worth playing
carefully with it - lots of testing is recommended. With great power
comes great responsibility, and all that.
There are several other guides on the web already, that may suit your needs better than this one.
Before reading on, you may find it helpful to have the
mod_rewrite cheat sheet and/or the
regular expressions cheat sheet handy. A basic grasp of the concept of regular expressions would also be very helpful.
What is "URL Rewriting"?
Most dynamic sites include variables in their URLs that tell the site
what information to show the user. Typically, this gives URLs like the
following, telling the relevant script on a site to load product number
7.
http://www.pets.com/show_a_product.php?product_id=7
The problems with this kind of URL structure are that the URL is not
at all memorable. It's difficult to read out over the phone (you'd be
surprised how many people pass URLs this way). Search engines and users
alike get no useful information about the content of a page from that
URL. You can't tell from that URL that that page allows you to buy a
Norwegian Blue Parrot (lovely plumage). It's a fairly standard URL - the
sort you'd get by default from most CMSes. Compare that to this URL:
http://www.pets.com/products/7/
Clearly a much cleaner and shorter URL. It's much easier to remember,
and vastly easier to read out. That said, it doesn't exactly tell
anyone what it refers to. But we can do more:
http://www.pets.com/parrots/norwegian-blue/
Now we're getting somewhere. You can tell from the URL, even when
it's taken out of context, what you're likely to find on that page.
Search engines can split that URL into words (hyphens in URLs are
treated as spaces by search engines, whereas underscores are not), and
they can use that information to better determine the content of the
page. It's an easy URL to remember and to pass to another person.
Unfortunately, the last URL cannot be easily understood by a server
without some work on our part. When a request is made for that URL, the
server needs to work out how to process that URL so that it knows what
to send back to the user. URL rewriting is the technique used to
"translate" a URL like the last one into something the server can
understand.
Platforms and Tools
Depending on the software your server is running, you may already
have access to URL rewriting modules. If not, most hosts will enable or
install the relevant modules for you if you ask them very nicely.
Apache is the easiest system to get URL rewriting running on. It
usually comes with its own built-in URL rewriting module, mod_rewrite,
enabled, and working with mod_rewrite is as simple as uploading
correctly formatted and named text files.
IIS, Microsoft's server software, doesn't include URL rewriting
capability as standard, but there are add-ons out there that can provide
this functionality.
ISAPI_Rewrite
is the one I recommend working with, as I've so far found it to be the
closest to mod_rewrite's functionality. Instructions for installing and
configuring ISAPI_Rewrite can be found at the end of this article.
The code that follows is based on URL rewriting using mod_rewrite.
Basic URL Rewriting
To begin with, let's consider a simple example. We have a website,
and we have a single PHP script that serves a single page. Its URL is:
http://www.pets.com/pet_care_info_07_07_2008.php
We want to clean up the URL, and our ideal URL would be:
http://www.pets.com/pet-care/
In order for this to work, we need to tell the server to internally
redirect all requests for the URL "pet-care" to
"pet_care_info_07_07_2008.php". We want this to happen internally,
because we don't want the URL in the browser's address bar to change.
To accomplish this, we need to first create a text document called
".htaccess" to contain our rules. It must be named exactly that (not
".htaccess.txt" or "rules.htaccess"). This would be placed in the root
directory of the server (the same folder as
"pet_care_info_07_07_2008.php" in our example). There may already be an
.htaccess file there, in which case we should edit that rather than
overwrite it.
The .htaccess file is a configuration file for the server. If there
are errors in the file, the server will display an error message
(usually with an error code of "500"). If you are transferring the file
to the server using FTP, you must make sure it is transferred using the
ASCII mode, rather than BINARY. We use this file to perform 2 simple
tasks in this instance - first, to tell Apache to turn on the rewrite
engine, and second, to tell apache what rewriting rule we want it to
use. We need to add the following to the file:
RewriteEngine On # Turn on the rewriting engine
RewriteRule ^pet-care/?$ pet_care_info_01_02_2008.php [NC,L] # Handle requests for "pet-care"
A couple of quick items to note - everything following a hash symbol
in an .htaccess file is ignored as a comment, and I'd recommend you use
comments liberally; and the "RewriteEngine" line should only be used
once per .htaccess file (please note that I've not included this line
from here onwards in code example).
The "RewriteRule" line is where the magic happens. The line can be broken down into 5 parts:
- RewriteRule - Tells Apache that this like refers to a single RewriteRule.
- ^/pet-care/?$
- The "pattern". The server will check the URL of every request to the
site to see if this pattern matches. If it does, then Apache will swap
the URL of the request for the "substitution" section that follows.
- pet_care_info_01_02_2003.php - The "substitution". If the pattern above matches the request, Apache uses this URL instead of the requested URL.
- [NC,L]
- "Flags", that tell Apache how to apply the rule. In this case, we're
using two flags. "NC", tells Apache that this rule should be
case-insensitive, and "L" tells Apache not to process any more rules if
this one is used.
- # Handle requests for "pet-care" - Comment explaining what the rule does (optional but recommended)
The rule above is a simple method for rewriting a single URL, and is the basis for almost all URL rewriting rules.
Patterns and Replacements
The rule above allows you to redirect requests for a single URL, but
the real power of mod_rewrite comes when you start to identify and
rewrite groups of URLs based on patterns they contain.
Let's say you want to change all of your site URLs as described in
the first pair of examples above. Your existing URLs look like this:
http://www.pets.com/show_a_product.php?product_id=7
And you want to change them to look like this:
http://www.pets.com/products/7/
Rather than write a rule for every single product ID, you of course
would rather write one rule to manage all product IDs. Effectively you
want to change URLs of this format:
http://www.pets.com/show_a_product.php?product_id={a number}
And you want to change them to look like this:
http://www.pets.com/products/{a number}/
In order to do so, you will need to use "regular expressions". These
are patterns, defined in a specific format that the server can
understand and handle appropriately. A typical pattern to identify a
number would look like this:
[0-9]+
The square brackets contain a range of characters, and "0-9"
indicates all the digits. The plus symbol indicates that the pattern
will idenfiy one or more of whatever precedes the plus - so this pattern
effectively means "one or more digits" - exactly what we're looking to
find in our URL.
The entire "pattern" part of the rule is treated as a regular
expression by default - you don't need to turn this on or activate it at
all.
RewriteRule ^products/([0-9]+)/?$ show_a_product.php?product_id=$1 [NC,L] # Handle product requests
The first thing I hope you'll notice is that we've wrapped our
pattern in brackets. This allows us to "back-reference" (refer back to)
that section of the URL in the following "substitution" section. The
"$1" in the substitution tells Apache to put whatever matched the
earlier bracketed pattern into the URL at this point. You can have lots
of backreferences, and they are numbered in the order they appear.
And so, this RewriteRule will now mean that Apache redirects all
requests for domain.com/products/{number}/ to
show_a_product.php?product_id={same number}.
Regular Expressions
A complete guide to regular expressions is rather beyond the scope of
this article. However, important points to remember are that the entire
pattern is treated as a regular expression, so always be careful of
characters that are "special" characters in regular expressions.
The most instance of this is when people use a period in their
pattern. In a pattern, this actually means "any character" rather than a
literal period, and so if you want to match a period (and only a
period) you will need to "escape" the character - precede it with
another special character, a backslash, that tells Apache to take the
next character to be literal.
For example, this RewriteRule will not just match the URL "rss.xml"
as intended - it will also match "rss1xml", "rss-xml" and so on.
RewriteRule ^rss.xml$ rss.php [NC,L] # Change feed URL
This does not usually present a serious problem, but escaping
characters properly is a very good habit to get into early. Here's how
it should look:
RewriteRule ^rss\.xml$ rss.php [NC,L] # Change feed URL
This only applies to the pattern, not to the substitution. Other
characters that require escaping (referred to as "metacharacters")
follow, with their meaning in brackets afterwards:
- . (any character)
- * (zero of more of the preceding)
- + (one or more of the preceding)
- {} (minimum to maximum quantifier)
- ? (ungreedy modifier)
- ! (at start of string means "negative pattern")
- ^ (start of string, or "negative" if at the start of a range)
- $ (end of string)
- [] (match any of contents)
- - (range if used between square brackets)
- () (group, backreferenced group)
- | (alternative, or)
- \ (the escape character itself)
Using regular expressions, it is possible to search for all sorts of
patterns in URLs and rewrite them when they match. Time for another
example - we wanted earlier to be able to indentify this URL and rewrite
it:
http://www.pets.com/parrots/norwegian-blue/
And we want to be able to tell the server to interpret this as the following, but for all products:
http://www.pets.com/get_product_by_name.php?product_name=norwegian-blue
And we can do that relatively simply, with the following rule:
RewriteRule ^parrots/([A-Za-z0-9-]+)/?$ get_product_by_name.php?product_name=$1 [NC,L] # Process parrots
With this rule, any URL that starts with "parrots" followed by a
slash (parrots/), then one or more (+) of any combination of letters,
numbers and hyphens ([A-Za-z0-9-]) (note the hyphen at the end of the
selection of characters within square brackets - it must be added there
to be treated literally rather than as a range separator). We reference
the product name in brackets with $1 in the substitution.
We can make it even more generic, if we want, so that it doesn't
matter what directory a product appears to be in, it is still sent to
the same script, like so:
RewriteRule ^[A-Za-z-]+/([A-Za-z0-9-]+)/?$ get_product_by_name.php?product_name=$1 [NC,L] # Process all products
As you can see, we've replaced "parrots" with a pattern that matches
letter and hyphens. That rule will now match anything in the parrots
directory or any other directory whose name is comprised of at least one
or more letters and hyphens.
Flags
Flags are added to the end of a rewrite rule to tell Apache how to
interpret and handle the rule. They can be used to tell apache to treat
the rule as case-insensitive, to stop processing rules if the current
one matches, or a variety of other options. They are comma-separated,
and contained in square brackets. Here's a list of the flags, with their
meanings (this information is included on the
cheat sheet, so no need to try to learn them all).
- C (chained with next rule)
- CO=cookie (set specified cookie)
- E=var:value (set environment variable var to value)
- F (forbidden - sends a 403 header to the user)
- G (gone - no longer exists)
- H=handler (set handler)
- L (last - stop processing rules)
- N (next - continue processing rules)
- NC (case insensitive)
- NE (do not escape special URL characters in output)
- NS (ignore this rule if the request is a subrequest)
- P (proxy - i.e., apache should grab the remote content specified in the substitution section and return it)
- PT (pass through - use when processing URLs with additional handlers, e.g., mod_alias)
- R (temporary redirect to new URL)
- R=301 (permanent redirect to new URL)
- QSA (append query string from request to substituted URL)
- S=x (skip next x rules)
- T=mime-type (force specified mime type)
Moving Content
RewriteRule ^article/?$ http://www.new-domain.com/article/ [R,NC,L] # Temporary Move
Adding an "R" flag to the flags section changes how a RewriteRule
works. Instead of rewriting the URL internally, Apache will send a
message back to the browser (an HTTP header) to tell it that the
document has moved temporarily to the URL given in the "substitution"
section. Either an absolute or a relative URL can be given in the
substitution section. The header sent back includea a code - 302 - that
indicates the move is temporary.
RewriteRule ^article/?$ http://www.new-domain.com/article/ [R=301,NC,L] # Permanent Move
If the move is permanent, append "=301" to the "R" flag to have
Apache tell the browser the move is considered permanent. Unlike the
default "R", "R=301" will also tell the browser to display the new
address in the address bar.
This is one of the most common methods of rewriting URLs of items
that have moved to a new URL (for example, it is in use extensively on
this site to forward users to new post URLs whenever they are changed).
Conditions
Rewrite rules can be preceded by one or more rewrite conditions, and
these can be strung together. This can allow you to only apply certain
rules to a subset of requests. Personally, I use this most often when
applying rules to a subdomain or alternative domain as rewrite
conditions can be run against a variety of criteria, not just the URL.
Here's an example:
RewriteCond %{HTTP_HOST} ^addedbytes\.com [NC]
RewriteRule ^(.*)$ http://www.addedbytes.com/$1 [L,R=301]
The rewrite rule above redirects all requests, no matter what for, to
the same URL at "www.addedbytes.com". Without the condition, this rule
would create a loop, with every request matching that rule and being
sent back to itself. The rule is intended to only redirect requests
missing the "www" URL portion, though, and the condition preceding the
rule ensures that this happens.
The condition operates in a similar way to the rule. It starts with
"RewriteCond" to tell mod_rewrite this line refers to a condition.
Following that is what should actually be tested, and then the pattern
to test. Finally, the flags in square brackets, the same as with a
RewriteRule.
The string to test (the second part of the condition) can be a
variety of different things. You can test the domain being requested, as
with the above example, or you could test the browser being used, the
referring URL (commonly used to prevent hotlinking), the user's IP
address, or a variety of other things (see the "server variables"
section for an outline of how these work).
The pattern is almost exactly the same as that used in a RewriteRule,
with a couple of small exceptions. The pattern may not be interpreted
as a pattern if it starts with specific characters as described in the
following "exceptions" section. This means that if you wish to use a
regular expression pattern starting with <, >, or a hyphen, you
should escape them with the backslash.
Rewrite conditions can, like rewrite rules, be followed by flags, and
there are only two. "NC", as with rules, tells Apache to treat the
condition as case-insensitive. The other available flag is "OR". If you
only want to apply a rule if one of two conditions match, rather than
repeat the rule, add the "OR" flag to the first condition, and if either
match then the following rule will be applied. The default behaviour,
if a rule is preceded by multiple conditions, is that it is only applied
if all rules match.
Exceptions and Special Cases
Rewrite conditions can be tested in a few different ways - they do
not need to be treated as regular expression patterns, although this is
the most common way they are used. Here are the various ways rewrite
conditons can be processed:
- <Pattern (is test string lower than pattern)
- >Pattern (is test string greater than pattern)
- =Pattern (is test string equal to pattern)
- -d (is test string a valid directory)
- -f (is test string a valid file)
- -s (is test string a valid file with size greater than zero)
- -l (is test string a symbolic link)
- -F (is test string a valid file, and accessible (via subrequest))
- -U (is test string a valid URL, and accessible (via subrequest))
Server Variables
Server variables are a selection of items you can test when writing
rewrite conditions. This allows you to apply rules based on all sorts of
request parameters, including browser identifiers, referring URL or a
multitude of other strings. Variables are of the following format:
%{VARIABLE_NAME}
And "VARIABLE_NAME" can be replaced with any one of the following items:
- HTTP Headers
- HTTP_USER_AGENT
- HTTP_REFERER
- HTTP_COOKIE
- HTTP_FORWARDED
- HTTP_HOST
- HTTP_PROXY_CONNECTION
- HTTP_ACCEPT
- Connection Variables
- REMOTE_ADDR
- REMOTE_HOST
- REMOTE_USER
- REMOTE_IDENT
- REQUEST_METHOD
- SCRIPT_FILENAME
- PATH_INFO
- QUERY_STRING
- AUTH_TYPE
- Server Variables
- DOCUMENT_ROOT
- SERVER_ADMIN
- SERVER_NAME
- SERVER_ADDR
- SERVER_PORT
- SERVER_PROTOCOL
- SERVER_SOFTWARE
- Dates and Times
- TIME_YEAR
- TIME_MON
- TIME_DAY
- TIME_HOUR
- TIME_MIN
- TIME_SEC
- TIME_WDAY
- TIME
- Special Items
- API_VERSION
- THE_REQUEST
- REQUEST_URI
- REQUEST_FILENAME
- IS_SUBREQ
Working With Multiple Rules
The more complicated a site, the more complicated the set of rules
governing it can be. This can be problematic when it comes to resolving
conflicts between rules. You will find this issue rears its ugly head
most often when you add a new rule to a file, and it doesn't work. What
you may find, if the rule itself is not at fault, is that an earlier
rule in the file is matching the URL and so the URL is not being tested
against the new rule you've just added.
RewriteRule ^([A-Za-z0-9-]+)/([A-Za-z0-9-]+)/?$
get_product_by_name.php?category_name=$1&product_name=$2 [NC,L]
# Process product requests
RewriteRule ^([A-Za-z0-9-]+)/([A-Za-z0-9-]+)/?$
get_blog_post_by_title.php?category_name=$1&post_title=$2 [NC,L]
# Process blog posts
In the example above, the product pages of a site and the blog post
pages have identical patterns. The second rule will never match a URL,
because anything that would match that pattern will have already been
matched by the first rule.
There are a few ways to work around this. Several CMSes (including
wordpress) handle this by adding an extra portion to the URL to denote
the type of request, like so:
RewriteRule ^products/([A-Za-z0-9-]+)/([A-Za-z0-9-]+)/?$
get_product_by_name.php?category_name=$1&product_name=$2 [NC,L]
# Process product requests
RewriteRule ^blog/([A-Za-z0-9-]+)/([A-Za-z0-9-]+)/?$
get_blog_post_by_title.php?category_name=$1&post_title=$2 [NC,L]
# Process blog posts
You could also write a single PHP script to process all requests,
which checked to see if the second part of the URL matched a blog post
or a product. I usually go for this option, as while it may increase the
load on the server slightly, it gives much cleaner URLs.
RewriteRule ^([A-Za-z0-9-]+)/([A-Za-z0-9-]+)/?$
get_product_or_blog_post.php?category_name=$1&item_name=$2 [NC,L]
# Process product and blog requests
There are certain situations where you can work around this issue by
writing more precise rules and ordering your rules intelligently.
Imagine a blog where there were two archives - one by topic and one by
year.
RewriteRule ^([A-Za-z0-9-]+)/?$ get_archives_by_topic.php?topic_name=$1 [NC,L] # Get archive by topic
RewriteRule ^([A-Za-z0-9-]+)/?$ get_archives_by_year.php?year=$1 [NC,L] # Get archive by year
The above rules will conflict. Of course, years are numeric and only 4
digits, so you can make that rule more precise, and by running it first
the only type of conflict you cound encounter would be if you had a
topic with a 4-digit number for a name.
RewriteRule ^([0-9]{4})/?$ get_archives_by_year.php?year=$1 [NC,L] # Get archive by year
RewriteRule ^([A-Za-z0-9-]+)/?$ get_archives_by_topic.php?topic_name=$1 [NC,L] # Get archive by topic
mod_rewrite
Apache's mod_rewrite comes as standard with most Apache hosting
accounts, so if you're on shared hosting, you are unlikely to have to do
anything. If you're managing your own box, then you most likely just
have to turn on mod_rewrite. If you are using Apache1, you will need to
edit your httpd.conf file and remove the leading '#' from the following
lines:
#LoadModule rewrite_module modules/mod_rewrite.so
#AddModule mod_rewrite.c
If you are using Apache2 on a Debian-based distribution, you need to run the following command and then restart Apache:
sudo a2enmod rewrite
Other distubutions and platforms differ. If the above instructions
are not suitable for your system, then Google is your friend. You may
need to edit your apache2 configuration file and add "rewrite" to the
"APACHE_MODULES" list, or edit httpd.conf, or even download and compile
mod_rewrite yourself. For the majority, however, installation should be
simple.
ISAPI_Rewrite
ISAPI_Rewrite is a URL
rewriting plugin for IIS based on mod_rewrite and is not free. It
performs most of the same functionality as mod_rewrite, and there is a
good quality
ISAPI_Rewrite forum where most common questions are answered. As ISAPI_Rewrite works with IIS, installation is relatively simple - there are
installation instructions available.
ISAPI_Rewrite rules go into a file named httpd.ini. Errors will go into a file named httpd.parse.errors by default.
Leading Slashes
I have found myself tripped up numerous times by leading slashes in
URL rewriting systems. Whether they should be used in the pattern or in
the substitution section of a RewriteRule or used in a RewriteCond
statement is a constant source of frustration to me. This may be in part
because I work with different URL rewriting engines, but I would advise
being careful of leading slashes - if a rule is not working, that's
often a good place to start looking. I never include leading slashes in
mod_rewrite rules and always include them in ISAPI_Rewrite.
Sample Rules
To redirect an old domain to a new domain:
RewriteCond %{HTTP_HOST} old_domain\.com [NC]
RewriteRule ^(.*)$ http://www.new_domain.com/$1 [L,R=301]
To redirect all requests missing "www" (
yes www):
RewriteCond %{HTTP_HOST} ^domain\.com [NC]
RewriteRule ^(.*)$ http://www.domain.com/$1 [L,R=301]
To redirect all requests with "www" (
no www):
RewriteCond %{HTTP_HOST} ^www\.domain\.com [NC]
RewriteRule ^(.*)$ http://domain.com/$1 [L,R=301]
Redirect old page to new page:
RewriteRule ^old-url\.htm$ http://www.domain.com/new-url.htm [NC,R=301,L]
Useful Links
Summary
Hopefully if you've made it this far you now have a clear
understanding of what URL rewriting is and how to add it to your site.
It is worth taking the time to become familiar with - it can benefit
your SEO efforts immediately, and increase the usability of your site.
