Url paths and regular expressions

In the Set up content, understand Django urls, templates and apps section in Chapter 1, you learned how Django projects use the urls.py file to define url paths in two parts: a basic string that matches a url request and an action to take when said string is matched. For example, specifically in listing 1-20 you learned how an empty string '' represents a site's root path or homepage that can be set to serve a static template, as well as how a string like 'admin/' can match all url requests prefixed with /admin/ and be processed by yet another url file with more specific strings.

In most circumstances, Django projects use the django.urls.path method to achieve url path matching/action mechanism, however, Django also offers the django.urls.re_path method. So what's the difference between the django.urls.path and django.urls.re_path methods ? The former is used to match simple url patterns, while the latter is used to match more complex url patterns. In addition to this difference, there's also a historical Django perspective for why there are two ways to declare url patterns. Prior to Django 2.0, the only way to define urls paths was through regular expressions and the django.conf.urls.url method, something that made creating url patterns unnecessarily complex. Therefore, the creation of django.urls.path was intended to simplify url path creation via simple strings, whereas django.urls.re_path became a newer equivalent of the older django.conf.urls.url method to create more sophisticated url patterns with regular expressions.

django.urls.path behaviors

As already mentioned, the django.urls.path method is likely to be your go-to choice for defining Django urls, however, there are certain behaviors you need to be aware of when using the django.urls.path method, because although for the most part it works matching simple strings in url paths, there are certain edge cases where its behavior isn't as clear cut. Let's first explore two clear cut cases for exact matches with django.urls.path. For example, if you need to perform an action on the /contact/ url path can you use the path('contact/',...) syntax, whereas if you need to perform an action on the /contact/email/ url path you can use the path('contact/email/',...) syntax.

A django.urls.path variation that doesn't precisely work with exact urls are django.urls.path strings with url parameters. For example, the path('stores/<int:store_id>/',...) syntax defines a url parameter named store_id that matches paths like /stores/1/, /stores/2/ or any other url that begins with /stores/ and is followed by an integer. Because urls with url parameters can have variable values depending on the url parameter type, these url definitions have a dynamic nature that can match various url values. Because the use of url parameters also requires special view method handling, django.urls.path strings with url parameters are described in the Url parameters, extra options & query strings section later in this chapter.

Another variation you'll often find the django.urls.path method used for is when it's chained to other django.urls.path methods, so it might not be obvious you're working with exact urls since url parts are spread out across multiple url files. You already saw one particular case for the Django admin site, where the url statement path('admin/', admin.site.urls) points toward admin.site.urls, which is another url file with more django.urls.path -- or django.urls.re_path -- declarations. So path('admin/', admin.site.urls) represents the initial part of the url path starting with /admin/ and the other parts of the url path are declared in admin.site.urls (e.g. as path('login/',...) to match the full path /admin/login; as path('logout/',...) to match the full path /admin/logout).

The use of chained django.urls.path statements can also surface an edge case related to url precedence, that was also mentioned earlier in listing 1-28. The last url declaration included in admin.site.urls uses the statement admin/(?P<url>.*)$ -- this is a url pattern for the django.urls.re_path method whose syntax will be described shortly -- where the .* syntax means match anything, therefore this declaration is a catch-all for any /admin url pattern that isn't declared before it.

Although catch-all url patterns are helpful for certain cases like SEO (Search Engine Optimization), they can also give way to url declarations that are never reached. In the case of path('admin/', admin.site.urls), because the last statement in admin.site.urls is a catch-all url, it means any statement below path('admin/', admin.site.urls) containing an admin prefix will never be reached, because the catch-all in admin.site.urls is invoked before. That's the reason why the path('admin/doc/', include('django.contrib.admindocs.urls')) statement in listing 1-28 must be declared prior to path('admin/', admin.site.urls), otherwise a call to the url /admin/doc would always be handled by the catch-all in admin.site.urls.

The concept of chained django.urls.path and django.urls.re_path url statements is explored further in a dedicated section Url consolidation and modularization later in this chapter.

django.urls.re_path behaviors

A few lines above, you saw how the django.urls.re_path method uses a slightly different syntax in the form admin/(?P<url>.*)$, which represents regular expressions.

Regular expressions provide a powerful approach in all programming languages to determine patterns, but with power also comes complexity, to the point there are entire books written on the topic of regular expressions^[1]. Although most Django urls will never require a fraction of the complexity illustrated in many regular expression books, it's important to take a closer look at the most common regular expressions patterns used by Django urls, as well as the most important behaviors of the django.urls.re_path method.

By design, regular expressions use a very specific syntax to indicate whether to match specific or broad patterns, a behavior that can have unintended consequences if you use the django.urls.re_path method and don't take care of declaring more granular url regular expressions first and broader url regular expressions last. Since Django url resolution triggers the action of the first matching url statement -- whether it's a django.urls.path or django.urls.re_path statement -- you can end up never reaching an intended action if broad url regular expressions are placed first.

You shouldn't underestimate how easy it can be to introduce two url regular expressions that match the same pattern, particularly if you've never worked with regular expressions which have cryptic syntax. Listing 2-1 illustrates the right way to declare django.urls.re_path url regular expressions statements, with more granular regular expressions toward the top and broad regular expressions toward the bottom.

Listing 2-1. Correct precedence for django.urls.re_path url regular expressions

from django.urls import re_path
from django.views.generic import TemplateView

urlpatterns = [
    re_path(r'^about/index',TemplateView.as_view(template_name='index.html')),
    re_path(r'^about/',TemplateView.as_view(template_name='about.html')),
]

Based on listing 2-1, lets walk through what happens if Django receives a request for the url /about/index. Initially Django reads the first regular expression which says match ^about/index, therefore this is a match for the /about/index url and therefore control is sent to the index.html template.

Now let's walk through a request for the /about/ url. Initially Django reads the first ^about/index regular expression and determines it's too specific for the /about/ url and continues to the next url regular expression statement. The second django.urls.re_path regular expression says match ^about/, therefore this is a match for the /about/ request url and therefore control is sent to the about.html template.

As you can see, listing 2-1 produces what can be said to be expected behavior. But now let's invert the order of the django.urls.re_path statements -- as shown in listing 2-2 -- and break down why declaring more granular regular expressions toward the bottom is the wrong way to declare Django django.urls.re_path url regular expressions.

Listing 2-2. Wrong precedence for re_path url regular expressions

from django.urls import re_path
from django.views.generic import TemplateView

urlpatterns = [
    re_path(r'^about/',TemplateView.as_view(template_name='about.html')),
    re_path(r'^about/index',TemplateView.as_view(template_name='index.html')),

]

The issue in listing 2-2 comes when a request is made for the /about/index url. Initially Django reads the first django.urls.re_path regular expression statement ^about which turns out to be a broad match for the requested /about/index url and therefore control is sent right away to the about.html template! Before it reaches the next and likely intended ^about/index regular expression. The thing is the ^about/ regular expression simply says 'match anything that begins with about/', therefore a request made for the /about/index url produces a match, so you must be very careful when using django.urls.re_path statements to declare more granular url regular expressions towards the top and broader url regular expressions towards the bottom.

I'll admit I intentionally used regular expressions that allowed broad url matching to prove my point. But in my experience, as Django projects grow you'll eventually face the need to use this type of broad url regular expressions -- but more on why this is so, shortly. Another possibility to avoid falling into this broad matching regular expression pitfall -- besides being careful with regular expression precedence/order -- is to use exact url regular expressions to remove any ambiguity introduced by the order of django.urls.re_path url regular expression statements.

Lets rework the url regular expressions from listing 2-2 and make them exact regular expressions so their order doesn't matter. Listing 2-3 illustrates exact regular expressions on basis of those in listing 2-2.

Listing 2-3. Exact regular expressions, where url order doesn't matter.

from django.views.generic import TemplateView

from django.urls import path, re_path

urlpatterns = [
    re_path(r'^about/$',TemplateView.as_view(template_name='about.html')),
    re_path(r'^about/index$',TemplateView.as_view(template_name='index.html')),
]

# Equivalent path() method statements
# urlpatterns = [
#     path('about/',TemplateView.as_view(template_name='about.html')),
#     path('about/index',TemplateView.as_view(template_name='index.html')),

Notice the regular expressions in listing 2-3 end with the $ character. This is the regular expression symbol for end of line, which means the regular expression urls only match an exact pattern.

For example, if Django receives a request for the /about/index url it only matches the last regular expression in listing 2-3 which says 'match ^about/index/$. However, it doesn't match the first ^/about/$ regular expression because this regular expression says match about/ exactly with nothing else after, since the $ indicates the end of the pattern.

In addition, notice toward the bottom of listing 2-3 the urlpatterns definition with django.urls.path statements, which is equivalent to the urlpatterns definition one before it that uses django.urls.re_path statements. It turns out that by using exact url regular expressions with django.urls.re_path statements, you obtain identical behavior to that of django.urls.path statements which also operate on the premise of exact matches. The takeaway here is the django.urls.path method offers a more simplified approach to define exact urls than the django.urls.re_path method which offers a more general purpose approach capable of defining both exact and broad matching url patterns.

While you might be tempted to think it's always best to declare all django.urls.re_path regular expressions with the $ character to make them exact url regular expressions and avoid any problems, it's important to take a look at some beneficial cases where broad url regular expressions are helpful. If you plan to run the same logic on different urls -- a technique that's common in SEO and A/B testing -- strict regular expressions ending with $ used by django.urls.re_path statements or django.urls.path statements that by design are strict, can lead to a lot more work.

For example, if you start to use urls like /about/index, /about/email,/about/address and they all use the same template or view for processing, exact regular expressions just make the amount of urls you declare larger. Similarly, if you use A/B testing or SEO where lengthier variations of the same url are processed in the same way (e.g. /about/landing/a, /about/landing/b, /about/the+coffeehouse+in+san+diego/) broad url matching is much simper than declaring exact url patterns.

In the end, whether you opt to use exact url regular expression ending in $ with django.urls.re_path statements or just use django.urls.path statements, I would always recommend you maintain the practice of keeping finer grained url patterns at the top and broader ones at the bottom. Following this advice -- granular urls first, broader urls last -- avoids the unexpected behaviors described in listing 2-2, as well as the possibility of never reaching expected urls as it was described for path('admin/', admin.site.urls) & path('admin/doc/', include('django.contrib.admindocs.urls')) urls.

Common url patterns

Although url regular expressions can have limitless variations -- making it next to impossible to describe each possibility -- I'll provide examples on some of the most common url patterns you're more likely to use. Table 2-1 shows individual regular expression characters for Django urls and table 2-2 shows a series of more concrete examples with url patterns.

Table 2-1. Regular expression syntax for Django urls: Symbol (Meaning)

Table 2-2. Common Django url patterns for django.urls.re_path and their regular expressions, with samples

http://127.0.0.1/

http://127.0.0.1/stores/
http://127.0.0.1/stores/long+string+with+12345

http://127.0.0.1/about/contact/

http://127.0.0.1/about/

http://127.0.0.1/stores/2/
http://127.0.0.1/stores/34/

http://127.0.0.1/stores/downtown/

http://127.0.0.1/drinks/mocha/

http://127.0.0.1/drinks/324/

http://127.0.0.1/drinks/mocha/
http://127.0.0.1/drinks/mochaccino/
http://127.0.0.1/drinks/espresso/

http://127.0.0.1/drinks/soda/

http://127.0.0.1/drinks/mocha/

http://127.0.0.1/drinks/mochaccino/

http://127.0.0.1/drinks/espresso/

http://127.0.0.1/drinks/espressomacchiato/

http://127.0.0.1/stores/sandiego/
http://127.0.0.1/stores/LA/
http://127.0.0.1/stores/1/

http://127.0.0.1/san-diego/

http://127.0.0.1/san-diego/

http://127.0.0.1/CA/

http://127.0.0.1/Ca/

1. https://www.regular-expressions.info/    ↑