Description:
From version 3 onwards, your Birds have improved functionality for custom webroots. This customisation revolves mainly around a single configuration file which allows you to set headers, cookies, redirects, dynamic variable replacements, and more (per page too!).
Before getting down to the low level configuration, make sure you have a webroot folder structure that we can work from.
Adding the configuration file:
First, lets create a configuration file for our webroot.
Create a new folder in your webroot called thinkst-canary-metadata then within that, create a new file called config.toml.
Your folder structure should look similar to the below:
This configuration file makes use of the TOML file format that is relatively easy to understand and use. You can read up about the basics here (we will be using fairly simple functionality).
Format of the configuration file: Redirects
(For the below examples, we'll use 10.0.0.2 as the IP address of our bird)
A simple example of a working config.toml based on the above webroot could look like:
[redirects]
"" = "redirect.html"
Here, we introduce the [redirects] section. All this does is allow you to add redirects for specific pages. In this case, we are saying that if you navigate to the root of our website (i.e http://10.0.0.2/) the user should be redirected to http://10.0.0.2/redirect.html.
This section allows you to list multiple redirects as needed. For example, the below is valid:
[redirects]
"" = "redirect.html"
"/test/endpoint" = "index.html"
"index" = "index.html"
"/some/other/page.html" = "/go/to/this/page.html"
Format of the configuration file: Page specific config
For each route/page that we want to handle, we can add a new section to our toml file:
[redirects]
"" = "redirect.html"
["redirect.html"]
...
["/another/page.html"]
...
[!] You'll notice that our redirect.html section name is quoted, this is due to how toml handles sections with punctuation (in this case, the "."). Quoting ensures that it gets handled correctly.
Each section allows us to specify page specific settings. Currently, this includes
- HTTP protocol
- Whether to enable Dynamic Variable Replacements for the page
- HTTP headers for the page
- HTTP cookies for the page
We'll walk through each of these in the following sections.
Format of the configuration file: HTTP Protocol
If needed, you can specify a different HTTP Protocol per page (instead of using the default HTTP/1.1).
All that's needed is to add the desired protocol for the page like so:
[redirects]
"" = "redirect.html"
["redirect.html"]
protocol = "HTTP/1.0"
You can do this on every page that you need to override.
Format of the configuration file: Dynamic Variable Replacements
There are many times that a page displays dynamic data based on the interaction between the client and server (eg. displaying the IP address of the client or the server on the page). Previously, this wasn't possible with custom webroots. But, with our new configuration, we're able to dynamically replace variables in your page. The process is fairly simple:
Let us know you have dynamic variables in your page by specifying replace = true:
[redirects]
"" = "redirect.html"
["redirect.html"]
protocol = "HTTP/1.0"
replace = true
Then, add the relevant dynamic variables to your page:
<!-- redirect.html -->
<html>
<head>
<link href='./stylesheets/style.css' rel='stylesheet' type='text/css'></link>
</head>
<body>
<h1>My Dynamic variable test</h1>
<p>request.headers: {{request.headers}}</p>
<p>request.cookies: {{request.cookies}}</p>
<p>request.path: {{request.path}}</p>
<p>request.client: {{request.client}}</p>
<p>request.client.ip: {{request.client.ip}}</p>
<p>request.client.port: {{request.client.port}}</p>
<p>request.host: {{request.host}}</p>
<p>request.host.ip: {{request.host.ip}}</p>
<p>request.host.port: {{request.host.port}}</p>
<p>date: {{date}}</p>
<p>client_ip_address: {{client_ip_address}}</p>
<p>host_ip_address: {{host_ip_address}}</p>
<p>date.strftime('%Y-%m-%d'): {{date.strftime('%Y-%m-%d')}}</p>
<p>date.time: {{date.time}}</p>
<p>date.hour: {{date.hour}}</p>
<p>date.minute: {{date.minute}}</p>
<p>date.second: {{date.second}}</p>
<p>date.year: {{date.year}}</p>
</body>
</html>
This above example includes all the possible dynamic replacements that we currently offer. You'll notice that the variables we want to replace are surrounded by {{ .. }} (double braces). When this page is rendered, we'll replace those with the dynamic data.
Here is a list of what is available:
Quick IP addresses:
client_ip_address : IP address of the client
host_ip_address : IP address of the bird
Date object:
date : current UTC date
date.time : current time in HH:MM:SS format
date.hour : current hour in 24 hour format
date.minute : current minute
date.second : current second
date.year. : current year
date.strftime : formats the current date according to python strftime formatting
Request object:
request.client : Address(<ip>, <port>) of the client
request.client.ip : IP address of the client
request.client.port : port of the client
request.host : Address(<ip>, <port>) of the host
request.host.ip : IP address of the host
request.host.port : port of the host
request.path : path of the current request
request.headers : current HTTP headers
request.cookies : current Cookies
Format of the configuration file: HTTP Headers
Let's say that we want to return extra headers on a specific page (but not all of them). In particular, let's say we want to add Pragma = "no-cache" as a header when requesting the redirect.html page.
In our configuration file, we can add a section to handle all config for our redirect page:
[redirects]
"" = "redirect.html"
["redirect.html"]
["redirect.html".headers]
Pragma = "no-cache"
Here you can see that we created a subsection for headers for our redirect page. The headers section will allow you to list multiple headers, depending on your needs:
[redirects]
"" = "redirect.html"
["redirect.html"]
["redirect.html".headers]
Pragma = "no-cache"
My-custom-header = "Some Value"
Content-Type = "text/html; charset=UTF-8"
Format of the configuration file: HTTP Cookies
Cookies become a little more involved than headers. Similar to headers, you'll create a ["page".cookies]
subsection where you will be able to list multiple cookies as needed. But, HTTP cookies tend to be a lot more dynamic than headers.
To expand on that, some cookies can easily be defined statically. Let's add a custom cookie and value to our redirect page:
[redirects]
"" = "redirect.html"
["redirect.html"]
["redirect.html".headers]
Pragma = "no-cache"
My-custom-header = "Some Value"
Content-Type = "text/html; charset=UTF-8"
["redirect.html".cookies]
"my-custom-cookie" = "'my static value'"
Note: notice how the value is surrounded by double quotes, as well as single quotes. We'll get to this in a bit.
When this page is returned, we'll return our custom cookie along with it. We can also define multiple cookies and return them all. But, so far we are only allowing static values for the cookies, which isn't always useful. Let's now look at dynamic cookies.
For defining dynamic cookies, we offer a few predefined functions:
random(<int>) : generate a random alpha-numeric string of length defined by <int>
base64(s) : generate base64 representation of string s
md5(s) : generate md5 hash of string s
sha1(s) : generate sh1 hash of string s
uuid() : generate a valid uuid
upper(s) : convert string s to uppercase
's' : single quoted values define a static string
These can be combined with string concatenation to create complex cookies:
"my-custom-cookie" = "base64(random(10)) + '; HttpOnly;'"
"my-second-cooke" = "upper(uuid())"
Format of the configuration file: Custom POST Error Response Codes
POST requests can happen when programs like cURL submit data to your website, or when someone tries log in with a browser.
Your Canary's web server can be configured to respond to POST requests on certain pages with a custom response code.
To get started, create a HTML file in your webroot named after your chosen code.
In the below example, we'll use the response code 405, (Method not allowed) to respond to POST requests on your websites root path. ("/") your directory should now look like this:
Note: This file's HTML will be served up when we respond, so be sure to add in any custom HTML and design you'd like.
If you haven't created your thinkst-canary-metadata folder yet, refer to the configuration file portion of this article.
Now that we have the HTML file we'd like to serve up, lets modify our config.toml file to refer to it.
Within your config.toml file, specify the path, and your chosen response code.
["/"]
response_code = 405
Your Canary's web server server will now return the request with the response code you specified in the config file, you're all done!
Note: Although any code would work, since this is an error being returned, it's highly recommended to use an error code (within the 400 and 500 ranges). A list of valid HTTP response codes can be found here.