Here at Follow My Vote, we rely on WordPress to host our website. We are an online company, so this website is our public face and we strive to keep it informative and attractive to guests. But that isn’t always easy. In order to keep a WordPress website secure and reliable, both WordPress itself and all of the plugins must be kept up to date, but sometimes updates cause breakage to the website that can impair its visual appeal or even result in downtime, leaving us scrambling to get all of the pieces we need working together again with the new version. Furthermore, sometimes we want to update the content of the site, and if it’s a small change, it’s (usually) easy and safe to just push it into production immediately. Still, sometimes we want to make big changes and see those in action ourselves before taking them live to show to the world.

In other words, when talking about making changes to the website, whether those be updates, new content, or whatever else… we want to try it before we buy it. We want to see it and try it out like it’s for real, but the public still sees our stable and reliable site while we tidy and polish the next version behind the scenes. And of course, it needs to be easy to set this up without asking our content creators to jump through complex technical hoops.

What we really want is a way to create a staging copy of our website, that we can make changes to and test out new content or configurations with, while the production website is all that is visible to the public eye. Well as it turned out, that wasn’t quite as easy as we expected…

What We Tried

One may suppose that the easiest solution to a problem is the best, so the most obvious thing to try was the features built into WordPress for making drafts of posts and pages. And indeed, these features are great for new content, and they’re reasonable for updating old content, but we find them best suited for quick and basic changes. They can be a bit clunky when doing certain kinds of edits or design changes, and it always feels a bit too easy for a stray click or keypress to publish a draft before it’s ready. For a lot of content work, we find ourselves wanting a bit more, and furthermore, these features leave us completely in the cold for testing updates before we commit to using them on our production website.

So we needed a stronger solution than we found built into WordPress itself. The next thing we tried was to simply copy the WordPress installation and host it on a staging domain, meaning we put the staging site on staging.followmyvote.com rather than regular followmyvote.com and made our updates there.

This was a quick and dirty solution that required minimal sophistication to get up and running, but the problems it created were endless. First of all, this is not particularly discreet. Anyone could type in staging.followmyvote.com and see what we’re up to. There are ways to put it behind a password, but it still feels a bit unprofessional to have this internal resource discoverable to the public, even if it is difficult to guess the password to see inside. Secondly, WordPress doesn’t take kindly to being hoisted up from one domain and plopped down on another. The website needs configuration changes to make it OK with that, and while those are relatively easy to do, it complicates the process, making it a less attractive solution.

We then realized that even if you make WordPress itself work when hosted on a new domain, the pages don’t. Lots of pages had hardcoded links to followmyvote.com rather than relative links based on the current domain, and it was difficult to train ourselves not to copy and paste links without removing the domain from the beginning of them. This made working on the staging site error-prone, and often resulted in us clicking a link that took us back to the public site without our realizing it, which could then lead to making changes there unintentionally! Furthermore, when we eventually did deploy the staging site into production, we realized to our chagrin that we had accidentally copied quite a number of links to the staging site into it, which was a rather embarrassingly public mistake.

All around, then, the approach of using a separate staging domain was painful, error-prone, and ridiculous. It was fraught with difficulties and risks, making it easier to do it wrong than to get it right. The failures of this approach made it far worse than if we had not attempted to use a staging site at all. I considered the possibility that with automation and filtering, we might be able to rescue the strategy, but quickly decided that this would be risky and complicated and would doubtlessly result in still more mistakes.

Nevertheless, I remained convinced that the staging site idea had merit, and for a few years, I contemplated what I would want from such a solution and how it could be done.

A Solution Begins to Take Shape

The first step to solving a problem is always figuring out what one would want in a solution. In the case of a staging solution for our website, I wanted the following features:

Same Domain First and foremost, the staging site and the production site must share the same domain (and URL). No more staging.followmyvote.com nonsense, no more reconfiguring WordPress for different domains, and no more changing links back and forth. Both sites must be accessed by browsing to followmyvote.com.

Not Just WordPress The solution must be built outside of and around WordPress, not using a plugin or any other feature of WordPress. Building such a solution into WordPress would be inherently more complicated than building it outside, and it wouldn’t provide the kind of isolation we would need to test different versions of plugins, themes, or WordPress itself without jeopardizing the production site. Moreover, I want a solution that works for other sites and site frameworks; not merely WordPress.

Professionally Discreet The existence of the staging site must not be publicly obvious. Our public web presence should be buttoned up and shouldn’t have technical details exposed. It should not be feasible for any passerby or bot to access the staging site, even if they happen know it existed. On the other hand, it should be relatively convenient for us to share access to it with others, without expecting them to possess unusual knowledge or dedication.

Zero Downtime Copying the production site into a new staging site, and subsequently deploying a stable staging site into production, should both incur zero downtime. The website should remain up and serving users the entire time with the only public indication that something happened being a change in content.

Straightforward Automation While it is true that a good staging solution will be reasonably convenient to operate manually, it should also be straightforward to implement full automation in the future so that staging sites can one day be configured, spun up, deployed, or retired with a simple back-office control interface.

The Path Comes into Focus

With these requirements in mind, I put on my sysadmin hat and considered our existing deployment infrastructure. Follow My Vote runs a public WordPress site as well as several internal cloud services to facilitate our business operations. All of these exist as containerized microservices in Docker utilizing Traefik as a TLS-terminating reverse proxy sitting in front of these services. This seems to be a typical deployment strategy, and it has worked well for us for years with minimal turbulence after the initial learning curve.

When we tried the staging domain, we just deployed a copy of WordPress with Traefik serving it on the staging domain rather than the primary domain. While easy to implement, this strategy failed due to the difference in domain. In theory, however, the same back-end architecture could be used successfully if instead of using the domain to route traffic between the production and staging back-end services, I used some other discriminant.

My first idea was to use client IP to select between serving the production and staging sites; however, I rejected this approach because it was clumsy and complicated. On the one hand, an IP address isn’t a good identifier of a client anymore due to NAT, and on the other, it makes the Traefik configuration annoying due to frequent updates to an excessive number of rules.

Then I considered that if the browser indicated that it was looking for the staging site in its request, this would also be adequate for Traefik to pick out and decide how to route the request. This was the approach I finally settled on: the client inserts a custom header into its request and Traefik determines whether the header is correct and, if so, routes the request to the staging back-end.

There was one wrinkle in the solution. WordPress sites regularly make requests to themselves, and since WordPress just does a DNS lookup for the domain it’s configured to run on and sends the request there, obviously without our fancy custom header, the staging service’s requests get routed to the production service rather than back to staging! Working around this required a bit of ingenuity, and I’ll cover it in the technical guide below.

This solution was highly successful for us. It’s easy and reliable to implement on the server side, and opting in or out of the staging site is completely client-side. A browser plugin can handle the injection of the header, and plugins exist for major browsers to configure a custom header and to enable or disable the header with a click, making it convenient to switch back and forth between production and staging. Staging environments can be kept private by making the value of the header into a password. If a staging environment is desired to go live, simply remove the custom header requirement from it and give it a higher priority than the production server, and Traefik will switch new traffic to it with zero downtime. The custom header strategy also looks like it will be easy to automate, although for the time being, this is still future work.

OK, now it’s time for the fun part.

How To Run It on Your Server

We show the setup of this staging solution in three stages. First is the foundation stage, where we set up a trivial Traefik reverse proxy and a placeholder webpage hosted by a fallback server, used when nothing else is working. Initially, no other servers will be defined, thus nothing else will be working and therefore the placeholder will be shown. The second phase adds our production website on top of the placeholder, and the third phase adds a staging site on top of that. We will explore the configs used in each of these phases here, but readers can also see the full config directory hierarchy in a git repository where each phase is represented by a commit. The git repository is at https://gitlab.followmy.vote/nathanielhourt/staging-site

The Foundation

We will be using Docker Compose to deploy our server infrastructure. Installing Docker and Docker Compose is left as an exercise for the reader; there is an abundance of tutorials easily discoverable online for virtually all imaginable servers and environments. The reader also must choose a domain on which they wish to host their website and set up DNS for it. If it is not convenient to set up public DNS to follow this tutorial, an entry into a client’s /etc/hosts file will suffice, but in this instance the reader must procure their own TLS certificate and manage it manually. Configurations for both manually managed certs and automatic Let’s Encrypt certs (for deployments on public domain) are shown below.

OK, with Docker and Compose ready to go and either a public domain or a cert for our private domain in hand, let’s write our service configuration. Begin with a new directory to contain the servers on this website, and within it, a subdirectory for the foundation of our deployment, which will hold configs for Traefik and the placeholder site. In this tutorial, we’ll call these directories mywebsite and foundation. In the mywebsite directory, let’s make a file of environment variables we may share with the various services in our deployment, called env. The only variable we need here for this tutorial is the domain upon which we’re hosting our website:

mywebsite/env

DOMAIN="stagingsite.followmy.vote"

In our foundation directory, we’ll add a compose.yaml file, and Docker Compose will look for environment variables in a .env file in the same directory as the compose.yaml file, so make a symlink at mywebsite/foundation/.env to the shared environment file, ../env.

Now let’s explore the contents of the compose config section-by-section, and remember that the full config of this phase is visible on the repo.

We begin by defining a Docker network (which is to say, a VLAN) to interconnect our production services.

mywebsite/foundation/compose.yaml

networks:
  prod:
    name: production-vlan
    ipam:
      config:
        - subnet: 172.100.0.0/24
          ip_range: 172.100.0.128/25

This creates a network, which we can refer to in this file as prod, but which will be known as production-vlan outside this file. That network will occupy the 172.100.0.0/24 subnet, but the first half of the subnet will be reserved for static IPs and Docker will only assign IPs dynamically out of the second half of the subnet.

Next, let’s make a service that will be Traefik, using the official public Traefik image for it, tracking the 3.1 series:

mywebsite/foundation/compose.yaml (cont’d)

services:
  traefik:
    image: traefik:v3.1
    networks:
      prod:
        ipv4_address: 172.100.0.10
    ports:
      - "80:80"
      - "443:443"
    restart: always
    healthcheck:
      test:
        - CMD
        - traefik
        - healthcheck

We assign our Traefik service the fitting (if not very original) name traefik and put it on our prod network with a static IP. Note that we’re only dealing with IPv4 in this tutorial, but Docker will invisibly forward inbound IPv6 traffic to IPv4 service networks if desired. Of course, readers may define IPv6 addresses for their services at their discretion.

We also bind the host’s ports 80 and 443 to the same ports on the service. Note that the ports section governs public ports belonging to the host system — these are not private ports on the VLAN we created! We are giving the Traefik service access to bind the host’s ports 80 and 443, and this is the only service to which we will grant direct access to public ports. All other services will have private VLAN networking only, and Traefik will forward traffic to them over said VLANs as appropriate.

We direct Docker to restart the Traefik service if it stops for any reason, and we define a healthcheck command that Docker will run regularly to verify that Traefik is healthy. Note that by default, Docker will take no corrective action if it becomes unhealthy, but it makes for pretty status reports.

Finally, let’s attach some volumes to our Traefik service. Docker containers, by default, have no persistent storage, and all the data they store is subject to be reset to the original image’s contents any time the service restarts or is rebuilt (NB, during normal maintenance). Docker Volumes allow us to persist paths within the container, and also give us a way to place specific files and directories into the container or share certain files or directories from the host with the container.

mywebsite/foundation/compose.yaml (cont’d)

    volumes:
      - type: bind
        source: /var/run/docker.sock
        target: /var/run/docker.sock
      - type: bind
        source: ${PWD}/volumes/certs
        target: /etc/certs
      - type: bind
        source: ${PWD}/configs/static.yml
        target: /etc/traefik/traefik.yml
        read_only: true

Here, we have done all three. We share the host’s Docker socket with the container, allowing Traefik to access Docker’s API, we make a persistent directory wherein Traefik can store (or access) TLS certificates, and we mount into the container Traefik’s config file which we will have created by the time we start this service and Docker goes looking for it.

Note that when we give Traefik access to Docker’s API, we give up all the security benefits of containerization, at least in Traefik’s case. Having access to the Docker API bestows the full power of the host’s root account. Those concerned about this can look into options like rootless Docker or podman and can still follow this tutorial, but details on how to set this up are out of scope here, and the benefits are questionable: either way, a successful attack on Traefik will compromise everything Docker has access to, and whether this equates to the host’s root or not, it’s probably still everything of value on your server. It would be nice if there were a way to expose a restricted Docker API to Traefik instead of the full gamut, but I am not aware of a way to do this today.

Pressing onward, we define a second service in this compose file, which is our placeholder website. We name the service greeter and assign it a bare nginx image. Attach it to our prod network with no config, and Docker will assign it a dynamic IP automatically. We’ll also mount in the website to host, although for this tutorial this is merely a single config file.

mywebsite/foundation/compose.yaml (cont’d)

 greeter:
    image: nginx:latest
    networks:
      prod:
    volumes:
      - type: bind
        source: ${PWD}/configs/greeter.cfg
        target: /etc/nginx/conf.d/default.conf
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.greeter.rule=Host(`$DOMAIN`)"
      - "traefik.http.routers.greeter.priority=1"

At the end, we include a section of labels. These are simple key-value pairs that can be attached to most constructs in Docker, and while Docker itself doesn’t use or care about them, they provide information that systems integrated with Docker (such as Traefik) can use to determine how to handle things running within Docker. These labels are how we tell Traefik how to host our service, and Traefik monitors Docker for services starting and stopping and consults the labels on those services to automatically update its routing according to the services available at the moment.

On our placeholder service, we put a label that tells Traefik that we want it to proxy for this service, another label telling Traefik what traffic this service is intended to host (in our case, traffic addressed to our domain), and finally, a label setting a very low priority for this service, so that of the requests that are addressed to our domain, Traefik will only forward those requests to the placeholder if nothing else is available to take it. Initially, since no other service is running, this will show us that our servers are working properly. In the next phase, when we start our real website, however, this placeholder will no longer be served unless the server of our real website crashes. This is because the real website server has a higher priority, granting it precedence over the placeholder server any time both are available.

OK, that’s our entire services config. Now let’s make the other things we need for our servers. Create a directory at mywebsite/foundation/volumes and a certs directory within. Next, create also a directory mywebsite/foundation/configs, and within, we’ll make two config files, one for Traefik, and one for our placeholder nginx website.

The nginx config is trivial:

mywebsite/foundation/configs/greeter.cfg

server {
    listen       80;
    listen  [::]:80;

    location / {
        add_header Content-Type text/plain;
        return 200 "This website is currently under construction. Check back later for total awesomeness! =)\n";
    }
}

Note that we listen only on port 80. No TLS is used within the VLANs; Traefik terminates the TLS and proxies raw HTTP traffic to the services. This way the services need not deal with certs or TLS options at all, simplifying their configuration immensely without exposing unsecured traffic to the internet. All of the TLS and certificate concerns can be handled centrally at Traefik.

Next, the Traefik config. I’ve included comments to elucidate inline, but we’ll discuss in more detail below:

mywebsite/foundation/configs/static.yml

# Define the ports Traefik listens on
entrypoints:
  # Listen on port 80, but by default just serve a 301 redirect to the same URL on HTTPS
  web:
    address: ":80"
    http:
      redirections:
        entrypoint:
          to: websecure
          scheme: https
  # Listen on port 443, and set a default certificate to use so services don't need to configure it individually
  websecure:
    address: ":443"
    asDefault: true # Since web is just a redir to websecure, make all routers default to using only websecure
    http:
      tls: {} # Set a default here for all routers using this entrypoint to use TLS; simplifies per-router config
      # When using Let's Encrypt, uncomment this line as well as the definition of the resolver below
      # Then you can remove the manual certs config from the providers section below
        #certResolver: letsencrypt

# Define a certificate resolver. Traefik uses Let's Encrypt by default, but we need to configure it to make it work
#certificatesResolvers:
#  letsencrypt:
#    acme:
#      email: "your@email.here"
#      storage: /etc/certs/acme.json
#      keyType: "EC256"
#      tlsChallenge: {}


# Configure Traefik to watch for docker services to show up asking to be hosted
providers:
  docker:
    watch: true
    # Matter of taste; I like to explicitly set which services Traefik should expose via the traefik.enable=true label
    exposedByDefault: false
  # During testing, using Let's Encrypt can be difficult, so we can manually configure cert in a dynamic config file
  file:
    filename: /etc/certs/manual.yml

# Enable the ping API endpoint, so docker can do health checks
ping: {}

### The rest of this is optional stuff that might be useful from time to time

# To enable Traefik's dashboard website and API
#api:
#  dashboard: true
#  insecure: true # To allow simply querying Traefik's port 8080 rather than needing to define a router to get to it

# To log each request like a webserver does
#accessLog: {}

# To increase logging verbosity and make it easier to diagnose problems
#log:
#  level: DEBUG

Traefik operates with two classes of configuration. The first is the static config, which is what we’re setting here, and it controls the constant concerns that are the same over an entire execution cycle of Traefik. The second configuration class is the dynamic configuration, which controls the services Traefik is expected to expose and the rules controlling how those services are accessed. The dynamic configuration changes over the course of Traefik’s execution, and Traefik constantly adjusts its behavior to align with this configuration: as new services show up or old ones shut down, Traefik updates its routing and adjusts how requests are directed accordingly. The dynamic config comes from “providers” which Traefik monitors for up-to-the-moment status updates and changes, and the providers Traefik should monitor are defined in the static config.

OK, with that context established, let’s look over the static config above. First, we define entrypoints. These are the ports Traefik should listen on for incoming traffic. We configure two: port 80, for HTTP traffic; and port 443, for HTTPS. We name the HTTP entrypoint web and the HTTPS, websecure. We configure Traefik not to connect HTTP traffic to services, but rather to respond with a 301 redirect to HTTPS. This is, of course, a matter of preference, but it’s a common enough pattern that it’s useful to show it here. We configure the HTTPS entrypoint, websecure, to be a default entrypoint, to simplify the per-service configuration, and we set TLS options so as to inform Traefik that this entrypoint deals with TLS traffic, as it will not infer this from the use of port 443 alone.

In the next sections, the config is different for those wishing to use a public domain and Let’s Encrypt certs than for those configuring their certs manually (as those using a private domain must do). Certificates are a dynamic concern (they change at runtime), so they aren’t set in the static config, but where to get them is. If configuring the cert manually, just put an empty tls config block to indicate that TLS is used, but to accept the defaults. We’ll then set a dynamic config provider that loads in the certs. Alternatively, if using Let’s Encrypt, uncomment the certResolver config so Traefik knows to get certs automatically, as well as the config lines defining the certificate resolver to be used. In this case, the dynamic configuration provider for manually managed certs can be omitted.

Moving ahead to the providers configuration, we configure Traefik to monitor Docker for dynamic configurations, and I prefer to select explicitly which services should be exposed with the traefik.enable=true label, but again, this is a matter of taste. Then, for those managing certs manually, we add a dynamic provider from a file, where we will hook in those certs.

Those using Let’s Encrypt can omit this, but for those who aren’t, here are some typical contents of that dynamic config file. Be sure also to put in your private key and certificate files alongside so the paths provided in the dynamic config point to them in the container’s filesystem (remember that our mywebsite/foundation/volumes/certs directory shows up at /etc/certs inside the container!).

mywebsite/foundation/volumes/certs/manual.yml

tls:
  stores:
    default:
      defaultCertificate:
        certFile: /etc/certs/followmy.vote.pem
        keyFile: /etc/certs/followmy.vote.key.pem

Returning our attention to the static config, hopefully the remainder of the configuration is self-explanatory. I do want to note, however, that the ping section is only needed if using the healthcheck on the Traefik service in compose.yaml. If no healthcheck is required, the ping section can be omitted as well. To opt into anonymous usage statistics collection, add a global section with a setting sendAnonymousUsage: true.

With that, our foundation is complete! Go ahead and start the servers by setting your working directory to foundation and running docker compose up. Note that this starts the servers in the foreground with logging to the console; to daemonize it all, pass an additional flag -d and monitor the status with docker compose ps and docker compose logs. You should now be able to open your website in a web browser, though if using Let’s Encrypt, it may take a few extra moments for the cert to be provisioned. If automatic certs don’t seem to be working, turn on the debug logging in the static config and monitor Traefik’s output for details on the ACME negotiations. The certs that Traefik has acquired go into acme.json in the certs volume — that file can also be monitored to determine whether certs are being issued successfully, without enabling debug logging.

Take a few moments to bask in the glory of your new foundational website infrastructure… and then move on to Phase II, the production website.

Pushing to Prod

The simplest phase of our deployment, now we create the production website. For our demo, this is a trivial single-page website, almost identical to the fallback site. We just create another bare nginx service and populate it with a slightly different config. In a real deployment, you might set up a WordPress here or any number of other apps; from the perspective of the staging system as well as Docker and Traefik, it’s all fundamentally the same.

As an aside, if you are setting up an app with dependencies, and those dependencies are in their own containers, I recommend putting these dependencies on a separate VLAN without Traefik and connecting the front end to both the prod VLAN and its support VLAN. This creates a better separation of concerns. If your front end is on multiple VLANs, you need to add another label to it so Traefik knows which VLAN to connect to it on, i.e. traefik.docker.network=production-vlan (remember to use the absolute name, not the local alias).

But back to our demo, let’s begin by creating a new directory mywebsite/production, and in that directory creating our .env symlink back to ../env. Next, create our compose file with our service definition:

mywebsite/production/compose.yaml

networks:
  prod:
    name: production-vlan
    external: true

services:
  website:
    image: nginx:latest
    networks:
      - prod
    labels:
      traefik.enable: true
      traefik.http.routers.prod.rule: "Host(`$DOMAIN`)"
    volumes:
      - type: bind
        source: ${PWD}/configs/website.cfg
        target: /etc/nginx/conf.d/default.conf

We don’t need to define the network this time, as it was already defined in the foundation configs. This time we just give it a local name, mention its absolute name, and set external to true so Docker knows to find it already created rather than creating it anew.

Then we create our service, essentially the same as before, but this time we don’t set an artificially low priority. I used a different syntax for the labels this time… but both formats do the same thing.

Finally, we create our nginx config:

mywebsite/production/configs/website.cfg

server {
    listen       80;
    listen  [::]:80;

    location / {
        add_header Content-Type text/plain;
        return 200 "Welcome to Follow My Vote's staging website demo. You are currently viewing the production site.\n";
    }
}

To launch it, we do the same thing as before. Make sure Traefik is still running and, from the new production directory, do a docker compose up -d to start the website in the background. Now go to your domain in your browser, and you should see the new production page rather than the placeholder.

And with that, we’re ready to make our staging site!

Staging the Next Thing

Now to launch our staging site, we could actually just copy our production site, tweak the Traefik rule, and be done… except that if your website or app is anything more than a trivial static webpage, it might at some point want to connect to itself as a client (for example, I know for a fact that WordPress does this to trigger background tasks) and if we are too naive about how we set this up, the staging service won’t connect to itself but will rather connect to the production service. This is because the staging service doesn’t know to set the magic header needed to reach itself through Traefik.

This posed a bit of a conundrum for me when I was designing this solution. I couldn’t just convince the app (I was working with WordPress) to connect to localhost (its own container) rather than using DNS to find Traefik because while it was configured to serve over port 80, it was also configured to know that its URL uses the https:// scheme, so it used that and thus couldn’t connect to itself directly — it had to go through Traefik for TLS termination. “OK,” I thought, “but I control the IP of the container, so I can just tell Traefik to route the request-to-self based on source IP, right?” Well, no, because the way Docker routes outgoing traffic, all containers’ outgoing traffic to a public IP address gets NATed together and shares a single source IP. So I have to force my app to connect to itself through Traefik, but also within Docker without going to the public IP associated with my domain on public DNS. Here is how I did that.

We create a new VLAN for the staging site, giving it a different subnet from the production VLAN. We then put Traefik on both VLANs, giving it an alias on the staging VLAN which is our domain. This means that when a container in the staging VLAN does a DNS lookup on our domain, Docker sees that this is an alias for a container on the VLAN and gives that container’s VLAN-private IP as the DNS result, rather than the result of a public DNS query! This causes the staging container to connect to our domain by connecting to Traefik directly over the staging VLAN, thus eliminating NAT and showing its real source IP to Traefik. Traefik is then configured to route any and all traffic destined for our domain coming from the staging VLAN to the staging frontend, even if it doesn’t have the magic header that says “route me to the staging service”!

So first things first, we edit our foundation compose config to add the new network and to give Traefik an alias on it. In the networks: section, we add:

mywebsite/foundation/compose.yaml (insert)

 staging:
    name: staging-vlan
    ipam:
      config:
        - subnet: 172.200.0.0/24
          ip_range: 172.200.0.128/25

Then in the services.traefik.networks: section, we add:

staging:
        ipv4_address: 172.200.0.10
        aliases:
          - $DOMAIN

Remember that you can see the entire config diff on the git repository.

Next, we just copy our entire production directory calling the new copy staging and update all the references to production things to refer to staging things instead, and we update the Traefik routing rule for the staging service like so:

traefik.http.routers.staging.rule: "Host(`$DOMAIN`) && (Header(`X-REQUEST-STAGING`, `Saex6Bu0`) || ClientIP(`172.200.0.0/24`))"

Note that we changed the label key as well as the value; the prod or staging in the label key are names that we define arbitrarily, and they must be globally unique. This is true in many of Traefik’s service labels, so copy and paste carefully!

Examining our rule, it now applies to traffic destined for our domain AND either bearing the magic header, or coming from our staging VLAN subnet. Also, by default a Traefik routing rule’s priority is equal to the number of characters in the rule (a cheap but remarkably effective heuristic for preferring more specific rules over less specific ones), meaning that while our staging traffic will match both the production rule and the staging rule (and the fallback rule too!), the staging traffic will be routed to the staging service by virtue of that service having the longest rule.

Make the appropriate changes to the staging site by updating its config, and we’re ready to run the staging site live! With yet another docker compose up -d we can make it so.

To access the staging site, it is necessary to set a custom header in our requests to our server. An easy way to do this is via the use of the Modify Header Value extension for popular browsers. Go ahead and give it a whirl! (Users of Chromium-based browsers may need to use ModHeader or some other option instead, as Chromium’s Modify Header Value doesn’t seem to work on all requests that match the filter).

Our staging site is now live and working, discreetly, for only those we want it to. We now can make our desired changes to it, and test them out as if they were in prod, on the normal domain and with the normal config. From an application perspective the staging site is exactly identical to the production site except that only we can see it, and only when we want to, while the rest of the world sees the production site without bothering to know any other option might exist.

In fact, the staging site is so similar to the production site that when we decide it’s ready, we can simply make it the production site — at least temporarily.

Zero-Downtime Updates

To deploy the new staging changes into production, we could shut down production, copy over the new changes, and re-deploy, and with the help of our fallback server we could even display a friendly “Scheduled maintenance” page in the meantime… but wouldn’t it be nicer if we could just switch over with no downtime at all? Well as it happens, there are a number of ways to do this. One of them would be to simply relaunch the staging site with a routing rule that makes it preferable to the production site for all traffic. With a health check on it, Traefik will watch it until it reports as healthy after its startup routine, and then switch new traffic over to it because it has a preferable rule. Once traffic switches over, we could shut down the production site, update it to the new code, start it up again, and once it’s healthy, shut down the staging site so traffic goes back to the service labeled as production. Voila! An upgrade with no downtime at all!

Of course, this isn’t the only way to accomplish the same result, and all such mechanisms have advantages and disadvantages, and depending on the exact details of your app, this approach may not work and possibly no zero-downtime approach will work at all. Nevertheless, this approach should work for many applications, and even if the zero downtime bit doesn’t work for your particular app, this overall strategy should be effective and compelling.

I wish you luck and I hope you’ve enjoyed this tutorial. If you get this far, or if you have any questions or trouble, drop me a line on Matrix: @I:nathaniel.land. I’ll see you around! =)

Read More Articles From The FMV Blog

The post WordPress Staging Website on the Same Domain appeared first on Follow My Vote.

As we are researching and developing new platform technologies necessary to someday support the development of online voting systems, we take note of others in the industry working toward such systems; especially those also using blockchain technology.

One such company is Voatz, and today I’d like to highlight their efforts and share some thoughts about their work. Let me preface this by saying that this is just one man’s opinion and perspective. I have no affiliation with Voatz, and have not closely examined their work. My words here should be viewed merely as the casual commentary of an interested bystander.

Voatz is working on solutions for an online voting app using blockchain as a durable record of the election procedures and results. They are developing the procedures and workflows necessary to conducting an online election, and creating the tools and interfaces to administrate and participate in such an election, and showing the world what it could look like.

In the course of developing these procedures, Voatz ran a pilot of their app, giving voters and administrators alike a taste of what online voting could be like, and sometime thereafter, they came under fire for the security stance of their pilot system. A team of advanced security researchers at MIT wrote a paper decrying the Voatz pilot as an insecure approach to online elections and warning the public that such solutions cannot be trusted to uphold the operations of modern democracy. Shortly thereafter, security firm Trail of Bits published a report outlining vulnerabilities in Voatz’ application and infrastructure in greater detail.

These events seem to have led to a great deal of uncertainty and doubt of Voatz’ suitability as a provider of solutions in the elections space. If their pilot was so insecure, can the company be trusted to produce solutions for use in secure elections?

As one trained in the same kind of cybersecurity analysis and techniques as were employed by the research team at MIT, I would like to share my view on these events. Elections are enormously complex endeavors that require coordination of many people and organizations, and this succeeds today by using various different procedures and methods, but a transition to online voting will massively disrupt these procedures, and nobody is yet sure how that can possibly work. Voatz is helping to find ways that it can, even while the full security architecture of a final solution is still uncertain. To do this, Voatz is working with security standards well understood by the industry today, building voting system demonstrations based on the standards we might expect from existing major tech companies.

The MIT review raises legitimate concerns about the viability of Voatz’ approach for secure elections. In a sense, the MIT researchers are pointing out that while they themselves don’t know what the security architecture of a fitting solution for online elections would be, they can clearly see that the Voatz approach isn’t it. And while this may be true, it completely misses the value of Voatz’ contribution. Voatz is showing us how online elections can look, even before the full picture of how they can work comes into focus. It doesn’t exhibit the final security architecture, and it doesn’t need to: Voatz is building to the security standards we understand today to show us how the final solution could look tomorrow.

Meanwhile, the Trail of Bits report was commissioned by Voatz to evaluate the security of their pilot against the standards Voatz was aiming for, and the vulnerabilities disclosed by that report, while serious in nature, are not fundamental flaws, and should be relatively easy to fix.

In conclusion, we at Follow My Vote are grateful to Voatz for the work they are doing to further the conversation on how online voting solutions may look in the future, how online elections can be orchestrated, and what it will be like to participate in them. We look forward to seeing what they produce next, even as we are researching and developing the foundations and underpinnings that will eventually enable applications to be deployed meeting the full security, privacy, and end-to-end verifiability requirements necessary to support modern political elections online.

The post A Word for Voatz appeared first on Follow My Vote.

I also explained that until such a foundation is mature, our ideal voting system would be deferred in favor of smaller applications to test, refine, and prove out the platform.

Today, I would like to announce the first such application: Pollaris. Named after Polaris, the star which has guided navigators throughout ages past, this initial application will serve as our guiding light. Pollaris will help us to ensure that the platform we build adequately addresses the difficulties of developing blockchain-based applications while enabling us to test our design principles and guidelines to ensure that our solutions are indeed intuitive to people of all walks of life.

In practical terms, Pollaris is a simple polling application intended for use in communities to establish consensus on any issue they may face. The app will be used by administrators to gain insight into member sentiments on the issues at hand, enabling faithful management of the community’s resources in accordance with member preferences. Initial target markets for the app will include Homeowner’s Associations (HOAs) and collegiate student government organizations.

Pollaris will be developed as a fully open source solution, with rigorous security and transparency standards. The technologies will be open for all to inspect, deploy, and modify as they see fit. Follow My Vote will also offer a fully supported and hosted option for customers who lack either the technical expertise or the desire to manage their own instance.

To ensure that Pollaris is both a commercial success and an effective North Star for our platform, Follow My Vote is pulling out all the stops and challenging all of our assumptions about software design. We are dissatisfied with current standards of software usability and intuitiveness, particularly in blockchain software. Rather than simply point out problems, we aspire to reconsider everything about how software works and lay the groundwork for a new paradigm of human/computer interaction.

Many of the most well-designed and intuitive applications in existence today required armies of developers over sprawling timelines backed by massive budgets, and despite this investment the results leave people frustrated all too often.

Follow My Vote plans to address these issues using a combination of design and technology. First, we will design our software to be self-describing by incorporating detailed information describing the application into the code that comprises it. We will also create systems to allow people to inspect the elements and controls of the application to determine what they do and why they are in a particular status. The application will feature a tutorial mode on first launch showing people how to use the app and how to inspect the components. Finally, we will build a live support system directly into the app to provide human help to users on demand.

Of course, all of these designs require technology to substantiate them and we do not want a one-time solution, but a reusable new approach to developing software. Therefore while we will first build, test, and refine these solutions as part of Pollaris development, they will be generalized into the platform to serve future projects both by our team and others.

While Pollaris is a conceptually simple app, I hope I’ve made clear that there is a great deal of care and attention going into its design and that getting it just right is no small undertaking. Moreover, Pollaris is only the beginning; we have some really exciting ideas to build upon the foundations described above. It will take time to prove all of this out and I can’t say yet how much will make it into the first release, but I do believe when it’s done it will have been well worth the wait.

I’ll be writing more posts as we go forward to announce more about what we’re working on here at Follow My Vote and to go into deeper technical detail on our designs and our ideas, so stay tuned for more. Thank you for reading!

The post Introducing Pollaris appeared first on Follow My Vote.

Things are looking quite a bit brighter now, though, and we are once again confident in our ability to deliver what we promised. Over the past years, the difficulties we encountered affected each of us who make up Follow My Vote differently, and tested our resolve, but over time, we have been refined by this fire, and it has clarified our vision and strengthened our resolve to continue. We have examined our situation and ascertained our position, charted a course forward, and embarked upon the journey.

Make no mistake, Follow My Vote is fundamentally the same today as it was when last we spoke. We are a small crew of three guys, brought together by a shared vision of a better world, based upon systems that work for everyone, where the best ideas are gathered, tested, refined, implemented, and improved. A world where communication is easier and more effective, where societies are formed by individuals empowered to speak out about the problems they see, propose solutions to those problems, experiment on how to implement those solutions, and collaborate on which implementations are most safe, effective, scalable, economical, and desirable.

Having surpassed the difficulties that beset us, we are as convinced as ever before that this vision can be realized, but we also have greater clarity on just how big that vision is, and how much work it will take to bring it to fruition. We have identified the necessary groundwork and have secured operational funding to begin laying the foundations. We have formulated a plan and are currently working on its execution.

In order to succeed in making our vision a reality, we have realized that we must be more open and transparent about that vision, the plans to realize it, and the path and progress to getting there. Today, we are sharing our vision and plans as they exist now, and we are committing to maintain an open dialogue as to our position and progress at every step along the way. As a first step along this path, in addition to this article, we are also giving to the world our pending patent for one of humanity’s most important solutions for scalable communication: a secure, anonymous, end-to-end verifiable online voting system.

The importance of a new solution for voting is clearer now than ever before. Confidence in election integrity would seem to be at an all time low, and the need is great for more scalable and accessible solutions, without compromising on integrity, security, and privacy. Some solutions are already being proposed and brought to the market, but these solutions are dangerously ill-prepared to meet the needs of modern elections, and we at Follow My Vote are gravely concerned that the market may be swayed by these ineffective half-measures. This makes ever more clear the pressing need for transparency and education on the concerns involved in designing an online voting system to meet the rigorous security and voter privacy requirements necessary to re-establish confidence in election integrity.

At the same time, we are fully aware that voting is only one of humanity’s communication needs. New solutions are necessary to support all kinds of communication with guarantees of integrity, authenticity, and privacy. While the internet has become the de facto solution for our communication problems, the technologies which exist today are inadequate to provide for the next generation of services and applications supporting communication at scale. By creating new internet technologies which adequately support the rigorous requirements of online elections, we simultaneously establish a foundation and platform, not only for voting solutions, but myriad other applications as well.

Since 2014, Follow My Vote has had a plan for an end-to-end verifiable, anonymous online voting system secure enough to support national elections. The difficulty is that the current solutions to problems fundamental to all online applications were simply not intended to withstand the kinds of attacks we expect and observe to be leveled against national elections. We need solutions to these problems the same as everyone else, but whereas the existing approaches may be adequate for other applications, if we relied upon these technologies, our voting system’s security would be decimated and it would be feasible that advanced or well-positioned attackers could successfully interfere with the results.

In the intermediate years, we have struggled to keep the company afloat while searching for solutions that could solve the problems well enough that we could build our system and deploy it with confidence that its foundations were strong enough to deliver on its promise without crumbling under the load of real-world usage and attacks. At the same time, we’ve been confronted by numerous dilemmas where we could compromise the principles informing the design of our solutions to make them more practical or popular, but at the cost of making the resultant system less trustworthy or easier to defeat.

Through these trials, we have come to understand that Follow My Vote’s purpose in the market, the company’s raison d’être, is to make, with absolute and honest purity of intent and principles, an online voting system with absolutely zero compromise on security and voter privacy. We know that honesty and principles are rigidly inflexible, and to hold ourselves to these standards, we will make decisions which will be unpopular and politically incorrect. Nevertheless, we will hold to principle and create a solution with zero compromise in the integrity of its design or its results, wherever that takes us. Our software will be open source and free to modify, and others will be welcome to make dilute versions which they find more palatable, but we will keep our designs on the straight and narrow path of the greatest possible integrity and resilience.

The first step on this path, then, is to share with the public our plans to make such a system logistically and technically feasible. As we have stated above, existing solutions for creating online applications are not adequate to support our online voting system. A secure online voting system must not only give voters incontrovertible proof that their decision was accurately recorded and included in the tally, but it must also be resilient enough that it would be infeasible to prevent voters from successfully casting their votes, and it must ensure that the voter’s privacy is absolute and unbreakable by any entity within the system, while simultaneously guaranteeing that only legitimate voters can successfully cast a vote and that each voter gets exactly one vote. To ensure that all of these constraints are met, a protocol, or the rules and procedures governing how the system works and how information is handled, is established and an application is used by voters to easily and intuitively interact with the system according to protocol. This application is written in software code and that code must be downloaded to the voter’s device or computer to enable them to interact with the system. Once the voter has the application, they use it and it locates and communicates with other computers in the system on their behalf. All of these processes must succeed, and all of the information must be handled with integrity, even under attack, even when the attackers are well funded, technically advanced, and enjoy privileged positions of power from which to corrupt the system.

There are two universal problems to be solved in making any application based on communication technology such as the internet. First is getting the application to the human so they can use it, and second is supporting the application’s communications as it operates on the human’s behalf. Both must be conducted without a reasonable possibility of corruption. The first problem, getting the application to the human, is typically solved today by downloading the application over the web or from an app store. The second problem, supporting the application while it locates and communicates with other computers online, is typically solved today using the Domain Name System (DNS) and Certificate Authorities (CAs). Unfortunately, all of these solutions — web downloads, app stores, DNS, and CAs — are susceptible to easy attack if the attacker is adequately advanced or enjoys a convenient position of power. These solutions were designed only to be secure against common attackers without powerful positions, but a voting system needs to be secure even against attackers who happen to control, or can seize control of, instrumental computers within the system. As a result, these technologies constitute a weak foundation for an online voting system, and any system based upon them, no matter how much security is added later on, is susceptible to attack.

Furthermore, the aforementioned technologies are not highly resilient to outages and failures. If key systems fail, the application may not load, may not function, or may not successfully locate other computers within the system. For highly sensitive applications, this is not adequate. In the immortal words of Apollo 13, “Failure is not an option.” In the face of outages and failures, the technology must automatically recover or choose a new path that does not fail. If the outages are insurmountable, then the human must be given clear and practical guidance on how to proceed successfully. While this kind of redundancy has been achieved with existing internet solutions, it requires enormously expensive and complex systems which require substantial expertise and resources to construct and maintain, and even then, the solutions are centrally managed, meaning that they are resilient to outages, but not to well-positioned attacks. To support an online voting system, simpler strategies for resilience and redundancy must be found, which do not require unusual skill or resources and central management.

To meet these needs, Follow My Vote has examined the technologies available and selected more fundamental technologies which have been around for longer and provide more basic services with few conveniences relied on by modern application developers. These older and less convenient, but more durable technologies remain in widespread use and are still fundamental to our modern systems today, and thus they constitute a strong foundation upon which to build. From this position of strength, new solutions can be implemented which leverage the experience gained from the modern web to provide all of the power and convenience to which modern developers have become accustomed and more, but without relying on less robust or resilient approaches to provide them.

Out of these solutions can be fashioned a new platform for communication applications. This is not just a solution for voting; indeed, if it were then it would be questionable whether the design could be securely maintained and withstand the tests of time. By creating a solution to benefit an entire new generation of online applications, we ensure that, as with the web today, our platform is continuously examined and improved as it supports a vast and growing array of vested interests, including many that are newly made possible by the greater architectural stability of the proposed platform.

Only once this platform is fully established will we continue with building our anonymous, end-to-end verifiable online voting solution. We will construct simpler applications to prove out the platform and demonstrate its proper use, but until the platform is ready, we cannot build a ruggedly secure solution for elections, and we dare not risk redirecting resources away from the foundation prematurely. This will ensure the platform is rugged and stable before we begin building a mission critical solution on top of it, but by building smaller solutions on top of it in the meantime, we use our own designs and ensure that the platform is practical and convenient enough for everyday usage as well.

Although Follow My Vote has had our struggles in the past, we’ve learned many lessons and we’ve gained experience so that now, we have confidence that we can build the system we set out to build years ago. These difficult lessons have shown us just how far we have yet to go, but also how far we have come, and they have given us the resourcefulness we will need to succeed in walking the path we have chosen for ourselves. We know that the work cut out for us is enormous, but we also see that the need for it is as great now as it ever has been, and we are galvanized by the threat that the market may lose hope that a truly secure and integral online voting system is possible and settle for a dangerous half-measure.

Having now fully ascertained our position and regained our footing, we know the magnitude and scope of the work we have yet to do, and it is surely daunting; nevertheless, we believe that the work can be done, and we know that it will be worth it when it is finished.

As we press onward, we know the value of transparency and openness and we commit to be more communicative as to the development of our solutions going forward. We hope that this visibility gives the market hope that true solutions are possible, not only for voting, but for a host of applications targeting critical social and societal issues, and we hope that others will support our efforts to create a solution not only for our own applications, but for everyone’s. We thank you for your attention in reading this article and following our progress as we follow through with all that is within us on the work we have described.

The post Allow Us to (Re)Introduce Ourselves appeared first on Follow My Vote.