A reverse proxy and tunneling tool for webhooks

Overview

If you developed modules involving payment processors (Stripe webhooks, Paypal IPN), you are surely aware that an end-to-end transaction can involve a number of two-way, asynchronous confirmation / validation parts, where the payment processor calls your server. In effect, it serves to confirm (for both parties) the outcome of a transaction, as well as it having been initiated by a legitimate agent (you).

An obstacle occurring quite frequently in development mode is developers lacking an Internet-facing https endpoint. To their credit, Stripe attempted to mitigate the issue with a command line tool, that listens for any incoming webhooks events from your developer account and routes such events to your app server. The intent is commendable, but somehow incomplete. You will not be able to validate the request back to Stripe. They admit it with a note: "Responses from your server aren’t returned to Stripe and won’t show up in the Dashboard". Since they remain unvalidated, the loop is not really closed.

When this happens, the payment processor may (after multiple attempts to obtain validation) disable the endpoint, which is something we don't want to happen.

A good way to solve this is to use a reverse proxy tool that would act as a listener on a publicly facing endpoint and set up a tunnel between that and your local app server. The following diagram summarizes the exchanges generally needed for a payment transaction, and how such a tool helps:

Note that Step 1 in the diagram above is not always necessary. For example, after the first successful payment, subsequent recurring payments in cases where the payment processor stores the payment method, would trigger a webhook call at the designated interval. This means that the flow effectively starts at step 2.

Frp is an open source client-server combo I used, and which did the trick for me: https://github.com/fatedier/frp

Server-Side Setup

For this part, you need an Internet facing node where you set up Nginx and the frp server component. Any budget Linux server, where you have sudo access and has a fixed IP address assigned to it, is a good candidate for this. Those 5$ plans from Linode, Digital Ocean, and many others would fit the bill very well.

Some payment processors require valid https certificates for the webhooks endpoints, so you will need a domain (subdomain) pointing to it, and a Let's Encrypt certificate.

Assumptions (for the purpose of this article):

• The frp server binary is present, and located at /opt/frp/server
• Nginx was installed and is fully configurable
• The subdomain you own is johndoe.acmedevelendpoint123.com and you set up the DNS entries to point to that server
• The Let's Encrypt tool (certbot) is installed on the server
• The public-facing endpoint is https://johndoe.acmedevelendpoint123.com/mytestendpoint
• Your Linux username is johndoe

First, configure the frp server by specifying the bind port, and the http port. The bind_port variable specifies the port used between the frp server and the frp client. The vhost_http_port specifies the http port that frp server listens on, where Nginx (acting as an SSL terminator) will proxy the request received on https://johndoe.acmedevelendpoint123.com/mytestendpoint

# frps.ini
[common]
bind_port = 7000
vhost_http_port = 8172

Save this as /opt/frp/server/frps.ini

Next, set up the frp server Systemd service file, to make sure it starts on each reboot (create, for example, a /etc/systemd/system/s-frp.service file)

[Unit]
Description = ACMEDevelEndpoint123 FRP Service

[Service]
Type=simple
WorkingDirectory=/opt/frp/server
User=johndoe
Group=johndoe
StandardOutput=syslog+console
StandardError=syslog+console
ExecStart=/opt/frp/server/frps -c /opt/frp/server/frps.ini

[Install]
WantedBy=multi-user.target

Next, we want to set up Nginx. Setting it up with a proper Let's Encrypt certificate is outside the scope of this article, but this is how the final configuration file could look like below:

server {
	    server_name johndoe.acmedevelendpoint123.com;

	    # HTML root folder (ideally, place a catch-all index.html here for security)
	    root /home/johndoes/html;

	    # Add index.php to the list if you are using PHP
	    index index.html index.htm index.nginx-debian.html;

	    location / {
		    # First attempt to serve request as file, then
		    # as directory, then fall back to displaying a 404.
		    try_files $uri $uri/ =404;
	    }

	    location /mytestendpoint {

            proxy_redirect off;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
            proxy_set_header Host $http_host;
            proxy_set_header X-NginX-Proxy true;
            proxy_set_header Connection "";
            proxy_http_version 1.1;

		    proxy_connect_timeout       6000;
		    proxy_send_timeout          6000;
		    proxy_read_timeout          6000;
		    send_timeout                6000;

            proxy_pass http://localhost:8172;
        }

        listen 443 ssl; # managed by Certbot
        ssl_certificate /etc/letsencrypt/live/johndoe.acmedevelendpoint123.com/fullchain.pem; # managed by Certbot
        ssl_certificate_key /etc/letsencrypt/live/johndoe.acmedevelendpoint123.com/privkey.pem; # managed by Certbot
        include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
        ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

    }

    server {
        if ($host = johndoe.acmedevelendpoint123.com) {
            return 301 https://$host$request_uri;
        } # managed by Certbot


	    server_name johndoe.acmedevelendpoint123.com;
        listen 80;
        return 404; # managed by Certbot
    }

In the Nginx configuration setup above, we enforce a mandatory redirect to https, which is good practice in any case.