In the previous post, we covered the design and architecture of our backconnect proxy. In this post, we will extend parts 2 and 3 to create a Minimum Viable Product (MVP).

Minimum Viable Product (MVP)—A product designed with a minimal scope to reduce the development time and cost.

The goal of the MVP is to minimize the scope and therefore minimize the time, and cost. The addition of more features increases the scope and therefore will require more time and increase the cost. MVPs are, by design, meant to quickly reach the viability stage. This ensures the developers maintain a feedback loop with the most crucial party; the user. With feedback, the development team can modify the development plan as needed to address concerns or suggestions by clients early.

With a longer development cycle, critical concerns or suggestions will come later. After significant effort has been invested in development. However, feedback given prior to a product's creation is primarily speculative. By limiting scope to only the necessary features a product can be produced within a shorter time frame to elicit feedback. Therefore an MVP should only consist of features that it absolutely needs. For example, an MVP for a calculator could do without a modulo operator. However, a calculator MVP should at least support the most basic operations (+, -, ×, ÷).

The scope of an MVP will vary depending on the specifics of the product. For the backconnect proxy, the scope of an MVP will need to proxy requests through our list of available IP addresses. Instead of dynamically loading the list of available IP addresses, we will simply load in a predefined list of IP addresses. The design for the MPV is shown below.

Side note—accessing multiple IP addresses on a local development machine is difficult. One alternatively, is to use a server with 2 or more IP addresses bound to it for development. This would be easier to set up, though the cost would be higher. We opted to make use of an existing server and purchase 2 additional IPs to use for development.

Building an MVP

With the features for the MVP decided, we will proceed with building the MVP. First we will start by setting up a basic proxy using goproxy.

proxy := goproxy.NewProxyHttpServer()
proxy.Verbose = true
log.Fatal(http.ListenAndServe(":8080", proxy))

Next we will integrate our custom request handler for goproxy. This request handler will forward the request to the destination using the alternative IP address. We will start by including a custom request handler definition.

proxy.OnRequest().DoFunc(
    func(r *http.Request,ctx *goproxy.ProxyCtx)(*http.Request,*http.Response) {
        // Handle using alternative IP
        return r,nil
    })

In this handler we have access to both the http.Request and an internal *goproxy.ProxyCtx. http.Request is the request that will be sent to the destination. Whereas goproxy.ProxyCtx is used by the library to process the request. If a goproxy.RoundTripper is defined then goproxy will use that goproxy.RoundTripper to perform the request. Below we define our custom Rotater struct. Rotater will maintain the list of available IP addresses and define the custom Dialer handler.

type Rotater struct {
    availableIPs []string
}

func (r *Rotater) nextIP() string {
    // TODO implement
    return ""
}

func (r *Rotater) CustomDialer(ctx context.Context, network string, addr string) (net.Conn, error) {
    altIP := r.nextIP()
    ipAddress := net.ParseIP(altIP)
    d := net.Dialer{
        LocalAddr: &net.TCPAddr{
            IP: ipAddress,
        },
        Timeout:   30 * time.Second,
        KeepAlive: 30 * time.Second,
    }
    return d.Dial(network, addr)
}

This differs slightly from our PoC as we define the custom dialer function as a member of a struct Rotater. We also define nextIP which we will implement later. For now we will define an implementation of the goproxy.RoundTripper interface using http.Transport.

type TransportWrapper struct {
    transport *http.Transport
}

func (t *TransportWrapper) RoundTrip(req *http.Request, ctx *goproxy.ProxyCtx) (*http.Response, error) {
    return t.transport.RoundTrip(req)
}

func WrapTransport(t *http.Transport) *TransportWrapper {
    return &TransportWrapper{t}
}
}

We then override goproxy.ProxyCtx.RoundTripper to use our custom dialer.

proxy.OnRequest().DoFunc(func(r *http.Request, ctx *goproxy.ProxyCtx) (*http.Request, *http.Response) {
    ctx.RoundTripper = WrapTransport(
        &http.Transport{
            Proxy:                 http.ProxyFromEnvironment,
            DialContext:           rotater.CustomDialer,
            MaxIdleConns:          1,
            IdleConnTimeout:       90 * time.Second,
            TLSHandshakeTimeout:   10 * time.Second,
            ExpectContinueTimeout: 1 * time.Second,
            DisableKeepAlives:     true,
        },
    )
    return r, nil
})

Now, we need to define nextIP. To do so, we'll update Rotater as follows:

type Rotater struct {
    availableIPs []string
    currentIndex int
    m            *sync.Mutex
}

currentIndex maintains the next IP in the list of availableIPs to use. Whereas m is a mutex for synchronization access to the currentIndex. Based off our pseudocode from part 4, we will want to

1. Obtain mutex lock
2. Perform bound checks
3. Retrieve the next IP
4. Increment currentIndex
5. Release lock
6. Return the next IP

In the code snippet we perform this synchronized task. However, for unlocking the mutex we make use of Golang's defer feature which will ensure a call is made post return.

func (r *Rotater) nextIP() string {
    r.m.Lock()
    defer r.m.Unlock()
    if r.currentIndex >= len(r.availableIPs) {
        r.currentIndex = 0
    }
    n := r.availableIPas[r.currentIndex]
    r.currentIndex += 1
    return n
}

For the bounds checking we ensure if currentIndex is out of range for availableIPs we wrap around the index. This handles the case where we've iterated through the list of availableIPs. However this will also help in cases where the list of availableIPs shrinks.

With that we've built a basic backconnect proxy that will rotate through a list of available IPs, we will simply define our Rotater with a predefined list of IP addresses and perform some testing.

rotater := &Rotater{
    availableIPs: []string{
        "x.x.x.1",
        "x.x.x.2",
    },
    m: &sync.Mutex{},
}

You can review the full example here.

Testing

With our MVP built we will perform some tests to ensure it works correctly. For the tests we will use the following configuration:

Host server IP (IPh): z.z.z.z
    IP1: x.x.x.1
    IP2: x.x.x.2

The backconnect proxy can be build and run using the following:

go build simple_proxy_rotate.go
./simple_proxy_rotate

In these tests we have provided our own client (testclient.go) to interact with the backconnect proxy. One can build testclient.go using the following:

go build testclient.go

This in part is due to our need to integrate the proxy with Go clients. However, each use of testclient can be replaced with a corresponding call to curl. For example in test 1 we will call:

./testclient http://ip-api.com/json http://localhost:8080

The corresponding curl call would be:

curl -x http://localhost:8080 http://ip-api.com/json

Test 1

In a separate terminal we will use testclient.go to make requests to a site that will respond with the requester’s IP address information. For HTTP requests, we will make requests to http://ip-api.com/json. For HTTPS requests, we will make requests to https://jsonip.com/. In this first test we will send 3 requests to rotate through all available IP addresses and wrap back to the first IP address.

$ ./testclient http://ip-api.com/json http://localhost:8080
  Request Took:  64.254512ms
  "{\"query\":\"x.x.x.1\",\"status\":\"success\",\"country\":\"Canada\",\"countryCode\":\"CA\",\"region\":\"XX\",\"regionName\":\"XXXX\",\"city\":\"XXXX\",\"zip\":\"XXX\",\"lat\":X.XXX,\"lon\":X.XXX,\"timezone\":\"America/XXXX\",\"isp\":\"XXXX\",\"org\":\"\",\"as\":\"XXXX\"}"
$ ./testclient http://ip-api.com/json http://localhost:8080
  Request Took:  58.893801ms
  "{\"query\":\"x.x.x.2\",\"status\":\"success\",\"country\":\"Canada\",\"countryCode\":\"CA\",\"region\":\"XX\",\"regionName\":\"XXXX\",\"city\":\"XXXX\",\"zip\":\"XXX\",\"lat\":X.XXX,\"lon\":X.XXX,\"timezone\":\"America/XXXX\",\"isp\":\"XXXX\",\"org\":\"\",\"as\":\"XXXX\"}"
$ ./testclient http://ip-api.com/json http://localhost:8080
  Request Took:  50.968575ms
  "{\"query\":\"x.x.x.1\",\"status\":\"success\",\"country\":\"Canada\",\"countryCode\":\"CA\",\"region\":\"XX\",\"regionName\":\"XXXX\",\"city\":\"XXXX\",\"zip\":\"XXX\",\"lat\":X.XXX,\"lon\":X.XXX,\"timezone\":\"America/XXXX\",\"isp\":\"XXXX\",\"org\":\"\",\"as\":\"XXXX\"}"

Our tests confirm the backconnect proxy successfully proxies the client requests. First through IP1 followed by IP2 and then wraps back to IP1. IPh is never exposed and the sequence is also correct.

Test 2

In the second test we will test using another site which automatically upgrades HTTP requests to HTTPS requests. This service, http://jsonip.com/ failed to correctly proxy due to this upgrade.

$ ./testclient http://jsonip.com/ http://localhost:8080
  Request Took:  753.205248ms
  "{\"ip\":\"z.z.z.z\",\"geo-ip\":\"https://getjsonip.com/#plus\",\"API Help\":\"https://getjsonip.com/#docs\"}"

IPh is exposed and neither IP1 or IP2 are used. The backconnect proxy has failed to correctly proxy the request. Instead the proxy handles it as a redirect (301) (see below)

[001] INFO: Got request / jsonip.com GET http://jsonip.com/
[001] INFO: Sending request GET http://jsonip.com/
[001] INFO: Received response 301 Moved Permanently
[001] INFO: Copying response to client 301 Moved Permanently [301]
[001] INFO: Copied 169 bytes to client error=
[002] INFO: Running 2 CONNECT handlers
[002] INFO: on 0th handler: &{2  0x65de40} jsonip.com:443
[002] INFO: Assuming CONNECT is TLS, mitm proxying it
[002] INFO: signing for jsonip.com
[003] INFO: req jsonip.com:443
[003] INFO: Sending request GET https://jsonip.com:443/json
[003] INFO: resp 200 OK
[002] INFO: Exiting on EOF

Test 3

For this test we will use a HTTPS site again. However, this time we will make a HTTPS request directly.

$ ./testclient https://jsonip.com/ http://localhost:8080
  Request Took:  264.933031ms
  "{\"ip\":\"z.z.z.z\",\"geo-ip\":\"https://getjsonip.com/#plus\",\"API Help\":\"https://getjsonip.com/#docs\"}"

Again the backconnect proxy uses IPh instead of IP1 or IP2. A Review of the backconnect proxy logs provides a clue for how to fix this issue. Instead of handling as a 301 redirect, the proxy handles the request as Accepting CONNECT to jsonip.com:443. Breaking this down a little, 443 is the port for HTTPS requests. However the key point is the request type CONNECT.

[001] INFO: Running 0 CONNECT handlers
[001] INFO: Accepting CONNECT to jsonip.com:443

Further research of goproxy revealed a HandleConnectFunc which allows for a custom handler for CONNECT requests to be defined. The default behavior is a pass through called OkConnect. Another potential handler is MitmConnect (Man In The Middle Connect) which will permit our request handler to modify incoming HTTPS requests. The following code will configure the proxy to handle all CONNECT requests with a MitmConnect.

proxy.OnRequest().HandleConnectFunc(func(host string, ctx *goproxy.ProxyCtx) (*goproxy.ConnectAction, string) {
    return goproxy.MitmConnect, host
})

With the addition of the above code we perform our test once again.

$ ./testclient https://jsonip.com/ http://localhost:8080
  Request Took:  619.138015ms
  "{\"ip\":\"x.x.x.1\",\"geo-ip\":\"https://getjsonip.com/#plus\",\"API Help\":\"https://getjsonip.com/#docs\"}"
$ ./testclient https://jsonip.com/ http://localhost:8080
  Request Took:  620.338359ms
  "{\"ip\":\"x.x.x.2\",\"geo-ip\":\"https://getjsonip.com/#plus\",\"API Help\":\"https://getjsonip.com/#docs\"}"
$ ./testclient https://jsonip.com/ http://localhost:8080
  Request Took:  637.21919ms
  "{\"ip\":\"x.x.x.1\",\"geo-ip\":\"https://getjsonip.com/#plus\",\"API Help\":\"https://getjsonip.com/#docs\"}"

With the use of MitmConnect the backconnect proxy successfully proxied the HTTPS requests. The requests correctly use the proxy IP addresses in sequence:

• Request 1 uses IP x.x.x.1
• Request 2 uses IP x.x.x.2
• Request 3 uses IP x.x.x.1 (wraparound)

Conclusion

We have successfully built a backconnect proxy in Go. Our tests have shown the backconnect proxy works for HTTP and HTTPS requests. The IPh remains hidden and only IP1 and IP2 are visible to the destination servers. The proxy is limited however as it fails to correctly handle HTTPS upgrade redirects. For now, solving this problem is left to a motivated reader. If you'd like to review the full code examples please see them below:

• simple_proxy_rotate.go
• testclient.go

In part 5 we will add in dynamic IP loading and discuss other possible extensions including:

• Geo-Location filtering
• Dynamic reconfiguration
• Potential alternative rotation algorithms

We will then end the series with a review of our build out of a backconnect proxy, as well as some lessons learned.

Discuss on Hacker News, send us thoughts, or join the discussion below.