How To Set Up a VPN Connection During Builds

Configuring VPN Connection during a Build

In case you need to connect to a private network during your builds, or if you want to restrict access to your environment to a specific IP address, we suggest configuring a VPN connection as follows.

Prerequisites:

Note that you must use one of the following executors:

machine [Linux] executor (Available machine images)

OpenVPN (2.x)

  • Base64-encode the OpenVPN client configuration file, and store it as an environment variable.
  • If the VPN client authentication is credentials-based (user-locked profile), you'll also need to add the username and password as environment variables (VPN_USER and VPN_PASSWORD).
version: 2.1
workflows:
  btd:
    jobs:
      - build
jobs:
  build:
    machine:
      image: ubuntu-2004:202201-02
    steps:
      - run:
          name: Install OpenVPN
          command: |
            sudo apt-get update
            sudo apt-get install openvpn openvpn-systemd-resolved
- run: name: Check IP before VPN connection command: | ip a echo "Public IP before VPN connection is $(curl checkip.amazonaws.com)"
- run: name: VPN Setup background: true command: | echo $VPN_CLIENT_CONFIG | base64 --decode > /tmp/config.ovpn

if grep -q auth-user-pass /tmp/config.ovpn; then
if [ -z "${VPN_USER:-}" ] || [ -z "${VPN_PASSWORD:-}" ]; then
echo "Your VPN client is configured with a user-locked profile. Make sure to set the VPN_USER and VPN_PASSWORD environment variables"
exit 1
else
printf "$VPN_USER\\n$VPN_PASSWORD" > /tmp/vpn.login
fi
fi

vpn_command=(sudo openvpn
--config /tmp/config.ovpn
--route 169.254.0.0 255.255.0.0 net_gateway
--script-security 2
--up /etc/openvpn/update-systemd-resolved --up-restart
--down /etc/openvpn/update-systemd-resolved --down-pre
--dhcp-option DOMAIN-ROUTE .)

if grep -q auth-user-pass /tmp/config.ovpn; then vpn_command+=(--auth-user-pass /tmp/vpn.login) fi

ET_phone_home=$(ss -Hnto state established '( sport = :ssh )' | head -n1 | awk '{ split($4, a, ":"); print a[1] }')
echo $ET_phone_home

if [ -n "$ET_phone_home" ]; then
vpn_command+=(--route $ET_phone_home 255.255.255.255 net_gateway)
fi

for IP in $(host runner.circleci.com | awk '{ print $4; }')
do
vpn_command+=(--route $IP 255.255.255.255 net_gateway)
echo $IP
done

for SYS_RES_DNS in $(systemd-resolve --status | grep 'DNS Servers'|awk '{print $3}')
do
vpn_command+=(--route $SYS_RES_DNS 255.255.255.255 net_gateway)
echo $SYS_RES_DNS
done

"${vpn_command[@]}" > /tmp/openvpn.log
- run: name: Wait for the connection to be established and check IP
command: |
counter=1
until [ -f /tmp/openvpn.log ] && [ "$(grep -c "Initialization Sequence Completed" /tmp/openvpn.log)" != 0 ] || [ "$counter" -ge 5 ]; do
((counter++))
echo "Attempting to connect to VPN server..."
sleep 1;
done

if [ ! -f /tmp/openvpn.log ] || (! grep -iq "Initialization Sequence Completed" /tmp/openvpn.log); then
printf "\nUnable to establish connection within the allocated time ---> Giving up.\n"
else
printf "\nVPN connected\n"
printf "\nPublic IP is now %s\n" "$(curl -s http://checkip.amazonaws.com)"
fi - run: name: Run commands in our infrastructure command: | # A command # Another command

  - run:
name: Disconnect from OpenVPN
command: |
sudo killall openvpn || true
when: always

 

OpenVPN Connect (OpenVPN 3)

version: 2.1
workflows:
  btd:
    jobs:
      - build
jobs:
  build:
    machine:
      image: ubuntu-2004:202201-02
    steps:
      - run:
          name: Install OpenVPN
          command: |
            sudo apt update && sudo apt install apt-transport-https
sudo wget https://swupdate.openvpn.net/repos/openvpn-repo-pkg-key.pub
sudo apt-key add openvpn-repo-pkg-key.pub

sudo wget -O /etc/apt/sources.list.d/openvpn3.list https://swupdate.openvpn.net/community/openvpn3/repos/openvpn3-$(sed 's/UBUNTU_CODENAME=//;t;d' /etc/os-release).list
sudo apt update && sudo apt install openvpn3 openvpn-systemd-resolved
- run: name: Check IP before VPN connection command: |
ip a
echo "Public IP before VPN connection is $(curl checkip.amazonaws.com)"
- run: name: VPN Setup background: true command: | echo $VPN_CLIENT_CONFIG | base64 --decode > /tmp/config.ovpn
ET_phone_home=$(ss -Hnto state established '( sport = :ssh )' | head -n1 | awk '{ split($4, a, ":"); print a[1] }')
## In case you're using an image with Ubuntu < 20.04, replace the above line with:
## ET_phone_home=$(ss -an | grep 'ESTAB .*:22' | head -n1 | awk '{ split($6, a, ":"); print a[1] }')
echo $ET_phone_home

if [ -n "$ET_phone_home" ]; then echo "route $ET_phone_home 255.255.255.255 net_gateway" >> /tmp/config.ovpn
fi

echo "route 169.254.0.0 255.255.0.0 net_gateway" >> /tmp/config.ovpn

for SYS_RES_DNS in $(systemd-resolve --status | grep 'DNS Servers'|awk '{print $3}')
do
echo "route $SYS_RES_DNS 255.255.0.0 net_gateway" >> /tmp/config.ovpn
echo $SYS_RES_DNS
done

for IP in $(host runner.circleci.com | awk '{ print $4; }')
do
echo "route $IP 255.255.255.255 net_gateway" >> /tmp/config.ovpn
echo $IP
done

# This will start the connection
sudo openvpn3 session-start --config /tmp/config.ovpn > /tmp/openvpn.log
- run: name: Wait for the connection to be established and check
command: |
counter=1
until sudo openvpn3 sessions-list|grep "Client connected" || [ "$counter" -ge 5 ]; do
((counter++))
echo "Attempting to connect to VPN server..."
sleep 1;
done

if ( ! sudo openvpn3 sessions-list|grep "Client connected"); then
printf "\nUnable to establish connection within the allocated time ---> Giving up.\n"
else
printf "\nVPN connected\n"
printf "\nPublic IP is now %s\n" "$(curl -s https://checkip.amazonaws.com)"
fi
- run: name: Run commands in our infrastructure command: | # A command # Another command

  - run:
name: Disconnect from OpenVPN
command: |
SESSION_PATH=$(sudo openvpn3 sessions-list | grep Path | awk -F': ' '{print $2}')
echo $SESSION_PATH
sudo openvpn3 session-manage --session-path $SESSION_PATH --disconnect
when: always

 

L2TP

To set up an L2TP VPN connection, we recommend referring to this guide.

We suggest storing VPN_SERVER_IPVPN_IPSEC_PSKVPN_USER and VPN_PASSWORD as environment variables. Ideally, you might want to base64-encode VPN_IPSEC_PSK before storing it; you'll need to decode it during the build.

Also, we suggest storing the default gateway IP address in an environment variable:

  • DEFAULT_GW_IP=$(ip route show default|awk '{print $3}')

 

macos executor (Supported Xcode versions)

  • Base64-encode the OpenVPN client configuration file, and store it as an environment variable.
  • If the VPN client authentication is credentials-based (user-locked profile), you'll also need to add the username and password as environment variables (VPN_USER and VPN_PASSWORD).
version: 2.1
workflows:
  btd:
    jobs:
      - build
jobs:
  build:
    macos:
xcode: "13.2.1" steps: - run: name: Install OpenVPN command: | brew install openvpn
curl https://raw.githubusercontent.com/andrewgdotcom/openvpn-mac-dns/master/etc/openvpn/update-resolv-conf --output /tmp/update-resolv-conf
chmod +x /tmp/update-resolv-conf
- run: name: Check IP before VPN connection command: | ifconfig echo "Public IP before VPN connection is $(curl checkip.amazonaws.com)"
- run: name: VPN Setup command: | echo $VPN_CLIENT_CONFIG | base64 --decode | tee /tmp/config.ovpn 1>/dev/null

if grep auth-user-pass /tmp/config.ovpn; then
if [ -z "${VPN_USER:-}" ] || [ -z "${VPN_PASSWORD:-}" ]; then
echo "Your VPN client is configured with a user-locked profile. Make sure to set the VPN_USER and VPN_PASSWORD environment variables"
exit 1
else
printf "$VPN_USER\\n$VPN_PASSWORD" > /tmp/vpn.login
sed -i config.bak 's|^auth-user-pass.*|auth-user-pass /tmp/vpn\.login|' /tmp/config.ovpn
fi
fi

touch /tmp/openvpn.log

phone_home="$(netstat -an | grep '\.2222\s.*ESTABLISHED' | head -n1 | awk '{ split($5, a, "."); print a[1] "." a[2] "." a[3] "." a[4] }')"
echo -e "\nroute $phone_home 255.255.255.255 net_gateway" | tee -a /tmp/config.ovpn
echo $phone_home

cat \<< EOF | sudo tee /Library/LaunchDaemons/org.openvpn.plist 1>/dev/null
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>org.openvpn</string>
<key>Program</key>
<string>/usr/local/opt/openvpn/sbin/openvpn</string>
<key>ProgramArguments</key>
<array>
<string>/usr/local/opt/openvpn/sbin/openvpn</string>
<string>--config</string>
<string>/tmp/config.ovpn</string>
<string>--route</string>
<string>169.254.0.0 255.255.0.0 net_gateway</string>
<string>--script-security</string>
<string>2</string>
<string>--up</string>
<string>/tmp/update-resolv-conf</string>
<string>--down</string>
<string>/tmp/update-resolv-conf</string>
</array>
<key>RunAtLoad</key>
<false/>
<key>TimeOut</key>
<integer>90</integer>
<key>StandardErrorPath</key>
<string>/tmp/openvpn.log</string>
<key>StandardOutPath</key>
<string>/tmp/openvpn.log</string>
<key>KeepAlive</key>
<false/>
</dict>
</plist>
EOF

echo "Public IP before VPN connection is > $(curl http://checkip.amazonaws.com)"

# This will start the connection
sudo launchctl load /Library/LaunchDaemons/org.openvpn.plist
sudo launchctl start org.openvpn
- run: name: Wait for the connection to be established and check
command: |
counter=1
until [ -f /tmp/openvpn.log ] && [ "$(grep -c "Initialization Sequence Completed" /tmp/openvpn.log)" != 0 ] || [ "$counter" -ge 5 ]; do
((counter++))
echo "Attempting to connect to VPN server..."
sleep 1
done

if [ ! -f /tmp/openvpn.log ] || (! grep -iq "Initialization Sequence Completed" /tmp/openvpn.log); then
printf "\nUnable to establish connection within the allocated time ---> Giving up.\n"
else
printf "\nVPN connected\n"
printf "\nPublic IP is now %s\n" "$(curl -s http://checkip.amazonaws.com)"
fi
- run: name: Run commands in our infrastructure command: | # A command # Another command

  - run:
name: Disconnect from OpenVPN
command: sudo launchctl stop org.openvpn
when: always

 

windows executor (Windows executor images)

  • Base64-encode the OpenVPN client configuration file, and store it as an environment variable.
  • If the VPN client authentication is credentials-based (user-locked profile), you'll also need to add the username and password as environment variables (VPN_USER and VPN_PASSWORD).
version: 2.1

orbs:
  win: circleci/windows@2.2.0
  
workflows:
  btd:
    jobs:
      - build
jobs:
  build:
    executor:
      name: win/default
      shell: bash.exe
      
    steps:
      - run:
          name: Install OpenVPN
          command: |
            choco install openvpn
            
      - run:
          name: Check IP before VPN connection
          command: echo "Public IP before VPN connection is $(curl checkip.amazonaws.com)"
          
      - run:
          name: VPN Setup
          command: |
echo $VPN_CLIENT_CONFIG | base64 --decode > /C/PROGRA~1/OpenVPN/config/config.ovpn if grep auth-user-pass "/C/PROGRA~1/OpenVPN/config/config.ovpn"; then
if [ -z "${VPN_USER:-}" ] || [ -z "${VPN_PASSWORD:-}" ]; then
echo "Your VPN client is configured with a user-locked profile. Make sure to set the VPN_USER and VPN_PASSWORD environment variables"
exit 1
else
printf "$VPN_USER\\n$VPN_PASSWORD" > /C/PROGRA~1/OpenVPN/config/vpn.login
sed -i 's|^auth-user-pass.*|auth-user-pass vpn\.login|' /C/PROGRA~1/OpenVPN/config/config.ovpn
fi
fi
### IMPORTANT: Include the following 3 lines to exclude the connection from CircleCI and the link-local range phone_home=$(netstat -an | grep ':22 .*ESTABLISHED' | head -n1 | awk '{ split($3, a, ":"); print a[1] }') echo -e "\nroute $phone_home 255.255.255.255 net_gateway" | tee -a "/C/PROGRA~1/OpenVPN/config/config.ovpn"
echo "route 169.254.0.0 255.255.0.0 net_gateway" | tee -a "/C/PROGRA~1/OpenVPN/config/config.ovpn"

# Create and start the OpenVPN service
sc.exe create "OpenVPN" binPath= "C:\PROGRA~1\OpenVPN\bin\openvpnserv.exe"
net start "OpenVPN" - run: name: Wait for the connection to be established and check command: |
counter=1 until [ $(cat /c/progra~1/openvpn/log/config.log|grep -c "Initialization Sequence Completed") == 0 ] || [ "$counter" -ge 5 ]; do
((counter++)) echo "Attempting to connect..." sleep 1; done
if [ ! -f /c/progra~1/openvpn/log/config.log ] || (! grep -iq "Initialization Sequence Completed" /c/progra~1/openvpn/log/config.log); then
printf "\nUnable to establish connection within the allocated time ---> Giving up.\n"
else
printf "\nVPN connected\n"
printf "\nPublic IP is now %s\n" "$(curl -s http://checkip.amazonaws.com)"
fi - run: name: Run commands in our infrastructure command: | # A command # Another command - run: name: Disconnect from OpenVPN command: net stop "OpenVPN" when: always

 

Additional Notes:

Please note: any VPN configuration steps are done at your own risk and can break if any changes occur in IP Ranges or Routing tables.

In the event, your configuration is met with This job was claimed but has not received a heartbeat in over 5 minutes error. Try to add routing exclusions to unblock your build using our example below which will require some modifications to meet your build needs.

OpenVPN Example:

  ET_phone_home=$(ss -Hnto state established '( sport = :ssh )' | head -n1 | awk '{ split($4, a, ":"); print a[1] }')
  DEFAULT_GW="$(ip route show default|awk '{print $3}')"
  echo "Original default gateway is $DEFAULT_GW"

  if [ -n "$ET_phone_home" ]; then
    sudo ip route add "$ET_phone_home"/32 via "$DEFAULT_GW"
    echo "Added route to $ET_phone_home/32 via default gateway"
  fi

  for IP in $(host runner.circleci.com | awk '{ print $4; }')
    do
      sudo ip route add "$IP"/32 via "$DEFAULT_GW"
      echo "Added route to $IP/32 via default gateway"
  done

  sudo ip route add 169.254.0.0/16 via "$DEFAULT_GW"

 

 

 

 

 

 

Was this article helpful?
12 out of 16 found this helpful

Comments

0 comments

Article is closed for comments.