Set up a dedicated virtualization server on Packet.net

October 3, 2017 — 2 Comments

Packet is a hardware-as-a-service vendor that provides dedicated servers on demand at very low cost. For me and my readers, Packet offers a solution to the problem of using cloud services to run complex network emulation scenarios that require hardware-level support for virtualization. Packet users may access powerful servers that empower them to perform activities they could not run on a normal personal computer.

In this post, I will describe the procedure to set up an on-demand bare metal server and to create and maintain persistent data storage for applications. I will describe a generic procedure that can be applied to any application and that works for users who access Packet services from a laptop computer running any of the common operating systems: Windows, Mac, and Linux. In a future post, I will describe how I run network emulation scenarios on a Packet server.

Table of Contents

  1. Packet.net
    1. Controlling costs when using bare metal servers
    2. Create a Packet account and Login
    3. Create a project
  2. Generate SSH Keys
    1. Windows
    2. Mac
    3. Linux
    4. Copy public key to Packet.net
  3. Deploy a Server
  4. SSH Server on local machine
    1. Windows
    2. Mac
    3. Linux
  5. Set up the remote server
    1. Test X11 forwarding
  6. Create block storage
    1. Create a volume in your Packet portal
    2. Route the Volume to the Server
    3. Run the Attach Scripts
    4. Partition the Block Device
    5. Build the file system
    6. Mount the block storage file system
  7. Install applications and load files into block storage
  8. Shutting down
    1. Detach the block storage volume
    2. Delete Server
  9. Starting up again
    1. Deploy a server
    2. Attach existing block storage to a new server
    3. All-in-one commands
    4. Run startup script
    5. Conclusion

Packet.net

Packet rents dedicated, bare metal servers by the hour. As far as I can tell, Packet offers the best value dedicated servers, compared to other major providers like Amazon AWS. For example, when this blog post was written Packet offers a high-powered server — which they call a Type 1 server — with a 4-core Intel Xeon processor running at 3.4 GHz, 32 GB of RAM, 120 GB SSD and a 2 Gbps network connection for US$ 0.40 per hour.

Controlling costs when using bare metal servers

Packet charges users for servers when they are in any state other than deleted. If you leave a Packet server running while you are not using it, you are still paying for it and the costs will add up. Packet charges forty cents per hour for their Type 1 server so if you let that server run it will cost you $292 per month.

To save money, I will delete a server when I am not using it. However, I cannot save the server’s system state so when I delete the server I will lose all software I installed and all configurations I changed. I will also lose all the data I saved from my network emulation exercises. I need to be able to save data to persistent storage.

My solution involves using Packet’s low-cost block storage service to save critical data, configuration files, and startup scripts that I will use to rebuild a server when I want to restart a network emulation scenario. For my purposes, twenty Gigabytes of block storage is sufficient and twenty Gigabytes costs only one dollar and fifty cents per month. After that, I pay only forty cents per hour for the server time I use. I use the server only when I am actively running network emulation scenarios. As soon as I am done, I delete the server again, after saving all my results to block storage.

Create a Packet account and Login

To get started, you must create an account on the Packet.net web site and log in.

Packet web site login page

Click on the Signup button on the Packet.net web site and enter the requested information. Follow the instructions provided by the web site.

Create a project

After logging in for the first time, create a project. Projects allow you to groups servers and other resources together. Most importantly, projects manage the billing information for the resources used in the project. You can have different credit cards for different projects. You can invite other Packet.net users to use the same project, allowing groups to collaborate on the same project while consolidating billing to one account.

For an individual researcher like me, creating a project is just the next administrative step I need to take to work with Packet servers. I don’t worry about Packet’s more complex collaboration features because I don’t need to use them.

To create a project, click on the Create Your First Project button on the Packet.net web page.

Next, enter you billing information. Then scroll to the bottom of the page and click on the Create Project button.

Now we have created a project and are able to add servers and resources to it.

Generate and store SSH Keys

You must use SSH to access the terminal interface on a Packet server. Packet servers do not support passwords for root access1. You must provide an SSH key to get root access.

In this chapter, I show how to generate an SSH key pair on your local computer, then copy the public key to the Personal Keys section on the Packet web app. Every server you create will automatically have this public SSH key installed.

Microsoft Windows

In the most up-to-date Windows systems, you may have access to command-line tools that will generate SSH keys. But, most users will use the PuTTYgen application to generate SSH keys. PuTTYgen is part of the PuTTY suite of applications.

You may download the PuTTY installer from the PuTTY developer’s web site. Install it by double-clicking on the installer file and following the prompts.

Next, Start the puTTYgen application. All the default setting are OK. Click on the Generate button.

Follow the instructions to move the mouse over the PuTTYgen window to provide random inputs for the key generator. After a while, the key will be generated.

Click on the Save private key button. Choose the key file name. I chose to call it private-key.ppk. Remember to keep it in a safe location.

Copy the public key text that appears in the Public key for pasting into OpenSSH authorized_keys file field to your system clipboard.

You now have a private key file saved on your hard drive and the public key text is available in your clipboard, ready to paste to Packet’s web app.

Mac

On your Mac host computer, open a Terminal window and create an SSH key pair with the command:

$ ssh-keygen -t rsa

The tool will ask you for the file name of the key pair. In my case, I want to create a key pair named packet so I enter packet at the prompt.

Enter file in which to save the key (/Users/fake/.ssh/id_rsa): packet

The tool will ask you for a passphrase. I skip the passphrase by pressing the Enter key twice.

The ssh-keygen tool creates the SSH key pair and stores both the private and public files in the active directory, unless you specify the full path in when prompted for the file name. So it is a good idea to navigate to the ~/.ssh directory before running the command. In my case, the files are:

  • packet is the private key file
  • packet.pub is the public key file

List the files in the directory:

$ ls -l
total 12
-rw-r--r--  1 blinklet  staff  3061 11 Jun  2016 known_hosts
-rw-------  1 blinklet  staff  1675 27 Sep 17:54 packet
-rw-r--r--  1 blinklet  staff   401 27 Sep 17:54 packet.pub

Next, copy the public key to the clipboard so you can add it to the Packet.net system in the next step. In Linux, list the content of the public key file in the terminal, select the text and copy it to the clipboard. In my example, the public key is named packet.pub.

$ cat ~/.ssh/packet.pub
ssh-rsa AAAAxxxxThis-is-a-Fake-key-xxxxBAQDaUf4Z0W2xxxxThis-is-a-Fake-key-xxxxPluzzfoHYHA+LBe+Z8lgnVpgsxxxxThis-is-a-Fake-key-xxxxtbuGovSb3HWDJCf1BeCtZUCWmxxxxThis-is-a-Fake-key-xxxxx2LxxxxThis-is-a-Fake-key-xxxx7H7bwgBl+n72BikqtzjKZGo2xxxxThis-is-a-Fake-key-xxxxa+YyHyD0zzzzz4S4YH4ry6o4LWxxxxThis-is-a-Fake-key-xxxxF7JXIzP5xxxxThis-is-a-Fake-key-xxxxzzzhEKK7/3u7ki2zz2tsfakedRU3 fake@iMac.local

Linux

On your Linux host computer, open a Terminal window and create an SSH key pair with the command:

$ ssh-keygen -t rsa

The tool will ask you for the file name of the key pair. In my case, I want to create a key pair named packet so I enter packet at the prompt.

Enter file in which to save the key (/home/fake/.ssh/id_rsa): packet

The tool will ask you for a passphrase. I skip the passphrase by pressing the Enter key twice.

The ssh-keygen tool creates the SSH key pair and stores both the private and public files in the active directory, unless you specify the full path in when prompted for the file name. So it is a good idea to navogate to the ~/.ssh directory before running the command. In my case, the files are:

  • packet is the private key file
  • packet.pub is the public key file

List the files in the directory:

$ ls -l
total 12
-rw------- 1 ubuntu ubuntu  395 Sep  6 17:33 authorized_keys
-rw------- 1 ubuntu ubuntu 1679 Sep 25 13:31 packet
-rw-r--r-- 1 ubuntu ubuntu  403 Sep 25 13:31 packet.pub

Next, copy the public key to the clipboard so you can add it to the Packet.net system in the next step. In Linux, list the content of the public key file in the terminal, select the text and copy it to the clipboard. In my example, the public key is named packet.pub.

$ cat ~/.ssh/packet.pub
ssh-rsa AAAAxxxxThis-is-a-Fake-key-xxxxBAQDaUf4Z0W2xxxxThis-is-a-Fake-key-xxxxPluzzfoHYHA+LBe+Z8lgnVpgsxxxxThis-is-a-Fake-key-xxxxtbuGovSb3HWDJCf1BeCtZUCWmxxxxThis-is-a-Fake-key-xxxxx2LxxxxThis-is-a-Fake-key-xxxx7H7bwgBl+n72BikqtzjKZGo2xxxxThis-is-a-Fake-key-xxxxa+YyHyD0zzzzz4S4YH4ry6o4LWxxxxThis-is-a-Fake-key-xxxxF7JXIzP5xxxxThis-is-a-Fake-key-xxxxzzzhEKK7/3u7ki2zz2tsfakedRU3 fake@t420

Copy public key to Packet.net

On the Packet.net web app, go to the SSH Keys tab and click on the blue “plus” sign to add a new SSH key.

In the screen that appears, paste the text you copied from the screen during the key pair generation step above to the Key text field.

Give the key a title in the Title field. I chose to call my key Public-Key.

Then, choose the location. I chose to store this in Personal Keys so that I can always use the same key for multiple projects. My use-case is simple so I am keeping my key-management scheme simple.

Then, click the blue Add button. You will now see the your public key is saved in the Packet.net web app.

Deploy a Server

Use the Packet.net web app to deploy a new server. Click on the Manage tab and then click on the project name, then click on the Deploy Server button.

Next, enter the information about the server:

  • Choose a server name
  • Select the type of server. I chose the Type 1 “Workhorse” server
  • Choose the server’s operating system
  • Choose the server’s location. I chose to deploy the server in a location that also supports Packet’s Elastic Block Storage service

Then click the blue Deploy button. The server will take several minutes to start.

Since we also plan to use Packet’s elastic block storage (EBS) service, we need to deploy a server in a location where EBS is available. At the time this post was written, EBS is only available in Packet’s EWR1, SJC1, and AMS locations. In my case, I chose to use Packet’s New Jersey location, EWR1.

SSH client and X server on local machine

To connect to the terminal on the remote Packet server, we must use SSH. Since we plan to eventually run software the supports X windows, we also need to have an X server running on our local machine. When we log into the remote Packet server, we will enable X tunneling.

If you have a Windows computer, you need to install and run both an SSH client and an X server application. If you have a Mac computer, you already have an SSH client available but you still need to install an X server. If you have a Linux computer, you already have everything you need installed.

See below for the procedures required to configure an SSH client and X server in each of the three major operating system and to log into the remote Packet server with X tunneling enabled.

First, make a note of — or copy to the clipboard — the IPv4 address of the Packet.net server you deployed in the previous step. You will need it for the SSH configurations below.

Windows

In Windows, we will use the PuTTY SSH client. First, we need to set up the parameters in the PuTTY application and then save them for future use. Then we will use PuTTY to log into the remote Packet server.

Click on the Session tab in PuTTY. Enter the Packet server’s IPv4 address. All other settings are OK.

Next, click on the Connection tab. Enable keepalives by setting the Seconds between keepalives field to a value other than zero. I chose 20 seconds. All other settings are OK.

Next, click on the Data tab. Enter the userid root into the Auto-login username field. All other settings are OK.

Next, click on the Auth tab. Click on the browse button and navigate to the private SSH key file you previously generated. Select that key. All other settings are OK.

Next, click on the X11 tab and chck the Enable X11 forwarding check box. Enter the display localhost:0 into the X display location field. All other values are OK.

Finally, go back to the Session tab. Enter a name for this session in the Saved Sessions field and click the Save button. Now you can re-use all the configurations when you need to login to the Packet server.

To login to the remote packet Server, click on the session in the Saved Sessions box and click load. Note that, if you have deleted a server and are starting again with a new server, you will need to change the IP address.

Then click the Open button. If you are logging into a new server for the first time you will see a security alert. Click either Yes or No to proceed.

Now a terminal window will appear on your desktop. You have root access to your Packet server. and may now configure it to suit your needs.

Mac

Mac OS X does not have an X Server installed by default. You need to install the XQuartz X server. Follow the instructions available at https://www.xquartz.org/.

Next, run the SSH command to log into the Packet server. Use the -Y option to set up X forwarding on the SSH tunnel. For example:

$ ssh -Y -i ~/.ssh/packet root@147.75.73.83

Need to use -Y option to tunnel X windows because Mac OS X has more strict security defaults.

Now a terminal window will appear on your desktop. You have root access to your Packet server.

Linux

From your Linux host computer you may SSH to the Packet server using the ssh -iX command with the private key file, and the Packet server’s userid and IP address (or URL):

$ ssh -X -i /.ssh/packet root@147.75.79.221
root@virtual:~#

The X flag enables X forwarding on this SSH connection. Depending on the Linux distribution you choose to run on your Packet server you may need to modify some configuration files to enable X forwarding over an SSH tunnel. I found that the default configuration worked without any modifications when I used the Ubuntu 16.10 LTS operating system provided by Packet.

Now a terminal window will appear on your desktop. You have root access to your Packet server. and may now configure it to suit your needs.

Set up the Packet.net server

After logging into the remote packet server, make some initial configuration changes to set up an X11 client. I ran the following commands to set up my server for a few basic tests:

# apt-get update
# apt-get install -y xorg

Test X11

To ensure that the Packet server can host X applications, I test the SSH tunnel using the xeyes command. If SSH and X forwarding is configured correctly, the xeyes application running on the Packet server should open an X window on my local desktop.

For example, the screenshot below shows the xeyes window on top of the Terminal window on my Mac.

Quit xeyes by entering ctrl-c on the keyboard.

Create block storage

Packet offers a block storage that can be added to any deployed server. Block storage allows users to create disks that may save data for future use. As mentioned above, one way to minimize the costs of using Packet servers is to store data on a block storage volume and delete servers when they are not is use. The saved data may be used to speed up the configuration of a new server when you are ready to use Packet again. Maintaining block storage is much less expensive than keeping a server running.

Packet.net offers block storage in a few of its locations. See the packet.net web site for supported locations. For example, I am using the New Jersey location because it supports block storage.

For my research, I use the Standard Tier block storage service. Standard Tier costs about one one-hundredth of a penny per Gigabyte per hour. For example, 20 GB of storage will cost about $1.50 per month. Standard Tier disk performance, in terms of operations per second, is five times higher than the performance of a typical 7200 RPM HD so it is more than enough for my individual needs.

Create a volume in your Packet portal

Packet.net provides documentation describing how to set up block storage.

In the Packet.net web app, click on your project. On the project page go to the Storage tab and click on the green New Block Storage button.

On the next screen, choose the block storage size, performance tier, and location. Choose a smaller size to start. You can increase it later, but you cannot decrease it. I am starting with 20 GB.

Click on the blue Deploy button. You will see the new block storage volume created. Note the volume name; you will use it later.

In this example, the volume name is volume-6e56c556.

Route the Volume to the Server

Once a block storage volume is created, or if one is already available, it must be connected to a server in order to be used. Use the Packet web app to set up the network connection between the block storage volume and a Packet server.

First, click on the block storage volume. In this example we have only one volume named volume-6e56c556.

In the Storage Details box, select the Packet server in the Connected to field and click the small Attach button next to it.

Then, click the Save button at the bottom of the box. Now, the new volume is connected to the server.

This is like connecting a new storage device to a computer. The “physical” connection is completed but in the next step we still need to configure the server so it recognizes the new device.

Run the Attach Scripts

To complete the attachment process, log in to your Packet server via SSH, and run the Packet attach script. This script should already be installed on the server. If it is not installed, you may download the script from Packet’s GitHub repository.

To run the Packet attach script, execute the following command on the Packet server:

# packet-block-storage-attach -m queue

The command will output a few lines of text. The last line contains the directory that the server will associate with the block storage device. In this example, the output is:

Block device /dev/mapper/volume-6e56c556 is available for use

Note the block device directory from the command output. You will use it later. In this case, the block device directory is /dev/mapper/volume-6e56c556.

Partition the Block Device

Since this is a new device, the server cannot use it until we partition it and build the file system. We will use the fdisk utility to partition the device, the kpartx utility to update the partition map, and the mkfs.ext4 utility to build an ext4 file system in the new partition.

As we saw after executing the attach command, the block device is situated at /dev/mapper/volume-6e56c556.

Run the fdisk command on the new block device:

# fdisk /dev/mapper/volume-6e56c556

Respond to the fdisk utility’s prompts with the following commands:

  • Type n to create a new partition,
  • Type p to choose the primary type,
  • Press the Enter key three times to accept the default settings.
  • Type w to write and save the changes.

You may see a warning message when the fdisk utility quits. Ignore it.

Run the fdisk -l command to see the list of partitions. In the output of the command, under the Device section, you will find the new partition name. The new partition name in this example is /dev/mapper/volume-6e56c556-part1.

# fdisk -l

The Device section of the output will look like this:

Device                           Boot Start      End  Sectors Size Id Type
/dev/mapper/volume-6e56c556-part1      2048 41943039 41940992  20G 83 Linux

See a screenshot of the output below:

The server will not recognize the new partition because the Linux kernel has not updated its own partition file. Normally you might reboot the system to do this but we can avoid rebooting by running the kpartx -u command which makes the system re-read its partition table.

Update partition device mappings for the new partition:

# kpartx -u /dev/mapper/volume-6e56c556-part1

Build the file system

Create a file system on the new partition:

# mkfs.ext4 /dev/mapper/volume-6e56c556-part1

Mount the block storage file system

To use the newly-created file system we need to mount it on the Packet server. In this example, I chose to create directory named /mnt/disk1 and use it as the mount point. Execute the following commands to mount the new file system to /mnt/disk1.

# mkdir /mnt/disk1
# mount -t ext4 /dev/mapper/volume-6e56c556-part1 /mnt/disk1

Should we need to reboot the server, we would lose the mount point. We need to update the fstab file so the disk remounts if we have to reboot the server. Add a new line to the fstab file with the following command:

# echo "/dev/mapper/volume-6e56c556-part1 /mnt/block ext4 _netdev 0 0" >> /etc/fstab

Install applications and Load files into block storage

At this point, you may configure the server to suit your needs. For example, you may install network emulation programs and file systems.

To make rebuilding the configuration easier after deleting a server, modify the configuration files so that data generated by your activity is saved, or backed up, to the block storage volume mounted at /dev/disk1.

List the files on the block storage volume with the commands:

# cd /mnt/disk1
# ls

In a future post, I will show how I installed and configured the Cloonix network emulator on a Packet server using block storage to greatly reduce the time required to rebuild a new Cloonix server whenever I want to run Cloonix on Packet.

Shutting Down

After using you Packet server to complete your work and after saving your data and any files you want to use again to the block storage volume mounted at /mnt/disk1, shut down the server so that you are not billed for running it while you are not using it.

To shut down, first unmount and detach the block storage volume. Then, delete the server.

Detach the block storage volume

Unmount the file system and detach the block storage volume from the server. It is important to properly unmount the file system to avoid the possibility of causing data corruption on it when you delete the server. Execute the command:

# umount /mnt/disk1

Next, detach the block storage volume by running the packet-block-storage-detach script:

# packet-block-storage-detach

You may need to wait about a minute before the device appears to be detached in the Packet portal.

Delete the Server

Delete the server on the Packet web app. Now you will no longer be charged by the hour for the server. You will continue to pay a very low fee for the block storage: US$ 1.50 per month in this case.

To delete the server, go to the Packet web app and select the project. Click the check-box next to the server you wish to delete, select the Delete action from the drop-down menu box below it, and click on the Apply button.

Starting up again

Imagine it is a few days later and you have some time to work on your project again. Now you need to start up a new Packet server, re-attach the block storage volume on which you saved your configurations and data, then re-install the software you need. To make this process faster, I recommend you write a startup script that you can save on the block storage volume.

Deploy a server

To start a new server, follow the same steps I listed in the Deploy Server chapter above. Use the Packet.net web app to deploy a new server.

Attach existing block storage to a new server

We already have a block storage volume saved in the Packet project. All our data files, configuration files, and startup scripts are saved on this volume. We need to attach the existing volume to the new server we deployed.

In the Packet.net web app, go to Storage tab and click on the volume you previously created. Follow the steps described in the Route the Volume to the Server chapter above. In this case, the volume is named volume-6e56c556.

In the Storage Details box, select the Packet server in the Connected to field and click the small ATTACH button next to it.

Then, click the Save button at the bottom of the box. Now, the new volume is connected to the server.

Log in to your Packet machine via SSH, and run the Packet attach script. Execute the following command on the Packet server:

# packet-block-storage-attach -m queue

Execute the following commands to mount the new file system to /mnt/disk1.

# mkdir /mnt/disk1
# mount -t ext4 /dev/mapper/volume-6e56c556-part1 /mnt/disk1

Update the fstab file so the disk remounts if we have to reboot the server. Add a new line to the fstab file with the following command:

# echo "/dev/mapper/volume-6e56c556-part1 /mnt/block ext4 _netdev 0 0" >> /etc/fstab

All-in-one commands

In summary, execute the following four commands to complete the process of mounting an existing block storage volume to a server. Note that the volume name will be different in your case.

# packet-block-storage-attach -m queue
# mkdir /mnt/disk1
# mount -t ext4 /dev/mapper/volume-6e56c556-part1 /mnt/block
# echo "/dev/mapper/volume-6e56c556-part1 /mnt/block ext4 _netdev 0 0" >> /etc/fstab

Run startup script

Now that the server is attached to the block storage volume and it is mounted to /mnt/disk1, run any startup scripts you may have saved on the volume and/or copy any files over to the server’s SSD. This will allow you to quickly rebuild your system so you can continue with your project.

List the files on the block storage volume with the commands:

# cd /mnt/disk1
# ls

Conclusion

Packet enables me to run multiple virtual machines with hardware virtualization support on a dedicated remote server. Packet provides me access to a powerful remote server when I need it at low cost, and allows me to delete that server when I am not using it while maintaining data files on persistent block storage.

In the near future, I will write a post describing how to install, set up, and run the Cloonix network emulator on a remote server provided by Packet. Cloonix uses QEMU/KVM to build the virtual machines that implement different network nodes so it must run on a machine that has access to hardware-level virtualization support. The Cloonix development team recently updated Cloonix to support Cisco router images, which means I need a computer that is more powerful than my laptop computer to run network emulation scenarios that include Cisco images.


  1. Packet gives you a temporary root password that expires after 24 hours and only works when using Packet’s Console access feature, which is outside the scope of this post 

2 responses to Set up a dedicated virtualization server on Packet.net

  1. Comments/Questions on: How to emulate a network using VirtualBox

    Hi Brian,

    I learned some stuff in Virtualbox that was not clear to me. Thank you.

    I was able to follow you with no issue until I got to the “Connect to each virtual machine” part.
    I keep receiving connection refused –

    How do I make this connection using putty? Sorry I am a newbie. I don’t see where to put that 14501/02 ports. I removed the port 22 and added it there – I got an error.

    And when you say host, for some reasons, it looks like you are referring to the guest. For instance, when you say the host port – the formula shows the guest port. Am I a bit confused, here?

    Thanks

    Klauss

    • Hi Klauss,
      Thanks for reading my blog.
      Regarding PuTTY, did you also enter the username in the Data tab? That could be the trouble in this case.
      Regarding use of the term “host”. I used my terminology this way:
      – Host Computer = the PC you are using
      – Host Port = a VirtualBox object that forwards a port on the Host Computer to a port on a Guest VM
      If I used The term Host all by itself I agree it could be confusing. I will check the post and clarify.
      Thanks,
      Brian

Leave a Reply

Text formatting is available via select HTML. <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

*